Update kubeadm install doc for v1.8

@kubernetes/sig-cluster-lifecycle-pr-reviews

This change is

Maybe add note about disabling swap ? it's new thing for 1.8, so if people are upgrading, they might end-up with non-starting kubelet

@kad Good idea. Done.
I tested this on Debian 9 and it works flawlessly as well. The only earlier blocker was that 7 and 8 didn't enable the memory cgroup, and we didn't want to tell the user to reboot & mock with their kernel cmdline to install k8s. Now that's not needed anymore :tada:

Yay! Thanks!


styling flash messages

Adding styles for 'error', 'warning', and 'notice' flash messages.

@tmaybe

also just sneaking in a small HTML change to fix the header in edit-article.html

Looks great!

Ooo! Love the colours.


fix this error

Info

Server version: Paper version git-Paper-1524 (MC: 1.12.2) (Implementing API version 1.12.2-R0.1-SNAPSHOT)
OldCombatMechanics version: 1.6.6
Server log file:
https://pastebin.com/Wni4LPVw
OldCombatMechanics config file:
https://pastebin.com/aZhX5MBR

Problem Description

error in console

https://github.com/gvlfm78/BukkitOldCombatMechanics/issues/209#issuecomment-408431344

Duplicate of #209


RegisterWithIron() overload to use default AWS credentials.

Not relevant any more since we attempt to just pick up from the environment if possible but don't completely fail otherwise.


[SPARK-15072][SQL][PySpark] FollowUp: Remove SparkSession.withHiveSupport in PySpark

What changes were proposed in this pull request?
This is a followup of https://github.com/apache/spark/pull/12851
Remove SparkSession.withHiveSupport in PySpark and instead use SparkSession.builder. enableHiveSupport
How was this patch tested?
Existing tests.

Test build #58411 has started for PR 13063 at commit 413a9ec.

@techaddict in the future can you try to put more changes in one patch instead of having many related patches?

Hi @andrewor14 support for enableHiveSupport was added(https://github.com/apache/spark/commit/0903a185c7ebc57c75301a27d215b08efd347f99) after I started working on removing withHiveSupport in https://github.com/apache/spark/pull/12851
That's why the Follow Up PR.

I see. That's fine then.

Test build #58411 has finished for PR 13063 at commit 413a9ec.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/58411/
Test PASSed.

Merging into master 2.0. Thanks.

There are still places where we use withHiveSupport. Seems our tests did not catch those.

https://github.com/apache/spark/pull/13069 is the fix


vendor: update appc/spec to 0.8.7

This updates appc/spec to latest version 0.8.7. It also anchors all
dependencies to specific versions, to avoid spurious updates and
floating dependencies.
Fixes part of https://github.com/appc/docker2aci/issues/195

LGTM


Added api.root() to be able to access root element

Just cherry picked your stuff and changed it slightly.
Now exposing rootEl directly.
See https://github.com/StephanHoyer/mithril-query#auto-rendering

great


Radio buttons break in RTL

http://plnkr.co/edit/HjAZk1pGefXDowt38ddu?p=preview
When using radio buttons (<ion-radio> or the CSS directly) with a dir="rtl" page, the labels disappear when they're clicked, and a small unstyled HTML radio button appears in the corner.

@imgx64
Thanks for the workaround, It woked


Linux support

When I try to build this project in docker I get all of these errors.
https://github.com/apple/swift-corelibs-foundation/blob/main/Sources/Foundation/Locale.swift
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:21:36: error: 'Lang
uageCode' is not a member type of struct 'Foundation.Locale'
93.15     public init(providers: [Locale.LanguageCode: any HXLocalisable], defaultLanguageCode: Locale.
LanguageCode? = nil, overrideLanguagePreference: ((_: Request) -> Locale.LanguageCode?)? = nil) {
93.15                             ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:21:98: error: 'Lang
uageCode' is not a member type of struct 'Foundation.Locale'
93.15     public init(providers: [Locale.LanguageCode: any HXLocalisable], defaultLanguageCode: Locale.
LanguageCode? = nil, overrideLanguagePreference: ((_: Request) -> Locale.LanguageCode?)? = nil) {
93.15                                                                                ~~~~~~
^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:21:171: error: 'Lan
guageCode' is not a member type of struct 'Foundation.Locale'
93.15     public init(providers: [Locale.LanguageCode: any HXLocalisable], defaultLanguageCode: Locale.
LanguageCode? = nil, overrideLanguagePreference: ((_: Request) -> Locale.LanguageCode?)? = nil) {
93.15
                                                                  ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HTTPHeaders+Language.swift:308:41: error
: 'LanguageCode' is not a member type of struct 'Foundation.Locale'
93.15     func language(fallback lang: Locale.LanguageCode) -> HXLanguageHeader {
93.15                                  ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:26:75: error: 'La
nguageCode' is not a member type of struct 'Foundation.Locale'
93.15     static func parse(directives: [[HTTPHeaders.Directive]], lang: Locale.LanguageCode) -> HXLang
uageHeader {
93.15                                                                    ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:24:43: error: 'La
nguageCode' is not a member type of struct 'Foundation.Locale'
93.15     public let defautLanguageCode: Locale.LanguageCode
93.15                                    ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:5:34: error: 'Lan
guageCode' is not a member type of struct 'Foundation.Locale'
93.15         public let value: Locale.LanguageCode
93.15                           ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:30:33: error: 'La
nguageCode' is not a member type of struct 'Foundation.Locale'
93.15     public var prefered: Locale.LanguageCode {
93.15                          ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:35:1: error: type
 'HXLanguageHeader.Preference' does not conform to protocol 'Equatable'
93.15 extension HXLanguageHeader.Preference: Comparable {
93.15 ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:5:20: note: store
d property type '<<error type>>' does not conform to protocol 'Equatable', preventing synthesized confo
rmance of 'HXLanguageHeader.Preference' to 'Equatable'
93.15         public let value: Locale.LanguageCode
93.15                    ^
93.15 Swift.==:1:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed to 'RawRepr
esentable'
93.15 @inlinable public func == <T>(lhs: T, rhs: T) -> Bool where T : RawRepresentable, T.RawValue : Eq
uatable
93.15                        ^
93.15 Swift.FloatingPoint:2:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed
to 'FloatingPoint'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool
93.15                        ^
93.15 Swift.BinaryInteger:2:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed
to 'BinaryInteger'
93.15     public static func == <Other>(lhs: Self, rhs: Other) -> Bool where Other : BinaryInteger
93.15                        ^
93.15 Swift._Pointer:2:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed to '_
Pointer'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool
93.15                        ^
93.15 Swift._Pointer:3:35: note: candidate would match if 'HXLanguageHeader.Preference' conformed to '_
Pointer'
93.15     @inlinable public static func == <Other>(lhs: Self, rhs: Other) -> Bool where Other : _Pointe
r
93.15                                   ^
93.15 Swift.Strideable:3:35: note: candidate would match if 'HXLanguageHeader.Preference' conformed to
'Strideable'
93.15     @inlinable public static func == (x: Self, y: Self) -> Bool
93.15                                   ^
93.15 Swift.StringProtocol:2:35: note: candidate would match if 'HXLanguageHeader.Preference' conformed
 to 'StringProtocol'
93.15     @inlinable public static func == <RHS>(lhs: Self, rhs: RHS) -> Bool where RHS : StringProtoco
l
93.15                                   ^
93.15 Swift.SIMD:4:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed to 'SIMD'
93.15     public static func == (a: Self, b: Self) -> Bool
93.15                        ^
93.15 Foundation.AttributedStringProtocol:5:24: note: candidate would match if 'HXLanguageHeader.Prefer
ence' conformed to 'AttributedStringProtocol'
93.15     public static func == <RHS>(lhs: Self, rhs: RHS) -> Bool where RHS : AttributedStringProtocol
93.15                        ^
93.15 Foundation.__BridgedNSError:2:24: note: candidate would match if 'HXLanguageHeader.Preference' co
nformed to '__BridgedNSError'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool
93.15                        ^
93.15 Foundation._BridgedStoredNSError:2:24: note: candidate would match if 'HXLanguageHeader.Preferenc
e' conformed to '_BridgedStoredNSError'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool
93.15                        ^
93.15 /build/.build/checkouts/swift-crypto/Sources/Crypto/Digests/Digest.swift:61:24: note: candidate w
ould match if 'HXLanguageHeader.Preference' conformed to 'Digest'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool {
93.15                        ^
93.15 /build/.build/checkouts/swift-crypto/Sources/Crypto/Digests/Digest.swift:74:24: note: candidate w
ould match if 'HXLanguageHeader.Preference' conformed to 'Digest'
93.15     public static func == <D: DataProtocol>(lhs: Self, rhs: D) -> Bool {
93.15                        ^
93.15 /build/.build/checkouts/swift-crypto/Sources/Crypto/Key Agreement/DH.swift:168:24: note: candidat
e has non-matching type '<D> (SharedSecret, D) -> Bool'
93.15     public static func == <D: DataProtocol>(lhs: Self, rhs: D) -> Bool {
93.15                        ^
93.15 /build/.build/checkouts/swift-crypto/Sources/Crypto/Message Authentication Codes/MessageAuthentic
ationCode.swift:35:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed to 'Messa
geAuthenticationCode'
93.15     public static func == (lhs: Self, rhs: Self) -> Bool {
93.15                        ^
93.15 /build/.build/checkouts/swift-crypto/Sources/Crypto/Message Authentication Codes/MessageAuthentic
ationCode.swift:48:24: note: candidate would match if 'HXLanguageHeader.Preference' conformed to 'Messa
geAuthenticationCode'
93.15     public static func == <D: DataProtocol>(lhs: Self, rhs: D) -> Bool {
93.15                        ^
93.15 Swift.Equatable:2:17: note: protocol requires function '==' with type '(HXLanguageHeader.Preferen
ce, HXLanguageHeader.Preference) -> Bool'
93.15     static func == (lhs: Self, rhs: Self) -> Bool
93.15                 ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLangaugeHeader.swift:10:32: error: typ
e 'Locale' has no member 'LanguageCode'
93.15             let value = Locale.LanguageCode(String(first.value))
93.15                         ~~~~~~ ^~~~~~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:4:35: error: 'Langu
ageCode' is not a member type of struct 'Foundation.Locale'
93.15     public var providers: [Locale.LanguageCode: any HXLocalisable]
93.15                            ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:5:44: error: 'Langu
ageCode' is not a member type of struct 'Foundation.Locale'
93.15     public var defaultLanguageCode: Locale.LanguageCode
93.15                                     ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:6:72: error: 'Langu
ageCode' is not a member type of struct 'Foundation.Locale'
93.15     public var overrideLanguagePreference: ((_ req: Request) -> Locale.LanguageCode?)?
93.15                                                                 ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:8:57: error: 'Langu
ageCode' is not a member type of struct 'Foundation.Locale'
93.15     public func localise(text: String, for code: Locale.LanguageCode) -> String {
93.15                                                  ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:12:51: error: canno
t infer contextual base in reference to member 'alpha2'
93.15         } else if let language = code.identifier(.alpha2),
93.15                                                  ~^~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:13:55: error: type
'Locale' has no member 'LanguageCode'
93.15                   let localisation = providers[Locale.LanguageCode(stringLiteral: language)]
93.15                                                ~~~~~~ ^~~~~~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:24:74: error: value
 of type 'Locale' has no member 'language'
93.15         self.defaultLanguageCode = defaultLanguageCode ?? Locale.current.language.languageCode ??
 Locale.LanguageCode("en")
93.15                                                           ~~~~~~~~~~~~~~ ^~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:24:106: error: type
 'Locale' has no member 'LanguageCode'
93.15         self.defaultLanguageCode = defaultLanguageCode ?? Locale.current.language.languageCode ??
 Locale.LanguageCode("en")
93.15
 ~~~~~~ ^~~~~~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:31:38: error: 'nil'
 requires a contextual type
93.15         overrideLanguagePreference = nil
93.15                                      ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:32:46: error: value
 of type 'Locale' has no member 'language'
93.15         defaultLanguageCode = Locale.current.language.languageCode ?? Locale.LanguageCode("en")
93.15                               ~~~~~~~~~~~~~~ ^~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXLocalisation.swift:32:78: error: type
'Locale' has no member 'LanguageCode'
93.15         defaultLanguageCode = Locale.current.language.languageCode ?? Locale.LanguageCode("en")
93.15                                                                       ~~~~~~ ^~~~~~~~~~~~
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXRequestLocalisation.swift:6:57: error:
 'LanguageCode' is not a member type of struct 'Foundation.Locale'
93.15     public func localise(text: String, for code: Locale.LanguageCode? = nil) -> String {
93.15                                                  ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 /build/.build/checkouts/VaporHX/Sources/VHX/Localisation/HXRequestLocalisation.swift:12:33: error
: 'LanguageCode' is not a member type of struct 'Foundation.Locale'
93.15     public var prefered: Locale.LanguageCode {
93.15                          ~~~~~~ ^
93.15 Foundation.Locale:1:15: note: 'Locale' declared here
93.15 public struct Locale : CustomStringConvertible, CustomDebugStringConvertible, Hashable, Equatable
, ReferenceConvertible {
93.15               ^
93.15 error: fatalError

Sorry about this. I haven't run it on Linux for quite some time. I should be able to set up a Linux VM tomorrow to confirm it. In any case, I was pretty sure that this API was available on Linux. It will be a bit sad if I was wrong.

The spcific steps I used to test are pretty straight forward.

Create a vapor project with the default template
Add VaporHX and import it to configure it
Run docker-compose build
See the error

I linked the swift-corelibs-foundation Locale.swift which is missing LanguageCore that was added to swift-foundation Locale.swift
From what I have heard, there are efforts to move away from corelibs and that it is in maintaince mode as the cross platform solution is made stable.
In the mean time I made a quick and dirty patch that uses the deprecated APIs for corelibs support.
https://github.com/RussBaz/VaporHX/compare/master...wSedlacek:VaporHX:master
I can’t say for sure if these changes actually work with localziation as I have yet to use any of that functionality, however I can say this patch at least compiles in docker and runs on linux.

It should work now. I also added a GitHub Action to run basic sanity tests on Ubuntu.
Could you check if it works for you and if it is safe to close this issue?

I will close it for now, but if it still does not work on Linux for any reason, then please let me know and I will reopen this issue. Thanks for trying and reporting!

Thank you! I have been busy at work so I haven’t gotten to play with this again.
I will give it a shot this weekend.


start on web design of the admin contest page

Add the icons just after the page title and add a gif/image here.

This isn't ready for review yet

merged as part of #944


Support for Kibana 6.2.x

Can you please support the latest Kibana Release 6.2.x. Thanks!
Plugin installation was unsuccessful due to error "Incorrect Kibana version in plugin [logtrail]. Expected [6.2.1]; found [6.1.3]"
The command '/bin/sh -c ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.25/logtrail-6.1.3-0.1.25.zip' returned a non-zero code: 70

👍

I see there is a branch for 6.2.1, is there anything left to do a release?

Some changes in Kibana 6.2.1 is breaking the UI in logtrail. This needs some work. Will release an update once these issues are fixed.

Download logtrail release for 6.2.x from https://github.com/sivasamyk/logtrail/releases/tag/v0.1.26

Thanks a lot!


robot_localization: 3.0.2-1 in 'dashing/distribution.yaml' [bloom]

Increasing version of package(s) in repository robot_localization to 3.0.2-1:

upstream repository: https://github.com/cra-ros-pkg/robot_localization.git
release repository: https://github.com/cra-ros-pkg/robot_localization-release.git
distro file: dashing/distribution.yaml
bloom version: 0.9.0
previous version for package: 3.0.1-1

its not a new distro release if it works the first time!


Creating the Featured Content Module molecule

Short description explaining the high-level reason for the pull request
Additions

Added macro code to generate FCM markup
Added prototyping content
Added FCM to landing-page demo

Testing

Run gulp build and navigate to /landing-page/

Review

@KimberlyMunoz
@sebworks
@anselmbradford
@kurtw
@kave

Screenshots

Checklist

[ ] Changes are limited to a single goal (no scope creep)
[ ] Code can be automatically merged (no conflicts)
[ ] Code follows the standards laid out in the front end playbook
[ ] Passes all existing automated tests
[ ] New functions include new tests
[ ] New functions are documented (with a description, list of inputs, and expected output)
[ ] Placeholder code is flagged
[ ] Visually tested in supported browsers and devices
[ ] Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)

:+1: :100: :tada:

:bird: :us:


Add exact seek option to ffmpeg config

We are currently using AvSeek flag backwards to perform seek operations but in some cases the AvSeek flag any (which is slower but more precise) is needed.
This should be part of ffmpeg config. Default remains flag backwards.
I will do the implementing if you are OK with it.

If you use any flag, ffmpeg will not seek to keyframe but to any frame. This means that you will get broken video (or solid gray image) after seeking, until the next key frame is decoded. I do not think that this is desirable. Only when you start decoding from a key frame, you will get valid output, which is what backwards flag does. I cannot imagine that anyone wants broken output after seek.
What we could do ist seek using backwards flag, then decode and drop frames until we are at the desired position. But it is not even needed, because media foundation will already do this for us. Sometimes you might notice that after seek it takes a short moment until playback starts. This happens when we seek to key frame and a lot of frames after that need to be decoded and dropped. So MF will already do the skip-ahead to exact position for us. Unfortunately we cannot even prevent this. At least, some time back I tried to disable it by setting the actual position in OnStarted, and it had no effect. So it is more like MF enforces exact seek, and we cannot disable it.

I was thinking of adding this flag specifically for audio. I didn't realize MF forces us to do exact seek. That is actually quite interesting.

I think for audio, ffmpeg will anyways seek to packet which contains that position, so using the any flag should not make a difference there. Key frame searching is only done for video.


Implementing rand_like

I was working on an implementation of rand_like, similar to that in PyTorch and I was unsure about how to handle RNGs. Right now, MLUtils does not depend on CUDA, so handling RNGs for CuArrays will be a problem because CUDA.default_rng() will not be accessible. Is there a way to write this so that the function still works for both CPU and GPU without introducing a CUDA.jl dependency?

rand!(similar(x, ...)) will automatically use a device-specific RNG. It would also be good to have an overload that takes a user-specified RNG as well. I see that getting more mileage downstream.

rand!(similar(x, ...)) will automatically use a device-specific RNG. It would also be good to have an overload that takes a user-specified RNG as well. I see that getting more mileage downstream.

Yep, that overload was what I was looking for. I am already using rand!(similar(x, ...)) for the general implementation, but how do I pass in a default RNG that handles both GPU/CPU? Something like
rand_like(x::AbstractArray, T::Type, sz=size(x); rng = rng_from_array(x)) = rand!(rng, similar(x, T, sz))

To my knowledge you can't without explicitly dispatching to each type as rng_from_array does.

Might be overkill, but with the recent GPU related issues with some of the utility functions maybe it makes more sense to have a FluxUtils package as well so that the heavy CUDA dep is less of a worry - but I'm not sure if there's just a way to solve this that I'm missing

Why not just have:
rand_like(x::AbstractArray, T::Type, sz=size(x)) = rand!(similar(x, T, sz))
rand_like(rng::AbstractRNG, x::AbstractArray, T::Type, sz=size(x)) = rand!(rng, similar(x, T, sz))

the one extra line adds negligible maintenance overhead, while calling rand! sans RNG gives that function more agency to choose the appropriate RNG type.

That makes sense! I've done that in #101


I followed the instructions in the README exactly, but I'm getting this error when running "python run.py --train=True --dataset=NYT". Can anyone tell me how to fix it? I appreciate any help! My environment has been modified according to everyone's suggestions.

(CasRel) yons@jmw-lab-server:~/lilinglicode/CasRel$ python run.py ---train=True --dataset=NYT
Using TensorFlow backend.
Traceback (most recent call last):
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcudnn.so.7: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 3, in
from model import E2EModel, Evaluate
File "/home/yons/lilinglicode/CasRel/model.py", line 2, in
from keras.layers import *
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/init.py", line 3, in
from . import utils
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/utils/init.py", line 6, in
from . import conv_utils
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/utils/conv_utils.py", line 9, in
from .. import backend as K
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/backend/init.py", line 1, in
from .load_backend import epsilon
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/backend/load_backend.py", line 89, in
from .tensorflow_backend import *
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 5, in
import tensorflow as tf
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/init.py", line 24, in
from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/init.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/home/yons/anaconda3/envs/CasRel/lib/python3.7/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: libcudnn.so.7: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.
(CasRel) yons@jmw-lab-server:~/lilinglicode/CasRel$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243
My understanding is whether my CUDA version being 11.4 might be causing issues since others have suggested using CUDA 10.0.？？

Resolved, following the #66 environment debugging approach, has been resolved.


multiple includefilters on more than 3rd level

Existing query on EF core looks this way
_db.Contracts.Include(x => x.ContractKind)
               .Include(x => x.ContractParticipants)
                   .ThenInclude(x => x.ParticipantType)
               .Include(x => x.BillObjects)
                   .ThenInclude(x => x.Rfsubject)
                  .Include(x => x.BillObjects)
                        .ThenInclude(x => x.LinkBillObjectWithEnergyLinkObjects)
                         .ThenInclude(x => x.EnergyLinkObject)
                         .ThenInclude(x => x.LinkEnergyLinkObjectWithBillPoints)
                         .ThenInclude(x => x.BillPoint)
                         .ThenInclude(x => x.LinkBillPointWithMeterPoints)

I need to filter by date on 7th and 9th query strings, like :
ThenInclude(x => x.LinkBillObjectWithEnergyLinkObjects).Where(link=>link,StartDate<param1)
and  .ThenInclude(x => x.BillPoint).Where (billPoint=>billPoint.StartDate<param2)
Can i do that?      If yes, please write query i don't understand how to do it.

Hello @vlapenkov ,
From what I remember, you cannot mixes Include or ThenInclude with IncludeFilter, you need to use IncludeFilter all the way.
We opened a task here to add support to ThenIncludeFilter, I don't think it will be hard (I might be wrong).
I don't have your model so the following code contains probably a lot of error but will give you an idea about how to do it:
_db.Contracts.IncludeFilter(x => x.ContractKind)
               .IncludeFilter(x => x.ContractParticipants)
                  .IncludeFilter(x => x.ContractParticipants.x.ParticipantType)
               .IncludeFilter(x => x.BillObjects)
                  .IncludeFilter(x => x.BillObjects.x.Rfsubject)
                  .IncludeFilter(x => x.BillObjects)
                         .IncludeFilter(x => x.BillObjects.SelectMany(y => y.LinkBillObjectWithEnergyLinkObjects.Where(link => link.StartDate<param1)))
                         .IncludeFilter(x => x.BillObjects.EnergyLinkObject)
                         .IncludeFilter(x => x.BillObjects.SelectMany(y => y.LinkEnergyLinkObjectWithBillPoints))
                         .IncludeFilter(x => x.BillObjects.SelectMany(y => y.BillPoint.Where(billPoint => billPoint.StartDate<param2))
                         .IncludeFilter(x => x.BillObjects.SelectMany(y => y.LinkBillPointWithMeterPoints));

I will try to work on the ThenIncludeFilter during the weekend to check if we can easily support it.
Best Regards,
Jonathan

Performance Libraries
context.BulkInsert(list, options => options.BatchSize = 1000);
Entity Framework Extensions • Entity Framework Classic • Bulk Operations •  Dapper Plus
Runtime Evaluation
Eval.Execute("x + y", new {x = 1, y = 2}); // return 3
C# Eval Function • SQL Eval Function

Thank you. I examined filters and they worked on either level.

Great :)
We will let you know when the ThenFilter will be ready.
We tried this week but it was more complicated than expected.

Hi @JonathanMagnan .
I would be interrested by the ThenIncludeFilter feature.
Is it still planned?
If not, any guideline on how to achieve it?
Thanks in advance.

Hello @omatrot ,
The last time we tried the ThenIncludeFilter, we didn't succeed to complete it. We will eventually make it happen but at this moment, we don't find enough time.
Everything you can do with the ThenIncludeFilter, you were able to do it in EF6 with the Include method. So to achieve it, you simply need something like:
context.A.Include(x => x.B).Include(x => x.B.SelectMany(y => y.C));

@JonathanMagnan I'm not sure I'm following:
Here is my actual query:
context.AlertDetailsDbSet

                    .Include(ad => ad.TrackedVehicule).ThenInclude(v => v.Device).ThenInclude(d => d.Registration)
                    .Include(ad => ad.TrackedVehicule).ThenInclude(v => v.GpsBoxTypeNavigation).ThenInclude(bt => bt.Icon)
                    .Include(ad => ad.TrackedVehicule).ThenInclude(v => v.VehiculeGpsBoxInfo).ThenInclude(a => a.VehiculeConfigurationNavigation)
                    //.Include(ad => ad.TrackedVehicule).ThenIncludeFilter(v => v.FleetDetail.Where(fd => fd.RowEnabled == true)).IncludeFilter(v2 => v2.FleetDetail.Select(u => u.Fleet).Where(f => f.RowEnabled == true && f.CompanyId == companyID))
                    .Include(ad => ad.Alert).ThenInclude(a => a.AlertInstructions)

I've added a commented out "ThenIncludFilter" to express what I'm trying to achieve.
Early in the thread, you say that we have to IncludeFilter all the way.
I've tried that but it fails, so the following query:
                    context.AlertDetailsDbSet

                    .IncludeFilter(ad => ad.TrackedVehicule).IncludeFilter(ad => ad.TrackedVehicule.Device).IncludeFilter(ad => ad.TrackedVehicule.Device.Registration)
                    .IncludeFilter(ad => ad.TrackedVehicule).IncludeFilter(ad => ad.TrackedVehicule.GpsBoxTypeNavigation).IncludeFilter(ad => ad.TrackedVehicule.GpsBoxTypeNavigation.Icon)
                    .IncludeFilter(ad => ad.TrackedVehicule).IncludeFilter(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo).IncludeFilter(ad => ad.TrackedVehicule.VehiculeGpsBoxInfo.VehiculeConfigurationNavigation)
                    //.Include(ad => ad.TrackedVehicule).ThenIncludeFilter(v => v.FleetDetail.Where(fd => fd.RowEnabled == true)).IncludeFilter(v2 => v2.FleetDetail.Select(u => u.Fleet).Where(f => f.RowEnabled == true && f.CompanyId == companyID))
                    .IncludeFilter(ad => ad.Alert).IncludeFilter(ad => ad.Alert.AlertInstructions)

Throws the following exception:
System.AggregateException
  HResult=0x80131500
  Message=One or more errors occurred. (Processing of the LINQ expression 'source => (IEnumerable<AlertInstructions>)source' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.)
  Source=System.Private.CoreLib
  StackTrace:
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at RTE.Technologies.SafeProtect.Common.DummyConsole.Program.Main(String[] args) in C:\Users\olivier.matrot\source\repos\SafeProtect\SafeProtect.Common\SafeProtect.Common.Data.ConsoleTest\Program.cs:line 226

  This exception was originally thrown at this call stack:
    Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ProcessSelectMany(Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.NavigationExpansionExpression, System.Linq.Expressions.LambdaExpression, System.Linq.Expressions.LambdaExpression)
    Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.VisitMethodCall(System.Linq.Expressions.MethodCallExpression)
    System.Linq.Expressions.MethodCallExpression.Accept(System.Linq.Expressions.ExpressionVisitor)
    System.Linq.Expressions.ExpressionVisitor.Visit(System.Linq.Expressions.Expression)
    Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.ExpandAndReduce(System.Linq.Expressions.Expression, bool)
    Microsoft.EntityFrameworkCore.Query.Internal.NavigationExpandingExpressionVisitor.Expand(System.Linq.Expressions.Expression)
    Microsoft.EntityFrameworkCore.Query.QueryTranslationPreprocessor.Process(System.Linq.Expressions.Expression)
    .(System.Linq.IQueryable, System.Action<System.Linq.IQueryable>, out Microsoft.EntityFrameworkCore.Query.RelationalQueryContext, out object)
    Z.EntityFramework.Extensions.EFPlusExtensions.EFPlusCreateCommand(System.Linq.IQueryable, System.Action<System.Linq.IQueryable>, out Microsoft.EntityFrameworkCore.Query.RelationalQueryContext, out object)
    Z.EntityFramework.Plus.BaseQueryFuture.CreateExecutorAndGetCommand(out Microsoft.EntityFrameworkCore.Query.RelationalQueryContext)
    ...
    [Call Stack Truncated]
Inner Exception 1:
InvalidOperationException: Processing of the LINQ expression 'source => (IEnumerable<AlertInstructions>)source' by 'NavigationExpandingExpressionVisitor' failed. This may indicate either a bug or a limitation in EF Core. See https://go.microsoft.com/fwlink/?linkid=2101433 for more detailed information.

But if I remove the last IncludeFilter line, then it works.
Do I have a solution to my problem to IncludeFiter all the way and achieving a ThenIncludeFilter ?

I made the ThenIncludeFilter work :)
Here is the uncommented line:
                   .IncludeFilter(ad => ad.TrackedVehicule).IncludeFilter(ad => ad.TrackedVehicule.FleetDetail.Where(fd => fd.RowEnabled == true)).IncludeFilter(ad => ad.TrackedVehicule.FleetDetail.Select(u => u.Fleet).Where(f => f.RowEnabled == true && f.CompanyId == companyId))
The remaining problem is that I can't IncludeFilter the last line:
.IncludeFilter(ad => ad.Alert).IncludeFilter(ad => ad.Alert.AlertInstructions)


GameObject active in hierarchy fixing for duplicate and moving

Fixing of issue #138 (Wrong status when duplicating disabled GameObjects)

Task linked: CU-86byjee0k Wrong status when duplicating disabled GameObjects #138

Sin conflictos con develop

@M1R4B3L can you approve again to close the PR? :D


RFC: New Proxy API & SWR 2.0

New Proxy API & SWR 2.0
This RFC outlines the implementation of tRPC's v10 proxy api in conjunction with SWR hooks, along with some following features:

New proxy API: tRPC's V10 proxy API on useSWR calls
SWR 2.0: Using the newest SWR beta
New client creator API: The hooks creator allows access to several API's
Better SSR support: Taking advantage of SWR's fallbackData & SWRConfig
Future API's: Prefetch data before React mounts directly into SWR cache & mutation hooks

Motivation
As tRPC v10 is nearing completion (main proxy API is stable), SWR hooks is a "must have" for many developers like myself.
The changes
Below are the changes I propose for trpc-swr (v1.0?).
New proxy API
trpc-swr will take advantage of the newly constructed Proxy technique used in tRPC V10. Examples:

// Given route cats.byId & cats.list
const {data: cat} = trpc.cats.byId.useSWR({id: 1});
const {data: cats} = trpc.cats.list.useSWR();

// Using configurations
const {data: cat} = trpc.cats.byId.useSWR({id: 1}, {revalidateOnFocus: false});

// Directly modifying tRPC client
const {data: cat} = trpc.cats.byId.useSWR({id: 1}, {
    trpc: { // TRPCRequestOptions
        context: {
            ...op context
        }
    }
});

SWR 2.0
trpc-swr will take advantage of the newest SWR release. This allows for optimisticUpdate support, and other features. With this RFC swr: ^2 will become a peer dependency rather than swr: ^1
Example
const {data: cat, isLoading} = trpc.cats.byId.useSWR({id: 1});

if(isLoading) { // New is loading prop!
    return
Loading...

}

New client creator API
Being able to access the raw tRPC proxy client is incredibly useful during calls outside of React. Previously, the API would've required you to create a new client instance separate from trpc-swr. AKA. Two tRPC instances running.
Therefore, I propose a new API for creating a client instance. This will allow for the following:
import { createTRPCClient } from '@trpc/client';
import { createSWRProxyHooks } from 'trpc-swr';

export const client = createTRPCClient<AppRouter>({
    url: '/api/trpc',
    transformer: {
        // ...
    },
});

export const trpc = createSWRProxyHooks(client); // New API

// Use the client instance directly
// Here createSWRProxyHooks internally creates a new TRPCProxy client
// This is for usage outside of React
const cat = await trpc.client.cats.byId.query({id: 1});

// Use the hooks
const {data: cat} = trpc.cats.byId.useSWR({id: 1});

Alternative API 1:
In this API no trpc.client will be available. Instead, you can reference the client directly. This also allows for usage outside of React. Here we are using named properties. trpc will always be the hooks instance, and client will always be the standard tRPC proxy client instance.
import { createSWRProxyHooks } from 'trpc-swr';

// Can also be [trpc, client] = ...
export const {trpc, client} = createSWRProxyHooks({
    url: '/api/trpc',
    transformer: {
        // ...
    },
});

// Use the client instance directly
const cat = client.cats.byId.query({id: 1});

// Use the hooks
const {data: cat} = trpc.cats.byId.useSWR({id: 1});

Alternative API 2:
Keep the raw client inside of a trpc.useContext() call as it is today. Allow passing in a client instance to createSWRProxyHooks to use the same client instance. This would allow for usage outside of React.
Better SSR support
trpc-swr will take advantage of SWR's fallbackData & SWRConfig to provide better SSR support. This requires a new API for createSWRProxyHooks:
.getKey:

// Global fallbackData
<SWRConfig value={{
    fallback: {
        [trpc.cats.byId.getKey({id: 1})]: { // Automatically generated & serialized to match SWR using unstable_serialize.
            id: 1,
            name: 'Mr. Cat',
        },
    }
}} >
{...}
</SWRConfig>

// Local
const {data: cat} = trpc.cats.byId.useSWR({id: 1}, {
    fallbackData: {
        id: 1,
        name: 'Mr. Cat',
    }
});

The reason this is wildy important is for SSR to work from a global state standpoint. Providing the fallback data to a single useSWR hook will not populate the cache for other hooks (Wierdly enough since SWR uses a global cache anyway).
The creation of this .getKey API also gives us a direct way to mutate SWR keys in the global cache. This is useful for optimistic updates, manual cache overwrites, etc.
// We are inside a React component here
const {client} = trpc.useContext(); // Native tRPC client for mutation
const {mutate} = useSWRConfig();

function doSomethingThenMutate(){
    // Do something
    const newCat = client.cats.byId.mutate({id: 1}, {
        id: 1,
        name: 'Mr. Garfield',
    });

    // Mutate the cache
    // All hooks using this key will now have the new data
    await mutate(trpc.cats.byId.getKey({id: 1}), newCat, false);
}

This is something thats been missing even from the current V2 SWR core. Since we use proxies in tRPC this is a very easy API to implement and will provide tremendous DX for SWR users.
Future & possible API's
Prefetch data before React mounts directly into SWR cache
This is a feature SWR is in the process of releasing. It will allow for prefetching data before React mounts. This is useful for SSR, and also for prefetching data before a user navigates to a page. A new API for trpc-swr might look like this:
Potential API 1
Using the .getKey method from above.
// somefile.js
import {preload} from "swr";
import {trpc, client} from "lib/trpc";

// Preload data into SWR cache
// !Note that we are outside of react and therefore using native trpc client to make the request.
const key = trpc.cats.byId.getKey({id: 1});
const fethcer = () => client.cats.byId.query({id: 1})
preload(key, fethcer);

function Cat() {
  // If network is fast then data is available faster
  const { data } = trpc.cats.byId.useSWR({id: 1});
  ...
}

Potential API 2
Using a special .preload() on queries.
// somefile.js
import {trpc} from "lib/trpc";

// Preload data into SWR cache
// This internally implements the above API
trpc.cats.byId.preload({id: 1});

// or

trpc.cats.byId.useSWR.preload({id: 1}); // The nature of proxies allows this too.

function Cat() {
  // If network is fast then data is available faster
  const { data } = trpc.cats.byId.useSWR({id: 1});
  ...
}

SWR's useSWRMutation API
SWR is also working on a brand new mutation API similar to react-query for triggerable mutations. This is a very exciting API and I'm looking forward to it. I'm not sure how this will work with tRPC yet, but I'm sure we can figure something out.
Footnotes
This is a very rough draft of what I'm thinking. I'm sure there are many things I'm missing. I'm also not sure if this is the best way to do things. I'm open to suggestions and feedback. Additionally, some API's in the current release are not accounted for here. I'll be sure to update this RFC once I have some more time.
Once I have some feedback on the above suggestions I am happy to start a PR for this new API. And huge thanks to @sachinraja for making this wonderful library :)

Hey thanks for writing all of this!

New client creator API
Being able to access the raw tRPC proxy client is incredibly useful during calls outside of React. Previously, the API would've required you to create a new client instance separate from trpc-swr. AKA. Two tRPC instances running.
Therefore, I propose a new API for creating a client instance. This will allow for the following:

import { createTRPCClient } from '@trpc/client';
import { createSWRProxyHooks } from 'trpc-swr';

export const client = createTRPCClient<AppRouter>({
    url: '/api/trpc',
    transformer: {
        // ...
    },
});

export const trpc = createSWRProxyHooks(client); // New API

// Use the client instance directly
// Here createSWRProxyHooks internally creates a new TRPCProxy client
// This is for usage outside of React
const cat = await trpc.client.cats.byId.query({id: 1});

// Use the hooks
const {data: cat} = trpc.cats.byId.useSWR({id: 1});

This is nice, but we can't do this because it breaks SSR. With this code example, client will be shared across all requests. Alternative API 2 seems like a good way to go here.
I like .getKey. For this specific example though:
The creation of this .getKey API also gives us a direct way to mutate SWR keys in the global cache. This is useful for optimistic updates, manual cache overwrites, etc.

// We are inside a React component here
const {client} = trpc.useContext(); // Native tRPC client for mutation
const {mutate} = useSWRConfig();

function doSomethingThenMutate(){
    // Do something
    const newCat = client.cats.byId.mutate({id: 1}, {
        id: 1,
        name: 'Mr. Garfield',
    });

    // Mutate the cache
    // All hooks using this key will now have the new data
    await mutate(trpc.cats.byId.getKey({id: 1}), newCat, false);
}

I think we can still do await trpc.cats.byId.mutate({ id: 1 }, newCat, false)

Prefetch data before React mounts directly into SWR cache

Again, instead of .getKey here I think trpc.cats.byId.preload would be better. So I prefer Potential API 2.
This also matches the @trpc/react API much better.

SWR's useSWRMutation API

Yes, would love to have support this. The API will probably work the same as useSWR.

Got it! Getting started as I have a bit more time on my hands :)

While building this I've met a slight problem. Essentially what we want is to only use a single TRPCClient instance, however, we cannot follow the API used by @trpc/react where you pass the client into the Context provider due to the fact SWRHook.preload() is supposed to kick off a request outside of React before the app mounts. Therefore the client must be available before React hydrates.
These are not alternatives, both can be used.
I overloaded the hooks creator function to accept an optional client like so:
// Option 1
export const trpc = createSWRProxyHooks<AppRouter>(config); // Similar to current API, generates the client

// Option 2 - If you need the native client for whatever reason
const client = createTRPCClient<>(config); // From @trpc/client

export const trpc = createSWRProxyHooks(
  null,
  client
); // Will use the provided client.

// Use the native client proxy for whatever reason (custom requests, custom mutations etc.), example: Init the @trpc/client with this
export const trpcNative = createTRPCClientProxy(client); // imported from @trpc/swr

Both of the options currently only use a single instance of the native TRPCClient. We do have the option of leveraging trpc.createClient() and injecting it through context, however it would require some modifications so .preload() has access to a global instance of this as long as its called top level before any .preload() calls

No effects will run before the App is mounted & hydrated, in the case of SSR. For large apps this can be quite a bit. Secondly preload() can be used to for instance kick off a request to trpc.users.byId when a user hovers a link. This is of course inside of react, but it cannot be one as a hook due to the fact that SWR's preload api is also not inside of react.
We can build a custom hook, but this ends up not reflecting how SWR's preload API works.
If we are making it compatible with SWR v2 either some magic global copy of the client should be used.
I did come up with another method however. Let me know what you think :)
// Option 1
export const trpc = createSWRProxyHooks<AppRouter>(config); // Similar to current API, generates the client

// Option 2 - If you need the native client for whatever reason
const client = createTRPCClient<AppRouter>(config); // From @trpc/client

export const trpc = createSWRProxyHooks.withClient(client);

I do see what you mean by the SSR sharing of the client (in standard nodejs ssr not lambdas). This might actually be a problem. In that case I propose a different solution:
Preload keeps its own trpc client
We keep the API as similar to @trpc/react meaning the client will be injected through context.
createSWRProxyHooks will only accept config which will be passed down.
// Client creator API
// lib/trpc.ts
export const trpc = createSWRProxyHooks<AppRouter>({
    links: [
        httpBatchLink({
            url: getApiUrl(),
        }),
    ]
})

// In _app.tsx
function MyApp({ Component, pageProps }) {
  const [client] = useState(() => trpc.createClient());

  return (
    <trpc.Provider client={client} >
      <Component {...pageProps} />
    </trpc.Provider>
  )
}

Minimal implementation:
// As preload is a browser only function, request sharing will not matter on SSR.
let client: TRPCClient<AnyRouter>;
const preload = (pathAndInput, ...args) => {
   if(typeof window === "undefined") return Promise.resolve();

    return _SWRPreload(pathAndInput, () => {
        if(!client) {
            client = createClient();
        }
        return client.query(...pathAndInput, ...args);
    });
}

With this approach we can keep the API as similar as possible to the react one. The client is not shared accross SSR requests and preload is a noop on SSR.
Once the trpc.Provider component mounts it will directly set the client variable (can be global cache too). This means preload() will not have to create another client if its used to preload data on hover or something similar.
If however, preload is used before React, it will create its own client and dispatch the request. trpc.Provider will take over once app mounts and the old client will be garbage collected. We can probably use refs to manually clear it from memory as well.

Sorry for the late response. I'm a bit confused, which API did you decide to go with for preload?

I will get started on SWRInfinite, however as this is a codesplit package, how do you propose we use it? Similar to the current API?

I like infinite.cats.use(...args).

I went with tRPC/react's api with a few modifications.

You create the hooks with the tRPC config
You call createClient and inject in the provider without config
Preload will use its own client until react hydrates then use the context provided client.
Preloads client will be Garbage collected

The only difference between this and tRPC/react'a api is that the config has to go inside the createHooks method and not create client. It's a necessary compromise to make preload work.

Update:
useSWRInfinite has been successfully implemented. All that remains now is typing the non-proxy hooks as we also export their methods.
97.7% Test coverage. Some quirks on infinite I have to iron out.
New infinite API's .use() & .useCursor()
react-query contains custom logic to use infinite queries. SWR lets this the user decide if they want to use cursors, pagination or any other kind of use case. To be more compliant with @trpc/react we now have two methods.
infinite.<endpoint>.use() & infinite.<endpoint>.useCursor(). The .use api is a tRPC ified version of SWR. The paging logic must be implemented by hand which can cover almost all use cases. useCursor()'s API is made to mimmic @trpc/react. Its a helper function providing the cursor paging logic out of the box, makes it really easy to integrate with prisma on the back-end like useInfiniteQuery does.
infinite.<endpoint>.use()
Manual integration of the paging. Any kind of logic can be used here. ID fetching, cursor etc.
const infinite = createSWRInfiniteProxy(trpc)

	const Component = () => {
		const { data, setSize, size } = infinite.people.get.use((index, previousPageData) => {
			if (index !== 0 && !previousPageData?.length) return null // Last page
			return { limit: 1, page: index }
		})

		if (!data) {
			return
Loading...

		}

		const people = data.flatMap((page) => page)

		const hasMore = (data.at(-1) ?? []).length > 0

		return (
			<>

					{people.map((user, index) => {
						return
{user.name}

					})}

				{hasMore && <button onClick={() => setSize(size + 1)}>Load More</button>}
			</>
		)
	}

infinite.<endpoint>.useCursor()
Prebuilt cursor logic
const infinite = createSWRInfiniteProxy(trpc)

	const Component = () => {
		const { data, setSize, size } = infinite.people.get.useCursor({limit: 5},
                         (data) => data?.nextCursor
                 )

		if (!data) {
			return
Loading...

		}

		const people = data.flatMap((page) => page.items)
		const hasMore = !!data.at(-1)?.nextCursor

		return (
			<>

					{people.map((user, index) => {
						return
{user.name}

					})}

				{hasMore && <button onClick={() => setSize(size + 1)}>Load More</button>}
			</>
		)
	}

Both these are now active in my fork and on<EMAIL_ADDRESS>on npmjs. I'll upgrade the latest trpc core packages to the newest beta later.

Thanks for the update! That looks awesome

Update
Was waiting for tRPC v10 to release and got caught with work. I have some time on my hands to finalize this RFC.
New /next entry point with SSG/SSR helpers
We now export Next.js helpers, this includes a withTRPCSWR HOC, and a proxy helper function using server side calls.
The flow is as follows:

trpc proxy ssr helper is initialized in server context (inits router.createCaller()
trpc.<endpoints...>.fetch() is called. This can be awaited  (for users wanting sync behavior) or not .dehydrate() will await either way.

As we are using SWR, there is no point in making a fetch call, thus we use the caller to make this a server to server call.
The key is then serialized using SWR's serialization function.

trpc.dehydrate() is called. This will serialize the data using any transformer provided or just return it. Same behavior as @trpc/next

Usage
In any server util file
// server/ssg

export const createSSG = () => {
	return createProxySSGHelpers({
		router: appRouter,
		ctx: {},
		transformer: SuperJSON,
	})
}

In next _app
// _app.tsx
const App = ({ Component, pageProps }: AppProps) => {
	const [client] = useState(() => trpc.createClient())
	return (
		<trpc.Provider client={client}>
			<Component {...pageProps} />
		</trpc.Provider>
	)
}

export default withTRPCSWR({
	transformer: SuperJSON,
})(App)

In any SSG/SSR route
export const getStaticProps: GetStaticProps = async () => {
	const trpc = createSSG()

	trpc.user.byId.fetch({ id: 1 })
        trpc.user.byId.fetch({ id: 2 })
       // Can decide to await or not. This is useful if the user wants parallell execution for faster builds/ssr

	return {
		props: {
			swr: await trpc.dehydrate(), // Must be awaited
		},
	}
}

And thats it, the hooks will have hydrated state.
Online documentation
I'm currently working on documenting everything in this RFC as an online documentation using Nextra as the static generator.
@sachinraja how does this look? We can most likely make something similar to @trpc/next when you init the trpc client as well.

@sannajammeh Amazing! This RFC looks awesome.

That looks great @sannajammeh!
Would you be interested in taking over development of trpc-swr? No pressure, but I don't have the bandwidth to manage this project on top of my other ones and I myself don't use SWR. I'd be happy to transfer the GitHub repo and/or the npm package if you're interested. Thanks!

That looks great @sannajammeh!
Would you be interested in taking over development of trpc-swr? No pressure, but I don't have the bandwidth to manage this project on top of my other ones and I myself don't use SWR. I'd be happy to transfer the GitHub repo and/or the npm package if you're interested. Thanks!

I definitely don't mind taking over maintenance of trpc-swr! Would you need the npmjs username or email?

Thank you so much! I just need your username to transfer ownership.

Invited you!

Invited you!

I've accepted the npmjs code. Would you mind transferring the github repo as well?

Can't transfer the repo since your fork has the same name: https://github.com/sannajammeh/trpc-swr

Makes sense. Renamed it!

I think you might have to delete your form too according to these docs? In any case I've invited you as a collaborator so feel free to push your branches here.

Hello @sachinraja
I've publish the first release candidate of this package on npm. My fork is now deleted and changes have been pushed here.
Would you mind transferring the repo over to me so I can deploy the documentation on Vercel?

@sachinraja Any possibility for this?

@sannajammeh I think I sent you an invite to transfer the repo a day after you asked

@sannajammeh

@sannajammeh

We may have to abort and retry the transfer. It appears GitHub wont fulfill this one

Done, sent another transfer request.

@sannajammeh Any updates about this RFC?

@sannajammeh Any updates about this RFC?

It's currently in RC mode! I'm testing it in my dev environments and trying to collect any bugs before I release the V1.
https://trpc-swr.vercel.app

Hi everyone!
I've successfully tested this library in most environments now. All thats missing is a an E2E test with Next 13.
v1 will release by the end of this week :)

Update
Hit some weird dependency resolution issues with both TypeScript and Next 13. For some reason the very same resolution method works in SWR, but not here. If I'm unable to resolve this by tomorrow, I'll be switching to a more safe packaging method instead, similar to tRPC.
Example:

@trpc-swr/react - previously trpc-swr
@trpc-swr/next -  previously trpc-swr/next
@trpc-swr/infinite - previously trpc-swr/infinite

Essentially the problem is as follows:
When testing inside of the monorepo, internal packages of trpc-swr like /infinite cannot resolve the trpc-swr core package. I am using the exact same technique as SWR itself, yet the resolution is somehow different.
This only appears to happen in the monorepo and not external Next.js projects.

All updates will now arrive in #40

Thank you for your work!


[WIP] Transparent Encryption: CI tests

testing CI with rolling key updates which should prevent any stalls due to hosts being out of sync and also has cleanup logic to remove old keys which will flush any keys on a node.

This change is

test-me-please

test-me-please

test-me-please

:woman_facepalming: forgot to push delay code wasted a test run oops

test-me-please

test-me-please

bookInfo failed

test-me-please

Runtime FQDN failed

test-me-please

Added delay and retry to cilium-health it seems to be failing here but theory we might just not be waiting long enough for ipsec to come online.

test-me-please

2019-03-21T16:24:43.945874465Z 2019-03-21 16:24:43.945751 I | embed: rejected connection from "<IP_ADDRESS>:53512" (error "tls: failed to verify client's certificate: x509: certificate specifies an incompatible key usage", ServerName "")

pulling etcd upgrades.

test-me-please

test-me-please

test-me-please

test-me-please

testing CI with retry for cilium-health

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please

test-me-please

Closing for now we never got this to reliably pass and the rolling update patches are moved. Going to retry with base CI PR.


MONDO:0008664 and MONDO:0006928[Merge]

Mondo term (ID and Label)
MONDO:0008664
autosomal dominant neovascular inflammatory vitreoretinopathy
Reason for deprecation
The only Mondo equivalent X-ref on this term is SCTID:232016005 , which maps to "Proliferative vitreoretinopathy"
THe MIM number on this term says is the preferred name is VITREORETINOPATHY, NEOVASCULAR INFLAMMATORY with alternate names including "PROLIFERATIVE VITREORETINOPATHY; PVR"
The CUI x-ref on the term to merge with is UMLS:C0242852 , which is where SNOMED 232016005 maps within UMLS.
Term to be merged with
MONDO:0006928
proliferative vitreoretinopathy
Your nano-attribution (ORCID)
If you don't have an ORCID, you can sign up for one here

Hello @kanems
I'm Sabrina, a new curator who recently joined the Mondo team.
We will address your issue as soon as possible. Thank you for your patience.
If this is a high priority, please let me know and we can prioritize it. Thank you!


Gamma migrations

Fix a small access bug in migrations.
Add a script to deactivate genesis.

Not merged properly.


FizzBuzz Unique Style of Showing

FizzBuzz solution using SVG HTML CSS and JS
I don't think this solution conflicts with any other solution other than logic(it might)
If it does let me know
Thank you :)

Neat! Thanks :)

Thank you
This was my second solution
I will try to do FizzBuzz in other way too
Happy coding :)


Feature request: Transfer multiple files at once

Multiple files are not accepted by select dialog neither drag&drop.

Multiple files would be nice...however you can just zip up the files and send the single archive

Apple's Airdrop allow's this and it's extremely useful.
One of the most common use-case's is to transfer multiple photos from a phone to a computer...
Is there any plans to develop this feature?

Who want's to start with this?
I guess we would need to update:
https://github.com/cowbell/sharedrop/blob/master/app/components/file-field.js
https://github.com/cowbell/sharedrop/blob/master/app/templates/components/peer-widget.hbs
https://github.com/cowbell/sharedrop/blob/master/app/controllers/index.js
I'm unsure if we need multiple file-field's or the file-field would need to pass the file array to the uploadFile method

I am really interested to help on this, will have some time this weekend 👍

Hmm, zip is a good idea but that makes it much more complex I guess. I imagine a smartphone where a user selects 500 images which would need to be packaged up to a zip file. Which either doesn't work at all due to ram limitations or would require a progress bar.
But let's create a protoype with https://www.npmjs.com/package/jszip
maybe we can also use something like this https://www.npmjs.com/package/zip-stream to stream the data right away so that the data can already transfered.

Actually is seems to be possible to use https://stuk.github.io/jszip/documentation/api_jszip/generate_internal_stream.html, https://stuk.github.io/jszip/documentation/api_zipobject/internal_stream.html to generate a zip stream.

@valoricDe Thanks again for implementing this feature! It is deployed to production 🎉


GPU is idle even when there are operations ready to be executed

Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04.1 LTS
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): 1.12.0
Python version: Python 2.7.15rc1
Bazel version (if compiling from source): 0.21
GCC/Compiler version (if compiling from source): (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
CUDA/cuDNN version: 10.0/7.4.2
GPU model and memory: NVIDIA Corporation GP100GL [Tesla P100 PCIe 12GB] (2 GPUs)

You can collect some of this information using our environment capture script
You can also obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the current behavior
I am running a toy matrix multiplication example (code here and timeline can be found here). If u look at the timeline, operations named MatMul_1 and MatMul_2 are placed on GPU1 while all other matrix multiplications are placed on GPU0. Operation named MatMul_2 takes input from operations named MatMul and MatMul_1. MatMul and MatMul_1 are completed almost at the same time while there is a large gap after completion of these operations before the execution of MatMul_2 starts. I am not sure why is this gap there ?
Describe the expected behavior
Expected behavior according to my understanding is that as soon as MEMCPYPtoP has completed (Op named MatMul is placed on GPU0), excution of MatMul_2 should be started. Is this a performance issue or am I missing something here ?
Code to reproduce the issue
toy_matmul code
Other info / logs
timeline for toy_matmul
Following is the snapshot of the timeline using chrome-tracing visualizer.

Un-assigning myself as this does not seem related to distribution strategy. Perhaps someone on the GPU team can look into it.

Can it be reproduced? I tried your script on GCE and it is executed as you expected

In this execution, it looks like this control dependency is ignored. Not sure why. Can you please explain this behavior ?
Thanks!

Sorry for delay.
First for the control dependency, I think it is expected. The control dependency is created on placeholder. When you call session.run, you already fulfilled the placeholder. So the ops depends on it is ready to run as well.
To fix it, you should move line 38 and 41 out from the control_dependency context.
And back to the original issue, it is actually caused by the profiler. In order to record the runtime of each kernel, our profiler calls cuCtxSynchronize to synchronize CPU and GPU. Unfortunately, right now we call this on the main thread, which blocks scheduling. This issue should go away if you turn off profiling.
We have an experimental feature that create private threads for scheduling, which can fix this issue as well. To turn it on, you need to set two environment variable before executing tf:
export TF_GPU_THREAD_MODE=gpu_private export TF_GPU_THREAD_COUNT=2
This is the timeline I generated after setting these two variables:

I am still investigating the gap between MatMul and memcpy.

Hi @qqfish thanks for your input. I moved the lines outside the control_dependency context and set the environment variables as you suggested. It appears to be working fine for me now (Not sure why are you still seeing a gap in your execution) as shown in the timeline below:
timeline JSON file can be found here
Updated toy_matmul_code can be found here

Next step for my use-case is to try similar settings for larger real models e.g. RRNLM (Recurrent Neural Network for Language Modeling), Inception, ResNet etc. Do you think these environment variable settings are safe to use for those bigger and more complicated control flow graphs as well ?

Yes, it should be safe. We may turn this feature on by default recently.
Previous we use one thread on GPU to compute CPU ops and schedule kernels on GPU. These variables create a private scheduling thread for each GPU. Then we don't need to do everything in one thread.
There are two gap in the original timeline: (1) between MatMul and MemCPYPtoP (2) between MemCPYPtoP and MatMul_2. The environment variables should address gap (2). However, (1) is non-deterministic
I am working on a fix to address (1).

Hi @qqfish
for variable TF_GPU_THREAD_COUNT. Value should be equal to number of GPUs or it should always be 2 ?
Also, after setting both the environment variables, for execution on my local machine 'both the gaps' [(1) and (2) in your last comment] are gone, but not for your case ?
Do you think for me GAP has disappeared by chance ? Like it may appear if I try to run the same thing again  or try to run a different example ?

If you set TF_GPU_THREAD_MODE=gpu_private then TF_GPU_THREAD_COUNT should always be 2. It means each GPU will have 2 private threads to launch ops.
Yes, I think gap (1) disappear in your case by change. If you rerunning the same thing, it will not appear. But the gap may appear again if running a different example.

@qqfish Thanks for the explanation. I will wait for your fix then before I try the other larger models. If I can be helpful in anyway, let me know.
Thanks!

Fixing this issue requires a much bigger change than I originally thought. I cannot find a easy workaround works for all cases. This may require to re-design op scheduling of TF to fully support this kind of fine-grained dependency.
The good news is, someone in our team has already started working on fine-grained dependency support. But it may take a while.

@qqfish Thanks for the update. Do we know approximately how much time will it take ?
Regards,
Ubaid

I am not sure as well. It is part of a larger project that is still in its early stage. So please stay tuned.

any updates ?

Sorry, not much yet. We are still working on it.

Just checking if there are any updates yet?

Can someone explain why TF_GPU_THREAD_COUNT should be 2 when gpu_thread_mode is private? Why not larger numbers?
Also, how about when thread_mode is shared? Shall I have larger TF_GPU_THREAD_COUNT be larger?
Thanks!

Can someone explain why TF_GPU_THREAD_COUNT should be 2 when gpu_thread_mode is private? Why not larger numbers?

When TF_GPU_THREAD_MODE is set to gpu_private, then TF_GPU_THREAD_COUNT is the number of threads per GPU for kernel launching. For some reference / benchmark models we find using 1 or 2 threads per GPU perform well. You can customize it to larger numbers, but be aware of contention if the total number of GPU private threads, data processing threads, and other CPU computation threads.

Also, how about when thread_mode is shared? Shall I have larger TF_GPU_THREAD_COUNT be larger?

When set to shared, it will be the total number of threads for kernel launching on all GPUs. It's supposed to be a larger number (for example, 2 * num_gpus).

Hi @Haoyuz,
Thanks. Is this TF_GPU_THREAD_COUNT thread pool for computation? I'm not very familiar with tensorflow runtime, when you said "kernel launching", do you mean a single GPU can only use 2 threads for computation?

Is this TF_GPU_THREAD_COUNT thread pool for computation?

Thread pool is for CPU, and this number is controlling how many CPU threads you want to schedule GPU kernels. You do not have to worry about the number of "threads" on GPU (technically GPU has different concepts controlling different levels of parallelism, and a single GPU stream running ops with a large enough batch size can usually saturate the GPU).

@qqfish, just checking, any updates yet ?

Sorry, there is still not much yet. I think you shouldn't be blocked by this issue.

Hi @qqfish,
I am trying to create a framework for speeding up TensorFlow programs using model parallelism. When one of the GPUs stays idle for sometime even when there are operations ready to be executed, performance improvement is not as expected and is sometimes worse than the CPU, negatively impacting the performance of my framework.
If you can share a temporary workaround for now. That will be helpful as well.
Thanks!

First, I think TF_GPU_THREAD_MODE is good enough. In our multi-gpu benchmark, we don't see any performance issues causing by this.
If you find any gap causing by this issue in timeline, I think you can use control dependency to work it around. You can add a tf.Identity with control dependency to make sure the Memcpy is scheduled before other heavy gpu ops.

@qqfish, Can you please elaborate this? I am not an expert in TensorFlow yet, sorry :( .
(If you can give a small toy example that will help a lot!)

If you find any gap causing by this issue in timeline, I think you can use control dependency to work it around. You can add a tf.Identity with control dependency to make sure the Memcpy is scheduled before other heavy gpu ops.

Thanks!

Let say you have a graph like this:
with tf.device('cpu:0'):
  X0 = tf.random_uniform([dim, dim], 0, 10)

with tf.device('gpu:0'):
  Y0 = tf.matmul(X0, X0)

with tf.device('gpu:1'):
  X1 = tf.random_uniform([dim, dim], 0, 10)
  with tf.control_dependencies([X0]):
    Y1 = tf.matmul(X1, X1)

If you notice memory copy is execute after Y1 on gpu, you can work around by adding following control dependency:
with tf.device('cpu:0'):
  X0 = tf.random_uniform([dim, dim], 0, 10)
  X0_ = tf.identity(X0)

with tf.device('gpu:0'):
  Y0 = tf.matmul(X0, X0)

with tf.device('gpu:1'):
  X1 = tf.random_uniform([dim, dim], 0, 10)
  with tf.control_dependencies([X0_]):
    Y1 = tf.matmul(X1, X1)

or:
with tf.device('cpu:0'):
  X0 = tf.random_uniform([dim, dim], 0, 10)

with tf.device('gpu:0'):
  Y0 = tf.matmul(X0, X0)

with tf.device('gpu:1'):
  with tf.control_dependencies([X0]):
    X1 = tf.random_uniform([dim, dim], 0, 10)
    Y1 = tf.matmul(X1, X1)

But like I said, it is a very rare issue, you don't need to worry about it too much.

Closing as per https://github.com/tensorflow/tensorflow/issues/25724#issuecomment-522719673: this is a rare issue that has a workaround.


feat(connector): [PayU] Use connector_response_response_id as referen…

Type of Change

[ ] Bugfix
[x] New feature
[ ] Enhancement
[ ] Refactoring
[ ] Dependency updates
[ ] Documentation
[ ] CI/CD

Description
Fixes #2344
Additional Changes

[ ] This PR modifies the API contract
[ ] This PR modifies the database schema
[ ] This PR modifies application configuration/environment variables

Motivation and Context
Fixes #2344
How did you test it?

Checklist

[x] I formatted the code cargo +nightly fmt --all
[x] I addressed lints thrown by cargo clippy
[x] I reviewed the submitted code
[ ] I added unit tests for my changes where possible
[ ] I added a CHANGELOG entry if applicable

@srujanchikke thanks for review. I have done required changes..

@srujanchikke I have done required changes, do I need to use clone as well here? Or not required?

@srujanchikke I have done required changes, do I need to use clone as well here? Or not required?

Yes , you need to use clone here
resource_id: types::ResponseId::ConnectorTransactionId(item.response.order_id.clone()),

Hey @harshalkh ,
Please use the following cargo commands -
cargo check --all-features
cargo clippy --all-features
ref - Cargo commands, and fix the errors thrown by clippy for the checks to pass .
Please run cargo +nightly fmt  to fix failing formatting check

Hey @harshalkh ,
Please use the following cargo commands -
cargo check --all-features cargo clippy --all-features
ref - Cargo commands, and fix the errors thrown by clippy for the checks to pass .
Please run cargo +nightly fmt to fix failing formatting check

Thanks I will check

Hey @harshalkh  👋,
Thanks a bunch for all your contributions! We've got a little something for you to show our appreciation.
Just take a moment to fill out this form, and get ready for some awesome swag coming your way.
Thanks!


Test on Windows

Test if node-lmdb can be built on Windows and test it by running the example code in the repository.

@JuniperChicago Is there a way to specify the minimum msvs version in the binding.gyp or package.json somehow?

@JuniperChicago Is there a way to specify the minimum msvs version in the binding.gyp or package.json somehow?

@Venemo I have to make a correction regarding the success of my build. I merged onilabs/node-lmdb/commit/a43d460eb1bbcd4160c6fc117509c42d36e39094 then used -msvs_version-2013 on the command line where the build spews lots of warnings but completes. As to your question, I do not know how to specify minimum msvs version other than at the command line.

Thank you @JuniperChicago :) I have a few more questions:

Does it actually work with your patches and -msvs_version-2013?
Could you pastebin the full output of the thing including warnings and everything?

@Venemo Here is the pastebin with my console.log output. Yes, the build works fine building off my fork, which I am calling from 'package.json'. Although I am getting some weirdness where txn.putString() is somehow causing keyIsUint32 error to be returned even though I am using a string for the key and I do not have the store set to expect `Uint32' types for keys, but I can avoid the situations where this error comes up so far.

@JuniperChicago Thanks for your help! About this keyIsUint32 error, could you give me some code that shows when it happens?

@Venemo Thanks for the help. I managed to sort out the ordering of transactions and edge cases cases where I was escaping functions before transactions were closed. So, I this error is not an issue anymore. Again, thanks.

@JuniperChicago That's nice, but I'd still appreciate a sample code to reproduce the issue. Maybe I can add more meaningful error messages.

on a windows 10 box with Node.js 5.2 and VisualStudio 2013 (able to compile other project like redis' c/c++ without issue so evn flags are all setup and working).
cd \
mkdir test
npm install --msvs_--save version=2013 node-lmdb

getting this as output.
><EMAIL_ADDRESS>install C:\test\node_modules\node-lmdb
> node-gyp rebuild

C:\test\node_modules\node-lmdb {git}
{lamb} if not defined npm_config_node_gyp (node "C:\nodejs\node_modules\npm\bin\node-gyp-bin\\..\..\node_modules\node-gyp\bin\node-gyp.js" rebuild )  else (node  rebuild )
Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
  mdb.c
  midl.c
  win_delay_load_hook.c
..\libraries\liblmdb\midl.c(43): warning C4267: 'initializing' : conversion from 'size_t' to 'unsigned int', possible loss of data [C:\test\node_modules\node-lmdb\build\node-lmdb.vcxproj]
..\libraries\liblmdb\midl.c(147): warning C4267: '+=' : conversion from 'size_t' to 'unsigned int', possible loss of data [C:\test\node_modules\node-lmdb\build\node-lmdb.vcxproj]
..\libraries\liblmdb\midl.c(177): warning C4267: 'function' : conversion from 'size_t' to 'int', possible loss of data [C:\test\node_modules\node-lmdb\build\node-lmdb.vcxproj]
..\libraries\liblmdb\mdb.c(76): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\test\node_modules\node-lmdb\build\node-lmdb.vcxproj]
gyp ERR! build error
gyp ERR! stack Error: `C:\Program Files (x86)\MSBuild\14.0\bin\msbuild.exe` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onExit (C:\nodejs\node_modules\npm\node_modules\node-gyp\lib\build.js:270:23)
gyp ERR! stack     at emitTwo (events.js:88:13)
gyp ERR! stack     at ChildProcess.emit (events.js:173:7)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:201:12)
gyp ERR! System Windows_NT 10.0.10586
gyp ERR! command "C:\\nodejs\\node.exe" "C:\\nodejs\\node_modules\\npm\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild"
gyp ERR! cwd C:\test\node_modules\node-lmdb
gyp ERR! node -v v5.2.0
gyp ERR! node-gyp -v v3.0.3
gyp ERR! not ok
npm WARN EPACKAGEJSON<EMAIL_ADDRESS>No description
npm WARN EPACKAGEJSON<EMAIL_ADDRESS>No repository field.
npm ERR! Windows_NT 10.0.10586
npm ERR! argv "C:\\nodejs\\node.exe" "C:\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "--msvs_version=2013" "--save" "node-lmdb"
npm ERR! node v5.2.0
npm ERR! npm  v3.3.12
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the node-lmdb package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-gyp rebuild
npm ERR! You can get their info via:
npm ERR!     npm owner ls node-lmdb
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     C:\test\npm-debug.log

I know this is an old thread but seeing if there is a known fix.

@delaneyj The trouble seems to be this line:
..\libraries\liblmdb\mdb.c(76): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory [C:\test\node_modules\node-lmdb\build\node-lmdb.vcxproj]

So it's one of LMDB's source files that would need unistd.h which it can't find.
I'd need to look around and see what needs to be done to build LMDB on Windows.

Not sure but would a library like this one provide you with what you need, also there is this?

I can get it to compile on Windows (and working) by just dropping these files into the directory:
liblmdb_win.zip

Can those files be added or a binary for Windows be provided?  I have a catch 22 where npm won't create the directly until its finished installing but can't finish install without those headers.

@delaneyj You can download the node-lmdb source from git, add the files posted above to 'node-lmdb-master/libraries/liblmdb/' and then just do 'npm install node-lmdb-master' and it will compile the code and you can use 'require("node-lmdb")' as normal

wouldn't it be more universal to just include the 3 headers?  This way is making the use of lmdb a special  case in a normal npm workflow.

@mrboggieman Can you give some more explanation on what exactly those files are? Could I include them in the source tree?

Personally I don't know, definitely not a c/c++ node guy..  but if i drop those files into folder with the rest of the code and use node-gyp configure; node-gyp build it does in fact build.  I know io-redis compiles on Windows without these types of workaround.  Any testing I can do to make this better from a user perspective please let me know.

The other question is that of licensing - do the files come with a license that is compatible with that of node-lmdb?

Gotcha, its iteresting the files linked above appear to be LGPL, but as found an MIT version.  The joys of licensing :P

@Venemo The files polyfill the linux functionality missing in windows for handling parsing command arguments and os specific calls like fork and wait. To make compiling work with linux and windows it is likely that os specific checks should be used to include the files (rename them to prevent a clash with the linux version). I am no C++ developer though so will leave it up to someone who knows what they are doing. I tried compiling with other polyfills with different licences but had issues linking.

@mrboggieman @delaneyj I'll try to do my best. I'll try to make it work with an MIT-licensed variant but if it doesn't work out, would you guys be okay with the LGPL implementation?

@Venemo the licence works for me. As this is a wrapper library I think LGPL is suitable

testing on windows 10 compared to OS X i would like to understand why example7-largedb states a db site of a constant 16384 MB while the node version on OS X comes up with 2.2GB-2.5GB .. i would assume thats because of system ram compression on OS X, but would prefer some more details on that.
PS: i added the files mentioned above to the windows folder before it would build. @Venemo did you find something MIT based ?

@matthiasg I believe the size difference is because memory-mapped files behave differently on Windows. I recall reading something about this, but can't find the link right now.

Okay, so it seems that as of now, node-lmdb can be built correctly on Windows. It is quite painful though (maybe because I don't develop it on Windows), so I added a section that details how to do it.


[Question] How to skip assembly when dependencies haven't changed

I have a plugin that uses sbt-assembly as a dependency of creating a Python .whl.  during development and testing of the .whl, the assembly process reruns every time, even if none of the Scala code or other dependencies have changed. Is there some way to have sbt-assembly skip rebuilding the assembly in this case? IOW, only run the full process when its dependencies have changed?

Is this a question or a feature request? (All questions are redirected to Stackoverflow per guideline)
I think some parts are cached, but maybe not fully.

Is this a question or a feature request?
🤷♂ is calling assembly twice in a row supposed to bypass building the second time? In our configuration that's not happening, so if it's supposed to do that and isn't, it's either a) a bug for me to report, or b) an SO question on which switch I'm not seeing in the docs, or c) a feature request.

is calling assembly twice in a row supposed to bypass building the second time?

https://github.com/sbt/sbt-assembly/blob/edd35cfbaf05c3465371b63d38fda8ac579d766c/src/main/scala/sbtassembly/Assembly.scala#L92-L103
Looking at the code, I think it's supposed to cache if nothing changed.
sbt:hello> clean
[success] Total time: 0 s, completed Oct 11, 2019 4:39:27 PM
sbt:hello> assembly
[info] Compiling 1 Java source to /Users/eed3si9n/work/hellotest/target/scala-2.12/classes ...
[info] Compiling 1 Scala source to /Users/eed3si9n/work/hellotest/target/scala-2.12/test-classes ...
[info] foo.SomethingTest
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] Strategy 'discard' was applied to a file (Run the task at debug level to see details)
[info] Strategy 'rename' was applied to 2 files (Run the task at debug level to see details)
[success] Total time: 4 s, completed Oct 11, 2019 4:39:34 PM
sbt:hello> assembly
[info] foo.SomethingTest
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] Strategy 'discard' was applied to a file (Run the task at debug level to see details)
[info] Assembly up to date: /Users/eed3si9n/work/hellotest/target/scala-2.12/hello-assembly-0.1.0-SNAPSHOT.jar
[success] Total time: 0 s, completed Oct 11, 2019 4:39:39 PM

Thanks for your help. I think the problem I had was that I'm using sbt-buildinfo (with a build date in it), so technically the source changes every time, triggering the rebuild. I appreciate you looking into it!


Kyber package & MSRV

Problem
At the moment, pqc_kyber has a critical vuln meaning that it should not be used (see #42).

pqc_kyber - has a non-constant time bug that has not been patched. I am swapping to the ml_kem crate for now which which has this specific issue handled.

The alternative package with a fixed implementation is ml_kem  which has an msrv of 1.74 which means that it's use will require bumping our msrv.

closing this because I am sticking with ml-kem / x-wing crate(s)


Maintenance generates multiple alerts on status pages

⚠ Please verify that this bug has NOT been raised before.

[X] I checked and didn't find similar issue

🛡 Security Policy

[X] I agree to have read this project Security Policy

Description
I have noticed that during the active period of a maintenance, the maintenance alert is displayed for each heartbeat. Thus causing an infinity of repeated alerts.
Maintenance configuration

Error

👟 Reproduction steps

Create a maintenance
Be in the maintenance interval
Check the status page and there you will see a notification for each maintenance heartbeat

👀 Expected behavior
That only one alert was to appear during active maintenance
😓 Actual Behavior
An alert appears for each maintenance heartbeat
🐻 Uptime-Kuma Version
1.19.2
💻 Operating System and Arch
Ubuntu 20.04.5 LTS
🌐 Browser
Google Chrome 108.0.5359.125
🐋 Docker Version
Docker version 20.10.22, build 3a2c30b
🟩 NodeJS Version
v16.13.2
📝 Relevant log output
No response

Happening to me as well on a recurring monthly monitor.


endianess of ip address data from __sk_buff

Hello,
When I first looked at the mask below (0xffffff00), I thought that it would incorrectly mask the MSB because I thought that the IP address would be in network byte order but when I printed the IP address, I realized that it was masking off the right byte but the address was backwards (<IP_ADDRESS> instead of <IP_ADDRESS> for instance). It would seem that the address is in little endian (which in my case is the host endianness) given the mask and the bytes being masked off (after the mask I get <IP_ADDRESS>). Why isn't the IP address in network byte order?
static FORCE_INLINE void ipv4_hash(struct in_addr ip, struct address_hash *a, struct address_hash *b)
{
        bpf_printk("IP: %pI4", &ip.s_addr);

        a->vals[ADDRESS_IP] = fasthash64(&ip, sizeof(ip), FH_SEED);
	b->vals[ADDRESS_IP] = hashlittle(&ip, sizeof(ip), L3_SEED);
	ip.s_addr &= 0xffffff00;
	a->vals[ADDRESS_NET] = fasthash64(&ip, sizeof(ip), FH_SEED);
	b->vals[ADDRESS_NET] = hashlittle(&ip, sizeof(ip), L3_SEED);
}

I also noticed that __sk_buff has a some ip address fields. Couldn't we use those fields instead of load_word and load_ipv6?
...
	__u32 remote_ip4;	/* Stored in network byte order */
	__u32 local_ip4;	/* Stored in network byte order */
...

If I remember correctly this is because we use load_word to load the ipv4 addresses, which does a byte swap. So rather than swapping again we just deal with little endian ips.
remote_ip4 and local_ip4 are not available from the socket filter context if I remember correctly.


Add tinting support to UIImage

Currently, UIImage always uses white color to render. This PR adds tinting support, mainly for animations that will be added in a follow-up.

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/607/
Hooray Jenkins reported success with all tests good!

Neat! :-)


LUTECE-2085 : Make daemons reactive

Add a daemon scheduler interface and implementation.
The scheduler can schedule/unschedule daemon runs, as was possible before. Add the possibility to run a daemon on demand, possibly after a delay.
Demand run and scheduled run rely on the same underlying mechanism.
Add a button to run a daemon immediately on the manage daemons page
Turn the MailSenderDaemon into a reactive one.  Enqueuing a mail signals the daemon.
Use the daemon signaling system for error recovery or rate limiting
Make the threadLauncherDaemon reactive.
Have the daemon signaled when the runnable ends.
This allows dead threads reclaiming, and execution of delayed runnable
which where blocked from execution because no slot was available for their
key.

Can you elaborate a little on why we replace the previous API (ScheduledThreadPoolExecutor) in favor of the lower level API (ThreadPoolExecutor + Timer) ?
Is it to allow to coalesce things together ?

This allows to have a unified behavior. The timer does not execute the daemon directly, but merely signals it. This also helps for the coalescing.

Pushed the change to config files

Merged, thanks !!

Jenkins got intermittent test failures on job lutece-core-postgresql-doc-QA/123/consoleFull
[ERROR] Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.96 s <<< FAILURE! - in fr.paris.lutece.portal.web.system.DaemonsJspBeanTest
[ERROR] testDoDaemonActionRun(fr.paris.lutece.portal.web.system.DaemonsJspBeanTest)  Time elapsed: 1.524 s  <<< ERROR!
java.util.concurrent.TimeoutException
	at fr.paris.lutece.portal.web.system.DaemonsJspBeanTest.testDoDaemonActionRun(DaemonsJspBeanTest.java:134)

Yeah, the daemon failed to start after 1.5s. The timeout could be bumped to accomodate the light jenkins infra.

Hi Remi,
I modified your tests. Could you review my recent patches:

b4d13cd4d2dfb75338da9718a72ce053abe1d5b6
f131fa29b28c5ff9314d85aac85a377ca8538ba5
4a83d4f9001240ce8882bd2e1d8ca6caf8545478

Thanks in advance

Yeah, the daemon failed to start after 1.5s. The timeout could be bumped to accomodate the light jenkins infra.

The tests are still failing from time to time, even with a 10s timeout. There's probably another problem here. Maybe 1bec3fa54aaa44de fixes it?

Still failing sporadically.
Here's a good run:
testDoDaemonActionStart
21/06/17 18:00:46 INFO  [Lutece-MainThread] lutece.event - New Daemon registered : JUNIT
21/06/17 18:00:46 INFO  [Lutece-Daemon-JUNIT] lutece.event - Daemon JUNIT - starts processing.
21/06/17 18:00:46 INFO  [Lutece-MainThread] lutece.event - Daemon scheduled after 3 ms
21/06/17 18:00:46 INFO  [Lutece-Daemon-JUNIT] lutece.event - Daemon JUNIT - end of process.
21/06/17 18:00:47 INFO  [Lutece-Daemon-JUNIT] lutece.event - Daemon JUNIT - starts processing.
21/06/17 18:00:47 INFO  [Lutece-MainThread] lutece.event - Daemon scheduled after 994 ms
21/06/17 18:00:47 INFO  [Lutece-Daemon-JUNIT] lutece.event - Daemon JUNIT - end of process.
21/06/17 18:00:47 INFO  [Lutece-MainThread] lutece.event - Stopping daemon 'JUNIT'
21/06/17 18:00:47 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:00:47 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:00:47 INFO  [Lutece-MainThread] lutece.event - Stopping daemon 'JUNIT'

and a bad run:
testDoDaemonActionStart
21/06/17 18:05:08 INFO  [Lutece-MainThread] lutece.event - New Daemon registered : JUNIT
21/06/17 18:05:18 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:05:18 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:05:18 INFO  [Lutece-MainThread] lutece.event - Stopping daemon 'JUNIT'
21/06/17 18:05:18 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:05:18 ERROR [Lutece-MainThread] lutece.error - Could not unschedule daemon JUNIT which was not scheduled
21/06/17 18:05:18 INFO  [Lutece-MainThread] lutece.event - Stopping daemon 'JUNIT'

I noticed that bad runs happen when classes are instrumented by cobertura.

I can imagine a scenario where this could happen : the daemon gets executed before AppDaemonService#scheduleThread entry.setIsRunning( true ) is executed. In this case, DaemonThread#run does not execute the daemon because of the check for isRunning.
I will submit a patch adding logs and fixing this scenario.

The tests stopped failing. Looks like this was the remaining bug. Thanks !