TF 2.11.0: tf.image.ssim return_index_map=True outputs wrong shape

Click to expand!
Issue Type
Bug
Have you reproduced the bug with TF nightly?
No
Source
source
Tensorflow Version
2.11.0
Custom Code
Yes
OS Platform and Distribution
MacOS 13.1
Mobile device
No response
Python version
3.10.6
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
The documentation for `tf.image.ssim` claims to output

> ... a tensor containing an SSIM value for each pixel for each image in batch if return_index_map is True

However, the output image is smaller than the source image (see example below).

Upon comparing with a more well-known library implemented in PyTorch, I believe the reason for such discrepency is due to the Conv2D padding used.

In PyTorch's implementation, a "SAME" padding is used

https://github.com/Po-Hsun-Su/pytorch-ssim/blob/3add4532d3f633316cba235da1c69e90f0dfb952/pytorch_ssim/__init__.py#L25

However, in the current Tensorflow implementation, "VALID" padding is used

https://github.com/tensorflow/tensorflow/blob/d5b57ca93e506df258271ea00fc29cf98383a374/tensorflow/python/ops/image_ops_impl.py#L4340

Please verify if this is the case.

Standalone code to reproduce the issue
import numpy as np
import tensorflow as tf # tf.__version__ == "2.11.0"
# B x T x n_mel
shape = (16, 2106, 80, 1)
image1 = np.arange(np.prod(shape))
image1 = (image1 / np.max(image1)) * 10 + 100
image1 = np.reshape(image1, shape)
image2 = np.linspace(0, 1, np.prod(shape))
image2 = np.exp(image2)
image2 = (image2 / np.max(image2)) * 10 + 100
image2 = np.reshape(image2, shape)

out_tf = tf.image.ssim(image1, image2, max_val=255, return_index_map=True)

which outputs a tensor of shape [16, 2096, 70].
Relevant log output
No response

@sapphire008,
I was facing a different issue/error while executing the mentioned code. Kindly find the gist of it here and provide the dependencies. Thank you!

@tilakrayal This is an issue with the latest release of tensorflow 2.11.0 as indicated in the original post under "Tensorflow Version". The colab version is still using 2.9.2.

@sapphire008,
Thank you for the issue.  In tf.image.ssim when we are using return_index_map = True, returns the index map; where as in the case of  return_index_map = False, it should return the reduced global value which was working as expected.
O/p for return_index_map = True :
2.12.0-dev20230201
(2695680,)
(2695680,)
(16, 2106, 80, 1)
(2695680,)
(2695680,)
(2695680,)
(16, 2106, 80, 1)
(16,)

O/p for return_index_map = False :
2.12.0-dev20230201
(2695680,)
(2695680,)
(16, 2106, 80, 1)
(2695680,)
(2695680,)
(2695680,)
(16, 2106, 80, 1)
(16, 2096, 70)

A tensor containing an SSIM value for each image in batch or a tensor containing an SSIM value for each pixel for each image in batch if return_index_map is True. Returned SSIM values are in range (-1, 1], when pixel values are non-negative. Returns a tensor with shape:
    broadcast(img1.shape[:-3], img2.shape[:-3]) or broadcast(img1.shape[:-1],
    img2.shape[:-1]).

Kindly find the gist of it here and also please have a look  at the reference where it is explicitly described that the global ssim value is the mean of the local ssim value: https://medium.com/srm-mic/all-about-structural-similarity-index-ssim-theory-code-in-pytorch-6551b455541e .

@tilakrayal I believe in your response has the return_index_map =True and return_index_map =False cases switched. However, the gist version correctly reproduced the problem I have described in the original post. Regardless, the output shape is (16, 2096, 70) when setting return_index_map=True, which is not consistent with the input shape of (16, 2106, 80). This is unexpected, because a index map of SSIM calculation is the scores corresponding to the original input, thus their shape should have been consistent. You can also find in my original post for a link to another implementation made in PyTorch, which do produce index map with shapes consistent with the inputs, and has been cited / reproduced by various repositories such as https://github.com/MoonInTheRiver/DiffSinger

Hi, @sapphire008
Apologize for the delayed response and I was able to replicate the same issue and I have executed the same code multiple times and It seems like working as expected with TF2.11 and I also tried to with our latest pre-release tensorflow==2.12.0rc1 and tensorflow==2.12.0rc0 for your reference I have added gist-file and I completely understood your point here the implementation in PyTorch with "SAME" padding  but in Tensorflow we are using "VALID" padding, If have I missed something here please let me know ? Thank you!

@gaikwadrahul8 Is there any reason why VALID padding is chosen instead of SAME padding in this specific implementation? The returned index_map is usually expected as the same size as the inputs, to my understanding. It's also how I see it could be used as a loss function.

Hi, @SuryanarayanaY
Could you please look into this issue ? Thank you!

@sapphire008 ,
The shape mismatch might be due to the padding as you noticed. As per the documentation the API the output is:
Returns a tensor with shape: broadcast(img1.shape[:-3], img2.shape[:-3]) or broadcast(img1.shape[:-1], img2.shape[:-1]).
But for our example the input is of shape (16, 2106, 80, 1) the API returns  output of shape (16, 2096, 70) which seems incorrect and the root cause is due to the padding='VALID'
I have tested the code by changing padding='SAME' and in this case the output is (16, 2106, 80).
All these details captured in attached gist.
It seems a bug for me and I am going to raise a PR for this. Thanks for reporting.

Hi @sapphire008, Thanks for your patience. I have tried by manually adding the padding to input that is being passed to the tf.image.ssim. I got the same shape as the input. please refer to this gist. Thank You.


"threathunting" index in Splunk is empty and not getting populated

Operating System Version: Arch Linux
Deploying via (VirtualBox/VMWare/AWS/Azure/ESXi): VirtualBox
Vagrant Version (if applicable): 2.2.16

I successfully set up the DetectionLab, and I get a ton of logs from the DC, WEF and WIN10.
However, the ThreatHunting app is kind of useless, because the index "threathunting" doesn't get populated. Scheduled tasks of ThreatHunting app don't fill this index, and I don't know why. ThreatHunting app's dashboards are all empty and show big zeros.
Did someone succeed getting this "threathunting" index populated ?
I can send logs if necessary.
Thanks in advance

I noticed something.
Let's take the example of the following query [T1136] Create Account :
`indextime` ((`sysmon` event_id=1) OR (`windows-security` event_id=4688)) (process_command_line="*New-LocalUser*" OR process_command_line="*net*user*add*")
| eval mitre_category="Persistence"
| eval mitre_technique="Create Account"
| eval mitre_technique_id="T1136"
| eval hash_sha256= lower(hash_sha256)
| `process_create_whitelist`
| eval indextime = _indextime | convert ctime(indextime) | table _time indextime event_description hash_sha256 host_fqdn user_name original_file_name process_path process_guid process_parent_path process_id process_parent_id process_command_line process_parent_command_line process_parent_guid mitre_category mitre_technique mitre_technique_id hunting_trigger
| collect `threathunting_index`

This query returns no results, even though I created a new account.
When I change windows-security event_id=4688 to windows-security EventCode=4688, it works properly !
So maybe there is a problem with the expansion of event_id ?
This problem happens for all ThreatHunting app queries, what explains why the threathunting index stays unpopulated...
In the search's log, I notice that in the case of windows-security, event_id seems to be expanded to RecordNumber instead of EventCode.
Logs are here : https://pastebin.com/dGuTm12U

I'm also getting the same error even after running a couple of tools. I can see the logs are visible to the Splunk discover tab but no data in Threat Hunting App

I've been digging into this for the last hour and there's really a bunch of issues going on, namely with the event_id and host_fqdn field aliases not being created widely enough. I'm honestly still trying to figure out the best way to "fix" this without breaking a bunch of stuff.

Hi @Vayper
This should be fixed now. Can you pull the latest changes down from master and try redeploying? To test, I just run the following on win10:
Import-Module "C:\Tools\AtomicRedTeam\invoke-atomicredteam\Invoke-AtomicRedTeam.psd1" -Force
$PSDefaultParameterValues = @{"Invoke-AtomicTest:PathToAtomicsFolder"="C:\Tools\AtomicRedTeam\atomics"}
Invoke-AtomicTest All

@clong it all works!! I can see now all the hits from the threat hunting apps after rebuilding it fresh from the latest master branch! Thank you so much!


[2.8.0] - Oauth client does not initialise

Type: breaking|critical
Describe the bug
The OAuth2 request resulted in an error.
Error: invalid_client
Description: Client authentication failed (e.g., unknown client, no client authentication included, or unsupported authentication method)
Hint: The requested OAuth 2.0 Client does not exist.
Debug:
You are seeing this default error page because the administrator has not set a dedicated error URL (environment variable OAUTH2_ERROR_URL is not set). If you are an administrator, please read the guide to understand what you need to do. If you are a user, please contact the administrator.

It is related to https://github.com/reactioncommerce/reaction-platform/issues/41
Might have been caused by changes in #602
I ran make with example-storefront, reaction and reaction-hydra at v2.7.0, the auth pages load fine, but when these at v2.8.0 Oauth client doesn't initialize. Then I tried reaction and reaction-hydra at v2.7.0 and example-storefront at v2.8.0, the was reproduced. After that, I ran reaction and reaction-platform at v2.8.0 and example-storefront at v2.7.0, auth client works.
To Reproduce
Steps to reproduce the behavior:

Click on Sign In or Create Account

Expected behavior
Auth page should load for Sign In/Sign up.

Seems to be introduced by #603
./bin/start is no longer called, consequently ./bin/create-hydra-client.js is not executed.
Manually calling ./bin/create-hydra-client.js from the container works around the issue.

How can one manually call it ./bin/start, sh /bin/start doesnt work for me.
I can't login on v2.9.0 all packages due to this same problem it seems, as my hydra-client never starts?

me too

Still getting this in 2.9

No reports of this happening after v3. Can this be closed? There's new Auth work going into storefront with https://github.com/reactioncommerce/example-storefront/pull/760 and closing out old auth issues seems prudent.


[APPLE] Screenshots and bgfx reset flags

In order to have screenshots working with Metal (for Validation tests), the bgfx reset flag with contain BGFX_RESET_CAPTURE or request screenshot will result in nothing to be done. Callback is not performed as well and validation tests will hang.
I don't want to add this flag for now on Apple platform until proper tests are being done.
This is annoying as bgfx Metal implementation differs from OpenGL/Direct3D.

Can we fix it in bgfx?

It should be fixed in bgfx so all platforms behave the same.

Maybe file a bug on bgfx.

With the bgfx version in this PR https://github.com/BabylonJS/BabylonNative/pull/467
The callback is performed but the data provided by bgfx are unusable.

screenshots now works on iOS but not on MacOS.

Issue created on bgfx repo : https://github.com/bkaradzic/bgfx/issues/2327

It works with this fix: https://github.com/bkaradzic/bgfx/pull/2329
I'll close this issue when bgfx update is done.


1620-check-if-array-pairs-are-divisible-by-k

Yet another leetcode solution :)

Review

Nice Job Buddy!!
Share and Star the repo!!
Happy Coding/Contributing :)
Follow My LinkedIn


clyde should always be online

i should just kill myself now

:eyes:

what's the point of this?

keeping the easter eggs up to date is important

💯

surely this would mean clyde.presence == clyde.presence would be false

presence.equals

😢


Get raw data for unsupported blocks

Not sure if this is the right place to ask this. But, it would really help if we can get the data for the unsupported blocks. It doesn't matter even if it's not structured properly. When you do add support for those blocks, you can add a proper structure to it.

I would love to see this being worked on, because even the simplest of blocks right now, like images, are not supported.

Is there a timeline for how long it will be until images are supported? would really appreciate it!

Hoping for code blocks as well.  Thanks!

Hi All, the team here is looking into supporting more, advanced blocks and currently laying the groundwork to enable additional features. We do not have a public roadmap or timeline we can offer at this time. I've taken the feedback here and added it to our internal feedback tracker and given it some weight (based on the thumbs up). If there are other feature requests specific to the API (independent of the SDK) please send them over to<EMAIL_ADDRESS>


Stomp alternative to sleep

Hi,
This is Krish, i'm working on Stomp with ruby..
In my Program  I kept sleep time as  0.5 and
Even after receiving the response from the Activemq Queue ..
The programs hangs for specified time mention with sleep..
I would like to have help on  below 2 points
1.what alternative to sleep , we have  in Stomp with activemq?
2.if response comes before sleep time mentioned how to move on with out waiting on complete the mentioned sleep time
please help me on this
Thanks & Regards,
Krish

The alternative to 'sleep' is for you to design your system so that it does not use sleep.
Design is your responsibility.
It certainly has nothing to do with STOMP or the stomp gem.


Cannot compile examples for Raspberry Pi Pico

When I try to compile the any example with the commands specified in README it fails with this.

We do not yet support libtock-rs for the Raspberry Pi Pico. From what I can read in the error, it seems that you are missing the armv6 toolchain.
Please try adding it using rustup target add thumbv6m-none-eabi before running cargo.

Looks to me like we do support the Raspberry Pi Pico: https://github.com/tock/libtock-rs/blob/master/runtime/layouts/raspberrypi_pico.ld

It used to build when I added it, but I think I had the armv6 toolchain already installed.

Got this failures after doing rustup target add thumbv6m-none-eabi
dump.txt.  Any ideas?
error: could not compile syn due to previous error__


Perf tweaks to prevent excessive calls to getHeaderId()

indexOutOfBounds() first to prevent an unnecessary call to getHeaderId()
getFirstHeaderPosition()  goes through every item until it finds a header id >= 0. By adding the check for position == 0 (after checking if the headerId is >= 0) we can remove the call to getFirstHeaderPosition() preventing potentially lots of getHeaderId().

Awesome! Thanks.  I'll do a thorough review this afternoon.

LGTM, thanks!


Breach of Github T&Cs?

Hi,
Just wondering if any sort of advice has been sought regarding whether or not this breaches the Github T&Cs?
I could do with using this action but I'm concerned it may end up getting my account limited/banned.
Cheers,
James

I haven't asked -- but why would you think this breaches T&C?
My warning about "voiding warranty" is more targeting at the facts, that

the action just rm -rfs preinstalled software without any consistency checks, and remounts storage in a way that could interfere with the VM setup ... and this in turn might or might not break your workflow (or even runner internals).
the action somewhat depends on the Azure VM type that is used by Github. Any change on their side might break the action.

As I understand it, each run on GH actions receives a fresh virtual machine that is terminated afterwards. So even if you totally break the VM, it should not affect other builds or customers, just your current build. But, I could totally understand that Github support rejects support requests when using this action, because it's just a hack.

I'm not saying it does, it just feels like something that MS would get upset about.
That being said, if you're confident enough of your last statement, then I can't see why there'd be a problem.


Support build har & publish package with CI

ref: https://docs.openharmony.cn/pages/v4.0/zh-cn/application-dev/tools/packing-tool.md

ohrs publish


Add Order Entry Retrieve PDF API Call

https://developer.intacct.com/api/order-entry/order-entry-transactions/#get-order-entry-transaction-pdf-data

@haislip, I reviewed the code and it looks good. I just changed the target branch to our 2.2.0 branch, which will be the next release and the date is currently TBD. Thank you for your contribution :)


Order of text fields in "Search for lyrics" window is inverted

The order/position of the Artist and Title fields in the Search for lyrics window does not match the order of the Artist/Title columns in the search result box.

Doesn't matter which field might be of higher relevance or whether the column order is customizable.
The order of the text fields should match the order of the columns - consistency is key.


fix(stdlib): Allow optional parameters and signature overloads in stdlib functions

Signature overrides weren't previously supported at all; now they are!  Optional parameters weren't previously supported in stdlib functions (e.g. StrI's radix parameter).  Those are now supported as well!
This includes a demo of that in the StrI function.

Hmm, considering that Rnd can accept 0 and return a float or can accept an integer and return an integer, this probably isn't quite the right solution here.  Might be better to support multiple signatures instead of a single signature with multiple argument types.
But if there's going to be multiple signatures, perhaps there's multiple implementations instead of trying to cram them together.  Does that mean this interpreter would just support multiple / dynamic dispatch of functions?

Is this a signature problem with Rnd() though?  Many functions will return
a dynamic value like an object or invalid.  Returning an integer or float
doesn't seem that big of a deal.  In the case of Sgn, can't it just accept
floats, and ints passed in are casted?  That one seems easy as the return
type is always an integer, no?
-Josh
On Wed, Nov 14, 2018 at 3:51 PM Sean Barag<EMAIL_ADDRESS>wrote:

Hmm, considering that Rnd can accept 0 and return a float or can accept
an integer and return an integer, this probably isn't quite the right
solution here. Might be better to support multiple signatures instead
of a single signature with multiple argument types.
But if there's going to be multiple signatures, perhaps there's multiple
implementations instead of trying to cram them together. Does that mean
this interpreter would just support multiple / dynamic dispatch of
functions?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sjbarag/brs/pull/69#issuecomment-438862546, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAwAJwqKCFOzQrMElYQX1arzMQSBJND_ks5uvKyVgaJpZM4Yey2j
.


1Password

//
//  1Password Extension
//
//  Lovingly handcrafted by Dave Teare, Michael Fey, Rad Azzouz, and Roustem Karimov.
//  Copyright (c) 2014 AgileBits. All rights reserved.
//
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>
#import <MobileCoreServices/MobileCoreServices.h>
#ifdef __IPHONE_8_0
#import <WebKit/WebKit.h>
#endif
#if __has_feature(nullability)
NS_ASSUME_NONNULL_BEGIN
#else
#define nullable
#define __nullable
#define nonnull
#define __nonnull
#endif
// Login Dictionary keys - Used to get or set the properties of a 1Password Login
#define AppExtensionURLStringKey                            @"url_string"
#define AppExtensionUsernameKey                             @"username"
#define AppExtensionPasswordKey                             @"password"
#define AppExtensionTOTPKey                                 @"totp"
#define AppExtensionTitleKey                                @"login_title"
#define AppExtensionNotesKey                                @"notes"
#define AppExtensionSectionTitleKey                         @"section_title"
#define AppExtensionFieldsKey                               @"fields"
#define AppExtensionReturnedFieldsKey                       @"returned_fields"
#define AppExtensionOldPasswordKey                          @"old_password"
#define AppExtensionPasswordGeneratorOptionsKey             @"password_generator_options"
// Password Generator options - Used to set the 1Password Password Generator options when saving a new Login or when changing the password for for an existing Login
#define AppExtensionGeneratedPasswordMinLengthKey           @"password_min_length"
#define AppExtensionGeneratedPasswordMaxLengthKey           @"password_max_length"
#define AppExtensionGeneratedPasswordRequireDigitsKey       @"password_require_digits"
#define AppExtensionGeneratedPasswordRequireSymbolsKey      @"password_require_symbols"
#define AppExtensionGeneratedPasswordForbiddenCharactersKey @"password_forbidden_characters"
// Errors codes
#define AppExtensionErrorDomain                             @"OnePasswordExtension"
#define AppExtensionErrorCodeCancelledByUser                    0
#define AppExtensionErrorCodeAPINotAvailable                    1
#define AppExtensionErrorCodeFailedToContactExtension           2
#define AppExtensionErrorCodeFailedToLoadItemProviderData       3
#define AppExtensionErrorCodeCollectFieldsScriptFailed          4
#define AppExtensionErrorCodeFillFieldsScriptFailed             5
#define AppExtensionErrorCodeUnexpectedData                     6
#define AppExtensionErrorCodeFailedToObtainURLStringFromWebView 7
// Note to creators of libraries or frameworks:
// If you include this code within your library, then to prevent potential duplicate symbol
// conflicts for adopters of your library, you should rename the OnePasswordExtension class
// and associated typedefs. You might to so by adding your own project prefix, e.g.,
// MyLibraryOnePasswordExtension.
typedef void (^OnePasswordLoginDictionaryCompletionBlock)(NSDictionary * __nullable loginDictionary, NSError * __nullable error);
typedef void (^OnePasswordSuccessCompletionBlock)(BOOL success, NSError * __nullable error);
typedef void (^OnePasswordExtensionItemCompletionBlock)(NSExtensionItem * __nullable extensionItem, NSError * __nullable error);
@interface OnePasswordExtension : NSObject

(OnePasswordExtension *)sharedExtension;

/*!
@discussion Determines if the 1Password Extension is available. Allows you to only show the 1Password login button to those
that can use it. Of course, you could leave the button enabled and educate users about the virtues of strong, unique
passwords instead :)
@return isAppExtensionAvailable Returns YES if any app that supports the generic org-appextension-feature-password-management feature is installed on the device.
*/
#ifdef __IPHONE_8_0

(BOOL)isAppExtensionAvailable NS_EXTENSION_UNAVAILABLE_IOS("Not available in an extension. Check if org-appextension-feature-password-management:// URL can be opened by the app.");
#else
(BOOL)isAppExtensionAvailable;
#endif

/*!
Called from your login page, this method will find all available logins for the given URLString.
@discussion 1Password will show all matching Login for the naked domain of the given URLString. For example if the user has an item in your 1Password vault with "subdomain1.domain.com” as the website and another one with "subdomain2.domain.com”, and the URLString is "https://domain.com", 1Password will show both items.
However, if no matching login is found for "https://domain.com", the 1Password Extension will display the "Show all Logins" button so that the user can search among all the Logins in the vault. This is especially useful when the user has a login for "https://olddomain.com".
After the user selects a login, it is stored into an NSDictionary and given to your completion handler. Use the Login Dictionary keys above to
extract the needed information and update your UI. The completion block is guaranteed to be called on the main thread.
@param URLString For the matching Logins in the 1Password vault.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block called with two parameters loginDictionary and error once completed. The loginDictionary reply parameter that contains the username, password and the One-Time Password if available. The error Reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/

(void)findLoginForURLString:(nonnull NSString *)URLString forViewController:(nonnull UIViewController *)viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;

/*!
Create a new login within 1Password and allow the user to generate a new password before saving.
@discussion The provided URLString should be unique to your app or service and be identical to what you pass into the find login method.
The completion block is guaranteed to be called on the main
thread.
@param URLString For the new Login to be saved in 1Password.
@param loginDetailsDictionary about the Login to be saved, including custom fields, are stored in an dictionary and given to the 1Password Extension.
@param passwordGenerationOptions The Password generator options represented in a dictionary form.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block which is called with type parameters loginDictionary and error. The loginDictionary reply parameter which contain all the information about the newly saved Login. Use the Login Dictionary keys above to extract the needed information and update your UI. For example, updating the UI with the newly generated password lets the user know their action was successful. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/

(void)storeLoginForURLString:(nonnull NSString *)URLString loginDetails:(nullable NSDictionary *)loginDetailsDictionary passwordGenerationOptions:(nullable NSDictionary *)passwordGenerationOptions forViewController:(nonnull UIViewController *)viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;

/*!
Change the password for an existing login within 1Password.
@discussion The provided URLString should be unique to your app or service and be identical to what you pass into the find login method. The completion block is guaranteed to be called on the main thread.
1Password 6 and later:
The 1Password Extension will display all available the matching Logins for the given URL string. The user can choose which Login item to update. The "New Login" button will also be available at all times, in case the user wishes to to create a new Login instead,
1Password 5:
These are the three scenarios that are supported:

A single matching Login is found: 1Password will enter edit mode for that Login and will update its password using the value for AppExtensionPasswordKey.
More than a one matching Logins are found: 1Password will display a list of all matching Logins. The user must choose which one to update. Once in edit mode, the Login will be updated with the new password.
No matching login is found: 1Password will create a new Login using the optional fields if available to populate its properties.

@param URLString for the Login to be updated with a new password in 1Password.
@param loginDetailsDictionary about the Login to be saved, including old password and the username, are stored in an dictionary and given to the 1Password Extension.
@param passwordGenerationOptions The Password generator options epresented in a dictionary form.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block which is called with type parameters loginDictionary and error. The loginDictionary reply parameter which contain all the information about the newly updated Login, including the newly generated and the old password. Use the Login Dictionary keys above to extract the needed information and update your UI. For example, updating the UI with the newly generated password lets the user know their action was successful. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/

(void)changePasswordForLoginForURLString:(nonnull NSString *)URLString loginDetails:(nullable NSDictionary *)loginDetailsDictionary passwordGenerationOptions:(nullable NSDictionary *)passwordGenerationOptions forViewController:(UIViewController *)viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;

/*!
Called from your web view controller, this method will show all the saved logins for the active page in the provided web
view, and automatically fill the HTML form fields. Supports both WKWebView and UIWebView.
@discussion 1Password will show all matching Login for the naked domain of the current website. For example if the user has an item in your 1Password vault with "subdomain1.domain.com” as the website and another one with "subdomain2.domain.com”, and the current website is "https://domain.com", 1Password will show both items.
However, if no matching login is found for "https://domain.com", the 1Password Extension will display the "New Login" button so that the user can create a new Login for the current website.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param yesOrNo Boolean flag. If YES is passed only matching Login items will be shown, otherwise the 1Password Extension will also display Credit Cards and Identities.
@param completion Completion block called on completion with parameters success, and error. The success reply parameter that is YES if the 1Password Extension has been successfully completed or NO otherwise. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/

(void)fillItemIntoWebView:(nonnull id)webView forViewController:(nonnull UIViewController *)viewController sender:(nullable id)sender showOnlyLogins:(BOOL)yesOrNo completion:(nonnull OnePasswordSuccessCompletionBlock)completion;

/*!
Called in the UIActivityViewController completion block to find out whether or not the user selected the 1Password Extension activity.
@param activityType or the bundle identidier of the selected activity in the share sheet.
@return isOnePasswordExtensionActivityType Returns YES if the selected activity is the 1Password extension, NO otherwise.
*/

(BOOL)isOnePasswordExtensionActivityType:(nullable NSString *)activityType;

/*!
The returned NSExtensionItem can be used to create your own UIActivityViewController. Use isOnePasswordExtensionActivityType: and fillReturnedItems:intoWebView:completion: in the activity view controller completion block to process the result. The completion block is guaranteed to be called on the main thread.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param completion Completion block called on completion with extensionItem and error. The extensionItem reply parameter that is contains all the info required by the 1Password extension if has been successfully completed or nil otherwise. The error reply parameter that is nil if the 1Password extension item has been successfully created, or it contains error information about the completion failure.
*/

(void)createExtensionItemForWebView:(nonnull id)webView completion:(nonnull OnePasswordExtensionItemCompletionBlock)completion;

/*!
Method used in the UIActivityViewController completion block to fill information into a web view.
@param returnedItems Array which contains the selected activity in the share sheet. Empty array if the share sheet is cancelled by the user.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param completion Completion block called on completion with parameters success, and error. The success reply parameter that is YES if the 1Password Extension has been successfully completed or NO otherwise. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/

(void)fillReturnedItems:(nullable NSArray *)returnedItems intoWebView:(nonnull id)webView completion:(nonnull OnePasswordSuccessCompletionBlock)completion;

/*!
Deprecated in version 1.5
@see Use fillItemIntoWebView:forViewController:sender:showOnlyLogins:completion: instead
*/

(void)fillLoginIntoWebView:(nonnull id)webView forViewController:(nonnull UIViewController *)viewController sender:(nullable id)sender completion:(nonnull OnePasswordSuccessCompletionBlock)completion attribute((deprecated("Use fillItemIntoWebView:forViewController:sender:showOnlyLogins:completion: instead. Deprecated in version 1.5")));
@end

#if __has_feature(nullability)
NS_ASSUME_NONNULL_END

//
// 1Password Extension
//
// Lovingly handcrafted by Dave Teare, Michael Fey, Rad Azzouz, and Roustem Karimov.
// Copyright (c) 2014 AgileBits. All rights reserved.
//
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>
#import <MobileCoreServices/MobileCoreServices.h>
#ifdef __IPHONE_8_0
#import <WebKit/WebKit.h>
#endif
#if __has_feature(nullability)
NS_ASSUME_NONNULL_BEGIN
#else
#define nullable
#define __nullable
#define nonnull
#define __nonnull
#endif
// Login Dictionary keys - Used to get or set the properties of a 1Password Login
#define AppExtensionURLStringKey @"url_string"
#define AppExtensionUsernameKey @"username"
#define AppExtensionPasswordKey @"password"
#define AppExtensionTOTPKey @"totp"
#define AppExtensionTitleKey @"login_title"
#define AppExtensionNotesKey @"notes"
#define AppExtensionSectionTitleKey @"section_title"
#define AppExtensionFieldsKey @"fields"
#define AppExtensionReturnedFieldsKey @"returned_fields"
#define AppExtensionOldPasswordKey @"old_password"
#define AppExtensionPasswordGeneratorOptionsKey @"password_generator_options"
// Password Generator options - Used to set the 1Password Password Generator options when saving a new Login or when changing the password for for an existing Login
#define AppExtensionGeneratedPasswordMinLengthKey @"password_min_length"
#define AppExtensionGeneratedPasswordMaxLengthKey @"password_max_length"
#define AppExtensionGeneratedPasswordRequireDigitsKey @"password_require_digits"
#define AppExtensionGeneratedPasswordRequireSymbolsKey @"password_require_symbols"
#define AppExtensionGeneratedPasswordForbiddenCharactersKey @"password_forbidden_characters"
// Errors codes
#define AppExtensionErrorDomain @"OnePasswordExtension"
#define AppExtensionErrorCodeCancelledByUser 0
#define AppExtensionErrorCodeAPINotAvailable 1
#define AppExtensionErrorCodeFailedToContactExtension 2
#define AppExtensionErrorCodeFailedToLoadItemProviderData 3
#define AppExtensionErrorCodeCollectFieldsScriptFailed 4
#define AppExtensionErrorCodeFillFieldsScriptFailed 5
#define AppExtensionErrorCodeUnexpectedData 6
#define AppExtensionErrorCodeFailedToObtainURLStringFromWebView 7
// Note to creators of libraries or frameworks:
// If you include this code within your library, then to prevent potential duplicate symbol
// conflicts for adopters of your library, you should rename the OnePasswordExtension class
// and associated typedefs. You might to so by adding your own project prefix, e.g.,
// MyLibraryOnePasswordExtension.
typedef void (^OnePasswordLoginDictionaryCompletionBlock)(NSDictionary * __nullable loginDictionary, NSError * __nullable error);
typedef void (^OnePasswordSuccessCompletionBlock)(BOOL success, NSError * __nullable error);
typedef void (^OnePasswordExtensionItemCompletionBlock)(NSExtensionItem * __nullable extensionItem, NSError * __nullable error);
@interface OnePasswordExtension : NSObject
(OnePasswordExtension )sharedExtension;
/!
@discussion Determines if the 1Password Extension is available. Allows you to only show the 1Password login button to those
that can use it. Of course, you could leave the button enabled and educate users about the virtues of strong, unique
passwords instead :)
@return isAppExtensionAvailable Returns YES if any app that supports the generic org-appextension-feature-password-management feature is installed on the device.
*/
#ifdef __IPHONE_8_0
(BOOL)isAppExtensionAvailable NS_EXTENSION_UNAVAILABLE_IOS("Not available in an extension. Check if org-appextension-feature-password-management:// URL can be opened by the app.");
#else
(BOOL)isAppExtensionAvailable;
#endif
/*!
Called from your login page, this method will find all available logins for the given URLString.
@discussion 1Password will show all matching Login for the naked domain of the given URLString. For example if the user has an item in your 1Password vault with "subdomain1.domain.com” as the website and another one with "subdomain2.domain.com”, and the URLString is "https://domain.com", 1Password will show both items.
However, if no matching login is found for "https://domain.com", the 1Password Extension will display the "Show all Logins" button so that the user can search among all the Logins in the vault. This is especially useful when the user has a login for "https://olddomain.com".
After the user selects a login, it is stored into an NSDictionary and given to your completion handler. Use the Login Dictionary keys above to
extract the needed information and update your UI. The completion block is guaranteed to be called on the main thread.
@param URLString For the matching Logins in the 1Password vault.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block called with two parameters loginDictionary and error once completed. The loginDictionary reply parameter that contains the username, password and the One-Time Password if available. The error Reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/
(void)findLoginForURLString:(nonnull NSString *)URLString forViewController:(nonnull UIViewController )viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;
/!
Create a new login within 1Password and allow the user to generate a new password before saving.
@discussion The provided URLString should be unique to your app or service and be identical to what you pass into the find login method.
The completion block is guaranteed to be called on the main
thread.
@param URLString For the new Login to be saved in 1Password.
@param loginDetailsDictionary about the Login to be saved, including custom fields, are stored in an dictionary and given to the 1Password Extension.
@param passwordGenerationOptions The Password generator options represented in a dictionary form.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block which is called with type parameters loginDictionary and error. The loginDictionary reply parameter which contain all the information about the newly saved Login. Use the Login Dictionary keys above to extract the needed information and update your UI. For example, updating the UI with the newly generated password lets the user know their action was successful. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/
(void)storeLoginForURLString:(nonnull NSString *)URLString loginDetails:(nullable NSDictionary *)loginDetailsDictionary passwordGenerationOptions:(nullable NSDictionary *)passwordGenerationOptions forViewController:(nonnull UIViewController )viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;
/!
Change the password for an existing login within 1Password.
@discussion The provided URLString should be unique to your app or service and be identical to what you pass into the find login method. The completion block is guaranteed to be called on the main thread.
1Password 6 and later:
The 1Password Extension will display all available the matching Logins for the given URL string. The user can choose which Login item to update. The "New Login" button will also be available at all times, in case the user wishes to to create a new Login instead,
1Password 5:
These are the three scenarios that are supported:
A single matching Login is found: 1Password will enter edit mode for that Login and will update its password using the value for AppExtensionPasswordKey.
More than a one matching Logins are found: 1Password will display a list of all matching Logins. The user must choose which one to update. Once in edit mode, the Login will be updated with the new password.
No matching login is found: 1Password will create a new Login using the optional fields if available to populate its properties.
@param URLString for the Login to be updated with a new password in 1Password.
@param loginDetailsDictionary about the Login to be saved, including old password and the username, are stored in an dictionary and given to the 1Password Extension.
@param passwordGenerationOptions The Password generator options epresented in a dictionary form.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param completion A completion block which is called with type parameters loginDictionary and error. The loginDictionary reply parameter which contain all the information about the newly updated Login, including the newly generated and the old password. Use the Login Dictionary keys above to extract the needed information and update your UI. For example, updating the UI with the newly generated password lets the user know their action was successful. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/
(void)changePasswordForLoginForURLString:(nonnull NSString *)URLString loginDetails:(nullable NSDictionary *)loginDetailsDictionary passwordGenerationOptions:(nullable NSDictionary *)passwordGenerationOptions forViewController:(UIViewController )viewController sender:(nullable id)sender completion:(nonnull OnePasswordLoginDictionaryCompletionBlock)completion;
/!
Called from your web view controller, this method will show all the saved logins for the active page in the provided web
view, and automatically fill the HTML form fields. Supports both WKWebView and UIWebView.
@discussion 1Password will show all matching Login for the naked domain of the current website. For example if the user has an item in your 1Password vault with "subdomain1.domain.com” as the website and another one with "subdomain2.domain.com”, and the current website is "https://domain.com", 1Password will show both items.
However, if no matching login is found for "https://domain.com", the 1Password Extension will display the "New Login" button so that the user can create a new Login for the current website.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param viewController The view controller from which the 1Password Extension is invoked. Usually self
@param sender The sender which triggers the share sheet to show. UIButton, UIBarButtonItem or UIView. Can also be nil on iPhone, but not on iPad.
@param yesOrNo Boolean flag. If YES is passed only matching Login items will be shown, otherwise the 1Password Extension will also display Credit Cards and Identities.
@param completion Completion block called on completion with parameters success, and error. The success reply parameter that is YES if the 1Password Extension has been successfully completed or NO otherwise. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/
(void)fillItemIntoWebView:(nonnull id)webView forViewController:(nonnull UIViewController )viewController sender:(nullable id)sender showOnlyLogins:(BOOL)yesOrNo completion:(nonnull OnePasswordSuccessCompletionBlock)completion;
/!
Called in the UIActivityViewController completion block to find out whether or not the user selected the 1Password Extension activity.
@param activityType or the bundle identidier of the selected activity in the share sheet.
@return isOnePasswordExtensionActivityType Returns YES if the selected activity is the 1Password extension, NO otherwise.
*/
(BOOL)isOnePasswordExtensionActivityType:(nullable NSString )activityType;
/!
The returned NSExtensionItem can be used to create your own UIActivityViewController. Use isOnePasswordExtensionActivityType: and fillReturnedItems:intoWebView:completion: in the activity view controller completion block to process the result. The completion block is guaranteed to be called on the main thread.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param completion Completion block called on completion with extensionItem and error. The extensionItem reply parameter that is contains all the info required by the 1Password extension if has been successfully completed or nil otherwise. The error reply parameter that is nil if the 1Password extension item has been successfully created, or it contains error information about the completion failure.
*/
(void)createExtensionItemForWebView:(nonnull id)webView completion:(nonnull OnePasswordExtensionItemCompletionBlock)completion;
/*!
Method used in the UIActivityViewController completion block to fill information into a web view.
@param returnedItems Array which contains the selected activity in the share sheet. Empty array if the share sheet is cancelled by the user.
@param webView The web view which displays the form to be filled. The active UIWebView Or WKWebView. Must not be nil.
@param completion Completion block called on completion with parameters success, and error. The success reply parameter that is YES if the 1Password Extension has been successfully completed or NO otherwise. The error reply parameter that is nil if the 1Password Extension has been successfully completed, or it contains error information about the completion failure.
*/
(void)fillReturnedItems:(nullable NSArray )returnedItems intoWebView:(nonnull id)webView completion:(nonnull OnePasswordSuccessCompletionBlock)completion;
/!
Deprecated in version 1.5
@see Use fillItemIntoWebView:forViewController:sender:showOnlyLogins:completion: instead
*/
(void)fillLoginIntoWebView:(nonnull id)webView forViewController:(nonnull UIViewController *)viewController sender:(nullable id)sender completion:(nonnull OnePasswordSuccessCompletionBlock)completion attribute((deprecated("Use fillItemIntoWebView:forViewController:sender:showOnlyLogins:completion: instead. Deprecated in version 1.5")));
@end
#if __has_feature(nullability)
NS_ASSUME_NONNULL_END

@ironfangyin What exactly am I looking at here besides a copy and paste of the source? I was not able to note any immediate differences. I am closing this until a more accurate understanding of what bug/feature you are reporting or requesting is provided.


InfluxDB 2.0: There is no way hot to configure Telegraf with my own plugin [area/api]

When I trying create a new Telegraf configuration with my own plugin than then the request fails:
-> POST
-> http://<IP_ADDRESS>:9999/api/v2/telegrafs
-> {
	"name": "TelegrafConfig1552908917061-IT",
	"description": "test-config",
	"agent": {
		"collectionInterval": 1000
	},
	"plugins": [{
		"comment": "Output to Influx 2.0",
		"name": "influxdb_v2",
		"type": "output",
		"config": {
			"urls": ["http://<IP_ADDRESS>:9999"],
			"token": "$INFLUX_TOKEN",
			"organization": "my-org",
			"bucket": "my-bucket"
		}
	}, {
		"name": "kernel",
		"type": "input"
	}],
	"organizationID": "03899e997851e000"
}

<- 400
<- Bad Request
<- {
	"code": "invalid",
	"message": "unsupported telegraf plugin ping-plugin, type input",
	"op": "unmarshal telegraf config raw plugin"
}

+1

So i have an idea about this one, currently if a invalid telegraf type posted, we will just show an error.
Alternatively, we can simply unmarshal it to a map[string]interface{}. In this way user may create whatever they want.

Fixed via https://github.com/influxdata/influxdb/pull/16132


vendor: Bump installer to destroy untagged subnets

... and security groups.  Pulling in openshift/installer@55630a304 (openshift/installer#2214).
Generated with:
$ sed -i 's/ae2baf820f22b9bf1ed40932e7b702d790087574/6fdffbb9c21d90ba97fc79dff3ccbeac22fb2a0e/' Gopkg.toml
$ dep ensure

The k8s.io/apimachinery/pkg/util/clock addition in Gopkg.lock is probably catching up with c759bfa2d2 (#518).

Thanks @wking
/lgtm


[WIP] Fix rank reduction for beamformers

As discussed in https://github.com/mne-tools/mne-python/pull/7168, the option reduce_rank=True does not behave as one might expect:
The code is actually not implementing lead field rank reduction but reducing the rank of the denominator of the beamformer formula.
This PR fixes this, and completely swaps the old rank reduction for the lead field one.
Tests are left to fail for first round, so we can see what will change with this in place.

Tests are left to fail for first round, so we can see what will change with this in place.

Oh okay :)

@larsoner on my machine one DICS test fails and actually has a higher localization error after this fix. To me that makes sense as we scratch one orientation and then acutally pick orientation in that test ... But I'd like to see if you agree.
I will get to your comments soon, thank you!

on my machine one DICS test fails and actually has a higher localization error after this fix. To me that makes sense as we scratch one orientation and then acutally pick orientation in that test ... But I'd like to see if you agree.

I see the same failure locally. Let me see if I can make the test a bit more robust/clear by refactoring a bit

@larsoner, I worked on your comments and rebased. Locally, 3 tests fail for me in test_dics, let's see whether Travis agrees and what to do about it!

Should I update the bug description if we now also reduce rank on the denominator?

I guess so -- up to you

Let's do that to have it clean. I just need to commit changes from another branch then I'll push it.

Good to go from my end, @larsoner.

Thanks @britta-wstnr !


Make a deploy from command line

A command example could be:
miactl -p project-id -environment development deploy

Added in #40 and released in v0.3.0
Also, it is added a command to check the status of the deploy.


[DROOLS-898] rename drools-osgi to kie-osgi

All related PRs:
https://github.com/droolsjbpm/droolsjbpm-build-bootstrap/pull/274
https://github.com/droolsjbpm/drools/pull/902
https://github.com/droolsjbpm/droolsjbpm-integration/pull/631

jenkins, execute full downstream build

The full downstream build failed, because Jenkins did not pick up the correct branches (even though they have the same name). I'll need to fix this directly in the Jenkins plugin, but that will likely take some time.

+1


Migrate senzing/senzingapi-runtime-beta to senzing/senzingsdk-runtime-beta

See https://github.com/senzing-garage/template-go/issues/176
Change
validate/Dockerfile:ARG IMAGE_FINAL=senzing/senzingapi-runtime-beta:latest

Pull requests:

https://github.com/senzing-garage/validate/pull/105


Bugfix0.11.1

closes #68

Coverage increased (+0.3%) to 85.602% when pulling 75d1c6706fa139365e57d0b7893d5dbcbcc69a34 on bugfix0.11.1 into ed5461ad425f53ae1f092cecf0f80bef13991106 on master.


Whitelisted site blocked

Hey,
I was just made aware by one of your users that myetheroll.com is blocked and tagged as 'phishing' despite already being on the whitelist (https://www.reddit.com/r/etheroll/comments/7jdwax/etheraddresslookup_extension_blocking/).
Cheers!

Hi @myetheroll - I'll investigate this, thanks for the report.
Which page is it redirecting you to? I am able to browse it without being redirected.
Are you able to attach a screenshot?

Thanks for having a look at this!
I'm not the one experiencing the problems, so not sure.
Might be a case of outdated code.

Merging issue into https://github.com/409H/EtherAddressLookup/issues/262


Docs 0.2

Description

Related Issues

Checklist

[ ] Is this code covered by new or existing unit tests or integration tests?
[ ] Did you run make unit-testing and make integration-testing successfully?
[ ] Do new classes, functions, methods and parameters all have docstrings?
[ ] Were existing docstrings updated, if necessary?
[ ] Was external documentation updated, if necessary?

Additional Notes or Comments

This PR is not to be merged temporarily, only for confirming if there are conflicts. If there are conflicts, I will rebase the main branch promptly.


Update docs with how to build standalone jar

I couldn't work out how to build since the shadowJar task advised in the README was missing. Actually, I discovered how in the './go' task, then later found this thread: https://groups.google.com/forum/#!topic/wiremock-user/Mt6R04oU0Nw
I don't know if there's other places in the docs to update, but this would help newcomers...

Thanks!


Can 2Captcha solve android app's ReCaptcha?

It is quite similar to normal ReCaptchas.
For example, this info is sent to https://www.recaptcha.net/recaptcha/api3/mri (originally in protonbuf)
{
  "string_1": "6LeMK_sfAAAAAJLd4foZmo7m3u8pV_J_xxxxx",
  "string_2": "com.example.myApp",
  "string_3": "18.6.1",
  "int_4": 1,
  "string_6": "a28e8f6c-8b3c-4c1b-b15c-4e0ea77189f0",
  "string_7": "30"
}

Where "30" is android version (sdk 30 is A11), com.example.myApp is app's unique package name and 18.6.1 is recaptcha android library version. Device infos are further sent on /mlg requests

For requests not related to the library, please use contact form on the website: https://2captcha.com/support/tickets/new


Autodetect the locale based on the preferred browser language

The browser communicates such intent using Accept-Language header that lists a number of locales in the order of priority.
For example, if HTTP request contains Accept-Language: uk,en, this means that the browser is asking the server to provide the content in Ukrainian, if possible, and fall back to English if Ukrainian is not available.
Currently, PyPI ignores this header completely.

curl log
$ curl -svH 'Accept-Language: uk,en' https://pypi.org | head -c 500
*   Trying 2a04:4e42:200::223:443...
* TCP_NODELAY set
* connect to 2a04:4e42:200::223 port 443 failed: No route to host
*   Trying 2a04:4e42::223:443...
* TCP_NODELAY set
* connect to 2a04:4e42::223 port 443 failed: No route to host
*   Trying 2a04:4e42:400::223:443...
* TCP_NODELAY set
* connect to 2a04:4e42:400::223 port 443 failed: No route to host
*   Trying 2a04:4e42:600::223:443...
* TCP_NODELAY set
* connect to 2a04:4e42:600::223 port 443 failed: No route to host
*   Trying <IP_ADDRESS>:443...
* TCP_NODELAY set
* Connected to pypi.org (<IP_ADDRESS>) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: none
  CApath: none
* loaded libnssckbi.so
* ALPN, server accepted to use http/1.1
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: CN=www.python.org,O=Python Software Foundation,L=Wolfeboro,ST=New Hampshire,C=US,serialNumber=3359300,incorporationState=Delaware,incorporationCountry=US,businessCategory=Private Organization
*       start date: Sep 18 00:00:00 2018 GMT
*       expire date: Oct 14 12:00:00 2020 GMT
*       common name: www.python.org
*       issuer: CN=DigiCert SHA2 Extended Validation Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US
> GET / HTTP/1.1
> Host: pypi.org
> User-Agent: curl/7.65.1
> Accept: */*
> Accept-Language: uk,en
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Security-Policy: base-uri 'self'; block-all-mixed-content; connect-src 'self' https://api.github.com/repos/ *.fastly-insights.com sentry.io https://2p66nmmycsj3.statuspage.io; default-src 'none'; font-src 'self' fonts.gstatic.com; form-action 'self'; frame-ancestors 'none'; frame-src 'none'; img-src 'self' https://warehouse-camo.cmh1.psfhosted.org/ www.google-analytics.com *.fastly-insights.com; script-src 'self' www.googletagmanager.com www.google-analytics.com *.fastly-insights.com https://cdn.ravenjs.com; style-src 'self' fonts.googleapis.com; worker-src *.fastly-insights.com
< Content-Type: text/html; charset=UTF-8
< ETag: "/r3JSqvtLeoGObDXqvsphw"
< Referrer-Policy: origin-when-cross-origin
< Server: nginx/1.13.9
< Content-Length: 21373
< Accept-Ranges: bytes
< Date: Thu, 17 Oct 2019 15:38:48 GMT
< Connection: keep-alive
< X-Served-By: cache-iad2148-IAD, cache-hhn4058-HHN
< X-Cache: MISS, HIT
< X-Cache-Hits: 0, 1
< X-Timer: S1571326729.506037,VS0,VE1
< Vary: Accept-Encoding, PyPI-Locale, Accept-Encoding
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Frame-Options: deny
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Permitted-Cross-Domain-Policies: none
<
{ [1518 bytes data]

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <meta name="defaultLanguage" content="en">
    <meta name="availableLanguages" content="en">

    <title>PyPI · The Python Package Index</title>
    <meta name="description" content="The Python Package Index (PyPI) is a repository of software for the Python programming language.">
* Failed writing body (1241 != 2759)
* Closing connection 0

Additionally, HTTP server should reply with the correct Content-Language header as per https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Language / https://tools.ietf.org/html/rfc7231#section-<IP_ADDRESS>.
What to do
PyPI should learn to detect this header and set a language accordingly.
If the _LOCALE_ cookie is present, Content-Language should probably be ignored as it indicates that the user selected the language explicitly.
I'm not sure whether PyPI should save the locale from Content-Language into a cookie if it's not present. I think, no.
There may also be a way for the registered user to save their preferred language in their profile. This should also influence the locale selection logic but is out of the scope of this issue.

If we work to support Accept-Language, the VCL will need to be updated to undo https://github.com/python/pypi-infra/pull/52 and replace it with something using Fastly's support for this header: https://docs.fastly.com/en/guides/accept-language-header-vcl-features

You could use a cookie too in order to store the user language preference. (For example when the browser is in english but the user choose French as default language for PyPI).

Well, that's what I described. But the initial language should be taken from the browser preferences.

I've created a pull request on python/pypi-infra with a change for the Fastly config.
Looks like warehouse already supports Accept-Language if the header is passed through. Hope caching by language works fine, too.

Seems ok to me!

https://github.com/pypi/infra/pull/109/ is live on https://test.pypi.org for validation and all looks OK from my end. We'll roll this out to https://pypi.org/ later today.

This is live on https://pypi.org/


[Umbrella] Kyuubi Spark TPC-DS Connector

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the proposal
Spark DataSource V2 API[1] is available since Spark 3.0, basically, it provides a bunch of APIs for developers to implement a connector, and Spark will expose them to SQL/DataFrame API automatically with few configurations.
TPC-DS[2] dataset is very useful for benchmarking and demonstration. Previously, we need to generate the dataset by using dsdgen or kyuubi-tpcds tool before running queries. With the connector proposed by this PR, users just need

Add jar kyuubi-spark-connector-tpcds_2.12-${kyuubi_version}.jar
Add conf spark.sql.catalog.tpcds=org.apache.kyuubi.spark.connector.tpcds.TPCDSCatalog

Then they can query the different scales of TPC-DS tables under tpcds.sf{scale} database. For instance,
0: jdbc:hive2://<IP_ADDRESS>:10009/> show tables in tpcds.sf1;
+------------+-------------------------+--------------+
| namespace  |        tableName        | isTemporary  |
+------------+-------------------------+--------------+
| sf1        | call_center             | false        |
| sf1        | catalog_page            | false        |
| sf1        | catalog_returns         | false        |
| sf1        | catalog_sales           | false        |
| sf1        | customer                | false        |
| sf1        | customer_address        | false        |
| sf1        | customer_demographics   | false        |
| sf1        | date_dim                | false        |
| sf1        | household_demographics  | false        |
| sf1        | income_band             | false        |
| sf1        | inventory               | false        |
| sf1        | item                    | false        |
| sf1        | promotion               | false        |
| sf1        | reason                  | false        |
| sf1        | ship_mode               | false        |
| sf1        | store                   | false        |
| sf1        | store_returns           | false        |
| sf1        | store_sales             | false        |
| sf1        | time_dim                | false        |
| sf1        | warehouse               | false        |
| sf1        | web_page                | false        |
| sf1        | web_returns             | false        |
| sf1        | web_sales               | false        |
| sf1        | web_site                | false        |
+------------+-------------------------+--------------+

[1] https://github.com/apache/spark/tree/v3.2.1/sql/catalyst/src/main/java/org/apache/spark/sql/connector
[2] https://tpc.org/TPC_Documents_Current_Versions/pdf/TPC-DS_v3.2.0.pdf
Task list

https://github.com/apache/incubator-kyuubi/pull/2531
https://github.com/apache/incubator-kyuubi/issues/2539
https://github.com/apache/incubator-kyuubi/issues/2540
https://github.com/apache/incubator-kyuubi/issues/2541
https://github.com/apache/incubator-kyuubi/issues/2542
https://github.com/apache/incubator-kyuubi/issues/2543

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

I'm interested in this. Could you assign me some tasks?

Thanks @wForget, since the https://github.com/apache/incubator-kyuubi/pull/2531 have not got merged, you can start from the review first.

@pan3793 I’m interested in working on the subtasks in this umbrella as well.  I can start with #2553.  Could you assign the subtask to me if no one is working on it?

Thx, assigned.


Improve test coverage

Description
I use the new Coveralls code coverage report to find a few places we could increase coverage. Now that we've had Coveralls run once we ought to see coverage go up in this PR so this is also serves as a test of our new reporting system.
When writing this I discovered that we didn't define sf::Shader move operations for OpenGL ES so I defaulted those implementations so they now exist.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 50.955%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
10242

Relevant Lines:
19103

💛  - Coveralls


ASP.NET Core 2.2 issue with setting up http/2 endpoint on Kestrel via code

Describe the bug
I want to set up http/2 endpoint on Kestrel. It only works if I do configuration in appSettings.json. Configuring the same settings from the code leads to serving requests via http1.1 only.
To Reproduce
Steps to reproduce the behavior:

Using this version of ASP.NET Core 2.2.300
On Windows 10 Pro, x64, Version 1903 OS build 18362.239
Create WebHosting cert in IIS10 and export it into a .pfx file
Create asp.net core web-api app in VS2019 v16.1.5
Run this code

   WebHost.CreateDefaultBuilder(args)
                 .UseKestrel(opt =>
                 {
                     opt.ListenAnyIP(44336, listenOptions =>
                     {
<EMAIL_ADDRESS>"123456");
                         listenOptions.Protocols = HttpProtocols.Http1AndHttp2;
                     });
                 })
   .UseStartup<Startup>();

Launch Chrome Version 75.0.3770.100 (Official Build) (64-bit)
Make a request at https://localhost:44336/api/values

Expected behavior
Chrome reports h2 in Protocol tab in DevTools
Actual behavior
Chrome reports http/1.1 in Protocol tab in DevTools
===========================================
And here is an odd thing. It works just fine via appSettings.json -
Configuration via appSettings.json
  "Kestrel": {
    "EndPoints": {
      "HttpsInlineCertFile": {
        "Url": "https://localhost:44336",
        "Protocols": "Http1AndHttp2",
        "Certificate": {
          "Path": "./testcert.pfx",
          "Password": "123456"
        }
      }
    }
  }

  WebHost.CreateDefaultBuilder(args)
                 .UseKestrel()
  .UseStartup<Startup>();

Expected behavior
Chrome reports h2 in Protocol tab in DevTools
Actual behavior
Chrome reports h2 in Protocol tab in DevTools
Additional context
Chrome reports the same info on Security tab for the both cases -
"The connection to this site is encrypted and authenticated using TLS 1.2, ECDHE_RSA with P-384, and AES_256_GCM."

Duplicate of https://github.com/aspnet/AspNetCore/issues/11833, try reversing the order:
                         listenOptions.Protocols = HttpProtocols.Http1AndHttp2;
<EMAIL_ADDRESS>"123456");

Closing as this is a duplicate.


NameError: name 'reportlab' is not defined

Details

Read the Docs project URL: https://glpi-install.readthedocs.io/
Build URL (if applicable): https://readthedocs.org/projects/glpi-install/builds/9066046/
Read the Docs username (if applicable): trasher

Expected Result
Build success.
Actual Result
Build failing with error NameError: name 'reportlab' is not defined

It seems that you are calling rst2pdf from your conf.py file. That's not needed when building on Read the Docs. RTD uses a different PDF builder.

I have used it in the past locally, but this is no longer required now. Did not pay attention since I think it was not used (and previousely do not breaks builds).
Thank you :)


GEO: Support more configuration for geo_client

What problem does this PR solve?
Support more configuration for geo_client.
What is changed and how it works?
set min_level, latitude_index and longitude_index in geo_client's config.ini before inserting data into DB, and they are immutable after data has been inserted.
[geo_client.lib]
...
min_level = 12
max_level = 16
latitude_index = 5
longitude_index = 4

min_level and max_level are parameters for data search algorithm in S2.
https://github.com/XiaoMi/pegasus/blob/d227b4c0b6f555572025f433de85ff6eca7abb86/src/geo/lib/latlng_extractor.cpp#L42
latitude_index and longitude_index are parameters for extracting latitude and longitude from string type value. https://github.com/XiaoMi/pegasus/blob/d227b4c0b6f555572025f433de85ff6eca7abb86/src/geo/lib/geo_client.cpp#L463 https://github.com/XiaoMi/pegasus/blob/d227b4c0b6f555572025f433de85ff6eca7abb86/src/geo/lib/geo_client.cpp#L510
Check List
Tests

Unit test
Manual test (add detailed scripts or steps below)

Code changes
Side effects
Related changes

Need to cherry-pick to the release branch
Need to be included in the release note

`min_level and max_level are parameters for data search algorithm in S2.```, but I didn't find the usage place in latlng_extractor::extract_from_value() in your commit description.

min_level and max_level are parameters for data search algorithm in S2., but I didn't find the usage place in latlng_extractor::extract_from_value() in your commit description.

fixed


Instructions fail to work

Tried configuring this for a while, without success. I have installed all the npm packages, both locally and globally (just to see if it made a difference), set the port to 9515 and set services to `['chromedriver'], but nothing ever happens:
$(npm bin)/wdio --logLevel verbose

[18:21:02]  COMMAND	POST 	 "/wd/hub/session"
[18:21:02]  DATA		{"desiredCapabilities":{"javascriptEnabled":true,"locationContextEnabled":true,"handlesAlerts":true,"rotatable":true,"maxInstances":5,"browserName":"chrome","chromeOptions":{"args":["//headless","disable-gpu"],"prefs":{"credentials_enable_service":false}},"loggingPrefs":{"browser":"ALL","driver":"ALL"},"requestOrigins":{"url":"http://webdriver.io","version":"4.6.2","name":"webdriverio"}}}
ERROR: unknown command: wd/hub/session
chrome
Error

Full config
exports.config = {

    specs: [
        'tests/acceptance/**/*-test.js'
    ],
    exclude: [
    ],
    maxInstances: 10,
    capabilities: [{
        maxInstances: 5,
        browserName: 'chrome',
        'chromeOptions': {
            'args': [
                'disable-gpu'
            ],
            prefs: {'credentials_enable_service': false }
        }
    }],
    port: '9515',
    sync: true,
    logLevel: 'silent',
    coloredLogs: true,
    bail: 0,
    screenshotPath: './errorShots/',
    waitforTimeout: 30000,
    connectionRetryTimeout: 90000,
    connectionRetryCount: 3,
     services: ['chromedriver'],

    framework: 'mocha',
    mochaOpts: {
        timeout: 30000
    },
    mochaOpts: {
        ui: 'bdd'
    },
}

Did you set path to '/', according to the instructions?

Don't mind me. I'm just walking away in shame ...
(I thought the path variable was just for specifying the root of the application ... so ignored that)


fixed #2429 Previewing a world breaks world preview for remainder of game session

Contains
Fixes #2429 and partially fixes #2509
How to test
For issue 2429

Click on Single Player > Create > Details
Wait for screen to show up
Then click on Back
Click on Details again
It shows null pointer exception and preview does not load
Can be tested by selecting any world generator or module(even stable ones will show this exception)

For issue 2509
The 1st problem is not yet tackled but the reason for lag is the setting the environment which causes the assets to reload and when back button is pressed the environment is reset which also causes assets to reload and hence I think the lag
The 2nd problem in this issue is actually the above issue which is fixed
The 3rd problem is tackled in the commit for issue 2509 in which the game crashes or will be needed process killed.

Click on Single Player > Create > Details
Wait for screen to show up
Then click on Back
Click on Back again
Click on Create
The game crashes showing a null pointer exception
Note: The 2nd and 3rd problem imployed the same solution as the problem was same that the variables(with @It above them) that referenced to Core Registry for values were set to null.

I don't think the fourth problem is actually because of preview Class. The null pointer exception in this case is due to world i.e. what we get from worldGenerator.getWorld() being null and further use of world variable causes null pointer exception. I think the problem in this case is with the world creation procedure in this specific module because the world is not even fully loaded when the player is spawned in the game.
PS: solved the tta preview problem. Now it opens preview but does not show anything as its preview functionality is not defined. I think that should be made into another issue and this issue should be closed.
pr 3 in ClimateConditions and pr from new branch 'shine1' in WoodAndStone are made for this issue

Can one of the admins please verify this patch?

Thanks for this @0shine0 !
@flo @msteiger either of you interested in checking this one out? If not I'll try to catch it over the weekend
@GooeyHub: add to whitelist

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

@Cervator Please test and review the changes. Solved the issue.

Hooray Jenkins reported success with all tests good!

Tested out and merged!
I saw the two module PRs and left some comments there, still need to merge those.
I'm adding some more notes on what's left in the GSOC scope #2774 - that way we can let the two associated issues to this count as being fixed :-)


[GHSA-mf45-g8hg-p9g4] Cross-site scripting (XSS) vulnerability in the...

Updates

Affected products
References
Source code location
Summary

Comments
Add source link and some patch links related to CVE-2015-3268.

Hey @sunSUNQ, do you happen to have the maven artifact id for this one?

Hello, I can only confirm that this vulnerability affects https://github.com/apache/ofbiz and I couldn't find any specific impacted components in Maven.

Hi @sunSUNQ, we appreciate you informing us of the repository this vulnerability affects. Unfortunately because the affect product is not apart of any of the GitHub Advisory Database supported ecosystems we would not be able to accept your contribution.


Feature request: add support for partial open

The Tailwind app allows me to configure an amount of time for a partial door open. I can then long press the garage door to partially open it by this amount. See here for example: https://youtu.be/sOODpMTewW0?t=87
It would be nice if there was a way for me to trigger a partial open via this home assistant integration.
Use case I have in mind is that I'd like to set up an automation to partially open my garage doors when the weather outside is nice so I can automatically air out my garage.

The local control API doesn't provide a way to use the partial open time configured via the Tailwind app, but does allow a "partial open time" to be specified (in milliseconds) when opening the door.  There doesn't seem to be a core HA "cover" service call which matches the partial open behaviour (the closest would probably be set_cover_position, but trying to use that would most likely cause confusion as it's not a great fit).  Do you think that a custom service call would be a reasonable way to implement this, and if so, would it make sense to include the partial open time as a parameter (so that you could have different automations which open the door to different degrees, for instance), or would you expect the time to be part of the integration's configuration/options?


guvcview: fix gsettings filechooser errors

Motivation for this change
Fix this error:
(guvcview:8975): GLib-GIO-ERROR **: 10:38:24.106: Settings schema 'org.gtk.Settings.FileChooser' is not installed

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg eval

I ran nix-review, and checked that the file dialog in guvcview works. If you feel brave, qstopmotion (a reverse dependency) is in need of wrapQTappsHook.

Nice thinking. Is there an open issue you've encountered regarding this?
The next set of commits introduces a little bit broader changes which I think would be appropriate under the approval of @leenaars - maintainer of qstopmotion, and @coreyoconnor - maintainer of guvcview. Let's hope they'll respond.

@worldofpeace I used qt5.mkDerivation I hope that's what you meant.

I built guvcview and qstopmotion with nix-review and they both seem to work (as in, a window shows up).

/ping @worldofpeace .

/ping worldofpeace 🙏

/ping @worldofpeace pray

Very close :smile: I can try to merge today.

@worldofpeace most of ofborg's tests have passed besides the build of qstopmotion for Linux aarch64 for which I marked it as broken.

:clap: @worldofpeace :pray:.

@doronbehar Was this an issue on 20.03?

Thanks :]

@doronbehar Was this an issue on 20.03?

I'm using unstable so I don't know.


Use a phantom type to restrict LengthLiteral to correct length type

I tried to set an img width to Percent 100.0, but in the DOM you'll only get back a img with width 0
https://developer.mozilla.org/en-US/docs/Web/HTML/Element/img#attr-width

The intrinsic width of the image in pixels. In HTML 4, either a percentage or pixels were acceptable values. In HTML5, however, only pixels are acceptable.

I was about to send a PR to remove LenghtLiteral altogether, but then I realized that object-like elements, like object, embed and iframe support a percent-size
This is the first time I actually used a phantom type, and I'm not sure if the names I've picked are actually good.

This approach seems like a good idea, but how about we squash the construction functions back into two options:
percent :: Number -> LengthLiteral Object
pixels :: forall a. Int -> LengthLiteral a

I was musing some naming like Absolute / Relative rather than Object / Element too, but it's a bit weird, it would probably have to be OnlyAbsolute / AllowRelative or something to make sense, which doesn't seem much better really.

Agree, I've done it.
re. naming:
I hoped it'd be possible to do something like
canvas :: forall p i. Leaf (width :: Size Absolute, height :: Size Absolute) p i
embed :: forall p i. Node (height :: Size (Absolute | Relative), src :: I, mediaType :: I, width :: Size (Absolute | Relative)) p i

But I'm afraid that to do something like this we'd need another ADT (called something like AllowRelative, which is not buying us any clarity)... maybe with rows inside rows? Or maybe Type families? (which I know haven't been implemented yet, but I was wondering what's the best we could theoretically do, given a sufficiently advanced compiler)

You could probably do something with DataKinds that would better express it, as then you'd restrict the phantom type to constructors of an AllowRelative type (DataKinds allows you to promote data constructors into the type system). Something like:
data LengthLiteral (a :: LengthRestriction) = ...
data LengthRestriction = AllowRelative | OnlyAbsolute

canvas :: forall p i.  Node (height :: Size s, src :: I, mediaType :: I, width :: Size s) p i

In embed here s's kind would also be LengthRestriction, and the only types of that kind are 'AllowRelative and 'OnlyAbsolute, so you wouldn't need to express the union explicitly as suggested above. (The prime prefix is a Haskell thing, we wouldn't necessarily require that, but I've not looked into the arguments and reasoning about why it's so to be able to say for sure).

This looks good to me :+1:. I'll keep the PR hanging around until #315 is resolved though, and merge it once that is in and we're ready to release, as I might forget to credit you in the release notes otherwise.

Thanks! Sorry about the ridiculous delay in getting it merged.

Don't worry... I've done worse :) (issues unanswered for 7 months :P )


trouble performing IETF QUIC handshake

Hello again,
I am receiving a timeout error when I attempt to perform a IETF QUIC handshake. I make a simple client and server using your echo example.
client:
package main

import (
	"crypto/rand"
	"crypto/rsa"
	"crypto/tls"
	"crypto/x509"
	"encoding/pem"
	"fmt"
	"io"
	"log"
	"math/big"
        "sync"

	quic "github.com/lucas-clemente/quic-go"
	"github.com/lucas-clemente/quic-go/internal/protocol"
)

const addr = "localhost:4242"

const message = "foobar"

// We start a server echoing data on the first stream the client opens,
// then connect with a client, send the message, and wait for its receipt.
func main() {
	var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        log.Fatal(clientMain())
        wg.Done()
    }()
    wg.Wait()
}

func clientMain() error {
	versions := protocol.SupportedVersions
    versions = append([]protocol.VersionNumber{protocol.VersionTLS}, versions...)
    session, err := quic.DialAddr(addr, &tls.Config{InsecureSkipVerify: true}, &quic.Config{Versions: versions})
	if err != nil {
		return err
	}

	stream, err := session.OpenStreamSync()
	if err != nil {
		return err
	}

	fmt.Printf("Client: Sending '%s'\n", message)
	_, err = stream.Write([]byte(message))
	if err != nil {
		return err
	}

	buf := make([]byte, len(message))
	_, err = io.ReadFull(stream, buf)
	if err != nil {
		return err
	}
	fmt.Printf("Client: Got '%s'\n", buf)

	return nil
}

// A wrapper for io.Writer that also logs the message.
type loggingWriter struct{ io.Writer }

func (w loggingWriter) Write(b []byte) (int, error) {
	fmt.Printf("Server: Got '%s'\n", string(b))
	return w.Writer.Write(b)
}

server:
package main

import (
	"crypto/rand"
	"crypto/rsa"
	"crypto/tls"
	"crypto/x509"
	"encoding/pem"
	"fmt"
	"io"
	"log"
	"math/big"
        "sync"

	quic "github.com/lucas-clemente/quic-go"
	"github.com/lucas-clemente/quic-go/internal/protocol"
)

const addr = "localhost:4242"

const message = "foobar"

// We start a server echoing data on the first stream the client opens,
// then connect with a client, send the message, and wait for its receipt.
func main() {
	var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        log.Fatal(echoServer())
        wg.Done()
    }()
    wg.Wait()
}

// Start a server that echos all data on the first stream opened by the client
func echoServer() error {
	versions := protocol.SupportedVersions
    versions = append([]protocol.VersionNumber{protocol.VersionTLS}, versions...)
    listener, err := quic.ListenAddr(addr, generateTLSConfig(), &quic.Config{Versions: versions})
	if err != nil {
		return err
	}
	sess, err := listener.Accept()
	if err != nil {
		return err
	}
	stream, err := sess.AcceptStream()
	if err != nil {
		panic(err)
	}
	// Echo through the loggingWriter
	_, err = io.Copy(loggingWriter{stream}, stream)
	return err
}

// A wrapper for io.Writer that also logs the message.
type loggingWriter struct{ io.Writer }

func (w loggingWriter) Write(b []byte) (int, error) {
	fmt.Printf("Server: Got '%s'\n", string(b))
	return w.Writer.Write(b)
}

// Setup a bare-bones TLS config for the server
func generateTLSConfig() *tls.Config {
	key, err := rsa.GenerateKey(rand.Reader, 1024)
	if err != nil {
		panic(err)
	}
	template := x509.Certificate{SerialNumber: big.NewInt(1)}
	certDER, err := x509.CreateCertificate(rand.Reader, &template, &template, &key.PublicKey, key)
	if err != nil {
		panic(err)
	}
	keyPEM := pem.EncodeToMemory(&pem.Block{Type: "RSA PRIVATE KEY", Bytes: x509.MarshalPKCS1PrivateKey(key)})
	certPEM := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: certDER})

	tlsCert, err := tls.X509KeyPair(certPEM, keyPEM)
	if err != nil {
		panic(err)
	}
	return &tls.Config{Certificates: []tls.Certificate{tlsCert}}
}

I receive the following error:
HandshakeTimeout: Crypto handshake did not complete in time.
I suspect the issue is that I am not passing in the appropriate tls config to the client. The only change I made was passing protocol.versionTLS, so I am not sure why else the handshake would fail.

Please note that IETF QUIC support is still experimental. You might want to have a look at the integration tests and enable logging to figure out what’s going on here.

I meet the same problem. In my case, I didn't add protocol.versionTLS and still handshake failed.


NotImplementedError: The create_virtual_mfa_device action has not been implemented

I'm trying to test some IAM methods with users that have MFAs, but I'm getting this error when calling create_virtual_mfa_device:
self = <moto.iam.responses.IamResponse object at 0x10e7e4f28>

    def call_action(self):
        headers = self.response_headers
        action = camelcase_to_underscores(self._get_action())
        method_names = method_names_from_class(self.__class__)
        if action in method_names:
            method = getattr(self, action)
            try:
                response = method()
            except HTTPException as http_error:
                response = http_error.description, dict(status=http_error.code)

            if isinstance(response, six.string_types):
                return 200, headers, response
            else:
                if len(response) == 2:
                    body, new_headers = response
                else:
                    status, new_headers, body = response
                status = new_headers.get('status', 200)
                headers.update(new_headers)
                # Cast status to string
                if "status" in headers:
                    headers['status'] = str(headers['status'])
                return status, headers, body

        if not action:
            return 404, headers, ''

        raise NotImplementedError(
>           "The {0} action has not been implemented".format(action))
E       NotImplementedError: The create_virtual_mfa_device action has not been implemented

Unfortunatly this is to be expected as the IMPLEMENTATION_COVERAGE.md clearly states, that this method has not been implemented yet 😢
https://github.com/spulec/moto/blob/0f816566df3aefc20b9054e4dc9f7ca7373bc6f6/IMPLEMENTATION_COVERAGE.md#L2255
If you need this feature quickly, feel free to have a look at the Contribution Guidelines on how you can get involved in moto yourself.

Thanks for opening. Setting this as a feature request.


Feature: PackageRepository can automatically auth to private registries when secretgen-controller is in the cluster

Describe the problem/challenge you have
Right now, the process of creating a package repository that can auth to a private registry is an entirely manual one. The user needs to create a secret with the authorization details, and then correctly reference it within their repository yaml. The manual nature of this makes it prone to human error.
Describe the solution you'd like
kapp-controller is able to automatically handle the auth details of a package repository.
Anything else you would like to add:
To do this, we plan on adding a feature to secretgen-controller (SGC) called placeholder secrets. Essentially, there will be a namespace with some exported secret, and when SGC detects that a placeholder secret matching some criteria has been created in a another namespace, it will fill in the details of the placeholder with the same content as the exported secret. With this feature in place, kapp-controller will just need to create the appropriate placeholder secret and add it to the auth details of a repo during reconciliation.

Vote on this request
This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.
👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"
We are also happy to receive and review Pull Requests if you want to help working on this issue.

We are estimating this as a larger story based on the following considerations:

We need as a starting point an integration test between kapp-controller and secretgen-controller for the happy path scenario. Seeing as this will be the first test written for this, it will take some time to fully figure out this pattern.
Additionally, there is likely to be more than one test needed to fully verify this feature:

multiple registries need auth in this case?
failure tests on both the secretgen-controller/kapp-controller sides to verify unhappy paths
unit tests where applicable for more quick feedback

Developing placeholder secret functionality for secretgen-controller (This should potentially be split out into a separate story?)
Developing the integration between kapp-controller and secretgen-controller


Make Terms of Service & Privacy Policy clickable

I can do this for you, its quite an easy task. Plus this would be my first issue.


1.21 support!

Let's make some fancy pictures or renders for your mod, but in exchange you'll make a 1.21 support 😅

I'm currently working on updating my other mod "You're in Grave Danger", but when I have some time at hand to work on this too, I will make sure it receives updates too

The other mod is done, and now I've published version 1.3.0 for Short Grass!
It will work on all versions between 1.20.4 and 1.21


ST_AsBinary return result of 44 length hex

By following the example, it returned "\x010100000000000000000024c00000000000003e40" to go.geo causing "go.geo: incorrect geometry" error
Any idea what's wrong? thanks!

I have 'SELECT ST_AsBinary(coordinate) FROM table'
Does it mean i have to convert it again before passing to it?

if you are Scanning it into a []byte and the resulting slice is 44 bytes then it's something to do with sql driver settings.
if it's 21 bytes then there is something else wrong

I guess you are right about 42hex chars to 21 bytes. What could i do to receive 21 bytes so it could scan into Point? I am using it with (go-pg) [https://godoc.org/github.com/go-pg/pg#DB.Query] if it matters

does it work if you use https://github.com/lib/pq directly?

Yeah i think it did work when i was briefly  testing with https://github.com/lib/pq

length of 44 is after data, ok := value.([]byte) conversion
at here

tried doing hex.DecodeString(string(data)) after data, ok := value.([]byte) and it worked, but couldnt find the equivalent in postgres that does that

looks like this was fixed in may by https://github.com/paulmach/go.geo/pull/67


React-Bootstrap CDN not working

Challenge Name

https://www.freecodecamp.com/challenges/build-a-recipe-box
Issue Description

While working on the recipe box challenge I decided to use react-bootstrap, but I kept getting errors in the console. So, I decided to try to build things locally and figure out the problem. I found that react-bootstrap didn't load in the first place. So, I think it would be nice to notify campers that the CDN doesn't work, so they don't waste their time.
Screenshot

@Harry97, your issue is nothing with the fcc site.  if you need a help. you could go to chat room. okay

I understand, but it does affect FCC one way or another. And, I've found out that the problem is solved by using non-minified version of both React and React-dom libraries, so I terminated the issue. You do have a point and I'mma keep it in my mind next time, thanks!


Unsupported HTTP response - 301

During launch of the collector I got following messages in log:
Unsupported HTTP response HTTP/1.1 301 Moved Permanently
REJECTED_REDIRECTED: http://www.site.com (Subject: HttpFetchResponse [crawlState=REDIRECT, statusCode=301, reasonPhrase=Moved Permanently (https://www.site.com)])

startUrls comes to config without any attribute, which means, according the documentation, stayOnDomain="false", stayOnPort="false", stayOnProtocol="false". So it should serve such redirection correctly, I guess.
Using 2.3.0 release.
Reference filter isn't set to pass http://: ^.*www\.site\.com/somefilterlogic$
Start url is http:// but it shouldn't cause problems.

Yes, but what tells you the redirect target is not processed?  You have nothing crawled after that message?  REJECTED_REDIRECTED means the URL http://www.site.com is being rejected because it redirects somewhere, but the redirect is followed and the target URL should be processed normally.  In this case, the target is https://www.site.com (https).  Does the log mention anything about https://www.site.com?

It also writes:
Cannot fetch document: .... url (handshake alert: unrecognized_name )
   REJECTED_ERROR: ... url

Handshake-related errors usually have to do with failing to resolve the SSL certificate.  Either it expired, you need to tell java you trust it, or whatever other possible SSL error.  One easy way to get around this usually is to accept all certificates (usually a bad idea but for crawling you often can live with that).  Try:
<httpClientFactory>
    ...
    <trustAllSSLCertificates>true</trustAllSSLCertificates>
</httpClientFactory>

I added following lines to the crawlerDefaults, but it continue delivering unrecognized name.
I'm sure there is no override in crawler configuration.

Do you need stack trace from the log? I post it here.

Your full config would help the most.

May I send it to your email?

Sure thing.  You can find it in my profile.

I was able to replicate. It turns this problem started appearing only after a certain version of Java.
Java introduced Server Name Indication (SNI) extension on Java and it is enabled by default.  There are servers not configured properly that will send a "Unrecognized Name" warning in the SSL handshake.  Most clients ignore this message (as they should), but Java fails on it.  You can disable SNI by adding the following JVM system property when launching the collector:
java -Djsse.enableSNIExtension=false

This won't resolve everything though.  Because the site you crawl also seems to be using an older, insecure encryption protocol (MD2).  Java no longer supports that protocol when it ships.  You can force it to support it by editing this file:
JDK_HOME/jre/lib/security/java.security

In it, change this line:
jdk.certpath.disabledAlgorithms=MD2

to this one:
jdk.certpath.disabledAlgorithms=

I will try to find a permanent fix in HTTP Collector itself, but in the meantime, the above instructions should allow you to keep crawling.

I released a new snapshot release with a fix.  Setting trustAllSSLCertificates to true will now also disable SNI Extension and enable unsafe algorithms.  You no longer have to change your java.security file.  Please confirm.

Thanks a lot, I'll provide feedback nearest time.

ERROR - www.site.com: Could not process document: https://www.site.com (javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints)
com.norconex.collector.core.CollectorException: javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Certificates does not conform to algorithm constraints
	at com.norconex.collector.http.fetch.impl.GenericDocumentFetcher.fetchDocument(GenericDocumentFetcher.java:173)
	at com.norconex.collector.http.pipeline.importer.DocumentFetcherStage.executeStage(DocumentFetcherStage.java:48)
	at com.norconex.collector.http.pipeline.importer.AbstractImporterStage.execute(AbstractImporterStage.java:31)
	at com.norconex.collector.http.pipeline.importer.AbstractImporterStage.execute(AbstractImporterStage.java:24)
	at com.norconex.commons.lang.pipeline.Pipeline.execute(Pipeline.java:90)
...

Used 2.4.0 snapshot. Any additional info I may provide?

Not sure why it is not working for you.  Which JRE version are you using?  Can you attach your config?

java version "1.8.0_05"
I've sent the config once, by email.

Luckily I still had it.  I just tested with 1.8 and I was able to reproduce the exception.  It appears the solution put in place works fine under 1.7.x.    I will investigate when I have a chance, but in the meantime, you may want to use the JRE-tweaking measures suggested before to enable that site with an insecure protocol (or less fun: downgrade your JRE).

Thank you for the detailed response. I'll try to downgrade JRE and provide feedback.

I finally managed to have it working on both Java 7 and Java 8 without having to change the JRE java.security file or adding a JVM argument.  I made a new snapshot release with this fix.  Please test it to confirm when you have a chance.

Thanks a lot! I confirm it delivers no more exceptions with SSL.

Awesome!  That was a nasty one.


Update azure-stack-release-notes-1904.md

Added an issue with deleting marketplace products

@avishwan : Thanks for your contribution! The author, @sethmanheim, has been notified to review your proposed change.

#sign-off

@avishwan: I'm sorry - only the author of this article, @sethmanheim, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

#sign-off


ReactTestUtils has been moved to react-dom/test-utils. Update references to remove this warning.

Received this warning when doing a git commit on a mern package.

This was fixed by adding the react-dom and react-test-renderer packages to the distribution.


fix case where the tmux server has not been created yet

hold on. i messed up something. pls ignore this


How To Build A USSD Application With Django

I wrote an article on how to create a USSD application with Django and Africastalking's API.
The article demonstrates this by building a USSD application that retrieves the price of few cryptocurrencies in some fiat currency. It also has an exchange rate functionality for US dollars to few other currencies.
Images used in this article include:

a .jpg hero image
a .jpg image that shows what the response from the SDK used to get cryptocurrency prices looks like.
a .png image to show what saving sensitive app information on .env file
.jpg images to demonstrate how to test the USSD application using africastalking's simulator

@Olamidun
Do we have any updates?

@Olamidun
Do we have any updates?
Let me know if you need any help.


Update repository link

crates.io for this package was pointing to a different repo. This should fix it.

Thanks @eholk!