Unpacks should check and  always do fetch if content not found

This patch adds an additional check in the unpack logic to "always" fetch the content if not present. This handles a case that can lead to incomplete images (images that have the RootFs in the snapshot but have a different digest in their oci config) and prevent snapshotters other than the default from working.
In terms of code changes...
This patch moves the content check and fetch to the start of doUnpackFn. This means that we "always" will check for content even if the snapshot diffid is unpacked already and download the content if missing.
Fixes https://github.com/containerd/containerd/issues/8674
Signed-off-by: Simon Kaegi<EMAIL_ADDRESS>

Another possibility here is to always do the content check and fetch before the sn.Stat check and then remove the later fetch.
e.g.
if _, err := cs.Info(ctx, desc.Digest); err != nil {
	if errdefs.IsNotFound(err) {
		return doFetchFn(i)
	} else {
		return fmt.Errorf("failed to get content info %s: %w", desc.Digest.String(), err)
	}
}

if _, err := sn.Stat(ctx, chainID); err == nil {
	// no need to handle
	return nil
} else if !errdefs.IsNotFound(err) {
	return fmt.Errorf("failed to stat snapshot %s: %w", chainID, err)
}

Here's a quick problem repeater...
crictl pull uk.icr.io/armada-master/ingress-alpine:3.18_249
crictl pull alpine:3.18.0
ctr -n k8s.io image export /tmp/img docker.io/library/alpine:3.18.0

This shows a bit more of the problem...
ctr -n k8s.io i check | grep incomplete
ctr -n k8s.io content ls | grep 8a49

images that have the RootFs in the snapshot but have a different digest in their oci config

What is this referring to and where is the difference?
In your case , is discard_unpacked_layers set?
The transfer service configuration is intended to solve the multiple snapshotter unpack issue. In the case here, is the devmapper snapshotter specified during the pull and it still fails? It seems that case would have a better solution than pulling the content even when it is not needed for the configured unpack and the content is to be discarded.

Makes sense. Just want to understand any potential impact to GC, since we may end up with multiple contents referencing the same snapshot. If we remove one of the images, will the snapshot remain intact? Is it still possible to start a container for the other image?

Great question. With this PR patch applied this is what I saw. I first applied the two pods... (the kata pod is using devmapper)
apiVersion: v1
kind: Pod
metadata:
  name: trusted-alpine-ingress
spec:
  containers:
    - name: alpine
      image: uk.icr.io/armada-master/ingress-alpine:3.18_249
      imagePullPolicy: IfNotPresent
      command:
      - sleep
      - "50000000"
---
apiVersion: v1
kind: Pod
metadata:
  name: untrusted-alpine
spec:
  runtimeClassName: kata
  containers:
    - name: alpine
      image: alpine:3.18.0
      imagePullPolicy: IfNotPresent
      command:
      - sleep
      - "50000000"

nsenter-ing on to the node I see...
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# alias c="ctr -n k8s.io"
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# c snapshots --snapshotter devmapper ls | grep bb01
37a0b1f65024713b470adf0ca38d7a2ba19303497c70cab0bdc1756894fc4225        sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c Active
sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c                                                                         Committed
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# c snapshots --snapshotter overlayfs ls | grep bb01
sha256:538b3bf128118798072d800141631b0524e0c5b8739b872f264e4b999480b17e sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c Committed
sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c                                                                         Committed
sha256:c0669ef34cdc14332c0f1ab0c2c01acb91d96014b172f1a76f3a39e63d1f0bda	528B	About a minute	containerd.io/distribution.source.docker.io=library/alpine,containerd.io/gc.ref.content.config=sha256:5e2b554c1c45d22c9d1aa836828828e320a26011b76c08631ac896cbc3625e3e,containerd.io/gc.ref.content.l.0=sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6

I then deleted the kata pod -- kubectl delete pod untrusted-alpine and nsenter-ed the node again and ran...
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# alias c="ctr -n k8s.io"
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# c i prune --all
INFO[0000] deleted image: docker.io/library/alpine:3.18.0
INFO[0000] deleted image: docker.io/library/alpine@sha256:02bb6f428431fbc2809c5d1b41eab5a68350194fb508869a33cb1af4444c9b11
INFO[0000] deleted image: sha256:5e2b554c1c45d22c9d1aa836828828e320a26011b76c08631ac896cbc3625e3e
INFO[0000] deleted image: sha256:8de5ed13778b0186780b382dd34d0f9ba7eee9d46f584d6441e90321f657f3ab
INFO[0000] deleted image: uk.icr.io/armada-master/ingress-alpine@sha256:2f1ad34e0a7030a8f9a030a1ec99d8ebaa0a9ee4e9cb59ee04e244337d94726c
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# c snapshots --snapshotter devmapper ls | grep bb01
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# c snapshots --snapshotter overlayfs ls | grep bb01
sha256:538b3bf128118798072d800141631b0524e0c5b8739b872f264e4b999480b17e sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c Committed
sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c                                                                         Committed
root@kube-cf431inl06scgp3lmpt0-gen2lon-seconda-00001c53:/# ctr -n k8s.io content ls | grep bb01
sha256:bc01fbd705408cb3b447eb0ad4059bf08f686442be80e65dd951ce884b64a595	3.511MB	7 minutes	containerd.io/distribution.source.uk.icr.io=armada-master/ingress-alpine,containerd.io/uncompressed=sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c

From what I saw removing the image did not clean up anything until I also did a prune --all. After the ctr i prune --all though what I saw looked correct and both the content and devmapper snapshotter cleaneup correctly and the snapshotter for the running container or the content for the other image were left as is as one would expect. Not shown... but if I delete the remaing pod and c i prune --all everything is cleaned up completely. (I hope this description makes sense!)

images that have the RootFs in the snapshot but have a different digest in their oci config

What is this referring to and where is the difference?

When using uk.icr.io/armada-master/ingress-alpine:3.18_249 and alpine:3.18.0 we have a case where two blobs of content from the remote repositories unzip to the same bits.
sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 from alpine:3.18.0 and sha256:bc01fbd705408cb3b447eb0ad4059bf08f686442be80e65dd951ce884b64a595 from uk.icr.io/armada-master/ingress-alpine:3.18_249 unzip to the same bits ... sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c

In your case , is discard_unpacked_layers set?

This is set to false in our environment

The transfer service configuration is intended to solve the multiple snapshotter unpack issue. In the case here, is the devmapper snapshotter specified during the pull and it still fails?

The unpack via CRI currently does not less us control this (I believe @mikebrow) and so uses the matching platform unpacker which pulls the content and unzips it to overlayfs by default. This just adds overhead!

It seems that case would have a better solution than pulling the content even when it is not needed for the configured unpack and the content is to be discarded.

Definitely! We currently see the content first downloaded and unzipped to overlayfs and then again unzipped to devmapper.

nod.. we are fixing the disconnect in k8s where we lost the annotation that specified the pod runtime handler(from class) on image status and pull from kubelet ..

This change breaks remote snapshotters which rely on the snapshotter being able to return errdef.ErrAlreadyExists from Prepare to skip content fetching. More details in the remote snapshotter doc.

This change breaks remote snapshotters which rely on the snapshotter being able to return errdef.ErrAlreadyExists from Prepare to skip content fetching.

Yep. This is why the CRI needs to pass down the snapshotter vs. just using the default snapshotter. If the correct snapshotter had been passed down then the existing logic would have worked. The patch above is a workaround (apparently for people not using a remote snapshotter) for using two snapshotters in 1.7.x.

This change breaks remote snapshotters which rely on the snapshotter being able to return errdef.ErrAlreadyExists from Prepare to skip content fetching. More details in the remote snapshotter doc.

Indeed, it seems the PR is not a complete fixup, one possible situation in the real world:

Some images in the pod (including the pause image) have used containerd's default snapshotter (such as the overlayfs snapshotter);
The discard_unpacked_layers option was previously set to true in containerd config;
Containerd has already deleted the blobs from the content store;
The user switches to another snapshotter (for example nydus/estargz lazy loading implementation) to pull the image.

A possible workaround: ensure that discard_unpacked_layers=false, then use the following command to restore the image:
ctr -n k8s.io content fetch pause:3.8

The pause:3.8 is just an example image, we should also fetch all images used by the Pod to ensure that there are no issues.
cc @skaegi @dmcgowan @Kern--

I agree with @imeoer that the correct solution for the specific error is discard_unpacked_layers=false. Long term, we really need to know the target snapshotter the pull is for if discard_unpacked_layers=true is used.

When using uk.icr.io/armada-master/ingress-alpine:3.18_249 and alpine:3.18.0 we have a case where two blobs of content from the remote repositories unzip to the same bits.
sha256:8a49fdb3b6a5ff2bd8ec6a86c05b2922a0f7454579ecc07637e94dfd1d0639b6 from alpine:3.18.0 and sha256:bc01fbd705408cb3b447eb0ad4059bf08f686442be80e65dd951ce884b64a595 from uk.icr.io/armada-master/ingress-alpine:3.18_249 unzip to the same bits ... sha256:bb01bd7e32b58b6694c8c3622c230171f1cec24001a82068a8d30d338f420d6c

This is a pretty uncommon case that we definitely should have a solution for. Always pulling doesn't make sense for the unpacker though as it would degrade performance for the vast majority of pulls which would not have this issue. I think this was mentioned before but the most ideal solution would be to have the pull target both snapshotters for unpack. This optimized unpacker logic was not designed for breaking apart pull and unpack like ctr does today.

Sorry guys, I left this here as for whatever reason a number of teams started using it. I do not think this is mergeable for both the remote snapshotter and unpack reasons. Although this patch kind of works for our use-case, we see still get the extra unpack to the "default" overlay snaphotter which takes both extra disk space and time.
We really need the snapshotter passed down and have been using the io.containerd.cri.runtime-handler annotation and it's associated logic to what we need as a stop-gap while we wait for KEP 4216 and associated changes in containerd.


Remove unique name and fileReference constraints from sitelog_associateddocument table

As suggested, I added deferrable="false", initiallyDeferred="true" for the two unique field constraints in liquibase change log and it works fine. So, I'll created a new PR for deferring checking constraints until transactions are committed.


[SPARK-29592][SQL] ALTER TABLE (set partition location) should look up catalog/table like v2 commands

What changes were proposed in this pull request?

Update AlterTableSetLocationStatement to store partitionSpec and make ALTER TABLE a.b.c PARTITION(...) SET LOCATION 'loc' fail if partitionSpec is set with unsupported message.
Why are the changes needed?

It's important to make all the commands have the same table resolution behavior, to avoid confusing end-users. e.g.
USE my_catalog
DESC t // success and describe the table t from my_catalog
ALTER TABLE t PARTITION(...) SET LOCATION 'loc' // report set location with partition spec is not supported.

Does this PR introduce any user-facing change?

yes. When running ALTER TABLE (set partition location), Spark fails the command if the current catalog is set to a v2 catalog, or the table name specified a v2 catalog.
How was this patch tested?

New unit tests

cc: @cloud-fan @rdblue @viirya

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17805/
Test PASSed.

Test build #112881 has started for PR 26304 at commit 400525e.

Test build #112881 has finished for PR 26304 at commit 400525e.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112881/
Test FAILed.

seems a related test failure.

LGTM if tests pass

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17855/
Test PASSed.

Test build #112942 has started for PR 26304 at commit 1f59072.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17865/
Test PASSed.

Test build #112932 has finished for PR 26304 at commit bc76fdf.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112932/
Test PASSed.

Test build #112942 has finished for PR 26304 at commit 1f59072.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112942/
Test PASSed.

thanks, merging to master!


Should we use spack develop environement instead of spack dev-build?

this is a larger questions - will be first investigated with prototype


Hide names index id

Make it a truely internal thing. It will allow us to change the names index ids and records/clusters any time as we improve the matching.

see https://github.com/Sp2000/colplus-backend/issues/298

https://github.com/Sp2000/colplus-backend/commit/1cacc80011e6a0393da64c977ccd605bab3661f2


Unable to Configure Checkout Extension from Remix App Home Page

I've developed a Shopify app that includes multiple checkout extensions. However, I'm encountering an issue when trying to install the checkout extension on dev store from only remix app. Specifically, I'm unable to configure the checkout extension from the Remix App home page.
I suspect there might be some mutations or APIs that I'm unable to locate. I need assistance in finding them.
Remix App Home Page
Screencast from 16-05-24 03:27:00 PM IST.webm

@ksyp-7 It's not clear from the video what you're expecting versus what you're seeing. Could you please add some additional details to help us understand the issue?


How to add authInfo to request with generated client

Generated Client does not include authInfo as second parameter
I want to include a bearer token as a Authentication header For each request I make with the generated go-swagger client.
Examples I've seen is doing something like this:
auth := httptransport.BearerToken(rawToken), nil
client.cart.CancelCart(params, auth)

Where the different methods takes authInfo of type runtime.ClientAuthInfoWriter as a second parameter like this:
type ClientService interface {
       CancelCart(params *CancelCartParams, authInfo runtime.ClientAuthInfoWriter) (*CancelCartOK, error)
}

However in my generated client it looks like this:
// ClientOption is the option for Client methods
type ClientOption func(*runtime.ClientOperation)
type ClientService interface {
      CancelCart(params *CancelCartParams, opts ...ClientOption) (*CancelCartOK, error)
}

Why don't I get authInfo as a second parameter?
Is it possible to achieve the same goal the way my client is generated?
Swagger specification
Client generated with a swagger specification of version 2.0
Environment
swagger version: 0.26.1
go version: 1.16.4

@martinostvik how did you resolve this?


I was trying to install platformio-ide-terminal but all i get is a failure to install log.

[ ] I have reset Atom to defaults prior to submitting report.
[ ] I have not reset Atom to defaults prior to submitting report.

Description

Steps to reproduce

Expected behavior:
Actual behavior:
Reproduces how often:
Versions

> atom --version

> apm --version

OS name and version:
Platformio-ide-terminal version:
Additional Information

@Lufianti thanks for bring this issue to our attention.
This is a duplicate of #700 see https://github.com/platformio/platformio-atom-ide-terminal/issues/707#issuecomment-514634895 for a possible workaround

ignoring the issue template is a sure way to get issue closed also


2feature/27-Localise-The-Home-Page

First pass at localisation.
Note there's no french/spanish...

thanks, i have some time to look at this now. in order to merge this i'd want to

make it so interface language is settable somehow
add french and spanish with google translate
take out the page numbering feature

do you want to do that or otherwise im happy to
seems like making it settable is the only one that's not super clear. perhaps to start it we would just use routes and not make ui changes:
pdf.code4recovery.org
pdf.code4recovery.org/es
pdf.code4recovery.org/fr

Making it configurable would be good.
I'm happy to make it respond to /es and /fr but I'm pressed for time until next year. That said, given that this is only a few week's away, if that timeframe is OK, I can do it.
I see removing the page numbering feature as a DIFFERENT PR, unless you mean it has already been removed and so it doesn't need a translation any longer?

NB. My French is good enough to spot any silly translations; and I can understand enough Spanish to catch any egregious errors.
It would be useful to find a "proper translator" to double check; given the type of organisation this serves, there's probably several who would be willing to double check the Google translate as a way of contributing back.

great, thanks. if you could take it to the finish line next year that'd be terrific
you added some page numbering logic as part of this PR that i would prefer to handle separately


After upgrading to Homebridge 1.3.1 I get warnings

Hi,
The ARSO plugin is still working, but warnings appear after upgrading to the latest version of Homebridge.
Warnings from Hambridge logs:
[03/03/2021, 18:22:42] [homebridge-arso] This plugin generated a warning from the characteristic 'Ozone Density': characteristic value expected float and received undefined. See https://git.io/JtMGR for more info.
[03/03/2021, 18:22:42] [homebridge-arso] This plugin generated a warning from the characteristic 'Sulphur Dioxide Density': characteristic value expected float and received undefined. See https://git.io/JtMGR for more info.
BR,
David

Hi @ortoberi!
Thanks for the heads up.
Fix is on the way.


default template function treats 0 value as unspecified

Hi!
I've encountered an interesting problem with helm templating engine: default function treats 0 (integer) values as unspecified and uses $default parameter.

A small example:
Having single templating yaml and values:
template.yaml:
apiVersion: v1
kind: Something
metadata:
  name: {{ default 1 .Values.myvalue0 }}
  surname: {{ default 1 .Values.myvalue555 }}

values.yaml
myvalue0: 0
myvalue555: 555

Running helm template test test
Received output:
---
# Source: test/templates/template.yaml
apiVersion: v1
kind: Something
metadata:
  name: 1
  surname: 555

Expected output:
---
# Source: test/templates/template.yaml
apiVersion: v1
kind: Something
metadata:
  name: 0
  surname: 555

Seems like helm treats zero values as unspecified and replaces with first argument for default templating function, which is not expected behavior.

Output of helm version:
version.BuildInfo{Version:"v3.12.3", GitCommit:"3a31588ad33fe3b89af5a2a54ee1d25bfe6eaa5e", GitTreeState:"clean", GoVersion:"go1.20.7"}

Output of kubectl version:
k version                      (minikube/default)
Client Version: v1.28.0
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.27.4

Cloud Provider/Platform (AKS, GKE, Minikube etc.):  Minikube (doesn't actually matter)

@Endevir please see https://masterminds.github.io/sprig/defaults.html

this is right behavor.

@yxxhero: this is right behavor.

Yes, it is the right behavior ... in such a case, I recommend using an explicit check for 0 as a value. Also coalesce will not help in this case - it is a go feature.


nbcsports.com "Unable to download XML"

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2023.02.17 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
USA
Provide a description that is worded well enough to be understood
The video here plays perfectly on the website when accessed via a VPN server in the US, and doesn't require a log-in.
However, attempts to download the video, with the page open or with it closed, result in the error "Unable to download XML".
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-F', 'https://www.nbcsports.com/video/extended-highlights-aston-villa-2-arsenal-4', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.02.17 [a0a7c01] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.4-full_build-www.gyan.dev (setts), ffprobe 4.4-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1782 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.02.17, Current version: 2023.02.17
yt-dlp is up to date (2023.02.17)
[NBCSports] Extracting URL: https://www.nbcsports.com/video/extended-highlights-aston-villa-2-arsenal-4
[NBCSports] extended-highlights-aston-villa-2-arsenal-4: Downloading webpage
[NBCSportsVPlayer] Extracting URL: https://www.nbcsports.com/vplayer/p/BxmELC/nbcsports/select/media/9SvBG9n83EcQ?form=html&autoPlay=true
[NBCSportsVPlayer] 9SvBG9n83EcQ: Downloading webpage
[ThePlatform] Extracting URL: https://link.theplatform.com/s/BxmELC/media/9SvBG9n83EcQ?auto=true&player=v1.8%20-%20NBCS%20Main&policy=194713254
[ThePlatform] 9SvBG9n83EcQ: Downloading SMIL data
ERROR: [ThePlatform] 9SvBG9n83EcQ: Unable to download XML: Remote end closed connection without response (caused by RemoteDisconnected('Remote end closed connection without response')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 693, in extract
  File "yt_dlp\extractor\theplatform.py", line 298, in _real_extract
  File "yt_dlp\extractor\theplatform.py", line 32, in _extract_theplatform_smil
  File "yt_dlp\extractor\common.py", line 1047, in download_content
  File "yt_dlp\extractor\common.py", line 1011, in download_handle
  File "yt_dlp\extractor\adobepass.py", line 1366, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 881, in _download_webpage_handle
  File "yt_dlp\extractor\common.py", line 838, in _request_webpage

  File "yt_dlp\extractor\common.py", line 820, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3726, in urlopen
  File "urllib\request.py", line 525, in open
  File "urllib\request.py", line 542, in _open
  File "urllib\request.py", line 502, in _call_chain
  File "yt_dlp\utils.py", line 1371, in http_open
  File "urllib\request.py", line 1358, in do_open
  File "http\client.py", line 1344, in getresponse
  File "http\client.py", line 307, in begin
  File "http\client.py", line 276, in _read_status
http.client.RemoteDisconnected: Remote end closed connection without response

I cannot reproduce the issue from US IP
[NBCSports] Extracting URL: https://www.nbcsports.com/video/extended-highlights-aston-villa-2-arsenal-4
[NBCSports] extended-highlights-aston-villa-2-arsenal-4: Downloading webpage
[NBCSportsVPlayer] Extracting URL: https://www.nbcsports.com/vplayer/p/BxmELC/nbcsports/select/media/9SvBG9n83EcQ?form=html&autoPlay=true
[NBCSportsVPlayer] 9SvBG9n83EcQ: Downloading webpage
[ThePlatform] Extracting URL: https://link.theplatform.com/s/BxmELC/media/9SvBG9n83EcQ?auto=true&player=v1.8%20-%20NBCS%20Main&policy=194713254
[ThePlatform] 9SvBG9n83EcQ: Downloading SMIL data
[ThePlatform] 9SvBG9n83EcQ: Checking video URL
[ThePlatform] 9SvBG9n83EcQ: Downloading JSON metadata
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 9SvBG9n83EcQ:
ID        EXT RESOLUTION │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
─────────────────────────────────────────────────────────────────────────────
http-5931 mp4 1920x1080  │ ~730.79MiB 5931k https │ unknown 5931k unknown  0k

Can you try via VPN? Even if you're already in the US, this could replicate the issue, in which case I'll need to figure out how to solve that. Maybe even the connection protocol can affect it. Thanks.


✨(backend) /api/v1.0/teams/{team_id}/accesses/me/

Feature Request
For the moment, we don't have a way:

to know the role of the current user per team
to know if the current user is the last owner

It would be nice to have the endpoint:
/api/v1.0/teams/{team_id}/accesses/me/
Response:
{
  "id": "03fd543f-883e-4d40-b415-a67b3445b685",
  "user": {
    "id": "2a24d537-42e0-47e9-8019-7a8e36e6f7bc",
    "name": null,
    "email": null
   },
  "role": "owner",
  "is_last_owner": true,
  "abilities": {
    "delete": false,
    "get": true,
    "patch": false,
    "put": false,
    "set_role_to": []
  }
}

Purpose
If we are member of a team, we don't want to propose the action buttons.

If we are the last_owner of a team, we don't want to be able to update the role.

I will read again what abilities return to make sure we need to update this response. Will comment my findings here.

Abilities should provide all required information to do what you meant, for a MVP. Let's jump in a call with @sampaccoud or me to clarify!

Yes, abilities give all all this info and it is a bit harder than knowing your role in the team because it also depends on the role of each access.
For example, if I'm an administrator, I can set a member to "administrator" role but I can not modify the role of an "owner"...
The backend does all these computations for you and gives you all the abilities information for the logged-in user in the abilities attribute of each team access.
If "delete" is true, you can display the "delete" item in the three dots menu
If "put"  is true, you can display the "modify the role" item in the three dots menu
When "put" is true, the "set_role_to" attribute will indicate to what new role you are able to update this team access...
I hope it clarifies?

I still see a problem though, if I am a member, set_role_to is init, I guess it should be empty:


Creating ragged tensors is incredibly slow

I'm running the following code on a Google Colab instance with GPU support enabled:
tf.ragged.constant(np.random.randint(10, size = 10_000_000))

The code takes 15 seconds to finish. In comparison, tf.constant(np.random.randint(10, size = 10_000_000)) takes only 50 milliseconds to finish.

I am able to replicate the issue reported on tf 2.3,2.4 and nightly, please find the gist here.

@bjourne Recently Colab has some problems in accessing gpu. So, even when you select gpu it is still running in cpu. Please check the gist here.
Adding the following code to check gpu usage throws error
device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  raise SystemError('GPU device not found')
print('Found GPU at: {}'.format(device_name))

Can you please run it on GPU and let us know if there is any issue there. Thanks!

Yes, same thing with a GPU:
import numpy as np
from time import time

device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  raise SystemError('GPU device not found')
print('Found GPU at: {}'.format(device_name))

st = time()
tf.ragged.constant(np.random.randint(10, size = 10_000_000))
print(time() - st)

The code takes 14.8 seconds to finish.

It looks like most of the time is being spent in _find_scalar_and_max_depth.  Two options to speed up that function:

short-circuit and return early once we find any scalar value.  (But this may prevent us from giving good error messages if the user supplies an input with scalar values at different nesting depths.)
change np.ndim(pylist) != 0 to (isinstance(pylist, np.ndarray) and np.ndim(pylist) != 0 -- calling ndim seems to take up a fair amount of time.  Not sure if this will break any existing use-cases.

But even with these fixes, tf.ragged.constant will probably still be significantly slower than tf.constant (especially if the input is a numpy array), since tf.constant is implemented in c and tf.ragged.constant is implemented in Python.
If you want to ingest ragged large tensors quickly, and it's possible to get them into a format other than nested python lists, then you might consider using one of the following:

The RaggedTensor factory methods, such as tf.ragged.from_row_splits or tf.ragged.from_row_lengths.  (Use validate=False to prevent validation ops from being added to the graph, which can slow things down.)
The RaggedTensor.from_tensor method, which has optional padding and lengths arguments you can use to specify which values are padding.
If you have a list of numpy arrays (for each row), then you could use tf.ragged.stack(list_of_numpy_arrays).

All of these options will be substantially faster than tf.ragged.constant.
What is the actual shape of ragged tensors you want to ingest?  This could impact what approach will be best for ingesting them.  Examples:

a 2d tensor [dense, ragged] where dense and ragged are both very large.
a 3d tensor [dense_1, ragged_1, ragged_2, dense_2], where dense_1 and ragged_1 are large, but ragged_2 and dense_2 are small.

(In the example you used for timing, you pass in a 1D list, which can't even be ragged -- RaggedTensors always have rank>1.)

It's a 2d-tensor [n, size]. n is 10k and size varies between 100 to 1000.

I struggle with the same issue, have you managed to improve the efficiency?

@bjourne Is this still an issue for you? I think this is intended and @edloper provided couple of suggestions to improve. Thanks!

Yeah it's still an issue! The code I pasted still runs dog slow.

I have a list of 2d numpy arrays (m,n) where m is variable (order of 1000) and n is fixed (say 1000). I have around 10000 such arrays. The final ragged tensor should be of shape (10000, None, 1000). RaggedTensor.stack is fast however it results into a ragged tensor of ragged_rank=2, shape=(10000, None, None). How can I convert this into a tensor which has a ragged rank of 1. RaggedTensor.constant gives correct results, however it is incredibly slow.

I have a list of 2d numpy arrays (m,n) where m is variable (order of 1000) and n is fixed (say 1000). I have around 10000 such arrays. The final ragged tensor should be of shape (10000, None, 1000). RaggedTensor.stack is fast however it results into a ragged tensor of ragged_rank=2, shape=(10000, None, None). How can I convert this into a tensor which has a ragged rank of 1. RaggedTensor.constant gives correct results, however it is incredibly slow.

Hi @deepakagrawal,
Try using factory class-methods such as:

tf.RaggedTensor.from_value_rowids,
tf.RaggedTensor.from_row_lengths, and
tf.RaggedTensor.from_row_splits.

Here's an example of using tf.RaggedTensor.from_row_splits to convert a (100000, 1000) numpy-array into a (1000, None, 1000) ragged-tensor. https://colab.research.google.com/drive/1WnyvzoB8oE5vGXq5iYHJAAv7RQDADVU8?usp=sharing
CPU times: user 16.5 ms, sys: 18.6 ms, total: 35.1 ms
Wall time: 39.7 ms

@deepakagrawal: +1 to using the RaggedTensor factory methods.  In your example, you could convert the numpy arrays into a RaggedTensor using RaggedTensor.from_row_lengths as follows:
import numpy as np
np_arrays = [np.zeros([i, 1000]) for i in range(1000)]  # example array.

ragged_tensor = tf.RaggedTensor.from_row_lengths(
    values=tf.concat(np_arrays, axis=0),
    row_lengths=[a.shape[0] for a in np_arrays])

Thanks, This worked perfectly

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The Tensorflow team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the Tensorflow space.


[BUG] Do not save endpoint to the endpoints object if the check for public was not successful

Describe the bug:

Do not save endpoint to the endpoints object if the check for public was not successful
To Reproduce:

initEndpoint({ id: 'test', url: 'https://someurlthatdoesnotexist' })
Expected behavior:

There should be no test endpoint existing if the call above fails and it should throw, right now there is.

Fixed in previous commit


✨ Feature: URI Component Encoding

Description
Implement an exact replicate of encodeURIComponent and decodeURIComponent from Javascript, as it is useful for concatenating raw string data into URIs.
Included as many tests for edge cases as I could think of.
Checklist
Ensure you completed all of the steps below before submitting your pull request:

[x] Ran forge fmt?
[x] Ran forge test?

Reference
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent

Interesting work. If I may, what’s the use case?
If there’s a good use case, we’ll rewrite it in fully optimised form and add it to LibString.
On authorship, we’ll just use Solady. We don’t usually add individual authors, unless it is a link to some kind of article that describes the full algorithm, or a link to a simpler reference implementation. Otherwise, the files will be choked full with authors.

@Vectorized sounds good!
also great question for use case.
for me? on-chain animation_url Data URI creation and on-chain tokenURI Data URI creation as well. if you want to use data:application/json;utf8,... all things after the utf8 must be URI encoded. the animation_url content must be double encoded, so having this abstracted away into a function is useful.
// for a tokenURI
string.concat(
    "data:application/json;utf8,",
    compiler.encodeURI(allTheJsonAsAString),
)

updating PR real quick to remove my author tag.

Ok, let’s add this.

@Vectorized hype! let me know if i can help at all with moving it over to libstring and optimizing. curious how you approach optimizing, good learning experience for me.


Added support for geometry function

Coverage remained the same at 98.507% when pulling 3445b7dfaa451263d897f67ac84c24016cb5930f on dragosprotung:geometry into 5829a5266033148ad94caf0688a7a1e43af6acdc on creof:master.

Coverage remained the same at 98.507% when pulling 3445b7dfaa451263d897f67ac84c24016cb5930f on dragosprotung:geometry into 5829a5266033148ad94caf0688a7a1e43af6acdc on creof:master.

Coverage remained the same at 98.507% when pulling 3445b7dfaa451263d897f67ac84c24016cb5930f on dragosprotung:geometry into 5829a5266033148ad94caf0688a7a1e43af6acdc on creof:master.

Coverage remained the same at 98.507% when pulling 8ae16cc8eb3cbe656db9cba3e16a169a26c6642f on dragosprotung:geometry into 5829a5266033148ad94caf0688a7a1e43af6acdc on creof:master.

I am not sure if I should do anything regarding the codeclimate check

No worries about code climate !


Change Area Within UCI App

Bug Report
Issues should only be created for items related to covered functionality.
Not covered functionality, feature requests, and questions should use the Feature Request or Question templates.
EasyRepro Version

[ ] Microsoft Dynamics 365 Online Version 8.2 (8.2.x) (DB 8.2.x) online
[ ] Microsoft Dynamics 365 Online Version 9.0 (9.0.x) (DB 9.0.x) online
[ ] Microsoft Dynamics 365 Online Version 9.0.2 (9.0.2) (DB 9.0.2) online
[X] Microsoft Dynamics 365 Online Version 9.1 (9.1.x) (DB 9.1.x) online

UCI or Classic Web

[X] UCI (XrmApp)
[ ] Classic Web (XrmBrowser)

Online or On Premise

[X] Online
[ ] On Premise

Browser

[X] Chrome
[ ] Firefox
[ ] IE
[ ] Edge

Describe the bug
Hi,
I want to check if all entities still are displayed in an app. The app contains 2 "Area's":

Management
Settings

By default the App opens in the area Stock Management and I can check all entities by using xrmApp.Navigation.OpenSubArea("Stock Management", "EntityA");
xrmApp.Navigation.OpenSubArea("Stock Management", "EntitiyB");
When I Try to Open Entity C on the Settings Area by using
xrmApp.Navigation.OpenSubArea("Settings", "Reports"); the test failes with the following error:

Unable to click element
Stacktrace:
"   at Microsoft.Dynamics365.UIAutomation.Browser.SeleniumExtensions.<>c.b__4_1(IWebDriver e) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Browser\Extensions\SeleniumExtensions.cs:line 80\r\n   at Microsoft.Dynamics365.UIAutomation.Browser.SeleniumExtensions.WaitUntilClickable(IWebDriver driver, By by, TimeSpan timeout, Action1 successCallback, Action1 failureCallback) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Browser\Extensions\SeleniumExtensions.cs:line 736\r\n   at Microsoft.Dynamics365.UIAutomation.Browser.SeleniumExtensions.ClickWhenAvailable(IWebDriver driver, By by, TimeSpan timeout) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Browser\Extensions\SeleniumExtensions.cs:line 72\r\n   at Microsoft.Dynamics365.UIAutomation.Browser.SeleniumExtensions.ClickWhenAvailable(IWebDriver driver, By by) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Browser\Extensions\SeleniumExtensions.cs:line 66\r\n   at Microsoft.Dynamics365.UIAutomation.Api.UCI.WebClient.<>c.b__24_0(IWebDriver driver) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Api.UCI\WebClient.cs:line 457\r\n   at Microsoft.Dynamics365.UIAutomation.Browser.DelegateBrowserCommand1.ExecuteCommand(IWebDriver driver, Object[] params) in C:\\-= Tools =-\\EasyRepro-develop\\Microsoft.Dynamics365.UIAutomation.Browser\\DelegateBrowserCommand.cs:line 29\r\n   at Microsoft.Dynamics365.UIAutomation.Browser.BrowserCommand1.Execute[T1,T2,T3,T4,T5,T6,T7,T8,T9](IWebDriver driver, T1 p1, T2 p2, T3 p3, T4 p4, T5 p5, T6 p6, T7 p7, T8 p8, T9 p9) in C:\-= Tools =-\EasyRepro-develop\Microsoft.Dynamics365.UIAutomation.Browser\BrowserCommand.cs:line 192"

Fixed By Adding ChangeAreas Code


sigpaths parameter datatype

The sigpaths parameter in the load_vw function expects a string, but is passed a list.
Changed:
def load_vw(
    sample_path: str,
    format: str,
    sigpaths: str,
    should_save_workspace: bool = False,
) -> VivWorkspace:

to:
def load_vw(
    sample_path: str,
    format: str,
    sigpaths: list,
    should_save_workspace: bool = False,
) -> VivWorkspace:

The paths are loaded by the get_signatures function, which returns a list.

Also an unrelated question, is the is_workspace_file() function used for something currently?
def is_workspace_file(sample_file_path):
    """
    Return if input file is a vivisect workspace, based on file extension
    :param sample_file_path:
    :return: True if file extension is .viv, False otherwise
    """
    if os.path.splitext(sample_file_path)[1] == ".viv":
        return True
    return False

Should I clean it up to
def is_workspace_file(sample_file_path):
    """
    Return if input file is a vivisect workspace, based on file extension
    :param sample_file_path:
    :return: True if file extension is .viv, False otherwise
    """
    return os.path.splitext(sample_file_path)[1] == ".viv":

Good catch. is_workspace_file can be removed as it seems unused.

thanks!


Mapview.center() is often working late or less accurate

Issue Type
[X] Bug
Description and/or steps/code to reproduce the problem
I am using Osmdroid for over 4 years weekly. I like using the new osmdroid very much. Thank you!
Since using the osmdroid 6.0.1 centering on the mapview is often done very late or less accurate.  This is a bit annoying.
No code has been changed on this part of the App.
Environment
Android 7.0.1. Osmdroid 6.0.1.
If it's a bug, version(s) of android this affects:
Mapview.center( ... )
Version of osmdroid the issue relates to:
6.0.1

centering on the mapview is often done very late or less accurate

@johan974 There were indeed changes in 6.0.1 around map centering.
Could you please be more specific about the inaccuracy you experience, like how to reproduce it (zoom level, expected center, actual center), and about the "very late" part of your comments?
Please note that MapView.setMapCenter is now deprecated, and that MapView.setExpectedCenter should be used instead.

Every say 1 second I get a new GPS update. Using that I center the map, draw a line from my current location to the target. I see that especially after awakening my mobile (during navigation) it takes 5 seconds to get a good center.
Zoom level - often near 18 - 20.
The map center is set via: mapView.getController().setCenter( gp);

Then I would say it's a problem of GPS updates sometimes taking too much time, and not being always accurate, rather than mapView.getController().setCenter(gp) not being accurate. Do you agree with that?

Do not agree. For about 4 years I used previous versions and had no significant issues with this. Now centering the map takes often a lot of time. Sometimes it cannot find the center at all.
Be sure - I am very thankful for using the Osmdroid! This issue - I hope you can solve it!

@johan974 I must say I just found something fishy inside MyLocationNewOverlay.setLocation.
Do you use MyLocationNewOverlay?
Anyway, as a debug, you can add the following overlay to your MapView:
mMapView.getOverlays().add(new Overlay() {
	@Override
	public void draw(Canvas c, MapView osmv, boolean shadow) {
		Log.e("DebugOverlay", "projection center:" + osmv.getProjection().getCurrentCenter());
	}
});

And a similar log statement each time you run mapView.getController().setCenter(gp) (just before):
Log.e("DebugOverlay", "expected center:" + gp);

This way we'll be able to see:

if setCenter is slow
if the center is inaccurate

Tell me what you see in those logs.

Yes, I use MyLocationNewOverlay.
I have to go in the field to replay the issues.
The first statement, is that just creating an empty overlay, but still logging on every redraw of the screen?
My last updates (without the logging): sometimes I and a fellow app user, see that the app doesn't center at all. The map is sometimes even grey.

I will add the debug code. I am sure that I am tracking (following) to a target waypoint. Allow me some time to add the debug code and do the field test. Another person has the same issue with the app, tracking to a waypoint and sometimes seeing just a grey area. It takes sometimes over 10s to get a good center.

Yes, I have the enableFollowLoction on. Today I revisited my code that I use for almost 5 years now. It is a very, very large app for supporting geocaching in all its facets. It takes quite some time to put the debug statements in different parts.
I hoped you found some clues in the new v6.0 osmdroid code that performs the centering more slowly or not at all. Since v6 I have issues with the centering.

@johan974 I frankly don't know what's going on. I'm really confident in the setCenter feature though.
I changed some suspicious code in MyLocationNewOverlay that might be related to this issue.

PR in #1010

@johan974 Can you try with version 6.0.2?


Remove unneeded TryStream .map(Ok) uses

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

11 of 13 (84.62%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.02%) to 98.776%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core/src/executor/fetch.rs
5
6
83.33%

core/src/plan/schema.rs
4
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
38746

Relevant Lines:
39226

💛  - Coveralls


Provide a general way to write an StreamMessage to a file

@trustin suggested an awesome idea to write a StreamMessage<T> into a File or Path. https://github.com/line/armeria/pull/4044#discussion_r792374144
interface StreamMessage<T> {
    default CompletableFuture<Void> writeTo(Path destination, Function<? super T, ? extends HttpData> mapper, options...) {
        // Delegate to StreamMessages.writeTo(...)
    }
}

Hello @ikhoon 😀  I would like to handle this issue.  May I try it?

Sure! It is all yours. 😄


[SPARK-13774][SQL] - Improve error message for non-existent paths and add tests

SPARK-13774: IllegalArgumentException: Can not create a Path from an empty string for incorrect file path
Overview:

If a non-existent path is given in this call
scala> sqlContext.read.format("csv").load("file-path-is-incorrect.csv")
it throws the following error:
java.lang.IllegalArgumentException: Can not create a Path from an empty string …..
It gets called from inferSchema call in org.apache.spark.sql.execution.datasources.DataSource.resolveRelation

The purpose of this JIRA is to throw a better error message.

With the fix, you will now get a Path does not exist error message.

scala> sqlContext.read.format("csv").load("file-path-is-incorrect.csv")
org.apache.spark.sql.AnalysisException: Path does not exist: file:/Users/ksunitha/trunk/spark/file-path-is-incorrect.csv;
  at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$12.apply(DataSource.scala:215)
  at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$12.apply(DataSource.scala:204)
  ...
  at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:204)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:131)
  at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:141)
  ... 49 elided

Details
Changes include:

Check if path exists or not in resolveRelation in DataSource, and throw an AnalysisException with message like “Path does not exist: $path”
AnalysisException is thrown similar to the exceptions thrown in resolveRelation.
The glob path and the non glob path is checked with minimal calls to path exists. If the globPath is empty, then it is a nonexistent glob pattern and an error will be thrown. In the scenario that it is not globPath, it is necessary to only check if the first element in the Seq is valid or not.

Test modifications:

Changes went in for 3 tests to account for this error checking.
SQLQuerySuite:test("run sql directly on files") – Error message needed to be updated.
2 tests failed in MetastoreDataSourcesSuite because they had a dummy path and so test is modified to give a tempdir and allow it to move past so it can continue to test the codepath it meant to test

New Tests:
2 new tests are added to DataFrameSuite to validate that glob and non-glob path will throw the new error message.
Testing:
Unit tests were run with the fix.
Notes/Questions to reviewers:

There is some code duplication in DataSource.scala in resolveRelation method and also createSource with respect to getting the paths.  I have not made any changes to the createSource codepath.  Should we make the change there as well ?

From other JIRAs, I know there is restructuring and changes going on in this area, not sure how that will affect these changes, but since this seemed like a starter issue, I looked into it.  If we prefer not to add the overhead of the checks, or if there is a better place to do so, let me know.

I would appreciate your review. Thanks for your time and comments.

Can one of the admins verify this patch?

Jenkins test this please

Test build #53414 has started for PR 11775 at commit e03e776.

Test build #53414 has finished for PR 11775 at commit e03e776.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/53414/
Test FAILed.

Merged build finished. Test FAILed.

@srowen Thanks for your comments.  I have addressed them in the last commit.

cc @liancheng

LGTM. Merging to master. Thanks for the fix!

@skambha Do you have a Apache JIRA account? Could you please register an account if you don't so that I can assign this JIRA ticket to you? Thanks!

@skambha Nvm, found your account. It was just not in the Spark contributor group. Added.

Thanks @liancheng for the review, merging the fix  and adding me to the spark contributor group in Apache JIRA.


Security violation when installing CockroachDB

When downloading https://binaries.cockroachdb.com/cockroach-v2.0.1.windows-6.2-amd64.zip and then attempting to install CockroachDB, I received a security violation error, so I didn't continue with the installation.

Hi, Jesse,
Sorry. I didn't get a screen shot of the error message that I received, so I don't have any more info. for you.
Thanks,Liz
On Monday, May 7, 2018 3:19 PM, Jesse Seldess<EMAIL_ADDRESS>wrote:

@semmelmeyer, can you provide more details about the error? Even better, if you could just paste the error here, that'd help.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Thanks, @semmelmeyer. This is something I'll ask engineering about. If it's an issue we've seen before and there's docs work to do, I'll reopen this.


Cli providers params

closes #1179
This is a breaking change, for command line usage.

I like a lot of what I'm seeing here, getting rid of the click-option-group package is nice. Keeping all of the click usage confined to one module is clean.
I think I would prefer if this could be made a little bit closer to the approach that the Typer project uses. It combines using docstrings with an annotated function signature. See it in action in the CLI Options with Help section of their docs.
I would really really really prefer to not have anything syntactic in the doc strings. Using the doc string in the generated docs is great, but don't parse it.
class SomeProvider(Provider):
    def __init__(
            self,
            username: Annotated[str, Option(help="OpenSubtitles username")],
	    password: Annotated[str, Option(help="OpenSubtitles password")],
	)
        """
        Downloads subtitles from OpenSubtitles.

        [username] and [password] are required if not set in config.toml
        """
        ...

Instead, use the signature annotations for anything where you'd need to parse a docstring and inline the docstring into the generated documentation.
I think it's worth raising that hard coding the parameters that an extension can use is pretty restrictive. Who knows what kind of requirements might be needed by some future extension. Is there not some way to allow more flexibility in this regard?
Perhaps allowing aping a narrow subset of click's api and exposing that to extensions would work?

I like the usage of Annotated type hints, thanks, it's much better than parsing the docstring.

I think it's worth raising that hard coding the parameters that an extension can use is pretty restrictive. Who knows what kind of requirements might be needed by some future extension. Is there not some way to allow more flexibility in this regard?

It's not the case! It takes all the arguments of the __init__ method and transform them into click options. I hard-coded arguments to SKIP for the refiners but not the providers.

I'll try and incorporate this and the examples you gave in #1177 into my provider soon.

I decided not to use Annotated type hint and parse the docstring. If needed we can do it in another PR.


[WIP][SPARK-29334][ML][MLLIB] Support for basic vector operators

What changes were proposed in this pull request?
Basic operator support for ml.linalg.Vector, and mllib.linagl.Vector:

negation
scalar multiplication and division
vector addition and subtraction

Why are the changes needed?
To improve parity with Vector operators provided with pySpark.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit tests in the appropriate places.  To run quickly:
sbt "mllib-local/testOnly org.apache.spark.ml.linalg.VectorsSuite"

Can one of the admins verify this patch?

Can one of the admins verify this patch?

I've marked this PR as WIP because I am interested in feedback.  There are any number of ways of implementing.  I've wrapped BLAS where I could, and had to write some custom routines for handling SparseVector operations.
Additional work on this PR could include:

[ ] implicit support for a more natural experience.  The scalar operations are one-sided, and appending colon for analogous right associative operators seemed clunky and unintuitive.
[ ] replicating this code/functionality on the mllib side (wanted to wait for feedback before doing that)

@srowen @zhengruifeng Thanks for the feedback!

I think the implementations could be further optimized, but I don't think it's vital now

Yep, I definitely agree.  When the time comes we can try a few different implementations and do a scaling analysis.

I wonder are there places in the code that can use these operations for clarity?

I will search for usage of the wrapped BLAS routines and see.  One thing to note, for example, is that BLAS.axpy is a function with side effects (the y arg is modified).  So depending on how the routine is used in code, it might not make sense to substitute unless the y arg is copied before the call.


Not showing for newer mac versions.

Hi again :)
Lights seem to not render in some of the newer macOS versions at all, is this a known issue?

@SlightlyMad
I found a solution to rendering with the new macOS High Sierra, so I'll share it with you guys :)
It seems that upgrading from #pragma target 4.0 to #pragma target 4.5 enables "metal" shading.
Search replace the included shaders, and you should be good!

hi, thanks for the info. i don't have mac. or time. it used to target 5.0 but some people complained it doesn't run on their hw so i dropped it to 4.0. and for that reason i won't make it 4.5 for now. until i find a way how to support both. but i really don't have time for that now


[WIP] feat: implement complex select with orderby

What this PR does:
In short words:

more complex SELECT statements with order-by:

select uid,name,birth_year as birth from student where ... order by -birth_year;
select uid,name,birth_year as birth from student where ... order by 2022-birth_year;
select uid,name,2022-birth_year as age from student where ... order by 2022-birth_year;
select uid,name,2022-birth_year as age from student where ... order by age
select uid,name from student where ... order by 2022-birth_year;

-- TODO --
select uid,name,2022-birth_year from student where uid between 1 and 20 order by 2022-birth_year;

remove unnecessary sub-query in SimpleQueryPlan, will use UNION-ALL directly.

Codecov Report

Merging #379 (c5ae4ba) into master (e947693) will increase coverage by 0.01%.
The diff coverage is 61.29%.

@@            Coverage Diff             @@
##           master     #379      +/-   ##
==========================================
+ Coverage   39.61%   39.63%   +0.01%
==========================================
  Files         110      110
  Lines       14738    14782      +44
==========================================
+ Hits         5839     5859      +20
- Misses       8183     8204      +21
- Partials      716      719       +3

Impacted Files
Coverage Δ

pkg/runtime/ast/expression_atom.go
28.63% <0.00%> (-0.28%)
:arrow_down:

pkg/runtime/optimize/dml/aggregate_visitor.go
74.11% <16.66%> (-2.25%)
:arrow_down:

pkg/runtime/optimize/dml/select_scanner.go
70.16% <68.51%> (-6.86%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Angular CLI 7 Version

I made an Angular CLI Lib of your lib:
https://github.com/kernelport/ngx-image-zoom-dev
And add options styleWidth and styleHeight to scale  the 'thumbImage' size
This was an feature request in #6
It is not possible to make a pull request because it is a new project. But you can review this commit:
https://github.com/kernelport/ngx-image-zoom-dev/commit/9ae8110596de56dfc0dfb6ddf1a00332a8b1091a

Any update?

It works, see example project
https://github.com/kernelport/ngx-image-zoom-test

Thank you. I'll check out your solution when adding it into this project for sure. It looks nice and clean. But to keep discussions in one place I'll close this as a duplicate of #6 , but that doesn't mean I don't appreciate you letting me know about your implementation.

What is niz-NgxImageZoom on the template? Should it be  tag instead?

I do not understand. The library is well loaded, and the thumbnail image picture is visible, but there is no lens when I put the mouse over the picture... I'm using angular 7... with TypeScript ~3.1.6
Any idea?


[CARBONDATA-1808] Inconsistency in create table and alter table usage for char and varchar

…har column
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

[ ] Any interfaces changed?
NO

[ ] Any backward compatibility impacted?
NO

[ ] Document update required?
NO

[ ] Testing done
New Unit Test Cases Are Added Foer The Same

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1527/

SDV Build Fail , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/1927/

Build Failed  with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1532/

retest this please

Build Success with Spark 2.1.0, Please check CI http://<IP_ADDRESS>:8080/job/ApacheCarbonPRBuilder1/1536/

SDV Build Success , Please check CI http://<IP_ADDRESS>:8080/job/ApacheSDVTests/1935/


Is it possible to create a page based on a custom template using the Graph API?

What type of issue is this?
Question
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Target SharePoint environment
SharePoint Online
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[X] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details
No response
Issue description
Basically we were exploring Graph API to create SP pages but our scenario are based on custom template created by us, so we wanted to clarify is there is a way to create a page based on a custom template. We didn't see anything as part of the documentation and we would like to know if this is in the roadmap or not. Thanks.

Hi @roberAlb! It seems that currently, there's no straightforward method available in the pages API. However, you can achieve the same effect by following this procedure:

Retrieve the template item unique ID to use in the pages API. A template is actually the page located in the Template folder. You can do this using the following endpoint:
https://graph.microsoft.com/v1.0/sites/{siteId}/lists/{listId}/items/{itemListId}?$select=contentType,sharepointIds
Make sure to get the sharepointIds.listItemUniqueId for the next step.
Obtain the page content of the template. Note that page templates are not included in the list of pages using https://graph.microsoft.com/v1.0/sites/{SiteId}/pages endpoint, but you can still access them directly:
https://graph.microsoft.com/v1.0/sites/{SiteId}/pages/{listItemUniqueId}/microsoft.graph.sitePage?$expand=canvasLayout
Extract the canvasLayout object from the result and remove properties such as<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Make a POST call to https://graph.microsoft.com/v1.0/sites/{siteId}/pages with the edited canvasLayout parameter and all required fields in the body.

Let me know if that works for you.

Hi @mkm17  thanks for your message,
but I don't think it's going to work, bacause the page template contains some custom webparts and they are not allowed as far as I'm reading.

Hi @roberAlb I quickly tested the same method with my custom SPFx web part, and it successfully created a new page with non-standard Microsoft web parts.


Bracket indentation settings.

Microsoft Visual Studio Code has become one of my favorite code editors to use, but transitioning from Visual Studio to Code as a game developer is confusing since the indentation changes.
In Visual Studio, bracket indentation is like this when coding in C# for Unity:
if (smth)
{
    // code here
}

but in VS Code, bracket indentation changes to this:
if (smth) {
    // code here
}

This can be confusing for me since I was always used to the indentation Visual Studio had, and in massive programs, it can be hard to trace where or what is in functions, or if statements. I have tried every single setting related to this and every solution I could find online but nothing seems to work, so I guess it is not a feature. One of the main reasons I want to use VS Code is because it is extremely customizable, so adding this as a feature would really help me and potentially other newcomers moving to VS Code.

Hi @Georgios1999 thank you for posting this issue. When you mention that bracket indentation changes to the second example, what do you mean by that? Do you mean that when you format the code the indentation is as in the second example? Could you please include a screen recording of the issue you are experiencing?

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Unable to install version 2.7

Hi,
I’m trying to install the latest version of creditpy, but I get an error when it tries to install gap-stat 2.0.3 (regarding its build dependencies).
It looks like gap-stat 2.0.3 requires an older version of Python, but creditpy requires a newer version of Python, so I was just wondering if there was a way around this?
Thanks

Hi
gap-stat 2.0.3 should work with Python 3.9 version, can you please try 3.9 version.
Thanks

Can you please share the relevant error message with me? Also in the meantime please try to install another version of gap-stat, please note that I use 3.9 version.

Hi, thank you, I've managed to install it now!


Production: Fix deployment script

Deployment script needs changing for storage links

In the workflow laravel release script cp -r $LATEST_RELEASE/storage $NEW_DIR/storage/ should become cp -r $LATEST_RELEASE/storage $NEW_DIR/

Closed in #269


Write size function

Create a new branch for developing size function

Added the image_size function and hence, closing the issue.


\yii\db\Expression error on gps point update

What steps will reproduce the problem?
a) create column with type: "point"
b) add spatial index to it
c) at model file create public attributes: 'trackLong and trackLat'
d) at model file update beforeSave function to:
 public function beforeSave($insert){
        if (parent::beforeSave($insert)){
            //if ($insert) {
                $this -> trackCoordinates = new \yii\db\Expression("PointFromText(:point)", [
                    ':point'=>'POINT('. $this -> trackLong.' '.$this -> trackLat.')'
                ]);
           // }else{
         //   }
            return true;
        }
        return false;
    }

d) try insert/update data from controller
What do you get instead?
On insert (first record) - everything works as it should
On update got error: "Malformed UTF-8 characters, possibly incorrectly encoded."
What is the expected result?
Update without error about utf-8
Additional info

Q
A

Tested on Yii version
2.0.19 and 2.0.20

PHP version
7.2.18

Operating system
Debian 9

Database
15.1 / 10.3.14-MariaDB

Column type
point

Column index
spatial

I added more information

Yii isn't dealing with encoding conversion. That means that one of values you supply contains incorrect unicode sequences.

Strange, because the problem is only with update when i use Expression for mysql column with POINT type also without data  (using empty POINT(0 0)).
$model -> trackCoordinates = new \yii\db\Expression("PointFromText(:point)", [
':point'=>'POINT(0 0)'
]);
    public function actionUpdate($id){
        $model = GpsPoints::find()-> select([
            '*',
            'ST_X(trackCoordinates) as trackLong',
            'ST_Y(trackCoordinates) as trackLat'
        ]) ->where(['id' => $id]) ->one();
        $container = ['points-grid'];
        if (Yii::$app->request->post()) {
            $post = yii::$app -> request -> post();
            Yii::$app -> response -> format = \yii\web\Response::FORMAT_JSON;
            if ($model->load($post)){
                $connection = \Yii::$app -> db;
                $transaction = $connection -> beginTransaction();
                try{
                    $model -> trackCoordinates = new \yii\db\Expression("PointFromText(:point)", [
                        ':point'=>'POINT(0 0)'
                    ]);
                    if ($model -> save()){
                        $transaction  -> commit();

Can you save it using SQL?

Yes, from phpmyadmin insert/update works

If you Google the error, Malformed UTF-8 characters, possibly incorrectly encoded. it all comes back as a JSON encoding issue so I'm guessing there's an error with the response.
Does the code get to your transaction commit?
Is the DB updated even though it had an error?

@zuicis It seems to me that this is not related to the update event of the active record, but rather to the output generated by the response.
             // ...
            Yii::$app -> response -> format = \yii\web\Response::FORMAT_JSON;      // <----------------- enable json response
            if ($model->load($post)){
                $connection = \Yii::$app -> db;
                $transaction = $connection -> beginTransaction();
                try{
                    $model -> trackCoordinates = new \yii\db\Expression("PointFromText(:point)", [
                        ':point'=>'POINT(0 0)'
                    ]);
                    if ($model -> save()){
                        $transaction  -> commit();
              // <------- ... The exception is raised here?

Could you provide a stack trace for the exception?


Update manifests/p/Piriform/CCleaner/5.88/Piriform.CCleaner.locale.en-US.yaml

Fixes an issue where this package has multiple different monikers
Microsoft Reviewers: Open in CodeFlow

[Policy] Reset Labels


[v1.7] Backport Gluon Activations Fix #18370

@ciyongch I will also need this fix about Gluon Activations in Numpy.

@ciyongch Would you help review or merge this?

Hi @TaoLv , please help to merge the PR. Thanks.

@TaoLv @sxjscience Missing in v1.x branch. Intentional?

No
Get Outlook for iOShttps://aka.ms/o0ukef

From: Chaitanya Prakash Bapat<EMAIL_ADDRESS>Sent: Saturday, September 19, 2020 9:37:40 PM
To: apache/incubator-mxnet<EMAIL_ADDRESS>Cc: Xingjian SHI<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/incubator-mxnet] [v1.7] Backport Gluon Activations Fix #18370 (#18700)
@TaoLvhttps://github.com/TaoLv @sxjsciencehttps://github.com/sxjscience Missing in v1.x branch. Intentional?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/apache/incubator-mxnet/pull/18700#issuecomment-695712719, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABHQH3QXCZUETTM7A726OHTSGWBJJANCNFSM4OY3GNBA.

Can we cherry-pick this into v1.x then? So that we can ensure it gets into v1.8 as well?

@ChaiBapchya Yes, feel free to go ahead and pick it to v1.x.


Do not check for Pci in serial com

I do not know what i was thinking while
scanning for Pci port. A Pci port may hold
the communication and it is unclear to know
if it is an external hardware wallet.

ACK c370eed42dbf729008440ce0f0110c9820dae5a4


winpr_HexLogDump does not print when compiled using VS 2013

Describe the bug
I've compiled latest sources (as of Nov 10, 2020) using VS 2013 with almost all debugging enabled.
however when i execute wfreerdp, there are no buffer dumps printed
To Reproduce
wfreerdp /v:target /log-level:trace > trace.log
connect, login, exit
check trace.log - there are no buffer dumps whatsoever
Expected behavior
when wfreerdp compiled by VS 2019, buffers are printed.
Application details
I've attached debugger and it seems that TrioParseQualifiers is failing to parse formats, usually ending at
case QUALIFIER_FIXED_SIZE:
...
return TRIO_ERROR_RETURN(TRIO_EINVAL, offset);
apparently format definitions such as
#define PRIuz		"Iu"
are not compatible with trio.c

@lazyruss I´d update the compiler to something that is C99 compilant.
Of course, if you are at it pull requests are welcome ;)

@akallabeth its not vs version actually, its platform toolset. im forced to use v120 for compatibility reasons.
BTW, night builds from https://ci.freerdp.com/job/freerdp-nightly-windows exhibit same behavior.

@lazyruss I do understand. But this is a compiler/toolchain bug and the fallback library does not support that, so you´re back to square 1.


Combined text when looking at history in tmux

I have a strange issue with combined history text: https://imgur.com/a/XP0Cx
In the screenshots you can see there's some nonsensical commands, and pressing the up arrow clears that line... but this is a very strange issue that keeps happening. Is there a way to fix this?

Invalid $TERM. See https://github.com/junegunn/fzf#rendering-issues
Also https://github.com/tmux/tmux/blob/master/FAQ#L3-L13

Whelp, guess I need to read better. Thanks, I think that fixed it. I had it set to xterm-256color.


GWS.GMAIL.13.1v0.1 Policy Changes

Policy re-evaluation post 0.2 release
Possibly taking "to avoid unintentional replies and remind users to treat external messages with caution" out of the policy statement itself; the rationale covers the reasoning; and we want the policy statements to be as concise as possible.

Branch was created and changes were made. Waiting until the release of version 0.2 of the baselines before creating PR. Changes made in "gmail-changes-post-0.2"

PR has been created. Handled under PR 310. Moving this to in review.


[RNN-T] clarify gradient clipping status

This change is to minimize confusion in the reference implementation.
More details in https://github.com/mlcommons/training/pull/461

training polices update https://github.com/mlcommons/training_policies/pull/433


New database provider list

Fixes #472
Fixes #378

Added a new table on the index.md of providers containing relevant information
Updated various stale links and descriptions
Added jet provider to the list
Removes pages in our docs for all providers that are not built by EF team

@mairaw this PR removes some low-value pages from the docs. Ideally they would all redirect to providers/index.md. Anything I need to do?

@mairaw FWIW, it appears that the pages don't get removed, at least in the preview.

@mairaw I think I got how to do the redirects now, but feel free to review the master redirect files.


Pop-up toolbar for small groups

For small groups the height is too small to accommodate the three switcher icons. To address this issue we could create a pop-up toolbar showing up when the user hovers over the first switcher column.
Coming from #246.

A slightly different issue, but since this was referenced from issue #348 I think  it is also relevant to have popup toolbar at the top of the column instead stretching in following way:

Done by @katush.

@thinkh I think this issue was originally ment as pop-up for aggregation switcher column icons (column not toolbars), it just somehow got mixed up...

https://github.com/Caleydo/taggle/commit/06bb22d8e53cbce4dcb7d9a5f33bcf53bce4b6d5 hides them for now if there is not enough space

From a functionality point of view, it is fine because you can still click on "aggregation".  It is more a visual problem, because the cropped icon look a bit ugly.


Aaron broke US AP table

This table needs fixing as I changed the table structure in the .csv file to match what was sent (us-ap-catch.csv).  I figure that those AP report appendices should simply be what they send us (though I had to integrate 2 reports into 1 from what the US sent us ("thanks for that guys, we aren't already busy enough!")).  Sensei Chris the master table maker can probably do this in a minute or two, but I can get to it later if not beaten to it...  :)

I already did it this morning, see 80d4a4b970cdbada92a67c96e3c9925473a7efc3.

Sensei Kelli then... thanks!  Kelli, you can have the satisfaction of closing this one then.  And checking the box...  :)
This task was completed.

[ ]

darn, try again...

[ ] Kelli


support Cluster-API

support helper for cluster API

@helayoty
Can you elaborate on how you would like to see Cluster-API supported ? Is it:

Management cluster awareness
Managed cluster awareness
Ability to reach managed cluster automatically via management cluster
Etc

@helayoty
Is this helper func for custerctl like we have helper package for CRUD operations of k8s resources ?
If so I could explore this and come with initial draft PR.

@vladimirvivien  & @ShwethaKumbla
The idea for being able to support Cluster API is to spin up a kind cluster, install cluster API cli on it and then have a helper func for clustercli. Also, add a sample on how to use it with a couple of providers.

I just created a duplicate issue. I think the main thing is that we just register the types so that you can use klient on those types as well. Currently you get a problem with the scheme not knowing about those types.
I'm working on implementing that now but the test cluster itself needs to have those types installed too. Going through that now we we can know it works as intended.

I've got it nearly ready; I've gotten the types registered, clusterctl installed, etc. However, as I kept testing I hit https://github.com/kubernetes-sigs/cluster-api/issues/2450. I just found the issue just 1s ago and haven't had time to read it or implement the fix. PR probably tomorrow for this.

@helayoty can this be closed?


Update ratatui organization links

ratatui-org -> ratatui 🧀

Whoops - was just doing the same and already merged this.

no probs :)


Increase line-spacing (hardcoded, for building locally)

Hello,
First, thank you so much for this project! I've spent many weeks testing out most of the Neovim-GUI apps and terminal emulators, and I think I've finally found my peace with Neovim-Mac. It's the fastest Neovim client in my tests and has minimal memory footprint!
I've already removed font-smoothing from my local build, by changing these lines to false on src/font.mm:
    CGContextSetAllowsFontSmoothing(context.get(), false);
    CGContextSetShouldSmoothFonts(context.get(), false);

This gives me much crisper looking fonts. Without the extra boldening, I also get some extra spacing between the lines.
I would still like to increase the line-spacing, just a little bit more.
Is there a simple way where I could hardcode this somewhere in the source code and rebuild Neovim-Mac with it?
I tried to find something that would resemble this setting in the sources but no success so far. (I don't know the languages used for Neovim-Mac)
Thanks!

Hey, thanks for using the project! I'd definitely want to add a configurable line spacing option in the future, but for now you can hardcode it in: https://github.com/JaySandhu/neovim-mac/blob/f1050ff1138e154db645fef1277144fc97287fe0/src/NVGridView.mm#L403-L444
This patch should work:
diff --git a/src/NVGridView.mm b/src/NVGridView.mm
index c90b31e..3e2d094 100644
--- a/src/NVGridView.mm
+++ b/src/NVGridView.mm
@@ -401,26 +401,28 @@ - (void)setFrameSize:(NSSize)newSize {
 }

 - (void)setFont:(const font_family&)font {
     fontFamily = font;

     CGFloat leading = floor(font.leading() + 0.5);
     CGFloat descent = floor(font.descent() + 0.5);
     CGFloat ascent = floor(font.ascent() + 0.5);
+    CGFloat lineSpacingAbove = 0;
+    CGFloat lineSpacingBelow = 0;

-    CGFloat cellHeight = leading + descent + ascent;
+    CGFloat cellHeight = leading + descent + ascent + lineSpacingAbove + lineSpacingBelow;
     CGFloat cellWidth = floor(font.width() + 0.5);

     cellSize.x = cellWidth;
     cellSize.y = cellHeight;
     backingCellSize = [self convertSizeFromBacking:NSMakeSize(cellWidth, cellHeight)];

     baselineTranslate.x = 0;
-    baselineTranslate.y = ascent;
+    baselineTranslate.y = ascent + lineSpacingAbove;

     CGFloat underlinePos = font.underline_position();
     uint16_t lineThickness = floor(font.underline_thickness() + 0.5);
     int16_t underlineTranslate;

     if (underlinePos >= 0) {
         underlineTranslate = floor(underlinePos + 0.5);
     } else {

Just change the values of lineSpacingAbove and lineSpacingBelow variables to something you like.

Perfect! 🎉
Thank you so much @JaySandhu for the fast reply!


HTML

HTML
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
so that I can easily find and read information
Acceptance criteria
GIVEN a prework study guide website
WHEN I visit the website in my browser
THEN I see for boxes titled HTML, CSS, GIT, and JavaScript with associated notes listed

#CSS
User Story
As a bootcamp student
I want the prework notes to be structured on a webpage
so that I can easily find and read information
Acceptance Criteria
GIVEN a prework study guide website
WHEN I visit the website
THEN I see a dark blue header and footer, and four boxes with a shadow

JavaScript
User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I view the study guide
THEN I can see the four topics I learned listed along with a suggestion on what I should study first

README
User Story
As a boot camp student,
I want the prework notes to be structured on a webpage,
So that I can easily find and read the information.
Acceptance Criteria
GIVEN the prework project repository,
WHEN I view the README for the repository,
THEN I can see a professional README that provides information on the project and how to use it

Deployment
User Story
As a boot camp student,
I want the prework notes to be structured on a webpage,
So that I can easily find and read the information.
Acceptance Criteria
GIVEN a Prework Study Guide website,
WHEN I visit the website using the URL,
THEN I can access my website from any browser


Save the docker-compose.yml in configmap during builds?

Currently we save the .lagoon.yml into a configmap at the start and end of a build for use in troubleshooting
Should we do the same with the docker-compose.yml?

done!


Create Terraform scripts for self hosting Flagsmith on all main cloud providers

We should have Terraform scripts that we can provide for people to run to host Flagsmith on:

[ ] AWS
[ ] GCP
[ ] Azure

Are contributions welcome? I would be interested in taking care of AWS stack.

Are contributions welcome? I would be interested in taking care of AWS stack.

Yes that would be great!

Hello
I have begun working on some kind of Flagsmith starter kit :)
https://github.com/deltacodepl/flagsmith-aws-terraform
I plan on creating a small demo that showcases a simple landing page use case, perhaps someone find it useful.
...Regarding the infrastructure itself, do you have any specific plans or requirements for it?

Wow this is awesome thanks so much @deltacodepl - we'll take a look soon. Would you be interested in us bringing this into our organisation>

Sure, Ben
I'm open to suggestions about the 'Starter kit' project,
in the meantime I'm working on the Django landing page, so maybe I could use Flagsmith for some A/B testing.
Also, I've created the PR you requested.


Added AudioPlayer functionality.

Issue #16

I've merged your PR here https://github.com/vanderfox/alexa-skills-kit-java and added you as a collab (and the other PRS too!) if you want to give it a go!

Hey @jtgrassie I've found some bugs with your audio support I've logged here https://github.com/vanderfox/alexa-skills-kit-java/issues/1 the directives seem OK but the incoming audio requests are throwing exceptions with jackson errors in the sdk.

If you'd pulled and merged my head commit you wouldn't see these errors.
Working perfectly here in a live skill.

Nope there are bugs with sending audio response events, fixes are at https://github.com/vanderfox/alexa-skills-kit-java

@rvanderwerf You are not required to return anything from an audio request. The docs are pretty clear that if you do, you can only return certain audio playback messages.
https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/custom-audioplayer-interface-reference
"Note: When responding to AudioPlayer requests, you can only respond with AudioPlayer directives. The response cannot include any of the standard properties such as outputSpeech. In addition, some requests limit the directives you can use, such as not allowing Play. Sending a response with unsupported properties causes an error. See the request types below for the limits on each request. ."
The error you were getting is because you were sending an incorrect response to a audio player request. If you return nothing at all, they pass through fine.

That said, I do like the fact you added responses to audio requests. My only comment would be you should either validate the response is correct for the type of request or perform no check at all. Currently your commit has a mixture of creating its own response (for example on your playback stopped request) and on others returning a user created response. You should also check for null response from user code as the docs do not require a response, hence a user could chose to not respond to the request which would break your code, which is now enforcing a response.

Live testing confirms null/void responses with out a speechletresponse envelope cause Alexa to return SystemError calls. I've confirmed that an empty list returned makes it happy and it doesn't call back with errors, even if no directives are allowed to be returned like PlaybackStopped and SystemError. I tested this on a Tap. I think the docs should clarify this point, returned directives for audio calls are 'optional' however some sort of response envelope appears to be required even if it's empty.

If you don't return anything it works fine. If you return a response and without directives, you get system exception. Makes sense as the doc's state you can only return certain responses to audio requests, and these are all audio responses (hence would require directives). I advise you update your changes to allow for both a response to an audio request and no response at all (don't send anything back after receiving a request).
I think you got caught in a rabbit hole as you started returning a response, which was an invalid response based on the types of response allowed, and then assumed you have to return a response, which is not the case.

If you pulled and merged my head commit you wouldn't see this error. The
code is working perfectly for me in a live skill.
On Oct 24, 2016 11:39, "Ryan Vanderwerf"<EMAIL_ADDRESS>wrote:

Hey @jtgrassie https://github.com/jtgrassie I've found some bugs with
your audio support I've logged here vanderfox#1
https://github.com/vanderfox/alexa-skills-kit-java/issues/1 the
directives seem OK but the incoming audio requests are throwing exceptions
with jackson errors in the sdk.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/amzn/alexa-skills-kit-java/pull/18#issuecomment-255777576,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAzJT5Ry7OKiO4zQ4qRmMBRnrIx1WzKEks5q3NFBgaJpZM4KOLrZ
.

nope your code doesn't work man, I'm not sure why you can't see it:
} else if (speechletRequest instanceof PlaybackStartedRequest) {

       speechlet.onPlaybackStarted((PlaybackStartedRequest) speechletRequest);

   } else if (speechletRequest instanceof PlaybackStoppedRequest) {

       speechlet.onPlaybackStopped((PlaybackStoppedRequest) speechletRequest);

   } else if (speechletRequest instanceof PlaybackFinishedRequest) {

       speechlet.onPlaybackFinished((PlaybackFinishedRequest) speechletRequest);

   } else if (speechletRequest instanceof PlaybackNearlyFinishedRequest) {

       speechlet.onPlaybackNearlyFinished((PlaybackNearlyFinishedRequest) speechletRequest);

   } else if (speechletRequest instanceof PlaybackFailedRequest) {

       speechlet.onPlaybackFailed((PlaybackFailedRequest) speechletRequest);
   } else {

=======
// New Audio Interface

void onPlaybackStarted(PlaybackStartedRequest request) throws SpeechletException;
void onPlaybackFinished(PlaybackFinishedRequest request) throws SpeechletException;
void onPlaybackStopped(PlaybackStoppedRequest request) throws SpeechletException;
void onPlaybackNearlyFinished(PlaybackNearlyFinishedRequest request) throws SpeechletException;
void onPlaybackFailed(PlaybackFailedRequest request) throws SpeechletException;
}
View

=====
This is code from YOUR pull request above (you keep saying check your HEAD not your branch, but that's not what you submitted in your PR here because PRs come from a branch and it shows this clearly in GH). There null response returned on all of the audio objects and the user has no way to return a SpeechletResponse from a void return type call. This results in Alexa returning with a System Error call because YOUR code is returning null. Since there was no supported added for the SystemError call you would never see the errors happening unless you turned debug on your skill and checked cloudwatch logs. I've fixed that too in my repo as well so if something happens and alexa calls back with and error it is logged so you know it happened.

I was never returing a null response from user code. The return type was void for the requests and I never sent any response to a playback request, hence no error.
Your change adds a return type of SpeechletResponse so always does a return / call in response to every playback request. That is incorrect as per docs.  Not sure how I am not being clear on this.

Not sure where I should be asking this question, so I'll start here.  Just started trying out the sample skills and the SDK for AudioPlayer, so forgive me if I'm in the wrong place...
Is there a reason the AudioPlayer and PlaybackController interfaces include Request but don't include Session (e.g., onPlaybackNearlyFinished and onNextCommandIssued) like the standard Speechlet interface does?  Is the Session information available elsewhere when these methods are called?
I'd like to maintain audio information (like current song, next song, playlist info, etc.) to be used to determine the next song to queue.  Seems like Session is the logical place, but I don't see how to get to it from the above methods.
And according to the documentation, my Speechlet implementation object can't store the information because multiple objects and even multiple servers might support a single Session.
Or am I missing something?
If it makes sense to add them, should I just make the changes and submit a pull request?  Anyone know a good reason not to include Session  that would result in me creating extra interfaces (one with Request and Session and the (current) one with just Request)?
Thanks for the help,
phlepper

Closing this since the AudioPlayer interface is now supported in both the v1 and v2 SDKs.
@phlepper The event requests you listed are out of session requests and thus do not have access to session attributes. If you want to store information out of session, the v2 SDK introduces first class support for persisted attributes, as described here.


Can't make a POST request without a Content-Type

This seems related to https://github.com/thoughtbot/capybara-webkit/issues/260
When I make ajax requests with jQuery via capybara-webkit, something is setting the content type to application/json, text/javascript, */*; q=0.01.
The following is an example of an AJAX request gets that Content-Type.
js = "window.my_request = $.ajax({type: 'POST', url: 'http://<IP_ADDRESS>:5000/my/path', dataType: 'json', headers: {'X-HTTP-Method-Override': 'DELETE'}})\; window.my_var"
page.evaluate_script(js)

However, when I run the same JavaScript from within Chrome, Firefox, or Safari, none of them add a Content-Type.
I know that POST requests should usually have a Content-Type. However, when a POST request has no body, it doesn't need a Content-Type. In my case I am using a header to better support browsers that can't handle DELETE requests. It is normal for a DELETE request to have no body (and thus no Content-Type).
It looks like the latest from Qt seems to insist on a Content-Type too: https://qt.gitorious.org/qt/qtbase/source/70accc289a077b2d05250676ed33369eeaa7df49:src/network/access/qhttpnetworkrequest.cpp#L162
However that code doesn't explain the Content-Type I am seeing.

Hey @thoughtless, does adding the Content-Type header change the semantics of the request? Is it causing problems?

Unfortunately it has been too long since I opened the issue to remember exactly why I cared about this enough to write it up. I tried looking back through the old code, but I can't give you a firm answer. I suspect it had to do with Ruby on Rails executing slightly different controller code. It may also have been causing some issues with Rack Lint.
Since nobody else has commented on this issue in the last year, you might as well close it.

Thanks @thougtless!


Failed to find zone & current A records

Hi,  I was using oznu/cloudflare-ddns previously and was attempting to migrate to your solution, however I keep getting this error upon deployment
😞 Failed to find the zone of "sub.domain.com"
😞 Failed to retrieve the current A records of "sub.domain.com"
Here is my compose, basically verbatim of your example.
version: "3"
services:
  cloudflare-ddns:
    image: favonia/cloudflare-ddns:latest
    network_mode: host
    # This makes IPv6 easier; see below
    restart: always
    # Restart the updater after reboot
    cap_add:
      - SETUID # Capability to change user ID; needed for using PUID
      - SETGID # Capability to change group ID; needed for using PGID
    cap_drop:
      - all
      # Drop all other capabilities
    read_only: true
    # Make the container filesystem read-only
    security_opt:
      - no-new-privileges:true # Another protection to restrict superuser privileges
    environment:
      - PUID=1000 # Run the updater with user ID 1000
      - PGID=1000 # Run the updater with group ID 1000
      - CF_API_TOKEN=token # Your Cloudflare API token
      - DOMAINS=sub.domain.com, # Your domains (separated by commas)
      -<EMAIL_ADDRESS>      - IP6_PROVIDER=none

I'm guessing I'm just missing something somewhere, so any help or a point in the right direction is appreciated.
 Is domain.com the supposed DNS zone?

No trouble at all. Appreciate the response.  I figured it was user error on my part.
To answer your question, yes, "domain.com" the DNS zone as far as I know.

@tfer6 Ah, could it be some typo in the string domain.com? :sweat_smile:

I'm glad that it's resolved. Closing this now. :grin:

Sorry when I said "domain.com" I actually meant my own domain.  I was simply substituting sub.domain.com for my actual subdomain.myowndomain.com.  Just didn't want to actually share it what it actually is.  So my docker compose matches up with my actual domain and subdomain.
Am I just completely missing something as far as the instructions go?

@tfer6 You are good. Yes, I understand that domain.com is fake, and I used it in the same "fake" way. I just wanted to know whether you are referring to typos when you said "user error". :slightly_smiling_face:

@favonia Good.  Although I think I might have confused the situation.
I'm still actually experiencing the issue.  I believe I've followed all the instructions properly,  but still getting the error as described above.  I've also tried using just my domain without the subdomain and that got the same error.

To be honest I am not sure what has caused this. Are you using any internationalized domain names?

nope. Just a .com.   I spun up a copy of linuxserver/docker-ddclient and it seems to work fine as well.  Not sure how that and oznu/cloudflare-ddns work versus your solution.  I'll close this as I doubt I'll figure it out.

@tfer6 I think I figured it out... could you remove CF_ACCOUNT_ID? Sorry that I did not notice you set it earlier. It's NOT your email, and maybe the tool should check its format. Read Cloudflare's documentation if you are curious about what "account ID" is: https://developers.cloudflare.com/fundamentals/setup/find-account-and-zone-ids/

That got it to work!  Very strange.  I only added the CF_ACCOUNT_ID variable after getting a different error the first time I launched this.  I don't recall the exact error unfortunately, and it didn't pop up again.
Thanks again

@tfer6 Now the edge tag and the next release will show this message when CF_ACCOUNT_ID is not empty and when the search of DNS zones fails:

Double-check the value of CF_ACCOUNT_ID; in most cases, you can leave it blank

@tfer6 This is a head-up---recent versions (1.13.0+) will benefit from an update to your configuration, and I am on the mission to eliminate the old template from this universe. In case you or anyone is confused by how to update the configuration, here's the tl;dr:

Remove cap_add: ... completely.
Add user: "1000:1000" (or other IDs you want to use).
Remove PUID=... and PGID=... (optional, but why not).

For more information, please see the CHANGELOG and README.
PS: the new template works for older versions of the updater as well!


Touching up Redirects to New Site

Description
these changes let the checklist url redirect without a hard /, and don't forward the querystring with the redirect for the checklist.
List of proposed changes:

Changes to checklist redirects

What to test for/How to test
Changes are live on dev.
Additional Notes
They can go live when ready. No need to plan a release.

Is ? required at the end of https://www.canada.ca/en/services/retirement/quiz-main.html??

Yes, it removes the query strings from the tail end.