Video upload always fails

I get the error:
INIT
Media ID:<PHONE_NUMBER>288546307
APPEND
986718 of 986718 bytes uploaded
Upload chunks complete.
FINALIZE
{'media_id':<PHONE_NUMBER>288546307, 'media_id_string': '1263253181288546307', 'media_key': '7_1263253181288546307', 'size': 986718, 'expires_after_secs': 86400, 'processing_info': {'state': 'pending', 'check_after_secs': 1}}
Media processing status is pending
Checking after 1 seconds
STATUS
Media processing status is failed
I tried other videos and it does the same thing

INIT
Media ID:<PHONE_NUMBER>569826817
APPEND
4194304 of 4891459 bytes uploaded
APPEND
4891459 of 4891459 bytes uploaded
Upload chunks complete.
FINALIZE
{'media_id':<PHONE_NUMBER>569826817, 'media_id_string': '1347896365569826817', 'media_key': '7_1347896365569826817', 'size': 4891459, 'expires_after_secs': 86400, 'processing_info': {'state': 'pending', 'check_after_secs': 1}}
Media processing status is pending
Checking after 1 seconds
STATUS
Media processing status is failed

INIT
Media ID:<PHONE_NUMBER>569826817
APPEND
4194304 of 4891459 bytes uploaded
APPEND
4891459 of 4891459 bytes uploaded
Upload chunks complete.
FINALIZE
{'media_id':<PHONE_NUMBER>569826817, 'media_id_string': '1347896365569826817', 'media_key': '7_1347896365569826817', 'size': 4891459, 'expires_after_secs': 86400, 'processing_info': {'state': 'pending', 'check_after_secs': 1}}
Media processing status is pending
Checking after 1 seconds
STATUS
Media processing status is failed

if you are getting this then you are on the right track .............
So after long research, we (Me & My GURU @Mr. Rohit Vinayak sir) came to know that the video format is not up to the mark as per Twitter...
we are sending MP4 with less duration but getting media processing status is failed. so what is the format?
ANS
Twitter currently supports the following video formats: MOV and MP4 formats for mobile apps. The minimum Twitter video length is 0.5 second and the maximum Twitter video length is 2 minutes 20 seconds (140 seconds) Twitter supports the following web video formats: MP4 video uploads with H264 format and AAC audio
Use mobile record video you will get your answer

if you are getting this then you are on the right track .............
So after long research, we (Me & My GURU @Mr. Rohit Vinayak sir) came to know that the video format is not up to the mark as per Twitter...
we are sending MP4 with less duration but getting media processing status is failed. so what is the format?
ANS
Twitter currently supports the following video formats: MOV and MP4 formats for mobile apps. The minimum Twitter video length is 0.5 second and the maximum Twitter video length is 2 minutes 20 seconds (140 seconds) Twitter supports the following web video formats: MP4 video uploads with H264 format and AAC audio
Use mobile record video you will get your answer


Cannot configure multiple versions of balo in Ballerina.toml

Description:
ATM we can configure as below.
[dependencies]
"waruna/module1" = { version="1.0.0", path = "../TestProject1/target/balo/module1.balo"}
But if we cannot add multiple versions of same module with balo path.
suggestion
[dependencies]
"waruna/module1" = [ {version="1.0.0", path = "../TestProject1/target/balo/module1-v1.balo"},
{version="2.0.0", path = "../TestProject2/target/balo/module1-v2.balo"}]

Implementation has change and we do not support importing multiple versions.


Tags are not correctly format for root node

Tags are not correctly format for root node

https://codesandbox.io/s/mind-elixir-2-x-react-q9glxt?file=/index.js

same issue here, actually the style attribute to the root node is totally unhandled.


Crash on Worldgen (Feature placement)

Hey There :)
Got some crashes with Apotheosis while exploring the world.
Forge 36.2.8
Apotheoisi 4.8.2
crash-2021-10-16_10.44.26-server.txt
crash-2021-10-16_10.27.10-server.txt
Cheers

Both of these would imply the structure that the second variant of the boss dungeon uses is not loaded, which would be impossible under default configuration.  You likely have something else interfering.


Feature: Add safe division in reciprocal operation

Please make sure that this is a feature request. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:feature_template
System information

TensorFlow version (you are using): 1.14 / 2.0 alpha
Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state.
Hello,
I wanted to add a feature to the tf.math.reciprocal operation to optionally perform "safe" reciprocal. If the input is zero, it returns zero instead of NaN. I found this feature useful when implementing one of the recent PR. This is similar to the tf.math.div_no_nan operation/PR.
Should this feature request be approved, I can either add an optional argument to the current tf.math.reciprocal() operation which indicates whether to perform safe division. The default for this would be False, thus retaining the current interface. OR I can create a new tf.math operation called tf.math.reciprocal_no_nan() similar to the safe division op. Would love inputs on this!
Will this change the current api? How?
Depending on the two options laid out above, the first option would change the current API slightly, while the second option adds a new operation.
Who will benefit with this feature?
Any researcher designing algorithms involving reciprocals as well as engineers/developers who would otherwise write wrapper functions to perform safe reciprocals.
Any Other info.

tf.math.reciprocal_no_nan is probably the best name for this

Thank you, @alextp I'll raise a PR for this soon! Would it be possible to assign this issue to me?

@alextp  I am planning to implement a full-fledged operation (in C++, will be part of gen_math_ops) and add a wrapper on Python api similar to the other core math ops. This would have better performance and could perhaps be easily extended to other (non-python) APIs like tfjs in the future.
The other option is to try and implement it with existing TF ops in just the python api, and so only affecting the Python API.  While easier, it seems more restrictive.
Should I continue with the first option or switch to the second one? Just wanted to check before I start.

To implement a full-fledged C++ operation you probably want to change eigen to get this to vectorize well (I think? Let's ask @rmlarsen).
If you don't you won't gain much over implementing this with existing ops, which is what I mildly prefer.

@kmh4321 This issue can be closed now!?

@kshitij12345 sure, I  haven't taken a look at this in a while so you can close it for now.

Closing this issue since the associated PR has been merged and the feature has been implemented. Thanks!


How to connect to SAP HANA using Data Provider for Microsoft .NET

Hi,
how does the connection string have to look like when trying to connect to a SAP Hana Trial Instance in the SAP Cloud?
Thanks,
Oliver

Closing old issue. Not HANA express.

Reassigning to Chris.

Thanks for your feedback. We are improving the tutorials and will take your comments into consideration when making the updates.

Thanks, but that doesn't help me right now :-/


Empty table cells in markdown when using additionalProperties

I use additionalProperties to define a dictionary as suggested here.
When the dictionary value is object, the resulting markdown will have rows for summaryAsHTML , descriptionAsHTML, generatedExample with empty cells in Type and Description columns. This somehow breaks the HTML parsing of the table.
openapi: 3.0.0

info:
  description: Test specs
  version: "0.0.0"
  title: Test specs

paths:
  /test:
    get:
      summary: retrieves a dictionary
      responses:
        '200':
          description: success response
          content:
            application/json:
              schema:
                type: object
                additionalProperties:
                  type: object
                  properties:
                    x:
                      type: string
                    y:
                      type: string

# Test specs

Test specs

## Table of Contents

* [Paths](#paths)
  - [`GET` /test](#op-get-test)
* [Schemas](#schemas)

## Paths

### `GET` /test

> retrieves a dictionary

#### Responses

##### ▶ 200 - success response

###### Headers
_No headers specified_

###### application/json

      Name
      Type
      Description
      Accepted values

        Response

          object

        Any

        Response.x

          string

        Any

        Response.y

          string

        Any

        Response.summaryAsHTML

        Any

        Response.descriptionAsHTML

        Any

        Response.generatedExample

        Any

Any suggestions?

Sorry, for some reason I missed this issue.
The reason is that the template doesn't iterate over additionalProperties and thus you don't get them. It would be just a matter of fixing the Markdown template.


[Windows] Issue with ReadinessProbe suddenly failing on all pods of one node

We have the problem, that on our Windows 2019 Datacenter Servers several times a day all pods of a node show as failing the ReadinessProbe.
This fixes itself after 5-10 minutes, but continues to happen over and over.
All our windows nodes on all our clusters have this problem. No linux nodes have the same behaviour.
We see the Pods becomeing 0/1 suddenly. The Node is then unreachable via SSH, but can be pinged and a kubectl exec on a specific Pod running on that nod is working without issues.
After looking for a few hours we could not find the logs to troubleshoot this further.
Expected Behavior

ReadinessProbes do not fail on Windows nodes without any reason.
Current Behavior
ReadinessProbes fail several times a day on all pods of a single Windows node, but fix themselves after a few minutes.

Possible Solution

Steps to Reproduce (for bugs)

Can't be actively reproduced. But happens quite often.
Context

Running Windows Pods on Windows Nodes with ReadinessProbe
Your Environment

Calico version 3.20.3 on WIndows (Same version on Linux Nodes)
Orchestrator version (e.g. kubernetes, mesos, rkt): k8s v1.22.8
Operating System and version: Windows Server 2019 Datacenter 10.0.17763.2305 / ContainerD 1.5.8

Attached several outputs to help out (kube-proxy & kubelet error log from respective node & some output from kubectl and directly from the Node).
errors.txt
kube-proxy.err.log
kube-proxy.err-20220607T134535.340.log
kubelet.err.log

@song-jiang @lmm

@eyenx According to errors.txt (see below), who owns the IP:Port of <IP_ADDRESS>:80?
$ kubectl get events -n app-int-app5
LAST SEEN   TYPE      REASON      OBJECT                    MESSAGE
42m         Warning   Unhealthy   pod/app5-cc5b588cb-t67xw   Readiness probe failed: Get "http://<IP_ADDRESS>:80/server/status": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

But it is reachable
Do you mean when you try to access <IP_ADDRESS>:80 from the Windows node which hosts the pod by yourself, it works fine?

this I cannot check as SSH is not working on theu node at that time. but kubectl exec works and the Pod is reachable. As mentioned above this happens for 5-10 minutes on all pod of a node at the same time. after that timeframe the pods go back to being healthy and the Node is again reachable via ssh.
I am assuming something with calico to be not okay.

It is rare Calico can affect traffic in this pattern. Normally if there is an issue with Calico, either the traffic is allowed or denied.

We managed to get a local Admin Account so we could access the node during that timeframe without SSH. Curl to the Pod gives back a timeout.
So you suggest Calico can't be the culprit in this case?

What's the interval for this happening?  Sometimes that can hint at the cause.
Without knowing the interval, I'd be guessing things like DHCP renewal.  Check windows logs to find out what's going on around the time it happens.

Hi,
@song-jiang
i am working with @eyenx at this issue. attached you can find the logs from c:\CalicoWindows\logs directory.
calico-felix-20220613T211108.090.log
calico-node.err.log
calico-node.log
calico-felix.err.log
calico-felix.log
calico-felix-20220513T164524.844.log
calico-felix-20220513T164524.846.log
calico-felix-20220601T194316.181.log
The issue happend on this node today (16 June) arround 12:55. The logs are from arround 12:58.
@lwr20 - i can't tell the interval, but its quite often (i would guess every 2-3 hours.). DHCP is not the case - as ping is possible during this time and the server has a fixxed ip address, no dhcp.

Some more informations about this issue - as it is still open.
As told before, no ssh connection is possible during the time of the issue. But port 22 is always open (tested with nc).
Also logged in connections via SSH are working. So i was logged in on one server and had a loop running. the loop curled to a website in the internet and did a ping.
when the issue came up, ping was working while curl faiuled (unable to connect to the remote server).
during the time of the issue, i could see that the node sends packages to the pods (i guess readynessprobe).
in netstat i could see (for all the pods):
<IP_ADDRESS>:65103     <IP_ADDRESS>:80       SYN_SENT
i dont know if it helps, but at least some more informations

@eyenx @sil-zh are you still facing this issue? Calico v3.20 is old at this point, were you able to upgrade to a more recent version? Hopefully that's fixed your issues, but checking in before closing this...

Closing as stale unless there is new info.


placement extensible scheduling proposal

Ref: https://github.com/open-cluster-management-io/enhancements/issues/32
Signed-off-by: haoqing0110<EMAIL_ADDRESS>

/assign @qiujian16 @elgnay

@yue9944882 (subscribing the thread to so as to track it in the "mentioned" list)

The API break needs to be handled before an API PR can be merged.  I'm not really clear on why its desireable to suspend a placement entirely when some of its AddOnPlacementScores aren't available.
I think we can reasonably expect that the AddOnPlacementScoring controller is created before the placement attempts to use it.  And if it isn't available, then scoring at zero (no priority) seems like a more robust failure mode.  The scoring controllers themselves could then choose a balance of "freshness" without fear that stale choices will cause the placement to stop placing.

@deads2k yes, thanks for your suggestion, and I update the proposal. For the API changes, claim the Name will be deprecated. For the unavailable scores, giving a default score of 0 makes sense for current use cases, will keep the logic simple until we know more use cases and feedback.

The behavior looks good to me.  The API change is significant, but if @qiujian16 is ok with it, I think it's ok during alpha.
/assign @qiujian16
/approve

/lgtm
thanks, I think it is a great extension.
I am ok with the api break change in alpha given we have not had many consumers on placement plugin yet.


JSII9000 Encountered use of module that is not declared in "dependencies" or "peerDependencies"

Describe the bug
I am using a package @XXXX as a dependency that is not prepared to use JSII so I added it to bundledDependencies.
import { RefTypes } from '@XXXX';
The Error:
src/builder/validator.ts:15:20 - error JSII9000: Encountered use of module that is not declared in "dependencies" or "peerDependencies": "@XXXX"
Expected Behavior
Should not stop the JSII process because of this because the package is in dependencies.
Current Behavior
src/builder/validator.ts:15:20 - error JSII9000: Encountered use of module that is not declared in "dependencies" or "peerDependencies": "@XXXX"
Reproduction Steps
npm run build
Possible Solution
No response
Additional Information/Context
JSII Config
"jsii": {
"outdir": "dist",
"versionFormat": "full",
"targets": {
"python": {
"distName": "X",
"module": "X"
},
"java": {
"package": "X",
"maven": {
"groupId": "X",
"artifactId": "X"
}
}
},
"tsc": {
"outDir": "dist-tsc",
"rootDir": "src"
}
},
SDK version used
5.3.7 (build e512679), typescript 5.3.3
Environment details (OS name and version, etc.)
MAC Node.js v18.12.0

Are you exporting anything from that module? This error will happen if JSii is trying to translate a type from the bundled package (since it can't).

@gocapa can you provide the complete package.json? Any deps in bundledDependencies must also be in dependencies. Plus like @TimothyJones said, you likely cannot reexport anything from bundled deps.

No reproduction. Closing.
Like stated before, bundled deps also need to be defined as dependencies.


Logos Not Showing Up in v1.4.0

If I update everything including both the update for Extra Metadata Extension and ModernUI, the logos in specifically the new [v1.4.0] ModernUI no longer work, but they do still work in other themes. If I update everything excluding the new [v1.4.0] ModernUI the logos still work in the ModernUI theme. Something appears to be incompatible between the two updates.

Have you ticked that you want logos in the DKG Theme Modifier and then clicked the Apply Changes button? That needs to be done each time the theme is updated as it goes back to default settings.

As there has been no replay on this I'm closing this issue.


Translate 1 file to zh - tsconfig.json

Files:

tsconfig.json.md

Thanks for the contribution! I'll take a look soon.

@laosb PTAL.

@Kingwl 😄 Thanks for code review. I've corrected all the questions.

Looks good. But I think we may need one more eye to review again.

Seems there's no more eyes.
It's looks basically good. I think we could move on.
So
LGTM.

I can always ask for more Chinese readers on twitter?

I'm planning seeking for help at some Chinese social network too.


Compute min and max of numerical columns over a Dataset.

This change is

Yay! The ball is rolling. Left a few comments, some of them for my own understanding

Reviewed 1 of 5 files at r1.
Review status: 1 of 5 files reviewed at latest revision, 16 unresolved discussions, some commit checks failed.

src/contrib/data/dataset.ts, line 50 at r1 (raw file):

  abstract async getStream(): Promise<DataStream<DatasetElement>>;

  // TODO(soergel): plumb this through, to distinguish e.g. webcam from files

you can elaborate a bit what this means? Since this is user facing, let's move the elaborate jsdocs from internal methods to here.

src/contrib/data/statistics.ts, line 24 at r1 (raw file):

import {ElementArray} from './types';

// TODO(soergel) This whole file is the barest stopgap.

for my own understanding, can you clarify what this means?

src/contrib/data/statistics.ts, line 26 at r1 (raw file):

// TODO(soergel) This whole file is the barest stopgap.

export class NumericColumnStatistics {

no need for this to be a class, classes have overhead. Make it an interface, of simply a type:
export type NumericColumnStatistics = {
  min: number;
  max: number;
}

do the pos_inf/neg_inf initialization inside the method that computes min/max

src/contrib/data/statistics.ts, line 40 at r1 (raw file):

 *
 * @param min the lower bound of the inputs, which should be mapped to 0.
 * @param min the upper bound of the inputs, which should be mapped to 1

typo: s/@param min/@param max/

src/contrib/data/statistics.ts, line 52 at r1 (raw file):

    } else {
      if (value instanceof Tensor) {
        const result = value.sub(minTensor).div(maxTensor);

I think it should be (x - min)/(max - min) (that is, divide by the range=max-min). Here and below. Also update tests to reflect this, with cases where min != 0

src/contrib/data/statistics.ts, line 54 at r1 (raw file):

        const result = value.sub(minTensor).div(maxTensor);
        return result;
      } else if (value instanceof Array) {

for my own understanding, in which case (what part of the pipeline) would a value be instanceof Array and not an ElementArray

src/contrib/data/statistics.ts, line 66 at r1 (raw file):

 * Gathers statistics from a Dataset (or optionally from a sample).
 *
 * Currently we gather only the minimum and maximum value from each numeric

let's remove this sentence and minimize things we need to keep in sync as code progresses

src/contrib/data/statistics.ts, line 76 at r1 (raw file):

 * some way.
 *
 * A truly random shuffle of the stream would of course solve this

Let's move all of this jsdoc to the user-facing public API, which is dataset.getStats() i believe

src/contrib/data/statistics.ts, line 88 at r1 (raw file):

 * @param shuffleSize The size of the shuffle window to use, if any.  (Not
 *   recommended, as described above).
 * @param sampleSize The number of examples to take from the (possibly shuffled)

switch @param shuffleSize and sampleSize to align with the order in the method

src/contrib/data/statistics.ts, line 93 at r1 (raw file):

export async function computeStatistics(
    dataset: Dataset, sampleSize?: number,
    shuffleSize?: number): Promise<DatasetStatistics> {

how do you feel renaming this to shuffleWindowSize? or shuffleBufferSize?

src/contrib/data/statistics.ts, line 120 at r1 (raw file):

          recordMax = value.reduce((a, b) => Math.max(a, b));
          recordMin = value.reduce((a, b) => Math.min(a, b));
        } else {

let's make sure this is actually a numeric value. Check for number !isNaN(value) && isFinite(value) otherwise throw an error in the last else for unsupported data type

src/contrib/data/statistics_test.ts, line 17 at r1 (raw file):

 * =============================================================================
 */
/*

why are tests commented out?

src/contrib/data/streams/data_stream.ts, line 122 at r1 (raw file):

  async resolveFully(): Promise<void> {
    let x = await this.next();
    while (x !== undefined) {

do != null here and elsewhere (captures both null and undefined)  - simplifies the world. we do this everywhere in our codebase and it would be good to be consistent.

src/contrib/data/streams/data_stream.ts, line 159 at r1 (raw file):

   */
  async forEach(f: (value: T) => void|Promise<void>): Promise<void> {
    return this.map(f).resolveFully();

This uses MapStream, which seems to push the result of f() into an outputQueue, which means we'll have an array with Nundefinedvalues in it, is there a more efficient thing we can do than map.

src/contrib/data/streams/data_stream.ts, line 411 at r1 (raw file):

    const item = await this.upstream.next();
    if (item == null) {
      return false;

Not in this PR. It's unnatural to skip null elements when mapping an array. We should be consistent with JS's map and Python's map behavior and pass nulls to transform as well - maybe the user wants to map null values to some reasonable defaults, or count the number of null values.

src/contrib/data/streams/data_stream.ts, line 417 at r1 (raw file):

      mapped = await mapped;
    }
    this.outputQueue.push(mapped);

Sorry if this is a silly question: why do MapStream or FilterStream need a queue? Not for this PR, but I think we should simplify the world and remove QueueStream and move that logic to BatchStream. The logic is so simple that is not worth distributing in a separate class, and makes it hard to follow. E.g. I thought outputQueue never gets cleaned, until I realized that this class extends QueueStream whose next() method does the unshift on outputQueue.

Comments from Reviewable

Yep, now that the basic framework is in place, there are lots of features to add!
Thanks for the review!  And sorry some parts were premature before. :S

Review status: 0 of 5 files reviewed at latest revision, 16 unresolved discussions, some commit checks failed.

src/contrib/data/dataset.ts, line 50 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
you can elaborate a bit what this means? Since this is user facing, let's move the elaborate jsdocs from internal methods to here.

Done-- elaborated this comment, and also moved the (unrelated) jsdocs on the following method.

src/contrib/data/statistics.ts, line 24 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
for my own understanding, can you clarify what this means?

Expanded the comment.

src/contrib/data/statistics.ts, line 26 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
no need for this to be a class, classes have overhead. Make it an interface, of simply a type:
export type NumericColumnStatistics = {
  min: number;
  max: number;
}

do the pos_inf/neg_inf initialization inside the method that computes min/max

Done.

src/contrib/data/statistics.ts, line 40 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
typo: s/@param min/@param max/

Done.

src/contrib/data/statistics.ts, line 52 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
I think it should be (x - min)/(max - min) (that is, divide by the range=max-min). Here and below. Also update tests to reflect this, with cases where min != 0

Oh wow, that is so embarrassing.  I must have been really tired when I wrote that, and then the tests assumed min==0 to boot.  Gah!  All fixed now.

src/contrib/data/statistics.ts, line 54 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
for my own understanding, in which case (what part of the pipeline) would a value be instanceof Array and not an ElementArray

ElementArray is a union type: number|number[]|Tensor|string, so this covers the number[] case.

src/contrib/data/statistics.ts, line 66 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
let's remove this sentence and minimize things we need to keep in sync as code progresses

Done.

src/contrib/data/statistics.ts, line 76 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
Let's move all of this jsdoc to the user-facing public API, which is dataset.getStats() i believe

Done.

src/contrib/data/statistics.ts, line 88 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
switch @param shuffleSize and sampleSize to align with the order in the method

Oops, i went back and forth on this and left it inconsistent; it was actually the method argument order that was switched.  My logic was that people might want sampleSize often, and shuffleBufferSize rarely, so this order allows getStats(1000) as opposed to getStats(undefined, 1000) etc.  Fixed throughout.

src/contrib/data/statistics.ts, line 93 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
how do you feel renaming this to shuffleWindowSize? or shuffleBufferSize?

Done.

src/contrib/data/statistics.ts, line 120 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
let's make sure this is actually a numeric value. Check for number !isNaN(value) && isFinite(value) otherwise throw an error in the last else for unsupported data type

Done.

src/contrib/data/statistics_test.ts, line 17 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
why are tests commented out?

Oops.  Git branch juggling / debugging artifact, sorry!
The tests pass--mostly.  I consistently get numerical errors on these tests and others, but only under certain WEBGL_ENVS.  I assume this is due to bugs in my (~4 year old) GPU (& drivers etc.).  Let's see if the Travis test passes.

src/contrib/data/streams/data_stream.ts, line 122 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
do != null here and elsewhere (captures both null and undefined)  - simplifies the world. we do this everywhere in our codebase and it would be good to be consistent.

Done.  There are a few more cases in unrelated code to fix in a separate CL.

src/contrib/data/streams/data_stream.ts, line 159 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
This uses MapStream, which seems to push the result of f() into an outputQueue, which means we'll have an array with Nundefinedvalues in it, is there a more efficient thing we can do than map.

If I don't use a queue for MapStream, then this issue evaporates.  So, let's punt this question to another PR where I'll try that.

src/contrib/data/streams/data_stream.ts, line 411 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
Not in this PR. It's unnatural to skip null elements when mapping an array. We should be consistent with JS's map and Python's map behavior and pass nulls to transform as well - maybe the user wants to map null values to some reasonable defaults, or count the number of null values.

The DataStream API represents the end of the stream as 'undefined'.  This mechanism avoids requiring a separate call for hasNext() (or isEmpty(), etc.), which would produce race conditions.  Since we don't want to distinguish between null and undefined, it follows that a stream cannot contain null values, so they cannot be transformed or counted.  Happy to discuss!
Anyway, given that, here we're saying that if the upstream stream is exhausted, then nothing can be pumped.

src/contrib/data/streams/data_stream.ts, line 417 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
Sorry if this is a silly question: why do MapStream or FilterStream need a queue? Not for this PR, but I think we should simplify the world and remove QueueStream and move that logic to BatchStream. The logic is so simple that is not worth distributing in a separate class, and makes it hard to follow. E.g. I thought outputQueue never gets cleaned, until I realized that this class extends QueueStream whose next() method does the unshift on outputQueue.

Good point-- I think map and filter probably don't need queues.  That approach was left over from my thinking about 1-to-many mappers, which would need a queue, but which I haven't implemented (map() is 1-to-1 so far).  I'll revisit this question in another PR.

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 16 unresolved discussions, some commit checks failed.

src/contrib/data/statistics_test.ts, line 17 at r1 (raw file):
Previously, davidsoergel (David Soergel) wrote…
Oops.  Git branch juggling / debugging artifact, sorry!
The tests pass--mostly.  I consistently get numerical errors on these tests and others, but only under certain WEBGL_ENVS.  I assume this is due to bugs in my (~4 year old) GPU (& drivers etc.).  Let's see if the Travis test passes.

Hm, the tests still didn't run.  How can I cause $BROWSERSTACK_KEY to be set during the Travis build?

Comments from Reviewable

 Thanks for adding the new envs! Left a few responses, but no action needed for this PR. Feel free to merge.

Reviewed 1 of 4 files at r2.
Review status: 1 of 6 files reviewed at latest revision, 2 unresolved discussions, some commit checks failed.

src/contrib/data/statistics_test.ts, line 17 at r1 (raw file):
Previously, davidsoergel (David Soergel) wrote…
Hm, the tests still didn't run.  How can I cause $BROWSERSTACK_KEY to be set during the Travis build?

Talked offline. Future PRs should originate from branches in the dljs repo. As for failing tests, use webgl_float_envs for all tests in tf.contrib.data (we can change that later when we fix float packing)

src/contrib/data/streams/data_stream.ts, line 411 at r1 (raw file):
Previously, davidsoergel (David Soergel) wrote…
The DataStream API represents the end of the stream as 'undefined'.  This mechanism avoids requiring a separate call for hasNext() (or isEmpty(), etc.), which would produce race conditions.  Since we don't want to distinguish between null and undefined, it follows that a stream cannot contain null values, so they cannot be transformed or counted.  Happy to discuss!
Anyway, given that, here we're saying that if the upstream stream is exhausted, then nothing can be pumped.

Not for this PR. While js does have asynchronous behavior, it is single threaded, so we won't have race conditions. I feel strongly we should align with python/js behavior when it comes to .map and .filter. This means we should add hasNext() and avoid encoding null/undefined as end of stream. We want for users to be able to count missing values early in the pipeline when things are still primitive values.

Comments from Reviewable

Thanks!

Review status: 1 of 6 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

src/contrib/data/streams/data_stream.ts, line 411 at r1 (raw file):
Previously, dsmilkov (Daniel Smilkov) wrote…
Not for this PR. While js does have asynchronous behavior, it is single threaded, so we won't have race conditions. I feel strongly we should align with python/js behavior when it comes to .map and .filter. This means we should add hasNext() and avoid encoding null/undefined as end of stream. We want for users to be able to count missing values early in the pipeline when things are still primitive values.

I did have both next() and hasNext() originally, but had to switch to this approach to avoid resulting bugs.  The problem is that we need next() (and hasNext()) to be async, so there's no guarantee that the stream state would not change between the hasNext() resolution and the next() resolution.  There's also no guarantee that the next() promises resolve in the order that they were created (especially given nested Promises).  The consequences that I observed were things like a) the order of elements in the stream could get scrambled, b) the stream could terminate prematurely, c) hasNext() might have no way of knowing what to return until all prior next() promises were resolved, etc.  So basically I think the standard JS map() behavior doesn't apply cleanly in the async case.  Happy to discuss other solutions (e.g., maybe taking the ChainedStream approach throughout would fix this).

Comments from Reviewable


Hello，would mind telling us the CARLA version you used and the required version of other libraries？ Thank you

Hi,
We have used CARLA 0.9.13. The required versions of packages are specified in the Dockerfile. However, please take into account that this is a part of the bigger project to bring the more realistic pedestrian movements to CARLA. It isn't intended for fully standalone use. Please see the main project README.md for details.
You won't be able to build this docker without the other components.


Help with config errors

Hello!
I'm trying to use Mopidy-youtube without success.
Followed instructions at github project page but get the error below after running "sudo mopidyctl config":
=================
[youtube]
enabled = false  ; Extension disabled due to config errors.
youtube_api_key =
threads_max = 16
search_results = 15
playlist_max_videos = 20
api_enabled = false
autoplay_enabled = false
strict_autoplay = false
max_autoplay_length = 600
max_degrees_of_separation = 3
I got the following warning after building Mopidy-youtube:
"ytmusicapi 0.19.3 has requirement requests>=2.22, but you'll have requests 2.21.0 which is incompatible."
But that doesn't seem to be the problem.
How can I try to solve this?

@natumbri:
Solved.
My fault. Sorry.
Extension enabled after I removed "allow_cache" config option.


Synthesis failed for nodejs-speech

Hello! Autosynth couldn't regenerate nodejs-speech. :broken_heart:
Here's the output from running synth.py:
b'ng Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:41:41,670 synthtool [DEBUG] > Generating code for: google/cloud/speech/v1.\nDEBUG:synthtool:Generating code for: google/cloud/speech/v1.\n2020-06-05 04:41:42,549 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmp_vuy8n8e is empty.\n2020-06-05 04:41:42,588 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:41:42,589 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 04:41:42,594 autosynth [DEBUG] > Running: git checkout autosynth-self\nSwitched to branch \'autosynth-self\'\n2020-06-05 04:41:42,600 autosynth [ERROR] > Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n2020-06-05 04:41:42,741 autosynth [DEBUG] > Running: git checkout 3af749c8f705d92dba12169ba97c176ff3006a31\nNote: checking out \'3af749c8f705d92dba12169ba97c176ff3006a31\'.\n\nYou are in \'detached HEAD\' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n  git checkout -b <new-branch-name>\n\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 04:41:42,746 autosynth [DEBUG] > Running: git checkout cd804bab06e46dd1a4f16c32155fd3cddb931b52\nPrevious HEAD position was a3a0bf0f BREAKING_CHANGE: Removing TimeSeriesQueryLanguageCondition as an alert condition type. The condition type is unsupported and unused. It was originally added for the Monitoring Query Language Alpha feature. refactor!: Drop support for TimeSeriesQueryLanguageCondition as an alert condition type.\nHEAD is now at cd804bab docs: cleaned docs for the Agents service and resource.\n2020-06-05 04:41:42,816 autosynth [DEBUG] > Running: git checkout d53a5b45c46920932dbe7d0a95e10d8b58933dae\nPrevious HEAD position was be74d3e build: do not fail builds on codecov errors (#528)\nHEAD is now at d53a5b4 docs: improve README (#600)\n2020-06-05 04:41:42,832 autosynth [DEBUG] > Running: git branch -f autosynth-128\n2020-06-05 04:41:42,836 autosynth [DEBUG] > Running: git checkout autosynth-128\nSwitched to branch \'autosynth-128\'\n2020-06-05 04:41:42,841 autosynth [INFO] > Running synthtool\n2020-06-05 04:41:42,841 autosynth [INFO] > [\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\n2020-06-05 04:41:42,843 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --\n2020-06-05 04:41:43,049 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-speech/synth.py.\nOn branch autosynth-128\nnothing to commit, working tree clean\n2020-06-05 04:41:43,179 synthtool [DEBUG] > Ensuring dependencies.\nDEBUG:synthtool:Ensuring dependencies.\n2020-06-05 04:41:43,185 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool\n2020-06-05 04:41:43,189 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 04:41:43,189 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 04:41:43,193 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 04:41:44,053 synthtool [DEBUG] > Generating code for: google/cloud/speech/v1.\nDEBUG:synthtool:Generating code for: google/cloud/speech/v1.\n2020-06-05 04:41:44,889 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpjjun0xi1 is empty.\n2020-06-05 04:41:44,928 autosynth [ERROR] > Synthesis failed\n2020-06-05 04:41:44,928 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 04:41:44,934 autosynth [DEBUG] > Running: git checkout autosynth\nSwitched to branch \'autosynth\'\n2020-06-05 04:41:44,938 autosynth [DEBUG] > Running: git clean -fdx\nRemoving __pycache__/\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main\n    return _inner_main(temp_dir)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 595, in _inner_main\n    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 371, in synthesize_loop\n    synthesize_inner_loop(toolbox, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 381, in synthesize_inner_loop\n    synthesizer, len(toolbox.versions) - 1\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 266, in synthesize_version_in_new_branch\n    synthesizer.synthesize(synth_log_path, self.environ)\n  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 119, in synthesize\n    synth_proc.check_returncode()  # Raise an exception.\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode\n    self.stderr)\nsubprocess.CalledProcessError: Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n'

Google internal developers can see the full log here.

Autosynth is still having trouble generating nodejs-speech. :sob:
Here's the output from running synth.py:
b' Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 10:34:12,106 synthtool [DEBUG] > Generating code for: google/cloud/speech/v1.\nDEBUG:synthtool:Generating code for: google/cloud/speech/v1.\n2020-06-05 10:34:13,021 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpg8le_sxj is empty.\n2020-06-05 10:34:13,065 autosynth [ERROR] > Synthesis failed\n2020-06-05 10:34:13,065 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 10:34:13,070 autosynth [DEBUG] > Running: git checkout autosynth-self\nSwitched to branch \'autosynth-self\'\n2020-06-05 10:34:13,075 autosynth [ERROR] > Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n2020-06-05 10:34:13,201 autosynth [DEBUG] > Running: git checkout 3af749c8f705d92dba12169ba97c176ff3006a31\nNote: checking out \'3af749c8f705d92dba12169ba97c176ff3006a31\'.\n\nYou are in \'detached HEAD\' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n  git checkout -b <new-branch-name>\n\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 10:34:13,206 autosynth [DEBUG] > Running: git checkout 40a354b0f47454be4986f9db504cca4d4371c8c3\nPrevious HEAD position was a3a0bf0f BREAKING_CHANGE: Removing TimeSeriesQueryLanguageCondition as an alert condition type. The condition type is unsupported and unused. It was originally added for the Monitoring Query Language Alpha feature. refactor!: Drop support for TimeSeriesQueryLanguageCondition as an alert condition type.\nHEAD is now at 40a354b0 chore: add protoc proto3_optional flag in bazelrc\n2020-06-05 10:34:13,283 autosynth [DEBUG] > Running: git checkout d53a5b45c46920932dbe7d0a95e10d8b58933dae\nPrevious HEAD position was be74d3e build: do not fail builds on codecov errors (#528)\nHEAD is now at d53a5b4 docs: improve README (#600)\n2020-06-05 10:34:13,299 autosynth [DEBUG] > Running: git branch -f autosynth-129\n2020-06-05 10:34:13,304 autosynth [DEBUG] > Running: git checkout autosynth-129\nSwitched to branch \'autosynth-129\'\n2020-06-05 10:34:13,308 autosynth [INFO] > Running synthtool\n2020-06-05 10:34:13,309 autosynth [INFO] > [\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\n2020-06-05 10:34:13,311 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --\n2020-06-05 10:34:13,537 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-speech/synth.py.\nOn branch autosynth-129\nnothing to commit, working tree clean\n2020-06-05 10:34:13,679 synthtool [DEBUG] > Ensuring dependencies.\nDEBUG:synthtool:Ensuring dependencies.\n2020-06-05 10:34:13,684 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool\n2020-06-05 10:34:13,689 synthtool [DEBUG] > Cloning googleapis.\nDEBUG:synthtool:Cloning googleapis.\n2020-06-05 10:34:13,689 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\nDEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis\n2020-06-05 10:34:13,693 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest\nDEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest\nlatest: Pulling from gapic-images/gapic-generator-typescript\nDigest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66\nStatus: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest\n2020-06-05 10:34:14,628 synthtool [DEBUG] > Generating code for: google/cloud/speech/v1.\nDEBUG:synthtool:Generating code for: google/cloud/speech/v1.\n2020-06-05 10:34:15,503 synthtool [DEBUG] > Wrote metadata to synth.metadata.\nDEBUG:synthtool:Wrote metadata to synth.metadata.\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__\n    return self.main(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main\n    rv = self.invoke(ctx)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke\n    return ctx.invoke(self.callback, **ctx.params)\n  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke\n    return callback(*args, **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main\n    spec.loader.exec_module(synth_module)  # type: ignore\n  File "<frozen importlib._bootstrap_external>", line 678, in exec_module\n  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed\n  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>\n    version=version)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library\n    return self._generate_code(service, version, "typescript", **kwargs)\n  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code\n    f"Code generation seemed to succeed, but {output_dir} is empty."\nRuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmpx2bcnhwe is empty.\n2020-06-05 10:34:15,546 autosynth [ERROR] > Synthesis failed\n2020-06-05 10:34:15,546 autosynth [DEBUG] > Running: git reset --hard HEAD\nHEAD is now at 3af749c build: update protos.js (#601)\n2020-06-05 10:34:15,552 autosynth [DEBUG] > Running: git checkout autosynth\nSwitched to branch \'autosynth\'\n2020-06-05 10:34:15,557 autosynth [DEBUG] > Running: git clean -fdx\nRemoving __pycache__/\nTraceback (most recent call last):\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main\n    "__main__", mod_spec)\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code\n    exec(code, run_globals)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>\n    main()\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main\n    return _inner_main(temp_dir)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 595, in _inner_main\n    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 371, in synthesize_loop\n    synthesize_inner_loop(toolbox, synthesizer)\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 381, in synthesize_inner_loop\n    synthesizer, len(toolbox.versions) - 1\n  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 266, in synthesize_version_in_new_branch\n    synthesizer.synthesize(synth_log_path, self.environ)\n  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 119, in synthesize\n    synth_proc.check_returncode()  # Raise an exception.\n  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode\n    self.stderr)\nsubprocess.CalledProcessError: Command \'[\'/tmpfs/src/github/synthtool/env/bin/python3\', \'-m\', \'synthtool\', \'--metadata\', \'synth.metadata\', \'synth.py\', \'--\']\' returned non-zero exit status 1.\n'

Google internal developers can see the full log here.

Autosynth is still having trouble generating nodejs-speech. :sob:
Here's the output from running synth.py:
odule
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>
    version=version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library
    return self._generate_code(service, version, "typescript", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code
    f"Code generation seemed to succeed, but {output_dir} is empty."
RuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmp03_8z_3a is empty.
2020-06-05 15:59:34,312 autosynth [ERROR] > Synthesis failed
2020-06-05 15:59:34,313 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 3af749c build: update protos.js (#601)
2020-06-05 15:59:34,318 autosynth [DEBUG] > Running: git checkout autosynth-self
Switched to branch 'autosynth-self'
2020-06-05 15:59:34,323 autosynth [ERROR] > Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.
2020-06-05 15:59:34,462 autosynth [DEBUG] > Running: git checkout 3af749c8f705d92dba12169ba97c176ff3006a31
Note: checking out '3af749c8f705d92dba12169ba97c176ff3006a31'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 3af749c build: update protos.js (#601)
2020-06-05 15:59:34,468 autosynth [DEBUG] > Running: git checkout 40a354b0f47454be4986f9db504cca4d4371c8c3
Previous HEAD position was a3a0bf0f BREAKING_CHANGE: Removing TimeSeriesQueryLanguageCondition as an alert condition type. The condition type is unsupported and unused. It was originally added for the Monitoring Query Language Alpha feature. refactor!: Drop support for TimeSeriesQueryLanguageCondition as an alert condition type.
HEAD is now at 40a354b0 chore: add protoc proto3_optional flag in bazelrc
2020-06-05 15:59:34,538 autosynth [DEBUG] > Running: git checkout 9c2913e1831eb00fa01317f3fea4ce25d3f657eb
Previous HEAD position was be74d3e build: do not fail builds on codecov errors (#528)
HEAD is now at 9c2913e feat: show output from microgenerator runs (#575)
2020-06-05 15:59:34,554 autosynth [DEBUG] > Running: git branch -f autosynth-135
2020-06-05 15:59:34,557 autosynth [DEBUG] > Running: git checkout autosynth-135
Switched to branch 'autosynth-135'
2020-06-05 15:59:34,561 autosynth [INFO] > Running synthtool
2020-06-05 15:59:34,561 autosynth [INFO] > ['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']
2020-06-05 15:59:34,563 autosynth [DEBUG] > Running: /tmpfs/src/github/synthtool/env/bin/python3 -m synthtool --metadata synth.metadata synth.py --
2020-06-05 15:59:34,794 synthtool [DEBUG] > Executing /home/kbuilder/.cache/synthtool/nodejs-speech/synth.py.
On branch autosynth-135
nothing to commit, working tree clean
2020-06-05 15:59:34,932 synthtool [DEBUG] > Ensuring dependencies.
DEBUG:synthtool:Ensuring dependencies.
2020-06-05 15:59:34,937 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/synthtool
2020-06-05 15:59:34,942 synthtool [DEBUG] > Cloning googleapis.
DEBUG:synthtool:Cloning googleapis.
2020-06-05 15:59:34,942 synthtool [DEBUG] > Using precloned repo /home/kbuilder/.cache/synthtool/googleapis
DEBUG:synthtool:Using precloned repo /home/kbuilder/.cache/synthtool/googleapis
2020-06-05 15:59:34,945 synthtool [DEBUG] > Pulling Docker image: gapic-generator-typescript:latest
DEBUG:synthtool:Pulling Docker image: gapic-generator-typescript:latest
latest: Pulling from gapic-images/gapic-generator-typescript
Digest: sha256:c9bc12024eddcfb94501627ff5b3ea302370995e9a2c9cde6b3317375d7e7b66
Status: Image is up to date for gcr.io/gapic-images/gapic-generator-typescript:latest
2020-06-05 15:59:35,836 synthtool [DEBUG] > Generating code for: google/cloud/speech/v1.
DEBUG:synthtool:Generating code for: google/cloud/speech/v1.
gapic-generator-typescript: https://github.com/googleapis/gapic-generator-typescript
Latest commit:
commit a858a371e225254fca1c32d8d6b142b42cb711ad
Author: release-please[bot]<EMAIL_ADDRESS>Date:   Thu Jun 4 16:54:54 2020 -0700

child_process.js:642
    throw err;
    ^

Error: spawnSync protoc EACCES
    at Object.spawnSync (internal/child_process.js:1041:20)
    at spawnSync (child_process.js:607:24)
    at Object.execFileSync (child_process.js:634:15)
    at Object.<anonymous> (/usr/local/lib/node_modules/@google-cloud/gapic-generator/build/src/gapic-generator-typescript.js:117:17)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10)
    at Module.load (internal/modules/cjs/loader.js:811:32)
    at Function.Module._load (internal/modules/cjs/loader.js:723:14)
    at Function.Module.runMain (internal/modules/cjs/loader.js:1043:10)
    at internal/main/run_main_module.js:17:11 {
  errno: 'EACCES',
  code: 'EACCES',
  syscall: 'spawnSync protoc',
  path: 'protoc',
  spawnargs: [
<EMAIL_ADDRESS>    '--typescript_gapic_out=/out',
    '--gapic-validator_out=.',
    '--typescript_gapic_opt="grpc-service-config=google/cloud/speech/v1/speech_grpc_service_config.json"',
    '--typescript_gapic_opt="package-name=@google-cloud/speech"',
    '--experimental_allow_proto3_optional',
    '-I/in',
    '/in/google/cloud/speech/v1/cloud_speech.proto',
    '-I/protos/api-common-protos-master'
  ],
  error: [Circular],
  status: null,
  signal: null,
  output: null,
  pid: 17,
  stdout: null,
  stderr: null
}
2020-06-05 15:59:36,665 synthtool [DEBUG] > Wrote metadata to synth.metadata.
DEBUG:synthtool:Wrote metadata to synth.metadata.
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 102, in <module>
    main()
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmpfs/src/github/synthtool/env/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/__main__.py", line 94, in main
    spec.loader.exec_module(synth_module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/kbuilder/.cache/synthtool/nodejs-speech/synth.py", line 41, in <module>
    version=version)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 66, in typescript_library
    return self._generate_code(service, version, "typescript", **kwargs)
  File "/tmpfs/src/github/synthtool/synthtool/gcp/gapic_microgenerator.py", line 195, in _generate_code
    f"Code generation seemed to succeed, but {output_dir} is empty."
RuntimeError: Code generation seemed to succeed, but /tmpfs/tmp/tmp161xwz6d is empty.
2020-06-05 15:59:36,701 autosynth [ERROR] > Synthesis failed
2020-06-05 15:59:36,701 autosynth [DEBUG] > Running: git reset --hard HEAD
HEAD is now at 3af749c build: update protos.js (#601)
2020-06-05 15:59:36,709 autosynth [DEBUG] > Running: git checkout autosynth
Switched to branch 'autosynth'
2020-06-05 15:59:36,713 autosynth [DEBUG] > Running: git clean -fdx
Removing __pycache__/
Traceback (most recent call last):
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 615, in <module>
    main()
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 476, in main
    return _inner_main(temp_dir)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 595, in _inner_main
    commit_count = synthesize_loop(x, multiple_prs, change_pusher, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 371, in synthesize_loop
    synthesize_inner_loop(toolbox, synthesizer)
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 381, in synthesize_inner_loop
    synthesizer, len(toolbox.versions) - 1
  File "/tmpfs/src/github/synthtool/autosynth/synth.py", line 266, in synthesize_version_in_new_branch
    synthesizer.synthesize(synth_log_path, self.environ)
  File "/tmpfs/src/github/synthtool/autosynth/synthesizer.py", line 119, in synthesize
    synth_proc.check_returncode()  # Raise an exception.
  File "/home/kbuilder/.pyenv/versions/3.6.9/lib/python3.6/subprocess.py", line 389, in check_returncode
    self.stderr)
subprocess.CalledProcessError: Command '['/tmpfs/src/github/synthtool/env/bin/python3', '-m', 'synthtool', '--metadata', 'synth.metadata', 'synth.py', '--']' returned non-zero exit status 1.

Google internal developers can see the full log here.

Autosynth passed, closing! :green_heart:


Nom 6.2 documentation on docs.rs broken

Hi,
I've been having a pretty difficult time finding documentation for the latest version of nom on docs.rs because all the links to the documentation are broken, for example this link: https://docs.rs/nom/6.2.0/nom/bytes/complete/fn.escaped_transform.html leads to a "no-such-resource" page.
I also noticed that there's a banner on the main project page that said the doc build is broken: https://docs.rs/crate/nom/6.2.0/builds/401812

hello, I just published nom 6.2.1 that should fix this. Let me know if you still find some problems

Thank you so much! That is a great help! :slightly_smiling_face:


[WIP] Add initial metrics for sentry

Description
Add initial metrics for sentry
Issue reference
n/a
Checklist
Please make sure you've  completed the relevant tasks for this PR, out of the following list:

[ ] Code compiles correctly
[ ] Created/updated tests
[ ] Unit tests passing
[ ] End-to-end tests passing
[ ] Extended the documentation
[ ] Specification has been updated
[ ] Provided sample for the feature

@yaron2 @shalabhms CSR status chart


Uploading file on Windows

Tauri build correctly artifact
     Running candle for "main.wxs"
     Running light to produce D:\a\Sparus\Sparus\src-tauri\target\release\bundle/msi/sparus_0.1.0_x64_en-US.msi
    Finished 1 bundle at:
        D:\a\Sparus\Sparus\src-tauri\target\release\bundle/msi/sparus_0.1.0_x64_en-US.msi

When using softprops action-gh-release I get
Pattern 'D:\a\Sparus\Sparus\src-tauri\target\release\bundle\msi\sparus_0.1.0_x64_en-US.msi' does not match any files.
🤔 D:\a\Sparus\Sparus\src-tauri\target\release\bundle\msi\sparus_0.1.0_x64_en-US.msi not include valid file.

I don't know if it come from tauri-build or softprop action

Can't say much without more input (a repro or project to look at) but I just ran the test suite again and it passes without issues (including uploading the msi) so it's most likely an issue with your setup

Log : https://github.com/Ludea/Sparus/actions/runs/4103480208
Workflow : https://github.com/Ludea/Sparus/blob/main/.github/workflows/release.yml

https://github.com/softprops/action-gh-release/issues/280#issuecomment-1322187892 can be relevant

Closing, this issue is not from tauri-build but from softprops action


Snippet suggestions don't appear in menu in React prop arrow-functions

VSCode Version: 1.20.1
OS Version: Linux localhost 4.15.3-2-ARCH #1 SMP PREEMPT Thu Feb 15 00:13:49 UTC 2018 x86_64 GNU/Linux

Steps to Reproduce:

Write a React component
Add an arrow function in a prop of a subcomponent
Open the suggestions window (ctrl+space) or add a snippet from the menu

Suggestions appear properly when writing normal functions and arrow functions, but since a few updates (I can't tell how far back), I will not get any snippet suggestions in arrow functions of React props. I do get Intellisense-suggestions, so the suggestions window does still work.

Please provide some example code that shows where snippet suggestions do no show up


Can I filter null object before asRxObservable() ?

I use storIO.get().listOfObjects().withGetResolver().asRxObservable()  to observable database change.
But the  result is not I want, because my  GetResolver will return null object , when i show the chane result , i skip the null object . Can I filter null object before asRxObservable() ?
the version is
compile 'com.pushtorefresh.storio:sqlite:1.8.0'
compile 'com.pushtorefresh.storio:content-resolver:1.9.0'

What is recorder?

sorry,I did not describe clearly.There is no record in my database, it is empty

StorIO should notify you with empty result then. storIO.get()…asRxObservable() emits initial result.
In case of empty db:

storIO.get().listOfObjects() will emit empty List
storIO.get().cursor() will emit empty Cursor
storIO.get().object() will emit null

i find my error,  i use storIO.get().listOfObjects()  and my GetResolver will return null object, when i show GetResolver return data , i skip the null object , so i see no data in my app. Can i filter null object before asRxObservable()

Sorry, not sure I follow, can you please rewrite your question (if it is a question) "Can i filter null object before asRxObservable()"?

I'm closing this for now, feel free to comment!


Fails and corrupts LogSeq commands

followed the directions, and from thenon "/" would only show the first slash command. Settings>>Advanced>>Clear cache reset LogSeq to where it was before I attempted this plugin.

same problem here.

That's not problem of the plugin, it's a Logseq problem. They fix it in the latest version of the app, pls update it. :)

Hmmm, I'm already using the latest version (0.6.1) on win10; however, the problem still appears.
Thanks for the quick response! I will check with Logseq teams.

After fewer investigation, It turns out that I misunderstood the way to install the plugins. I git clone the repo and load plugins from that repo floder, not knowing that I should actually use the release zip file. 🤣 Maybe @BobWalsh can give it another try.
Now the plugin works faultlessly! Sorry for bothering @DenaroCF !


Gsheets

I dunno about the strings. It's a matter of whether the special_docs.csv should be viewed as the main source of truth, or whether in theory any old worksheet can be used.
The pdfs seem to work on HF now, the ebooks won't be shown as I disabled them for copyright reasons

I'm not sure what the best way would be to provide the ebooks to the chat bot :/
Well, the best way would be to just get permission to add them. Other than that, the chatbot will have to manually fetch them, or maybe even have them on disk when deployed? That might be the best option, as adding them to the github repo would probably also be a bad idea. Seems worth discussing during the meeting


Fix for volumemounts indents and also added the serviceAccount feature from kibana as have a need for it

[ ] Chart version not bumped (the versions are all bumped and released at the same time)
[ ] README.md updated with any new values or changes
[ ] Updated template tests in ${CHART}/tests/*.py
[ ] Updated integration tests in ${CHART}/examples/*/test/goss.yaml

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

Thanks for working on and fixing this! I'm planning to add a complex example to the automated example tests to make sure we can test all of these sorts of features and be sure that they are configured properly. Right now the integration tests only focus on testing the features used by the default install.

Would really like to see it merged! @jamestutton this looks like a very simple and straightforward fix. Do you plan to submit it anytime soon?
Thanks!

Hello @jamestutton,
Thank for your PR.
I'll close it as extraVolumes was fixed in #245 and Service Account was implemented in another way in #265.


Failure after installing from pip

I get the following error after installing from pip on Ubuntu 14.04:
Traceback (most recent call last):
  File "/usr/local/bin/wal-e", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: requests>=2.8.1,!=2.9.0

+1 same problem

Hi @Aitem , I also ran into this bug. My workaround was to add the --upgrade option to pip install.

This bug is related to pip, reinstall or upgrade, see #241 it's the same.
Sometimes you can just upgrade distribute too

Unfortunately upstream dependencies break WAL-E installs all the time. The problem is getting worse as more backends are added. A more satisfying mitigation that could be done in WAL-E is to start making backends optional.
They are already lazily-loaded, so this is mostly packaging work and adding good error messages on import failures.


Document that CookieProvider bails out the route segment from static generation

I noticed while building my application that the entire app now opted out of static generation.
After debugging the problem I concluded the cause to be wrapping the root layout with <CookieProvider>.
In my case, I only needed client cookies on a single page, so wrapping the root was an overkill.
I think it would be beneficial to document this.

Added: https://github.com/moshest/next-client-cookies/commit/de68ab7dd29b7f3b5c89158176963e729a8f9837


Fix "cannot find "orientation.h" error

When manually linking react-native-orientation for iOS, an extra step is needed otherwise it'll show `cannot find "orientation.h" error -
Add $(SRCROOT)/node_modules/react-native-orientation/iOS/RCTOrientation/ to Project Name -> Build Settings -> Header Search Paths

Ah true! Thanks for pointing that out. Will merge.


Fix sync_s3 for python3

@trbs
I didn't test this with py2 and other regions than eu-central. Kudos if someone could do that for me.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling f5e14ca9ba71a1ef0d1cb778572d9331c91e18e9 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Coverage decreased (-0.01%) to 29.56% when pulling 4a1bbbff6bd949eb88da9b2fe3ad8caf9b459030 on Kondou-ger:sync_s3_fix_py3_eu-central into 90c34556b35ae7b46255abde3571e96c21115eee on django-extensions:master.

Thanks !


Add bronhouder names

What about adding a test that checks whether the dictionary matches the current one found on het kadaster?
The following code compares the top row of the table presented on https://www.kadaster.nl/-/bgt-bronhoudercodes with the current implemented version
import requests
from lxml import html

# The URL of the page
url = 'https://www.kadaster.nl/-/bgt-bronhoudercodes'

# Send a GET request to fetch the raw HTML content
response = requests.get(url)

# Parse the HTML content
tree = html.fromstring(response.content)

# Use the XPath to find the desired text
xpath = '//*[@id="collapse1"]/div/ul/li[1]/text()'
text = tree.xpath(xpath)[0]

assert text == "Versie 2024: 1-1-2024", "Bronhoudercodes are not up to date. Update `get_bronhouder_names()`."

What about adding a test that checks whether the dictionary matches the current one found on het kadaster?

Good point. I can add a check for that. Is there a formal protocol to do this in nlmod? For the knmi data platform the issue is the same I think with an open api key that can expire.
Another option would be to download and read the excel / ods file, but I'd advise against that since it looks unstructured.

In my previous comment, I was a bit confused if you wanted a check in the get_bronhouder_names or a test. But I now added a test to assert if the date from the last url/file on the kadaster website is from 2024.

Perfect! Thank you for the consideration.
The reason to opt for only checking the year opposed to whether it is the most current version is because, an update once a year is sufficient?

The reason to opt for only checking the year opposed to whether it is the most current version is because, an update once a year is sufficient?

Yes, generally there is no big update. But you're right; in 2022 the file was updated twice. So I can also check the date as well.


Handle multiple select in forms

Closes https://github.com/germsvel/phoenix_test/issues/70

@germsvel There are some test cases still failing. See if you have any feedbacks. I'll give another attempt tomorrow.

@sasikumar87 thanks for opening this PR! 🎉 I've given it a quick glance, but I need to give it a better look. I don't have time right now, but I'll do it as soon as I can.
Aside from the TODO to raise an error, do you see it as done? Or are there any other open questions that I might be able to help with?

@germsvel Thanks.
I have rebased for recent changes. With this, apart from TODO, I see it done.


fix: Avoid to call import and export Arrow array for native execution

Which issue does this PR close?
Rationale for this change
Performance improvement
What changes are included in this PR?
This PR has changes to avoid to call import and export Arrow array for native execution
How are these changes tested?
Exisiting tests

Before

After

I tried running benchmarks from this PR but I am running into an error:
ExecutorLostFailure: The executor with id 21 exited with exit code 134(unexpected).

I also see this glib error in the executor log:
corrupted size vs. prev_size

Codecov Report
Attention: Patch coverage is 36.06557% with 39 lines in your changes missing coverage. Please review.

Project coverage is 34.32%. Comparing base (845b654) to head (ec72117).
Report is 16 commits behind head on main.

Files with missing lines
Patch %
Lines

...ava/org/apache/comet/vector/CometNativeVector.java
0.00%
15 Missing :warning:

...in/java/org/apache/comet/parquet/ColumnReader.java
0.00%
11 Missing :warning:

...ain/scala/org/apache/comet/vector/NativeUtil.scala
0.00%
8 Missing :warning:

...ain/java/org/apache/comet/parquet/BatchReader.java
0.00%
3 Missing :warning:

...ava/org/apache/comet/parquet/LazyColumnReader.java
0.00%
1 Missing :warning:

.../src/main/java/org/apache/comet/parquet/Utils.java
0.00%
1 Missing :warning:

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1055      +/-   ##
============================================
- Coverage     34.46%   34.32%   -0.15%
- Complexity      888      893       +5
============================================
  Files           113      114       +1
  Lines         43580    42916     -664
  Branches       9658     9339     -319
============================================
- Hits          15021    14732     -289
+ Misses        25507    25336     -171
+ Partials       3052     2848     -204

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@kazuyukitanimura. I can run the benchmarks now, but I did not see any difference in performance compared to the main branch. Are you still seeing a performance benefit after the recent changes?

Thank you @andygrove
That's odd. My local queries are showing clear improvement. Just checking, are you using iceberg? I haven't done the DSv2 yet.
I will try to run more queries as well.

5 iterations
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q27: Comet (Scan, Exec)                            4218           4320         116         68.8          14.5       1.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q27: Comet (Scan, Exec)                            3534           3784         191         82.1          12.2       1.0X

Before
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q39a: Comet (Scan, Exec)                          14077          14413         252         28.4          35.2       1.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q44: Comet (Scan, Exec)                            2160           2235          73        133.4           7.5       1.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q68: Comet (Scan, Exec)                            5023           5091          61         58.0          17.3       1.0

After
OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q39a: Comet (Scan, Exec)                          12968          13770        1085         30.8          32.5       1.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q44: Comet (Scan, Exec)                            1968           2106         129        146.5           6.8       1.0X

OpenJDK 64-Bit Server VM 17.0.11+9-LTS on Mac OS X 14.7
Apple M1 Max
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q68: Comet (Scan, Exec)                            4232           4530         195         68.8          14.5       1.0

I see these results from 3 runs of q39 (both a+b) with 1TB input:
main: 59.3s/57.0s/57.1s
this PR: 58.2s/58.9s/57.9s
I am using these configs:
    --conf spark.executor.instances=4 \
    --conf spark.executor.cores=6 \
    --conf spark.executor.memory=8g \
    --conf spark.memory.offHeap.enabled=true \
    --conf spark.memory.offHeap.size=32g \

Thanks @andygrove
I tried with offHeap but I still see the advantage. Wondering if any of the following configs make difference. I use 100G input
spark.comet.regexp.allowIncompatible=true
spark.sql.shuffle.partitions=4
spark.driver.memory=3g
spark.executor.memory=3g
spark.sql.autoBroadcastJoinThreshold=20971520 // (20 * 1024 * 1024)
spark.sql.crossJoin.enabled=true
parquet.enable.dictionary=true
spark.shuffle.manager=org.apache.spark.sql.comet.execution.shuffle.CometShuffleManager

I guess the diff is only observable on very small scale (ms). Once the query time is second level, the diff is insignificant.

Thank you @andygrove @viirya I addressed memory issues and added DSv2 support (except Iceberg). This is ready for review again.
I also run with 1TB by myself. I still see 10% ish speed up that is the order of seconds not ms. I think there are some environmental differences between @andygrove 's and mine.
Before (1TB)
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q39a: Comet (Scan, Exec)                          27979          33349         580         28.0          35.7       1.0X

TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q44: Comet (Scan, Exec)                           24877          25216         271        115.8           8.6       1.0X

After (1TB)
TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q39a: Comet (Scan, Exec)                          27281          28714         111         28.7          34.8       1.0X

TPCDS Micro Benchmarks:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
q44: Comet (Scan, Exec)                           23534          23880         388        122.4           8.2       1.0X

It looks like less than 5%?
27979 - 27281 =  698 ~= 2%
24877 - 23534 = 1343 ~= 5%
So I believe that the diff is insignificant for longer query time.

Thanks @viirya

It looks like less than 5%?
27979 - 27281 = 698 ~= 2%
24877 - 23534 = 1343 ~= 5%

I used Ave Time (5 iterations) when I mentioned 10% ish
100G q39a (14413-13770)/14413 ~= 4.5%
1T q39a (33349-28714)/33349 ~= 14%
The performance gain can easily swing between 2%-15% due to the noise

So I believe that the diff is insignificant for longer query time.

The performance gain is proportional to the input size by design because it saves the overhead for all input arrow vectors.
If you mean longer query time by deeper query plan, then yes, the performance gain gets less effective as the benefit is one-time per data read and query procesing time takes more portions in the entire runtime.
But there are many use cases I think for shallow query plans as well and we can still save $$$?

1T q39a (33349-28714)/33349 ~= 14%

33349 obviously looks like it is affected by some noises.
q44 avg diff is also ~ 5%.

The performance gain is proportional to the input size by design because it saves the overhead for all input arrow vectors.

I mean it is less significant with longer query time. For queries running many minutes or hours, it has no difference.
If this is a trivial change, it will be good. But as I took a quick look, the change is not small and doesn't look like in good design to me.
I doubt that if it is worth.

@viirya  Let me try to convince one more time.

For queries running many minutes or hours, it has no difference.

If the query time is long because the data size, this PR still helps (5% at least?). If the query time is long because the query itself is complex, this PR has less value.

the change is not small

The latest change is pretty small after following your change on Arrow spec memory model. There are only 3 main changes

avoid importing: common/src/main/java/org/apache/comet/parquet/ColumnReader.java
avoid exporting: common/src/main/scala/org/apache/comet/vector/NativeUtil.scala
type handling fix: native/core/src/execution/utils.rs

The rest of changes are only for passing the information of the new mode is used.

doesn't look like in good design to me.

Do you have any recommendations here? What if I add a feature flag to enable/disable this new code flow. The latest change is fully backward compatible. We can easily enable/disable with a single flag to manipulate hasNativeOperations boolean.

I understand what this PR is doing now. By using CometNativeVector when importing from native, and then reexporting CometNativeVector we are just passing memory addresses and avoiding extra serde costs.

Currently this PR only handles scan + exec, but if we can cover shuffle similarly the design may become clearner

The approach in this PR still serializes the schema with each batch (as is necessary when using the Arrow C data interface). I am not sure that the fast path reduces much cost. I think it is worth exploring more but I am not sure we should merge this PR yet, especially as the comet-parquet-exec work may replace some of this code.

Thanks @andygrove

I am not sure that the fast path reduces much cost.
In the flamegraph, currentBatch() and exportBatch() blocks are significantly shortened/disappeared
https://github.com/apache/datafusion-comet/pull/1055#issuecomment-2456050303

I think it is worth exploring more but I am not sure we should merge this PR yet,
I can close this for now

especially as the comet-parquet-exec work may replace some of this code.
I thought this PR part does not change as we still need to send back the results to JVM for pure scan case, but I could be wrong...

Closing this for now


Edit distance

Submission Title
Edit distance
Submission Type
Algorithm
Submission Description
problem statement:- The edit distance between two strings is the minimum number of operations (insertions, deletions, and
substitutions of symbols) to transform one string into another. It is a measure of similarity of two strings.
Edit distance has applications, for example, in computational biology, natural language processing, and spell
checking. Your goal in this problem is to compute the edit distance between two strings.
Overall time complexity :- O(m*n) ,where m,n are length of strings.
Contributors

[x]  C++ -
[ ]  C -
[ ]  Python -
[x] Java -
[ ] JavaScript -
[ ] TypeScript -
[ ] Golang -
[ ] Ruby -
[ ] Scala -

I want to implement this algorithm in c++. please assign it to me.

I want to implement this algorithm in Java. please assign it to me.

@ahampriyanshu
Can you please assign this to me for Golang?

May I try to implement the problem in python?

I want to implement this algorithm in C. please assign it to me.

I would really like to implement this in Python. Please assign it to me @yogesh-kansal

I would like to implement this algorithm in JavaScript. Please assign it to me.

thanks everyone :v:


Fix get_type_hints() for linen modules

What does this PR do?

Fixes #1947
I removed some unnecessary forward references in nn.Module, which were preventing the types of the module's parent field from being resolved correctly. Happy to add tests if it's desired/appropriate.
Checklist

[x] This PR fixes a minor issue (e.g.: typo or small bug) or improves the docs (you can dismiss the other
checks if that's the case).
[x] This change is discussed in a Github issue/
discussion #1947
[ ] The documentation and docstrings adhere to the
documentation guidelines.
[ ] This change includes necessary high-coverage tests.
(No quality testing = no merge!)

Codecov Report

Merging #1948 (d2e6648) into main (96c78cd) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1948   +/-   ##
=======================================
  Coverage   70.92%   70.92%
=======================================
  Files          58       58
  Lines        4918     4918
=======================================
  Hits         3488     3488
  Misses       1430     1430

Impacted Files
Coverage Δ

flax/linen/module.py
93.56% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 96c78cd...d2e6648. Read the comment docs.


FEAT: add tagging support to axolotl for DPOTrainer

Description
Add axolotl tag to HuggingFace model cards created when fine tuning with RL (using the DPOTrainer).
Motivation and Context
This PR is required to extend #1004 to include the DPOTrainer. The relevant issue is #979
How has this been tested?
I have tested this by fine tuning a model and pushing it to the HuggingFace hub.
Types of changes

create a new AxolotlDPOTrainer class which subclasses the trl DPOTrainer class
move _sanitize_kwargs_for_tagging outside the AxolotTrainer class

Social Handles (Optional)
X/Twitter: @olgias

@younesbelkada @winglian This is a follow up for #1004.
I am not entirely convinced by my solution though:

probably _sanitize_kwargs_for_tagging can be move to utility module
there is currently code duplication (the two def push_to_hub(self, *args, **kwargs) -> str:) inside AxolotlTrainer and AxolotlDPOTrainer)

Thank you @younesbelkada 🙏🏻
I rebased the branch and force-pushed a last change to fix a linting issue which was causing a test to fail.

@filippo82 I've fixed the issue and rebased again for you


Add new device form does not submit

I attempted to add a new device to a library but the form does not submit. Here are the values I was using if it's helpful:
Device Name: bennington-egress-wired
Device ID: 0ce3deb83d74a4f01fa44909b11cde455afc9c69937369a3dcdfd3bbdb5dd7
Network Type: public (though if "egress" was an option I would select it)
Connection Type: wired
MAC Address: 00:1e:06:37:07:4e

@critzo tested on latest demo instance and I'm able to add a device successfully, so closing for now! Let me know if this issue persists.


Cancel download

Hi! I am downloading file at componentDidMount. I want to cancel downloading when user tap button.
Example:
componentDidMount(){
var task = RNFetchBlob
.config({
// add this option that makes response data to be stored as a file,
// this is much more performant.
fileCache : true,
path : dirs.DocumentDir + '/1.pdf'
})
.fetch('GET', 'URL', {
  })
  .progress((received, total) => {
      thiz.setState({percent:{recieved:received,total:total}});
  })
  .then((res) => {
    console.log('The file saved to ', res.path())
    thiz.setState({"pdf":res.path()})
  })

}
onCancel(){
// HERE I WANT TO CANCEL DOWNLOAD WHEN USER TAP CANCEL BUTTON
}

35.0 RN version

Hi @YusufBesim , have you tried the API ? perhaps it's what you need ?


Loan Options print styles

The Loan Options pages could use some print CSS love.
There are some guidelines for print styles being discussed here: https://github.com/cfpb/design-manual/issues/281

Current print view


Disable completion without disabling auto-imports?

Do you mind explaining your usecase here?

I have Roblox LSP and Luau LSP active since I like Roblox LSP's autocompletion, but I like Luau LSP's typing. With both of them it duplicates the autocompletion. Is there a way to disable it for Luau LSP?

Is there a way to disable it for Luau LSP?

No, there's currently no way to disable completion in Luau LSP without also disabling auto-imports. This is because the setting responsible for auto-imports, luau-lsp.completion.imports.enabled, is under the luau-lsp.completion namespace rather than having it as its own independent setting like how Roblox LSP's config has it.

This is because the setting responsible for auto-imports, luau-lsp.completion.imports.enabled, is under the luau-lsp.completion namespace rather than having it as its own independent setting like how Roblox LSP's config has it.

This is not the reason. It does not make a difference where the config is located, and can be doable either way. Auto import is a type of autocomplete. Making it separable requires changes to our underlying completion operation structure.

This is not the reason. It does not make a difference where the config is located, and can be doable either way.

I didn't intend to imply that it was. I was just trying to give a simple explanation, but I guess it was misleading.

Unfortunately I do not think we are going to implement this anytime soon, so going to close this issue.


[SPARK-19951][SQL] Support || for string concatenation in Spark

https://issues.apache.org/jira/browse/SPARK-19951

Fixed in https://github.com/apache/spark/commit/b526f70c16caa2062a77117808860e1d3662114c


Unable to download course - fix

Summary of the Pull Request
Thanks for contributing!
Please be sure you are following the guidelines at
https://github.com/heliomarpm/udemy-downloader-gui/blob/master/docs/CONTRIBUTING.md
What is this about:

This is a fix for issue #106 which uses the 'path' module instead of hardcoding directory slashes

What is included in the PR:

a commit for fixing the issue in the app.js file and another commit for commenting debugger which triggers on every file download

How does someone test/validate:

run the app in dev mode - tested in both Linux (Ubuntu 22.04) and Windows (Win 11) based systems in dev environment

Quality ChecklistV

[ ] Linked issue: #xxx
[ ] Tests: Added/updated and all pass
[ ] Localization: All end user facing strings can be localized
[ ] Docs: Added/ updated

Hello, thanks for the contribution, this change already exists in the other PR