proxy: easier usage of N sockets per backend

mcp.backend({ etc, connections = N })
allow creating N tcp sockets for this described backend.
when a request is matched against a pool to retrieve a backend, a container object is pulled:

the container object holds all tcp sockets
a tcp socket is selected from the container just before the request is executed
otherwise all execution is identical to the previous code.

Changing the connection count does not interfere with key distribution within a pool.
TODO:

[ ] specific tests. the test suite does pass as-is, so single connection counts seem to work fine.
[ ] replace the conn selection loop for something more simple
[ ] if not, move depth to a small array on the container object if possible (probably not; as there's no way to have two dynamic arrays in one struct)
[ ] remove second bad backend checker in backend_run code.
[ ] some benchmark testing to ensure no major regressions.

This also deduplicates a little code. Since most related code doesn't change this should be simple to verify.
@feihu-stripe - probably of interest.

Debating setting a hard limit of 4 or 8 sockets:

Setting like 100 sockets is guaranteed to be terrible performance, and I know people do that (ie; setting 72 worker threads on a 4 core system)
Lets me keep the "lowest depth" algorithm. Looping within a 64 byte cacheline is nearly free.
This still ends up with idle established sockets, which can be problematic for people with dumb firewalls.

tcp keepalive is supposed to be the answer here and I would love to keep the better algorithm. Will reach out to people for feedback.

Code audit looks pretty good actually; very little code actually changed. A few loops were added, some code deduplicated, and a passed argument was changed a bunch with no change to its functions.
TODO's still look relevant. Need to decide on a more final chooser function before writing a lot of tests to avoid excess code churn.

spent a few days moving my work environment to a new computer. will wrap this up soon.

Alright I think most of the work is done now. Needs some benchmarking so I'll come back to that later.
Still debating on hard limiting the connection count. So that's one more TODO item.

rebased. this fixed a bug I ended up wasting two hours tracking down the other day :( leaving PR's up for months sucks.
it's been passing mcshredder test suite so far. a few of the tests run better with multiple connections but it's all the heavily pipelined stuff.
going to hold merging for one more day so I can add bandwidth limited tests to the perf suite and re-run everything. Also still need to add the connection limit.. I'll set it to 8 for now.

It's passed a bunch of benchmarks.
Think I'll fix up the staging branch and push this in there, so I can start developing on top of it. I intend to include this in the next release but am being extra careful..

squashed and merged to staging. I probably won't let it sit in staging for too long, but this lets me write new code on top of it if I need to.

rebased over next... going to run the bench suite against it once more and merge.

mmm.. can't tell if test suite is hanging due to rare bug or problem with this commit. takes forever to re-run so I guess this may miss the tarball cutoff.

Sigh. Paranoia getting the best of me. I'm going to leave this staging for this release and just develop on top of it until I can get more tests built.
I have a lot of benchmark tests now but none that cause timeout/retries mid-test.

Leaving a note while I work on this: Just noticed a weird regression that I just pinned to this commit. One specific section of t/proxy.t is running extremely slowly (at timeout speeds?). This is only seen in this one test suite and none of the benchmark tests picked it up so far as I can tell.
Will update once fixed.

Fixed. Apparently there's only one test in the whole stupid test suite that bothers testing bad backends (or there are but they don't notice the test is taking too long).

Going to hold this a bit more to write a test that would fail at least somewhere other than the legacy t/proxy.t file. Then squash merge and rebase the staging branch.

keeping these commits separate for another day while I try to write some tests for the perf suite to see if they'd catch the bug. None of the existing ones do since they don't deal in dead backends.

The load generator's "break" tests now reproduce this bug very obviously (traffic goes to zero, 3s lag measurements).
running the whole new break suite now then I can finally merge this damn thing.

This is re-merged into staging. it's survived hours and hours of break tests so far.
I'm going to just keep developing on top of staging until I feel the need to move stuff into next

merged to next. going to close this..


"Add more" url is broken

Is the issue already present in https://github.com/cloudera/hue/issues or discussed in the forum https://discourse.gethue.com?
What is the Hue version or source? (e.g. open source 4.5, CDH 5.16, CDP 1.0...)
CDP 1.0 (Cloudera Runtime 7.0.3)
Is there a way to help reproduce it?

The URL should be
https://gethue.com/new-sql-editor/

https://github.com/cloudera/hue/pull/1007?email_source=notifications&email_token=AAAEMGJQHN4D636MZ6CMFCTQW3XVLA5CNFSM4JVAQZY2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4H53AVAA


⚠ Bofrost.be has degraded performance

In b6f8561, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1752 ms

Resolved: Bofrost.be performance has improved in 3e31a34 after 13 minutes.


Add GitHub Actions

to make the tests run on every push and every pull-request.
There are two tests still failing as reported in #36
This PR is part of my Daily CI series
A detailed description of my process reaching this PR can be found here

Very good @szabgab! I'll review it soon!

Hi @jonatas,
I am trying out a new idea to improve the state of open source projects and to increase the cross-contribution.
If you have time please check out this issue: https://github.com/OSDC-Code-Maven/OSDC-Code-Maven.github.io/issues/5
where I invite people who received an open source contribution from me to pass on the favor by sending a small contribution to some other project.


Fix CI

flake.nix tried to run make tests, which doesn't exist (it's make test since https://github.com/AeneasVerif/aeneas/pull/93). This resulted in "nothing to be done for tests", which didn't error because there is a directory called tests x))

It is extremely embarrassing to set up a CI and realise that it actually doesn't fail when it ought to... Thanks a lot for this: that's a nice catch!


Transfer project to new maintainer

Hello,
I'm the director of Free Law Project (https://github.com/freelawproject). We've become dependent on jsondate and we want to upgrade it to py3. This library feels pretty much abandoned, so I wonder if we can take it over somehow. It's a very small library, so I think that'd just mean:

Figuring out how to transfer the code to us on Github

Figuring out how to transfer the pypi things to us so we can upload new versions

Working out any licensing issues (looks like it's MIT, so should be easy).

We've already vendored the code in freelawproject/juriscraper#249 and are considering a fork, but it'd be much better to get this one fixed and alive again.
Thank you,
Mike

There are several forks here that seem like winners:

umeboshi2 made and publishes jsondate3. It supports Python 2.7 and 3: https://github.com/umeboshi2/jsondate/
I made jsondate3-aware, which is a version that converts ISO-8601 dates into timezone aware datetime objects: https://github.com/freelawproject/jsondate3-aware


Adding G7 certificate to WWDRCA list

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[ ] I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Apple recently released a new Worldwide Developer Relations intermediate certificate. This change adds the new certificate to the WWDRCA_CERTIFICATES list so it can be automatically installed on systems when it is missing.
https://www.apple.com/certificateauthority/
Description

Testing Steps

Also, would you mind resolving the merge conflicts in this PR? :)

Thanks for the follow up @rogerluan. Yea, this one seems out of date now, so I'll close it.


Able to select PDF files after clicking in the select PDF Files button

Description: After clicking on the select PDF Files, the file explorer and can only select the PDF files.
Labels: Feature
Assignees: @otaku0304

#14


RaggedTensor Hierarchical Model Issue

I have the following model:
   import numpy as np

   def example():

        # Encode each timestep
        in_sentence = Input(shape=(None,),  dtype='int64', name="Input1")
        embedded_sentence = Embedding(1000, 300, trainable=False)(in_sentence)
        lstm_sentence = LSTM(300)(embedded_sentence)
        encoded_model = Model(in_sentence, lstm_sentence)

        section_input = Input(shape=(None, None), dtype='int64', name="Input2")
        section_encoded = TimeDistributed(encoded_model)(section_input)
        section_encoded = LSTM(300)(section_encoded)
        section_model = Model(section_input, section_encoded)

        document_input = Input(shape=(None, None, None), dtype='int64', name="Input3")
        document_encoded = TimeDistributed(section_model)(document_input)
        document_encoded = LSTM(300, return_sequences=True)(document_encoded)
        document_encoded = Dense(1)(document_encoded)
        document_model = Model(document_input, document_encoded)
        document_model.compile(loss='categorical_crossentropy',
                      optimizer='rmsprop',
                      metrics=['accuracy'])

        print(section_model.summary())
        print(document_model.summary())
        return document_model

And I call it:

y = []
for doc in documents:
    tocs = []
    for token in doc:
        tocs.append(np.random.random_integers(0, high=1))
        y.append(tocs)

output = tf.ragged.constant(y)

input = tf.ragged.constant([[[[4, 5, 2, 6]], [[10, 12, 9], [26, 20, 21, 22], [35, 34, 31]]]])

example().fit(input, output, use_multiprocessing=True, epochs=10, batch_size=1)

I get the following error:
Traceback (most recent call last):
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\util\nest.py", line 318, in assert_same_structure
expand_composites)
ValueError: The two structures don't have the same nested structure.
First structure: type=RaggedTensorSpec str=RaggedTensorSpec(TensorShape([5, None, None, None]), tf.int32, 3, tf.int64)
Second structure: type=Tensor str=Tensor("Input3:0", shape=(None, None, None, None), dtype=int64)
More specifically: Substructure "type=RaggedTensorSpec str=RaggedTensorSpec(TensorShape([5, None, None, None]), tf.int32, 3, tf.int64)" is a sequence, while substructure "type=Tensor str=Tensor("Input3:0", shape=(None, None, None, None), dtype=int64)" is not
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:/Development/competitions/Kaggle/Tensorflow 2.0 - Q_a_A/tensorflow_2.0/QuestionAnswerContextAttention.py", line 392, in
qa_model.fit(input=documents, output=output)
File "D:/Development/competitions/Kaggle/Tensorflow 2.0 - Q_a_A/tensorflow_2.0/QuestionAnswerContextAttention.py", line 163, in fit
self.example().fit(input, output, use_multiprocessing=True, epochs=10, batch_size=1)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 728, in fit
use_multiprocessing=use_multiprocessing)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 224, in fit
distribution_strategy=strategy)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 547, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py", line 594, in _process_inputs
steps=steps)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\keras\engine\training.py", line 2497, in _standardize_user_data
nest.assert_same_structure(a, b, expand_composites=True)
File "C:\Development\Python\Python37\lib\site-packages\tensorflow_core\python\util\nest.py", line 325, in assert_same_structure
% (str(e), str1, str2))
ValueError: The two structures don't have the same nested structure.
First structure: type=RaggedTensorSpec str=RaggedTensorSpec(TensorShape([5, None, None, None]), tf.int32, 3, tf.int64)
Second structure: type=Tensor str=Tensor("Input3:0", shape=(None, None, None, None), dtype=int64)
More specifically: Substructure "type=RaggedTensorSpec str=RaggedTensorSpec(TensorShape([5, None, None, None]), tf.int32, 3, tf.int64)" is a sequence, while substructure "type=Tensor str=Tensor("Input3:0", shape=(None, None, None, None), dtype=int64)" is not
Entire first structure:
.
Entire second structure:
.
Process finished with exit code 1
Please make sure that this is a bug. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary):
TensorFlow version (use command below):
Python version:
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with: 1. TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)" 2. TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"
Describe the current behavior
Describe the expected behavior
Code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate the problem.
Other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.

@nectario, Please provide the Tensorflow version and complete standalone code to reproduce the reported issue. Thanks!

Hi @gadagashwini ,
I have attached a zip file that contains the code and data that recreates the problem. Just unzip the file and type:
python RaggedProblem.py
ragged_problem.zip
Thanks!
Nektarios

I could replicate the issue with Tf 1.15.
Please find the gist here. Thanks!

Thank you! For me this happened with TensorFlow 2.0 also.

@nectario Can you please take a look at this issue and let me know if it helps. Thanks!

I tried specifiy ragged = True on all Input layers, but now I get a different issue:
TypeError: Failed to convert object of type <class 'tensorflow.python.ops.ragged.ragged_tensor.RaggedTensor'> to Tensor. Contents: tf.RaggedTensor(values=Tensor("Input1/flat_values:0", shape=(None,), dtype=int64), row_splits=Tensor("Input1/row_splits_0:0", shape=(None,), dtype=int64)). Consider casting elements to a supported type.

@nectario This is very similar to the issue #27170. Lets track the issue there and close it here. Please let me know if you think otherwise. Thanks!

@nectario did you manage to resolve the issue after all?
I am having exactly the same problem...
Ragged tensor support still seems to be lacking...
I do not want to have to clip and pad my sentences/words to represent documents.

@nectario did you manage to resolve the issue after all?
I am having exactly the same problem...
Ragged tensor support still seems to be lacking...
I do not want to have to clip and pad my sentences/words to represent documents.
@Jordy-VL Never resolved this issued. I actually gave up on ragged tensors and padded my inputs. I am hoping for better support for ragged tensors in Keras.

Thanks for the heads-up @nectario
Quite sad to still wait for this support as the concept behind ragged tensors is beautiful.
It deserves to be embedded deeper in the Tensorflow toolbox.
For anyone wanting to pad and clip ragged tensors I have a handy helper function:
def hierarchical_tensorize(variable_length_sequences, max_sentences=None, max_document_len=None):
    '''
    since support for ragged tensors is still incipient; we need to convert ragged to full tensors
    Args:
        variable_length_sequences (list): 2D variable length "sentences x words" elements
        max_sentences (int, optional):
        max_document_len (int, optional): IS abused for max words per sentence

        -> .to_tensor()[:,:max_sentences,:max_document_len]
    if using ragged tensor -> can only use batch without padding

    Returns:
        3D tensor
    '''
    docs = tf.ragged.constant(variable_length_sequences)
    docs = docs.to_tensor()
    if max_sentences:
        docs = docs[:, :max_sentences, :]
    if max_document_len:
        docs = docs[:, :, :max_document_len]
    return docs


Changed explode to true

Related https://github.com/nemtech/catapult-rest/issues/270

aren't we going with the deepObject option?
if so, we need to turn style to deepObject as well

@Vektrat Fernando didn't confirm if deepObject (?type[]=16974&type[]=16718) is compatible with current OpenAPI generators. In his last message, he suggested using explode:true (?type=16974&type=16718 ) instead.

I think the default open API option form exploded is the best middle ground,
?type=16724&type=16705&type=17220
This works in java and ts without custom templates
I've just tried the deepObject style and java generated clients don't support it. Could we go with the form exploded @dgarcia360 @Vektrat @rg911?
I wouldn't use deep object query param anyway, if we want to serialize complex objects, I would send a post JSON body.
https://github.com/OAI/OpenAPI-Specification/issues/1706

I'd go with explode: true too, waiting for confirmation from @Vektrat and @martinayora to merge this PR.

i'm unsure if it's directly supported by the REST api so this will need to wait from our side until i try it


flutter -v should tell you what it's running while its running it.

That's a @danrubel question. :)


Cache pygments rendering between runs.

Brings my compile time down from 4 seconds to 24 milliseconds.
Instant livereload works again, as it should.

Merged in 15463d3
Some code intensive sites will breathe lighter after this ...


Fix IE bugs

Fix jedi-validate/example/bootstrap.html
Add polyfills or rewrite to support IE 11.
Change env preset targets to IE 11.
Look at useBuiltIns option
Check work in IE 11.

reserved :relaxed:


Read password from environment variable

If the password is not provided via the command line, try to read it from the HTTPIE_PASSWORD environment variable.
This is meant as a simple way to avoid leaking the password via the process title in scripted runs.

Is this functionality available in some other form?


Revamp command line results table

Currently, the command line results table is one big chunk. We should do the following changes:

Separate queries from other statistics (as they take up quite some space)
Use another format, ideally one that we can directly paste as Github flavored Markdown

Thanks to @rjernst for the suggestions.

Hi @danielmitterdorfe, I would be happy to give this one a bash. I will have a look at the current blob - and see what options we have to display. When we agree on the display format, i can then implement it.

Current stats look like this:
Racing on track [geonames] and challenge [append-no-conflicts] with car [defaults]
Benchmarking indexing at 16729.2 docs/s                            [100% done]
Benchmarking stats (warmup iteration 100/100)                      [100% done]
Benchmarking stats (iteration 100/100)                             [100% done]
Benchmarking search (warmup iteration 1000/1000)                   [100% done]
Benchmarking search (iteration 0/1000)                             [  0% done]
Benchmarking search (iteration 1000/1000)                          [100% done]
_______             __   _____

/ ()  ____ / /  / /_____  ________
/ /_  / / __ / __ `/ /   __ / / __ / / _
/ __/ / / / / / // / /   / / // // / /  /  __/
//   /// //_,//   /__/_/_//   ___/
                                               Metric     Value

                     Min Indexing Throughput [docs/s]     16322
                  Median Indexing Throughput [docs/s]     16758
                     Max Indexing Throughput [docs/s]     17354
                                  Indexing time [min]   65.2369
                                     Merge time [min]   14.2558
                                   Refresh time [min]   5.48265
                                     Flush time [min]   0.41285
                            Merge throttle time [min]   1.16163

Query latency country_agg_cached (90.0 percentile) [ms]    1.4993
Query latency country_agg_cached (99.0 percentile) [ms]   1.64125
Query latency country_agg_cached (99.9 percentile) [ms]   5.20068
Query latency country_agg_cached (100 percentile) [ms]   5.48076
Query latency default (90.0 percentile) [ms]   39.4852
Query latency default (99.0 percentile) [ms]   50.1238
Query latency default (99.9 percentile) [ms]   51.3614
Query latency default (100 percentile) [ms]   55.6018
Query latency scroll (90.0 percentile) [ms]    20.775
Query latency scroll (99.0 percentile) [ms]   21.4549
Query latency scroll (99.9 percentile) [ms]   21.7414
Query latency scroll (100 percentile) [ms]   21.8215
Query latency term (90.0 percentile) [ms]    1.4881
Query latency term (99.0 percentile) [ms]   4.03789
Query latency term (99.9 percentile) [ms]   5.02154
Query latency term (100 percentile) [ms]   5.15831
Query latency phrase (90.0 percentile) [ms]   1.90016
Query latency phrase (99.0 percentile) [ms]   4.76495
Query latency phrase (99.9 percentile) [ms]   5.29573
Query latency phrase (100 percentile) [ms]   8.38323
Query latency country_agg_uncached (90.0 percentile) [ms]   264.426
Query latency country_agg_uncached (99.0 percentile) [ms]   304.563
Query latency country_agg_uncached (99.9 percentile) [ms]   332.257
Query latency country_agg_uncached (100 percentile) [ms]   366.988
Median CPU usage (index) [%]     692.5
Median CPU usage (stats) [%]     142.9
Median CPU usage (search) [%]    217.15
Total Young Gen GC [s]    53.877
Total Old Gen GC [s]     8.131
Index size [GB]   3.24356
Totally written [GB]   18.3658
Heap used for segments [MB]   21.3325
Heap used for doc values [MB]  0.112911
Heap used for terms [MB]   19.7005
Heap used for norms [MB]  0.078125
Heap used for points [MB]  0.771404
Heap used for points [MB]  0.669525
Segment count       101
Indices Stats(90.0 percentile) [ms]    4.2005
Indices Stats(99.0 percentile) [ms]   4.53104
Indices Stats(100 percentile) [ms]   4.67669
Nodes Stats(90.0 percentile) [ms]   4.31434
Nodes Stats(99.0 percentile) [ms]    4.7264
Nodes Stats(100 percentile) [ms]   5.35763

Maybe a screen shot:

Thoughts in this are:

Choose your output format eg --output=(json | markdown | summary)
Choose your output destination --output-destination=(screen | file | elasticsearch)

Choose your output format eg --output=(json | markdown | summary)

I think this is a good idea and I sense the default should be Markdown. Btw, I also had a short look on this and it seems that tabulate supports Markdown with `tablefmt="pipe" (see also the tabulate page on PyPI).

Choose your output destination --output-destination=(screen | file | elasticsearch)

All data are already kept in the metrics store. If your metrics store is Elasticsearch you already have everything. I would just write the report to stdout and the log file (which is already the case). It could make sense though to support an option to write the report directly to a file.
What do you think?

I have now added two new options:

--report-format: markdown (default) or csv
--report-file: which specifies a file to write the report to. The report will always be shown on the console in the selected report format


ConnectionError: Login Failed for user

This is a guideline to help us to understand SQLTools Issues.
You don't need to stick to this, but please try to guide us to reproduce the errors or understand your feature requests.

Issue Type

[ ] Feature Request
[x] Bugfix/Errors
[ ] Questions
[ ] Other

Prerequisites (For bugfixes)

[x] Are you running the latest version?
[x] Did you check the logs?
[x] Did you check the Setup?

Description
As stated in issue nr 65. I'm still not able to connect to sql server. Resolving Issue #60 didn't bring a solution to my problem. I'm using the following information to connect:
"sqltools.connections": [
{
"name": "Test",
"server": "svr-t-evasql01.otapp.local",
"dialect": "MSSQL",
"port": 1433,
"database": "EvaSfh",
"username": "\Jeordiep",
"askForPassword": false,
"connectionTimeout": 60
}
]
I use integrated security = true to connect to sql server
The following error occurs in the log:
Query error: { ConnectionError: Login failed for user '\Jeordiep'.
at ConnectionError (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\errors.js:12:12)
at Parser. (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\connection.js:507:33)
at emitOne (events.js:96:13)
at Parser.emit (events.js:191:7)
at Parser. (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\token\token-stream-parser.js:54:15)
at emitOne (events.js:96:13)
at Parser.emit (events.js:191:7)
at addChunk (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:284:12)
at readableAddChunk (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:271:11)
at Parser.Readable.push (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:238:10)
message: 'Login failed for user '\Jeordiep'.',
code: 'ELOGIN' }
Error while preparing columns completions { ConnectionError: Login failed for user '\Jeordiep'.
at ConnectionError (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\errors.js:12:12)
at Parser. (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\connection.js:507:33)
at emitOne (events.js:96:13)
at Parser.emit (events.js:191:7)
at Parser. (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\tedious\lib\token\token-stream-parser.js:54:15)
at emitOne (events.js:96:13)
at Parser.emit (events.js:191:7)
at addChunk (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:284:12)
at readableAddChunk (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:271:11)
at Parser.Readable.push (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.0\node_modules\readable-stream\lib_stream_readable.js:238:10)
message: 'Login failed for user '\Jeordiep'.',
code: 'ELOGIN' }

I have to use a domain name when logging in so: \jeordiep. I also tried it with double \

You are right. i forgot to consider de domain parameter in the options as de TDS driver needs (http://tediousjs.github.io/tedious/api-connection.html#function_newConnection).
I'll release a fix for it later today, ok? Then you will be able to connect using this setup:
{
  "name": "Test",
  "server": "svr-t-evasql01.otapp.local",
  "dialect": "MSSQL",
  "port": 1433,
  "database": "EvaSfh",
  "domain": "domainname",
  "username": "Jeordiep",
  "askForPassword": false,
  "connectionTimeout": 60
}

if you are using an azure server, you probably need to include the encrypt flag as following:
{
  "name": "Test",
  "server": "svr-t-evasql01.otapp.local",
  "dialect": "MSSQL",
  "port": 1433,
  "database": "EvaSfh",
  "domain": "domainname",
  "username": "Jeordiep",
  "askForPassword": false,
  "connectionTimeout": 60,
  "dialectOptions": {
    "encrypt": true
  }
}

Closed on #68.

Hi I'm still not able to connect: I'm setting up a connection using the connection wizard (great add on). After that I'm trying to connect. The log shows me this error:
[2018-03-05T07:49:34.][v0.12.1] ERROR: Error selecting connection:
[2018-03-05T07:49:34.][v0.12.1] Error: Connection got disposed.
at Object.dispose (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\main.js:825:25)
at Object.dispose (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-languageclient\lib\client.js:57:35)
at LanguageClient.handleConnectionClosed (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-languageclient\lib\client.js:1864:38)
at LanguageClient.handleConnectionClosed (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-languageclient\lib\main.js:106:15)
at closeHandler (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-languageclient\lib\client.js:1852:18)
at CallbackList.invoke (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\events.js:71:39)
at Emitter.fire (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\events.js:135:36)
at closeHandler (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\main.js:221:26)
at CallbackList.invoke (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\events.js:71:39)
at Emitter.fire (C:\Users\jeordiep.vscode\extensions\mtxr.sqltools-0.12.1\node_modules\vscode-jsonrpc\lib\events.js:135:36)


test 24746 CI

sanitycheck -t samples/posix/gettimeofday/ locally doesn't catch the fact that the example produces a build diagnostic.  See if real CI does.

confirmed


TSnake41/raylib-lua updated to 3.5

raylua v3.5a has been released

@TSnake41 Nice! Thanks for the update! 👍😄


Need trading/ evolve pokemon without trading

I put Pokémon blue on GPC and I can’t evolve some of my Pokémon without trading which I don’t think is possible. Any solutions?

I think the only way to address this would be with link cable support (#55) or manipulating save states (#46). With that in mind, I'm going to close this as a 'duplicate' (which isn't entirely true, but...).


Jilemnice is missing

Some places disappeared. Check if it is because of wrong join or not in data.

Fixed in #156


Use RBAC permissions to provide user access to pages

As a user, I expect UI pages to be shown according to my user's entitlements and RBAC permissions. Pages should only be shown if the following conditions are met.

The user has entitlements
If the user is an org admin, all pages are available
A page is shown if the user has RBAC permissions for specific resource types

As an example, consider the cost-demo user. Since that user is an org admin, they should have access to all pages.
The cost-user user is not an org admin and only has the cost_model resource type defined. In this scenario, the user only has access to the cost models page.
When users don't have permissions, they see the NotAuthorized empty state.
For more details, see https://issues.redhat.com/browse/COST-646
The cost-user user and the overview page

The cost-user user and the cost models page

Codecov Report

Merging #1709 into master will decrease coverage by 1.34%.
The diff coverage is 28.82%.

@@            Coverage Diff             @@
##           master    #1709      +/-   ##
==========================================
- Coverage   77.72%   76.37%   -1.35%
==========================================
  Files         246      251       +5
  Lines        4036     4136     +100
  Branches      779      796      +17
==========================================
+ Hits         3137     3159      +22
- Misses        815      893      +78
  Partials       84       84

Impacted Files
Coverage Δ

src/utils/permissions.ts
22.97% <22.97%> (ø)

src/app.tsx
44.15% <26.31%> (-15.85%)
:arrow_down:

src/utils/permissionsComponent.tsx
46.66% <46.66%> (ø)

src/routes.tsx
90.47% <100.00%> (+1.58%)
:arrow_up:

src/pages/state/maintenance/index.ts
100.00% <0.00%> (ø)

src/pages/state/notAuthorized/index.ts
100.00% <0.00%> (ø)

src/pages/state/notAvailable/index.ts
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aaa8b47...b637c6f. Read the comment docs.


remove unused direct and proxy connection handlers

These two files look to be unused, so it seems helpful to remove them.

@keep94 this is a small change extracted from #120 🙇♂


move source code for microsites over to angular/microsites

...and populate a README that directs folks to file issues here, and outlines how the branches work.

I'll bring over the sites and make sure the build process is sane. I'll also document it, so that it is easy to use.

Can be found https://github.com/angular/microsites


Upgrade command to ignore software that has no version

Description of the new feature / enhancement
As I use winget upgrade --all extensively (almost every day), I end up "upgrading" software that might not even need upgrading. This is due to the fact that some of the software apparently doesn't expose their version info correctly (as shown in picture below).
Thus, I suggest an extra option that could skip the software that has no version to compare to - and then I could skip them if needed to - most of the time ending up to download lots of bytes for no reason.

Proposed technical implementation details
If (version == "Unknown" && options.Contains('skipUnknownVersions')) { skipToNext() }? Something like this?

Your version of Winget must be out of date, ignoring unknown versions is already a thing

This has been resolved in the current release.


pull request for e2e test at 01/13/2017 07:26:13

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


TypeError: gitChangedFiles is not a function

Hi Kevin,
When running the command with the --staged flag, it throws TypeError: gitChangedFiles is not a function. That's happening because the gitChangedFiles is the default library's export.
When changing it the gitChangedFiles.default() it works. The issue is in this line. The * should be removed.

Awesome! Thx for creating this issue and the research! Will try to fix this soon

Should be fixed with version 1.4.4. Thx for reporting this. Let me know if somethings not working yet.

It works great, thanks. I'd also consider adding a --silent option.


Not found handler for subrouters

Consider this example:
func Create() {
  r := chi.NewRouter()
  r.NotFound(notFound) // weird, if you set NotFound after Mount - it won't affect subrouter at all
  r.Mount("/api", protected(authService))
}

func protected() chi.Router {
  r := chi.NewRouter()
  r.Route("/group", func(r chi.Router) {
     ....
  }
  return r
}

func notFound(rw http.ResponseWriter, r *http.Request) {
  rw.Write("my not found")
}

Now let's try some urls:
/unknown - "my not found"
/api/unknown - "my not found"
/api/group/unknown - "default not found"

It's very unexpected behavior for me. There is workaround, I need to set my not-found handler for protected router explicitly too. Maybe on mount it's better to go through all subrouters and set not-found router there? If that's alright, I can make pull request.

@smacker a PR would be great, at a minimum it will help understand possible situations better.

perhaps write a quick one to hack it, see how you can make it work as you expect, and we can iterate/refine from there

Done. Please take a look. If the problem isn't clear from my description - try tests from PR on master branch. Thanks!


test: Bump version in TestReleaseImages

This is so that the installation step doesn't install libvirt.

LGTM


Context menu grouping is broken

Does this issue occur when all extensions are disabled?: No, issue is with extension contributed context menus.

VS Code Version: 1.89.0-insider 2fb3fa8dd2f85d7f452066f8a64e8bf259d21412
OS Version: Mac OS X 10_15_7 (ARM)

Steps to Reproduce:

Open a context menu in Stable and compare it to the same menu in Insiders. I verified the issue reproduces for multiple extensions including GitLens.

Stable:

Insiders:

If this is a confirmed regression, it would help to know which change caused it. We maintain a node.js based CLI tool  vscode-bisect that automatically downloads and runs previous VS Code insiders and asks for each build if the issue reproduces. It takes up to 8 steps to find exactly the build that caused the regression.
Steps:

install node.js (if not yet done)
npx --yes @vscode/vscode-bisect@latest
follow the instructions until you found the offending build (its fine to leave the commit empty)
report back the commit range in this issue

Thanks!

Bisect points to https://github.com/microsoft/vscode/compare/063d715c4504380317f9ca8eefc602d71e56adb5...9cae075e983bed1020eaba0ed4deb2e9b8d71d39
My money would be on https://github.com/microsoft/vscode/commit/7b1acd91466e0be4be50629623b3902c4d5ddefa


Improve background video of container below Navbar.

Issue:
The background video of the container below the navbar is blurred. It is not looking aesthetic and not going well with the UI of the website.
Solution:
Its background can be improved and make it look good if we are adding copyright-free video from Unsplash.
@akshitagupta15june  Please assign me this issue under gssoc'23. I will fix it and make it look awesome.

its not required
thanks

It's okay


Add pages for Juno, JunoCam & JIRAM

Add basic structure/placeholder pages based on New Horizons page structure.  Initial page will have 3 tabs; Juno Mission Information, Access JunoCam & Access JIRAM using our convention, that page will sit in a directory "Juno", index.html, junocam.html, jiram.html, all w/tabs.  Note - at the moment, we don't need the tabs until we have the tables and additional data for the rest of the pages.
"We are in the process of integrating the JunoCam and Juno JIRAM data into our systems.  We will provide more detailed pages and links, but for now, you can access the ... :
Links in viewmaster:
{Raw JIRAM Images] https://pds-rings.seti.org/viewmaster/volumes/JNOJIR_1xxx
[Reduced JIRAM Images] https://pds-rings.seti.org/viewmaster/volumes/JNOJIR_2xxx
[JunoCam Jupiter Image Collection] https://pds-rings.seti.org/viewmaster/volumes/JNOJNC_0xxx


[RFC] Overhaul and Upgrade Link Definition Management

This is a request for comment on an evolution to the way in which link definitions are handled by hosts and within a lattice.
Motivation
The current state of link definition management is not nearly robust enough to support real-world, production workloads. It is far too easy for the state of host caches and stream caches to diverge over time and to potentially cause multiple instances of the same provider to have differing states. Further, dealing with link definition removal is awkward at best, and there's really no good way to support editing link definitions other than a non-atomic delete and put pair.
At any given time, you can inspect the contents of the link definition cache (a JetStream stream) and not be entirely sure if the running capability providers have the same view of the world. Worse, due to the "fire and forget" nature of much of this process (illustrated below in a sequence diagram), today we do not have a reliable way of detecting when application of link definitions fail or succeed.
Current State
There are a couple of important flows necessary for understanding how link definitions work in the current (0.57.x) version of wasmCloud.

Host Startup - When a host starts, it will create an ephemeral consumer on the cache stream and receive all known link definitions, ignoring those with the deleted flag set to true.
Provider Startup - When a capability provider first starts, the host will supply it with a list of known applicable link definitions for that provider+link name. This data comes from the host's own in-memory cache of link definitions which is populated by virtue of an always-on subscription to the linkdef cache.
Control Interface Request - An explicit request made on the wasmCloud control interface topic by a consumer. This request can be to put or remove a link definition. As mentioned below, put (or even remove in narrow edge cases) can cause divergence.

The following sequence diagram illustrates the basic flow by which a link definition is published and ultimately set in a capability provider.
sequenceDiagram
    participant CC as Control Client
    participant H as Host (ctl server)
    participant P as Provider
    participant C as Cache Stream
    CC->>+H: Put  link definition
    H-->>P: Advertise Link
    H-->>C: Put Link
    H->>-CC: ACK

In this flow, a consumer utilizing a control interface client makes a request to the entire lattice, asking that a given link definition be put. A psuedo-randomly chosen host (via queue subscription) handles the request. It then publishes the link definition advertisement message, which may or may not be delivered to a waiting provider because providers are not required to be online when link definitions are set. Additionally, another message is published on a stream topic on which the lattice cache stream is listening.
Because the cache stream will, in the absence of low-level network failure, always accept the incoming message while capability providers can fail to accept the message or (hopefully rarely) panic during the application of a link definition, the link definitions known to capability providers and those known to the host and those known to the lattice cache can all diverge.
Link Definition Cache
The link definition cache is a JetStream stream. This stream will be automatically created by a host on startup if it does not yet exist. By default, this stream is in-memory and will completely disappear when the host for the given JS domain stops or is rebooted.
📖 It's worth pointing out that this link definition cache feature actually predates the formalization of key-value and object stores on top of NATS's JetStream functionality. The raw stream was literally the only tool available to us at the time w/out adding another product into the list of requirements.
Recommended Changes
The following is a list of changes that we think will correct the problems outlined at the beginning of this RFC.
Convert Cache to NATS Key-Value Bucket
This would involve no longer using the LATTICECACHE_{lattice-id} raw stream and instead using a key-value bucket. The key for the link definition would remain similar to the topics used in the stream today as a hash of the following data:

Actor public key
Provider public key
Provider link name

Providers currently only expose an implementation for a single contract and so the contract ID is technically not a piece of uniquely differentiating information. If, at some point in the distance future, we ever allow providers to support multiple contracts (which this author currently opposes), then we can always revisit the hash key at that point.
We get a couple of really good benefits from this approach out of the box. First, this gives us atomic read/write for each individual link definition. Second, we gain the ability to query the list of all stored link definitions by asking for the key list. We also get a robust history mechanism built-in, which comes in handy for the next section.
Note: we're going to want to move the claims cache over from the legacy "stream" style access mode to a key-value bucket as well. Right now we store a 1-deep "most recent wins" claims JSON payload for each public key we discover/start. We can probably use the same general-purpose lattice cache bucket and migrate to query model versus push.
Emit and Store Errors and Success
When a capability provider attempts to apply a link definition, it should emit a failure event indicating that the application failed and, if available, the reason for the failure. This will go out on the standard wasmbus.evt.{lattice-id} topic.
In addition to emitting success and failure, we should store this. To do so, we'll use a key in the bucket that is the hash of the link definition (as discussed above) followed by a .state suffix. This gives us the ability to atomically push state data without risking last-write-wins conflicts on the actual raw data (configuration values) of the link definition.
Using the history functionality on a key value bucket, we can actually query the history of the changes to the {hash}.state key on demand. This gives dashboards, monitors, and control interface clients the ability to not only see whether a link definition is currently active, but whether and how many times it failed. We'll limit the number of historical copies of a key to something like 5 so we can retrieve just the most relevant state changes for a given link definition.
The combination of having a predictable key and a useful and authoritative history of state changes feeds directly into the next section.
Use the Key-Value Bucket as the Source of Truth
A problem with the current implementation of link definitions it that we have conflicting sources of truth. There is the in-memory cache of each host, the contents of the lattice cache stream, and the list of valid and accepted link definitions within each capability provider's memory.
The state of a link definition might have the following values (subject to change during actual implementation):

Idle - the link definition exists, the most recent state is not an error, and one or more of the participants (actor or provider) are not online/discoverable
Active - as above, but both the relevant actor and provider each have at least one running and discoverable instance in the lattice.
Error - the most recent state of the link definition is a failure. This state occurs regardless of the online status of link participants.

The above states can correspond roughly to a "traffic light" indicator where active is green, error is red, and idle is yellow. Idle being yellow indicates that nothing is wrong, but it can't be used yet (e.g. "not ready").
During host start up, the host could simply query the cache for the data it needs.
The host could periodically true-up against the data in the key-value bucket cache to examine state (e.g. to purge failed link definitions) and to ensure that the host knows about all of the current link definitions.
Finally, treating the linkdef cache this way allows external entities to write configuration values. For example, there may be situations where we want to preload link definitions with secrets so that clients using wash or the control interface client don't see that data (encrypting link definitions a discussion for another day.
Default to Disk Backed
When creating the key-value bucket for the first time, default to a persistent, disk-backed bucket. This may be the default in NATS now, but this is being included here for completeness to show that we're explicitly deciding we want this cache to be durable.
Rationale and Alternatives
The rationale for each decision is explained in-context in all of the above discussion. There were no alternatives considered other than using an external cache other than NATS (which was quickly rejected).
Drawbacks
There are no drawbacks here. Determining the current state of a link definition may require a tiny calculation, but since we don't have this capability today, that hardly seems like a drawback.
If a failure occurs when attempting to write the state data (e.g. a failure to accept the link by the provider), and we don't retry until it works, then we will introduce an inconsistency. This will be no worse than it is today, but we'll need to make sure we implement the appropriate "guaranteed write" logic when storing state changes.

I edited the RFC to indicate that the claims cache basically comes along for the ride here since we're using the same stream right now for claims and link defs (the stream listens on the lc.> prefix)

I know that https://github.com/wasmCloud/wasmcloud-otp/issues/361 has plagued us for a while, how do we think this could address this issue? Should capability providers be, rather than delivered link definitions at startup introducing this race condition, directly querying the keyvalue bucket instead? I may have missed this specifically in the RFC but this would be awesome to eliminate while we're at it.
For drawbacks, I would only add that this will be a breaking change for capability providers and we may create an incompatibility where capability providers will not work between versions of the host that support the stream and versions of the host that support the kv bucket.

Another question that we never really solved with actor health requests, how do we deal with a link definition where one instance of a provider succeeded and one instance failed to establish the linkdef? In this case, the link definition would both be Active and Error (or Idle) but that would depend on which provider you query 🤔
In theory, this could be a flaw in the provider design itself, but it is possible.

@autodidaddict I believe this was partially addressed with the Jetstream KV changes. We're going to need to prioritize the linkdef success/error state issue then

I am going to close this RFC as we've been running on "real" key-value store link definition data for a while now, and that addresses one of the main concerns of this RFC. I still think there is a lot that can and should be done to overhaul the way in which link definitions are managed within a lattice, but I think that merits an entirely new RFC.

We're going to need to prioritize the linkdef success/error state issue then

I've been doing a lot of thinking about this and the more I think about it the more this feels like addressing a symptom rather than the problem. I'm hoping there's a more holistic way to take care of some of the friction in linkdef management.


Update cats-effect, cats-effect-laws to 1.3.0

Updates

org.typelevel:cats-effect
org.typelevel:cats-effect-laws

from 1.2.0 to 1.3.0.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a nice day!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [{ groupId = "org.typelevel" }]

Codecov Report

Merging #281 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #281   +/-   ##
=======================================
  Coverage   93.44%   93.44%
=======================================
  Files          15       15
  Lines         747      747
  Branches       62       62
=======================================
  Hits          698      698
  Misses         49       49

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64f002d...67cd861. Read the comment docs.

Codecov Report

Merging #281 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #281   +/-   ##
=======================================
  Coverage   93.44%   93.44%
=======================================
  Files          15       15
  Lines         747      747
  Branches       62       62
=======================================
  Hits          698      698
  Misses         49       49

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 64f002d...67cd861. Read the comment docs.


Can't backspace or delete after selecting all text in the input field

Demo with latest tagged releases: http://plnkr.co/edit/nWoLd5d4jFScWBcUjvBq?p=preview
Demo with master branches: http://plnkr.co/edit/h0H9wAypPw27v0bXtrSS?p=preview

Can you update the Plunker to be based off of the latest releases?

Sure: http://plnkr.co/edit/KNGnFyMnwZsgrN8maIU0?p=preview
Seems to be mostly fixed except that left arrow now acts like backspace when you have fully selected the input text.

Issue still happens when reset-search-input attribute is set to false. Select all and backspace doesn't work


Please build the library and components using node make generic components

I know I am just being dense here, but I followed the build instructions and built using:
node make generic components

I then copied the build folder into my project and included the pdf.js file:
<script src="/backend/newdesign/assets/js/pdfjs/build/pdf.js"></script>

This is for version:
PDFJS.version = '1.3.148';
PDFJS.build = '4e9ea35';

And then I get the pop-up saying:
Please build the library and components using node make generic components

Can you kindly let me know what I am doing wrong? Thanks!

If you are using PDFViewer or PDFPageView, you also need to add "/backend/newdesign/assets/js/pdfjs/build/components/pdf_viewer.js". For better results, use pdf.js, pdf.worker.js and pdf_viewer.js included in the pdfjs-dist
Closing as answered.

That worked great, thank you!


fix: return null when encryption is invalid rather than throw exception

Proposed changes
Issue Number: close #xxx
Problem summary
Describe your changes.
Checklist(Required)

Does it affect the original behavior:

[ ] Yes
[ ] No
[ ] I don't know

Has unit tests been added:

[ ] Yes
[ ] No
[ ] No Need

Has document been added or modified:

[ ] Yes
[ ] No
[ ] No Need

Does it need to update dependencies:

[ ] Yes
[ ] No

Are there any changes that cannot be rolled back:

[ ] Yes (If Yes, please explain WHY)
[ ] No

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 35.25 seconds
load time: 549 seconds
storage size:<PHONE_NUMBER>1 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230130062742_clickbench_pr_86842.html


Add a note about token expiry for managed identity for Azure MySQL

Its unclear from the page whether this token expires (say after 60 mins) so if any connecting programs need to:

Reconnect to MySQL after the token expires, and/or
Ensure that before making a new connection, a fresh token is retrieved from http://<IP_ADDRESS>/metadata/identity/oauth2/token

This information is important because if we're trying to make a legacy or third-party application talk to MySQL using a Managed Identity Token, it's unclear if we can do this by simply adding a startup script that retrieves a fresh token and inserts it into the application config, or if we need to make changes to the application itself so that it periodically retrieves tokens for use.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f9784fd1-8949-dac6-4c79-26dfb335ffb3
Version Independent ID: 6137488f-0b66-0854-e882-e2f629f6e973
Content: Connect with Managed Identity - Azure Database for MySQL
Content Source: articles/mysql/howto-connect-with-managed-identity.md
Service: mysql
GitHub Login: @lfittl-msft
Microsoft Alias: lufittl

@dev-d  We are checking if we can enhance the document with this information.

@dev-d  We are checking if we can enhance the document with this information.

@lfittl-msft Can you please verify the token expiration for the Managed Token.

@lfittl-msft Can you please verify the token expiration for the Managed Token.

Adding @dimadhus for this feedback

We sincerely apologize for the delayed response to this issue. I collaborate with the owners of the GitHub Issues related to PostgreSQL to help either get them resolved or determine how to update our documentation.
As such, is this GitHub Issue still a concern for you? If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion, and we will reopen the issue.
Due to the age of this issue, we will continue to close this issue.
Please visit https://docs.microsoft.com/en-us/answers/support/ for more information on asking questions about the product or service.
Thank you.
#please-close


Training slower in beta6

Issue Description
I've just upgraded from beta5 to beta6 and am finding that each training iteration now takes about 50% longer than it did before.
I am using MultiLayerNetwork.fit passing an INDArrayDataSetIterator
Version Information

Deeplearning4j version -  1.0.0-beta6
Platform - Ubuntu 18.04.3 LTS
CUDA - 10.1

Please let me know if you require any further information to help diagnose the issue.
Thanks, Damon

Can you post your network please?

Hi,
The simplified network below has an even bigger performance difference between beta5 and beta6 (about 3-4x slower).
It seems to be something to do with the amount of data per batch.  With batch size 64 this is 64 * 1024 * 128 * 3 * 4 = 100,663,296 bytes.  However if I reduce the batch size by 1 to 63 (99,090,432 bytes) then the performance is the same in both versions.  (Normally increasing the batch size makes it faster so I wonder if it is exceeding some internal buffer limit.)
Regards, Damon
public static MultiLayerConfiguration createDummyNetwork() {
	int id = 0;
	MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder() //
			.seed(12345) //
			.weightInit(WeightInit.XAVIER_UNIFORM) //
			.updater(new Adam(7.8125e-5)) //
			.activation(Activation.RELU) //
			.list() //
			.layer(id++, new SubsamplingLayer.Builder(new int[] { 32, 32 }, new int[] { 32, 32 }, new int[] { 1, 1 }) //
					.name("pool1") //
					.build()) //
			.layer(id++, new OutputLayer.Builder() //
					.name("output") //
					.nIn(384) //
					.nOut(8192) //
					.activation(Activation.SIGMOID) //
					.lossFunction(LossFunctions.LossFunction.XENT).build()) //
			.backpropType(BackpropType.Standard) //
			.setInputType(InputType.convolutional(1024, 128, 3)) //
			.build();
	return conf;
}

can you make OutputLayer microscopic please?

Nevermind. you're on CUDA.
What's your GPU model name? Are you using cuDNN? Can i see your pom.xml?

2020-04-23 23:59:55,248 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - ND4J CUDA build version: 10.1.243
2020-04-23 23:59:55,250 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112]
pom.xml.zip

Hmmm..Your pom.xml looks ok. Can you please post more of your console output?

beta5
2020-04-24 00:15:19,141 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
ND4J CUDA build version: 10.1.243
CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112];
2020-04-24 00:15:22,246 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for OpenMP: 32
2020-04-24 00:15:22,294 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 0
2020-04-24 00:15:22,305 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Linux]
2020-04-24 00:15:22,305 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [32]; Memory: [4.0GB];
2020-04-24 00:15:22,305 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
2020-04-24 00:15:22,306 [main] INFO org.nd4j.linalg.jcublas.ops.executioner.CudaExecutioner - Device Name: [GeForce GTX 1080]; CC: [6.1]; Total/free memory: [8512602112]
2020-04-24 00:15:22,352 [main] INFO org.deeplearning4j.parallelism.ParallelInference - Initializing ObservablesProvider...
2020-04-24 00:15:24,846 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:24,946 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,005 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,008 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,016 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,306 [application-akka.actor.default-dispatcher-2] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
2020-04-24 00:15:25,737 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,754 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,759 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,787 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,795 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,840 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,845 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,847 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,854 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,865 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,872 [main] WARN application - system properties: play.crypto.secret is deprecated, use play.http.secret.key instead
2020-04-24 00:15:25,886 [main] INFO play.api.Play - Application started (Prod) (no global state)
2020-04-24 00:15:26,230 [main] INFO play.core.server.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
2020-04-24 00:15:26,231 [main] INFO org.deeplearning4j.ui.play.PlayUIServer - DL4J UI Server started at http://localhost:9000
2020-04-24 00:15:26,234 [main] INFO org.deeplearning4j.ui.play.PlayUIServer - StatsStorage instance attached to UI: InMemoryStatsStorage(uid=1a3db575)
2020-04-24 00:15:26,248 [main] INFO org.deeplearning4j.nn.multilayer.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
2020-04-24 00:15:33,429 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 1 complete in 4s
2020-04-24 00:15:37,173 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 2 complete in 3s
2020-04-24 00:15:40,824 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 3 complete in 3s
2020-04-24 00:15:44,375 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 4 complete in 3s
2020-04-24 00:15:47,783 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 5 complete in 3s
2020-04-24 00:15:51,297 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 6 complete in 3s

beta6
2020-04-24 00:11:45,182 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
2020-04-24 00:11:47,923 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 32
2020-04-24 00:11:47,962 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 0
2020-04-24 00:11:47,973 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Linux]
2020-04-24 00:11:47,973 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [32]; Memory: [4.0GB];
2020-04-24 00:11:47,973 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
2020-04-24 00:11:47,983 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - ND4J CUDA build version: 10.1.243
2020-04-24 00:11:47,984 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112]
2020-04-24 00:11:48,025 [main] INFO org.deeplearning4j.parallelism.ParallelInference - Initializing ObservablesProvider...
2020-04-24 00:11:50,461 [vert.x-eventloop-thread-0] INFO org.deeplearning4j.ui.VertxUIServer - Deeplearning4j UI server started at: http://localhost:9000
2020-04-24 00:11:50,471 [main] INFO org.deeplearning4j.ui.VertxUIServer - StatsStorage instance attached to UI: InMemoryStatsStorage(uid=d667b772)
2020-04-24 00:11:50,494 [main] INFO org.deeplearning4j.nn.multilayer.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
2020-04-24 00:12:04,670 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 1 complete in 11s
2020-04-24 00:12:19,373 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 2 complete in 14s
2020-04-24 00:12:33,148 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 3 complete in 13s
2020-04-24 00:12:47,343 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 4 complete in 14s
2020-04-24 00:13:01,483 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 5 complete in 14s
2020-04-24 00:13:15,760 [main] INFO nz.argonaut.rva.core.NeuralNetwork - Epoch 6 complete in 14s

One more thing to try here to help isolate the cause of the problem: Can you give the same output without the UI enabled (i.e., no UIServer.getInstance()) or UI listener (StatsListener) added to the network? Thanks
The UI (StatsListener) has a fair bit of overhead, especially if the frequency is small

Hm, why there's no cuDNN helpers messages? Are you sure deeplearning4j-cuda dependency is actually enabled?

One more question - are you using Eclipse IDE by any chance?

Disabling the UI and stats makes no difference.
I don't think it's using cuDNN.  I also remember there being logging about that so I think I've lost that after a few version updates and switching from Mac OSX to Linux.  How can I  confirm/debug this?  Is there a bit of code I can step through to see if/why it isn't finding CuDNN?  The CuDNN libs are in my Cuda installation (as per NVIDIA instructions).  I also tried using the javacpp presets but that didn't make a difference either.
I was running from within Eclipse but have just tried it outside and the behaviour is the same.

Disabling the UI and stats makes no difference.

I wish this was true, but we know for sure, that UI is very expensive computationally. So if it doesn't make any difference for you - it just means you're doing something wrong.

If I remove the deeplearning4j-cuda dependency then I get a log message about cuDNN not found, so that suggests that it is using cuDNN.  However there is no difference in performance with or without cuDNN.  Is that what you'd expect for the above dummy network (with no CNN layers)?
Should there be cuDNN helper messages in the log?  Where is the code that writes that?
Alex, why did you ask about Eclipse?

I added a CNN layer to the network and the deeplearning4j-cuda dependency makes it about 3 times faster so it's definitely using cuDNN.
Both with and without cuDNN beta6 is a lot slower than beta5 when the batch size is 64 but isn't slower with a batch size of 63 (or less).

Hm. Can you please try the same network on the same hardware but use snapshots instead of beta6?

I'm getting 8s per epoch with snapshot which is faster than beta6 but still double what I get in beta5.  Also no longer seeing an improvement lowering the batch size.  This is with Cuda 10.1 and I'm getting a message "Warning: Versions of org.bytedeco:javacpp:1.5.3 and org.bytedeco:cuda:10.1-7.6-1.5.2 do not match." so I will try Cuda 10.2.  This means upgrading the NVIDIA driver which I'm having trouble with so I'll get back to you when that's working.
If Cude 10.2 doesn't fix the issue (with either snapshot or beta6) then I'll create a standalone example that I can send you.

Hm. That sounds weird. Okay, we'll investigate this then. Thanks for flagging this issue.

I haven't been able to try Cuda 10.2 yet.
Here's a standalone example.  (Just a pom file and 1 Java class.)
dl4jperftest.zip
These are the times that I am seeing.  Please let me know if you're able to reproduce the issue on your hardware.
beta5
2020-04-26 11:18:14,165 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
ND4J CUDA build version: 10.1.243
CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112];
2020-04-26 11:18:17,284 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for OpenMP: 32
2020-04-26 11:18:17,331 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 0
2020-04-26 11:18:17,343 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Linux]
2020-04-26 11:18:17,343 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [32]; Memory: [15.7GB];
2020-04-26 11:18:17,343 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
2020-04-26 11:18:17,343 [main] INFO org.nd4j.linalg.jcublas.ops.executioner.CudaExecutioner - Device Name: [GeForce GTX 1080]; CC: [6.1]; Total/free memory: [8512602112]
2020-04-26 11:18:18,704 [main] INFO org.deeplearning4j.nn.multilayer.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
Epoch  1 took  3625ms ( 3594 +    31)
Epoch  2 took  2098ms ( 2055 +    43)
Epoch  3 took  2087ms ( 2050 +    37)
Epoch  4 took  2125ms ( 2080 +    45)
Epoch  5 took  2065ms ( 2019 +    46)
Epoch  6 took  2084ms ( 2042 +    42)
Epoch  7 took  2078ms ( 2041 +    37)
Epoch  8 took  2079ms ( 2040 +    39)
Epoch  9 took  2055ms ( 2006 +    49)
Epoch 10 took  2069ms ( 2033 +    36)

Average = 2236ms

beta6
2020-04-26 11:19:40,336 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
2020-04-26 11:19:43,527 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 32
2020-04-26 11:19:43,580 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 0
2020-04-26 11:19:43,596 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Linux]
2020-04-26 11:19:43,596 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [32]; Memory: [15.7GB];
2020-04-26 11:19:43,596 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
2020-04-26 11:19:43,663 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - ND4J CUDA build version: 10.1.243
2020-04-26 11:19:43,665 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112]
2020-04-26 11:19:45,020 [main] INFO org.deeplearning4j.nn.multilayer.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
Epoch  1 took  4690ms ( 4662 +    28)
Epoch  2 took  3397ms ( 3361 +    36)
Epoch  3 took  3878ms ( 3842 +    36)
Epoch  4 took  3906ms ( 3867 +    39)
Epoch  5 took  3692ms ( 3650 +    42)
Epoch  6 took  3653ms ( 3617 +    36)
Epoch  7 took  3636ms ( 3602 +    34)
Epoch  8 took  3882ms ( 3833 +    49)
Epoch  9 took  3821ms ( 3784 +    37)
Epoch 10 took  3843ms ( 3799 +    44)

Average = 3839ms

I've just run that using SNAPSHOT and it's faster!
Warning: Versions of org.bytedeco:javacpp:1.5.3 and org.bytedeco:cuda:10.1-7.6-1.5.2 do not match.
2020-04-26 11:36:19,177 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
2020-04-26 11:36:22,066 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 32
2020-04-26 11:36:22,110 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Linux]
2020-04-26 11:36:22,110 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [32]; Memory: [15.7GB];
2020-04-26 11:36:22,110 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
2020-04-26 11:36:22,120 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - ND4J CUDA build version: 10.1.243
2020-04-26 11:36:22,122 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - CUDA device 0: [GeForce GTX 1080]; cc: [6.1]; Total memory: [8512602112]
2020-04-26 11:36:23,372 [main] INFO org.deeplearning4j.nn.multilayer.MultiLayerNetwork - Starting MultiLayerNetwork with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
Epoch  1 took  4058ms ( 4023 +    35)
Epoch  2 took  2362ms ( 2330 +    32)
Epoch  3 took  1562ms ( 1525 +    37)
Epoch  4 took  1532ms ( 1496 +    36)
Epoch  5 took  1465ms ( 1432 +    33)
Epoch  6 took  1556ms ( 1520 +    36)
Epoch  7 took  1592ms ( 1550 +    42)
Epoch  8 took  1460ms ( 1429 +    31)
Epoch  9 took  1583ms ( 1549 +    34)
Epoch 10 took  1546ms ( 1508 +    38)

Average = 1871ms

What changed between this run, and previous attempt when it was still slower?

I'm getting 8s per epoch with snapshot which is faster than beta6 but still double what I get in beta5.

vs

I've just run that using SNAPSHOT and it's faster!

What was changed?

The difference is in how it's creating the NDArrays. In the latest example they reside in GPU memory. In my real code they are created in off-heap memory (and only loaded onto the GPU when I call the fit method). I found this was the fastest way to cycle through large amounts of training data.How do you normally handle large training data efficiently? I have about 500GB, with only 64GB RAM and 8GB GPU memory. I developed a custom solution that caches the NDArrays on disk and then loads them in pages of about 40GB each into off-heap memory.Sent from my Samsung Galaxy smartphone.
-------- Original message --------From: raver119<EMAIL_ADDRESS>Date: 26/04/20  7:14 PM  (GMT+12:00) To: eclipse/deeplearning4j<EMAIL_ADDRESS>Cc: damonhnz<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [eclipse/deeplearning4j] Training slower in beta6 (#8863)
What changed between this run, and previous attempt when it was still slower?
I'm getting 8s per epoch with snapshot which is faster than beta6 but still double what I get in beta5.
vs
I've just run that using SNAPSHOT and it's faster!
What was changed?
—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.
[
{
"@context": "http://schema.org",
"@type": "EmailMessage",
"potentialAction": {
"@type": "ViewAction",
"target": "https://github.com/eclipse/deeplearning4j/issues/8863#issuecomment-619499457",
"url": "https://github.com/eclipse/deeplearning4j/issues/8863#issuecomment-619499457",
"name": "View Issue"
},
"description": "View this Issue on GitHub",
"publisher": {
"@type": "Organization",
"name": "GitHub",
"url": "https://github.com"
}
}
]

Cuda 10.2 doesn't make much difference for me over 10.1.
I've done a performance comparison of Nd4j.create(float[]) between the 3 versions and beta6 is the same as beta5 but SNAPSHOT is quite a bit faster.
My attached example shows beta6 is slower than beta5 for training (for this particular network on my hardware) but the snapshot version is faster than both.
Overall I expect SNAPSHOT should be faster than beta5 and yet when I use a test case more similar to my real code it is definitely slower.  It's going to take more time for me to figure out just what else is different between my code and the simple test cases....but I've spent way too much time on this already, so I'm going to park this for now.
Whatever was slower in beta6 seems to have been more than made up for in the latest code so I'm happy if you want to close this ticket.  I can go back to using beta5 for now (since that's the fastest version for my code) and I'll wait for the next DL4J release.  If that's still slower for me than beta5 then I'll put more effort into pinpointing exactly where it is slower before raising a new ticket.
Thanks for your help.

Actually I've just tried my real code using the latest snapshot and it runs about 20% faster than beta5 (whereas beta6 is about 50% slower).  (When I created a simplified example to post on here I reduced the volume of training data significantly so I could do quick comparisons between versions.  The numbers I obtained from that weren't meaningful because it had the overheads of my caching/paging code for handling large data volumes but with only a small amount of data.)  In my real code I'm seeing the same performance improvement as in the standalone test case I posted above.
So...a 20% improvement sounds attractive to me (since my model takes several days to train and I'm still in the process of growing my training data).  What do you think is my best option for getting that performance improvement?

Wait for the next release - when is that expected?
Build off master - how stable is that right now?
Cherry-pick some performance improvements since beta6 without all the other changes - is that even viable?

Thanks, Damon

Use snapshots for a next few days.
Once we're done with performance profiling/polishing - we'll publish release. So, snapshots are almost release-ready at this point.


NCZarr fill_value is Nan instead of NaN on Windows

In this xarray's PR I found out that I'm having issues reading with python NCZarr data created on Windows.
Looks like the issue has to do with fill_value, which is Nan instead of NaN, and therefore can't be properly parsed by python's json.
Sorry, I don't have much more details as I don't have a Windows machine to explore this a bit better. I thought I would open an issue here in case you already have some ideas about what's causing the problem. I guess it's possible that the problem is actually related to netcdf4-python as I'm using it to create the NCZarr file.
This is the Python code I'm using to create the NCZarr data
from xarray.tests.test_dataset import create_test_data
ds = create_test_data()
ds = ds.drop_vars("dim3")
ds.to_netcdf("file://test_data.zarr#mode=nczarr")

Looks like a typo on my part.
You should be able to fix as follows:

edit the file netcdf-c/libnczarr/zcvt.c
Near line 308 you should see this line: strcpy(s,"Nan"); break;
Change to this: strcpy(s,"NaN"); break;

I will submit a PR for this after I hear if it fixes the problem.

I don't have an easy way to test that change on Windows.
However, I replaced myself all Nan in .zattrs and .zarray with NaN, and I can confirm that I'm able to correctly read the data. Therefore, I'm pretty sure that changing strcpy(s,"Nan") to strcpy(s,"NaN") will fix the problem and I suggest to go ahead with the PR.

See PR https://github.com/Unidata/netcdf-c/pull/2267


Networking - Enable Session Affinity in Kubeproxy for Windows

See https://github.com/kubernetes/kubernetes/pull/91701
Preview of 1.19 docs: https://deploy-preview-21534--kubernetes-io-vnext-staging.netlify.app/docs/setup/production-environment/windows/intro-windows-in-kubernetes/#load-balancing-and-services

This is available now in Kubernetes 1.19


New version: Google.Chrome.Beta version 124.0.6367.60

Created by 🥟 Dumplings in workflow run #4423.
Logs
Updated: 124.0.6367.49 → 124.0.6367.60
Submitting WinGet manifests
The manifests for the last version 124.0.6367.49 will be removed

Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

Installer Verification Analysis Context Information:
Specified hash doesn't match.
Actual hash:   BE750C3C5401E1BCDE78DF4446EC503F61CD411D8E7E335F66119EFA146BE914
Expected hash: 13cd0db18b7db3c4444334984ccc04c13f76d9f40c93216d540c7ace466d7ddf
Specified hash doesn't match.
Actual hash:   3DCF4D16E8324CEDE845D515D7B3AA6ACCC194EA7AB4B791D52F7B06C75F5282
Expected hash: 8e418b0949ee25f32e9cd4378a1dbd1f3889f4e1d1ee4e1d7f611fb251f3dddc

4/15/2024 6:20:25 PM

(Automated response - build 876.)


feat(stream) Allow temporal join on prefix of key columns.

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Allow the temporal join to only contain equi-join condition with a prefix of the key columns.

Checklist For Contributors

[ ] I have written necessary rustdoc comments
[ ] I have added necessary unit tests and integration tests
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] I have demonstrated that backward compatibility is not broken by breaking changes and created issues to track deprecated features to be removed in the future. (Please refer to the issue)
[ ] All checks passed in ./risedev check (or alias, ./risedev c)

Checklist For Reviewers

[ ] I have requested macro/micro-benchmarks as this PR can affect performance substantially, and the results are shown.

Documentation

[ ] My PR DOES NOT contain user-facing changes.

Click here for Documentation
Types of user-facing changes
Please keep the types that apply to your changes, and remove the others.

Installation and deployment
Connector (sources & sinks)
SQL commands, functions, and operators
RisingWave cluster configuration changes
Other (please specify in the release note below)

Release note

Currently the cache is setup such that the value would be a Vec<OwnedRow>, and due to the introduced change in #9134, the EstimateSize trait has to be implemented.
Implementation of EstimateSize for containers (such as HashSet) seems to be left as a TODO, and I similarly added and marked an empty generic implementation for Vec.
However, some concrete impl for Vec<u8>, Vec<u32>, etc are already added by a macro, to avoid overlapping implemenation, I enabled the specialization feature.
I doubt this is the best way to use the EstimateSize API, but since it is kind of unfinished, please feel free to correct me!


Add more customizable blocks in import.html

Problem
Fields detail and definition cannot easily be overriden in import.html unless you overrite the whole import_form block.
Solution
This PR adds sub-blocks in import_form block to allow overide forms parts more easily.

Thanks, feel free to add your name to AUTHORS

Done, thank you!

Thanks - we had an issue with an upstream lib which caused the build to fail.  I've fixed that.  I'd be grateful if you could merge the main branch and re-push your change.

Done.


HTTPError when requesting INSEE dataflow

Dear developper,
I get a HTTPError when requesting INSEE dataflow.
import sdmx
insee  = sdmx.Request('INSEE')
insee.dataflow()

results in
>>> insee.dataflow()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/bbk/.local/lib/python3.8/site-packages/sdmx/api.py", line 414, in get
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error:  for url: https://www.bdm.insee.fr/series/sdmx/dataflow/INSEE/latest

However, please note that INSEE does support the dataflow request, at the following addresses:
https://www.bdm.insee.fr/series/sdmx/dataflow/ALL/latest
https://www.bdm.insee.fr/series/sdmx/dataflow/FR1/latest
I guess a hook similar to ISTAT's would fix it, but not sure how.
Cheers,
Bertrand

@beberking good find and thanks for the report! This will be a simple fix, exactly as you say.
From the information you have given, a work around in the meantime is to use either of the following lines:
…
insee.dataflow(provider="ALL")
insee.dataflow(provider="FR1")

The first returns a slightly larger set.

Version 1.5.0 will be released after #24 is merged, and will include this fix.


fsc netcore doesnt produce full pdb

Doing fsc --debug:full doesnt work if fsc is built for .net core.
the target framework doesnt matter (netcoreapp1.1 or net451 is the same result).
zero pdb, zero warning, zero errors. it's just not created. Maybe a warning/error?
As a note from https://github.com/dotnet/netcorecli-fsc/issues/89 bug report with repro, csc netcore support full (/cc @DaveEmmerson)
Repro, using .net core sdk 1.0 rtm because is easier.
dotnet new console -lang f#
change fsproj adding `<DebugType>full</DebugType>`
dotnet restore
dotnet build -v n

the -v n show fsc args :D
Expected:
the full pdb is output dir
Result:
no pdb, no errors, no warning

Yes, full pdb’s are not built by the netcore compiler.  For F# portable pdbs have all the same information and are teeny-tiny.
Yes C# supports desktop PDBs on coreclr, this is not something we intend to address at this time.
Kevin
From: Enrico Sada<EMAIL_ADDRESS>Sent: Thursday, March 9, 2017 4:56 AM
To: Microsoft/visualfsharp<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [Microsoft/visualfsharp] fsc netcore doesnt produce full pdb (#2565)
Doing fsc --debug:full doesnt work if fsc is built for .net core.
the target framework doesnt matter (netcoreapp1.1 or net451 is the same result).
zero pdb, zero warning, zero errors. it's just not created. Maybe a warning/error?
As a note from dotnet/netcorecli-fsc#89https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Fnetcorecli-fsc%2Fissues%2F89&data=02|01|Kevin.Ransom%40microsoft.com|a3a7a5352ead41bfc26608d466daec49|72f988bf86f141af91ab2d7cd011db47|1|0|636246537828731186&sdata=yUgEaT8n0ytyd%2FpJEj1UfBtvt7NiRieceVKKFf9HV94%3D&reserved=0 bug report with repro, csc netcore support full (/cc @DaveEmmersonhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FDaveEmmerson&data=02|01|Kevin.Ransom%40microsoft.com|a3a7a5352ead41bfc26608d466daec49|72f988bf86f141af91ab2d7cd011db47|1|0|636246537828731186&sdata=Ai%2BIlEmTkhu0qZZ3DjMAumqrowwiGVV%2BQcLWi2OxD6k%3D&reserved=0)
Repro, using .net core sdk 1.0 rtm because is easier.
dotnet new console -lang f#
change fsproj adding <DebugType>full</DebugType>
dotnet restore
dotnet build -v n
the -v n show fsc args :D
Expected:
the full pdb is output dir
Result:
no pdb, no errors, no warning
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoft%2Fvisualfsharp%2Fissues%2F2565&data=02|01|Kevin.Ransom%40microsoft.com|a3a7a5352ead41bfc26608d466daec49|72f988bf86f141af91ab2d7cd011db47|1|0|636246537828731186&sdata=44sqFQL7Z6NQ9%2FX7lp1yhmbpfMFJRhBLZXXitLpUyTI%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAE76FsNXjoLJIbVsV3qJgJ23hXjVcJoPks5rj9rUgaJpZM4MX7gH&data=02|01|Kevin.Ransom%40microsoft.com|a3a7a5352ead41bfc26608d466daec49|72f988bf86f141af91ab2d7cd011db47|1|0|636246537828731186&sdata=4M5K5p%2Bs7z5KcrhY7399Vw1OWspXA9d%2B9nmA9qs1HRg%3D&reserved=0.

Closing by-design.

@KevinRansom that need a warning/error? because atm no pdb is generated and no messages about that, so can be confusing.
happy to add the warning if you think is something who may be useful.

Oh does the help still allow full and pdb only that is certainly an issue.

@KevinRansom
--debug:{full|pdbonly|portable|embedded} Specify debugging type: full, portable, embedded, pdbonly. ('full' is the default if
                                         no debuggging type specified and enables attaching a debugger to a running program,
                                         'portable' is a cross-platform format, 'embedded' is a cross-platform format
                                         embedded into the output file).

I'm not sure if we should break the cli interface. Could we print a warning instead as @enricosada suggested and change the help text? Or is there a good reason to break it?
Or maybe I understood that wrong :)

Or we could we print a warning and just generate portable PDBs?

as a note, current behaviour:
using a wrong value for DebugType property, the FscTask will not pass --debug to fsc, just -g, and no warning/errors.

we print a warning and just generate portable PDBs

warning and no pdb is better ihmo, at least i see there are no pdb generated.
The full is usually used generate  net45 assemblies/apps (legacy maybe), so using a wrong pdb format is not good (i get the error later at debug time)
As default value embedded is a good alternative, probably better than portable for the long run

Warning works for me ...

Certainly it feels like -g should generate portable

This has both resolution-by-design and bug labels.
Is this by design, or a bug? If it's by design, we should close it.

Ping - should this still be resolution-by-design?

--debug:portable on the coreclr compiler generates a portable pdb.


Add support for health check timeout configuration

What's the user value of this feature request?
Health check configuration will be made more convenient for users not familiar with the CF APIs.
Who is the functionality for?
CF users willing to configure CF health checks via the CF CLI.
Who else is affected by the change?
This feature isn't a breaking change. It suggest to extend the current options supported by cf set-health-check.
Describe the solution you'd like
The CF V3 APIs support a timeout parameter as described in the docs. I'm suggesting to add an additional option to the cf set-health-check command which is called --timeout and can be used to configure the duration in seconds that health checks can fail before the process is restarted.
Describe alternatives you've considered
A workaround is to use the CF APIs directly to set the health check timeout. This can work like this:
cf app <my-app-name> --guid #get app guid
cf curl /v3/apps/<app-guid>/processes #list app processes information
cf curl -H "Content-Type: application/json" -X PATCH -d @health-conf.json /v3/processes/<process-from-previous-command>

The content of health-conf.json could be something like this:
...
"health_check": {
        "type": "http",
        "data": {
          "timeout": 120,
          "invocation_timeout": null,
          "endpoint": "/health"
        }
      }
...

Hi @beyhan. Thank you for opening the issue. Will discuss this with the team and get back to you.


yewtube: 2.10.4 -> 2.10.5

Diff: https://github.com/mps-youtube/yewtube/compare/refs/tags/v2.10.4...v2.10.5
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[x] (Package updates) Added a release notes entry if the change is major or breaking
[x] (Module updates) Added a release notes entry if the change is significant
[x] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 252094 run on x86_64-linux 1

2 packages built:

yewtube
yewtube.dist


ChoicePrompt with ActionCards

Original issue.
I would like more detailed documentation regarding the proper use of ChoicePrompt with ActionCards.
There seems to be a duplicate when Choice contains an ActionCard, and it seems that their behavior is unpredictable.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8a9577ff-cf56-e5de-84c3-bfe10aab5368
Version Independent ID: b5d62753-f87b-4673-b09b-a5a3fd9360e8
Content: Reuse dialogs - Bot Service
Content Source: articles/v4sdk/bot-builder-compositcontrol.md
Service: bot-service
Sub-service: sdk
GitHub Login: @v-ducvo
Microsoft Alias: v-ducvo

From original issue:
Sure. I remember this from V4.3, but I can't find the code right now.

I used "very very" because a one "very" is still a card.
JSON for the short message:
{
  "attachments": [],
  "channelId": "emulator",
  "conversation": {
    "id": "3e91a080-7a59-11e9-b27e-b7f2dc2a2f8f|livechat"
  },
  "entities": [],
  "from": {
    "id": "2",
    "name": "Bot",
    "role": "bot"
  },
  "id": "4279fe40-7a59-11e9-99a0-d9c48d8a8fbc",
  "inputHint": "acceptingInput",
  "localTimestamp": "2019-05-19T20:12:19+03:00",
  "locale": "",
  "recipient": {
    "id": "d1ff7fc6-7de6-4208-bed8-0dd025aaaa26",
    "role": "user"
  },
  "replyToId": "4240ece0-7a59-11e9-99a0-d9c48d8a8fbc",
  "serviceUrl": "https://a943a825.ngrok.io",
  "suggestedActions": {
    "actions": [
      {
        "title": "short title",
        "type": "imBack",
        "value": "recommendations"
      }
    ]
  },
  "text": "Hi, select what you want",
  "timestamp": "2019-05-19T17:12:19.748Z",
  "type": "message"
}

JSON for the long title:
{
  "attachments": [],
  "channelId": "emulator",
  "conversation": {
    "id": "9476c340-7a59-11e9-b916-434698c80b5d|livechat"
  },
  "entities": [],
  "from": {
    "id": "2",
    "name": "Bot",
    "role": "bot"
  },
  "id": "9a384dd0-7a59-11e9-a6cb-c56ac337ce96",
  "inputHint": "acceptingInput",
  "localTimestamp": "2019-05-19T20:14:46+03:00",
  "locale": "",
  "recipient": {
    "id": "d1ff7fc6-7de6-4208-bed8-0dd025aaaa26",
    "role": "user"
  },
  "replyToId": "99fea030-7a59-11e9-a6cb-c56ac337ce96",
  "serviceUrl": "https://8b80b63c.ngrok.io",
  "text": "Hi, select what you want\n\n   1. A very very long title",
  "timestamp": "2019-05-19T17:14:46.956Z",
  "type": "message"
}

And this is how I create the PromptOptions dialog:
internal PromptOptions CreateChoicesPromptOptions(string text, IDictionary<string, string> keyToTitle)
{
    return new PromptOptions
    {
        Prompt = MessageFactory.Text(text),
        Choices = keyToTitle.Select(v =>
            new Choice(v.Key)
            {
                Action = new CardAction(ActionTypes.ImBack, v.Value, value: v.Key)
            })
            .ToList()
    };
}

I'm not sure I understand your issue. Does the following information help?
If you make this change in the C# 05.multi-turn-prompt sample:
In .\Bots\DialogBot.cs, line 37
AddDialog(new ChoicePrompt(nameof(ChoicePrompt)) { Style = ListStyle.List });

The bot formats the choice as a list:

But even I set Style = ListStyle.SuggestedAction, if the choices are too long, it will be shown as a list.

I understand that the documentation on how to use prompts is not complete.
What behavior are you trying to get from the choice prompt? Are you trying to force it to always use suggested actions?

Closing. Please reopen if the issue still exists.