c++ client Producer with name 'xxx-18-31667' is already connected to topic

Describe the bug
C++ producer may blocked sometimes, and from the logs in broker we can see

cc @BewareMyPower

I have checked the C++ client, it looks like the error is not caused by the issue like https://github.com/apache/pulsar-client-go/pull/639. See https://github.com/apache/pulsar/blob/8625ffe0cce476539faf38c7d8a97adbfbe95855/pulsar-client-cpp/lib/ProducerImpl.cc#L68-L70
The userProvidedProducerName_ field is only initialized once when the ProducerImpl is created.

And I simulated the reconnection by modifying the SDK implementation just now.
diff --git a/pulsar-client-cpp/lib/ClientConnection.cc b/pulsar-client-cpp/lib/ClientConnection.cc
index 3ad6f4062f4..43a3cdc83ce 100644
--- a/pulsar-client-cpp/lib/ClientConnection.cc
+++ b/pulsar-client-cpp/lib/ClientConnection.cc
@@ -49,6 +49,8 @@ static const uint32_t DefaultBufferSize = 64 * 1024;

 static const int KeepAliveIntervalInSeconds = 30;

+static std::atomic_int reconnect_{0};
+
 // Convert error codes from protobuf to client API Result
 static Result getResult(ServerError serverError) {
     switch (serverError) {
@@ -805,6 +807,12 @@ void ClientConnection::handleIncomingCommand() {
             // Handle normal commands
             switch (incomingCmd_.type()) {
                 case BaseCommand::SEND_RECEIPT: {
+                    reconnect_++;
+                    if (reconnect_ == 3) {
+                        LOG_INFO(cnxString_ << " trigger reconnection manually");
+                        close();
+                        return;
+                    }
                     const CommandSendReceipt& sendReceipt = incomingCmd_.send_receipt();
                     int producerId = sendReceipt.producer_id();
                     uint64_t sequenceId = sendReceipt.sequence_id();
@@ -812,8 +820,9 @@ void ClientConnection::handleIncomingCommand() {
                     MessageId messageId = MessageId(messageIdData.partition(), messageIdData.ledgerid(),
                                                     messageIdData.entryid(), messageIdData.batch_index());

-                    LOG_DEBUG(cnxString_ << "Got receipt for producer: " << producerId
-                                         << " -- msg: " << sequenceId << "-- message id: " << messageId);
+                    LOG_INFO(cnxString_ << "Got receipt for producer: " << producerId
+                                        << " -- msg: " << sequenceId << "-- message id: " << messageId
+                                        << " | reconnect_: " << reconnect_);

                     Lock lock(mutex_);
                     ProducersMap::iterator it = producers_.find(producerId);
diff --git a/pulsar-client-cpp/lib/ProducerImpl.cc b/pulsar-client-cpp/lib/ProducerImpl.cc
index f81e205475d..1ee938116d4 100644
--- a/pulsar-client-cpp/lib/ProducerImpl.cc
+++ b/pulsar-client-cpp/lib/ProducerImpl.cc
@@ -150,6 +150,8 @@ void ProducerImpl::connectionOpened(const ClientConnectionPtr& cnx) {
     SharedBuffer cmd = Commands::newProducer(topic_, producerId_, producerName_, requestId,
                                              conf_.getProperties(), conf_.getSchema(), epoch_,
                                              userProvidedProducerName_, conf_.isEncryptionEnabled());
+    LOG_INFO(getName() << " newProducer userProvidedProducerName_: " << userProvidedProducerName_
+                       << ", producerName_: " << producerName_);
     cnx->sendRequestWithId(cmd, requestId)
         .addListener(std::bind(&ProducerImpl::handleCreateProducer, shared_from_this(), cnx,
                                std::placeholders::_1, std::placeholders::_2));
@@ -191,6 +193,8 @@ void ProducerImpl::handleCreateProducer(const ClientConnectionPtr& cnx, Result r
         producerName_ = responseData.producerName;
         schemaVersion_ = responseData.schemaVersion;
         producerStr_ = "[" + topic_ + ", " + producerName_ + "] ";
+        LOG_INFO(getName() << " handleCreateProducer userProvidedProducerName_: " << userProvidedProducerName_
+                           << ", producerName_: " << producerName_);

         if (lastSequenceIdPublished_ == -1 && conf_.getInitialSequenceId() == -1) {
             lastSequenceIdPublished_ = responseData.lastSequenceId;

i.e. there will be a reconnection for the 3rd time to send a message. And I also added some logs for debugging.
Here are the output:
2021-12-15 16:34:38.488 INFO  [0x7000035ed000] ClientConnection:825 | [[::1]:51361 -> [::1]:6650] Got receipt for producer: 0 -- msg: 0-- message id: (1,0,-1,-1) | reconnect_: 1
2021-12-15 16:34:38.488 INFO  [0x11ae9f600] SampleProducer:45 | Message sent: Ok
2021-12-15 16:34:39.497 INFO  [0x7000035ed000] ClientConnection:825 | [[::1]:51361 -> [::1]:6650] Got receipt for producer: 0 -- msg: 1-- message id: (1,1,-1,-1) | reconnect_: 2
2021-12-15 16:34:39.497 INFO  [0x11ae9f600] SampleProducer:45 | Message sent: Ok
2021-12-15 16:34:40.505 INFO  [0x7000035ed000] ClientConnection:812 | [[::1]:51361 -> [::1]:6650]  trigger reconnection manually
2021-12-15 16:34:40.505 INFO  [0x7000035ed000] ClientConnection:1551 | [[::1]:51361 -> [::1]:6650] Connection closed
2021-12-15 16:34:40.505 INFO  [0x7000035ed000] HandlerBase:142 | [persistent://public/default/my-topic, standalone-0-1] Schedule reconnection in 0.1 s
2021-12-15 16:34:40.505 ERROR [0x7000035ed000] ClientConnection:591 | [[::1]:51361 -> [::1]:6650] Read operation failed: Bad file descriptor
2021-12-15 16:34:40.505 INFO  [0x7000035ed000] ClientConnection:257 | [[::1]:51361 -> [::1]:6650] Destroyed connection
2021-12-15 16:34:40.605 INFO  [0x7000035ed000] HandlerBase:64 | [persistent://public/default/my-topic, standalone-0-1] Getting connection from pool
2021-12-15 16:34:40.605 INFO  [0x7000035ed000] ConnectionPool:86 | Deleting stale connection from pool for pulsar://localhost:6650 use_count: -1 @ 0x0
2021-12-15 16:34:40.605 INFO  [0x7000035ed000] ClientConnection:183 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000
2021-12-15 16:34:40.606 INFO  [0x7000035ed000] ConnectionPool:96 | Created connection for pulsar://localhost:6650
2021-12-15 16:34:40.607 INFO  [0x7000035ed000] ClientConnection:369 | [[::1]:51368 -> [::1]:6650] Connected to broker
2021-12-15 16:34:40.610 INFO  [0x7000035ed000] ProducerImpl:154 | [persistent://public/default/my-topic, standalone-0-1]  newProducer userProvidedProducerName_: 0, producerName_: standalone-0-1
2021-12-15 16:34:40.611 INFO  [0x7000035ed000] ProducerImpl:190 | [persistent://public/default/my-topic, standalone-0-1] Created producer on broker [[::1]:51368 -> [::1]:6650]
2021-12-15 16:34:40.611 INFO  [0x7000035ed000] ProducerImpl:197 | [persistent://public/default/my-topic, standalone-0-1]  handleCreateProducer userProvidedProducerName_: 0, producerName_: standalone-0-1
2021-12-15 16:34:40.614 INFO  [0x7000035ed000] ClientConnection:825 | [[::1]:51368 -> [::1]:6650] Got receipt for producer: 0 -- msg: 2-- message id: (1,3,-1,-1) | reconnect_: 4
2021-12-15 16:34:40.614 INFO  [0x11ae9f600] SampleProducer:45 | Message sent: Ok

From following lines:
2021-12-15 16:34:40.610 INFO  [0x7000035ed000] ProducerImpl:154 | [persistent://public/default/my-topic, standalone-0-1]  newProducer userProvidedProducerName_: 0, producerName_: standalone-0-1
2021-12-15 16:34:40.611 INFO  [0x7000035ed000] ProducerImpl:197 | [persistent://public/default/my-topic, standalone-0-1]  handleCreateProducer userProvidedProducerName_: 0, producerName_: standalone-0-1

We can see userProvidedProducerName_ is always false (0). I also checked from the broker side as well.

I think there is something wrong at broker side. When tryOverwriteOldProducer is called, the old Producer is not removed from producers cache. The exception is caused by following validation.
    public boolean isSuccessorTo(Producer other) {
        return Objects.equals(producerName, other.producerName)
                && Objects.equals(topic, other.topic)
                && producerId == other.producerId
                && Objects.equals(cnx, other.cnx) // the connections are not the same
                && other.getEpoch() < epoch; // the epochs are not the same
    }

The code is from latest master, you could also check your own Pulsar source code.

For example, the image above is the result after I modified AbstractTopic#removeProducer like:
    @Override
    public void removeProducer(Producer producer) {
        checkArgument(producer.getTopic() == this);

        //if (producers.remove(producer.getProducerName(), producer)) {
        //    handleProducerRemoved(producer);
        //}
    }

https://github.com/apache/pulsar/issues/13342

I'm going to close this issue for the same justification provided here https://github.com/apache/pulsar/issues/13342#issuecomment-1145151323.
It's like that this issue was solved in a subsequent release of Pulsar.


Module Versioning

Documentation is needed for standardizing module versioning. This should probably follow Go's v1, v2, v3 approach.

This type of versioning will allow for simpler jablkomods version handling. Instead of providing a specific commit tag, the major version tag will have all the latest updates for each branch (v1, v2, v3).

The update process could be going through all installed jablkomods and check if the latest commit to the master or main branch is of a newer time than the currently installed package (Git JSON API).
If it is newer, Jablko can download the latest source replace the data in the current install directory. The building should be done in a temporary directory to make sure the new source works. If the source doesn't work, the module will not be updated.

There should still be a way for downloading the source at a specific version tag. In an ideal world, all V1 minor versions should be backwards compatible; however, there may be accidental breaking changes introduced. Users should be able to certain version tags to get a source that works for them.

Closed in favor of #118


Refactor using string_view and fix undefined behaviour

This PR achieves the following things:
Refactor: const std::string& --> std::string_view wherever possible and sensible.
This was done for performance reasons: const std::string& automatically converts const char* and std::string r-values resulting in unnecessary heap allocations. A string_view avoids any allocations at the risk of dangling references which is not a problem if common rules are followed: generally only use in function parameters, never return string_view, never store string_view.
This point alone touches many files of the repo and creates a large diff (sorry).
Feature: Allow Heterogenous Lookup (HL) in maps using std::string as key.
One of the features added by this PR is HL: looking up the value of a st::map/unordered_map associated with a key of type std::string via key-simile types const char* and std::string_view. HL is crucial in ensuring the performance of e.g. map.find(str) lookups to not be dominated by the forced conversion to std::string (heap allocations). It works by supplying a custom Comparator (ordered maps) or custom Hash and Equality (unordered maps) types for the map templates which allow to compare/hash/eq-check objects within the string family (currently defined as string, string_view, const char*) and forego the need to convert every lookup to std::string. Two points of interest:

HL requires a little template metaprogramming machinery to filter out unaccepted types (SFINAE and static asserts). This  is provided in the new files utils/heterogenous_lookup.h and utils/type_traits.h.
C++17 does not support HL. Abseil containers however do and since this repo relies on abseil machinery in many parts I have gone ahead and used the drop-in container replacements of abseil for the std counterparts. This may lead to changes in the expected performance of some core parts in the repo (whether for the better or worse I dare not say, abseil claims their containers are better).

Bugfix: Fix undefined behaviour in GameParameter
So far, the class GameParameter has held a member of type std::map< std::string, GameParameter> which represents the usage of an incomplete type as the value type of a map template. According to my understanding, incomplete value types are not defined behaviour in this context in c++17 (and AFAIK still unchanged in later standards). Only smart pointers and some specific containers are allowed to hold incomplete value types (e.g. vector/list). Therefore, I refactored the member as a mapping of std::shared_ptr<GameParameter> and moved it to an abseil container to also enable HL. This has consequences to nearly the entire repo. For example, the tests construct parameter objects via GameParameter(args...) which up until now provided values. These calls are now replaced everywhere by MakeGameParameter(args...) which automatically returns a shared ptr of type GameParameter. Other places now needed to deref these pointers (-> instead .), comparison of two objects of type GameParameters (heed the plural, it is meant) is no longer meaningfully provided by the default comparison of keys and values in a map due to the values being shared pointers; a custom equality function is needed. Many other small changes were necessary. While doing so, I also refactored some of the functions around GameParameter. When in Rome...
Minor refactors of unnecessary copies
E.g. pass-by-value instances of std::shared_ptr and other points in which copies were created without need.
Overall this PR touches a vast amount of the c++ codebase, so apologies for the massive diff.

Hi @maichmueller,
I appreciate this, but since you have already made a few large PRs, could I kindly ask you to touch base with us before you do this? It could save a lot of wasted effort.
There is a reason string_view is not more widely used in OpenSpiel: IMO, it's overkill. It's mostly solving a non-problem, and for 99% of use cases you add nonstandard syntax for a few nanoseconds benefit. Also, I have seen it lead to several nontrivial bugs that would not have happened with string references.
I think this is a clash with the first principle of the design philosophy of the project (keep it simple). IMO string_view complicates things for very miniscule benefit and I would rather keep things simple (and only use it in cases where it is really beneficial).
Anyway. Like I said, I appreciate the enthusiasm regarding PRs but please try to reach out before spending effort on large changes like this. Thanks.

For similar reasons, we usually discourage anything that requires template meta programming (which IMO leads to even bigger potential problems and deeper knowledge of C++).

Hi @lanctot ,
noted, I will open an issue next time if I consider major work again. However, a PR is communication too IMO and I would expect to discuss and receive feedback there as well. At the least, I am using my work in my own fork so it wouldn't be wasted effort.
Regarding string views, I understand your concerns. However, as function parameters they are harmless and signal intent well I find. Users who need to modify the string can simply make an actual string out of it so that errors like the one in your example wouldn't be possible. Understanding the difference between a string view and a string is quite similar to understanding a raw pointer's relation to smart ptrs in the end (sans implicit conversion).
Anyhow, the commits are split into two parts and I could cut out the fix for the UB if you don't want the view refactoring. If you also don't want the UB fix (or would like to discuss it first in an issue) then feel free to just close the PR.

Anyhow, the commits are split into two parts and I could cut out the fix for the UB if you don't want the view refactoring. If you also don't want the UB fix (or would like to discuss it first in an issue) then feel free to just close the PR.

More than happy to have a fix for undefined behavior, thanks! :)
(Though I may have to fullt understand it first, at first glance I see no problem)

Hold on, don't you need the string_view for the heterogenous lookup?
Also: I'm a bit confused:

According to my understanding, incomplete value types are not defined behaviour in this context in c++17 (and AFAIK still unchanged in later standards)

I'm not sure what problem is. Wouldn't it just not compile? This map has been used everywhere from the start.. we've never had a problem. Is it easy to construct a concrete example where we'd run into a problem with a map<std::string, GameParameter>?

Only smart pointers and some specific containers are allowed to hold incomplete value types (e.g. vector/list).
Again, wouldn't it just not compile if it wasn't "allowed" ? Do you have a reference for this. I'd like to understand the problem better. Why is having a map of a templated type problematic? I can imagine the fix is rather involved if it means introducing smart pointers...?

Going to summon @elkhrt to help me out :)

Hold on, don't you need the string_view for the heterogenous lookup?

You can have HL for any types you define it for (in this pr: string, string_view, const char*), so removing string_view would not necessarily remove HL as long as you have function overloads. But if the  const string& signature is preferred, then taking on HL is pointless and would not need to be needed, yes.

Also: I'm a bit confused:

According to my understanding, incomplete value types are not defined behaviour in this context in c++17 (and AFAIK still unchanged in later standards)

I'm not sure what problem is. Wouldn't it just not compile? This map has been used everywhere from the start.. we've never had a problem. Is it easy to construct a concrete example where we'd run into a problem with a map<std::string, GameParameter>?

As with all things UB, compilers are free to do whatever they see fit, including making the situation work. Major compiler vendors like clang and GCC do just that it seems. For example, MSVC is not as forgiving AFAIK. Compilers are free to drop supporting this in future versions though which would leave a tricky UB bug to triage. Possible ways of dealing with this is:

ignore and bet on the compilers to keep playing along
switch to a container implementation that explicitly allow this (see later in the message)
Fix by changing the mapped_type to a container which allows incomplete types by the standard (shared/unique_ptr).

Only smart pointers and some specific containers are allowed to hold incomplete value types (e.g. vector/list).

Again, wouldn't it just not compile if it wasn't "allowed" ? Do you have a reference for this. I'd like to understand the problem better. Why is having a map of a templated type problematic?

UB is free real estate for compilers, so it may compile just fine.
As for references see e.g. https://stackoverflow.com/a/38060437 ('Container Improvements' and https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4510.html) or https://stackoverflow.com/a/41913654.
Boost equivalent containers do allow incomplete types in line with boost's general aim to provide more functionality than the std library.
Abseil equivalent containers actively disallow them (static_assert error) which matches their attempt at being standard compliant with better implementations.

I can imagine the fix is rather involved if it means introducing smart pointers...?

What I attempted to fix it is described in the initial post. See GameParameters.h  as an example of the changes. I have since then noticed on my own branch that the type caster for the pybind11 GameParameter binding needs changing still which isn't in this PR yet.

Thanks for the contributions & the enthusiasm for improving OpenSpiel!
1 - Now that std::string_view has been in the C++ standard for 5 years, I think it's arguable that void foo(std::string_view s) is more idiomatic and more readable than void foo(const std::string& s). For most naive users, either one is just a magic incantation for passing a string from one function to another. Given the usability advantages, I think it's worth changing. I'll attempt to convince Marc in a side channel.
2 - I think you're technically correct about GameParameters, but the fix seems like overkill. Couldn't we just move the definition of GameParameters after the declaration of GameParameter? That means we can't use the type alias inside GameParameter, but has no wider implications.
https://www.youtube.com/watch?v=0ZEuWJ4muYc
3 - The HL sounds it's probably a necessary part of the string_view switch. If the template metaprogramming can be hidden from ordinary users, and this just looks like a set / map at the point of use, then I think this is probably okay. But obviously without string_view there's limited (but not zero!) point.

Hi @elkhrt
Regarding 1:
One argument to add to the debate of whether to view or not to view this is that the decision against would probably have to be firm, as the cost of future refactoring only grows with time and the evolution of the library. Tackling all the parts in the repo was a fair bit of work in the end.
Regarding 2:
I agree that the fix is sort of overkill. It appeared to be the least divergent solution at the time since it merely needed to replace values with pointers.
In general, I wasn't quite able to understand the intention behind the design of GameParameter (a tree data structure of further parameters + a union type itself which isn't using a union/variant though). Definitely would have been wiser to discuss this fix in an issue first.
Could you elaborate on how we could move the definition of GameParameters after the definition of GameParameter? At the time of GameParameter's definition, the member types need to be complete or reference types after all (struct's memory layout is otherwise indeterminable)

1 - Now that std::string_view has been in the C++ standard for 5 years, I think it's arguable that void foo(std::string_view s) is more idiomatic and more readable than void foo(const std::string& s). For most naive users, either one is just a magic incantation for passing a string from one function to another. Given the usability advantages, I think it's worth changing. I'll attempt to convince Marc in a side channel.

I don't think we should change something just because it has been in the standard for 5 years. If it's arguably the new community idiom, then someone will need to convince me. I have searched several times on this topic (including most recently when this PR came up) and all I have seen are people asking the same questions as me and essentially everyone shrugging.
I don't see how in OpenSpiel we will benefit from performance enhancements (without buying into it quite heavily-- and the benefits are not clear to me). I have only seen hard to fix bugs arise from it so far.
What are the usability advantages compared to a string reference? Will they improve OpenSpiel?
I am open to being convinced, but I am still not seeing the case for it in OpenSpiel.

I have read a few blogs and I am starting too see the benefits.
But let me ask: the thing I need most is to be convinced that the community is indeed moving away from const string ref, and I am not seeing any evidence of that.
Can you show me some? Has there been any videos on people trying to make the case to move away from const tring ref? To me, it seems quite up in the air still, after 5 years. I tend to be slow on these because frivolous changes for performance don't make much sense if 99% of use cases do not benefit from it and add chances of subtle bugs. The correctness and clarity of the code is so much more important IMO.

Regarding 2:
I agree that the fix is sort of overkill. It appeared to be the least divergent solution at the time since it merely needed to replace values with pointers.
In general, I wasn't quite able to understand the intention behind the design of GameParameter (a tree data structure of further parameters + a union type itself which isn't using a union/variant though). Definitely would have been wiser to discuss this fix in an issue first.
Could you elaborate on how we could move the definition of GameParameters after the definition of GameParameter? At the time of GameParameter's definition, the member types need to be complete or reference types after all (struct's memory layout is otherwise indeterminable)

Delete the line class GameParameter;
Move the block
using GameParameters = std::map<std::string, GameParameter>;
std::string GameParametersToString(const GameParameters& game_params);
GameParameter GameParameterFromString(const std::string& str);
GameParameters GameParametersFromString(const std::string& game_string);

to be after the end of the definition of the GameParameter class.
Inside the definition of GameParameter, either:
a - replace GameParameters with std::map<std::string, GameParameter> throughout
b - add using GameParameters = std::map<std::string, GameParameter> inside the class, and using GameParameters = GameParameter::GameParameters; outside it (replacing the other previous using)

Regarding 2:
I agree that the fix is sort of overkill. It appeared to be the least divergent solution at the time since it merely needed to replace values with pointers.
In general, I wasn't quite able to understand the intention behind the design of GameParameter (a tree data structure of further parameters + a union type itself which isn't using a union/variant though). Definitely would have been wiser to discuss this fix in an issue first.
Could you elaborate on how we could move the definition of GameParameters after the definition of GameParameter? At the time of GameParameter's definition, the member types need to be complete or reference types after all (struct's memory layout is otherwise indeterminable)

The fix I had in mind doesn't work, but how about this? Replace the member variable:
GameParameters game_value_ = {}; with std::shared_ptr<std::map<std::string, GameParameter>> and then make a few local changes in that class to preserve the existing interface as far as possible.
Also move the block:
using GameParameters = std::map<std::string, GameParameter>;
std::string GameParametersToString(const GameParameters& game_params);
GameParameter GameParameterFromString(const std::string& str);
GameParameters GameParametersFromString(const std::string& game_string);

To after the class definition.
This will only have any impact on call sites using the GameParameter::GameParameter(GameParameters) constructor which is pretty rare.

@lanctot :

But let me ask: the thing I need most is to be convinced that the community is indeed moving away from const string ref, and I am not seeing any evidence of that.

I am not even sure how to provide such evidence. What community do you mean?
AI researchers? I doubt they would, but I assume they are most at home with other languages like Python and may tend to use C++ pythonically anyway.
High-performance library devs? I expect them to inspect const string& usage more carefully, yes.
P-l-e-n-t-y of sources advice using string_view, albeit with caveats of course. And that is also the point, string_view is not meant to simplify your life primarily. It is supposed to add to the string usage performance:

no automatic conversion to std::string
no double indirection to the underlying chars
substring operations are now constant complexity, not linear anymore
(not applicable to open_spiel) allows interaction with different string implementations.

Point 1's relevance is arguably dampened by small string optimization (23 chars on clang AFAIK, so potentially even most of all 'const char' usages in open_spiel, less with other compilers).
Point 2 asserts that you forego many cache misses which double indirections are very prone to.
But these points are all performance based, so when you ask

What are the usability advantages compared to a string reference? Will they improve OpenSpiel?

the answer is 0. Everything that implicitly converts to string_view in this repo will do the same to a const string &.
Now, I didn't time the performance gain of slapping string_view on function parameters in all honesty. Maybe it doesn't change a thing. This PR is merely a reflection of what I consider best practice.
I sympathize with your feeling on string_view and so do others. Personally, I find it safe to use in the ways I described above. However, I am not here to convince you of something that you seem all but decided on.
@elkhrt
Correct me if I am wrong, but I think std::shared_ptr<std::map<std::string, GameParameter>> is, technically, still UB due to the same map problem. At the point of this definition, the map specialisation is already complete. So, even if shared_ptr accepts incomplete types, the type std::map<std::string, GameParameter> is complete and does not.

But these points are all performance based, so when you ask

What are the usability advantages compared to a string reference? Will they improve OpenSpiel?

the answer is 0.

Ed specifically mentioned usability advantages, so I guess that one was mostly directed at him.

Now, I didn't time the performance gain of slapping string_view on function parameters in all honesty. Maybe it doesn't change a thing. This PR is merely a reflection of what I consider best practice.
I sympathize with your feeling on string_view and so do others. Personally, I find it safe to use in the ways I described above. However, I am not here to convince you of something that you seem all but decided on.

I've got my reasons to be hesitant that I've made clear, but mostly trying to understand the motivation + justification for such a large change (more so now that Ed wants to convince me too), and weighing the pros and cons. I've had a hard stance before and managed to be convinced; in those cases, I often learn something. I appreciate the effort and patience, thanks.


Unhandled rejection error.

What version of bluebird is the issue happening on?
v3.4.7

What platform and version? (For example Node.js 0.12 or Google Chrome 32)
Chrome 55.0.2883.95

Did this issue happen with earlier version of bluebird?
Don't know.

I have the following code. And it works as expected without throwing an unhandled rejection error.
p = new Promise (fulfill, reject) ->
  reject new Error 'some error'

p.catch (error) ->
  console.log 'error'

Now, in the second code example something weird is happening.
I'm getting an unhandled rejection error even tho the 'error' string is logged to the console.
Can someone explain to me why this is happening when im clearly handling the error.
This seems like a bug to me.
Wouldn't it be more sensible to call the catch callback or throw an unhandled rejection error.
But not both at the same time.
p = new Promise (fulfill, reject) ->
  reject new Error 'some error'

p.then ->
  console.log 'ok'

p.catch (error) ->
  console.log 'error'

You're forking the promise, in order to handle the error you need to handle the error both in the .then and in the original promise.
p   ---> p.then fork
|
\ ---> caught
Someone can listen to p.then and not handle the error (what if the code console.log "ok" throws?)

So whats the method i need to call to listen for the 'done' event without forking the promise.

p = new Promise (fulfill, reject) ->
  reject new Error 'some error'

p.then ->
  console.log 'ok'
.catch (error) ->
  console.log 'error'

Also, please ask support questions in the appropriate support channels :)

Ok thanks i will


tests: rearange fixtures in integration tests to limit their numbers

*INSPIR-3425

After our discussion, we decided to create only one fixture inspire_appwith app, db and es_clear and have separate fixtures for redis,cli etc. @pazembrz


Add Astroneer template

Have you tried running this using Proton? You can add Steam AppID 1580130 as an update stage that only applies to Linux.

Merging Windows only for now, but I'll try modifying it to accommodate Proton.


SILCombine: optimize casts of existential boxes.

Optimize the unconditional_checked_cast_addr in this pattern:
   %box = alloc_existential_box $Error, $ConcreteError
   %a = project_existential_box $ConcreteError in %b : $Error
   store %value to %a : $*ConcreteError
   %err = alloc_stack $Error
   store %box to %err : $*Error
   %dest = alloc_stack $ConcreteError
   unconditional_checked_cast_addr Error in %err : $*Error to ConcreteError in %dest : $*ConcreteError

to:
   ...
   retain_value %value : $ConcreteError
   destroy_addr %err : $*Error
   store %value to %dest $*ConcreteError

This lets the alloc_existential_box become dead and it can be removed in following optimizations.
The same optimization is also done for conditional_checked_cast_addr.
There is also an implication for debugging:
Each "throw" in the code calls the runtime function swift_willThrow. The function is used by the debugger to set a breakpoint and also add hooks.
This optimization can completely eliminate a "throw", including the runtime call.
So, with optimized code, the user might not see the program to break at a throw, whereas in the source code it is actually throwing.
On the other hand, eliminating the existential box is a significant performance win and we don't guarantee any debugging behavior for optimized code anyway. So I think this is a reasonable trade-off.
I added an option "-Xllvm -keep-will-throw-call" to keep the runtime call which can be used if someone want's to reliably break on "throw" in optimized builds.
To make this optimization work smoothly with the Result type from the library, I had to make the Result methods inlinable. For a use case see the SILOptimizer/existential_box_elimination.swift test file.
rdar://problem/66055678

@swift-ci test

@swift-ci benchmark

@fredriss do you have any concerns regarding the implication for debugging (see my comment in the description)?

Performance: -O

Improvement
OLD
NEW
DELTA
RATIO

DropFirstArrayLazy
14
13
-7.1%
1.08x (?)

Code size: -O
Performance: -Osize

Improvement
OLD
NEW
DELTA
RATIO

FlattenListFlatMap
4225
3797
-10.1%
1.11x (?)

FlattenListLoop
1522
1419
-6.8%
1.07x (?)

Code size: -Osize
Performance: -Onone
Code size: -swiftlibs

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: 6-Core Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

Build failed
Swift Test Linux Platform
Git Sha - 61d8bbd92dff1a6ee29c5412c911e16782dc64a3

Build failed
Swift Test OS X Platform
Git Sha - 61d8bbd92dff1a6ee29c5412c911e16782dc64a3

@swift-ci test

@swift-ci test

Build failed
Swift Test Linux Platform
Git Sha - 61d8bbd92dff1a6ee29c5412c911e16782dc64a3

Build failed
Swift Test OS X Platform
Git Sha - 61d8bbd92dff1a6ee29c5412c911e16782dc64a3

@swift-ci test

@swift-ci test


We can load models in 4-bit with and without bitsandbytes support. so what is the major difference between both?

System Info
We can load models in 4-bit with and without bitsandbytes support.
so what is the major difference between both?

from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained(
    model_path,
    load_in_4bit = True,
    device_map="auto"
)

Using bitsandbytes

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant="False",
)

FineTunedModel = AutoModelForCausalLM.from_pretrained(
    model_path,
    quantization_config=bnb_config,
    device_map="auto"
)

Reproduction
We can load models in 4-bit with and without bitsandbytes support.
so what is the major difference between both?

from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained(
    model_path,
    load_in_4bit = True,
    device_map="auto"
)

Using bitsandbytes

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant="False",
)

FineTunedModel = AutoModelForCausalLM.from_pretrained(
    model_path,
    quantization_config=bnb_config,
    device_map="auto"
)

Expected behavior
We can load models in 4-bit with and without bitsandbytes support.
so what is the major difference between both?

from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained(
    model_path,
    load_in_4bit = True,
    device_map="auto"
)

Using bitsandbytes

from transformers import AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant="False",
)

FineTunedModel = AutoModelForCausalLM.from_pretrained(
    model_path,
    quantization_config=bnb_config,
    device_map="auto"
)

https://huggingface.co/docs/transformers/main_classes/quantization
By default your model is loaded with bnb.
https://github.com/huggingface/transformers/blob/main/src/transformers/modeling_utils.py#L3609

@puyuanOT Okay
so
from transformers import AutoModelForCausalLM
AutoModelForCausalLM.from_pretrained(
    model_path,
    load_in_4bit = True,
    device_map="auto"
)

in the above case, the model by default uses the bitsandbytes support for the load_in_4bit quantization, even though we didn't specify it externally in the code right?

Thanks @puyuanOT for chipping in. Also in my understanding this is correct. load_in_4bit can be understood as a shorthand when not needing the extra settings of the BitsAndBytesConfig.
From what I understand, so far it was possible to specify both load_in_4bit and BitsAndBytesConfig which is confusing, but which is being fixed to be XOR with the HF Transformers modeling_utils refactor that should be merged and released any day now.
@younesbelkada Correct me if I'm wrong?
Can we close this issue then?

Hi @pradeepdev-1995 @Titus-von-Koeller true yes, if you don't specify a BitsAndBytesConfig and simply pass load_in_4bit=True the model will be loaded in 4bit with the default bnb config values for 4bit. In the future, load_in_4bit and load_in_8bit will be deprecated in favor of quantization configs.


video Callback not working

Hello Guys!
when i try this:
this.state = {
  camera: {
    aspect: Camera.constants.Aspect.fill,
    captureMode: Camera.constants.CaptureMode.still,
    captureAudio: true,
    captureQuality: Camera.constants.CaptureQuality.high,
    captureTarget: Camera.constants.CaptureTarget.cameraRoll,
    type: Camera.constants.Type.back,
    orientation: Camera.constants.Orientation.auto
  },
  isRecording: false
};

_takeVideo = () => {
if (this.camera) {
this.camera.capture()
.then((data) => console.log(data))
.catch(err => console.error(err));
this.setState({
isRecording: true
});
}
}
The callback into this.camera.capture works as well BUT if i change
captureMode: Camera.constants.CaptureMode.video
isnt work anymore, why?
This happen into ios 10 simulator using
"react-native": "0.38.0",
"react-native-camera": "^0.4.1"
Thanks

I have the same issue the only difference is I have updated to react-native 0.40.0

@blaziken105 Dude...the callback works! but only into device, in simulator won´t work

I tried it out, doesn't work on the device or the emulator probably a bug within 0.40.0. Could you try upgrading to the latest version and testing if it works? I am using the example code...

@leonciocaminha did you find out how to resolve this issue?

This issue is closed because of inactivity.
Furthermore RCTCamera is deprecated now, use RNCamera instead. If this issue is also related to RNCamera, please open a new one.


Fix dtype for streaming confusion matrix

Currently, dependents of _streaming_confusion_matrix don't actually work correctly with floating-point weights.

Can one of the admins verify this patch?

@poxvoculi your name was all over blame, I was confused too. Probably a merge you did.
@alextp can you take a look?

Updated.

Jenkins, test this please.

Please address the test failure as it is relevant.

Oops, my bad. I updated the wrong weight for the test. Fixed now.

Anyway, be aware that if you have any internal consumers of _streaming_confusion_matrix without weights in Python 2 that divide by integers (e.g. for building a streaming normalized confusion matrix metric), this would constitute a bit of a breaking change in practice, as in the no-weights case, the confusion matrix tensors will now be of type int64 rather than float64 as before.

Jenkins, test this please.

Jenkins, test this please.

One change request from API review.

Jenkins, test this please.


CLA Corporate Console Approved List date is not accurate

Ticket: SUPPORT-12530
SUPPORT-13416
Reporters: Bob Killen (Google LLC), FINOS
Organization: All
Project: All
Issue Summary:
The CLA Corporate Console displays the "date_modified" date from cla-prod-signatures table in DynamoDB to all the Approval Criteria Added On date, which is not accurate.
When a CLA Manager is added or removed from the Organization, the "date_modified" date changes and displays the same data for all the Approval Criteria options.
For Google LLC Organization scenario, the last change was on August 19 2022:
This change modified the "date_modified" date and now all the Approval Criteria for the Organization is showing the same Added On date.
https://jira.linuxfoundation.org/browse/LFX-3194
Observe difference in this section of approval list

vs Event log

Hello @mrbobbytables I created this to publicize the issue you and others have raised about the dates in the Approval list being incorrect. The team is investigating the issue and will come back with an ETA or a fix soon.

We've discussed this internally, and we're committed to resolving it promptly. However, EasyCLA currently can't accurately track the timestamp for corporate contributor additions. We have a proposed solution that requires performance testing. We're finalizing our next release in the next couple weeks and we would like to address this issue in following releases. Unfortunately, I can't provide an exact ETA until we complete evaluation of our proposed solution.
If you urgently require access to the dates when individual contributors were added, we are more than willing to perform a one-time manual export from our database within a short timeframe to assist you.

No worries @mlehotskylf it's not urgent. I appreciate you looking into it 👍

Correct, @krook. The date isn't lost; it's stored in our "event log" within the DynamoDB table. Currently, the UI doesn't display this date; instead, it shows the update date for the organization's CCLA. We need to figure out an efficient method to access or track this date so that we can display it accurately and efficiently in the UI.

This is related to Jira UI ticket CD-2367 where UI changes needs to be implemented.

Moving this to QA as verified in CD-2367

The existing approval list record is not visible at corporate cla page

This was a UI issue that has been resolved. Kindly reverify @thakurveerendras

@nickmango to confirm if this is BE or UI issue.

This is BE issue and @nickmango will create PR soon.

Added indexing and resolved the child project events fetched. @thakurveerendras  kindly review

Tested issue on reported child project and found that it is fixed as expected

Blank approval list records appear after deployment, So reopening this ticket

@thakurveerendras  This is a UI deployment issue not yet resolved

Deployment and infra changes have been completed. @thakurveerendras kindly verify

We are improving performance for this solution so moving back to progress.

@mlehotskylf do we know if this issue is planned for deployment soon?

Hi @krook. This fix is in progress and our goal is to fix this in next major release (in next few weeks or so).

@nickmango Retested the issue on the dev site and found that the Approved List date is still not accurate

Retested on the dev site and found that for some records still wrong date & time appear,
Refer to the screenshot

Tested on dev site & found that records date & time are proper & expected. So updating this ticket status

Issue is still reproducible on prod, So reopening this issue

@thakurveerendras kindly reverify.. A migration was still running in prod

Yes @krook, we resolved the issue last week and have recently completed updating the Added On date for contributors across all projects and organizations. However, please note that if a contributor was added to the "approval list" before August 2022, the Added On date might not be accurate. This inaccuracy stems from our migration from version V1 to version V2, where V1 did not closely track this date.
Closing this ticket.


Unclear implementation in the documentation

Hi,
I followed the initial integration documentation (https://developers.google.com/admob/flutter/quick-start#platform_specific_setup) and it said "See the Android guide for more information on configuring AndroidManifest.xml and setting app ID". In that link it shows other steps to do (like add dependency 'implementation' com.google.android.gms: play-services-ads: 21.1.0''), do I have to do them or do them the plugin itself (same question for iOS (like SKAdNetworkItems))? (I wonder because I haven't made them at the moment but it seems to work (maybe, in case they aren't needed, you can write that they are not necessary)).
I had asked this question here (https://groups.google.com/g/google-admob-ads-sdk/c/gS9bAD1Y7FA/) but they told me to ask here.

Hi @Francesco-FL, I would like to answer your questions as below:

For Android, the depencency 'com.google.android.gms:play-services-ads:21.0.0' has already added to Android library so you don't need to add it into your project's build.gradle, see:

https://github.com/googleads/googleads-mobile-flutter/blob/219a3ccf218c29ffe7aa0b16106fb4f0bcd6049e/packages/google_mobile_ads/android/build.gradle#L34

For iOS, you can see there is no Info.plist file in iOS library so you need to update your project's Info.plist (add GADApplicationIdentifier and SKAdNetworkItems for example)

Does the above explanation make sense?

Now it's clear, thanks

Closing, as it's solved above. Thank you!


Error when downloading artifact

I'm getting this error when trying to download the artifact in 1 of the VMs of the environment:
EISDIR: illegal operation on a directory, open 'C:\azagent\B1_work\1\drop???.zip'
at Error (native)
##[debug]Microsoft.VisualStudio.Services.Agent.Util.ProcessExitCodeException: Exit code 7 returned from process: file name 'C:\azagent\B1\externals\node\bin\node.exe', arguments '"C:\azagent\B1\_work\_tasks\DownloadBuildArtifacts_a433f589-fce1-4460-9ee6-44a624aeb1fb\0.178.0\main.js"'.
   at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.NodeHandler.RunAsync()
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync()
   at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 861ce7a5-4fa4-55f1-2c97-46a9cc6a2f66
Version Independent ID: bdf954b9-9bbe-8b34-b674-71ad8153519d
Content: Publish and consume artifacts in pipelines - Azure Pipelines and TFS
Content Source: docs/pipelines/artifacts/pipeline-artifacts.md
Product: devops
Technology: devops-cicd-artifacts
GitHub Login: @ramiMSFT
Microsoft Alias: rabououn

@hosseinnassiri Hossein, sorry for the delay in response. It looks like you're working through an issue with your scenario or implementation, rather than an issue with the documentation. Please consider these resources:

Azure DevOps Services
Azure DevOps Support Bot
Azure DevOps on Stack Overflow

it's an issue when agent tries to download a zip artifact. it works when it's not zip.

@hosseinnassiri If this is a product issue, to make sure it gets in front of the right people, please submit your bug here.


Article link is wrong?

See screen shot

Article of "Fine-tune Mistral-7b with SFT" is the same as "Fine-tune Llama 2 with SFT".
Maybe we should remove the row? I cannot find the article about "Fine-tune Mistral-7b with SFT" in the blog website.

Yes it's not super clear but the idea is that you can replace Llama 2 7B with Mistral-7B in this case and it would be the same article. I didn't write one specific to Mistral-7B because there's basically zero change

oh, I see, will close this.


Update the gitops guide to use new kfctl and kustomize instead of ksonnet

https://www.kubeflow.org/docs/use-cases/gitops-for-kubeflow/
The guide needs updating for Kubeflow v0.6 and v0.7 - it should use kustomize instead of ksonnet, and should use the new kfctl binary rather than kftcl.sh.

@dthomson25 Is this something you can take a look at?

Hallo @dthomson25 just wondering if you have bandwidth to update the Kubeflow gitops guide to use the latest kfctl and config?
https://www.kubeflow.org/docs/use-cases/gitops-for-kubeflow/

I would suggest just deleting this page. Our gitops story is incomplete so my suggestion would be to just delete this page until we have a more complete story for gitops and as part of that add documentation.

+1 to deleting this page. That is, remove the file entirely. I've searched the docs for references to this page. There's just one reference - we should remove the following line:
https://github.com/kubeflow/website/blob/master/content/_redirects#L101

/assign


Migrations being called while running the app

Hi,
This is a question for a weird behaviour, which I don't know.
So, after taking some time monitoring the app, I have noticed the migrations is being called for no reason since we haven't added any code for checking the migrations.
Scenario: After entering the page, the ReceiptsController is called (GetReceipts) and then, it goes to the database to select some data and return.
Why Am I getting the select below? And Is there any reason about the exception that I got?
Log:
2016-11-07 10:44:49 [Debug] Selected output formatter '"Microsoft.AspNetCore.Mvc.Formatters.JsonOutputFormatter"' and content type '"application/json"' to write the response.
2016-11-07 10:44:49 [Information] Executing ObjectResult, writing value "Microsoft.AspNetCore.Mvc.ControllerContext".
2016-11-07 10:44:49 [Information] Executed action "Company.Server.Host.Controllers.ReceiptsController.GetReceipts (host)" in 2896.5183ms
2016-11-07 10:44:49 [Information] Request finished in 3040.7953ms 200 application/json; charset=utf-8
2016-11-07 10:44:49 [Debug] Connection id ""0HL06UA07TIB2"" completed keep alive response.
2016-11-07 10:44:49 [Information] Executed DbCommand (477ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
SELECT EXISTS (SELECT 1 FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid=c.relnamespace WHERE c.relname='__EFMigrationsHistory');
2016-11-07 10:44:49 [Information] Executed DbCommand (321ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
SELECT "MigrationId", "ProductVersion"
FROM "__EFMigrationsHistory"
ORDER BY "MigrationId";
2016-11-07 10:44:49 [Error] An unhandled exception has occurred while executing the request
System.InvalidOperationException: An operation is already in progress.
at Npgsql.NpgsqlConnector.StartUserAction(ConnectorState newState)
at Npgsql.NpgsqlCommand.d__4.MoveNext()
.NET Command Line Tools (1.0.0-preview2-003148)
Product Information:
Version: 1.0.0-preview2-003148
Commit SHA-1 hash: 18ca244551
Runtime Environment:
OS Name: Mac OS X
OS Version: 10.11
OS Platform: Darwin
RID: osx.10.11-x64

I'd like to see a bit more info on exactly what's happening when you get the InvalidOperationException - can you please post some code?

Closing as no additional info was provided, but feel free to post back here and I'll reopen if necessary.

Sorry for the delay @roji I thought I had already posted. This is was solved. A middleware was running the migrations when using the dev environment.
Thanks


Why the document's guide and this sample is different? ViewBinding Sample

Question
In Android Developers guide  (https://developer.android.com/topic/libraries/view-binding)
we nullify binding vairable after super.onDestroyView()
override fun onDestroyView() {
    super.onDestroyView()
    _binding = null
}

But when we look at the viewbinding sample,
the binding variable is nullified before super.onDestroyView()
override fun onDestroyView() {
        // Consider not storing the binding instance in a field, if not needed.
        fragmentBlankBinding = null
        super.onDestroyView()
    }

What is the correct way? Official developers site's guide or sample?

private var _binding: ResultProfileBinding? = null
// This property is only valid between onCreateView and
// onDestroyView.
private val binding get() = _binding!!

override fun onCreateView(
    inflater: LayoutInflater,
    container: ViewGroup?,
    savedInstanceState: Bundle?
): View? {
    _binding = ResultProfileBinding.inflate(inflater, container, false)
    val view = binding.root
    return view
}

override fun onDestroyView() {
    super.onDestroyView()
    _binding = null
}

Just use like above (Sample in official docs)
closing this issue


Vendor code

Use godep https://github.com/tools/godep to include all third-party library code into the khepri repository.

There is currently a discussion about a new built tool called gb, which solves this problem more elegantly than to vendor code and rewrite import paths: https://github.com/constabulary/gb
Slides: http://go-talks.appspot.com/github.com/davecheney/presentations/reproducible-builds.slide

I need to get an overview over existing tools and how they work before I'm able to decide this.
From the top of my head, I have the following requirements:

Source code should be contained in a subdir of this repository, so that cloning it gives us all the code needed to build restic (apart from the stdlib)
No import path rewriting, that's ugly
Ideally, restic is still 'go-gettable', although this then doesn't use the vendored dependencies (but that's okay)
Ideally, users who would like to compile restic shouldn't need to be bothered with setting up $GOPATH etc, just clone the repository, run a command, and that's it.

There is also wgo, which might be a solution: https://github.com/skelterjohn/wgo

This is addressed in #197. We're still using Godeps for vendoring, but that can easily be exchanged for something else. End users only need to run go run build.go, without having to set up a GOPATH. So I consider this issue closed once #197 is merged.


generate_workspace is generating invalid shas

Migrated from issue #1131 in original repository.
It's including the rest of the .sha1 file it downloaded, not cutting it off after the sha. See https://groups.google.com/d/msg/bazel-discuss/gu8RbSdD0xY/YvI3f9azAQAJ.
One potential cause: https://github.com/bazelbuild/bazel/blob/master/src/tools/generate_workspace/src/main/java/com/google/devtools/build/workspace/maven/Resolver.java#L275?

To add one invalid case, I got something like sha1 = "html", which seems to be also relevant to the potential exception mentioned above. It should return null whenever the sha is not queryable.


Azurite 3.24.0

Resolves previously-ignored CVE in xml2js.
Does not resolve new dottie CVE. sequelize is patched, azurite is not.
@aruniverse

@mergifyio backport release/4.0.x


fix(table-react): add customized click event on row

affects: @fremtind/jkl-table-react
📥 Proposed changes
Added customRowCilck to enhance the flexibility on the table component. This is a proposal to evaluate a customized click-row-approach and remove the current onRowClick.
☑ Submission checklist

[x] I have read the CONTRIBUTING doc
[x] yarn build works locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)

💬 Further comments
We need feedback.

Delete this branch, the reason we contacted you yesterday was that it was the first thing we tried and it did not work, probably some issue with the open localhost that was tested.


[Openload] Unable To Download Video From This Openload Website

Unable To Download Video From This Openload Website

I would say I have that too, but I downgraded to v2017.02.04 and the problem happens there too, so I guess it is some change on the openload site.
I use youtube-dl via mpv, which plays video streams without downloading first and it did work properly yesterday. In case you are looking for a log to help, here
http://paste.debian.net/913133/
The pastebin link expires in 3 days and I use the youtube-dl binary from your page stored and run from my ~/bin file because my distro's version is 2 months old.

Here try this fix! #12002

How do I apply a patch to an already executable file?
$ file bin/youtube-dl
bin/youtube-dl: a /usr/bin/env python script executable (binary data)

Traceback (most recent call last):
File "/usr/local/bin/youtube-dl", line 11, in
sys.exit(main())
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/init.py", line 452, in main
_real_main(argv)
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/init.py", line 442, in _real_main
retcode = ydl.download(all_urls)
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/YoutubeDL.py", line 1796, in download
url, force_generic_extractor=self.params.get('force_generic_extractor', False))
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/YoutubeDL.py", line 696, in extract_info
ie_result = ie.extract(url)
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/extractor/common.py", line 369, in extract
return self._real_extract(url)
File "/usr/local/lib/python2.6/dist-packages/youtube_dl/extractor/openload.py", line 85, in _real_extract
first_three_chars - fifth_char * int(float(ol_id[num + 3:][:2])))
ValueError: unichr() arg not in range(0x110000) (wide Python build)
~# python --version
Python 2.6.6

Didn't try fix from PR yet.

Please move to #10408 for further discussion.


Adding (and editing) links to livestream archives

Changing title format from "Live Stream Archive" to "Coding Train Live"
Adding or editing links for:
3, 36, 59,60, 65-72, 107-116
Edit title for 87 (all challenges were in 87.3)

Thank you!!!


Reports showing data for all Customers if no location created

If you did not create any location and then have a look at the reports you will see data for all cucumber usersers.
For example 106.000 individual users, created vouchers, splash access methods and so on.

Thanks for the heads up, all fixed. Changes on way!


feat: add eslint plugin

Fixes #568 #508 #523
What this PR does:

Add TSLintPlugin to detect typescript-related lint issues and add these issues to report, including the use of any. TSLintPlugin runs at the end of migrate and upgrade, so the line numbers are settled before we run the plugin.
Change the name of the original LintPlugin to FormattingPlugin to differentiate the purposes of these two eslint-related plugins.
Filter TS diagnostics so that only those with category as 'error' will get a comment added in the file, and will be reported.

This is great! Nice job!

Rehearsal shouldn't be explicitly inlining any type; however on the chance it gets through its good to have a error in the report.
We need to be sure rehearsal is leveraging the eslintrc settings from the consuming app and not rehearsal own config.

I tried this out locally but I'm not seeing any lint violations in the report. How are you testing this?

@chadhietala Please see this screenshot for some of the eslint errors for ember-cli-pemberly-m3/addon/services/m3-schema.ts. Also in the rehearsal-js tests, additional blocks of eslint errors newly show up for existing tests.

@chadhietala I will add a count of ts lint errors in the report to increase the visibility of ts lint errors.

@lynchbomb @chadhietala When handling lint issues, we can mimic how diagnostics are handled: first round auto fixing fixable diagnostics, second round checking and reporting remaining unfixable diagnostics.
For lint issues, we do the same: first round auto fixing lint issues, second round checking remaining unfixable lint issues.
The flow goes like this:

auto fixing diagnostics (line numbers may change)
auto fixing lint issues, reformatting (line numbers may change)
checking and reporting unfixable diagnostics, adding comments (line numbers may change, but change is predictable, because we make sure the added comment is only one line, so we can report line numbers)
auto fixing lint issues, reformatting, checking and reporting unfixable lint issues (line numbers will not change, since auto fixing lint issues is mainly indenting, so we can report line numbers)

I got it to work on that project but not upload-manager. Also it seems like the lines are not correct because we pushing the items into the report before formatting it seems.

@chadhietala added eslint config to pre-emptively address the formatting issue.


As an operator, I want to review our current ALB and ELB usage for upgrades and old instances.

It appears we have a few "Classic" ELBs and possibly a few ELB's that are not in use. The task is to evaluate our current ALB/ELB usage and report back on a plan to upgrade Classic ELBs to ALB if possible as well as a plan to delete unused resources.

All load balancers are ALBs now.


Failed to use tf.ones with brackets

version: 1.4
for example:
a = tf.constant(2, dtype=tf.int32)
tf.ones(shape=(a))
# ValueError: Shape must be rank 1 but is rank 0 for 'ones_18' (op: 'Fill') with input shapes: [], [].

but
a = tf.constant(2, dtype=tf.int32)
tf.ones(shape=[a])
# works well

@jiemojiemo With 1.7.0 the issue is resolved:
$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.7.0'
>>> a = tf.constant(2, dtype=tf.int32)
>>> tf.ones(shape=(a))
<tf.Tensor 'ones:0' shape=(?,) dtype=float32>
>>>

I think you can try update your tensorflow version.


ORT Server version information is wrong

Although the latest tag is 0.1.0-RC6, both the recently published images and the UI display something prefixed with "0.0.1-RC.001..."

The SHA1s seem to be recent, though.

The tagged images at least show the correct versions, e.g.:
ORT Server: 0.1.0-RC6
ORT Core: 42.0.0

The tagged images at least show the correct versions

Sorry, where exactly are these correct versions shown?

The tagged images at least show the correct versions

Sorry, where exactly are these correct versions shown?

In the UI about screen.

In the UI about screen.

Ah, ok. So it's correct (for you) when using a tagged image exactly. It's probably wrong for us when deploying main as we probably to not fetch tags along with the branch, so that the version can be calculated correctly relative to the latest tag.

I just tested with the most recent image and see the same image. Could the fix be to make sure that the Docker build job fetches the tags from the repository? Because my local builds work correctly.

Could the fix be to make sure that the Docker build job fetches the tags from the repository?

I guess so, yes.


Dynamic fields for schemas for stats and preaggregation

Hi,  I'm  to make analytics for number of download per item.  Here is my doc structure
{ _id: "20101010/site-1/apache_pb.gif", metadata: { date: ISODate("2000-10-10T00:00:00Z"), hourly: { "0": 227850, "1": 210231, ... "23": 20457 }, minute: { "0": { "0": 3612, "1": 3241, ... "59": 2130 }, "1": { "0": ..., }, ... "23": { "59": 2819 } } }

As you see I need  to  store the hits very often in a new subdoc .  Every thing is explained here  http://docs.mongodb.org/ecosystem/use-cases/pre-aggregated-reports/ but it applies only for native mongodb. How to do it using mongoose ?

I am also facing same issue with mongoose.
Do i need to use another mongodb middleware?

What's the exact issue? Mongoose does support the aggregation framework, see api docs on aggregate.


ETCD-495: UPSTREAM : [3.5] Bump golang.org/x/net to v0.17.0

Handling in one PR: https://github.com/openshift/etcd/pull/222


i3wm inside a container

Hello,
I wanted to ask if you could give me a little tutorial how you build these images, is it the same as i would without x11docker or do I have to do some special tweaks to run x11docker --desktop test/x11i3 for example. Where the starting point of the image is i3wm.

is it the same as i would without x11docker or do I have to do some special tweaks

There is nothing special in creating Dockerfiles for x11docker.
Example:
FROM debian
RUN apt-get update
RUN apt-get install -y i3-wm
CMD i3

This should just work with your command example.


feat(logging): single line JSON tracing info on console

BREAKING CHANGE: Prints JSON tracing info as a single line for better compatibility with
log aggregators

Which problem is this PR solving?
Tracing info displayed on the console by ConsoleSpanExporter is multiline, as it is the
standard behaviour when using console.log(obj) - However, this causes trouble for people
using log aggregators (like Loki) which handle single line log messages better. Moreover, it is
a safe assumption that people using a tracing library is probably using a log aggregator as well.
This option makes it easier for the case I described while being still possible to copy->paste->format
from the terminal as well for local development or if no log aggregator is in place.

Short description of the changes
Prints JSON tracing info as a single line

Nevermind. I can roll out an exporter here on my own.
Sorry about the nuisance.


Updating CNCF Custom Functions Guide

JIRA:

Description:
This guide was outdated. Parameter signature does not need to a json node. Also KogitoProcessContext functionality needs to be documented

Please make sure that your PR meets the following requirements:

[ ] You have read the contributions doc
[ ] Pull Request title is properly formatted: KOGITO-XYZ Subject
[ ] Pull Request title contains the target branch if not targeting main: [0.9.x] KOGITO-XYZ Subject
[ ] The nav.adoc file has a link to this guide in the proper category
[ ] The index.adoc file has a card to this guide in the proper category, with a meaningful description

How to setup automatic cherry-pick to another branch ?

The cherry-pick action allows to setup automatic cherry-pick from main to a specific branch.
To allow it, you will need to add the corresponding label with pattern: backport-{RELEASE_BRANCH}.
For example, if a backport to branch 1.26.x is needed, then the label should be backport-1.26.x.
Once the PR is merged, the action will retrieve the commit and cherry-pick it to the desired branch.
NOTE: You can still add label after the merge and it should still be cherry-picked.

@hmanwani-rh @fjtirado all good with this PR? Can we merge it?


Publish 0.25.1

Publish changes to types.

Would it be possible to publish a version of magic-string that includes these updated TypeScript definitions?

Done! Thanks

Thanks Rich!


add service and seata config

What this PR does:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

Codecov Report

Merging #413 (7ea146b) into master (85c2f18) will increase coverage by 0.04%.
The diff coverage is 61.11%.

@@            Coverage Diff             @@
##           master     #413      +/-   ##
==========================================
+ Coverage   40.48%   40.52%   +0.04%
==========================================
  Files         141      142       +1
  Lines        8747     8765      +18
==========================================
+ Hits         3541     3552      +11
- Misses       4910     4917       +7
  Partials      296      296

Impacted Files
Coverage Δ

pkg/tm/service_config.go
0.00% <0.00%> (ø)

pkg/client/config.go
62.29% <100.00%> (+3.73%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Init gdirs and retrieve climate data in a more memory efficient way

I suspect the huge memory usage right now comes from xarray and the opening of huge .nc files with the climate data for each gdir. We should find a way to make this more memory efficient and avoid having too much stuff open at the same time. This might be worth a discussion with @fmaussion to see how it's done in OGGM.

How are you doing this? Opening files does not lead to memory usage unless if they are compresses since xarray reads only the selected data.
In OGGM however we process the data independantly for each gdir and add the time series as single point netcdf to the gdir

OK, I haven't properly benchmarked this, but I have an intuition from where this comes from.
Since we are working in 2D, I'm retrieving the climate data in a very similar fashion as you do in the MB_Sandbox, but I am downscaling the temperature to the DEM of the glacier with the lapse rate from W5E5. This means that I have 1000s of more data points compared to a flowline approach. I'm then putting everything together again in an xarray Dataset, and I store this again in the gdir folder like you do in OGGM. This means that when we simulate a few glaciers, I need to load all the climate data (temp, precip and snow) for each glacier, which we do before running all simulations in parallel in order to have everything available in memory. This is of course really fast, but it has a huge memory cost.
@fmaussion I'm sure you have thought about these issues. I remember thinking my implementation was pretty naive, but since we still not modelling MB per se, I didn't care much at the time. We'd need to come up with a way to make this both memory efficient and fast.

The trivial fix that comes to mind is to not store climate data on a 2D grid but compute it on the go (as OGGM does). Yes indeed, in 2D you have many more grid point, but the actual info you need for long term storage is still one single climate time series and from this you can recompute everything.
For a model run, you compute what you need for the time step you need (in OGGM we do 12 months or 365 days to cover 1 year), and store only the yearly or monthly diagnostic output (e.g. mass balance). If you don't want to store all time slices in memory you can write each time step in a pre-formatted netcdf file, i.e. dumping memory to disk when possible.
Would be interesting to know how the ice sheet people deal with that.

If I only compute the MB for the timestep of interest (e.g. for a given month) this would indeed reduce the amount of data loaded, even if it's parallelized for many glaciers. However, I'm afraid that the data manipulation might be quite computationally expensive. I'll have to benchmark this. Is this what you are doing in OGGM? You retrieve and downscale the climate series for a given glacier only for the time interval of interest? I know that often the bottleneck in simulations is read/write of files, so I'm really afraid of doing this.
Just to be clear, when I mean memory usage I mean RAM, there's no problem in reading and writing big files on disk right now.
This also makes me think that there might be either a ton of redundancy or there's something wrong with how I computed the 2D climate data. 18 glaciers and their 2D monthly climate series for 50 years take up to 140GB of RAM, so this is not normal 🥲

Is this what you are doing in OGGM? You retrieve and downscale the climate series for a given glacier only for the time interval of interest?

We retrieve the entire climate data (this is only timeseries) for the entire period of available climate, but we compute distributed MB only when asked to (usually for one year). We store the distributed MB only on demand (e.g. when people want to do hydrology).

the bottleneck in simulations is read/write of files

Yes. We read the climate data only once, and write only once. If memory becomes an issue I think that dumping to disk every now and then should be fine though.

18 glaciers and their 2D monthly climate series for 50 years take up to 140GB of RAM, so this is not norma

you should be able to compute the size of your stored arrays and estimate the RAM they would need? One trick to save memory would be to store 1D arrays of glacier pixels only instead of the 2D ones which have loads of NaNs in them.

Some progress on this side. I implemented the retrieval, processing and computation of the MB in the above mentioned way. I have a raw climate file in .nc in the folder of each gdir, which is loaded in memory at the beginning of a simulation. Then, whenever the solver is interrupted with a callback for a given timestep for the computation of the mass balance (e.g. monthly), the climate is downscaled to the current surface elevation of the glacier in 2D, and then the MB is computed with a given MB model.
However, this didn't seem to fix the memory issue. After quite a lot of debugging, I've isolated that the issue is coming from the multiprocessing of OGGM. If I stop using multiprocessing for OGGM (even if I use it for ODINN), the memory goes down to reasonable levels. I think the issue must be related to the inability of PyCall.jl to correctly call the Python garbage  collector. Moreover, I've noticed by checking all the processes using top, that for each run, OGGM seems to spawn new processes, instead of reusing the previously open ones. This makes the RAM increase a lot, and it is highly inefficient. I will investigate how to deal with this issue, but for now I think I'll stop using multiprocessing with OGGM, or at least I will severely limit the number of processes.

thanks - sorry this is causing you such trouble.

OGGM seems to spawn new processes, instead of reusing the previously open ones.

This is how multiprocessing works in python - we send N processes for N glaciers, but once a process is done it closes...
Sounds like a pycall thing - that's bad, but the multiprocessing part of OGGM is extremely trivial, it should be pretty straightforward to port this to Julia.

Yeah, I just need to figure out how to properly close "old" processes in Python from Julia. For now I'll stay off multiprocessing with OGGM, it's a small price to pay to have reasonable memory usage. I'll update this once I have more information.


Package cannot be installed

Hello
I get an error when trying to install android_packages_apps_GmsCore built from sources today.
I already have android_packages_apps_GmsCore installed that I built three weeks ago. But It seems the recent apk cannot be installed. Is my OS making problems or is the problem that I already got a version of GmsCore installed?
Using Android 7.
Anyways, here's the error-log:

01-23 12:14:13.907  1423  2667 I ActivityManager: START u0 {dat=file:///storage/emulated/0/Download/play-services-core-23.01.2017.apk cmp=com.android.packageinstaller/.InstallAppProgress (has extras)} from uid 10017 on display 0
01-23 12:14:13.952 20794 20794 W InstallAppProgress: Replacing package:com.google.android.gms
01-23 12:14:13.985   269   269 I         : free_cache(2620077) avail<PHONE_NUMBER>
01-23 12:14:14.054  1423  1520 I ActivityManager: Displayed com.android.packageinstaller/.InstallAppProgress: +135ms
01-23 12:14:14.118 20794 20808 D OpenGLRenderer: endAllActiveAnimators on 0x7f778b0400 (RippleDrawable) with handle 0x7f778a3960
01-23 12:14:14.849  1423 16972 W art     : Long monitor contention with owner Binder:1423_8 (12335) at void com.android.server.pm.PackageInstallerService$InternalCallback.onSessionSealedBlocking(com.android.server.pm.PackageInstallerSession)(PackageInstallerService.java:1278) waiters=0 in android.content.pm.PackageInstaller$SessionInfo com.android.server.pm.PackageInstallerService.getSessionInfo(int) for 227ms
01-23 12:14:15.207  1423  2254 D NativeLibraryHelper: Library 'libvtm-jni.so' is not page-aligned - will not be able to open it directly from apk.
01-23 12:14:15.207  1423  2254 W NativeHelper: Failure copying native libraries [errorCode=-2]
01-23 12:14:15.208  1423  2254 E PackageInstaller: Commit of session 904515950 failed: Failed to extract native libraries, res=-2
01-23 12:14:15.215  1423  1433 I art     : Background sticky concurrent mark sweep GC freed 47597(3MB) AllocSpace objects, 27(1272KB) LOS objects, 27% free, 12MB/17MB, paused 5.528ms total 52.589ms
01-23 12:14:15.221 20794 20794 D InstallAppProgress: Installation error code: 4
01-23 12:14:15.450  1423  2667 W art     : Long monitor contention with owner PackageInstaller (2254) at void com.android.server.pm.PackageInstallerService$InternalCallback$1.run()(PackageInstallerService.java:1253) waiters=0 in android.content.pm.PackageInstaller$SessionInfo com.android.server.pm.PackageInstallerService.getSessionInfo(int) for 237ms

@motatuc: Does the version provided here work?

@motatuc: You must zipalign the apk.

Hm. I don't remember doing this with the build three weeks ago. Anyways, zipalign did the trick.
Thanks.

@motatuc: With Android Studio it is zipaligned automatically.
Since now native libs are used directly without extraction (when possible), they need to be zipaligned.

@mar-v-in: I have never tried to compile it but doesn't the compilation script auto zipalign the file?

The gradle build process should automatically zipalign the file, the apk from the microG F-Droid repo is just the output of that build process and works perfectly on various android versions.
@motatuc
I guess the reason for your problem is that you signed manually and not as part of the build process. If you sign manually or resign the apk, you need to zipalign afterwards as the signing breaks previous zipalign. To sign as part of the build process, create a file play-services-core/user.gradle with the following content:
android {
    signingConfigs {
        release {
            storeFile file('path/to/keystore/file')
            storePassword 'keystore password'
            keyAlias 'key name'
            keyPassword 'key password'
        }
    }

    buildTypes {
	release {
            signingConfig signingConfigs.release
        }
    }
}

As this is gradle, you can also read the info from env or sth like that by using groovy code.
I will update the wiki with these build instructions later.


Add Python bindings for fast libtrue access.

Python programs have a desperate need for guaranteed truth. By binding
libtrue with cffi we allow Python programs fast access to the libtrue C
library. This is altogether a better option that re-implementing libtrue
in pure Python.
To install and use, you need the cffi port installed, then simply do the following:
% cd py-libtrue
% python setup.py build
% sudo python setup.py install
...
>>> import true
>>> true.get_true()
True

Is cffi the right approach, or would a more generic approach like Swig make more sense?

swig is python 2 only, cython allows for python 3 as well as 2


#8723 BaseVarDumper var_export() fix

Changed var_export to VarDumper::dumpAsString()

Fix is unacceptable, because dumpAsString does not maitain the PHP syntax for the complex data structure.


Unable to install on Python 3.11

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: [Windows 10]
scrcpy version: [e.g. 1.12.1]
device model:
Android version: [e.g. 12]

Describe the bug
I have Python 3.11 and wanted to install scrcpy. But unable to install it. As this version install on python version 3.7-3.10.
Please paste terminal output in a code block.
Below is command prompt output
ERROR: Ignored the following versions that require a different python version: 0.0.1 Requires-Python >=3.7,<3.10; 0.0.3 Requires-Python >=3.7,<3.10; 0.0.4 Requires-Python >=3.7,<3.10
ERROR: Could not find a version that satisfies the requirement scrcpy (from versions: none)
ERROR: No matching distribution found for scrcpy

Scrcpy is not developed for Python, does not provide Python bindings, and does not rely on Python for compilation.
So I guess you seem to be asking questions in the wrong repo?


DBeaver - Number values formatted in thousands by default.

Hi,
Is there a setting that allows us to disable the thousands number formatting in our cells?
Ex.: My database value for cell X is 34567 and it is displayed as 34,567 in DBeaver.  I'd rather set it to show the raw value with no formatting.
Thank you!

You can change numbers/datetime formats in Preferences:

Because they significantly reorganized the  Preferences menu and I had to spend 10 minutes looking for it again.

Because they significantly reorganized the Preferences menu and I had to spend 10 minutes looking for it again.

Just a tip: since the IDE is just eclipse in desguise, you can use the 'type filter text' box, type in 'data formats' and it will show you where the menu item is ;-)

Things have been rearranged slightly again, so now "Editors" is one level higher up in the tree, so do not look for "Database".

Things have been rearranged slightly again, so now "Editors" is one level higher up in the tree, so do not look for "Database".

For dbeaver 7.3.0 on Linux
select in Type = number and unmark use grouping

On DBeaver 21.1.3 (Linux) I did this: in Window > Preferences > Editors > Data Editor > Data Formats check the box for "Disable numeric formatting"

Is there a way to maintain the group separators in the TEXT version of the result set ? I see it in GRID, but then the TEXT tab does not show it. I use it to copy paste the entire result set with headers in a simple way.

Answered my own question : change Value display format from Editable to Display(default)


Consider using MEAN stack for backend

Tracking my research time with this issue. Will close if we decide against it, or if it gets incorporated in a PR.

#45


Not able to find model for type - inkedService and subType - undefined

Trying to use the @microsoft/azure-data-factory-utilities to generate arm template for af but keep receiving this error
Failed to find model for the resource, ignoring this resource. Error: {"stack":"Error: Not able to find model for type - inkedService and subType - undefined\n    at Function.e.getModelByResourceType

I tried from version 0.1.3 to 0.1.6 but keep having the same errors.
I have a repo with only some link services:

When I run this command it fails:
npm run build validate . /subscriptions/***/resourceGroups/***/providers/Microsoft.DataFactory/factories/***

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5b5392af-90b9-8dbb-663d-9ea12defe00e
Version Independent ID: 54296020-8b98-ea17-7915-5b918b173793
Content: Automated publishing for continuous integration and delivery - Azure Data Factory
Content Source: articles/data-factory/continuous-integration-delivery-improvements.md
Service: data-factory
Sub-service: ci-cd
GitHub Login: @nabhishek
Microsoft Alias: abnarain

Hi @CrazyTuna,
Thanks for leveraging Azure docs feedback channel to raise this question and appreciate for sharing your findings here.
This channel is reserved for ‘Azure docs’ feedback- for sharing doc feedback and suggesting content improvements. We didn't determine any changes for this documentation upon reviewing this feedback.
Going forward if your issue is specific to a use case, we request you to please reach out in forums like Microsoft Q&A or Stackoverflow. We have our engineers and other community members who will be happy to assist. Forums have a better suited audience for such queries.
If you have a document which you are following and if you think it needs an update, please let us know about that here.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply and reopen the issue. We will gladly continue the discussion.
Thank you


SPI

Is this library able to be used with SPI?   If I use the same variables, does it matter?

Hello, The library is intended to use with I2C only. If my memory is correct the SPI version needs different interrupts.


Handling of datatypes when migrating schema

Hi,
We are currently evaluating pg_chameleon for migrating our database from mysql to postgresql with minimal downtime.
One blocking issue we found is that in mysql booleans are represented as tinyint(1) or bit(1) while in postgresql our ORM mapper expects real booleans.
We didn't find a way to do this kind of transformation with pg_chameleon. Other tools like pgloader have this built in.
Question is can this functionality be added?

Hi, currently I have no flexibility in type handling.  I need to check if is possible to have it implemented in the version 1 or if I have to bump it to the version 2 (probably out by the end of the year.

the override is possible, however the risk of breaking the replica is very concrete.
for example if you insert a value larger that 1 in a tinyint(1) the boolean on postgres will cause an error.
I need to investigate a solid way to make it work so adding this feature to the version 1.7 at least.

@agseekda you can test the functionality on the branch type_override. The configuration file requires a mapping for the type override. the configuration-example.yaml have the example override for tinyint(1).
The override_to key specifies the type to override. This must be a postgresql type.
The override_tables allow to specify which tables will get the override. Leave the star if you want to apply the override to all the tables.
Currently the override works only at create time.
Please let me know if it works for you.
Thanks


Use WeakMap to save the listeners

__ee__ only goes so far. This property being set prevents me from freezing my object before it's set.

Having the same problem: https://github.com/medikoo/event-emitter/issues/5

It will work as advertised only with native WeakMap, as polyfill again will need to define property on key object.
Other risk implied with that solution is that two different listener maps may be created for same object if for some reason someone has two versions of event-emitter in his setup (imagine two packages, one requiring v0.2.1 and other v0.2.2 of event-emitter). Unfortunately it's not that unlikely to happen, and I've already approached similar issue with other package I maintain.
Solution I see:
Switch to WeakMap, at least for ES6 environments it'll assure that we can handle events on frozen objects.
However keep listeners map accessible on a global object (via some obscure variable), so in case of two different event emitters initialized in a process, same map is used.
Unfortunately It'll imply other problem, as we need to assign something to global object, we need to retrieve it, and the only reliable way to do it, is via new Function("return this")(), and this breaks CSP. Are we ok with that?

No knowledge about WeakMap, but  I will do my researches. Thank you :+1:


Minor corrections to the HA documentation page

@warferton the commit message misses the NO-JIRA prefix, for further details see typical-development-cycle i.e.
NO-JIRA Minor corrections to the HA documentation page


Fix Time Picker's place holder content not showing in high contrast mode

We were using the wrong resource.

/azp run


Bookmark status not updated in SpeakersDetailActivity

Actual Behaviour
Bookmark status not updated properly.
Expected Behaviour
Bookmark status to be updated properly.
Steps to reproduce it

Go to the SpeakersDetail Activity
Click on any session of that particular speaker
Bookmark that session in the SessionDetailAcitivty
On pressing back button you would observe bookmark status not updated in the SpeakersDetailActivity

LogCat for the issue

Screenshots of the issue

Would you like to work on the issue?
Yes.

@arp95  It is working fine on my device.

@mohitmanuja please follow the steps again!

@arp95 It needs to be resolved :+1:

@arp95 Go ahead


Is the A3D dataset available now?

Yes! Please give us one or two weeks to set up the website and liscenses.

Thanks a lot! @MoonBlvd

@Flawless1202
This repo has been updated and the dataset is provided!

OK! Thanks a lot!


BalDatepicker fail because allowedDates is not a function

Using the component like that
<BalDatepicker name="datepicker" v-model="datepicker" placeholder="TT.MM.JJJJ" trigger-icon />
will generate an error with  TypeError: this.allowedDates is not a function and the component will not be rendered.

Adding :allowed-dates="() => { return true; }" is a workaround ...

Hi @gagarine
This issue will be fixed with this PR => https://github.com/baloise/design-system/pull/310
Cheers,
hirsch


Can I upload the errors captured by the app?

Hi, I want to upload the app error caught by 'FlutterError.onError'. Which function should I call to implement uploading to matomo system? Thank you very much.
I tried to use the following line of code, however, I do not see the logs in the matomo system:
MatomoTracker.instance.trackEvent(
eventCategory: 'Error',
eventName: details.exception.toString(),
action: details.stack!.toString()
);
However, I can upload with the following line of code and the number of characters becomes less：
MatomoTracker.instance.trackEvent(
  eventCategory: 'Error',
  eventName: 'handleError', //details.exception.toString(),
  action: 'onError', //details.stack!.toString()
);

Thanks a lot. I think I should use ‘Matomo + Sentry‘ to implement the combination.


TERA 100 hour baseline

Hi,
I have read paper for SUPERB benchmark, and I found that in the paper , TERA was trained with LS360 data, I'm wondering is there TERA checkpoint that follow similar training method but trained with LS100 in the default upstream(I only find one trained with fbank for 200000 step, but it use only time and frequency masking, and I'm not sure whether 200000 step is enough.), or can I get one?

Hi,
I think -u tera_100hr is what you are looking for (https://github.com/s3prl/s3prl/blob/301c8db2a32afe7bacea86f56039bd8c9be5eee5/upstream/tera/hubconf.py#L60-L65).
It is trained with LS100, with time + frequency + magnitude masking.
Btw, 200k steps are more than enough for 100hrs of data.
Andy


Wrong definition of SaveAsCsv

It is:
static void SaveAsCsv(double[] data, int[] labels)
should be:
static void SaveAsCsv(double[] data, long[] labels)
Generates a compilation error when uncommenting the SaveAsCsv call.

Indeed we switched to long labels and didn't update the signature.
Good catch, thank you for fixing it!


Added Web-Image Filter python application

Added Image filter app using open-AI

I've added Web-Image filter application in python projects subsection. The app works for 3 different filters as mentioned in the issue, and displays the result to the user.
For the purpose of easy execution, i'm sharing the link of the colab notebook of the project: View colab notebook of project
Closes #3046

Checklist:

[x] I have mentioned the issue number in my Pull Request.
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md

Hey @MohitGupta121 @pranjay-poddar please check me PR. Thanks!

Hi @MohitGupta121 I've resolved the issues. Can u please mrege the PR now? Thanks!


Fix dashboard log watching

When the dashboard was running as an executable, the logic we used to detect replicas in broken. Instead of trying to be clever, watch all dashboard resources and log their output. This is more robust and future proof as we only have one dashboard running at once.
Deleted the timeout

Fixes #3612
Microsoft Reviewers: Open in CodeFlow

Enabling auto merge, need to get this into p6.

/backport to release/8.0


Resin notification setting error

161~200 resin notification setting is not available
Fix plan : 1.24.1

Fix on v1.24.1


fix: inconsistency between request redirect filter script and config.json

…json

Description:

Testing done:

Affected area:

Functional Area

New Functionality
[ ]

CI System
[ ]

CLI Tool
[ ]

Certificate Management
[ ]

Control Plane
[ ]

Demo
[ ]

Documentation
[ ]

Egress
[ ]

Ingress
[ ]

Install
[ ]

Networking
[ ]

Observability
[ ]

Performance
[ ]

SMI Policy
[ ]

Security
[ ]

Sidecar Injection
[ ]

Tests
[ ]

Upgrade
[ ]

Other
[ ]

Please answer the following questions with yes/no.

Does this change contain code from or inspired by another project?

Did you notify the maintainers and provide attribution?

Is this a breaking change?

Has documentation corresponding to this change been updated in the fsm-docs repo (if applicable)?

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (179b2fb) 47.71% compared to head (179b2fb) 47.71%.

:exclamation: Current head 179b2fb differs from pull request most recent head 6d6e399. Consider uploading reports for the commit 6d6e399 to get more accurate results

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@              Coverage Diff              @@
##           release/v1.2     #157   +/-   ##
=============================================
  Coverage         47.71%   47.71%
=============================================
  Files               194      194
  Lines             14883    14883
=============================================
  Hits               7101     7101
  Misses             7738     7738
  Partials             44       44

Flag
Coverage Δ

unittests
47.71% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Hackathon: Add Quilt Data San Jose Co-Working Location

Now that this is approved, what is the process to get it merged?

You can just hit merge whenever you are ready 🙂

I don't have write access to this repo, just the code one. Which Slack channel should I use to ask?

#github-invitations, but I merged it now


docs: Showcase users

Description
Showcase some users of Cerbos!
Checklist

[X] The PR title has the correct prefix
[X] PR is linked to the corresponding issue
[X] All commits are signed-off (git commit -s ...) to provide the DCO

Codecov Report

Merging #983 (d0e5aa2) into main (57a0bc4) will increase coverage by 0.18%.
The diff coverage is n/a.

:exclamation: Current head d0e5aa2 differs from pull request most recent head ddf308e. Consider uploading reports for the commit ddf308e to get more accurate results

@@            Coverage Diff             @@
##             main     #983      +/-   ##
==========================================
+ Coverage   54.84%   55.03%   +0.18%
==========================================
  Files         106      106
  Lines       11663    11663
==========================================
+ Hits         6397     6419      +22
+ Misses       4732     4712      -20
+ Partials      534      532       -2

Impacted Files
Coverage Δ

internal/engine/engine.go
65.44% <0.00%> (+0.78%)
:arrow_up:

internal/storage/index/builder.go
84.35% <0.00%> (+5.02%)
:arrow_up:

internal/compile/errors.go
55.55% <0.00%> (+15.87%)
:arrow_up:


Factorio instance, map-gen-settings within UI along with server-settings?

This is not for requesting support for new games/applications
To do this you should go to https://github.com/CubeCoders/AMPTemplates and first attempt to build a configuration yourself - otherwise you can request a template from this repo.
Feature Request
The ability to edit the mapgen settings of factorio within amps UI along with server settings. I have to do a work around by making a new folder, renaming it and making the mapgen setting in an extremely inconvenient way.  With LinuxGSM I did this by "cp map-gen-settings.example.json map-gen-settings.json" Within factorio
Feature Information:

I confirm:

[ x] that I have searched for an existing feature request matching the description.

Not possible at this time, AMPs configuration can't handle configuration files with nested structures. For now use AMPs file manager to change these.


Need regularly build using by CI service

Have to check dependency of Alpine Linux's package.

Build using my private VPS and CircleCI API.


[CSI Sanity] Should fail when the volume source snapshot is not found

Fix CSI sanity test failure.
    unexpected error: invalid csi snapshot type: . Must be snap or bak
    Expected
        <codes.Code>: 3
    to equal
        <codes.Code>: 5

https://github.com/longhorn/longhorn/issues/2271

It seems that the CSI should return error NotFound once there is an empty result from decodeSnapshotID(snapshot.SnapshotId)

Updated.


GraphQL Client with Schema showcase

Describe your feature request
With the addition of GraphQL to Surreal, it would be nice to view the schema in Surrealist and send queries and mutations (and perhaps save named queries similar to SQL).
Inspiration can be taken from Apollo GraphQL Studio which is considered the best GraphQL IDE in the market.
Are you willing to contribute this feature?
Unfortunately, I cannot due to time constraints
Related pull request (if applicable)

Initial support for a GraphQL view has been implemented in #394. While currently it does not offer introspection, documentation, and auto completion, these are features we intend to implement in the near future