[fix][broker] fix the global topic policy config override the local topic policy config

Fixes #21202
Motivation
fix the global topic policy config override the local topic policy config
Modifications

Divide variable org.apache.pulsar.common.policies.data.PolicyHierarchyValue.topicValue into globalTopicValue and variable localTopicValue
Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:

@mattisonchao  This pr does not fix the problem you mentioned(same event key), do I need to submit another pr to fix it or fix it together in this pr?

topicValue in PolicyHierarchyValue could only be local policies.
https://github.com/apache/pulsar/blob/66271e3bf3cf0699789c759d852c24e6f00f90cd/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SystemTopicBasedTopicPoliciesService.java#L387-L397

@Technoboy-
There are two places where the topicValue in PolicyHierarchyValue will be modified

Initialization(you mentioned above),  the current implementation is only replay the local policy. I think we should replay both of local and global policies to avoid losing information.
update in org.apache.pulsar.broker.service.SystemTopicBasedTopicPoliciesService#readMorePolicies (mentioned in this issue)

2. update in org.apache.pulsar.broker.service.SystemTopicBasedTopicPoliciesService#readMorePolicies (mentioned in this issue)

ah, good catch.  please raise a new pr to fixing it.

there are three issues here. right?

local and global topic policies using the same variable org.apache.pulsar.common.policies.data.PolicyHierarchyValue.topicValue,  there's have chance to overwrite each other
during init policesCache, only replay the local policy message, global policies will not take effect.
global and local topic policies using the same key. it will be compacted by the compactor.

I fixed the first two in this pr,  the third issue I'll fix with another pr.
@Technoboy-  @aloyszhang PTAL

Could you please add a test to verify the changes and explain how you can avoid the compactor to override local policy with global policy(same event key) :)

I have fixed this issue and added a test case in my latest commit, PTAL,  thanks
CC @Technoboy- @aloyszhang @coderzc

/pulsarbot run-failure-checks

Also, we need to clean the global topic policy in cleanCacheAndCloseReader.
https://github.com/apache/pulsar/blob/09a17203702a032274a56d7d0ff03c9e32b4529f/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SystemTopicBasedTopicPoliciesService.java#L443-L461

Also, we need to clean the global topic policy in cleanCacheAndCloseReader.
https://github.com/apache/pulsar/blob/09a17203702a032274a56d7d0ff03c9e32b4529f/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SystemTopicBasedTopicPoliciesService.java#L443-L461

fixed and added a test to cover this change, PTAL

Plus, we didn't support the global policy very well. There are some issues I know so far.

The global topic policy and the local topic policy use the same message key, and they will overwrite each other by topic compression.
We did not use the global policy correctly. In some places, we only get local policies.

I'm sorry, but I don't think this change suits me.
There are some reasons are as follows:

Apache Pulsar has the broker/namespace/topic level policy. The global topic policy is just a particular topic policy based on the geo-replication. We should refrain from introducing this concept into the topic entity.
We should encapsulate this concept into topic policy service, which can decide to give the topic a global or local policy because the topic doesn't care if the policy is global or not.

Therefore, I would like to let [SystemTopicBasedTopicPoliciesService.java](https://github.com/apache/pulsar/pull/21212/files#diff-9d2948d863c111e4be6d508a1c573667a1326b98c4314e917ba9e344bb61dc27) decide to update global or local policy to the topic to avoid isGlobal param everywhere.
WDYT? /cc @codelipenghui @Technoboy- @poorbarcode

I agree, It is indeed we should encapsulate this concept into topic policy service. I will make some changes as below:

rollback PolicyHierarchyValue to its previous state (use topicValue replace localTopicValue and globalTopicValue)
merge the local and global topic policies into one TopicPolicies at SystemTopicBasedTopicPoliciesService before init topic and notify topic.  therefore  org.apache.pulsar.broker.service.persistent.PersistentTopic#onUpdate method doesn't care if the policy is global or not.  but the org.apache.pulsar.common.policies.data.TopicPolicies#isGlobal becomes meaningless in the topic context. so we should not use this variable in the topic context.
Any suggestions?

I'm sorry, but I don't think this change suits me.
There are some reasons are as follows:

Apache Pulsar has the broker/namespace/topic level policy. The global topic policy is just a particular topic policy based on the geo-replication. We should refrain from introducing this concept into the topic entity.
We should encapsulate this concept into topic policy service, which can decide to give the topic a global or local policy because the topic doesn't care if the policy is global or not.

Therefore, I would like to let [SystemTopicBasedTopicPoliciesService.java](https://github.com/apache/pulsar/pull/21212/files#diff-9d2948d863c111e4be6d508a1c573667a1326b98c4314e917ba9e344bb61dc27) decide to update global or local policy to the topic to avoid isGlobal param everywhere.
WDYT? /cc @codelipenghui @Technoboy- @poorbarcode

I agree, It is indeed we should encapsulate this concept into topic policy service. I will make some changes as below:

rollback PolicyHierarchyValue to its previous state (use topicValue replace localTopicValue and globalTopicValue)
merge the local and global topic policies into one TopicPolicies at SystemTopicBasedTopicPoliciesService before init topic and notify topic.  therefore  org.apache.pulsar.broker.service.persistent.PersistentTopic#onUpdate method doesn't care if the policy is global or not.  but the org.apache.pulsar.common.policies.data.TopicPolicies#isGlobal becomes meaningless in the topic context. so we should not use this variable in the topic context.
Any suggestions?

those changes have been done in my latest commit, PTAL, thanks @mattisonchao

@chenhongSZ Thanks for your great work! I will review it today.

Sorry for some other tasks. I have to postpone this review.


Flowtype highlighting off in Classes with @-decorators and assigned members

Thanks for getting a fix to #248 out so quickly!
Version 3.7.1 does fix the example I posted, but I've noticed that the bug persists with a very specific arrangement using decorators and assigned class members. I can't describe it exactly, so hopefully the screenshot makes sense. (I'm working on a project using mobx, which provides a @observable decorator):

Ok i'll take a look. It's quite tricky when the code is using ASI as I have to determine when a new statement starts without having a nice semi-colon to let me know. Not an issue when using an AST based parser but difficult when using textmate grammars as used in Atom. It looks straightforward though.

You're right that there isn't a problem when you put the semi-colons in,
which is what I normally do. My ESlint config apparently doesn't require
them inside of classes.
On Sun, Sep 11, 2016 at 12:02 PM, Graham<EMAIL_ADDRESS>wrote:

Ok i'll take a look. It's quite tricky when the code is using ASI as I
have to determine when a new statement starts without having a nice
semi-colon to let me know. Not an issue when using an AST based parser but
difficult when using textmate grammars as used in Atom. It looks
straightforward though.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/gandm/language-babel/issues/249#issuecomment-246190933,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJxfg79DJ2yw_A5EIqcTzesRHWcixDFZks5qpDQbgaJpZM4J6CyM
.

This should be fixed in 2.38.0


 See below:
RETURN CODE -6

main: seed =<PHONE_NUMBER>
llama_model_load: loading model from '/usr/src/app/weights/7B-native.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx = 512
llama_model_load: n_embd = 4096
llama_model_load: n_mult = 256
llama_model_load: n_head = 32
llama_model_load: n_layer = 32
llama_model_load: n_rot = 128
llama_model_load: f16 = 2
llama_model_load: n_ff = 11008
llama_model_load: n_parts = 1
llama_model_load: type = 1
llama_model_load: ggml ctx size = 4273.34 MB
llama_model_load: mem required = 6065.34 MB (+ 1026.00 MB per state)
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc

### Confirmations

- [X] I'm running the latest version of the main branch.
- [X] I checked existing issues to see if this has already been described.

Same happening to me

Is this still happening with the latest tag @sshadmand @zhukovgreen

Tested. All works fine now. Thank you


Make better use of Jackson

Better use of Jackson will help us to:

Eliminate repetitive conversion code from many methods
Replace remaining Maps with strongly-typed POJOs.

Progress report

Create POJOs for JsonRpcRequest and JsonRpcResponse: 0523aeb58c14e9ba578ed1060af7bff40ce987bb
Start customizing the ObjectMapper with a CoinDeserializer: 91f7cc1d67ad92956865ca9e543b76f01f839cab

What do you think, @dexX7 ?

TODO:

Split current bitcoinj-json project into two modules: -json (jackson stuff) and -server (services)
Make bitcoinj-rpcclient depend on bitcoinj-json
Move serializers and deserializers from -rpcclient into a Jackson module in bitcoinj-json.

These changes are fantastic, really a great improvement!
In Omni Core we currently have a few results where some fields are omitted, depending on the state, e.g.  whether a crowdsale is closed, or active, whether a transaction is confirmed or unconfirmed, which makes mapping to plain data objects probably pretty difficult. Makes me wonder..

POJOs to create:

Block object - getBlock
Array/List of block hashes - setGenerate
"Raw" transaction as a map - signRawTransaction. getRawTransactionMap
List of ?? - listReceivedByAddress
List of UTXO - getTxOut
?? - getTransaction (what should this return?)
Information object - getInfo
?? - getChainTips

some fields are omitted, depending on the state ... which makes mapping to plain data objects probably pretty difficult.

Can't those fields just be nullable in the POJO?

List of UTXO - getTxOut

It returns a single unspent output or null, if the output is spent.

Can't those fields just be nullable in the POJO?

Ah, sure.

List of UTXO - getTxOut

It returns a single unspent output or null, if the output is spent.

Thanks. Fixed, above.

@dexX7 Finished all but (8) and (9) in the POJOs to create list. Any ideas or suggestions on (8) or (9)?

If getTransaction refers to gettransaction, then this is a call to get a wallet transaction, which has some wallet specific information (such as listing potential conflicts, or the time of creation). If getRawTransactionMap refers to getrawtransaction, then it might either be mapped to a BitcoinJ transaction, or a new type: the verbose version has information about the chain (blocktime, blockhash, confirmations (stateful)).

Yes. In addition to mapping to org.bitcoinj.core.Transaction we're going to need to create POJOs for both gettransaction and getrawtransaction. They're required for making the server version and, as you point out, they contain info that is not available in bitcoinj's Transaction.

Finished in v0.0.10:  ff680a00f946af20f60a6ff52851609ee1f60671.


Add Test Cases for 'Accessibility Testing'

Add Test Cases for 'Accessibility Testing'

I would like to work on it


Errors

made a fresh install of gridsome using "yarn global add @gridsome/cli" and so on from the "getting started"
When calling gridsome deveolp the process stops after several errors. Any help, please?
here the errors
PS:: using MacOS with uptodate npm
$ gridsome develop
/Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/bindings/bindings.js:135
  throw err;
  ^

Error: Could not locate the bindings file. Tried:
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/build/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/build/Debug/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/build/Release/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/out/Debug/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/Debug/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/out/Release/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/Release/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/build/default/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/compiled/12.1.0/darwin/x64/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/addon-build/release/install-root/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/addon-build/debug/install-root/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/addon-build/default/install-root/sharp.node
 → /Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/lib/binding/node-v72-darwin-x64/sharp.node
    at bindings (/Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/bindings/bindings.js:126:9)
    at Object.<anonymous> (/Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/lib/constructor.js:10:34)
    at Module._compile (internal/modules/cjs/loader.js:759:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:770:10)
    at Module.load (internal/modules/cjs/loader.js:628:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Module.require (internal/modules/cjs/loader.js:666:19)
    at require (internal/modules/cjs/helpers.js:16:16)
    at Object.<anonymous> (/Users/XXX/Desktop/Airtable/gridsome/airtable/node_modules/sharp/lib/index.js:3:15)
    at Module._compile (internal/modules/cjs/loader.js:759:30)
XXX airtable
$

I get the same on Linux Mint - fresh install of node, npm, yarn, vue and gridsome cli

Sorry to hear. I thought I will be the only one. Also tried to fresh install everything you mentioned too.
One thought of mine was, if there is something with python? I am not sure, but I have a feeling that linking is broken.

Do you use Node.js v12? The Sharp version Gridsome use doesn't support v12. We'll publish a fix in a few hours.

Yes. I am using node 12.2.0 and npm 6.9.0, yarn 1.6.0, vue 3.5.5, gridsome/cli 0.0.9
So I will wait. Thanks

Same here (Windows 10, latest build). Thank you. Will wait also.

I think I was using Node.js v12, but not sure as I am not near that PC right now.

Seems to be a problem with sharp - getting similar errors with Gatsby. Hours of searching and no solutions found.

Node.js 12 and Sharp should work better with Gridsome 0.6 (published a few hours ago).

Closing, as this should not be a problem anymore in 0.6. Please reopen if problem persists.


ci: add release please workflow

What kind of change does this PR introduce (bug fix, feature, docs update, ...)?
CI
What is the current behaviour (you can also link to an open issue here)?
Current release CI is broken and requires manually editing CHANGELOG
What is the new behaviour (if this is a feature change)?

Merge develop into main.
Release Please will create the release PR going to main.
Merge the generated release PR.
Package will be published to NPM.

Other information:
the first time a new release is run the CHANGELOG will still need to be manually updated, this is because the way the previous CI did things broke the CHANGELOG for the next release. After this the CHANGELOG should be running correctly again

Codecov Report

Merging #206 (d2fd293) into develop (edb0c5e) will decrease coverage by 0.20%.
Report is 22 commits behind head on develop.
The diff coverage is 8.13%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@             Coverage Diff             @@
##           develop     #206      +/-   ##
===========================================
- Coverage    19.72%   19.53%   -0.20%
===========================================
  Files           16       15       -1
  Lines         1034     1065      +31
  Branches       358      373      +15
===========================================
+ Hits           204      208       +4
- Misses         808      834      +26
- Partials        22       23       +1

Files Changed
Coverage Δ

src/lib/lsp-factory.ts
16.66% <0.00%> (ø)

src/lib/services/key-manager.service.ts
17.14% <0.00%> (-0.51%)
:arrow_down:

src/lib/services/universal-receiver.service.ts
23.33% <0.00%> (ø)

src/lib/services/universal-profile.service.ts
15.74% <3.44%> (+0.02%)
:arrow_up:

src/lib/helpers/deployment.helper.ts
22.13% <9.09%> (-4.25%)
:arrow_down:

src/lib/services/digital-asset.service.ts
7.10% <16.66%> (+0.37%)
:arrow_up:

src/lib/helpers/config.helper.ts
100.00% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


unable to show graphics

looks like yahoo finance is not working, im getting an animation of progressive dots

any thoughts ?

Running into the same issue here. I can still see the Volume and Options charts on my end though

Me too, getting progress dots... no data coming in. Maybe yahoo is blocking these.
Suggest using a free API such as: twelve-data1.p.rapidapi.com which allows every user to provide their own API key (on linux systems RAPIDAPIKEY is obtained from user ~./bashrc).

Thought it was just me, also getting this issue.

same

Same problem for me. kmymoney stopped updating from yahoo and the new URL that works is https://query1.finance.yahoo.com/v8/finance/chart/%1
If I could manage to compile this without having to go through dependency hell I would get in there and try to fix this.

BTW the only thing that would need to change in the code is likely V7 to V8

Tried to change it and compile locally as well, likewise ran into dependency hell. :(

Hey all, I just did a quick check and it appears to be the https://query1.finance.yahoo.com/v10/finance/quoteSummary/SPY endpoint which is failing. I use this to get company profile info & current price info. I already use v8 for chart endpoint, I've confirmed that works.
I'm getting {"finance":{"result":null,"error":{"code":"Unauthorized","description":"Invalid Cookie"}}} on the v10/finance/quoteSummary endpoint. Does anyone know an alternative endpoint to get this information?

This has been fixed in https://github.com/tarkah/tickrs/pull/146
The quotes endpoints now require a cookie & crumb value to work. I'll cut a release soon.

This has been fixed in #146
The quotes endpoints now require a cookie & crumb value to work. I'll cut a release soon.

Thanks for getting this fixed! I can also confirm it's now working in Arch after update pacman -Syu

it's still not working for me, 0.14.9 :(
i remember a couple of weeks ago there was an issue with the yahoo enpoints and the country where the request was made from, for example, those in the EU had a different signature (or something like that) that those in the US, i'll try to find the issue

@jbrodriguez Do you have a long update interval by any chance? I noticed that when I start tickrs the issue still appears until the first update tick.
For example, I have the interval set to 120s. So the issue still appears until 120s has elapsed after running the app.

thank you @JacquardCauseway, i had the interval set to 60, changed to 5, but same issue,  this is my command
tickrs --show-x-labels --summary --update-interval 5  --symbols VOO,BTC-USD,CL=F,^GSPC

it's still not working for me, 0.14.9 :(
i remember a couple of weeks ago there was an issue with the yahoo enpoints and the country where the request was made from, for example, those in the EU had a different signature (or something like that) that those in the US, i'll try to find the issue

If you have a VPN or if you can sign up to a free VPN to test your theory that would help.

Actually if you can tell me what country you would like to test I can try it on my VPN if you're okay with that

hi @SigmaticThar , thanks for the offer ! does it work for you from mexico ?

hi @SigmaticThar , thanks for the offer ! does it work for you from mexico ?

Yes I just tested Mexico and it works.

At this point I think you should try from a free VPN on your end and use a US server and that will tell you for sure if it's something on your computer or a country connection. I also use the program mop and while I was connected to London it wasn't working so your country theory is possible.

Are you using a pihole or adblocker? I noticed the fc.yahoo.com domain was blocked on mine and I had to whitelist it. We have to hit this endpoint to get a cookie.

thanks guys, i'll try both the vpn and the adblocker check, yes i'm running adblock on my router, i'll test and report back

success !! i whitelisted fc.yahoo.com and got it working ! thank you !


Add Invoice Macaroon to Settings

We are adding a CLN connection to Zaprite. To make it easier for users to connect, it would be great if they had an option to copy an Invoice Macaroon from the Settings screen.
Maybe there could be a toggle to allow the user to choose which one they needed?

I am trying to create an "invoice" rune via the cli. i basically need a limited access rune to create an invoice and to read if that invoice was paid. i tried creating a base64 macaroon using the cli:
lightning-cli createrune restrictions='[["method=invoice,method=listinvoices"]]'

But i get this error when hitting the rest api
Bad Macaroon!

Any ideas?

Restrictions are the array of alternatives, so your command should look something like:
lightning-cli createrune restrictions='[["method=invoice"], ["method=listinvoices"]]'

https://github.com/ElementsProject/cln-application/pull/56

Here is what I have it looking like.
@ShahanaFarooqui is the rest 'Macaroon' supposed to be blank?

Here is what I have it looking like. @ShahanaFarooqui is the rest 'Macaroon' supposed to be blank?

Do you have c-lightning-rest running? If yes, have you set APP_CORE_LIGHTNING_REST_CERT_DIR correctly in your environment for macaroon to be read successfully from the location?
Also, adding invoice rune on the c-lightning-rest screen doesn't make sense. It is the old rest API library which generates its own macaroon. It does not support runes. We do not have a connect screen & configured variables (like clnrest-port, host, etc.) required for clnrest library yet.

Oh okay thanks, so I'll update the pr to hide the Invoice Rune when REST is selected.

@john-zaprite c-lightning-REST only supports access.macaroon (admin). Without c-lightning-REST generating invoice macaroon, the UI cannot provide this information. Also, with upcoming sunset of c-lightning-REST, it will not be possible to add that in future also.

Thanks @evansmj for looking into this.
@ShahanaFarooqui so would you recommend using LNMessage instead of the REST endpoint? https://github.com/aaronbarnardsound/lnmessage

There are many ways to connect to Core Lightning node as explained here. You can choose any one of them according to your requirement. lnmessage is a NodeJS commando client. For REST API solution, clnrest is the built-in Python REST server.

Updated https://github.com/ElementsProject/cln-application/pull/56 to hide Invoice Rune from the rest and grpc menus.

Closing with PR #56

Thanks @evansmj and @ShahanaFarooqui ! Super excited to integrate this into Zaprite!


'tkill' is not found in libc now in Android 10

I tried to replace tkill by tgkill, but it seems useless in Android 10.
Any ideas? T T
Device: Huawei P30

Actually tgkill report an exception:

Error: illegal instruction


ignore them

#128

@yegor256 Thanks for the contribution! You've earned +5 points for this: +20 as a basis; +5 for 20 hits-of-code; -15 for the lack of code review; -5 for 0 comments. Please, keep them coming. Your running balance is +40.


Zephyr IP Stack Leaks in Promiscuous Mode

Describe the bug
I have a networking based application that provides 2 interfaces under the Native-POSIX board target, a simulated wired and simulated wireless interface.
The wired interface provides support for 2 UDP receive sockets, which can optionally (conditional compilation) be replaced with a singular raw layer 2 socket in a single thread.  The raw socket is the method that I need as I need Layer 2 support to be able to forward on ARPs and ICMPs across a network.
There is also a separate thread bound to a different UDP port.
A Telnet server is active on this interface to provide status and control.
In my current Zephyr configuration, when data is ingress into the raw socket (or the UDP sockets) in combination with the CONFIG_NET_PROMISCUOUS_MODE kernel configuration - the Zephyr stack appears to be leaking, resulting in an application failure.
This is confirmed by viewing the results of the shell output to "net allocs" command showing network buffer allocation and corresponding free (where present, lack of free shows the buffer leak during the clone procedure).
So it would seem that the stack allocations are not being properly de-registered and freed.
I am looking for some suggestions on where to dig into determine the problem and solution.
I can delay the amount of ingress traffic before the exception/failure by increasing the number of network buffers, but eventually it will happen.
I can get the leak to not occur if I disable CONFIG_NET_PROMISCUOUS_MODE in the kernel configuration, but then I can not pass Layer 2 traffic.
The problem is I need support for this and Layer 2 support in general using a raw socket to handle L2 traffic to
the primary wired interface to the application.  I need this support as well as a standard UDP socket and Telnet, all within the same application.  This setting makes a copy of the packet that is not being properly freed, but I am not
sure what mechanism would do the free.  I am reading the raw socket using the Linux/Zephyr standard
multiplexed POSIX i/o using the zephyr_select() statement, wherein I would expect the Zephyr cloned packet
to be freed after the read occurs.
I am using the standard POSIX multiplexed i/o "select" call to handle the Ethernet interfaces similar to the following:
        // Wait for data to read or write between Ethernet interfaces or a timeout
        // NOTE: Need to set CONFIG_NET_SOCKETS_POLL_MAX to the total number of sockets to poll in select()
        int count = zsock_select(maxFD + 1, &readfds, 0, 0, (struct zsock_timeval *)&tvTimeout);

The read from the raw socket uses code as follows:

            if (m_wiredInterface.isOpenRawSocket())
            {
                sock = m_wiredInterface.getRawSocket();

                // Host wired interface reception of a packet
                if (ZSOCK_FD_ISSET(sock, &readfds))
                {
                    onWiredRawSocketReceive(tvNow, true);
                }
            }

The receive handler in POSIX is the following:
    // Receive packet from wired interface
    int bytesRead = receivePacket(m_wiredInterface.getDeviceName(),
                                  m_wiredInterface.getRawSocket(),
                                  m_wiredInterface.m_recvPacket,
                                  sizeof(m_wiredInterface.m_recvPacket),
                                  m_wiredInterface.m_recvPktSize);

Where:
int App::receivePacket(const char *ifName, const int sock, unsigned char *pkt, int pktSize, int &size)
{
    size = 0;

    // Reads packets from socket from wired or wirelesss interface
    int bytesRead = zsock_recv(sock, (void *)pkt, pktSize, 0);

    // Negative value indicates read error
    if (bytesRead <= 0)
    {
        LOG_ERR("zsock_recv(): error '%s' on interface %s, socket %d, bytes read %d", log_strdup(strerror(errno)), ifName, sock, bytesRead);
        return 0;
    }
    // Success on read
    else
    {
        LOG_DBG("zsock_recv() read %d bytes on interface %s on socket %d", bytesRead, log_strdup(ifName), sock);
    }

    // Assign count of bytes read
    size = bytesRead;

    // Return count of bytes read from the interface
    return bytesRead;
}

My expectation would be that after the POSIX API call to zsock_recv, any cloning of the network packet or dynamic memory would be automatically freed.  Using the POSIX APIs, I have no access to the underlying Zephyr specific buffers to allow the application to free the cloned packet.
Clearly I do not fully understand what is required by the application using this combination of a raw socket in promiscuous mode via the kernel configuration, a separate UPD socket in another thread, and a Telnet server to co-exist properly, and the de-referencing process that occurs to mark a network packet for deallocation.
Please also mention any information which could help others to understand
the problem you're facing:

What target platform are you using?  Native-POSIX
What have you tried to diagnose or workaround this issue? Yes
...

To Reproduce
Steps to reproduce the behavior:

west build -p always -b native_posix_64 ${workspaceFolder}

Expected behavior
Network allocations are properly freed after the POSIX read when using a promiscuous raw socket.
Impact
Showstopper
Logs and console output
Here is a capture of the callstack at the point of the exception occurence.
net_pkt_cursor_init(struct net_pkt * pkt) (/home/robocom/zephyrproject/zephyr/subsys/net/ip/net_pkt.c:1516)
net_if_recv_data(struct net_if * iface, struct net_pkt * pkt) (/home/robocom/zephyrproject/zephyr/subsys/net/ip/net_if.c:3929)
process_data(_Bool is_loopback, struct net_pkt * pkt) (/home/robocom/zephyrproject/zephyr/include/zephyr/net/net_pkt.h:286)
processing_data(struct net_pkt * pkt, _Bool is_loopback) (/home/robocom/zephyrproject/zephyr/subsys/net/ip/net_core.c:141)
net_rx(struct net_if * iface, struct net_pkt * pkt) (/home/robocom/zephyrproject/zephyr/subsys/net/ip/net_core.c:365)
net_process_rx_packet(struct net_pkt * pkt) (/home/robocom/zephyrproject/zephyr/include/zephyr/net/net_pkt.h:286)
tc_rx_handler(struct k_fifo * fifo) (/home/robocom/zephyrproject/zephyr/subsys/net/ip/net_tc.c:250)
z_thread_entry(k_thread_entry_t entry, void * p1, void * p2, void * p3) (/home/robocom/zephyrproject/zephyr/lib/os/thread_entry.c:36)
posix_thread_starter(void * arg) (/home/robocom/zephyrproject/zephyr/arch/posix/core/posix_core.c:305)
libpthread.so.0!start_thread(void * arg) (/build/glibc-sMfBJT/glibc-2.31/nptl/pthread_create.c:477)
libc.so.6!clone() (/build/glibc-sMfBJT/glibc-2.31/sysdeps/unix/sysv/linux/x86_64/clone.S:95)

Here is a dump of "net allocs" from the Telnet interface menu with a simple 1 pulse per second PING traffic ingress to the primary Native-POSIX Ethernet interface.  Note that there is no corresponding free to the packet clone.
0x4f06d0/1       used      RX   net_pkt_clone_internal():1823
0x4f0658/1       used      RX   net_pkt_clone_internal():1823
0x4f05e0/1       used      RX   net_pkt_clone_internal():1823
0x4f0568/1       used      RX   prepare_non_vlan_pkt():305
0x4f04f0/1       used      RX   net_pkt_clone_internal():1823
0x4f0478/1       used      RX   net_pkt_clone_internal():1823
0x4f0400/1       used      RX   net_pkt_clone_internal():1823
0x4f0388/1       used      RX   net_pkt_clone_internal():1823
0x4f0310/1       used      RX   net_pkt_clone_internal():1823
0x4f0298/1       used      RX   net_pkt_clone_internal():1823
0x4f0220/1       used      RX   net_pkt_clone_internal():1823
0x4f01a8/1       used      RX   net_pkt_clone_internal():1823
0x4f0130/1       used      RX   net_pkt_clone_internal():1823
0x4f00b8/1       used      RX   net_pkt_clone_internal():1823
0x4f0040/1       used      RX   net_pkt_clone_internal():1823
0x4eff50/1       used      RX   net_pkt_clone_internal():1823
0x4efed8/1       used      RX   prepare_non_vlan_pkt():305
0x4effc8/1       used      RX   net_pkt_clone_internal():1823
0x4efde8/1       used      RX   net_pkt_clone_internal():1823
0x4efd70/1       used      RX   prepare_non_vlan_pkt():305
0x4efe60/1       used      RX   net_pkt_clone_internal():1823
0x4efc80/1       used      RX   net_pkt_clone_internal():1823
0x4efc08/1       used      RX   net_pkt_clone_internal():1823
0x4efb90/1       used      RX   net_pkt_clone_internal():1823
0x4efb18/1       used      RX   net_pkt_clone_internal():1823
0x4efaa0         free      RX   prepare_non_vlan_pkt():305 -> processing_data():159
0x4efcf8         free      RX   net_pkt_clone_internal():1823 -> zpacket_recvfrom_ctx():253

0x4f0040/1       used      RX   net_pkt_clone_internal():1823
0x4eff50/1       used      RX   net_pkt_clone_internal():1823
0x4efed8/1       used      RX   prepare_non_vlan_pkt():305
0x4effc8/1       used      RX   net_pkt_clone_internal():1823
0x4efde8/1       used      RX   net_pkt_clone_internal():1823
0x4efd70/1       used      RX   prepare_non_vlan_pkt():305
0x4efe60/1       used      RX   net_pkt_clone_internal():1823
0x4efaa0/1       used      RX   net_pkt_clone_internal():1823
0x4efc80/1       used      RX   net_pkt_clone_internal():1823
0x4efc08/1       used      RX   net_pkt_clone_internal():1823
0x4efb90/1       used      RX   net_pkt_clone_internal():1823
0x4efb18/1       used      RX   net_pkt_clone_internal():1823
0x4efcf8/1       used      RX   net_pkt_clone_internal():1823
0x4ef9b0/1       used      RX   net_pkt_clone_internal():1823
0x4ef938/1       used      RX   net_pkt_clone_internal():1823
0x4ef8c0/1       used      RX   prepare_non_vlan_pkt():305
0x4efa28/1       used      RX   net_pkt_clone_internal():1823
0x4dff60         free      TX   arp_prepare_reply():502 -> ethernet_send():717
0x4a9710         free   TDATA   ethernet_fill_header():504 -> ethernet_remove_l2_header():586

Environment (please complete the following information):

OS: (Linux)
Toolchain (e.g Zephyr SDK zephyr-sdk-0.13.1, .Booting Zephyr OS build v2.7.99-7456-g8566b00df3e1.)

Additional context
Project Kernel Configuration is as follows:
# Zephyr Project Kernel Configuration (common for all build board targets)

# C++ support
CONFIG_CPLUSPLUS=y
CONFIG_LIB_CPLUSPLUS=y
CONFIG_STD_CPP2A=y
CONFIG_CBPRINTF_FP_SUPPORT=y

# General kernel options
CONFIG_INIT_STACKS=y

# fs_dirent structures are big
CONFIG_MAIN_STACK_SIZE=2048

# Size of heap memory pool
CONFIG_HEAP_MEM_POOL_SIZE=16384

# Co-operative threading disabled, want pre-emption
# A cooperative thread remains the current thread until
# it performs an action that makes it unready
CONFIG_NET_TC_THREAD_COOPERATIVE=n

# Base 64 conversions support for XML interface
CONFIG_BASE64=y

# Flash storage settings
# Optionally force the file system to be recreated
CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_FLASH_PAGE_LAYOUT=y
CONFIG_FILE_SYSTEM=y
CONFIG_FILE_SYSTEM_LITTLEFS=y
CONFIG_FS_LOG_LEVEL_DBG=y
# CONFIG_APP_WIPE_STORAGE=y

# Networking config
# Need TCP for Telnet session for shell interface
CONFIG_NETWORKING=y
CONFIG_NET_UDP=y
CONFIG_NET_TCP=y
CONFIG_NET_IPV6=n
CONFIG_NET_IPV4=y
CONFIG_NET_DHCPV4=n
CONFIG_NET_ARP=y
CONFIG_NET_NATIVE=y
CONFIG_NET_L2_ETHERNET=y
CONFIG_NET_DISABLE_ICMP_DESTINATION_UNREACHABLE=y

# Network drivers buffers
# Maximum is unsigned short as 65,535
CONFIG_NET_PKT_RX_COUNT=650
CONFIG_NET_PKT_TX_COUNT=650
CONFIG_NET_BUF_RX_COUNT=650
CONFIG_NET_BUF_TX_COUNT=650

# IP Address ranges
CONFIG_NET_IF_UNICAST_IPV4_ADDR_COUNT=258
CONFIG_NET_IF_MCAST_IPV4_ADDR_COUNT=0
CONFIG_NET_MAX_CONTEXTS=10

# To support prj.conf based IP setup
# We will be assigning IPs dynamically runtime
# so will not use the configuration based approach
CONFIG_NET_CONFIG_SETTINGS=n
CONFIG_NET_CONFIG_NEED_IPV6=n
CONFIG_NET_CONFIG_NEED_IPV4=n

# IP address options, leave blank so can be assigned runtime
# Primary interface IP and peer IP
CONFIG_NET_CONFIG_MY_IPV4_ADDR=""
CONFIG_NET_CONFIG_PEER_IPV4_ADDR=""

# Network interface statistics logging
CONFIG_NET_STATISTICS=y
CONFIG_NET_STATISTICS_USER_API=n
CONFIG_NET_STATISTICS_PERIODIC_OUTPUT=n
CONFIG_NET_STATISTICS_IPV4=y
CONFIG_NET_STATISTICS_IPV6=n
CONFIG_NET_STATISTICS_IPV6_ND=n
CONFIG_NET_STATISTICS_ICMP=y
CONFIG_NET_STATISTICS_UDP=y
CONFIG_NET_STATISTICS_TCP=y
CONFIG_NET_STATISTICS_MLD=n
CONFIG_NET_STATISTICS_ETHERNET=y
CONFIG_NET_STATISTICS_ETHERNET_VENDOR=n
CONFIG_NET_STATISTICS_LOG_LEVEL_DBG=y
CONFIG_NET_STATISTICS_PER_INTERFACE=y

# Enable promiscuous mode support on sockets for L2 access
# Necessary for Layer 2 Bridge behaviors (ARP and ICMP) on wired interface
CONFIG_NET_PROMISCUOUS_MODE=y

# Enable BSD sockets but don't use POSIX names for clashes with build machine
CONFIG_NET_SOCKETS=y
CONFIG_NET_SOCKETS_POSIX_NAMES=n

# Maximum file descriptors
CONFIG_POSIX_MAX_FDS=6

# Packet raw socket configuration to allow for raw sockets bound to iface
CONFIG_NET_SOCKETS_PACKET=y

# Random number generator
CONFIG_ENTROPY_GENERATOR=y

# Logging
CONFIG_NET_LOG=y
CONFIG_LOG=y
CONFIG_LOG_MODE_DEFERRED=n
CONFIG_LOG_RUNTIME_FILTERING=y
CONFIG_LOG_BUFFER_SIZE=2048
CONFIG_PRINTK=y
#CONFIG_LOG_MODE_OVERFLOW=y
#CONFIG_LOG_PROCESS_THREAD=y
#CONFIG_LOG_PROCESS_TRIGGER_THRESHOLD=10
#CONFIG_LOG_PROCESS_THREAD_STARTUP_DELAY_MS=0
#CONFIG_LOG_PROCESS_THREAD_SLEEP_MS=1000
#CONFIG_LOG_PROCESS_THREAD_STACK_SIZE=4096
#CONFIG_LOG_STRDUP_BUF_COUNT=128
#CONFIG_LOG_STRDUP_MAX_STRING=128
CONFIG_LOG_STRDUP_POOL_PROFILING=y
CONFIG_LOG_PROCESS_TRIGGER_THRESHOLD=0
CONFIG_LOG_STRDUP_BUF_COUNT=64

# - 0 OFF, do not write by default
# - 1 ERROR, default to only write LOG_LEVEL_ERR
# - 2 WARNING, default to write LOG_LEVEL_WRN
# - 3 INFO, default to write LOG_LEVEL_INFO
# - 4 DEBUG, default to write LOG_LEVEL_DBG
CONFIG_LOG_DEFAULT_LEVEL=3

# Prefix log output with function name
CONFIG_LOG_FUNC_NAME_PREFIX_ERR=y
CONFIG_LOG_FUNC_NAME_PREFIX_WRN=y
CONFIG_LOG_FUNC_NAME_PREFIX_INF=y
CONFIG_LOG_FUNC_NAME_PREFIX_DBG=y

# Disable logging using '%s' where strdup()s occur
# Only applies to logging using separate thread
CONFIG_LOG_DETECT_MISSED_STRDUP=n

# Additions to the Shell/Console Interface
# Note Telnet control of UART shell requires
# specifying the UART backend in the Telnet
# command (e.g. "log backend log_backend_uart status")
CONFIG_SHELL_STACK_SIZE=4096
CONFIG_SHELL_CMDS=y
CONFIG_SHELL_TAB_AUTOCOMPLETION=y
CONFIG_SHELL_HISTORY=y
CONFIG_NET_SHELL=y
CONFIG_KERNEL_SHELL=y
CONFIG_NET_STATISTICS=y
CONFIG_DEVICE_SHELL=y

# Shell and logging device backends
CONFIG_SHELL_LOG_BACKEND=n
CONFIG_SHELL_BACKEND_TELNET=y
CONFIG_LOG_BACKEND_UART=y
CONFIG_SHELL_TELNET_INIT_LOG_LEVEL_NONE=y

# Debug Options for debugging
CONFIG_DEBUG=y
CONFIG_DEBUG_OPTIMIZATIONS=y
#CONFIG_LOG_MODE_DEFERRED=n

# Memory buffer leak verification shell extensions
CONFIG_NET_BUF_POOL_USAGE=y
CONFIG_NET_DEBUG_NET_PKT_ALLOC=y

# File System Shell
CONFIG_FILE_SYSTEM=y
CONFIG_FILE_SYSTEM_SHELL=y

# Gather network statistics
CONFIG_NET_STATISTICS_ETHERNET=y

# Name threads
CONFIG_THREAD_NAME=y

# Max number of IPv4 network interfaces in the system
# Will be a value of 1 for real radio
CONFIG_NET_IF_MAX_IPV4_COUNT=2

# How many Tx traffic classes to have for each network device
# 1 = all the network traffic is handled equally.
# 0 = all the network traffic is pushed to the driver directly without any queues
CONFIG_NET_TC_TX_COUNT=1

# How many network connections is supported (default is 4)
CONFIG_NET_MAX_CONN=8

# Total number of sockets to poll in select()
CONFIG_NET_SOCKETS_POLL_MAX=5

Are you familiar with the promiscuous mode sample app at samples/net/promiscuous_mode, it shows how to use the promiscuous mode API. In short, if you enable promiscuous mode by calling  net_promisc_mode_on(), then you need to have a handler that will need to unref the received packet, otherwise there is a leak.

I understand, but this then implies that using standard POSIX APIs at the network layer is incompatible with promiscuous mode for the network interface, that is Zephyr is not compatible with that standard.  Is there any way that the underlying collection pointer to be freed could be exposed to say the zsock_recv() API so that the application could free it?, or would it just make sense that Zephyr perform the free itself as a result of that call?, meaning as soon as the content is copied to the user space buffer, there is no reason to keep the allocation around anymore.  I think this approach would be the most elegant, keeping Zephyr compatible to the POSIX standard and freeing the developer adhering to POSIX standard APIs out of the business of knowing deallocations.  Using the POSIX APIs provides efficiencies to folks porting existing applications, and adding promiscuous support would make Zephyr compatible with the Linux APIs.

I think we've discussed this already in https://github.com/zephyrproject-rtos/zephyr/issues/36748.
TLDR the conclusion was that the module enabled with CONFIG_NET_PROMISCUOUS_MODE is a standalone API, not integrated with sockets, therefore I think it's safe to say that Zephyr does not support promiscuous RAW sockets. Threfore I think the issue should rather be cosidered as an enhancement, not a bug.

Yes, the discussion from the past is the same one, over time I have found that I really needed that raw socket, which is not uncommon for any Layer 2 networking device, and as the Zephyr code base as it has evolved since my original posting, changes/updates have not addressed that issue.
Is the level of effort described to have the zsock_recv API simply free up the local allocation a complex issue to implement?
I really don't know the scope of what I am asking, and you guys being the subject matter experts, know the full scope of the design

You could have a somewhat hackish workaround by having a "dummy" promiscuous mode handler that just discards the received packets, and then your raw socket should also receive all the same packets. This is not optimal of course but should work until we get support for proper promiscuous mode sockets.

Yeah, I was thinking a thread that would set the interface to promiscuous mode and have a while loop like you have above.
After some code digging there seems to be a simpler way. The promiscuous packets are processed in https://github.com/zephyrproject-rtos/zephyr/blob/bb8575962da291e3b75dfb3abe2fcb9341f76be2/subsys/net/ip/net_if.c#L3900 and if the interface is set to promiscuous mode but there is no handler i.e., the net_promisc_mode_on() is not called, then the packet should be unreffed properly and no leaks should happen.
So if you want to receive packets in promiscuous mode, but do not want to use the promiscuous mode API, you can enable promiscuous mode in config file, and then call net_if_set_promisc(iface); to set the interface to promiscuous mode. You should then be able to receive all the network traffic in your device without any buffer leaks and without any dummy handlers etc. Not tested but should work ok.

Yes, the discussion from the past is the same one, over time I have found that I really needed that raw socket, which is not uncommon for any Layer 2 networking device, and as the Zephyr code base as it has evolved since my original posting, changes/updates have not addressed that issue.
Is the level of effort described to have the zsock_recv API simply free up the local allocation a complex issue to implement?
I really don't know the scope of what I am asking, and you guys being the subject matter experts, know the full scope of the design

Given that you are really asking for a feature request here, I will convert this to a discussion and then ask you to open  a new issue (marked as enhancement), along the lines of: "Implement promiscuous mode for AF_PACKET sockets"


Update Command.php

Refactor constructor if else

:-1: This doesn't need changing. It's already more readable in it's current form.

:-1: I agree with @GrahamCampbell


Link fields in administration

Description
Currently it's only possible to have a full URL in the administration on CMS elements.
No matter if you choose the link-to property of an image or create an inline link in the content.
There should be a way to select a category as link destination so that SEO urls are used properly (in case somebody changes them).
Example

i don't get why this isn't getting more attention. See https://github.com/shopware/platform/issues/304#issuecomment-557523170 for a "developer"-friendly solution. But for content managers this is really a problem.

The same is an issue for products etc.
It would already help if one could do something like this:

product://
etc.

Of course, even better would be to provide an option to link to records directly using the admin interface / search.

It should also be possible to link to symfony routes nicely. Doesn't need to be all but some like cart, login etc. would be great.

It would also be good to support this for other non RTE fields like e.g. the external URL field in the category admin as maybe I want to link directly to the user registration or a product from menu entries - especially as this might be usable in other custom entities or CMS blocks then.


Update ERC-20 (CEP-18) js client for enhanced NFTs (CEP-78)

As we rework the ERC-20 contract to be compatible with CEP-78 standards, there will be updates necessary to the js client to be released.  This epic captures the necessary work to update and republish the js client.

new ticket opened closed as a duplicate


Form's state is null by default

I'm trying to jump from 0.1.3 to 0.2.4 and found a pitfall which I couldn't get rid of.
The form is rendered within bootstrap popup.

I'm using reselect for lazy evaluations and in this particular case to define whether user can add widget or not.
import { createSelector } from 'reselect';
const widgetNameSelector = state => state.addWidgetForm.data.name;
const widgetTypeSelector = state => state.addWidgetForm.data.type;
const canAddSelector = createSelector(
  [widgetNameSelector, widgetTypeSelector],
  (name, type) => !!name && !!type  // both fields should not be empty
);

@connect(
  (state) => {
    return {
      form: state.addWidgetForm
      canAdd: canAddSelector(state)
    };
  }
)
class AddWidgetForm extends Component { ... }

addWidgetForm  wasn't equal null at start in<EMAIL_ADDRESS>so I could rely on the structure in reselect state.addWidgetForm.data.name.
In<EMAIL_ADDRESS>it equals null.

Of course I have error in this function:
const widgetNameSelector = state => state.addWidgetForm.data.name;

Any suggestions? Is it correct behaviour?

First of all, brilliantly presented issue. Thank you.
Yes, that changed just now in v0.2.4, that the state objects are lazily initiated on the first action received. This is what allows the reducer to figure out if it is a multi-reducer or not. My ridiculous destructuring on my example was masking this issue.
I see two solutions:

Force people to do null checks.
Force people wanting to use multi-reducers to tell the reducer that in createFormReducer().

I prefer the latter. Thoughts? @janmarek?

In this specific case I'd go with null check, but in general I'd prefer more intuitive solution. Is there any chance to implement optional lazy initialization only for multiple records?

v0.3.0 provides a solution to this. Or, rather, defers a solution to this to another library. See Calculating props from Form Data.

Thanks, awesome work!


ingest/ledgerbackend: Debug slow ingestion during catchup in captive-core

What version are you using?
1.10.0
What did you do?
Start Horizon + Captive Core from empty DB. Wait for state ingestion. Then it should catchup from checkpoint ledger up to the latest ledger.
What did you expect to see?
Ingestion is fast.
What did you see instead?
Every 4-5 ledgers ingestion is slower even when number of ops/changes is comparable to previous ledgers. See my catchup from earlier today:
time="2020-10-14T21:14:48.059Z" level=info msg="Processed ledger" commit=true duration=0.0413032 ledger=true pid=1 sequence=1228494 service=expingest state=true stats_accounts_created=0 stats_accounts_removed=0 stats_accounts_updated=2 stats_data_created=0 stats_data_removed=0 stats_data_updated=0 stats_offers_created=0 stats_offers_removed=0 stats_offers_updated=0 stats_operations=1 stats_operations_account_merge=0 stats_operations_allow_trust=0 stats_operations_begin_sponsoring_future_reserves=0 stats_operations_bump_sequence=0 stats_operations_change_trust=0 stats_operations_claim_claimable_balance=0 stats_operations_create_account=0 stats_operations_create_claimable_balance=0 stats_operations_create_passive_sell_offer=0 stats_operations_end_sponsoring_future_reserves=0 stats_operations_in_failed=0 stats_operations_in_successful=1 stats_operations_inflation=0 stats_operations_manage_buy_offer=0 stats_operations_manage_data=0 stats_operations_manage_sell_offer=0 stats_operations_path_payment_strict_receive=0 stats_operations_path_payment_strict_send=0 stats_operations_payment=1 stats_operations_revoke_sponsorship=0 stats_operations_set_options=0 stats_transactions=1 stats_transactions_failed=0 stats_transactions_successful=1 stats_trust_lines_created=0 stats_trust_lines_removed=0 stats_trust_lines_updated=1
time="2020-10-14T21:14:48.060Z" level=info msg="Ingestion system state machine transition" current_state="resume(latestSuccessfullyProcessedLedger=1228493)" next_state="resume(latestSuccessfullyProcessedLedger=1228494)" pid=1 service=expingest
GCH27 [Herder INFO] Quorum information for 1228565 : {"agree":3,"delayed":0,"disagree":0,"fail_at":2,"hash":"59d361","lag_ms":13,"ledger":1228565,"missing":0,"phase":"EXTERNALIZE"}

time="2020-10-14T21:14:48.083Z" level=info msg="Processing ledger" commit=true ledger=true pid=1 sequence=1228495 service=expingest state=true
GCH27 [Ledger INFO] Got consensus: [seq=1228567, prev=f58820, txs=8, ops=8, sv: [ SIGNED@sdf_testnet_3 txH: 8ef9b8, ct:<PHONE_NUMBER>, upgrades: [ ] ]]
GCH27 [Ledger INFO] Close of ledger 1228567 buffered
time="2020-10-14T21:14:48.089Z" level=info msg="Trimmed offers table" offer_rows_removed=0 pid=1 service=expingest
GCH27 [Tx INFO] applying ledger 1228499 (txs:2, ops:2)
GCH27 [Tx INFO] applying ledger 1228500 (txs:5, ops:84)
time="2020-10-14T21:14:49.118Z" level=warning msg="error updating stellar-core cursor" error="Setting stellar-core cursor failed: http request errored: Get \"http://localhost:11626/setcursor?cursor=1228495&id=HORIZON\": context deadline exceeded" pid=1 service=expingest
time="2020-10-14T21:14:49.119Z" level=info msg="Processed ledger" commit=true duration=1.0357502 ledger=true pid=1 sequence=1228495 service=expingest state=true stats_accounts_created=4 stats_accounts_removed=0 stats_accounts_updated=14 stats_data_created=0 stats_data_removed=0 stats_data_updated=0 stats_offers_created=0 stats_offers_removed=0 stats_offers_updated=0 stats_operations=5 stats_operations_account_merge=0 stats_operations_allow_trust=0 stats_operations_begin_sponsoring_future_reserves=0 stats_operations_bump_sequence=0 stats_operations_change_trust=0 stats_operations_claim_claimable_balance=0 stats_operations_create_account=4 stats_operations_create_claimable_balance=0 stats_operations_create_passive_sell_offer=0 stats_operations_end_sponsoring_future_reserves=0 stats_operations_in_failed=0 stats_operations_in_successful=5 stats_operations_inflation=0 stats_operations_manage_buy_offer=0 stats_operations_manage_data=0 stats_operations_manage_sell_offer=0 stats_operations_path_payment_strict_receive=0 stats_operations_path_payment_strict_send=0 stats_operations_payment=1 stats_operations_revoke_sponsorship=0 stats_operations_set_options=0 stats_transactions=5 stats_transactions_failed=0 stats_transactions_successful=5 stats_trust_lines_created=0 stats_trust_lines_removed=0 stats_trust_lines_updated=1

Ledger 1228494 was ingested in 0.04 but 1228495 with almost the same number of ops/changes in more than 1s!
I think it can be related to buffering of ledgers loaded in captive core backend which I was thinking had been fixed in https://github.com/stellar/go/pull/2763 but apparently it still can be super slow.

I wrote a small script based on captive-core-start-tester to help me debug this. I think it comes down to the read-ahead buffer size (now just 2 ledgers). There are two situations in which this issue can occur:

If ingestion is faster than Stellar-Core applies ledgers then Horizon waits for next available ledger for 1s because GetLedger is not blocking in online mode (instead of blocking it just returns false in first return argument). Horizon could ingest more but there are only 2 in a buffer.
If Stellar-Core is faster it can't apply ledgers because ingestion package doesn't read more than 2.

I think we should increase the buffer size but it's not trivial because there are two constraints that must be met:

The buffer shouldn't exceed X of RAM but...
It should fill at least one ledger no matter how big it is (even if exceeds allow memory constraint).

This is because some ledgers can exceed allow size and having a memory constraint only would result in a deadlock.
I'll try to code a solution for this next week.


the kv-sidecar pod remains in CrashLoopBackOff.

Hi @sethvargo @gdwikir
Thanks for the amazing workshop. I am almost done with setting up everything, and everything seemed to be working fine until I came across this error. Not so sure whether anyone else is facing this issue or not.
Here are the logs
$ kubectl logs -l app=kv-sidecar -c vault-authenticator

2019/09/25 07:36:48 failed to get successful response: &http.Response{Status:"500 Internal Server Error", StatusCode:500, Proto:"HTTP/2.0", ProtoMajor:2, ProtoMinor:0, Header:http.Header{"Cache-Control":[]string{"
no-store"}, "Content-Length":[]string{"52"}, "Content-Type":[]string{"application/json"}, "Date":[]string{"Wed, 25 Sep 2019 07:36:48 GMT"}}, Body:http2.transportResponseBody{cs:(*http2.clientStream)(0xc00009ea00)}
, ContentLength:52, TransferEncoding:[]string(nil), Close:false, Uncompressed:false, Trailer:http.Header(nil), Request:(*http.Request)(0xc0000d6000), TLS:(*tls.ConnectionState)(0xc000082580)}, {"errors":["could no
t load backend configuration"]}

Image is attached below:

I even tried changing the vault-authenticator image version to latest, still the error seems to persist.

And because of this failed container, consul-template remains in initialising stage(as it is waiting for vault-authenticator to come up.)

This error corresponds to when there's no configuration for the Kubernetes auth method. Can you share more about what steps you took and where you got stuck?

Hey. was able to sort that out, there was some problem with rbac permission, so now stuck to this. It says that the certificate is signed by unknown authority. I am using ca.crt .. That said, I have configmap and secrets configured properly already. This certificate is self-signed, hence it is showing as invalid. I have configured as shown in this slide.
kubectl get secret vault-tls

kubectl get configmap vault

https://codelabs.developers.google.com/codelabs/vault-on-gke/index.html?index=..%2F..cloud#8
kubectl logs kv-sidecar-5459db6969-228s2 vault-authenticator
019/09/25 14:08:25 failed to get successful response: &http.Response{Status:"500 Internal Server Error", StatusCode:500, Proto:"HTTP/2.0", ProtoMajor:2, ProtoMinor:0, {"errors":["Post h
ttps://<K8s_host>/apis/authentication.k8s.io/v1/tokenreviews: x509: certificate signed by unknown authority"]}


Useless serialization into immediate deserialization

https://github.com/excalidraw/excalidraw/blob/bc8ba08ad0369fdf13e138e094677707a99622c9/src/scene/zoom.ts#L27
This line above ultimately chops off everything after the 2nd digit after the decimal place. If this behavior is truly intended, one could use Math.floor(zoom* 100) / 100 instead.
Also why is this operation called "normalization" (as the normalizedZoom variable name suggests)?

When we're at it, I'd actually use parseInt() and get rid of the precision completely.
As for normalizedZoom naming, true that the first variable isn't the normalized output yet, so either none of those 2 variables should be called normalized, or the second one. Ultimately, not important.

Feel free to take this on.


#140 disallow arbitrary attributes on Bundle-types, Concat

@ThomasPluck - please check out the changes to Scalar here:
https://github.com/dan-fritchman/Hdl21/blob/e060483c1289c64a4db3a77af97f48fc2a322710/hdl21/scalar.py
This is how I would prefer that Scalar type/ "thing" to work - more like Union[Prefixed, Literal] than "wrapper thereof".
It means that all the call sites throughout PDKs like this:
@h.paramclass
class Params:
  xyx = h.Param(dtype=h.Scalar, desc="", default=h.Scalar(inner=1 * µ))

Need to become:
@h.paramclass
class Params:
  xyx = h.Param(dtype=h.Scalar, desc="", default=1 * µ)

(Note the latter version always worked; now just have to use it; Scalar is not a constructible thing.)
I know you've got PDK changes brewing. This should just effect those call-sites - presumably all of which are parameter-defaults - and nothing else. (I think.)
You ready for this to go in?
Or wanna get PDK changes merged first?

Sure, this can go in and I'll refactor Scalar calls.

Error discovered in a couple tests having scrubbed out h.Scalar calls:
        if not isinstance(orig, h.Scalar):
>           raise TypeError(f"Invalid Scalar parameter {orig}")
E           TypeError: Invalid Scalar parameter number=Decimal('0.42') prefix=<Prefix.MICRO: -6>

Is this something that you'd like to build an exception for, or would you prefer to keep this type definition strict?

Yep, I noticed the same, example here:
https://github.com/dan-fritchman/Hdl21/blob/1cf539ed660c7c4ce5ae58cd169004206d72a497/pdks/Sky130/sky130/pdk_logic.py#L369
I think that's the right approach, and consistent with the idea that `Scalar "is" Union[Prefixed, Literal]".


made changes to supported the jartobundle script for all JDK versions

Purpose

Currently the jartobundle script is only supported with JDK 8. Made this script changes to support it with all compatible JDK versions with the WSO2 product.

Approach

Removed the JDK version checking condition.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


specify a more clear hint for Dockerfile

the example dockerfile will not work for (default) docker desktop.
also, a default latest tag does not exist.
for 2019 one may use FROM mcr.microsoft.com/windows/servercore:1903-KB4505057

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 55e2558f-e434-e5cc-acd4-0d3bd2dd979f
Version Independent ID: be6bfe5f-bfdd-6550-588d-6d627c16c3f8
Content: Use Docker to run PowerShell in Azure Stack
Content Source: azure-stack/user/azure-stack-powershell-user-docker.md
Service: azure-stack
GitHub Login: @mattbriggs
Microsoft Alias: mabrigg

Hi @bottkars Thank you for your feedback! We will review and provide an update as appropriate.

@bottkars Thanks for reporting. Dockerfiles base image has to be changed.
Better to change that to mcr.microsoft.com/windows/servercore:ltsc2019
I will assign this issue to the author to investigate and update the document.
CC: @TravisCragg-MSFT

@bottkars, @jakaruna-MSFT
Thanks for providing feedback about this. It looks like latest was deprecated, and it looks like there have been some updates that may also cause some issues. Here is a thread that discusses it.
https://github.com/docker/for-win/issues/3761
I will update shortly when I work out the kinks. #in-progress

Yes, we are facing some issues as well. When pullend from docker, we have eben able to Extract Digest, Headers etc from Go. Since pulling the new ones from mcr, the Go-libraries Face Various issues...

@bottkars Thanks for the note. Pulling the image using the tag ltsc2019 I am able to use the dockerfile. Iv'e updated the dockerfile to mcr.microsoft.com/windows/servercore:ltsc2019. I am running this is on Windows 10 Enterprise 1809. I will close this thread for now, but please me know if you have additional feedback or questions. Thanks again! Matt

#please-close


Add WithError(), and name to log level conversion

Add a WithError() shortcut for WithField("error", err) so it's easy to log errors parametrically
Add some name-to-log-level conversion to make it easer to parse these directly from service configs or env vars instead of needing to know the log level number.

I've kept the existing WithLevel() function signature the same for backwards compatibility.

Seems ok to be, but should we also not add a WithInfo() and a WithWarn() for consistency and expectation pattern?

Seems ok to be, but should we also not add a WithInfo() and a WithWarn() for consistency and expectation pattern?

I guess the idea here is to add a field "error" of value of type Error.
WithInfo() or WithWarn() would not really make sense, as there is no type Info or Warning

Seems ok to be, but should we also not add a WithInfo() and a WithWarn() for consistency and expectation pattern?

I guess the idea here is to add a field "error" with a value of the built-in interface type Error to the logs. WithInfo() or WithWarn() would not really make sense, as there is no type Info or Warning

Indeed. @AtzeDeVries the intended usage is logrus-like, so you would do something like:
err := someOperationThatFailed()
if err != nil {
  logger.WithError(err).Fatal("That didn't go well")
}

Logs in other levels can just be written with logger.Info("Starting up")


[cmake]: Second try at making swift-api-digester work correctly.

Running ninja check-swift didn't rebuild swift-api-digester.
Previous: https://github.com/apple/swift/pull/5505
and: https://github.com/apple/swift/pull/5509
rdar://problem/28953348

@swift-ci Please smoke test

The problem is that swift-api-digester is only built on Apple platforms.
(I mean, maybe we want to change that—cc @nkcsgexi—but for now I don't think we can just turn it on and expect things to work.)

Can we exclusively build swift-api-digester on Apple platforms for now?

@swift-ci Please smoke test

@swift-ci Please smoke test

swift-api-digester now builds on linux but the smoke test failed for other reasons. Try again..
@swift-ci please smoke test

Cleaned up the OSX special casing for swift-api-digester.

@swift-ci Please smoke test


health_checker/redis_proxy/thrift_proxy: remove deprecated v2 config handling

Risk level: Low
Testing: Removed/updated and verified coverage of deprecated tests.
Signed-off-by: Harvey Tuch<EMAIL_ADDRESS>

CC @tyxia

/retest

I think I was too hasty with the v2 link removal for Windows..


workbench.editor.closeOnFileDelete

I very recommend to turn the setting workbench.editor.closeOnFileDelete to false by default. This is a very appreciated feature of other editors like sublime-text that prevent (careless) developers loosing their work that was accidentally removed and to my surprise vscode automatically closed the editors by default.
I always have to manually put it in false on each of my installations. I wonder what are the reasons to have it on "true" but I really really think developers (that do lots of rm -rf manually) really expect this setting to be in false.
Sorry for the opinionated - not-an-issue - feedback but vscode is IMO the best and I think this little detail is playing against it. Thanks and keep doing this great work!

VSCode Version: Code 1.18.1 (929bacba01ef658b873545e26034d1a8067445e9, 2017-11-16T18:34:22.110Z)
OS Version: Linux x64 4.14.0-0.1-MANJARO
Extensions:

Extension
Author (truncated)
Version

vscode-docker
Pet
0.0.22

spellright
ban
2.0.6

Thanks for the suggestion, but I like to not have deleted files stay around when deleted. We do this to clean up text buffers automatically and it helps free resources and helps language servers to do less work.

Thanks for the answer. The common scenario is user edited few files and accidentally rm -rf *  so in general there won't be much memory used - just edited/changed ones.
It's wrong that, by default, the editor makes a decision about deleting data that I explicitly decide to edit without even warn me about it I'm the owner of those users, not the editor!!!! Remember that not saved changes are not files - are memory buffers - and the editor is deciding to remove these!!!! It's almost immoral...
Know that this decision will make users loose their changes and probably hate vscode. As an user of other similar product I expect the editor to keep my buffers or at least ask me before delete them.
I don't know any other editor that does this, and the reason is sincerely unacceptable (again, take this as an user feedback that consider this very important for this EXCELLENT editor)
I'm sincerely surprised that the reason for this decision is performance. As a user I rather prefer my system slow/heavier than loosing work that I'm explicitly editing. The performance gain you intend to gain by implementing this "intelligent" behavior could be still archived by 1) ignore these buffers in language servers 2) back up buffers to disk and show just a fragment visually - or don't show an editor at all and if users focus the tab give them possibility to restore the data..


InvalidMessageException after v62.5.3 (PR 537)

Describe the bug
Using v62.5.3 or later causes InvalidMessageException when client connects. Tested with 2019.4.18f1 LTS, using Basic sample scene.

v62.5.2 works
v62.5.3 doesn't (tested with v62.9.1 as well)

To Reproduce

In a blank project, follow standard installation: https://github.com/MirrorNG/MirrorNG#installation
Add sample: Basic
Follow Basic setup steps (build example scene)
Play standalone build & editor, using one as host and the other as client (doesn't matter which).
Observe InvalidMessageException on client.

Desktop (please complete the following information):

OS: Windows 10
Build target: standalone
Unity version: 2019.4.18f1 LTS
Mirror branch: MirrorNG

Exception
: Closed connection: connection([::1]:7777). Invalid message Mirror.InvalidMessageException: OnDeserialize failed for: object=Player(Clone) component=Mirror.Examples.Basic.Player sceneId=0. Possible Reasons:
  * Do Mirror.Examples.Basic.Player's OnSerialize and OnDeserialize calls write the same amount of data?
  * Was there an exception in Mirror.Examples.Basic.Player's OnSerialize/OnDeserialize code?
  * Are the server and client the exact same project?
  * Maybe this OnDeserialize call was meant for another GameObject? The sceneIds can easily get out of sync if the Hierarchy was modified only in the client OR the server. Try rebuilding both.

  at Mirror.NetworkIdentity.OnDeserializeSafely (Mirror.NetworkBehaviour comp, Mirror.NetworkReader reader, System.Boolean initialState) [0x00021] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkIdentity.cs:898
  at Mirror.NetworkIdentity.OnDeserializeAllSafely (Mirror.NetworkReader reader, System.Boolean initialState) [0x0002a] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkIdentity.cs:915
  at Mirror.ClientObjectManager.ApplySpawnPayload (Mirror.NetworkIdentity identity, Mirror.SpawnMessage msg) [0x000db] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\ClientObjectManager.cs:339
  at Mirror.ClientObjectManager.OnSpawn (Mirror.SpawnMessage msg) [0x00118] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\ClientObjectManager.cs:374
  at Mirror.NetworkConnection+<>c__DisplayClass24_0`1[T].<RegisterHandler>b__0 (Mirror.INetworkConnection _, T value) [0x00001] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkConnection.cs:152
  at Mirror.NetworkConnection+<>c__DisplayClass22_0`1[T].<MessageHandler>g__AdapterFunction|0 (Mirror.INetworkConnection conn, Mirror.NetworkReader reader, System.Int32 channelId) [0x00025] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkConnection.cs:121
  at Mirror.NetworkConnection.InvokeHandler (System.Int32 msgType, Mirror.NetworkReader reader, System.Int32 channelId) [0x00014] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkConnection.cs:259
  at Mirror.NetworkConnection.TransportReceive (System.ArraySegment`1[T] buffer, System.Int32 channelId) [0x00011] in A:\unity_tests\net_test_2019_4_LTS\Library\PackageCache\com.mirrorng.mirrorng@62.0.0\Runtime\NetworkConnection.cs:295
UnityEngine.Logger:LogError(String, Object)
Mirror.ILoggerExtensions:LogError(ILogger, Object) (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/LogFactory.cs:43)
Mirror.NetworkConnection:TransportReceive(ArraySegment`1, Int32) (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/NetworkConnection.cs:303)
Mirror.<ProcessMessagesAsync>d__40:MoveNext() (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/NetworkConnection.cs:352)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`1:Run() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/CompilerServices/StateMachineRunner.cs:189)
Cysharp.Threading.Tasks.AwaiterActions:Continuation(Object) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1:TrySetResult(Int32) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTaskCompletionSource.cs:139)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2:SetResult(Int32) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/CompilerServices/StateMachineRunner.cs:328)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskMethodBuilder`1:SetResult(Int32) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/CompilerServices/AsyncUniTaskMethodBuilder.cs:201)
Mirror.KCP.<ReceiveAsync>d__35:MoveNext() (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/Transport/Kcp/KcpConnection.cs:248)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTask`2:Run() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/CompilerServices/StateMachineRunner.cs:313)
Cysharp.Threading.Tasks.AwaiterActions:Continuation(Object) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1:TrySetResult(AsyncUnit) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTaskCompletionSource.cs:139)
Cysharp.Threading.Tasks.AutoResetUniTaskCompletionSource:TrySetResult() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTaskCompletionSource.cs:386)
Mirror.KCP.KcpConnection:InputReliable(Byte[], Int32) (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/Transport/Kcp/KcpConnection.cs:161)
Mirror.KCP.KcpConnection:RawInput(Byte[], Int32) (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/Transport/Kcp/KcpConnection.cs:135)
Mirror.KCP.<ReceiveLoop>d__7:MoveNext() (at Library/PackageCache/com.mirrorng.mirrorng@62.0.0/Runtime/Transport/Kcp/KcpClientConnection.cs:51)
Cysharp.Threading.Tasks.CompilerServices.AsyncUniTaskVoid`1:Run() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/CompilerServices/StateMachineRunner.cs:104)
Cysharp.Threading.Tasks.AwaiterActions:Continuation(Object) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTask.cs:21)
Cysharp.Threading.Tasks.UniTaskCompletionSourceCore`1:TrySetResult(Object) (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTaskCompletionSource.cs:139)
Cysharp.Threading.Tasks.DelayPromise:MoveNext() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/UniTask.Delay.cs:520)
Cysharp.Threading.Tasks.Internal.PlayerLoopRunner:RunCore() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/Internal/PlayerLoopRunner.cs:175)
Cysharp.Threading.Tasks.Internal.PlayerLoopRunner:Update() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/Internal/PlayerLoopRunner.cs:145)
Cysharp.Threading.Tasks.Internal.PlayerLoopRunner:Run() (at Library/PackageCache/com.cysharp.unitask@7d21a75ea8/Runtime/Internal/PlayerLoopRunner.cs:104)

Might be connected to: https://github.com/MirrorNG/MirrorNG/issues/552 (an issue with PR 537)

Might be connected to: https://github.com/MirrorNG/MirrorNG/issues/552 (an issue with PR 537)

Ok,  I reverted #537  in 12a1d73061766d3c8d1bfb406a20ba4f44d18d50
can you check if this is still happening?

Ok,  I reverted #537  in 12a1d73061766d3c8d1bfb406a20ba4f44d18d50
can you check if this is still happening?

Tested with most recent version (63.3.2) and it's working again, Thank you!


Refactor selendroid to be able to run tests in a remote host

Attaching asm suggestions!

cc @lukeis @DominikDary

I'm confused about the use case for this. Why wouldn't that remote host be running the selendroid server? How would one use this?

We're reconsidering this approach. In our case the remote host doesn't run the selendroid standalone, but we hook up already running emulators to selendroid standalone.


Cannot type accented vowels on the integrated terminal

VSCode Version: 1.34.0
OS Version: GNU/Linux (Manjaro with kernel 4.19 LTS)

I cannot type accented vowels in the integrated terminal. That's the info I can give to you:

If I type «á» (dead key «´» + «a» key) it will transform into «a».

It doesn't matters if I use the canvas terminal o the dom one.
It isn't a GUI or font issue, because the integrate terminal can show accented vowels (for example, from a file using cat). It's an input issue.
I'm using a spanish keyboard.
This issue happens on all my different machines.
It happens on insider build too.

Steps to Reproduce:

Open visual studio code.
Open the integrate terminal.
Press dead key «´» and then a vowel key.

Does this issue occur when all extensions are disabled?: Yes

/duplicate https://github.com/microsoft/vscode/issues/68262

@Tyriar    I do not think the description matches #6282 exactly. That one is only about the ~ sign with the Alt-Gr and only on Windows. Not the other Alt-Gr characters in the Spanish keyboard, not the accented characters. Everything except the ~ works fine (I guess the same in the Freanch/Belgian because nobody reported other behabiour)
On Linux, with my Spanish keyboard everything works perfectly.
My setup on Linux, Spanish keyboard, only ~ fails:
Versión: 1.34.0 (system setup)
Commit: a622c65b2c713c890fcf4fbf07cf34049d5fe758
Fecha: 2019-05-15T21:59:37.030Z
Electron: 3.1.8
Chrome: 66.0.3359.181
Node.js: 10.2.0
V8: 6.6.346.32
OS: Windows_NT x64 10.0.17763
My setup on Linux, Spanish keyboard, everything works right:
Version: 1.34.0
Commit: a622c65b2c713c890fcf4fbf07cf34049d5fe758
Date: 2019-05-15T21:55:35.507Z
Electron: 3.1.8
Chrome: 66.0.3359.181
Node.js: 10.2.0
V8: 6.6.346.32
OS: Linux x64 4.18.20-041820-lowlatency

@bcxpro I think they all have the same root cause


(Do not merge) Test native image build

com.google.cloud.spanner.connection.AbstractStatementParser accessed. Registering. found in native-image build logs.


rename app selector to 'my-app'

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

feature

What is the current behavior? (You can also link to an open issue here)

AppComponent selector is app

What is the new behavior (if this is a feature change)?

AppComponent selector is my-app

Other information:

Uses a hyphenated, lowercase element selector value. Follows Angular styleguide. Is a valid custom element name. Solves HTML validation error: "Element app not allowed as child of element div in this context. (Suppressing further errors from this subtree.)".

Sorry, but I close this PR. This depends upon taste, and the settings of your tslint.


update for backwards compatibility to IE

polyfill must be added to your html
<script src="https://polyfill.io/v3/polyfill.min.js?features=Array.prototype.values" type="text/javascript"></script>
please let me know if you see anything you would like changed.
Thanks

Maybe you want consider issue #105? polyfill solves that?

@luca-vercelli Hi yes, the changes to remove the tick marks along with this polyfill script tag solves the IE issues, I haven't found a work around for the tick marks in JS other than removing them. This should resolve issue #105


Update pip dir

🦙 MegaLinter status: ✅ SUCCESS

Descriptor
Linter
Files
Fixed
Errors
Elapsed time

✅ EDITORCONFIG
editorconfig-checker
1

0
0.03s

✅ REPOSITORY
checkov
yes

no
12.77s

✅ REPOSITORY
gitleaks
yes

no
0.15s

✅ REPOSITORY
git_diff
yes

no
0.0s

✅ REPOSITORY
secretlint
yes

no
1.36s

✅ REPOSITORY
trivy
yes

no
6.89s

✅ YAML
prettier
1
0
0
0.59s

✅ YAML
v8r
1

0
1.88s

✅ YAML
yamllint
1

0
0.25s

See detailed report in MegaLinter reports
Set VALIDATE_ALL_CODEBASE: true in mega-linter.yml to validate all sources, not only the diff
MegaLinter is graciously provided by


Could you please place a sample implementation for proper msal silent and popup authentication for usage in a MS Teams app web tab (not the standard web url tab)

When trying to create a valid aad login method for a pure html/js/css web app, we found that the vanillajs examples from here do not work for teams web tabs added via "real" teams apps (by creating and uploading a manifest or by using the teams appstudio).
Could you please provide a working example for a MSAL silent authentication with redirection after successful login to an apps web URL? If silent fails, could you ask for a login via a popup? Can the token be placed in localstorage so that one authentication will be enough for multiple teams tabs?
I have tried to achieve this result by trying many available documentation pages - no luck so far. A simple and working example would be of great help to enter the space of teams app bulding...

It seems like the login form is denied to appear inside of an iframe when using a teams app web tab while being allowed when using a teams website tab. So IF a login form is needed, it has to appear in a popup.

I changed console.log() to alert().

All examples for Teams use adal instead of msal
https://docs.microsoft.com/en-us/microsoftteams/platform/tabs/how-to/authentication/auth-silent-aad
https://docs.microsoft.com/en-us/microsoftteams/platform/tabs/how-to/authentication/auth-tab-aad#samples

Yes, but only on the outdated documentation, as said in:

https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-migration
https://github.com/AzureAD/azure-activedirectory-library-for-js

Maybe the following links can help you, I recommend only to view github code of MSAL.js and github doc, but not any docs on microsoft.com.

https://github.com/AzureAD/microsoft-authentication-library-for-js/tree/dev/samples
https://github.com/Azure-Samples/ms-identity-javascript-v2
https://github.com/OfficeDev/msteams-tabs-sso-sample-nodejs
https://github.com/wictorwilen/teams-sso-tab-demo/blob/master/src/app/api/graphRouter.ts

I'm trying to authenticate in my node.js restify server a MS teams dialog (TaskInfo), to query user names from the MS graph API, but I fail to do so.

Hi @BerndGewehr ! Thanks for the feedback. We agree that we should have samples for Microsoft Teams. In the mean time, please take a look at this sample that was put together by a Microsoft employee.

Thank You. None of these contain a sample for a MSAL using Teams app Web tab implementation. That's why I asked for it. Any recommendation welcome!￼￼

Need a silent authentication with pop up fall back and token persistence in local storage on success￼

@technical-boy this example relies on on disk files that are not part of the repository. Sorry to say but it did not help me in that state of completeness.

@BerndGewehr it seems the sample was configured to use a locally installed copy of the "msal" package. Have you tried deleting the package-lock.json file and running npm install? I believe this would help if your intention is to test the sample with the published version of<EMAIL_ADDRESS>

@technical-boy I tried the example - and failed. It somehow does not manage to show a real login procedure and fails on trying what looks like a second attempt for a silent login:

@technical-boy I found that in a web tab of teams the code works fine. In an app web tab (created from a manifest.json), the redirect fails without further notice.

Possibly similar problem here:
https://stackoverflow.com/questions/62461493/msal-login-redirect-window-inside-microsoft-toolkit-wpf-ui-controls-webview-is-l

I found that the referenced version from the sample (1.2.2) does not work successfully while the current released version (1.1.3) works much better.

@BerndGewehr glad to hear 1.3.4 works better, has your issue been solved then?

Well, partly.
I achieved a working solution for the following flow:

build a teams app web tab (manifest.json, tab, Config.html) from appstudio
use your web app in the app tab
try to get a AAD token silently
if no luck, get a token in a Popup silently (have not yet completely understood why this is helpful)
if no luck even there, let the user sign in interactively
use the accessToken for graph API access etc.

What I did not yet understand is how this all could help me to use a AAD SAML enabled enterprise app silently in a teams app web tab. This was my initial problem why I started to go deeper into all this. I always hoped that a teams user, logged on with his AAD user would never have to log on again in teams tabs, be it for graph API apps or for SAML federated apps with Azure AD. But this is not the case. Users have to login again in those tabs while they experience seamless SSO in a web URL tab. To use the advantages of the app web tab (opens inside mobile app instead of breaking out to browser, integration with a bot, whitelisting of domains involved, icons, preset names, etc.), I wanted to use those better than url web tabs.
Any suggestions for this?

A similar issue is found on our end, oddly enough the pop-up login prompts the standard browser on the machine to open, basically hijacking the entire experience to the browser.
The Desktop app so far doesn't appear to be able to deal with the pop-up login.

@akleimrey which version of MSAL do you use? Had similar experiences with the wrong version, with 1.3.4 it worked as designed.

@akleimrey which version of MSAL do you use? Had similar experiences with the wrong version, with 1.3.4 it worked as designed.

I'm using the Angular Wrapper, but I checked my dependencies, and we had the version of 1.3.3 of MSAL apparently. I installed the 1.3.4 version of MSAL but sadly, same issue.

Can anybody tell what to do to use a MSFT AAD SAML federated web app seamlessly in a teams app tab?
The manifest already has some switches and fields to handle an AAD enterprise app for correct SSO but I did not see any effect on these entries.
(Recognizing being off-topic a bit)

@BerndGewehr You will need to open a ticket on Azure for that question--I'm not sure if that's in the scope of this particular team.


Update readme

Fixes #26
Signed-off-by: Antoine Galataud<EMAIL_ADDRESS>

@antoine-galataud Thanks for the patch. Please let me verify it.


Add Japanese

for japanese users
please dm to any translation errors to Assault#9807

Shouldn't the file be named "active.jp.toml" instead of ".ja."? I thought jp was the i18n code for Japanese


How to run this example?

First of all: Thanks for creating this plugin!
I followed your instructions, but get the following error:
Error: Cannot find native module 'ShareExtensionNative', js engine: hermes
at node_modules/react-native/Libraries/Core/ExceptionsManager.js:102:17 in reportException
at node_modules/react-native/Libraries/Core/ExceptionsManager.js:148:19 in handleException
at node_modules/react-native/Libraries/Core/setUpErrorHandling.js:24:39 in handleError
at node_modules/expo/build/errors/ExpoErrorManager.js:25:19 in errorHandler
at node_modules/expo/build/errors/ExpoErrorManager.js:30:24 in
at node_modules/@react-native/polyfills/error-guard.js:49:36 in ErrorUtils.reportFatalError
at node_modules/metro-config/node_modules/metro-runtime/src/polyfills/require.js:225:40 in guardedLoadModule
at node_modules/metro-config/node_modules/metro-runtime/src/polyfills/require.js:127:23 in metroRequire
at http://<IP_ADDRESS>:19000/node_modules/expo/AppEntry.bundle?platform=ios&dev=true&hot=false&strict=false&minify=false:null in global
After cloning the repo I just did a "npm install" and then a "npx expo start" and tried to run expo app on the iOS simulator and my own phone.
Since you wrote "Run the expo app and open it once (to setup the api entry into the keychain)" I assumed this was the correct way to do it.
Am I doing something wrong?

the documentation leaves MUCH unanswered and needs to be updated to use EAS

Hi @croepke
Have you tried the following steps?

Clone
npm i
npx expo prebuild --no-install
npx expo run:ios


Reveal in explorer opens explorer twice

When I right-click a file in the sidebar explorer and click "reveal in explorer", the file's folder is opened in the windows explorer. Sometimes after several seconds, the explorer opens the folder again.
This doesn't always happen but every once in a while. Is there maybe a way to log what vscode is doing here ?

Can you reproduce with vscode insiders https://code.visualstudio.com/insiders/
If yes
F1 > developer tools > sources > put a breakpoint here
And figure out what is going on since I can not reproduce

Ok I installed vscode insiders and am sometimes able to reproduce the error. I noticed now that when it happens, the context menu freezes after I right-click and select "reveal in explorer". The second time the explorer opens is after the frozen contextmenu disappears, after some seconds.
I'm still trying to go to the sources but when I do F1 > developer tools I don't get a third option "sources".

Sources is a tab under the developer tools window

Ah ok thank you.

I don't have qttabbar and I get the exact same problem: when I click reveal in explorer, the explorer window opens immediately and VS Code freeze for 30 seconds and then the explorer window open again.


NotifyPropertyChanged not triggered correctly

Bug Information
Version Number of Plugin: 1.4.0
Device Tested On: none
Simulator Tested On: iPhone 13 Pro iOS 15.2
Version of VS: VS for Mac 8.10.19 (build 2)
Version of Xamarin: <IP_ADDRESS>7
Versions of other things you are using:
Steps to reproduce the Behavior
In a simple page, create an Entry and a Label, both bound to the same Validatable(string)'s "Value" member.
Expected Behavior
When writing into the Entry, the Label's text should update automatically.
Actual Behavior
The Label's text is not updated, because "Value"'s setter also sets "ValueFormatted" property which in turn sets the backing field of "Value" and when "Value"'s setter wants to trigger the notify property changed event, it finds out that its own backing field is equal to the new value passed to the setter, so it simply returns false.
prtected bool SetProperty<T>(ref T storage, T value, [CallerMemberName] string propertyName = null)
{
      if (EqualityComparer<T>.Default.Equals(storage, value))
      {
          return false;
      }

      storage = value;
      NotifyPropertyChanged(propertyName);

      return true;
}

Removing "return false" fixes the issue.

Close with #19


[Python] Use tzdata timezone database on Windows

In ARROW-13168, we enabled setting the path of the text-based database engine at runtime. This allowed R to use the tzdb package for the timezone database, since it uses the text format.
However, it doesn't seem like tzdata Python package ships that text format. They do have a "compact" text format, which might be compatible with our vendored date library. Otherwise, we'd likely have to wait for binary format support in https://github.com/HowardHinnant/date/issues/564
Reporter: Will Jones / @wjones127
Watchers: Rok Mihevc / @rok
Related issues:

[C++] Timezone database configuration and access (is related to)

Note: This issue was originally created as ARROW-16054. Please see the migration documentation for further details.

PR to skip the failing tests on Windows that require a tz database: https://github.com/apache/arrow/pull/36996 (interim solution)

Are things still being backported to the branch for 13.0? If so, I think https://github.com/apache/arrow/pull/36996 would make sense, as it will cause test failures on windows otherwise.
CC @raulcd

Are things still being backported to the branch for 13.0? If so, I think #36996 would make sense, as it will cause test failures on windows otherwise.
CC @raulcd

From my understanding the original error was introduced on https://github.com/apache/arrow/issues/36323 which is not part of 13.0.0

From my understanding the original error was introduced on #36323 which is not part of 13.0.0

Ah OK great! Thanks for the info.
Background: I remember seeing a tzdata failure when I had built from source a while ago, but I had already tested one of the rc's, and it passed as well on windows.


Permutation and Combination Calculator

Calculator Functions
Calculator will take the values of 'n' and 'r' and compute the no. of permutations and combinations.
Contribution Event
GSSOC22
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign


fix: excel preview can fail the first time

Fix #360

:tada: This PR is included in version 6.16.1 :tada:
The release is available on:

GitHub release
v6.16.1

Your semantic-release bot :package::rocket:


feat: account list screen

closes #20

functional comment

Show/hide accounts button is always showing, even if I don’t have any hidden account

should show only if there’re hidden accounts


Failed to add egress IP due to failed to find interface

Hi,
I follow this installation : https://github.com/nirmata/kube-static-egress-ip#installation and I get this error after apply my deployment (given at the end of this post).
I deploy only one static-egressip-controller pod (by nodeSelector selection) on node with <IP_ADDRESS> IP.
Here is the error :
I0219 16:55:17.086137       1 controller.go:233] Processing update to StaticEgressIP: default/egressip
I0219 16:55:17.092408       1 gateway.go:86] Created ipset name: EGRESS-IP-2XT2FC5FTMR7KU3B
I0219 16:55:17.095040       1 gateway.go:95] Added ips [<IP_ADDRESS>] to the ipset name: EGRESS-IP-2XT2FC5FTMR7KU3B
I0219 16:55:17.097205       1 gateway.go:108] Added rules in filter table FORWARD chain to permit traffic
E0219 16:55:17.099137       1 controller.go:314] Failed to add egress IP <IP_ADDRESS> for the staticegressip default/egressip on the gateway due to failed to find interface
I0219 16:55:17.099162       1 controller.go:199] Successfully synced 'default/egressip'

Can anyone help me  ? Am I missing something?
Thanks!!!
My deploment :
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: busybox
  name: busybox
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      nodeSelector:
        egress-busybox: "true"
      containers:
      - name: busybox
        image: busybox:latest
        args:
        - /bin/sh
        - -c
        - while (true); do date; wget <third_app_ip>; sleep 1; done;
---
apiVersion: v1
kind: Service
metadata:
  name: busybox
  namespace: default
spec:
  ports:
  - name: web
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: busybox
---
apiVersion: staticegressips.nirmata.io/v1alpha1
kind: StaticEgressIP
metadata:
  name: egressip
  namespace: default
spec:
  rules:
  - egressip: <IP_ADDRESS>
    service-name: busybox
    cidr: <third_app_ip>/32

I have the same issue. Have you managed to find the solution for it yet @WeAreHadock ?

Unfortunately not. We are currently trying to use Calico features instead, as described here: https://www.projectcalico.org/calico-ipam-explained-and-enhanced/

Thank you for suggestion. Very appreciate.
Does this approach work on your side? @WeAreHadock

We manage to create a pool and use it for a busybox, with natOutgoing: false , but this busybox can't reach an external nginx (it works with default ippool with NAT)
Dunno yet where the problem come from (ippool configuration, cidr choosen...), we're currently doing some test.
But give a look if you already use Calico, the MvP described in the article is quite simple.

Hi @murali-reddy, it would be nice if you can take a look on this issue, i am very close to make it work.
Thank you

@tung-huynh @WeAreHadock
Please see https://github.com/nirmata/kube-static-egress-ip/issues/38
I ran into some issues making current implemented solution work across CNI. So at the moment taking differnt approach to find a solution that work across the CNI's and subnets (in case of direct routing).

Same issue here with master:

Does not work as well for me, it fails with same message.
When I log some stuff, I can find that I got:
Route  {
  Ifindex: 1
  Dst: 51.xxx.xxx.160/32
  Src: 51.xxx.xxx.160
  Gw: <nil>
  Flags: []
  Table: 254
}

However here are the ifaces:
{1 65536 lo  up|loopback}
{2 1500 ens3 dd:ff:aa:bb:cc:99 up|broadcast|multicast}

The address 51.xxx.xxx.160 being on ens3, which is index 2 in my case.
In the gateway code there is the following condition that prevent the interface to be detected in my case: if iface.Index == routeIndex
(Btw, I'm on a managed cluster)

I have same issue , plz advise on how to solve this.


change the yaml anchors order to make sure anchors defined before they are referenced

There is a change in bosh cli release v4.0.1 to enforces that YAML anchors are already defined before they are referenced.

https://github.com/cloudfoundry/bosh-cli/releases/tag/v4.0.1

LGTM


fix: clear pending TLS socket handle

What kind of change does this PR introduce?
I recognised when calling canDownload the @puppeteer/browsers package makes a HEAD request, e.g. to https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/116.0.5793.0/mac-arm64/chrome-mac-arm64.zip. This causes the Node.js process to stale as there stay pending TLSSocket around
Did you add tests for your changes?
I am currently working on https://github.com/webdriverio/webdriverio/pull/10767 and have been encountered this issue. Adding given code line resolves it.
My current workaround is:
setTimeout(() => {
    for (const handle of process._getActiveHandles()) {
        if (handle.servername && handle.servername.includes('edgedl.me')) {
            handle.destroy()
        }
    }
}, 10)

If relevant, did you update the documentation?
Summary

Does this PR introduce a breaking change?
I don't think so.
Other information

Thank you for your PR, I left a small comment and after #10668 is merged test should pass as well.

Thanks for the review @Lightning00Blade!
You are totally right, request.resume() is the right thing to call here. Update the PR and PR message, everything should pass now.


TRACE logging should log which classes/fields are planning cloned for debugging purposes

Debugging if a certain class in planning cloned is hard to do.
It would be much easier if at solving start, after the TRACE logging logs the planning entities and planning variables, it also logs which classes are being planning cloned.
Not sure if this is possible, as the planning cloning is dynamic. For example: if a Animal field has a Cat instance, then even if Animal nor Dog isn't planning cloned, Cat might be.

Nice to have


Enforce consistency of Naming Conventions (e.g. GeoTiff/GeoTIFF)

Eo-datasets currently uses both GeoTiff and GeoTIFF internally as defaults, it should be brought into some kind of alignment internally.
If there is a requirement to support both (for historical reasons) this should be configurable at run-time.

Note that the metadata class should convert all of them to "GeoTIFF", see here:
https://github.com/GeoscienceAustralia/eo-datasets/blob/968307311ff87f947c7b1c400285254a58ce8a2b/eodatasets/type.py#L247
Whiich is used when initialising a FormatName field: https://github.com/GeoscienceAustralia/eo-datasets/blob/968307311ff87f947c7b1c400285254a58ce8a2b/eodatasets/type.py#L210
As far as I can see, all other variants of capitalisation are on construction of FormatName()s (particulary during tests). The tests themselves will fail if another capitalisation is output in a package.
(The choice of GeoTIFF is not particular pretty, but predates the use of eodatasets: it was chosen for consistency with the legacy systems)


Support NumPy 1.16

Close #1879

jenkins, test this please.

Jenkins CI test (for commit 8f3ef1f89e1fe1a05df257acf35938353e03af7f, target branch master) succeeded!

I removed unused import.


Add ssh subcommand

Add command to ssh to the (a) master node. Something like:
haas cluster ssh <cluster-name>

Added an IP command in 23d0eea7762d2491919ce7f6ec5cef4b3e333ffa
That may be enough. @oxhead What do you think?

I agree.  We can close this issue.


Clear old def-use record when re-analyze an ID

This is to fix the bug in #336

-1

+2 please squash first.
Why does ClearDef return a bool when you never use the result?

+2  please squash

oh, there's only one commit.  so it's already squashed. :-)


Minor: publish static analysis reports in PR builds

Ok, it seems to work, but it is pretty ugly https://ci-builds.apache.org/job/Kafka/job/kafka-pr/view/change-requests/job/PR-9246/Checkstyle_20Report

Closing this since ASF's Jenkins will strip off any CSS for HTML reports which seriously limits the usefulness of viewing these reports. Will move streams quickstart fix to a separate PR


Update README.md

to close code blocks and enable syntax highlighting

Thanks, David!  I did a big refactoring last night that screwed up all my documentation, but it's great to know why I was having this bug!