Error settings socket options for ICMP during Ping

Hello,
The question is very simple, how can I get the ping error number 106, Error settings socket options for ICMP doing Ping if the Ping itself doesn't need any socket?
Ping command, which uses the Internet Control Message Protocol (ICMP), does not directly involve the creation of a socket in the same way that other network communication methods might. ICMP operates at a lower level than TCP or UDP and is used for network diagnostics rather than establishing a connection.
But it's not so simple. I'm receiving one positive answer:
+UUPING: 1,32,"www.google.com","<IP_ADDRESS>",109,160
and then right after that
+UUPINGER: 106

Hi there: for our information, can you indicate which WiFi module this is with?

Oh, sorry :)
It's not Wi-Fi.
I'm using SARA-R510M8S-01B with nRF+Zephyr.
Ping through LTE-M1 network.

Oh!  Not so much a ubxlib question then (since ubxlib does not send this), more a module behaviour question.
The AT command manual uses phrases like "If the first +UUPING URC reports...", so I'd guess that you may receive multiple ping responses and the second one is indicating that the PDP context has been gone (which appears to be what 10 indicates)...?

omg, I'm so sorry, it's very late night already and I needed to go to sleep instead of posting.
There's a typo, not 10 but 106
Updated that post.

Ah, now I see, 106 == Error creating socket for ICMP.  ICMP is a protocol and so would likely be opened as a socket type?  Not really sure I'm afraid.  I will ask internally what it should be interpreted as.

It seems that AT+UPING, by default, performs four pings, so likely the second one has failed because of a connectivity issue.
The full command: AT+UPING=<remote_host>[,<retry_num>,<p_size>,<timeout>,<ttl>] allows you to set <retry_num> to some other value (1 to 64); it also allows you to set the ping size (<p_size>, range 4 to 1460, default 32), the response timeout in milliseconds (<timeout>, range 10 to 60,000, default 5000) and the time to live (<ttl>, range 1 to 255, default 32).

You are right, yes, exactly four.
I use default settings for the ping command but it doesn't matter, the result is the same.
If it is a connectivity issue why is it always happening the same? only 1 result.
The main question is why is it even possible to get errors about the sockets if the ICMP doesn't use them?

The main question is why is it even possible to get errors about the sockets if the ICMP doesn't use them?

Well, error code 106 is, as written in the AT manual, Error creating socket for ICMP, so either (a) the wording is somehow incorrect or (b) a socket is being used for ICMP.  A search on the internet shows examples of (b) (e.g. this one), it would seem to be an implementation matter, down to how the code inside the module that handles ICMP happens to be implemented in relation to the IP stack within the module.

If it is a connectivity issue why is it always happening the same? only 1 result.

I suspect the only way we will find that out is to set you up for logging the internal behaviour of the module but that will be quite involved, so before we do that, some questions:

could you indicate the FW version number of the module (the response to ATI, sent near boot)?
could you indicate what cellular network/APN you happen to be using?
is the application using any other sockets or IP-related channel at the time, or is AT+UPING the only thing going on?
do you know if performing other sockets or IP-related transactions, instead of AT+UPING work correctly?

Let's do that

02.03,A00.01
Telia EE, The APN is "jtm2m.mnc050.mcc234.gprs"
Just right before Ping I do (uSockGetHostByName(devHandle, "www.google.com", &(address.ipAddress)) and that's it.
No, I don't know that, I just started to work with u-blox, this is my first experience with the modems at all and I just started this summer.

I can't find this
socket(bind_if_addr->ss_family, SOCK_RAW, IPPROTO_ICMP);
Where is that and how can I use this knowledge in resolving my problem? :)

I can't find this
socket(bind_if_addr->ss_family, SOCK_RAW, IPPROTO_ICMP);
Where is that and how can I use this knowledge in resolving my problem? :)

No, sorry, that is the module code, I was simply posting it to confirm that the module is opening a raw ICMP socket to perform the ping; a socket is being used.

Please make contact with me at<EMAIL_ADDRESS>and we can go through the instructions for setting up logging that way rather than trying to do it here.

I will do my test with the HW tomorrow as I need to go to the office and do the physical reconnection as you described in your email.
But in the meantime, by the way, I have to mention that if I turn off uSockGetHostByName just right before my ping command, I don't even get any positive response from the ping, only +UUPINGER: 106. It seems like uSockGetHostByName opens somehow a required socket that ping uses and after it closes it is not immediately so that 1 ping could be served before the required socket closes.

Understood: obviously none of this makes any sense, we need to see what the module is seeing.

Assuming you're using the ubxlib uAtCient to implement your AT+UPING function, could you post the code for that also, in case there's something up with the sequencing?

Sure, after the connection I execute ping function, here it is
static void UUPING_urc(uAtClientHandle_t atHandle, void* pParameter) {
    char ipStr[64];
    uPingContext_t* pPingCtx = (uPingContext_t*)pParameter;

    // default to error
    pPingCtx->status = U_PING_STATUS_ERROR;
    // retry_num
    uAtClientReadInt(atHandle);
    // p_size
    uAtClientReadInt(atHandle);
    // remote_hostname
    uAtClientReadString(atHandle, NULL, 256, false);
    // remote_ip
    if (uAtClientReadString(atHandle, ipStr, sizeof(ipStr), false) > 0) {
        int32_t tmp;
        // Use a temporary output variable to avoid "Attempt to cast away " lint issue
        uSockAddress_t tmpAddr;
        tmp = uSockStringToAddress(ipStr, &tmpAddr);
        pPingCtx->resultSockAddress = tmpAddr;
        if (tmp == (int32_t)U_ERROR_COMMON_SUCCESS) {
            pPingCtx->status = U_PING_STATUS_IP_RECEIVED;
        }
    }
    // ttl
    uAtClientReadInt(atHandle);
    // rtt
    uAtClientReadInt(atHandle);
}

static void UUPINGER_urc(uAtClientHandle_t atHandle, void* pParameter) {
    (void)atHandle;
    uPingContext_t* pPingCtx = (uPingContext_t*)pParameter;

    // we received an error
    pPingCtx->status = U_PING_STATUS_ERROR;
}

void UbloxManager::ping(uDeviceHandle_t devHandle, char const* pHostName) {
    int32_t tmp;
    uAtClientHandle_t atHandle;
    uCellPrivateInstance_t* pInstance;
    static uPingContext_t gPingContext;

    // Check parameters
    if (pHostName != NULL) {
        pInstance = pUCellPrivateGetInstance(devHandle);
        atHandle = pInstance->atHandle;

        // Register the ping URCs
        uAtClientSetUrcHandler(pInstance->atHandle, "+UUPING:", UUPING_urc, (void*)&gPingContext);
        uAtClientSetUrcHandler(pInstance->atHandle, "+UUPINGER:", UUPINGER_urc, (void*)&gPingContext);

        // Send UPING AT command
        gPingContext.status = U_PING_STATUS_WAITING;

        uAtClientLock(atHandle);

        // AT command starts with
        uAtClientCommandStart(atHandle, "AT+UPING=");

        // remote_host
        uAtClientWriteString(atHandle, pHostName, true);

        // retry_num
        uAtClientWriteInt(atHandle, 10);

        // p_size
        uAtClientWriteInt(atHandle, 64);

        // timeout
        uAtClientWriteInt(atHandle, 10);

        // ttl
        uAtClientWriteInt(atHandle, 64);

        uAtClientCommandStopReadResponse(atHandle);
        uAtClientUnlock(atHandle);

        // Regardless if there is an error or not we unregister the URC
        if (pInstance) {
            uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPING:");
            uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPINGER:");
        }
    }
}

Thanks: the only thing I'd say is that you are removing the URC handler here:
        // Regardless if there is an error or not we unregister the URC
        if (pInstance) {
            uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPING:");
            uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPINGER:");
        }

...pretty much immediately after getting the OK back from the AT+UPING command; you probably only want to do that at the end of your application, or at least after a timeout when you are sure you have received all of the expected responses.

Attachment available until 29. Feb 2024
No, but now I put a delay of 30 seconds before those handlers' removers and the result is better! But anyway, ping gives strange results like this:
+UUPING: 1,64,"www.google.com","<IP_ADDRESS>",105,124
+UUPING: 2,64,"www.google.com","<IP_ADDRESS>",105,-1
+UUPING: 3,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 4,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 5,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 6,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 7,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 8,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 9,64,"www.google.com","<IP_ADDRESS>",0,-1
+UUPING: 10,64,"www.google.com","<IP_ADDRESS>",0,-1
TTL and RTT are 0 and -1
and I'm sending you this trace as well.
Click to Download
https://www.icloud.com/attachment/?u=https%3A%2F%2Fcvws.icloud-content.com%2FB%2FAeYda04oWPszOJoZaAqEp6D0G3IPAaK2grywGQekr2l0ewS7Oy7ny_ZH%2F%24{f}%3Fo%3DAruCULk4RQ_4AknkhFDKL01PymWTMkGCB9en1bohNxg-%26v%3D1%26x%3D3%26a%3DCAogEwd_HgWqdIlFO27g51g1pj_6XD3jY3O1Jive-oCKz3gSeBCI29Pi1TEYiOvOtt8xIgEAKgkC6AMA_x9kTdJSBPQbcg9aBOfL9kdqJtyEQQpOAFhQfzyfbshm6AitV9tqA1QpMAwWAjEOi2v-lUfnWzluciZ91SMRoQm8KfwsSL5SjRsX8A7RHL5MqEGPtMEoYgc8kHHAkHsxuw%26e%3D1709243086%26fl%3D%26r%3D2BCCBB46-8496-4235-8921-E7478A83D3FB-1%26k%3D%24{uk}%26ckc%3Dcom.apple.largeattachment%26ckz%3DD6F08130-227B-43B0-B442-60292C62145A%26p%3D62%26s%3DxXvX_DIcgp1AxCi5lKhQzQ_LuCw&uk=27dyTMc2gEX9or42mqc8WQ&f=mc-trace.20240130233555.01.trace&sz=20763769mc-trace.20240130233555.01.trace
20,8 MB

Best regards,
Aleksandr Karabatov
@.***
+372-5397-5449

On 29. Jan 2024, at 22:46, Rob Meades @.***> wrote:
Thanks: the only thing I'd say is that you are removing the URC handler here:
    // Regardless if there is an error or not we unregister the URC
    if (pInstance) {
        uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPING:");
        uAtClientRemoveUrcHandler(pInstance->atHandle, "+UUPINGER:");
    }

...pretty much immediately after getting the OK back from the AT+UPING command; you probably only want to do that at the end of your application, or at least after a timeout when you are sure you have received all of the expected responses.
—
Reply to this email directly, view it on GitHub https://github.com/u-blox/ubxlib/issues/189#issuecomment-1915533442, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADTNWYOIUS7VEFRLOY3GAYLYRADATAVCNFSM6AAAAABCMNXH5WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMJVGUZTGNBUGI.
You are receiving this because you authored the thread.

Thanks: that looks much better, I have downloaded those traces and deleted the links from your post.
Could the UUPINGER: be occurring because your application has moved on and closed down the link before there has been a chance for the responses to arrive?

I set a 30s delay before the deletion of handlers (uAtClientRemoveUrcHandler) and now there is no more UUPINGER #106, so yes, the "socket" problem was because of the immediate deletion of those handlers.
But why do I see these results of 10 pings, I don't understand.

Understood.

But why do I see these results of 10 pings, I don't understand.

We will need to wait for the relevant expert to look at the log, I will update this issue when I know more.

The problem is solved by adding more time to wait for ping results

The problem is solved by adding more time to wait for ping results

Understood: I've not heard anything back from the person who was looking into this internally; I will prod them and update this issue when they get back to me.


Requirement for the 2021 unified agent is not explicit for windows server 2012 and 2016 devices

[Enter feedback here]
The requirements for windows server 2012 and 2016 devices are not explicit regarding the need for the new unified agent.
Live Response does not work if the onboarded devices connect to MDE through MMA, so it should appear as requirement that the 2021 unified agent is required to establish a session (it's only referred when the necessary KB is clicked on)
cheers,

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 81b2a4c2-e698-dda0-9919-80ca2d05b358
Version Independent ID: 19b5f609-fcaa-cb5b-96b7-462285fd32e7
Content: Investigate entities on devices using live response in Microsoft Defender for Endpoint
Content Source: microsoft-365/security/defender-endpoint/live-response.md
Product: m365-security
Technology: mde
GitHub Login: @mjcaparas
Microsoft Alias: macapara

@MasterBlaster4 Thank you for your feedback. Before you can initiate a session on a device running Windows Server 2012 R2 or Windows Server 2016, it is required to have Microsoft Defender for Endpoint update for EDR Sensor. I think it is mentioned as a requirement as the section talks about requirements based on the platform.
Let us know if there is any specific change you would like to make in the document.
Hope this helps!
Thanks
Sri

@msbemba Thank you very much for the contribution and sharing this explanation. @MasterBlaster4 Hope this comment is helpful for you. If you see a documentation update is required, please feel free to open an issue for the same. We proceed here to close it. Thanks for taking out some time to open the issue. Appreciate and encourage you to do the same in future also.


⚠ Bofrost.be has degraded performance

In b15c12c, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1167 ms

Resolved: Bofrost.be performance has improved in e03be7a.


Capacitors values

Thank you for your work on this.
It's possible somehow to identify values for capacitors in this schematic?
using RLC/LCT tester LINK

Measuring capacitors in circuit is always very hit and miss. It depends what else is connected to the capacitors and whether that forms a path across the capacitor somehow. It also depends how it is measured, Very expensive LCR meters use AC of a selectable frequency, say 1 kHz but cheaper ones like the one you linked and all multimeters use a method where they periodically charge the cap and measure how long it takes before it reaches a certain voltage. Longer time = higher capacitance. This method works normally well but has problems one needs to be aware of. For example a very leaky capacitor will show as a cap which much higher capacitance because it takes much longer to accumulate charge which is mistakenly interpreted as a higher capacitance. So if you measure an old  electrolytic cap of say 10uF and it shows as 15 or 20uF it probably means it has too much leakage current.
Some testers use a very low test voltage (below 0.6V) to avoid turning most diodes or transistors on. That works better in circuit but still has problems with Schottky diodes or, in a a capacitor-dropper arrangement, the discharge resistor across the cap. In short, measuring in circuit works sometimes but I would not call it a reliable.
On Wednesday, 31 August 2022 at 09:00:02 BST, Radu<EMAIL_ADDRESS>wrote:

Thank you for your work on this.
It's possible somehow to identify values for capacitors in this schematic?
using RLC/LCT tester LINK
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


fix: use nvim_set_hl instead of highlight.link

Fix #38

Thanks for PR, can you fix code styling?

Thanks for PR, can you fix code styling?

done

Thanks !!


write_scrollback_file doesn't include what's on the current screen

First off, great work on the achievement of this project!
The write_scrollback_file action was suggested in https://github.com/ghostty-org/ghostty/issues/189#issuecomment-1824543708 to be an alternative to the currently missing scrollback search functionality.
When testing this out, I however noticed it's only contents of the scrollback which is outside of the current screen which get written to the file, which means - if you want to search through the scrollback - you will only be able to search through the lines which are not visible on the screen.
Is this intended? Would it not be better to output everything in the scrollback, including what is visible on the current screen?
I can't think of a use case where you'd only want the contents that aren't visible on the screen to be written in the file.
As a quick way of testing this, you can open a new tab, enter for i in $(seq 1 100); do echo $i; done to print out numbered lines, and then perform the write_scrollback_file action by pressing super + shift + j and notice it doesn't include all the lines outputted from the loop.

write_screen_file is what you want. write_scrollback_file only writes the scrollback (not the active screen)

Sorry! Right after creating this, I noticed the write_screen_file (link) action is available which does include the contents of the current screen.

FWIW, write_screen_file and write_scrollback_file are absolutely awful names for these operations given what they do. I think that we should re-open this issue as a springboard to rename them and to change the default keybinding.

This is the terminology we use internally:
https://github.com/ghostty-org/ghostty/blob/1fa2e699d142b9bc2795a4d3db98b035cc5a825c/src/terminal/point.zig#L6-L48

I’m not trying to suggest it. I’m just posting it as information.


cross-env: command not found

I'm running node 4.4.5 locally. Cloned the repo, ran setup command without issue. On 'npm run start' I get an error:

<EMAIL_ADDRESS>start /Users/kdolbear/Documents/Jobs/mower/react-boilerplate
npm run start:tunnel
<EMAIL_ADDRESS>start:tunnel /Users/kdolbear/Documents/Jobs/mower/react-boilerplate
cross-env NODE_ENV=development node server

sh: cross-env: command not found
npm ERR! Darwin 15.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start:tunnel"
npm ERR! node v4.4.5
npm ERR! npm  v2.15.5
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR<EMAIL_ADDRESS>start:tunnel: cross-env NODE_ENV=development node server
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start:tunnel script 'cross-env NODE_ENV=development node server'.
npm ERR! This is most likely a problem with the react-boilerplate package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     cross-env NODE_ENV=development node server
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs react-boilerplate
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls react-boilerplate
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/kdolbear/Documents/Jobs/mower/react-boilerplate/npm-debug.log
npm ERR! Darwin 15.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start"
npm ERR! node v4.4.5
npm ERR! npm  v2.15.5
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>start: npm run start:tunnel
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>start script 'npm run start:tunnel'.
npm ERR! This is most likely a problem with the react-boilerplate package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm run start:tunnel
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs react-boilerplate
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!
npm ERR!     npm owner ls react-boilerplate
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/kdolbear/Documents/Jobs/mower/react-boilerplate/npm-debug.log
And here's the log:
0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'start' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose run-script [ 'prestart', 'start', 'poststart' ]
5 info prestart<EMAIL_ADDRESS>6 info start<EMAIL_ADDRESS>7 verbose unsafe-perm in lifecycle true
8 info<EMAIL_ADDRESS>Failed to exec start script
9 verbose stack Error<EMAIL_ADDRESS>start: npm run start:tunnel
9 verbose stack Exit status 1
9 verbose stack     at EventEmitter. (/usr/local/lib/node_modules/npm/lib/utils/lifecycle.js:217:16)
9 verbose stack     at emitTwo (events.js:87:13)
9 verbose stack     at EventEmitter.emit (events.js:172:7)
9 verbose stack     at ChildProcess. (/usr/local/lib/node_modules/npm/lib/utils/spawn.js:24:14)
9 verbose stack     at emitTwo (events.js:87:13)
9 verbose stack     at ChildProcess.emit (events.js:172:7)
9 verbose stack     at maybeClose (internal/child_process.js:827:16)
9 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
10 verbose pkgid<EMAIL_ADDRESS>11 verbose cwd /Users/kdolbear/Documents/Jobs/mower/react-boilerplate
12 error Darwin 15.5.0
13 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "start"
14 error node v4.4.5
15 error npm  v2.15.5
16 error code ELIFECYCLE
17 error<EMAIL_ADDRESS>start: npm run start:tunnel
17 error Exit status 1
18 error Failed at the<EMAIL_ADDRESS>start script 'npm run start:tunnel'.
18 error This is most likely a problem with the react-boilerplate package,
18 error not with npm itself.
18 error Tell the author that this fails on your system:
18 error     npm run start:tunnel
18 error You can get information on how to open an issue for this project with:
18 error     npm bugs react-boilerplate
18 error Or if that isn't available, you can get their info via:
18 error
18 error     npm owner ls react-boilerplate
18 error There is likely additional logging output above.
19 verbose exit [ 1, true ]

npm install cross-env will fix your problems! I wonder why that's not installed…

Thanks for the prompt reply. I installed cross-env, restarted CLI, re-cloned repo just to be sure I had a clean slate and am getting the same error as before.

I upgraded node to the most recent release - 6.2.1 - and re-ran. This time I got the following:
npm ERR! Darwin 15.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "setup"
npm ERR! node v6.2.1
npm ERR! npm  v3.9.3
npm ERR! code MODULE_NOT_FOUND
npm ERR! Cannot find module 'github-url-from-git'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     https://github.com/npm/npm/issues
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/kdolbear/Documents/Jobs/mower/react-boilerplate/npm-debug.log
with this in the log:
0 info it worked if it ends with ok
1 verbose cli [ '/usr/local/bin/node', '/usr/local/bin/npm', 'run', 'setup' ]
2 info using<EMAIL_ADDRESS>3 info using<EMAIL_ADDRESS>4 verbose stack Error: Cannot find module 'github-url-from-git'
4 verbose stack     at Function.Module._resolveFilename (module.js:440:15)
4 verbose stack     at Function.Module._load (module.js:388:25)
4 verbose stack     at Module.require (module.js:468:17)
4 verbose stack     at require (internal/module.js:20:19)
4 verbose stack     at Object. (/usr/local/lib/node_modules/npm/node_modules/read-package-json/node_modules/normalize-package-data/lib/fixer.js:2:22)
4 verbose stack     at Module._compile (module.js:541:32)
4 verbose stack     at Object.Module._extensions..js (module.js:550:10)
4 verbose stack     at Module.load (module.js:458:32)
4 verbose stack     at tryModuleLoad (module.js:417:12)
4 verbose stack     at Function.Module._load (module.js:409:3)
5 verbose cwd /Users/kdolbear/Documents/Jobs/mower/react-boilerplate
6 error Darwin 15.5.0
7 error argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "setup"
8 error node v6.2.1
9 error npm  v3.9.3
10 error code MODULE_NOT_FOUND
11 error Cannot find module 'github-url-from-git'
12 error If you need help, you may report this error at:
12 error     https://github.com/npm/npm/issues
13 verbose exit [ 1, true ]

I'm also having some weird issues with this lib.  I was able to fix them by deleting and reinstalling cross-env and then creating my own symlink manually in $boilerplate-root/node_modules/.bin/
obviously that isn't a solution

Hmm, that's really weird, not sure what we can do to fix this though…

I can't reproduce it regularly either.  We need to add it as a production dependency anyway so hopefully it just goes away;

As I mentioned above, in trying to fix the issue I had upgraded Node locally. I did a little searching on NPM issues and found this. I manually removed Node and NPM, then did a clean install of 6.2.1, cloned the boilerplate repo and everything works perfectly.

Okay, perfect, this doesn't seem like something we can fix from our side. Glad this is resolved for you @krispind, I'll close this for now!

Hm,
I ran across the same exact issue when yarn run start was running in one terminal, and yarn bild in the other.
One of the log messages read as follows:
error An unexpected error occurred: "ETXTBSY: text file is busy, open '/home/dmitry/project/node_modules/ngrok/bin/ngrok'".

Stopping the yarn run start process magically fixed it.

hello after running npm install cross_env it works but i still have some troubles.It said that their and error for compiling the file

This maybe an unnecessary addition but I found using Git Bash to access Npm and setup the boilerplate caused issues. Usually I can use Git Bash for Npm commands but here I had to clone the repo in Git Bash then switch to windows CMD to complete the setup.

thanks men!

sh: 1: cross-env: not found
npm ERR! Linux 3.10.0-514.6.2.el7.x86_64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "run" "build"
npm ERR! node v6.2.2
npm ERR! npm  v3.9.5
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR<EMAIL_ADDRESS>build: cross-env NODE_ENV=production language=en  webpack --progress --hide-modules
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>build script 'cross-env NODE_ENV=production language=en  webpack --progress --hide-modules'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the disney-game package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     cross-env NODE_ENV=production language=en  webpack --progress --hide-modules
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs disney-game
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls disney-game
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /home/jenkins/workspace/Disney_node_demon/DocMcstuffins/Doc.McS.S01F024/act2_cache_the_red_car/src/page/npm-debug.log
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Hello men, I still didn't slove this problem. Can anybody puts some  specific methods?

@LisaYF did u try @krispind suggestion?


Backport2.1: Show error message only if there was an error

Description
Backport of #919 to mbedtls-2.1
Status
READY

Needs a Changelog entry, but that can be done at Gatekeeping.


Websocket onerror:  Negotiate Unknown

A few days ago i'm receiving this error from the websocket listen:
[2017-12-20T20:19:32+00:00] Websocket bound
[2017-12-20T20:47:27+00:00] Websocket disconnected
[2017-12-20T20:47:27+00:00] Websocket auto reconnecting
[2017-12-20T20:47:27+00:00] Connection aborted
[2017-12-20T20:47:27+00:00] Websocket bound
[2017-12-20T23:07:48+00:00] Websocket disconnected
[2017-12-20T23:07:48+00:00] Websocket auto reconnecting
[2017-12-20T23:07:48+00:00] Connection aborted
[2017-12-20T23:07:48+00:00] Websocket bound
[2017-12-20T23:44:51+00:00] Websocket disconnected
[2017-12-20T23:44:51+00:00] Websocket auto reconnecting
[2017-12-20T23:44:52+00:00] Websocket onerror:  Negotiate Unknown
[2017-12-20T23:44:53+00:00] Connection aborted

and after that it does not try to reconnect again, it was stuck at 'Connection aborted' from few hours now.
My code looks like this:
BittrexService.websocketListen = (callback) => {
    bittrex.websockets.client((wsClient) => {
        wsClient.serviceHandlers.connected = (wsConnection) => {
            wsClient.call('CoreHub', 'SubscribeToSummaryDeltas').done((err, result) => {
                if(err) return console.error(err);

                if(result === true) {
                    bittrex.websockets.listen((data, client) => {
                        if(data.M === 'updateSummaryState') {
                            _.forEach(data.A, (data) => {
                                callback(data)
                            });
                        }
                    })
                }
            });
        };
    });
}

Is something that i'm done wrong?

Anyone ? Up


Implement Issue Forms

https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/configuring-issue-templates-for-your-repository#creating-issue-forms

This responsibility has been handled by the template.


Resultant SVG has partial duplicates left and right

Thank you, Anemy, for this awesome, versatile tool! I am noticing that parts of my image are duplicated to the left and right of the exported SVG. Minor issue, but does mean the SVGs have to be manually edited before plotting.

Hmmm... this is definitely a bug I'll look into it and report back once I've got a fix. Thanks for reporting @cleversomeday ! Sorry on the late response - I must have had notifications off for this project.


Is unfair comparison?

Why choose the 79.3 record?
Maybe I miss some details.

All the data in Figure.1 are pass@1,  0-shot without any feedback, thanks.


Entrypoint changes for watchloop & CRIO restart

Tomo, here's the changes for the watch loop & the CRIO restart. I added a parameter to enable the CRIO restart. Let me know if you see anything that needs changes. Feel free to push commits to my branch, too.
I'll also follow up with a PR for the cluster-network-operator to hopefully enable this.

Pull Request Test Coverage Report for Build 847

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 71.089%

Totals

Change from base Build 836:
0.0%

Covered Lines:
927

Relevant Lines:
1304

💛  - Coveralls

Looks good to me, but could you please test with daemonset rolling update?
https://kubernetes.io/docs/tasks/manage-daemon/update-daemon-set/
I'd like to double check that config files invalidated->validated.

Tomo, great idea on validating that works as intended during a rolling update, I was able to have some successful results with this procedure:
As the 0th step: I built a new nfvpe/multus:latest image using this branch as a basis.
Then, I modified the daemonset YAML to use the new parameter for config cleanup...
[centos@kube-singlehost-master multus-cni]$ git diff
diff --git a/images/multus-daemonset.yml b/images/multus-daemonset.yml
index 5551520..fd8525c 100644
--- a/images/multus-daemonset.yml
+++ b/images/multus-daemonset.yml
@@ -138,6 +138,7 @@ spec:
         command: ["/entrypoint.sh"]
         args:
         - "--multus-conf-file=auto"
+        - "--cleanup-config-on-exit=true"
         resources:
           requests:
             cpu: "100m"
@@ -199,6 +200,7 @@ spec:
         command: ["/entrypoint.sh"]
         args:
         - "--multus-conf-file=auto"
+        - "--cleanup-config-on-exit=true"
         resources:
           requests:
             cpu: "100m"
@@ -228,3 +230,4 @@ spec:
             items:
             - key: cni-conf.json
               path: 70-multus.conf
+

Next, I created Multus daemonset normally... This is using :v3.2 tagged image.
[centos@kube-singlehost-master multus-cni]$ kubectl create -f images/multus-daemonset.yml
customresourcedefinition.apiextensions.k8s.io/network-attachment-definitions.k8s.cni.cncf.io created
clusterrole.rbac.authorization.k8s.io/multus created
clusterrolebinding.rbac.authorization.k8s.io/multus created
serviceaccount/multus created
configmap/multus-cni-config created
daemonset.extensions/kube-multus-ds-amd64 created
daemonset.extensions/kube-multus-ds-ppc64le created
[centos@kube-singlehost-master multus-cni]$ watch -n1 kubectl get pods --all-namespaces
[centos@kube-singlehost-master multus-cni]$ kubectl logs kube-multus-ds-amd64-fdj78 --namespace=kube-system
WARNING: unknown parameter "--cleanup-config-on-exit"
Generating Multus configuration file ...
Config file created @ /host/etc/cni/net.d/00-multus.conf
Entering sleep... (success)

The configuration file is created as usual.
Next, I remove the Flannel configuration, and kick off a rolling update by updating the image
[centos@kube-singlehost-master multus-cni]$ sudo rm /etc/cni/net.d/10-flannel.conflist
[centos@kube-singlehost-master multus-cni]$ kubectl set image ds/kube-multus-ds-amd64 kube-multus=nfvpe/multus:latest -n kube-system
daemonset.extensions/kube-multus-ds-amd64 image updated
[centos@kube-singlehost-master multus-cni]$ watch -n1 cat /etc/cni/net.d/00-multus.conf
[centos@kube-singlehost-master multus-cni]$ kubectl rollout status ds/kube-multus-ds-amd64 -n kube-system
Waiting for daemon set "kube-multus-ds-amd64" rollout to finish: 0 out of 1 new pods have been updated...
Waiting for daemon set "kube-multus-ds-amd64" rollout to finish: 0 out of 1 new pods have been updated...
Waiting for daemon set "kube-multus-ds-amd64" rollout to finish: 0 out of 1 new pods have been updated...
Waiting for daemon set "kube-multus-ds-amd64" rollout to finish: 0 of 1 updated pods are available...
daemon set "kube-multus-ds-amd64" successfully rolled out

Once the rollout is complete, I check the contents of the Multus configuration...
[centos@kube-singlehost-master multus-cni]$ cat /etc/cni/net.d/00-multus.conf
{Multus configuration intentionally invalidated to prevent pods from being scheduled.}

I then copy a Flannel config back into the CNI config directory, and we can see the contents of the multus config are restored to a working config.
[centos@kube-singlehost-master multus-cni]$ sudo cp /tmp/10-flannel.conflist /etc/cni/net.d/
[centos@kube-singlehost-master multus-cni]$ cat /etc/cni/net.d/00-multus.conf
{ "name": "multus-cni-network", "type": "multus", "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig", "delegates": [ { "name": "cbr0", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } ] }

The multus logs resulted in:
[centos@kube-singlehost-master multus-cni]$ kubectl logs kube-multus-ds-amd64-t282f --namespace=kube-system
2019-08-20T19:18:52+0000 Generating Multus configuration file using files in /host/etc/cni/net.d...
2019-08-20T19:18:53+0000 Attemping to find master plugin configuration, attempt 0
2019-08-20T19:18:53+0000 Multus configuration intentionally invalidated to prevent pods from being scheduled.
2019-08-20T19:18:58+0000 Attemping to find master plugin configuration, attempt 5
2019-08-20T19:19:03+0000 Attemping to find master plugin configuration, attempt 10
2019-08-20T19:19:08+0000 Attemping to find master plugin configuration, attempt 15
2019-08-20T19:19:11+0000 Using /host/etc/cni/net.d/10-flannel.conflist as a source to generate the Multus configuration
2019-08-20T19:19:11+0000 Config file created @ /host/etc/cni/net.d/00-multus.conf
{ "name": "multus-cni-network", "type": "multus", "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig", "delegates": [ { "name": "cbr0", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } ] }
2019-08-20T19:19:11+0000 Entering watch loop...

Doug, thank you for your test!!
Looks good to me!


fix get_url for windows

os.sep is different on Windows , which result in wrong url parth

Thanks!


Generate preview link for each PR

Purpose

The purpose of this PR is to fix #349 #320
Goals

Generate a preview link for each Pull request.

Approach

Create a workflow to create a preview link for each Pull request.
Update pull request template.

Checklist

[x] This PR doesn't commit any keys, passwords, tokens, usernames, or other secrets.
[x] I have read and understood the development best practices guidelines
[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

@kumuditha-udayanga is this PR ready for review?


Add ToadicusTools from Kerbal Stuff

This pull request was automatically generated by Kerbal Stuff on behalf of toadicus, to add ToadicusTools to CKAN.
Please direct questions about this pull request to toadicus.

#rebuild

#rebuild


fix: Prevent mypy error on Starlette v0.35+

As witnessed here:
https://github.com/kolonialno/trex/actions/runs/7499504189/job/20416417776?pr=4080

Ah, I "took a guess" here, and didn't really run the linters... 🙈

Now it looks better 😅


Use the private memo as the airdrop description line in transaction view, without a quote marker

@keybase/picnicsquad CC @patrickxb @cecileboucheron

@cjb The color of the text should be purpleDark.

@cecileboucheron Thanks, done.


Change Management

Add a system for change management for changes triggered by AutoTransform. The way this will work is the following:
A changemanager.py script will be run that leverages a ChangeRequestManager class to do the following:

Get all existing outstanding change requests (A generic name for a pull request)
Handle outstanding change requests

If the request is more than X days stale, trigger a "rebase" which effectively spawns a worker to run on the same input/schema on a more recent checkout of master, then updates the request to the new changes.
If the request has been approved, land it (i.e. merge)
If the request has been denied, close it

A GithubChangeRequest and GithubChangeRequestManager class will be included for supporting pull requests on Github.
Handling of outstanding requests will be done using ChangeRequestManagerStep classes. Additional steps can be added by an organization to provide their own handling behavior.
In the future, code may be added that will allow organizations to host a webpage that can display information on outstanding change requests. The security/custom stuff required for this may make this too difficult to do generically, meaning organizations will need to host this content themselves if they desire it.

Management is in place :)


Fix documentation of http1_title_case_headers()

Case is not preserved either way, so the old text was misleading.

Thank you! <3


[Bug]: unable to push new changes

Describe the bug
Unable to perform a git push from the command palette

Relevant errors (if available)
No response
Steps to reproduce
this was working fine, I updated the plugin and something broke
unfortunately I can't provide exact steps to reproduce it, just the error message
Expected Behavior
No response
Addition context
No response
Operating system
macOS
Plugin variant
Obsidian Git
Plugin version
1.29.0

Should be fixed now in new release.


Meta-issue: Steel inference, framing, and selectors

[x] Move all modules to use the PCM-based memory model (Nik)

[ ] Revise the formulation of frame preservation and remove the assume Steel.Effect.Atomic.witness_h_exists (Aseem)

[ ] Add an interface to Steel.FramingEffect so that we can change its representation to use action trees directly rather than actions (once we have cumulativity)

[ ] Remove the assumes in Steel.FramingEffect related to sl_implies

[ ] Use Steel.FramingEffect to simplify existing modules, starting with Steel.HigherReference

[ ] Extend SteelAtomic effect to support 2 additional indexes for requires/ensures

[ ] Extend the multi-effect system to incorporate SteelAtomic

[ ] Add an interface to Steel.Preorder so that its SMT patterns are hidden (Nik)

[ ] Continue work on selectors and viewables (Aymeric)

Older typing glitches:

The post-hprop of a return is often not inferred

Trouble mixing refinement types. Not sure why the middle one (test_fails) fails.

assume
val t (n:nat) : Type0

noeq
type s (n:nat) =
  | MkS : st:t n -> s n

assume val ps (#n:nat) (x:t n) : hprop

let test_ok (n:nat) (x:t n)
  : SteelT (y:s n{y.st == x}) (ps x) (fun _ -> ps x)
  = return #_ #(fun _ -> ps x)  (MkS x)

let test_fails (n:nat) (x:t n)
  : SteelT (y:s n{y.st == x}) (ps x) (fun (y:s n{y.st == x}) -> ps y.st)
  = h_admit #(y:s n{y.st == x}) (ps x) (fun (y:s n{y.st == x}) -> ps y.st)

let ss #n (x: t n) = y:s n{y.st == x}
let test_ok_also (n:nat) (x:t n)
  : SteelT (ss x) (ps x) (fun (y:ss x) -> ps y.st)
  = return #(ss x) #(fun (y:ss x) -> ps y.st) (MkS x)

[ ] Move all modules to use the PCM-based memory model (Nik)

[ ] Revise the formulation of frame preservation and remove the assume Steel.Effect.Atomic.witness_h_exists (Aseem)

[ ] Add an interface to Steel.FramingEffect so that we can change its representation to use action trees directly rather than actions (once we have cumulativity)

[ ] Remove the assumes in Steel.FramingEffect related to sl_implies

[ ] Use Steel.FramingEffect to simplify existing modules, starting with Steel.HigherReference

[ ] Extend SteelAtomic effect to support 2 additional indexes for requires/ensures

[ ] Extend the multi-effect system to incorporate SteelAtomic

[ ] Add an interface to Steel.Preorder so that its SMT patterns are hidden (Nik)

[ ] Continue work on selectors and viewables (Aymeric)

Posting here several observations from ongoing work:
Example 4 is indeed due to the discrepancy between hide u0 and v0. We might be able to apply an slprop canonicalization pass before running the tactic to solve this specific case.
Example 5, 6, 7 all suffer from program implicits that appear in the slprop indices (such as the permission, or the ghost value). In the current state, these implicits can only be inferred by unification on the slprop indices, but the tactic does not handle it correctly, since it considers for instance pts_to r0 full_perm v0 and pts_to r0 ?u1 ?u2 to be different terms. Passing these implicits explicitly in rread and rwrite (for instance rread #_ #full_perm #v0 r0) allows example 5 to verify.
In addition to requiring explicit program implicits, examples 6 and 7 also need their bodies to end with an explicit () to verify. When designing the framing tactic in our idealized calculus, we made the assumption that each body would terminate with a return, and hence a SteelF function. This invariant does not hold here, and that seems to be the cause for an slprop mismatch. I'm investigating some more.
Minor typo: In example 7, the first rwrite should have r0 as an argument

More comments about why we currently require a return at the end of each function.
Let's consider the following code:
  : SteelT unit (pts_to r0 full_perm v0)
                (fun _ -> pts_to r0 full_perm v0)
  = let u0 = rread #_ #full_perm #v0 r0 in
    rwrite_alt #_ #v0 r0 v0 u0```

As this is a bind between two annotated Steel functions, the inferred type for the body of the function would be of the shape `SteelT unit (?u1 * ?u2) (?u3 * ?u4)`. But an extra indirection is added when performing subcomp, adding the following constraints:

?u5 == ?u3 * ?u4
?u6 == (fun _ -> pts_to r0 full_perm v0)
can_be_split_forall ?u5 ?u6

In the current state of framing, we first solve the trivial equalities (i.e. where one side contains no implicit, so `?u6` here, as well as ?u3 which corresponds to the postresource of rwrite), then the postresource of subcomp (i.e. `can_be_split ?u5 ?u6` here) and then the rest of the constraints. But when solving `?u5`, there is a chance we pick a term that will be equivalent to `(fun _ -> pts_to r0 full_perm v0) * ?u4`, but unfortunately not equal, leading to a failure when trying to solve the equality.

When the last command in the body is a return (or a lifted pure variable), and that the previous command was a `Steel` (which corresponds to almost all cases so far), we know the shape of the corresponding `?u3 * ?u4`: `?u3` will be the complete postresource, and `?u4` will be an empty frame.
Hence we can rig the solving of subcomp_post to make sure that the equality corresponding to `?u5 == ?u3 * ?u4` can be solved when finally encountered.

So what can we do in the general case? It seems that we need some knowledge of the shape of the inferred postcondition to correctly solve subcomp_post. A proposed solution is to solve all the equalities before solving subcomp_post, which will create new restricted uvars.
In our example, this would give us the constraint `can_be_split (fun _ -> pts_to r0 full_perm v0 `star` ?u7) (fun _ -> pts_to r0 full_perm v0)` when solving subcomp_post.

How is it different from the previous naive solution which caused scoping issues? Here, we *only* solve the equalities, not the slprop equivalences which require calls to `canon`. Especially, given the cascaded structure of let-bindings, the only scope restriction that should happen is for the postcondition of the very last term, when restricting its scope to the top-level scope. All the other equalities should be restricted to inner constraints in binds, and should not leak outside these binds since they are then "blocked" by slprop equivalences.

One possible issue is that we are breaking the invariant that we have at most one slprop uvar when calling canon. We need to identify when we might have several uvars, and investigate how to tweak the tactic accordingly.

An additional advantage is that reading constraints after solving equalities is much easier. We don't have extra indirections, only the slprop equivalences.

We also discussed how to typecheck the following example:
assume val p: slprop

let f (#a:Type) (r0:reference a) (v0:erased a)
  : SteelT unit (p `star` pts_to r0 full_perm v0) (fun _ -> p `star` pts_to r0 full_perm v0)
  = rwrite_alt #_ #v0 r0 v0 (Ghost.reveal u0)

This only involves a subcomp from Steel to Steel. As Aseem remarked this morning, a subcomp from Steel to Steel only occurs when the body consists of a single function call. As soon as we have a bind, the inferred effect becomes SteelF. Furthermore, we also observed that having a refinement in the return type was also generating a bind (for instance if we were calling rread, and returning x:a{x == Ghost.reveal v0}). This seems to be due to an assert implicitly generated for the refinement.
To solve this, a solution might be to support framing inside of the Steel <: Steel subcomp:
given a f : SteelT a pre_f post_f and an annotation SteelT a pre_g post_g, subcomp would succeed if we can solve pre_g <==> pre_f * frame and forall x. post_g x <==> post_f x * frame.


⚠ GitLab Server has degraded performance

In fccd60f, GitLab Server (https://gitlab01.its-telekom.eu) experienced degraded performance:

HTTP code: 200
Response time: 11966 ms

Resolved: GitLab Server performance has improved in 07b7e51.


Negative health effects with a time do not reduce when in menu

I believe health effects should count down until 0 while out of raid, i don't think they get reduced while in menu.
Would need confirmation of this, e.g. muscle fatigue after using Gym
We do something 90% similar to what's required as part of game start: updateProfileHealthValues(), perhaps this can be repurposed into a service that runs every 60 secs?

Originally written by chomp

Current status:
Effect time is calculated/adjusted when logging into game
Effect time is calculated/adjusted when exiting a raid
Effect time is NOT calculated/adjusted when sitting in menu / doing menu actions

Originally written by chomp


[Dashboard cards] Need "pop out" expand icon in card header

Summary
Need "expand" option in dashboard card header.

Additional context
Health & Predict are following the pattern that Monitor set for expanding cards to show a larger view of the data, but it looks like the expand button isn't available in the IoT cards.  We need to expand a value card first, but also need this option on chart cards.
Click on the expand icon in the upper right

to open a dialog that has additional info

Specific timeline issues / requests
Do you want this work within a specific time period? Is it related to an
upcoming release?
Health needs this for our October 2020 release.
Lack of this button blocks creation of the dialog that is invoked when the user clicks the button.
Available extra resources
What resources do you have to assist this effort?
The Predict team hard-coded this into their chart cards, copying the Monitor team, and should hopefully be able to contribute it back.

Would like the button to fire any event, not just a Carbon dialog, since our dialogs in Graphite are different.

:tada: This issue has been resolved in version 2.120.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Test for transposing Square SGSparseMatrix.

Adding a test for transposing a square SGSparseMatrix.
#2748

@karlnapf : The build error has nothing to do with the test I included. I also ran a fresh build locally on my system and it worked fine. Additionally I ran the test with ctest. Is there anyway that you could restart the build ?

No worries about the new pr. keep in mind though that you can always clone your fork. Never delete your fork!!

@karlnapf: Squashed it, take a look. Would you be available some time (15-20 minutes), I was trying to add testing data for meta examples and was stuck. Need a little help. I did read the docs and some other recent accepted PRs regarding that but to no avail.

@karlnapf: Reminder, this is ready to merge, implemented your suggestions.

Sorry for the delay, I was on holiday.
Yes ping me in irc anytime if you need help

@karlnapf: Done. I replaced the EXPECT_EQ macro in my test with EXPECT_NEAR as you suggested and have squashed the commit as well. Review.

Now we are game :) Thanks!


[META] Determine API design for performing maintainer actions

What's the problem this feature will solve?
We've had several requests for an API that would let users perform various maintainer actions only possible via the web UI right now:

Deleting releases/files (https://github.com/pypi/warehouse/issues/12934)
Yanking releases
Generating API tokens (https://github.com/pypi/warehouse/issues/6396)
Regenerating/invalidating API tokens (https://github.com/pypi/warehouse/issues/12765)
Configuring trusted publishers (https://github.com/pypi/pypi-oidc-private-beta-community/issues/34)

Describe the solution you'd like
Before attempting to implement any of them, we should figure out how we would generally want to expose these types of APIs to maintainers, the API format, authentication mechanism, etc.

API format: some sort of REST + JSON, though I guess this is now about as generic as saying 'implement it using computers' :wink: . E.g. for deletion, it would be nice if each file and release had an API url, and you used the HTTP DELETE verb on that URL to delete it. Obviously there are a whole lot of details to work out here, but I imagine those will mostly be hammered out for each specific piece of API.
Authentication: I imagine some flavour of Oauth is the obvious answer? Please ensure there's a clear story for people authenticating scripts etc. from their own machine, though. Too many APIs with Oauth assume that you'll only ever sign in through another server which can have a 'client secret', and force users to 'create an application' and copy and paste stuff around just to do something simple.

@jaraco ^

Just doing some sketching:
The current trusted publishing APIs live under /_/, which (IMO) is a fitting place for PyPI-specific APIs as well. In terms of routes and verbiage, it probably makes sense to roughly mirror the HTML routes/views where possible while being RESTful, e.g.:

/_/api/manage/project/release: DELETE for deletion, PATCH for yanking (?)
/_/api/manage/token: DELETE for "burning" a token

The equivalent routes here are under /manage/account/token, but with trusted publishing a token is no longer necessary associated with an account.

For additive operations (like adding new trusted publishers, creating new API tokens, etc.) we should think carefully about the permission model: if it's too loose, we might end up encouraging users to create "god" tokens for API interactions that they end up sharing/using somewhat haphazardly. For example, we should probably be very careful about how (if) we allow an API to mint more API tokens, and ensure that the majority of users are funneled to trusted publishing for that kind of use case instead.
CC @tnytown for visibility

Another thing to consider: we currently offer "user-scoped" API tokens, which just means that the token is valid for every project the user has. We probably don't want to inadvertently broaden the scope of those "user" tokens to all user management tasks as well, so we should probably think about changing the name of that Macaroon caveat 🙂

I guess authentication is one of the first pieces to figure out. I see a couple of options for how this will work from a user perspective:

You enter your PyPI username, password & 2FA code in the app (e.g. at terminal prompts), and it sends these to PyPI, exchanging them for a token it can use to authenticate later requests.
The app opens a web browser, or gives you a URL to open, where you log in to PyPI and grant it access.

I think 1. is more flexible - no need for a browser - and a nicer user experience - no need for a disorienting switch into a browser and back. But 2. is probably more secure - you have a chance to check what permissions you're granting, and it's harder for a clumsily coded app to save your password. 2. also means apps don't need to deal with 2FA security keys.
What do you get once authenticated? Would the token let you upload packages directly, or would you use the API token to create an upload token? Presumably the token you get would have some expiry date? But upload tokens are perpetual by default - does anyone want to come up with advice for tools on which sort of token to store?

I might have misunderstood, but I thought the assumption here was that these APIs would be limited to API tokens only.
In other words: the authentication flow here would be to create an appropriately scoped API token through the PyPI web interface, and then authenticate to the API with that token. That in turn would allow us to reuse all of the existing scaffolding we have around API token creation, without having to solve problems like doing TOTP challenges in the user's terminal.

I believe almost all of the APIs would use API Tokens yes.
One of the things people have asked for is an API to generate tokens though, which would involve some kind of login flow.

Yup, I'd like to have a better way to upload projects than manually creating, copying and pasting a token for each project (besides storing a user-scoped token, which is basically as powerful as my password).

I'd like Read-Only API Tokens for user accounts (not projects), or the equivalent OAuth scope'd token if a PyPI OAuth provider is implemented in the future.

For an API-based reimplementation of our PyPI management tool the we'd need:

an authentication mechanism;
endpoints for invitation management;
a way of listing projects for an authenticated account;
a way of listing the contributors (including their roles and 2FA status) to those projects.

I suppose some of the required functionality could become obsolete once https://blog.pypi.org/posts/2023-04-23-introducing-pypi-organizations/ is fully rolled out for us.


Missing sealed classes

The following class are missing the sealed keyword.

PlayerMovement.cs
InputParser.cs
GridList.cs

This in imported because it's in the code convention and it's alot nicer to have.

Sealed cannot be used for GridList.cs since it is not valid for the type.


No need to base64 decode secret data with client-go

OSD-13907 The following error was happening when attempting to use the AWS credential override:
{"level":"error","ts":"2023-01-27T18:30:10Z","msg":"Reconciler error","controller":"vpcendpoint","controllerGroup":"avo.openshift.io","controllerKind":"VpcEndpoint","VpcEndpoint":{"name":"avo-sc-test","namespace":"openshift-aws-vpce-operator"},"namespace":"openshift-aws-vpce-operator","name":"avo-sc-test","reconcileID":"5dad7071-549c-4136-9fa4-fed776605e88","error":"failed to describe subnets: operation error EC2: DescribeSubnets, exceeded maximum number of attempts, 3, https response error StatusCode: 0, RequestID: , request send failed, Post \"https://ec2.us-east-2.amazonaws.com/\": net/http: invalid header field value for \"Authorization\""}

because we were doing an extra base64 decode... client-go returns the non-encoded value of the secret after converting []byte to string

/lgtm


Nancy 2.0-alpha upgrade feedback

We are trying to collect feedback on the 2.0-alpha pre-release of Nancy. We are looking for both people with existing applications, whom are willing to update to 2.0-alpha (evaluation purpose only, we are not asking you to put it into production) and share their experience as well as people whom would just like to try out the new bits and share their though and experiences.
Currently we are mostly looking for:

Pain points while upgrading from 1.x to 2.0-alpha
Any stability issues (hanging requests, exceptions, performance and so on)?

But, of course, if you come across anything else then we will happily accept that feedback as well.
If you find any problems, please open an issue and provide as much information, as possible, to help us identify your problem. Alternatively, feel free to submit a pull-request if you think you can help resolve the problem. If you need a guiding hand, on how to get started with pull-requests, please have a look at our contributing guidelines. Make sure you don't miss the style guidelines, I won't :D

NOTE: Because Nancy 2.0-alpha is a pre-release, you are going to have to install packages using the -pre toggle, i.e install-package Nancy -pre

Please use this issue for posting comments that are not directly related to any specific reported bug or pull-request.
Resources

We have put together a brief (living documentation, will be updated as we move forward) Nancy v2 Upgrade Notes post that highlights some of the things that have changes, along with proposed workarounds
You can find a draft of the new Configuration API documentation here

THANK YOU for helping us with testing 2.0-alpha! Your feedback is going to help us move forward with a stable release

Next week I should have the time to send a PR for #1577 so we can get that in to v2
I've also got a site for my own use that I could try updating to the v2 betas. I've been running it on preview builds all along so if v2 is far enough along now I'll give that a try.

Replaced by #2659


Add Emission Enabled bolean flag to Mint Module

Purpose of Changes and their Description
This change adds a mint module parameter that allows network administrators or the /x/gov governance module to control whether or not any new tokens are minted, basically entirely turning off paying for rewards and creating new tokens / inflation.
Link(s) to Ticket(s) or Issue(s) resolved by this PR
PROTO-3083
Are these changes tested and documented?
New tests added to x/mint/migrations/v5/migrate_test.go and x/mint/module/module_test.go

forgot to add upgrade in app folder, in addition to migration, will do in a bit


CMakeLists.txt is not detected as CMake code

Here's a file that is reported as text although it is clear form the file name that it is a CMake code:
https://github.com/cppformat/cppformat/blob/757a523195325d73eb87469875990dd6ac8f82ff/test/CMakeLists.txt

The last released version of Linguist detects this file as CMake so your statistics are probably just not up-to-date. It looks like the highlighting is though. It should update with the next commit or if you contact GitHub support.
This kind of incorrect detections (when the detection is made based on the filename) should be fixed with #2006.

Thanks for the information. Do you mean it should update with the next commit to this specific file? Doesn't GitHub update language statistics periodically?

Doesn't GitHub update language statistics periodically?

Hi @vitaut - I'm afraid language statistics are only updated after a update is pushed to your repo

@arfon Well, I've just pushed to the master branch of the repo containing the file, but it's still misclassified.

@vitaut - can you tell me what URL you are using to determine that it's being classified as text? As @pchaigno mentioned, the highlighting suggests that we think it's CMake

@arfon I think @vitaut saw that it was classified as text through this page. Maybe I didn't have the last released version when I tested...

Ah ok. The search indexes are sometimes a few minutes behind these updates. https://github.com/cppformat/cppformat/search?l=text doesn't show any results now.

Actually, it does classify it correctly now. Probably needed more time after the push. Thanks!

FWIW I was checking the classification here: https://github.com/cppformat/cppformat/search?l=text

:+1: - glad things are looking better for you now @vitaut


chore(gateway): remove orphaned admin.rs file

This file wasn't included in any module and the last real commit to it was over a year ago.
Looked into some tools to automate detecting this problem and found cargo-modules which does work to catch these, but there are some annoyances like it being a little slow and that packages must be specified manually in a workspace (no --workspace option). There aren't any simple lints available since knowledge about which files are included/excluded is required, which is beyond what tools like clippy can do.
IMO this is a fairly niche problem so pulling in a dependency isn't that worth it, but if you feel otherwise then I can setup a script in lint.yaml
for name in $(cargo metadata --no-deps --format-version 1 | jq -r '.packages[].name');
do
   cargo modules orphans --package $name --cfg-test --deny;
   echo "finished checking: $name";
done;

 sample output

No orphans found.
finished checking: axelar-gas-service

No orphans found.
finished checking: axelar-soroban-interfaces

No orphans found.
finished checking: axelar-soroban-std

1 orphans found:

error: orphaned module `orphan` at contracts/axelar-gateway/src/orphan.rs
  --> contracts/axelar-gateway/src/lib.rs
   |  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ orphan module not loaded from file
   |
 help: consider loading `orphan` from module `axelar_gateway`
   |
   |  mod orphan;
   |  +++++++++++
   |

Error: Found 1 orphans in crate 'axelar_gateway'
finished checking: axelar-gateway

No orphans found.
finished checking: axelar-operators

No orphans found.
finished checking: integration-tests

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 99.09%. Comparing base (6dddb12) to head (79324f8).

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   99.06%   99.09%   +0.03%
==========================================
  Files          28       28
  Lines        2885     2885
==========================================
+ Hits         2858     2859       +1
+ Misses         27       26       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


chemiscope local web app size 0

I have an input file I'm creating with write_input, and I want to use the species of the atoms as a property to control some aspect of visualization, probably size. I add it to the Atoms object with atoms.new_array("Z", atoms.numbers), and then interactive set size as coming from "Z".  However, for Cu Al, Zs 29 and 13, I don't get circles that are off by a factor of ~2, but rather tiny specks, and reasonable size circles.
Is setting size from a property reasonably tested, such that this is surprising? Or is it just something that's not well supported?  Either way I can try to do a more complete bug report.

Hello! It should work pretty well, this is something we do often. See e.g. this to give you an idea of how property value and size should compare

One can never exclude a bug though. If you upload the chemiscope.json file, we can have a look.

Actually, I think I know what your problem is. The size is "rescaled" so that the smallest value is "tiny" and the largest a given "reasonable" size. There isn't really a way to set it to be actually proportional to the value in an "absolute" way. One would basically need to add "range" properties as there are for other indicators such as color and axis range.

Even with size fixed, something weird is going on.  I did symbol by a categorical property (config_type), and the squares (circled in red pen) are much smaller than the circles.  Is this also normal?

So, I tweaked a few things that should now work better, pls have a look. Your second issue looks strange. Here point size, color and z coordinates are all expressions of the same quantity, and all sizes look proportionate independent of the symbol

See if it also breaks with the branch (if you want to run locally you need the whole node.js stack, and launch npm install; npm start from the repo root, then you should be able to open localhost:8080 and see the web interface.

That's more like what I expected.  Let me check the size-cleanup branch and see how it behaves.

size-cleanup definitely looks better with fixed size now - the squares are comparable to the circles, which I like. It also works sensibly with Z for size, as long as Z is a number.  I had a string Z, so I could use it for symbol type, and then it behaved weirdly.  Not sure if it's worth checking for that situation.
Also, the default colormap has an issue - it goes black to yellow, and if I have many spots at the bottom of the range (which is the case if it's a categorical property, like config_type) the black spots merge into a uniform blob, because the outlines are also black.

Excellent, we'll go ahead and merge the PR so you have something that works. I also added a proportional mode which doesn't stretch the range, but is proportional to the actual magnitude.
I'd be tempted to say that the default colormap is a wontfix issue, as you can always pick a different colormap. we could always switch to viridis but it's a bit boring :-)

Thanks.  What about making the marker outlines always be contrasty with the actual color as selected by the colormap for each marker?  This only happens because the marker and the outline are both black.

That requires substantial additional complication and it looks like a sufficiently narrow corner case to warrant that. Maybe one thing that you didn't know: you can select the color palette when you call write_input (the settings option accepts a dictionary with the entire option set), so you can make sure that your .json file will load already with a palette that does not have this problem.
You can experiment in a notebook using chemiscope.show and then only write when you're happy with the visualization. You can also edit the visualization options in the online viewer and then save the .json again, and it'll incorporate the settings.

@bernstei  just to add that if you feel this "contrasty outline" is important for your workflow and you can't fix it with the workarounds we discussed you should open another issue, but it'll be low priority on our side so code contributions will also be much welcome ^_^.

I can definitely live with using a different palette.  If I do feel that it's worth a proper fix, I'll try to see if I can understand the code structure enough to make a PR.


It's weird that I can't run fab deploy from the graphic folder

Can I resolve this with a symlink or something?

I can't figure out any sane way of doing this, sadly. :/


Type safe prepared statements in parser

Proposal
Add syntax to the PromQL language, that allows making type safe prepared statements. I think syntax and actual functionality is heavily discussable so I wouldn't focus on this part just yet (I would rather first come to consensus that this type of feature is something we want rather than the nitty gritty details), but I imagine something along the lines of:
rate(http_requests_total{%selector})
Which reserves the templatable selector in the query's AST, and can be added at a later point. (potentially these could even be named variables, but that could also be a client concern and for the parser it's just a list of variables)
I haven't thought through possible clashes with other PromQL edge cases, obviously the syntax would need to be designed to not interfere with other PromQL features or break them in any way.
Use case. Why is this important?
The obvious use is prevention of typical query injection type attacks, but that's not the only use I see.
PromQL statements are being text-templated all over the place in the ecosystem. A list of examples (things that I know off the top of my head, and work with regularly so I feel the pain a lot):

monitoring mixins
k8s-prometheus-adapter
argocd

Alternatives that have been mentioned when discussing this on IRC was just doing text templating and seeing if the query parses. This is problematic, as in the above example uses if the surrounding PromQL query is incorrect it is only noticed at runtime, even though the failure could have been detected when writing the prepared PromQL statement. Both the query and input variables being unstructured also makes parser errors very difficult to understand as both query, input and or the combination of both could be the culprit of a malformed resulting query.
@slrtbtfs @brian-brazil @SuperQ @bwplotka @csmarchbanks @beorn7 @gouthamve @codesome @juliusv @tomwilkie @krasi-georgiev @simonpasquier

I see the pain in the use case and agree it would be nice to be able to be able to copy-and-paste one of those mixin queries and check that it at least parses correctly with placeholders.
Just a thought: I wonder about the granularity level at which placeholders would be supported. It would seem easiest to make placeholders possible only as stand-ins for any current full PromQL AST nodes (of type string, scalar, vector, matrix). But your example for templating only part of a vector selector node would make things more complex. Then again, this seems like an extremely common use case in the mixins, so it would be good to support somehow.

Is it proposed that such changes would be visible in the Prometheus binary, or just at the code level?

Alternatives that have been mentioned when discussing this on IRC was just doing text templating and seeing if the query parses. This is problematic, as in the above example uses if the surrounding PromQL query is incorrect it is only noticed at runtime, even though the failure could have been detected when writing the prepared PromQL statement.

I'm confused by this claim. If you put in a dummy value you can see if the query parses in advance.
I'm not exactly seeing why a problem that appears solvable entirely within mixins given the existing parser needs fundamental language changes to the user-exposed PromQL.

I'm confused by this claim. If you put in a dummy value you can see if the query parses in advance.

Parser errors would still be confusing, and it doesn't solve the injection case.

Is it proposed that such changes would be visible in the Prometheus binary, or just at the code level?

I can see uses in Prometheus itself as well, but for a start I think just having it in the parser to allow ecosystem integrations would be sufficient to get some experience with it.

I'm not exactly seeing why a problem that appears solvable entirely within mixins given the existing parser needs fundamental language changes to the user-exposed PromQL.

I don't think just because something is difficult or a lot of work, diminishes the value of a feature. Obviously someone must be willing to put in that work (should it be agreed on) and not break any existing features. Also I'd like to see how difficult this would actually be, before claiming something like this. My hope is that with the recent work from @slrtbtfs, this shouldn't be all that wild (I may be completely off though).

But your example for templating only part of a vector selector node would make things more complex. Then again, this seems like an extremely common use case in the mixins, so it would be good to support somehow.

I honestly haven't looked too deep into implementation details yet, I generally agree though. I think we could even just start with what I feel are the two most common things:

Selectors
Groupings

Parser errors would still be confusing,

I'm not following you here, you'd have the usual PromQL errors of the usual standard. Can you provide a concrete example?

it doesn't solve the injection case.

I'm not following you here either, applying string escaping is not difficult.

ecosystem integrations

Hmm, do any the mixins use the parser today? How does jsonnet integrate with Go code?

I don't think just because something is difficult or a lot of work, diminishes the value of a feature.

I never said anything about the level of effort. My point was more why should problems external to Prometheus complicate Prometheus itself, as if the solution to every problem was to change Prometheus itself that's not likely to result in a coherent system.

Regarding the implementation difficulty:
The canonical way to achieve this probably would be to make individual selectors and groupings AST Nodes, too. This was already suggested in #6942 but not deemed necessary at that point in time. Since there would be some nontrivial refactoring across package boundaries and the addition of new node types involved, this probably would be the hardest part.
With that in place the parser could  be extended with a new node type that serves as a placeholder for things that would be inserted into queries. Since AFAIK PromQL doesn't have too much function operator type information can be added to those nodes, that can be used for type verification.

I'm not following you here, you'd have the usual PromQL errors of the usual standard. Can you provide a concrete example?

When having a query like foo{a=%b} and claiming %b would be a label matcher (i.e. of the form label="value",..., what you're suggesting would be parsing something like foo{a=label="value"}. The resulting error message isn't really helpful: 1:7: parse error: unexpected identifier "label" in label matching, expected string.

I'm not following you here either, applying string escaping is not difficult.

When inserting more complex things like whole vector selectors, string escaping alone won't do it.

I never said anything about the level of effort. My point was more why should problems external to Prometheus complicate Prometheus itself, as if the solution to every problem was to change Prometheus itself that's not likely to result in a coherent system.

The debate whether PromQL should be extended by something that can be already done with a template system seems to come up in a new Proposal every month or so and it seems to be more or less the same debate every time. Maybe some more general discussion about whether such features are desirable would be beneficial.

The resulting error message isn't really helpful:

There's two problems here. One is verifying that foo{a=%b} is correct in itself, the other is that %b has an appropriate value.
Tracing the conversation this point is about the former, and substituting in "param_b" to check if the query parses doesn't seem difficult to me and will have the usual error message.
For the latter, the existing matcher parser can check that it's valid, and will have an appropriate error message.
I think a step has been missed in your explanations, is automatic type inference what you're going for or is it all statically typed by the user?

Maybe some more general discussion about whether such features are desirable would be beneficial.

The question isn't whether some people desire them. The question is more what layer should they go in, and in particular if a given feature is appropriate for inclusion in Prometheus itself as a 1st class feature or should be built on top of it.

I think a step has been missed in your explanations, is automatic type inference what you're going for or is it all statically typed by the user?

Both should be possible to implement with similar effort, but it doesn't really make sense to discuss about these details yet.

It makes a big difference, as if you're going with user provided static types then what you want should already be doable with the parser today. Inferring the types from the query on the other hand I would imagine would require some pretty invasive changes.

If we extend the parser to parse place holders as such and do static type checking based on the types the user assigned to these place holders, the step to automatic type inference isn't that big any more.
But getting there in the first place would indeed be quite invasive.

My point above is that you can do static type checking without having to change the parser at all. For a label value (i.e. string) try substituting in a quoted label value to the query and see if it parses. In the other direction try parsing it as a label value.

Isn't the main use case just to be able to directly copy rate(http_requests_total{%selector}[5m]) from one of the mixin files in your code editor and paste it into a PromQL input field that responds with "Yup this parses, but it's incomplete and to evaluate it you'll also need to provide the placeholder values"? Otherwise the editing flow is painful, as you'll have to manually replace all the selectors with something that yields a fully valid PromQL expression.

Collected thoughts:
I have seen (and used) templating of many different parts of a PromQL expression. Some were, in a way, utilizing "injection attacks" by requiring fragments of PromQL to be inserted. Or in other words: In some use cases, the user doesn't even want type safety. (One might argue, that's bad usage, but that gives the discussion a different angle, from "we want to help users" to "we want to change bad behavior of users".)
Even if we assume type safety is desired, then my concern would be about the many many different types possible. A user needed to understand parsing and the AST first to even start using the feature effectively. Then there needed to be some kind of sub-language to specify the type. And even then, after all the efforts put into it by us and the user, some users might be dissatisfied, with being restricted to node types in the AST rather than saying: "This needs to be bottom_k or top_k with a 1st parameter less than 10`. Going the other way and don't let the user specify types at all (I guess that's meant by automatic inference above), we might not gain so much because a non-trivial template is in many cases expandable in a valid way by a non-intended type. I guess it boils down to a powerful typesafe templating system being a very deep rabbit hole…
I'm not quite sure if the prime goal here is checking un-expanded templates ("you want this to be expanded to a range selector, but that doesn't result in valid PromQL") or checking the parameters used for expansion ("this parameter is not a range selector") or both. In any case, this seems to be mostly about meaningful error messages. As already said above, how does that integrate with the templating engine used by the user? To deep-integrate with the editing workflow (I edit, for example, Jsonnet and get error messages as described above in my editor), I guess you needed a specific glue tool that understands both (for example) Jsonnet and PromQL (syntax and AST types). Such a solution would be specific to a single templating engine, but as we see, users use many different templating engines. One step away from it would be what Julius hinted at: copy&pastie an unexpanded PromQL expression into a PromQL input field. But then you still have the problem of different templating syntax from different engines. And that's just the first part (about the un-expanded template), not the second about checking parameters. To work in a generic way with all templating engines, we could come up with our own intermediate templating syntax, which is then the one the user has to create with their templating engine (which resembles SQL prepared statements a bit, see the discussion about this being part of the SQL syntax, but note that SQL prepared statements are just about binding values while we, quite often, want to template PromQL code). But then you are still far away from an integrated editing experience, despite having inflicted the cost of another layer of templating…
Over all, I'm very sceptic about the "return of investment" here. However we turn this, it seems to be a lot of effort, the benefits of which will be hard to make generally usable.

(I guess that's meant by automatic inference above),

Yes, that was my meaning.

To work in a generic way with all templating engines, we could come up with our own intermediate templating syntax, which is then the one the user has to create with their templating engine

Just to clarify, my assumption was that the intention was indeed for the PromQL parser to have its own templating syntax (like the rate(http_requests_total{%selector}[5m]) example), but in your config templating engine you would just write that entire expression out as a constant string and then provide the placeholder values out of band to the query evaluator. For example, in rule definitions this could be another section in parallel to the expression field:
- alert: TheRateIsTooDamnHigh
  expr: rate(http_requests_total{%selector}[5m]) > 10
  placeholder_values:
    selector: 'job="api-server"'

And similar for the HTTP API, where the placeholder values could be passed as separate parameters.

Over all, I'm very sceptic about the "return of investment" here. However we turn this, it seems to be a lot of effort, the benefits of which will be hard to make generally usable.

Yeah, it's starting to feel like that to me as well. On one hand templating selector matchers would be the most important use case, on the other hand just doing that feels "oddly specific" and partial on the implementation side. And allowing placeholders in all arbitrary places would be a lot of complexity.

Sounds like there is rough consensus that this is too high in complexity over the gain.


Add default HTTP User-Agent for specific system

New Pull Request Checklist

[x] I have read and understood the CONTRIBUTING guide

[x] I have read the Documentation

[x] I have searched for a similar pull request in the project and found none

[x] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)

[ ] I have added the required tests to prove the fix/feature I am adding

[ ] I have updated the documentation (if necessary)

[ ] I have run the tests and they pass

[ ] I have run the lint and it passes (pod lib lint)

This merge request fixes / reffers to the following issues: ...
Pull Request Description
Let's tell the server what we are. Server maybe forbidden us if he think it's not the valid request.
Don't needs to reinventing the wheel, borrow snippet from AFNetWorking.
Fixed #2408 .

Codecov Report

Merging #2409 into master will decrease coverage by 0.12%.
The diff coverage is 78.57%.

@@            Coverage Diff             @@
##           master    #2409      +/-   ##
==========================================
- Coverage   75.09%   74.97%   -0.13%
==========================================
  Files          36       36
  Lines        3895     3908      +13
==========================================
+ Hits         2925     2930       +5
- Misses        970      978       +8

Impacted Files
Coverage Δ

SDWebImage/SDWebImageDownloader.m
78.45% <78.57%> (-0.09%)
:arrow_down:

SDWebImage/SDWebImageDownloaderOperation.m
85.43% <0%> (-1.95%)
:arrow_down:

SDWebImage/SDImageCache.m
63.84% <0%> (-0.42%)
:arrow_down:

SDWebImage/SDWebImageManager.m
75.22% <0%> (+1.35%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f18747...c77310b. Read the comment docs.

Use headersFilter in 4.x instead.
For 5.x, use requestModifier instead.
We should not bind complicated business code to a common library. If the user desired to use something, we just provide a abstract protocol or way to let them provide the information.

@dreampiggy It has no any business code, it just add the device and system version. just like all common user agent. you can see UIWebView/WKWebView or Safari.
Like below, you can see how Apple uses the user-agent to identify the device.

@zhongwuzw I agree we need to set the User-Agent. I would also consider the comment from @dreampiggy and actually make use of our existing features to do so.
Just that we need this User-Agent configured properly for all our users. We can't expect them to do it. We only offer ways to alter this behavior, if they need that.

@bpoplauschi the propose @dreampiggy suggested is used for custom for each request, so any opinion about this PR?

I've looked at this PR again. Guys, I think an API like requestModifier is too advanced for the users that simply use our UIImageView (or similar) categories. I see AFNetworking also sets the User-Agent in the same way, so I say we do the same.
This is not a case where some users want some custom behaviour. From my point of view, this is the User-Agent set for each request of all our users, so we need to set it properly.
The only feature I say we could add (but it's just nice to have) is another field in the DownloaderConfig that allows passing in the User-Agent.

@bpoplauschi Emm 🤔 , User-Agent is only a field of Headers, user can override these values by (void)setValue:(nullable NSString *)value forHTTPHeaderField:(nullable NSString *)field of SDWebImageDownloader or requestModifier, I think it's maybe enough. Would you think we must add the BooleaninDownloaderConfig`?

@zhongwuzw Don't need to add one extra config in SDWebImageDownloadConfig or somewhere. If you want to add this, it shuold be one built-in HTTP headers. Use can use current APIs to change or override it. It's not suitable to add any property just for User-Agent this specify headers.

You are right guys, I forgot about setValue:forHTTPHeader:. That is sufficient for the users to customize. So we just need to calculate the User-Agent for the default case, which is most of the users (that will never deal with that). Basically this PR + the minor change request by @dreampiggy

I see no reason to keep this un-merged. Good job @zhongwuzw


Deprecate Basecoat and update to npm-hosted Primer

GitHub's internal basecoat stylesheet framework thing has been deprecated and folded into primer which is now hosted on npm instead of bower. Yayyyy!
https://github.com/primer/primer-css
Would love to get our bower deps out of the mix. @simurai do you have a sense of how easy/hard it would be to upgrade to the new stuff?

Probably a lot has changed, but yeah, would be nice to start using the npm version. I can give it try and see how much it breaks.

Gave it a try, but quite a few things (file names, mixins, variables, etc.) changed.
Chatted with @zeke and we'll keep this on hold until the next redesign comes around.

https://twitter.com/githubprimer/status/995061407581384704
https://github.com/primer/primer-primitives

This is a monorepo for color, spacing, and typography primitives for use with Primer, GitHub's design system.


NFT Notification button appears on sell offer [Bug]

What happened?
Steps to reproduce:

Create an offer to sell an NFT you own
Share the offer on dexie
BUG: The button for "Notify Owner" appears. You are the owner; this button should not appear in this scenario.

Out of curiosity, I notified the owner and indeed received the notification in the same wallet. I accepted the offer and it was successful. So I sold an NFT to myself.
The main consequence of this (beyond end users being confused) is that if the offer is accepted, the royalty will be deducted.
Version
release/1.7.0 branch
What platform are you using?
Windows
What ui mode are you using?
GUI
Relevant log output or stacktrace
No response

This issue has already been fixed and merged.


fix issue #66 intro message not rendered as HTML

fixed #66

Make sure to squash commits before merging or else they won’t appear correctly in the release notes. And also the PR and commit title must be good so the extension users can read it.


Support applying a process to a neighbourhood of pixels

Use case: run a machine learning model that works on 128x128 'patches' or 'tiles' of EO data, and retain only the 64x64  inner pixels as a result. So the resulting datacube is the combination of these 64x64 results, that can be computed in parallel.
Possible solution:
This sounds like an apply_dimension, but:

x and y are separate dimensions, while we want to work on both of them
it is not necessary to work on the 'full' space dimension, this could be very large, we simply want a small window

So:

allow specifying more than one dimension in apply_dimension?
or create a separate function to work on a 'window' of data. Where the extent and overlap of windows needs to be specified for each dimension.

So my window would be:

all bands
64 pixels along x and y, padded with 32 pixels on each side
1 date

Similar function in xarray:
First 'chunk' the dataset:
http://xarray.pydata.org/en/stable/generated/xarray.Dataset.chunk.html
Then apply function to each chunk:
http://xarray.pydata.org/en/stable/generated/xarray.map_blocks.html
But also rolling:
xarray.DataArray.rolling
In fact I believe I need a combination of rolling and chunking in my use cases. Rolling provides the ability to 'pad' data in space and time, chunking allows me to for instance specify that I want to transform 64x64 tiles.
Earth Engine has this creative solution:
ee.Image.neighborhoodToBands
Which would help in some cases: for a time window, we could add the required dates as bands, maybe even in a udf. Then a reduction of the bands dimension would also have temporal info.

Isn't that exactly a use case for apply_kernel? Although I acknowledge that the kernel array is getting pretty large.

It's certainly related to apply_kernel, but that process is restricted to 2D convolutions, while I want to do a more generic transformation. So I would maybe not overload apply_kernel with this behavior.

I think it was supporting your usecase in 0.4, but we restricted it for 1.0 (can't remember why).
I'm still not 100% sure what is missing from apply_kernel so maybe you could propose a draft process in a PR? We could discuss that in the next dev meeting or so. I feel like it should be a separate process to also not overload apply_dimension.

I added a pull request for a 'chunk' process. Advantage is that we do not need to modify/overload existing processes.
Disadvantage is that people may expect chunking to have an effect on certain other processes, that we did not or do not foresee ourselves. So we'll need to try and document this.

I'm just wondering:

Would apply_neighbourhood also need a border parameter like apply_kernel?
We should rename neighbourhood to neighborhood. We use American English in process names AFAIK and removing the u makes the word comply to AE (and easier to write?)

There's something to say for adding a border parameter, although less relevant compared to apply_kernel: this function will mostly be used with UDF's, so it's a lot easier for users to implement their own gap filling scheme. IMO default behaviour should remain to fill gaps with nodata.
Renaming to AE is fine.

So should we just say that missing values at the borders are always filles with nodata/null?
I'll do the rename then.

This has now been merged and is scheduled to be released in 1.0.0.


added mocks for test_geonames

Hi @DenisCarriere ,
I have created a branch to mock the tests with https://pypi.python.org/pypi/requests-mock, and tested the lib on test_geonames.py and it looks straightforward.
If you agree, let's work together on this branch, and let's use this pull request for discussion.
Please have a try with the existing tests and let me know what you think about it. Meanwhile, I will play a bit on the support for multiple results.
Cheers
Manu

Good suggestion for test data. Applied.
I have also added a few mocks and fix the issue that was breaking the build for Google.
Oddly enough, I do not succeed in mocking all google calls...

google_reverse does not recognize the mocked url
google_timezone does not recognize the json data

I think we reach a state where you can merge the PR :)
Il the build pass on master, I reckon you could also push a new version on pypi :)

👍 Awesome

Il the build passes on master, I reckon you could also push a new version on pypi :)

You've been pushing quite a few commits, let me know whenever you're ready for a new version on PyPi.

he he :)
and the build passes now 👍
you can push to pypi, indeed ! I will stop for this week and add a bit of documentation on next week-end I think

haha awesome! 👍 I'll publish a new PyPi version

Ok i've published v1.24.0, I updated the Makefile with the appropriate commands.
$ python setup.py sdist upload -r pypi
$ python setup.py bdist_wheel upload -r pypi

Also you need to add your PyPi credentials ~/.pypirc.
[distutils]
index-servers =
  pypi

[pypi]
repository: https://upload.pypi.org/legacy/
username: <USERNAME>
password: <PASSWORD>


JavaScript tooltips

Hi,
Is the experimental JavaScript support supposed to work with tooltips? If not, just filing as a request.
If so, I'm not sure what I'm doing wrong as not getting any (e.g., hovering over the call to a locally imported function).

Remember that atom-ts only activates on TypeScript grammar by default. See https://github.com/TypeStrong/atom-typescript/blob/master/docs/faq.md#i-want-to-use-atom-typescript-with-javascript-too, or manually run typescript:activate command from the command palette.

Yes, thanks, I did that too... But not working...

Maybe I've disabled something somewhere? For atom-ide-datatip, I have "Show datatip automatically on 'mouse' hover" checked.

Not sure what's going wrong there. It's not that easy to disable tooltips actually. Perhaps you're using a non-standard js grammar or file extension? Check javascript file extensions and javascript syntax scopes in Atom-TypeScript settings. To get editor's syntax scope, run editor:log-cursor-scope in the editor with a JS file open, you're interested in the first scope shown.
If that doesn't help, make sure you're running the latest Atom version (1.55.0 at the time of writing). Try checking developer console (window:toggle-dev-tools, then open console tab) for relevant error messages (be warned, there almost definitely will be some irrelevant ones).
If the issue persists with no obvious cause, you can run window.atom_typescript_debug = 1 in the developer console, that will enable verbose logging of communications with tsserver -- for the purpose of datatips you're interested in seeing sending request { ..., command: "quickinfo", ...} and (perhaps more importantly) received response for quickinfo in ... ms with data { ... }. At least with this we'll know where the issue originates:

If no new log messages of either sending... or recieved ... appear in the dev console after enabling atom_typescript_debug and messing about in the editor (hovering mouse, opening/closing text editors, etc), this means atom-ts didn't actually activate, or is failing to recognize the current editor as relevant (most probably due to unexpected file extension or unknown grammar name).
If there are no sending request { ..., command: "quickinfo", ...}, that means datatip isn't initiated.
If there are no  received response for quickinfo ...  that means tsserver fails to find anything relevant
If both are there, it means that atom-ts does what it's supposed to, but tooltips are somehow suppressed.

Ah, fixed it, thanks so much... Your suggestions prompted me to look in the config file for my extensions, and in core -> customFileTypes -> source.js I did not have "js" and adding it brought tooltips! Wonderful!
However, I don't think this has caused for other programs, so don't know if you want to add that one by default. But this is great, thanks!


google_authenticator

2 step verification not working on ly .
Is there any hope we can use auth required pam_google_authenticator.so in /etc/pam.d/ly?

@mshzsh Could you please detail a bit more? Like, how doesn't it work?


When is the next point release?

Last one was in November :disappointed:

@Unknwon please build and push a new release so normies like me can run gogs easily

Hi,

This is not the place to ask question as stated very clear in the issue template.
I plan to release new version on 3/31 EDT.


[Enhancement] Be node memory manage need to optimize

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
My doris version:1.1.1
problem:
When be running a few weeks laster.Spend more and more memory, and never release it.
be节点占用内存不释放，逐渐攀升，直到达到限制。导致集群响应变慢，最终无法对外提供服务！
Solution
reboot be node one by one.
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

No one answer.Closed.


Create test inputs to RNA-Seq pipeline

From @jvivian on August 10, 2015 23:26
null
Copied from original issue: BD2KGenomics/toil#225

This was sort of a failure as I was able to find and create small fastqs but one step in the pipeline now takes 300x longer than it used to on these small files which is rather unintuitive.   Right now the smallest zipped sample we have is 7.5G and I'm timing a run on a 32 core machine.