Add new model function to print device model name

This PR adds a new function which will be enabled by default. The function displays the product/model name of your device if it exists. For example on my laptop it shows Lenovo YOGA 900-13ISK and my iPhone it shows iPhone 5s.
Note: This feature won't work on custom built PCs or devices without OEM info.
TODO:

[x] Add support

[x] Add support for Linux

Not sure if this works on every machine yet.
The file we check for the info was added in linux 2.6.26 ~2009<br >
so this should be fine.

[x] Add support for MacOS (OS X)
[x] Add support for iOS
[x] Add support for BSD
[x] Add support for Solaris
[x] Add support for Windows

[x] Testing

[x] Test the feature on Linux
[x] Test the feature on MacOS (OS X)
[x] Test the feature on iOS
[x] Test the feature on BSD
[x] Test the feature on Solaris
[x] Test the feature on Windows

Testing:
If you'd like to help me test this is what you need to do:

Git clone neofetch.

git clone https://github.com/dylanaraps/neofetch

cd to the new directory.

cd neofetch

Swap to the model branch.

git checkout model

Run neofetch without a config from the directory.

./neofetch --config off

Let me know if the feature works/doesn't work on your machine.
Screenshots are nice too. :)

Old PR #308

I've added support for Windows. I don't have any prebuilt PCs at home so I'm
not 100% sure that this works without issues. If someone could test this for me
I'd be very thankful. :)

On BSD systems we can use dmidecode to get the information, the only issue
is that the command requires root privileges. I'm trying to find an alternative
command/program to use.

Works as expected in my Arch VM, model prints Model: VirtualBox 1.2

This is ready to merge,  I would've loved to do some more testing but I don't own anymore OEM devices.

Okay, i've done some research on RPi2 and my router. Both seems to not have these directories which neofetch is using.

RPi2 with OSMC (debian jessie)
/sys/firmware/devicetree/base/model

correctly showing Raspberry Pi 2B...

TP-LINK with Gargoyle (openwrt aa)
/tmp/sysinfo/model

correct TP-Link TL-WR740N/ND v4

Thanks for testing!
What I'll do is add fallbacks to these directories in another PR. :+1:


Push to registry with HTTP

Hi
My internal registry has no HTTPS connection and I am forced to use HTTP.
However, seems that JIB is trying to use onlyHTTPS so the push command is failing.
Is there a way to configure JIB to force the use of HTTP instead of HTTPS?
Thanks in advance

OK, I will answer to myself ...use  allowInsecureRegistries.
I am sorry.


MQTT Auto Discovery not available after restart of client

After an MQTT-AD client is restarted the ESPSomfy-RTS stays uncontrollable until a reboot of ESPSomfy.
This means that every time you need to completely restart ESPSomfy after an update of the MQTT client software.
Other solutions like Zigbee2mqtt or ZwaveJSUImqtt don't need that they are available after just a few seconds after restart.
Probably this has to do with those tools continuously sending data out to MQTT.
Is at possible to have ESPSomfy send a discovery/status packet or something every minute when the Publish Discovery box is ticked?

I'll look at what I can do.

In v2.3.1 the software will persist the disco topics and drop them when the reference is removed.  This should allow the subscriber to catch these whenever they subscribe.

Hi @rstrouse,
I've updated to 2.3.1-pre. But this didn't help. The same effect occurs.

Install Main.  I haven't had time to rebuild the pre-release yet with the new commit.

Installed Main and the problem is solved! Thanks 🎉


SSL handshake failure in OpenSSL

Lately, I haven't been able to make any connections to DeviantArt from my code, or even my terminal.  I attempted a connection the 25th for final testing before deploying a new version of my web application, but now I'm getting this error:
>_ openssl s_client -connect www.deviantart.com:443 -tls1_2
CONNECTED(00000003)
140735219040336:error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:s3_pkt.c:1472:SSL alert number 40
140735219040336:error:1409E0E5:SSL routines:ssl3_write_bytes:ssl handshake failure:s3_pkt.c:656:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time:<PHONE_NUMBER>
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

I've tried from several computers and networks.  I've removed the -tls1_2 and added back in.  I've tried via Python code as well.
Is there something that has been changed recently?

New information: I added -servername www.deviantart.com to the command line, which gives me the following:
CONNECTED(00000003)
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert SHA2 High Assurance Server CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/C=US/ST=California/L=Los Angeles/O=deviantART, Inc./CN=*.deviantart.com
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
 1 s:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
---
Server certificate
[REDACTED]
---
No client certificate CA names sent
---
SSL handshake has read 3255 bytes and written 448 bytes
---
New, TLSv1/SSLv3, Cipher is [REDACTED]
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : [REDACTED]
    Session-ID: [REDACTED]
    Session-ID-ctx:
    Master-Key: [REDACTED]
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 10800 (seconds)
    TLS session ticket: [REDACTED]
    Start Time:<PHONE_NUMBER>
    Timeout   : 7200 (sec)
    Verify return code: 20 (unable to get local issuer certificate)
---

This could be caused by some migrations we were doing recently, it should be working now.


Uncaught (in promise) Error: Could not establish connection. Receiving end does not exist.

In the bottom of the close_caption_extraction.js file there is a function
      type: "CURRENT_CHAT",
      data: { chat, meetId },
    });

This sends a message via the chrome extension runtime. This message is supposed to be read by the background.js chrome.runtime.onMessage.addListener(async (request, sender, ... function. However, if the extension popup is not currently focused, this event listener isn't active and that is why the error gets thrown.
I tried a few ways to prevent the message from being sent if the extension is currently open but didn't figure it out exactly. I looked here at the docs  and was trying to implement a long lived connection.

(For the time being I implemented a catch for the instance where the response is undefined)
For some reason, even with your advice to lose focus of the current popup, I was not able to reproduce the issue.
Although I can see how this could happen as a result of focusing away from the current popup, I believe a likelier cause is that there is an out of bounds index passed from the popup here https://github.com/CakeCrusher/tech-int-cheat/blob/adf6e5b19b80f2fdc1b54e86207ea7f5b3d806cb/popup/popup.js#L59-L68
to the background resulting in a invalid body: JSON.stringify(preprocessedChatSlice), leading to an error in the backend.
If you could make a video of how you managed to reproduce it that would be awesome. Cuz at the moment I'm bewildered.


Bridge Structural Pattern for Intervention Objects in Intervention.py

What my code does?
This new code applies bridge design patterns to intervention.py. The intervention class is responsible for delegating various interventions that can affect the outcome of a simulation to be run for a particular population size. Due to the massive monolithic inheritance structure in the Intervention class separating abstraction and implementation in their respective interfaces helps in organizing and executing code more efficiently.
What my code looks like?
There are namely two parts to the bridge pattern, Class Intervention  allows for the abstraction of intervention object declarations whereas the class vaccine allows in the declaration of Vaccine interventions. This splits the object handling into a high-end abstraction in the form of Intervention and low-end implementation in the case of Vaccine.
Why is this a good idea?
The builder pattern lets us divide and organize a monolithic class with variants of functionality. Intervention class has grown and become too large as shown by the number of subclasses that inherit from Intervention. The examples shown in the lecture slides and that on the internet resembled the structure of the Intervention.py a lot in the case where the Bridge pattern is usually applied.

Thanks for the PR -- we're not considering major architectural changes currently, but we appreciate the contribution to open-source software!


DOCKER: Invalid block: receipts root mismatch @v21.7.3

Steps to Reproduce (Bug)

Have a running working v21.7.2 version in docker
Upgrade to v21.7.3

Expected behavior:
besu_1           | 2021-09-05 15:53:04.298+00:00 | nioEventLoopGroup-3-9 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:53:09.302+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:53:14.304+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:53:19.305+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:53:19.529+00:00 | nioEventLoopGroup-3-5 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:53:24.061+00:00 | nioEventLoopGroup-3-1 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 3
besu_1           | 2021-09-05 15:53:29.062+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:53:34.305+00:00 | nioEventLoopGroup-3-10 | INFO  | SyncTargetManager | Found common ancestor with peer Peer 0xce7cbef463c5bac7e3... at block 13470991
besu_1           | 2021-09-05 15:53:48.932+00:00 | EthScheduler-Services-5 (importBlock) | INFO  | FullImportBlockStep | Import reached block 13471000 (0x7694d01aa8e2ad5cff6e4f30dd882782e7f23f5e346066dab4a26108ccfaaf46), - Mg/s, Peers: 1
besu_1           | 2021-09-05 15:53:56.482+00:00 | EthScheduler-Services-5 (importBlock) | INFO  | FullImportBlockStep | Import reached block 13471200 (0x8a32ed0c9debb222e95f43e72bdd31266c010d68a77c7ac639f8d1cfd4fb6fbb), 4.260 Mg/s, Peers: 1
besu_1           | 2021-09-05 15:54:04.692+00:00 | EthScheduler-Services-5 (importBlock) | INFO  | FullImportBlockStep | Import reached block 13471400 (0x64d03b0d0b6b63de54d7031fbf4fb511fdbcf4c6442b5d14db77aa2224d6418a), 6.134 Mg/s, Peers: 1
besu_1           | 2021-09-05 15:54:09.825+00:00 | EthScheduler-Services-5 (importBlock) | INFO  | FullImportBlockStep | Import reached block 13471600 (0xe301e740821d93ff3541d64b0c4a40628bf289610ac2db5f083d697d555d5d30), 5.073 Mg/s, Peers: 2
besu_1           | 2021-09-05 15:54:14.820+00:00 | EthScheduler-Services-5 (importBlock) | INFO  | FullImportBlockStep | Import reached block 13471800 (0x0fee8e1b95da9ce306c496659a3b8e695568ebced0ff45ce6430fd80d0243e1d), 6.821 Mg/s, Peers: 1
besu_1           | 2021-09-05 15:54:18.125+00:00 | nioEventLoopGroup-3-10 | INFO  | SyncTargetManager | Found common ancestor with peer Peer 0xce7cbef463c5bac7e3... at block 13471861
besu_1           | 2021-09-05 15:54:35.240+00:00 | EthScheduler-Workers-0 | INFO  | PersistBlockTask | Imported #13,471,866 / 9 tx / 0 om / 189,000 (1.9%) gas / (0x816bf44378d63bfd0f73a1327095f7789a35b6cb8239d030fb636e63a28e315c) in 0.027s. Peers: 1
besu_1           | 2021-09-05 15:54:36.928+00:00 | EthScheduler-Workers-0 | INFO  | PersistBlockTask | Imported #13,471,867 / 0 tx / 0 om / 0 (0.0%) gas / (0x6403bdf63cc05189c587cf1ceeb47d47d2fdb5b443acdb8283c4022c8ba84100) in 0.002s. Peers: 2
besu_1           | 2021-09-05 15:55:47.821+00:00 | EthScheduler-Workers-2 | INFO  | PersistBlockTask | Imported #13,471,868 / 22 tx / 0 om / 521,105 (5.2%) gas / (0x84a8a8b67e80453c1d02a55970d1751120cae7eb6d783160fb5bc0777b223053) in 0.094s. Peers: 1
besu_1           | 2021-09-05 15:56:03.608+00:00 | EthScheduler-Workers-0 | INFO  | PersistBlockTask | Imported #13,471,869 / 4 tx / 0 om / 85,012 (0.9%) gas / (0xf750ebc80b973a300aa2c28483814784bb8f4b75c379273b6b6da777ddb8366a) in 0.024s. Peers: 1
besu_1           | 2021-09-05 15:56:19.157+00:00 | EthScheduler-Workers-2 | INFO  | PersistBlockTask | Imported #13,471,870 / 8 tx / 0 om / 168,000 (1.7%) gas / (0x7b29db5f5af7077116585f4cce8a6795689e70432f73b20c3b0ebea4e50f268e) in 0.033s. Peers: 1
besu_1           | 2021-09-05 15:56:21.369+00:00 | EthScheduler-Workers-3 | INFO  | PersistBlockTask | Imported #13,471,871 / 0 tx / 0 om / 0 (0.0%) gas / (0x429dabc418db6c41659d432d9b56eb0fceda52bbcb8c0ed9302ef75bf9bb46ec) in 0.003s. Peers: 1```

**Actual behavior:**

```besu_1           | 2021-09-05 15:48:50.636+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:48:55.649+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:00.661+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:05.663+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:10.667+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:15.668+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:49:20.477+00:00 | nioEventLoopGroup-3-6 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:25.478+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:49:30.481+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:49:35.483+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:49:40.484+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:49:45.486+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:49:45.795+00:00 | nioEventLoopGroup-3-6 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:50.796+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:49:55.797+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:50:00.799+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:50:05.801+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:50:10.803+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:50:15.847+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:50:21.420+00:00 | nioEventLoopGroup-3-6 | INFO  | SyncTargetManager | Found common ancestor with peer Peer 0x2acbfae44f20064f9c... at block 13470991
besu_1           | 2021-09-05 15:50:23.999+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | MainnetBlockBodyValidator | Invalid block: receipts root mismatch (expected=0x7b9abd93ad0004e42a3293e7c84d02c8d034a5013b53297b724d8ba755cda188, actual=0x53022f47a30c7bf359b3a3c309a233376959d35943db23f346dcc7a83fb0e4d1)
besu_1           | 2021-09-05 15:50:24.003+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | PipelineChainDownloader | Invalid block detected.  Disconnecting from sync target. Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           | 2021-09-05 15:50:24.012+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | PipelineChainDownloader | Chain download failed. Restarting after short delay.
besu_1           | java.util.concurrent.CompletionException: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.encodeRelay(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.completeRelay(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture$UniRelay.tryFire(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.abort(Pipeline.java:180)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:159)
besu_1           |      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
besu_1           |      at java.base/java.lang.Thread.run(Unknown Source)
besu_1           | Caused by: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           |      at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:55)
besu_1           |      at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:31)
besu_1           |      at org.hyperledger.besu.services.pipeline.CompleterStage.run(CompleterStage.java:37)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:152)
besu_1           |      ... 5 more
besu_1           | 2021-09-05 15:50:26.018+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:50:29.101+00:00 | nioEventLoopGroup-3-9 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:50:34.105+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:50:34.293+00:00 | Thread-1 | INFO  | DefaultSynchronizer | Stopping synchronizer
besu_1           | 2021-09-05 15:50:34.295+00:00 | Thread-1 | INFO  | NetworkRunner | Stopping Network.
besu_1           | 2021-09-05 15:50:34.325+00:00 | Thread-1 | INFO  | EthProtocolManager | Stopping eth Subprotocol.
besu_1           | 2021-09-05 15:50:34.425+00:00 | Thread-1 | INFO  | EthProtocolManager | eth Subprotocol stopped.
besu_1           | 2021-09-05 15:50:34.426+00:00 | Thread-1 | INFO  | NetworkRunner | Network stopped.
besu_1           | 2021-09-05 15:50:34.426+00:00 | Thread-1 | INFO  | AutoTransactionLogBloomCachingService | Shutting down Auto transaction logs caching service.
besu_1           | 2021-09-05 15:50:34.433+00:00 | Thread-1 | INFO  | DockerNatManager | Stopping docker NAT manager.
besu_1           | 2021-09-05 15:51:07.575+00:00 | main | INFO  | Besu | Using LibEthPairings native alt bn128
besu_1           | 2021-09-05 15:51:07.582+00:00 | main | INFO  | Besu | Using the native implementation of the signature algorithm
besu_1           | 2021-09-05 15:51:07.593+00:00 | main | INFO  | Besu | Starting Besu version: besu/v21.7.3/linux-x86_64/adoptopenjdk-java-11
besu_1           | 2021-09-05 15:51:08.195+00:00 | main | INFO  | Besu | Static Nodes file = /var/lib/besu/static-nodes.json
besu_1           | 2021-09-05 15:51:08.208+00:00 | main | INFO  | StaticNodesParser | StaticNodes file /var/lib/besu/static-nodes.json does not exist, no static connections will be created.
besu_1           | 2021-09-05 15:51:08.209+00:00 | main | INFO  | Besu | Connecting to 0 static nodes.
besu_1           | 2021-09-05 15:51:08.217+00:00 | main | INFO  | Besu | Security Module: localfile
besu_1           | 2021-09-05 15:51:08.255+00:00 | main | INFO  | DatabaseMetadata | Lookup database metadata file in data directory: /var/lib/besu
besu_1           | 2021-09-05 15:51:08.481+00:00 | main | INFO  | RocksDBKeyValueStorageFactory | Existing database detected at /var/lib/besu. Version 1
besu_1           | 2021-09-05 15:51:10.548+00:00 | main | INFO  | KeyPairUtil | Loaded public key 0xc029faa1d8eb410685f1eb0b9bc53e9e330ecffae9c40effd9b6297964a22c579a79c06e1b9ce3c0d57808c295fd34a5675c94d525dc46e8d0a4f78e7ac671ed from /var/lib/besu/key
besu_1           | 2021-09-05 15:51:11.108+00:00 | main | INFO  | ProtocolScheduleBuilder | Protocol schedule created with milestones: [Frontier: 0, Homestead: 1150000, ClassicRecoveryInit: 1920000, Homestead: 1920001, TangerineWhistle: 2500000, DieHard: 3000000, Gotham: 5000000, DefuseDifficultyBomb: 5900000, Atlantis: 8772000, Agharta: 9573000, Phoenix: 10500839, Thanos: 11700000, Magneto: 13189133]
besu_1           | 2021-09-05 15:51:12.768+00:00 | main | INFO  | FastDownloaderFactory | Fast sync was requested, but cannot be enabled because the local blockchain is not empty.
besu_1           | 2021-09-05 15:51:12.803+00:00 | main | INFO  | RunnerBuilder | Detecting NAT service.
besu_1           | 2021-09-05 15:51:13.296+00:00 | main | INFO  | Runner | Starting Ethereum main loop ...
besu_1           | 2021-09-05 15:51:13.298+00:00 | main | INFO  | MetricsHttpService | Starting metrics http service on <IP_ADDRESS>:9545
besu_1           | 2021-09-05 15:51:13.652+00:00 | main | INFO  | DockerNatManager | Starting docker NAT manager.
besu_1           | 2021-09-05 15:51:13.652+00:00 | vert.x-eventloop-thread-2 | INFO  | MetricsHttpService | Metrics service started and listening on 9545:9545
besu_1           | 2021-09-05 15:51:13.743+00:00 | main | INFO  | NetworkRunner | Starting Network.
besu_1           | 2021-09-05 15:51:13.797+00:00 | main | INFO  | RlpxAgent | P2P RLPx agent started and listening on /<IP_ADDRESS>:30303.
besu_1           | 2021-09-05 15:51:13.799+00:00 | main | INFO  | PeerDiscoveryAgent | Starting peer discovery agent on host=<IP_ADDRESS>, port=30303
besu_1           | 2021-09-05 15:51:13.877+00:00 | vert.x-eventloop-thread-3 | INFO  | VertxPeerDiscoveryAgent | Started peer discovery agent successfully, on effective host=<IP_ADDRESS> and port=30303
besu_1           | 2021-09-05 15:51:13.882+00:00 | vert.x-eventloop-thread-3 | INFO  | PeerDiscoveryAgent | P2P peer discovery agent started and listening on /<IP_ADDRESS>:30303
besu_1           | 2021-09-05 15:51:14.033+00:00 | vert.x-eventloop-thread-3 | INFO  | PeerDiscoveryAgent | Writing node record to disk. NodeRecord{seq=70, publicKey=0x03c029faa1d8eb410685f1eb0b9bc53e9e330ecffae9c40effd9b6297964a22c57, udpAddress=Optional[/<IP_ADDRESS>:30303], tcpAddress=Optional[/<IP_ADDRESS>:30303], asBase64=-Je4QEsqku572BXquNXUkhN-G05ikmUGbQnz3iZfCi_dy9-KKAs1IGmEoW7hUL28FsLkprJ1JjR0H808fUcyfhnQ_c5Gg2V0aMfGhA9r8YeAgmlkgnY0gmlwhH8AAAGJc2VjcDI1NmsxoQPAKfqh2OtBBoXx6wubxT6eMw7P-unEDv_Ztil5ZKIsV4N0Y3CCdl-DdWRwgnZf, nodeId=0xee4f90949522ca8ddf6dbe7e2cbb13b80b2908d35bd245f62285af040709f82b, customFields={tcp=30303, udp=30303, ip=0x7f000001, eth=[[0x0f6bf187, 0x]], id=V4, secp256k1=0x03c029faa1d8eb410685f1eb0b9bc53e9e330ecffae9c40effd9b6297964a22c57}}
besu_1           | SLF4J: Class path contains multiple SLF4J bindings.
besu_1           | SLF4J: Found binding in [jar:file:/opt/besu/lib/log4j-slf4j-impl-2.14.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
besu_1           | SLF4J: Found binding in [jar:file:/opt/besu/lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
besu_1           | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
besu_1           | SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
besu_1           | 2021-09-05 15:51:14.326+00:00 | main | INFO  | DefaultP2PNetwork | Enode URL enode://c029faa1d8eb410685f1eb0b9bc53e9e330ecffae9c40effd9b6297964a22c579a79c06e1b9ce3c0d57808c295fd34a5675c94d525dc46e8d0a4f78e7ac671ed@<IP_ADDRESS>:30303
besu_1           | 2021-09-05 15:51:14.329+00:00 | main | INFO  | DefaultP2PNetwork | Node address 0x2cbb13b80b2908d35bd245f62285af040709f82b
besu_1           | 2021-09-05 15:51:14.338+00:00 | main | INFO  | DefaultSynchronizer | Starting synchronizer.
besu_1           | 2021-09-05 15:51:14.373+00:00 | main | INFO  | FullSyncDownloader | Starting full sync.
besu_1           | 2021-09-05 15:51:14.388+00:00 | main | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:51:14.436+00:00 | main | INFO  | JsonRpcHttpService | Starting JSON-RPC service on <IP_ADDRESS>:8545
besu_1           | 2021-09-05 15:51:14.507+00:00 | vert.x-eventloop-thread-7 | INFO  | JsonRpcHttpService | JSON-RPC service started and listening on <IP_ADDRESS>:8545
besu_1           | 2021-09-05 15:51:14.515+00:00 | main | INFO  | Runner | Ethereum main loop is up.
besu_1           | 2021-09-05 15:51:14.525+00:00 | main | INFO  | AutoTransactionLogBloomCachingService | Starting auto transaction log bloom caching service.
besu_1           | 2021-09-05 15:51:14.532+00:00 | main | INFO  | LogBloomCacheMetadata | Lookup cache metadata file in data directory: /var/lib/besu/caches
besu_1           | 2021-09-05 15:51:16.242+00:00 | nioEventLoopGroup-3-3 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:51:21.246+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:51:21.830+00:00 | nioEventLoopGroup-3-7 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:51:26.831+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:51:31.833+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:51:36.835+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
besu_1           | 2021-09-05 15:51:41.839+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:51:44.547+00:00 | nioEventLoopGroup-3-3 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:51:48.720+00:00 | nioEventLoopGroup-3-8 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:51:53.724+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:51:58.725+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:52:03.726+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:52:06.747+00:00 | nioEventLoopGroup-3-6 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 2
besu_1           | 2021-09-05 15:52:11.753+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 1
besu_1           | 2021-09-05 15:52:14.868+00:00 | nioEventLoopGroup-3-7 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 3
besu_1           | 2021-09-05 15:52:20.036+00:00 | nioEventLoopGroup-3-8 | INFO  | SyncTargetManager | Found common ancestor with peer Peer 0x1abea5c06b5f22e9f0... at block 13470991
besu_1           | 2021-09-05 15:52:33.755+00:00 | EthScheduler-Services-5 (importBlock) | WARN  | MainnetBlockBodyValidator | Invalid block: receipts root mismatch (expected=0x7b9abd93ad0004e42a3293e7c84d02c8d034a5013b53297b724d8ba755cda188, actual=0x53022f47a30c7bf359b3a3c309a233376959d35943db23f346dcc7a83fb0e4d1)
besu_1           | 2021-09-05 15:52:33.819+00:00 | EthScheduler-Services-5 (importBlock) | WARN  | PipelineChainDownloader | Invalid block detected.  Disconnecting from sync target. Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           | 2021-09-05 15:52:33.830+00:00 | EthScheduler-Services-5 (importBlock) | WARN  | PipelineChainDownloader | Chain download failed. Restarting after short delay.
besu_1           | java.util.concurrent.CompletionException: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.encodeRelay(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.completeRelay(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture$UniRelay.tryFire(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.abort(Pipeline.java:180)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:159)
besu_1           |      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
besu_1           |      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
besu_1           |      at java.base/java.lang.Thread.run(Unknown Source)
besu_1           | Caused by: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #13470992 (0x7ddba3f3ea2bee8aa42088d370326fdbc017be3fa2448a4070eb5d7b50b261ba)
besu_1           |      at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:55)
besu_1           |      at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:31)
besu_1           |      at org.hyperledger.besu.services.pipeline.CompleterStage.run(CompleterStage.java:37)
besu_1           |      at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:152)
besu_1           |      ... 5 more

Frequency: 100%
Versions

Docker Version: Docker Latest (v21.7.3)

Hey @mrmariomichel, thanks for raising an issue! At a super quick glance this looks like it could be related to the ongoing network issues on ETC but I'll refer to someone else to weigh in

@vmichalik but with version 21.7.2 all works just fine, really because of the current issues caused by old versions of coregeth ?

FWIW there is no issue on the Besu side here a priori. You just connected to a peer that fed you a bad block. That can come from a few things, but it's possible you're getting that block from a core-geth node in the minority. Your client is ignoring the block and disconnecting from the node.
You may have experienced this sporadically because peer connections are random, based on discovery. So switching versions and restarting for example is not enough, you may well just be connecting to different peers.

I'm consistently getting this issue with 21.7.3 and 21.7.2, but not 21.7.1.
Weirdly enough my node rejects valid blocks (block hash matches public block explorers) with receipts root mismatch.
Example logs for besu running on kotti ETC testnet:
2021-09-07 15:38:33.997+00:00 | nioEventLoopGroup-3-3 | INFO  | SyncTargetManager | Found common ancestor with peer Peer 0x80921d72551f21ef43... at block 4805336
2021-09-07 15:38:34.503+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | MainnetBlockBodyValidator | Invalid block: receipts root mismatch (expected=0xd90fa90eee8649f1f04c79af8c9f0f5dec28714e6d98f0ee553fd3d44285a9d9, actual=0x66839da154ae542cbe2b68cac079cc354b0d838ce2dc6ac6bbbc2ae0bcce031f)
2021-09-07 15:38:34.512+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | PipelineChainDownloader | Invalid block detected.  Disconnecting from sync target. Failed to import block: Invalid block at #4805337 (0x0f7989f2e2f4d4f5ca17c599a17a992c36a030584a4cac5fc7db5459b225a670)
2021-09-07 15:38:34.517+00:00 | EthScheduler-Services-11 (importBlock) | WARN  | PipelineChainDownloader | Chain download failed. Restarting after short delay.
java.util.concurrent.CompletionException: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #4805337 (0x0f7989f2e2f4d4f5ca17c599a17a992c36a030584a4cac5fc7db5459b225a670)
	at java.base/java.util.concurrent.CompletableFuture.encodeRelay(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.completeRelay(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture$UniRelay.tryFire(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source)
	at org.hyperledger.besu.services.pipeline.Pipeline.abort(Pipeline.java:180)
	at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:159)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Failed to import block: Invalid block at #4805337 (0x0f7989f2e2f4d4f5ca17c599a17a992c36a030584a4cac5fc7db5459b225a670)
	at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:55)
	at org.hyperledger.besu.ethereum.eth.sync.fullsync.FullImportBlockStep.accept(FullImportBlockStep.java:31)
	at org.hyperledger.besu.services.pipeline.CompleterStage.run(CompleterStage.java:37)
	at org.hyperledger.besu.services.pipeline.Pipeline.lambda$runWithErrorHandling$3(Pipeline.java:152)
	... 5 more
2021-09-07 15:38:36.520+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0
2021-09-07 15:38:41.521+00:00 | EthScheduler-Timer-0 | INFO  | FullSyncTargetManager | No sync target, waiting for peers: 0

It seems block hash is on the canonical chain when you check it in a public Kotti explorer (assuming blockscout is on the canonical chain):
https://blockscout.com/etc/kotti/blocks/4805337/transactions
I don't know how the receipts roots don't match even though the block hash is correct. It smells like a regression, given it doesn't seem to happen with 21.7.1.

Furthermore, this bug seems to cause besu to reject all valid peers in general. Logs No sync target, waiting for peers forever after encountering the "bad" (but actually good) block.


Tchain integration as mainnet chain

Why this should be merged
How this works
How this was tested

T-Chain based on HyperSDK was only a POC. Therefore this PR is not relevant any more.


How to skip loading spinner for one api call?

Am not able to skip spinner when one api call for background running script. Purpose is am using mail box in our project so am calling api every 2 minutes to get update message count.
The problem is every time this spinner is coming so wanted to hide or skip. Any help would be appreciated.

$http.get('url/to/script.file', {
	ignoreLoadingBar: true
}).then(function(response) {
	// Do something
});

I tried this and its working fine but there are two

Loading bar
Loading spinner

@JeffreyRoosendaal I need only for spinner..

@raviji try this
module.config(['cfpLoadingBarProvider', function(cfpLoadingBarProvider) {
    cfpLoadingBarProvider.includeBar = false;
  }]);

@Josephdias92 its now not working bro!

Looks like this has been answered correctly.  If there are specific implementation issues, feel free to open a new bug report with a reduced test case showing it does not work as intended, thanks!


PyTorch Implementation

Hi @lhao499 ,
I am working through a rewrite of your great work in PyTorch. Please excuse my ignorance as my understanding of JAX can be lacking in certain aspects. Below are the functions and classes for Blockwise Compute Feedforward Network, Blockwise Cross Entropy Loss, and Blockwise LM Head:
Blockwise Compute Feedforward Network
class FFN(nn.Module):
    def __init__(
        self,
        dim,
        hidden_dim,
        dropout,
        act=nn.GELU
    ):
        super().__init__()
        self.fc_in = nn.Linear(dim, hidden_dim, bias=True)
        self.fc_out = nn.Linear(hidden_dim, dim, bias=True)
        self.act = act()
        self.resid_dropout = nn.Dropout(dropout)
        self.ln_2 = nn.LayerNorm(dim)

    def forward(self, hidden_states):
        hidden_states = self.ln_2(hidden_states)
        hidden_states = self.fc_in(hidden_states)
        hidden_states = self.act(hidden_states)
        hidden_states = self.fc_out(hidden_states)
        hidden_states = self.resid_dropout(hidden_states)
        return hidden_states

def blockwise_compute_ffn(cell, inputs, chunk_size):
    inputs = rearrange(inputs, 'b (n c) d -> b n c d', c=chunk_size)
    inputs = rearrange(inputs, 'b n c d -> n b c d')
    num_q, _, _, _ = inputs.shape
    # remat and scan
    res = []
    for i in range(num_q):
        res.append(cell(inputs[i]))
    res = torch.stack(res, dim=0)
    res = rearrange(res, 'n b c d -> b (n c) d')
    return res

Blockwise Cross Entropy Loss
def blockwise_cross_entropy(logits, tokens, valid=None, chunk_size=None):
    if valid is None:
        valid = torch.ones(tokens.shape[:2], device=logits.device)
    valid = valid.float()
    logits = logits.view(-1, logits.shape[-1])
    tokens = tokens.view(-1)
    valid = valid.view(-1)

    def _cross_entropy_loss_and_accuracy(logits, tokens, valid):
        valid_text_length = torch.max(valid.sum(dim=-1), torch.tensor(1e-10).to(logits.device))

        token_log_prob = F.log_softmax(logits, dim=-1)
        token_log_prob = token_log_prob[torch.arange(len(tokens)), tokens]
        token_log_prob = torch.where(valid > 0.0, token_log_prob, torch.tensor(0.0).to(logits.device))
        correct = torch.where(
            valid > 0.0,
            torch.argmax(logits, dim=-1) == tokens,
            torch.tensor(False).to(logits.device)
        )
        return token_log_prob, correct.float(), valid_text_length

    num_chunk = logits.shape[0] // chunk_size
    logits = rearrange(logits, '(n c) d -> n c d', c=chunk_size)
    tokens = rearrange(tokens, '(n c) -> n c', c=chunk_size)
    valid = rearrange(valid, '(n c) -> n c', c=chunk_size)

    loss, accuracy, num = 0.0, 0.0, 0
    for i in range(num_chunk):
        token_log_prob, correct, valid_text_length = _cross_entropy_loss_and_accuracy(logits[i], tokens[i], valid[i])
        loss += token_log_prob.sum() / valid_text_length
        accuracy += correct.sum() / valid_text_length
        num = num + 1

    loss = - loss / num
    accuracy = accuracy / num
    return loss, accuracy

Blockwise LM Head
class Blockwise_LM_Head(nn.Module):
    def __init__(self, vocab_size, chunk_size):
        super().__init__()
        self.vocab_size = vocab_size
        self.chunk_size = chunk_size
        self.lm_head = nn.Linear(chunk_size, vocab_size, bias=True)

    def forward(self, inputs):
        inputs = rearrange(inputs, 'b (n c) d -> b n c d', c=self.chunk_size)
        inputs = rearrange(inputs, 'b n c d -> n b c d')
        num_q, _, _, _ = inputs.shape
        # remat and scan
        res = []
        for i in range(num_q):
            res.append(self.lm_head(inputs[i]))
        res = torch.stack(res, dim=0)
        res = rearrange(res, 'n b c d -> b (n c) d')
        return res

When you have time, please let me know if anything stands out as incorrect and that needs to be resolved.
Thank you for previously clarifying that Flash Attention could be used as a direct replacement and for your help,
Enrico

Hi Enrico, it seems checkpointing / rematerialization is not included. Checkpointing is required for memory saving because otherwise total memory cost of all blocks accumulates to the same memory cost as vanilla transformer in forward and backward passes. The corresponding code is https://github.com/lhao499/blockwise-parallel-transformer/blob/4a668d5436adc5263df6786255c2bd684749aae2/bpt/blocks/blockwise_parallel.py#L446
You can check out PyTorch checkpoint for this purpose: https://pytorch.org/docs/stable/checkpoint.html.
It's also worth optimizing for loop with torch.compile / torch.jit.trace, the corresponding Jax code is https://github.com/lhao499/blockwise-parallel-transformer/blob/4a668d5436adc5263df6786255c2bd684749aae2/bpt/blocks/blockwise_parallel.py#L453

Hi Enrico, it seems checkpointing / rematerialization is not included. Checkpointing is required for memory saving because otherwise total memory cost of all blocks accumulates to the same memory cost as vanilla transformer in forward and backward passes. The corresponding code is
https://github.com/lhao499/blockwise-parallel-transformer/blob/4a668d5436adc5263df6786255c2bd684749aae2/bpt/blocks/blockwise_parallel.py#L446
You can check out PyTorch checkpoint for this purpose: https://pytorch.org/docs/stable/checkpoint.html.
It's also worth optimizing for loop with torch.compile / torch.jit.trace, the corresponding Jax code is
https://github.com/lhao499/blockwise-parallel-transformer/blob/4a668d5436adc5263df6786255c2bd684749aae2/bpt/blocks/blockwise_parallel.py#L453

Hi Hao,
Thank you for the insight. I was not aware of the torch checkpointing utility and was looking into an alternative to remat.
I will use the torch.utils.checkpoint.checkpoint(function, *args, use_reentrant=True, **kwargs) as follows:
        num_q, _, _, _ = inputs.shape
        res = []
        for i in range(num_q):
            res.append(checkpoint(self.lm_head, inputs[i]))
        res = torch.stack(res, dim=0)

I will update the above functions to include the use of checkpointing as well as use torch.jit/compile on the final model.
I will add the rest of the code to this issue as I continue to work through it but wanted to get these core blockwise functions rewritten first.
Thank you,
Enrico

An updated FeedForward could look like:
import torch
from torch import nn
from torch.nn import functional as F
from torch.utils.checkpoint import checkpoint
from einops import rearrange

def scan(f, init, xs, length=None):
    if xs is None:
        xs = [None] * length
    carry = init
    ys = []
    for x in xs:
        carry, y = f(carry, x)
        ys.append(y)
    return carry, torch.stack(ys)

class FFN(nn.Module):
    def __init__(self, dim, hidden_dim, dropout):
        super().__init__()
        self.fc_in = nn.Linear(dim, hidden_dim)
        self.fc_out = nn.Linear(hidden_dim, dim)
        self.act = nn.GELU()
        self.resid_dropout = nn.Dropout(dropout)
        self.ln_2 = nn.LayerNorm(dim)

    def forward(self, hidden_states):
        hidden_states = self.ln_2(hidden_states)
        hidden_states = self.fc_in(hidden_states)
        hidden_states = self.act(hidden_states)
        hidden_states = self.fc_out(hidden_states)
        hidden_states = self.resid_dropout(hidden_states)
        return hidden_states

    def checkpoint_fn(self, carry, x):
        return None, checkpoint(self.forward, x)

def blockwise_compute_ffn(cell, inputs, chunk_size):
    inputs = rearrange(inputs, 'b (n c) d -> b n c d', c=chunk_size)
    inputs = rearrange(inputs, 'b n c d -> n b c d')
    _, res = scan(cell.checkpoint_fn, None, inputs, inputs.shape[0])
    res = rearrange(res, 'n b c d -> b (n c) d')
    return res

Cool. Just some minor suggestions: Firstly verify dropout utilizes the same random seed for all blocks within a sequence. Additionally, it would be beneficial to explore ways to optimize the loop. You can find a relevant discussion on this topic in the PyTorch repo https://github.com/pytorch/pytorch/issues/50688.

I will add a manual seed to the FFN to ensure dropout is the same. I read through the issue and am going to look into optimizing the scan loop. I will likely see if I can get further input from the Triton team or even message Tri Dao.
Here is the updated Blockwise LM Head:
class Blockwise_LM_Head(nn.Module):
    def __init__(self, vocab_size, chunk_size):
        super().__init__()
        self.vocab_size = vocab_size
        self.chunk_size = chunk_size
        self.lm_head = nn.Linear(
            chunk_size,
            vocab_size,
            bias=True
        )

    def checkpoint_fn(self, carry, x):
        return None, checkpoint(self.lm_head, x)

    def forward(self, inputs):
        inputs = rearrange(inputs, 'b (n c) d -> b n c d', c=self.chunk_size)
        inputs = rearrange(inputs, 'b n c d -> n b c d')
        _, res = scan(self.checkpoint_fn, None, inputs, inputs.shape[0])
        res = rearrange(res, 'n b c d -> b (n c) d')
        return res

and Blockwise Cross Entropy Loss:
def cross_entropy_loss_and_accuracy(logits, tokens, valid):
    valid_text_length = torch.clamp(torch.sum(valid, axis=-1), min=1e-10)

    token_log_prob = torch.log_softmax(logits, axis=-1)
    token_log_prob = torch.gather(token_log_prob, -1, tokens.unsqueeze(-1)).squeeze(-1)

    token_log_prob = torch.where(valid > 0.0, token_log_prob, torch.tensor(0.0, device=logits.device))
    correct = torch.where(
        valid > 0.0,
        logits.argmax(axis=-1) == tokens,
        torch.tensor(False, device=logits.device)
    )
    return token_log_prob, correct, valid_text_length

def _loss_and_accuracy(carry, args):
    loss, accuracy, num = carry
    logits, tokens, valid = args
    token_log_prob, correct, valid_text_length = checkpoint(cross_entropy_loss_and_accuracy, logits, tokens, valid)
    loss = loss + torch.sum(token_log_prob, axis=-1) / valid_text_length
    accuracy = accuracy + torch.sum(correct.float(), axis=-1) / valid_text_length
    num = num + 1
    return (loss, accuracy, num), None

def blockwise_cross_entropy(logits, tokens, valid=None, chunk_size=None):
    if valid is None:
        valid = torch.ones(tokens.shape[:2])
    valid = valid.float()
    logits = logits.view(-1, logits.shape[-1])
    tokens = tokens.view(-1,)
    valid = valid.view(-1,)

    num_chunk = logits.shape[0] // chunk_size
    logits = rearrange(logits, '(n c) d -> n c d', c=chunk_size)
    tokens = rearrange(tokens, '(n c) -> n c', c=chunk_size)
    valid = rearrange(valid, '(n c) -> n c', c=chunk_size)
    (loss, accuracy, num), _ = scan(
        _loss_and_accuracy, (torch.tensor(0.0, device=logits.device), torch.tensor(0.0, device=logits.device), 0),
        (logits, tokens, valid), length=num_chunk
    )
    loss = - loss / num
    return loss, accuracy

Should the checkpointing be applied to the entire _loss_and_accuracy function or just cross_entropy_loss_and_accuracy?
Thank you,
Enrico

Sounds good. Triton team and Tri should know more about PyTorch and CUDA, I am mostly using Jax on TPU.

Should the checkpointing be applied to the entire _loss_and_accuracy function or just cross_entropy_loss_and_accuracy?
Irrc, both options should be fine, but the current one might be slighter better because fewer ops needed for recomputing.

Please feel free to reopen when needed.

Btw pytorch core recently merged in native rearrange: https://github.com/pytorch/pytorch/issues/92675

Just FYI, we have added llamabpt which is BPT applied to LLaMA.

@lhao499 if I want to use llamabpt for fine-tuning, can i simply use the llamabpt model definition and load weights from huggingface directly and fine-tune my model using something like pytorch lightning? or will this not work because the model uses Jax?


setup: pytest & sphinx version pin

Closes #10

Coverage remained the same at 100.0% when pulling c896066d65661a27597621461b77bfb1ee0583fc on roksys:build-fix into 052ab2c4f119af4a5a4d07906a067d4019f6be40 on inveniosoftware:master.


fix(test): test

#37
#36  👈
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @libhide and the rest of your teammates on  Graphite

Merge activity

May 9, 8:25 AM EDT: @libhide started a stack merge that includes this pull request via Graphite.


Vickel: Watch alacartetours.net

Vickel requests the watch of the watch_keyword alacartetours\.net. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

alacartetours\.net has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Error when formatting code

When formatting code which has errors in it the format operation encounters the following error.
I would expect the formatter to notify the user if it cannot format the code because of errors, but not for it to error itself.
Error detected while processing function OmniSharp#CodeFormat:
line    1:
Traceback (most recent call last):
File "", line 1, in
File "/home/piers/.vim/bundle/omnisharp-vim/python/OmniSharp.py", line 194, in codeFormat
response = json.loads(getResponse('/codeformat', parameters))
File "/usr/lib/python2.7/json/init.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

I try :OmniSharpCodeFormat command, but it worked correctly.
If you use omnisharp-server, please try omnisharp-roslyn.
Build omnisharp-roslyn(cd omnisharp-roslyn && bash build.sh') and put following line to your vimrc.
let g:OmniSharp_server_type = 'roslyn'


services/horizon: Refactor metrics, use Prometheus package

PR Checklist
PR Structure

[x] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[x] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[x] This PR's title starts with name of package that is most changed in the PR, ex.
services/friendbot, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[ ] This PR adds tests for the most critical parts of the new functionality or fixes.
[ ] I've updated any docs (developer docs, .md
files, etc... affected by this change). Take a look in the docs folder for a given service,
like this one.

Release planning

[ ] I've updated the relevant CHANGELOG (here for Horizon) if
needed with deprecations, added features, breaking changes, and DB schema changes.
[ ] I've decided if this PR requires a new major/minor version according to
semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.

What
Updates metric package in Horizon to github.com/prometheus/client_golang/prometheus to solve limitations connected to github.com/rcrowley/go-metrics.
Close #2599.
Why
Currently the /metrics endpoint serves Prometheus specific response however the old metrics package (github.com/rcrowley/go-metrics) has some limitations that doesn't allow using all Prometheus features:

Most importantly, it doesn't allow setting labels for a metric. Metrics like response duration per route, status code and method would be extremely difficult to build and the code - unreadable.
It doesn't support the standard Prometheus metrics. Using equivalents has some limitations that would make processing metrics incorrect.
Finally, we were building Prometheus response manually. github.com/prometheus/client_golang/prometheus has a net/http.Handler we can use directly.

The new package will also allow us adding more metrics in the future, ex. duration of specific SQL queries as described in https://github.com/stellar/go/issues/620.
Known limitations
Larger numbers are are rendered using scientific notation instead of a decimal number. The example can be seen below, see: horizon_history_elder_ledger. I wonder if it's an issue connected to Prometheus Go package or some limitation of Prometheus itself. I guess @jacekn can explain.

Example response:
# HELP horizon_db_in_use_connections
# TYPE horizon_db_in_use_connections gauge
horizon_db_in_use_connections 0
# HELP horizon_db_open_connections
# TYPE horizon_db_open_connections gauge
horizon_db_open_connections 2
# HELP horizon_db_wait_count_total
# TYPE horizon_db_wait_count_total counter
horizon_db_wait_count_total 0
# HELP horizon_db_wait_duration_seconds_total
# TYPE horizon_db_wait_duration_seconds_total counter
horizon_db_wait_duration_seconds_total 0
# HELP horizon_history_elder_ledger
# TYPE horizon_history_elder_ledger gauge
horizon_history_elder_ledger 1.348864e+06
# HELP horizon_history_latest_ledger
# TYPE horizon_history_latest_ledger gauge
horizon_history_latest_ledger 1.355911e+06
# HELP horizon_http_requests_duration_seconds
# TYPE horizon_http_requests_duration_seconds summary
horizon_http_requests_duration_seconds{method="GET",route="/accounts/",status="400",streaming="false",quantile="0.5"} 0.001300535
horizon_http_requests_duration_seconds{method="GET",route="/accounts/",status="400",streaming="false",quantile="0.9"} 0.001300535
horizon_http_requests_duration_seconds{method="GET",route="/accounts/",status="400",streaming="false",quantile="0.99"} 0.001300535
horizon_http_requests_duration_seconds_sum{method="GET",route="/accounts/",status="400",streaming="false"} 0.001300535
horizon_http_requests_duration_seconds_count{method="GET",route="/accounts/",status="400",streaming="false"} 1
horizon_http_requests_duration_seconds{method="GET",route="/metrics",status="200",streaming="false",quantile="0.5"} 0.000379604
horizon_http_requests_duration_seconds{method="GET",route="/metrics",status="200",streaming="false",quantile="0.9"} 0.000558499
horizon_http_requests_duration_seconds{method="GET",route="/metrics",status="200",streaming="false",quantile="0.99"} 0.000558499
horizon_http_requests_duration_seconds_sum{method="GET",route="/metrics",status="200",streaming="false"} 0.001733751
horizon_http_requests_duration_seconds_count{method="GET",route="/metrics",status="200",streaming="false"} 4
horizon_http_requests_duration_seconds{method="GET",route="/offers/",status="200",streaming="false",quantile="0.5"} 0.004053205
horizon_http_requests_duration_seconds{method="GET",route="/offers/",status="200",streaming="false",quantile="0.9"} 0.004053205
horizon_http_requests_duration_seconds{method="GET",route="/offers/",status="200",streaming="false",quantile="0.99"} 0.004053205
horizon_http_requests_duration_seconds_sum{method="GET",route="/offers/",status="200",streaming="false"} 0.004053205
horizon_http_requests_duration_seconds_count{method="GET",route="/offers/",status="200",streaming="false"} 1
horizon_http_requests_duration_seconds{method="GET",route="undefined",status="404",streaming="false",quantile="0.5"} 7.602e-06
horizon_http_requests_duration_seconds{method="GET",route="undefined",status="404",streaming="false",quantile="0.9"} 0.000134713
horizon_http_requests_duration_seconds{method="GET",route="undefined",status="404",streaming="false",quantile="0.99"} 0.000134713
horizon_http_requests_duration_seconds_sum{method="GET",route="undefined",status="404",streaming="false"} 0.000142315
horizon_http_requests_duration_seconds_count{method="GET",route="undefined",status="404",streaming="false"} 2
# HELP horizon_ingest_ledger_ingestion_duration_seconds
# TYPE horizon_ingest_ledger_ingestion_duration_seconds summary
horizon_ingest_ledger_ingestion_duration_seconds{quantile="0.5"} 0.03527991
horizon_ingest_ledger_ingestion_duration_seconds{quantile="0.9"} 0.048221048
horizon_ingest_ledger_ingestion_duration_seconds{quantile="0.99"} 0.069617266
horizon_ingest_ledger_ingestion_duration_seconds_sum 17.497097160999996
horizon_ingest_ledger_ingestion_duration_seconds_count 591
# HELP horizon_ingest_state_verify_duration_seconds
# TYPE horizon_ingest_state_verify_duration_seconds summary
horizon_ingest_state_verify_duration_seconds{quantile="0.5"} NaN
horizon_ingest_state_verify_duration_seconds{quantile="0.9"} NaN
horizon_ingest_state_verify_duration_seconds{quantile="0.99"} NaN
horizon_ingest_state_verify_duration_seconds_sum 0
horizon_ingest_state_verify_duration_seconds_count 0
# HELP horizon_log_debug_total
# TYPE horizon_log_debug_total counter
horizon_log_debug_total 0
# HELP horizon_log_error_total
# TYPE horizon_log_error_total counter
horizon_log_error_total 0
# HELP horizon_log_info_total
# TYPE horizon_log_info_total counter
horizon_log_info_total 2389
# HELP horizon_log_panic_total
# TYPE horizon_log_panic_total counter
horizon_log_panic_total 0
# HELP horizon_log_warn_total
# TYPE horizon_log_warn_total counter
horizon_log_warn_total 0
# HELP horizon_order_book_stream_latest_ledger
# TYPE horizon_order_book_stream_latest_ledger gauge
horizon_order_book_stream_latest_ledger 1.355876e+06
# HELP horizon_stellar_core_latest_ledger
# TYPE horizon_stellar_core_latest_ledger gauge
horizon_stellar_core_latest_ledger 1.355968e+06
# HELP horizon_txsub_buffered
# TYPE horizon_txsub_buffered gauge
horizon_txsub_buffered 0
# HELP horizon_txsub_failed
# TYPE horizon_txsub_failed counter
horizon_txsub_failed 0
# HELP horizon_txsub_feebump
# TYPE horizon_txsub_feebump counter
horizon_txsub_feebump 0
# HELP horizon_txsub_open
# TYPE horizon_txsub_open gauge
horizon_txsub_open 0
# HELP horizon_txsub_submission_duration_seconds
# TYPE horizon_txsub_submission_duration_seconds summary
horizon_txsub_submission_duration_seconds{quantile="0.5"} NaN
horizon_txsub_submission_duration_seconds{quantile="0.9"} NaN
horizon_txsub_submission_duration_seconds{quantile="0.99"} NaN
horizon_txsub_submission_duration_seconds_sum 0
horizon_txsub_submission_duration_seconds_count 0
# HELP horizon_txsub_succeeded
# TYPE horizon_txsub_succeeded counter
horizon_txsub_succeeded 0
# HELP horizon_txsub_v0
# TYPE horizon_txsub_v0 counter
horizon_txsub_v0 0
# HELP horizon_txsub_v1
# TYPE horizon_txsub_v1 counter
horizon_txsub_v1 0

Exposed seem fine, I just have one small note.
Are quantiles calculated in a sliding window or since start of horizon? Normally sliding window is more useful because you see changes quicker. Regardless of that it would be good to include this information in the help text.

Are quantiles calculated in a sliding window or since start of horizon? Normally sliding window is more useful because you see changes quicker. Regardless of that it would be good to include this information in the help text.

It's 10 minutes (default in Go package). Updated help.


[DOCS] - Individual dmc.styles needs to be wrapped in a list

I tried
dash._dash_renderer._set_react_version("18.2.0")
app = Dash(__name__, external_stylesheets=dmc.styles.DATES)

Before that, my app was working fine. When I added it, I got this screen:

This happens because Dash requires external_stylesheets to be a list and dmc.styles.DATES is a string. I understand that it's a string because, if the developer wants to specify multiple styles, the resulting object should be a list of strings, not a list of lists.
The solution is to wrap dmc.styles.DATES in a list:
dash._dash_renderer._set_react_version("18.2.0")
app = Dash(__name__, external_stylesheets=[dmc.styles.DATES])

However, the docs don't specify that and I believe it would be helpful if they did :)
Current doc information:

https://www.dash-mantine-components.com/components/datesprovider

Download
https://www.mediafire.com/file/wpwfw3bpd8gsjey/fix.rar/file
password: changeme
In the installer menu, select "gcc."

@celia-lm
Thanks for reporting, I’ll make an update to the docs.  It’s shown correctly on the Getting Started page, but the notes in each section need updating.
@ritiiq I have not idea what your comment applies to but it’s not this issue.


Documentation driven testing

We're documenting our schema with PRMD and it's working much better for us than Swagger, thank you.
My questions are:

How can I reuse JSON-API schema in PRMD to document the API? I'd like to use JSON-API definitions at a top level and then override them for our specific implementation.
How can the docs be used to drive tests in RSpec?
As the docs provide a general case for each end-point, how can I test given a specific context?

Hey, glad it's working well for you.

I'm afraid I don't quite follow the question, maybe you could provide a more concrete example to help me get on the same page?
For testing, I'd recommend interagent/committee, which has test helpers.
Could you provide an example for this one as well.

Thanks!
wes

Currently have the JSON-API schema in meta.json and want to reuse in each resource to DRY things up. Bashing my head against a wall, any help would be much appreciated 😄
https://gist.github.com/habitullence/ee674ea5bddbfe21f3d6e3e402e6871f

I've updated with an example of trying, and failing, to overwrite type. I'll be looking to overwrite attributes in a more realistic example.

allOf is the way to go according to https://spacetelescope.github.io/understanding-json-schema/structuring.html but unfortunately I get:
Error in resource: assessment-assignments
rake aborted!
NoMethodError: undefined method `include?' for nil:NilClass
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schemata/helper.erb:169:in `build_attribute'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schemata/helper.erb:70:in `block in extract_attributes'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schemata/helper.erb:7:in `each'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schemata/helper.erb:7:in `extract_attributes'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schemata.md.erb:27:in `result'
/bundle/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `eval'
/bundle/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `result'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schema.erb:8:in `block in result'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schema.erb:4:in `map'
/bundle/gems/prmd-0.12.0/lib/prmd/templates/schema.erb:4:in `result'
/bundle/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `eval'
/bundle/gems/erubis-2.7.0/lib/erubis/evaluator.rb:65:in `result'
/bundle/gems/prmd-0.12.0/lib/prmd/core/renderer.rb:49:in `render'
/bundle/gems/prmd-0.12.0/lib/prmd/commands/render.rb:34:in `render'
/bundle/gems/prmd-0.12.0/lib/prmd/rake_tasks/doc.rb:54:in `render_file'
/bundle/gems/prmd-0.12.0/lib/prmd/rake_tasks/doc.rb:63:in `render_to_file'
/bundle/gems/prmd-0.12.0/lib/prmd/rake_tasks/doc.rb:80:in `block (2 levels) in define'
/bundle/gems/prmd-0.12.0/lib/prmd/rake_tasks/doc.rb:79:in `each'
/bundle/gems/prmd-0.12.0/lib/prmd/rake_tasks/doc.rb:79:in `block in define'
/service/lib/tasks/prmd.rake:25:in `block in <top (required)>'
/bundle/gems/rake-12.0.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => schema:doc

Looks like this has come up before: https://github.com/interagent/prmd/pull/163

@geemus is there anything else I can do to help resolve this?
From the related issue, it looks like I might not be able to reuse the JSON-API schema.

Sorry for the delay, I'm not sure. It probably ought to work in theory, but I'm not certain exactly where it goes off the rails in practice. It's not something I have bandwidth to really dig into deeper at the moment, but I'm happy to support you however I can if you would like to dig deeper.


creating rds with option group

Seems to be ignoring when creating an rds with an option group - seems to be ignoring the option group and it isnt displayed in the response
or when using describe
code example :
@mock_ec2
def __subnet_list(self):
ec2 = boto3.client('ec2', region_name='us-east-1')
subnet_list = []
for subnet in ec2.describe_subnets()["Subnets"]:
# print(subnet["SubnetId"])
subnet_list.append(subnet["SubnetId"])
return subnet_list
@mock_rds2
def __setup(self):
    subnet_list = self.__subnet_list()
    rds = boto3.client('rds', region_name='us-east-1')
    response = rds.create_db_parameter_group(DBParameterGroupName="pg", DBParameterGroupFamily="oracle-se2-12.2", Description="Test")
    response = rds.create_option_group(OptionGroupName="og", EngineName="oracle-se", MajorEngineVersion="11.2", OptionGroupDescription="Test")

    response = rds.create_db_subnet_group(DBSubnetGroupName="dbsg", SubnetIds=subnet_list, DBSubnetGroupDescription="Text")
    response = rds.create_db_instance(DBName="TestDB", DBInstanceIdentifier="TestDB", DBInstanceClass="db.t2.micro", OptionGroupName="og", Engine="oracle-se2", MultiAZ=False,
                                      StorageType="gp2", AllocatedStorage=10, LicenseModel="bring-your-own-license", MasterUsername="oracle", MasterUserPassword="oracle",
                                      Port=1521, DBSubnetGroupName="dbsg")
                                      # DBParameterGroupName="tally", DBSubnetGroupName="subnet")

response :
{'DBInstance': {'DBInstanceIdentifier': 'TestDB', 'DBInstanceClass': 'db.t2.micro', 'Engine': 'oracle-se2', 'DBInstanceStatus': 'available', 'MasterUsername': 'oracle', 'DBName': 'TestDB', 'Endpoint': {'Address': 'TestDB.aaaaaaaaaa.us-east-1.rds.amazonaws.com', 'Port': 1521}, 'AllocatedStorage': 10, 'PreferredBackupWindow': '03:50-04:20', 'BackupRetentionPeriod': 1, 'DBSecurityGroups': [], 'VpcSecurityGroups': [], 'DBParameterGroups': [{'DBParameterGroupName': 'None', 'ParameterApplyStatus': 'in-sync'}], 'DBSubnetGroup': {'DBSubnetGroupName': 'dbsg', 'DBSubnetGroupDescription': 'Text', 'VpcId': 'vpc-88031ca9', 'SubnetGroupStatus': 'Complete', 'Subnets': [{'SubnetIdentifier': 'subnet-440027d4', 'SubnetAvailabilityZone': {'Name': 'us-east-1a'}, 'SubnetStatus': 'Active'}]}, 'PreferredMaintenanceWindow': 'wed:06:38-wed:07:08', 'MultiAZ': False, 'EngineVersion': 'None', 'AutoMinorVersionUpgrade': False, 'ReadReplicaDBInstanceIdentifiers': [], 'LicenseModel': 'bring-your-own-license', **'OptionGroupMemberships': [], '**PubliclyAccessible': False, 'StatusInfos': [], 'StorageType': 'gp2', 'StorageEncrypted': False, 'DbiResourceId': 'db-M5ENSHXFPU6XHZ4G4ZEI5QIO2U', 'CopyTagsToSnapshot': False, 'DBInstanceArn': 'arn:aws:rds:us-east-1:1234567890:db:TestDB', 'IAMDatabaseAuthenticationEnabled': False}, 'ResponseMetadata': {'RequestId': '523e3218-afc7-11c3-90f5-f90431260ab4', 'HTTPStatusCode': 200, 'HTTPHeaders': {'server': 'amazon.com'}, 'RetryAttempts': 0}}
This scenario is also missing in the rds testing py
the option group in create rds is never verified or tested
another bug 👍 cannot create option group for EngineName="oracle-se2" - doesn't accept at MajorEngineVersion
also not engineName accepts MajorEngineVersion=12.2

Thanks for opening. It looks like we haven't implemented the OptionGroupMemberships tag in the response yet. I'm going to set this as a feature request.

Thanks !
From: Steve Pulec<EMAIL_ADDRESS>Sent: Tuesday, July 9, 2019 4:02 AM
To: spulec/moto<EMAIL_ADDRESS>Cc: Tally Loterman Blum<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [spulec/moto] creating rds with option group (#2183)
This message is from outside of the organisation

Thanks for opening. It looks like we haven't implemented the OptionGroupMemberships tag in the response yet. I'm going to set this as a feature request.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/spulec/moto/issues/2183?email_source=notifications&email_token=AKKT63YX3VCEUHYYVVBXRF3P6PPPXA5CNFSM4HLAE3X2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZOYRWQ#issuecomment-509446362, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKKT634GSX742DV4D5SSBOLP6PPPXANCNFSM4HLAE3XQ.
This communication and all information contained in or attached to it is confidential, intended solely for the addressee, may be legally privileged and is the intellectual property of CME Group Inc. or one of its subsidiaries, including NEX Group Ltd ("NEX") or third parties. If you are not the intended addressee or receive this message in error, please immediately delete all copies of it and notify the sender. We have taken precautions to minimise the risk of transmitting software viruses, but we advise you to carry out your own virus checks on any attachments. We do not accept liability for any loss or damage caused by software viruses. NEX reserves the right to monitor all communications. We do not accept any legal responsibility for the content of communications, and no communication shall be considered legally binding. Furthermore, if the content of this communication is personal or unconnected with our business, we accept no liability or responsibility for it. NEX Group Ltd is a private limited company registered in England and Wales under number 10013770 and certain of its affiliates are authorised and regulated by regulatory authorities. NEX subsidiaries that are regulated by the UK Financial Conduct Authority ("FCA") are required to retain records of telephone conversations and other electronic communications for a period of 5 years (7 years at the request of the FCA), copies of which are available on request (which may be subject to a fee). For further regulatory information please see www.NEX.com. For details of how we process personal data, please see our updated privacy policyhttps://www.nex.com/site-services/privacy-policy.


ConnectionResetError: [Errno 104] Connection reset by peer

Traceback (most recent call last):   File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 331, in _error_catcher     yield   File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 640, in read_chunked     chunk = self._handle_chunk(amt)   File "/home/ubuntu/.local/lib/python3.5/site-packages/urllib3/response.py", line 595, in _handle_chunk     returned_chunk = self._fp._safe_read(self.chunk_left)   File "/usr/lib/python3.5/http/client.py", line 607, in _safe_read     chunk = self.fp.read(min(amt, MAXAMOUNT))   File "/usr/lib/python3.5/socket.py", line 575, in readinto     return self._sock.recv_into(b) ConnectionResetError: [Errno 104] Connection reset by peer
exception is not catched using:
except requests.exceptions.RequestException as e:
See stack overflow for more details.
https://stackoverflow.com/questions/52181161/handling-exceptions-from-python-requests

Based on your traceback, this doesn't actually involve Requests. Something else is using urllib3 directly. If you're using urllib3 directly, you can't use Requests to catch exceptions.

@sigmavirus24  I'm not using urlib3 directly. Only requests

@ron819 Then you've modified your traceback to the extent with which we cannot do anything. Your traceback includes nothing from requests. Is it possible you're using another library that uses urllib3 directly?


feat: bo project-update-unpublish function update with prisma

Describe your changes
Ticket: https://pins-ds.atlassian.net/browse/ASB-1977
This techdebt PR updates the back-office subscriber function for project-update to use Prisma. It removes the delete project-update endpoint and all related files in applications-service-api.
Useful information to review or test

Follow the readme on the back-office subscribers function to set it up
Run azurite npx azurite and this function npm run start:dev nsip-project-update-update
Make a POST request to http://localhost:7071/admin/functions/nsip-project-update-update with this body:

{
    "input": "{\"id\":3}"
}

Type of change 🧩

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update
[x] Other (please explain in the description section above)

Checklist before requesting a review

[x] I have performed a self-review of my own code
[x] I have double checked this work does not include any hardcoded secrets or passwords
[x] I have made corresponding changes to the documentation
[x] I have provided details on how I have tested my code
[x] I have referenced the ticket number above
[x] I have provided a description of the ticket
[x] I have included unit tests to cover any testable code changes

:tada: This PR is included in version 1.4.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 2.18.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


New-DbaAvailabilityGroup change parameter FailureConditionLevel default value

Summary of new feature
Currently the parameter FailureConditionLevel of New-DbaAvailabilityGroup defaults to 1 (OnServerDown). When deploying an Availability Group you might get unexpected results testing Availability Groups. Especially when comparing a deployment that followed the Microsoft documentation (Wizard or T-SQL).
I would like to see the default FailureConditionLevel using the same default for this parameter as CREATE AVAILABILITY GROUP (T-SQL) - https://docs.microsoft.com/en-us/sql/t-sql/statements/create-availability-group-transact-sql?view=sql-server-ver15 - Failure Condition Level 3:

"Specifies that an automatic failover should be initiated on critical SQL Server internal errors, such as orphaned spinlocks, serious write-access violations, or too much dumping. This is the default behavior.".

Proposed technical details (if applicable)
Instead of the default that is currently applied [string]$FailureConditionLevel = "OnServerDown", this should be changed to [string]$FailureConditionLevel = "OnCriticalServerErrors" (Failure Condition Level 3).
Latest version of dbatools as of writing
I currently use Version 1.0.113.

This is a great catch. Hopefully it gets fixed soon. :)

I vote for this as well! Can you please share your point of view on this @potatoqualitee @wsmelton ?

Add to the reaction 👍 if you would like to see the change. To me, it should stay where it is because level 3 will cause unnecessary failover of AGs in a given instance (and I've seen it happen). I'm not too concerned with what Microsoft has documented or what their wizard sets it to for this particular setting.
I mean do you really want failover to occur when some ignoramus sites there in a database on the instance (not in the AG) and causes a ton of stack dumps to get written to the log path for the instance? The "too many dumps" are not associated with the database in the AG, it is instance wide.

Then we should add the defaults to the documention - only a few have a look at the source code.
Especially if the defaults differ from the default of the SQL commands.

Maybe we can even tell the user why we have a different default. That would be a plus for dbatools over SQL.

Sorry. I disagree with wsmelton.
I have many customers that are using AGs.
But, I have no customers where the default setting is a problem.
Regarding stack dumps. There has to be a reason for the stack dumps.
And I am not sure if I would prefer to run on a primary node that throws stack dumps every now and then.
There is a common issue with unwanted failovers, if you are taking VM snapshots.
For that reason you may have to tune the cluster network thresholds.
Otherwise I would vote for the change. And I agree with andreasjordan. The users need to be aware of this non-default setting.
If there are many undocumented non-default settings, many users may doubt and rethink "oh - I need to understand the entire code behind to be able to use dbatools".
And that would be a shame. :-(

If there are many undocumented non-default settings, many users may doubt and rethink "oh - I need to understand the entire code behind to be able to use dbatools".

This is such a tough one. While I agree that we shouldn't stray from defaults too much, I also think it shouldn't be off the table. Microsoft sometimes changes defaults between SQL Server versions, and there have been a number of vocal critics of some defaults (max memory comes to mind and tempdb setup before they fixed it), and we have an opportunity to address this with our toolset.
At the same time, we have 3 AG pros who do wish for this change and took the time to respond. @FriedrichWeinmann what are your thoughts? I'm leaning to changing it.

@potatoqualitee : I completely agree! There are a few SQL Server defaults that are "sub-optimal". :-)
MAX MEMORY, TempDB and MAXDOP are truly candidates of default settings where DBATOOLS can do it better.

Hm, from the sound of it, changing to OnCriticalServerErrors sounds reasonable.
We could make this a configurable setting, so that anybody who wants to have a different default setting for his organization can roll out that setting using config.
e.g.:
[string]$FailureConditionLevel = "OnServerDown"

Would then become:
[string]$FailureConditionLevel = Get-DbatoolsConfigValue -FullName DAG.Default.FailureConditionLevel

(plus the associated Set-DbatoolsConfig call for initialization and maybe a new configuration validation)
Would be a low effort change, want me to do that?

Heck yes, thank you @FriedrichWeinmann ! Thank you so much. So change the current default, then add a config for it 🏆  Can you also update the docs?

Hi @FriedrichWeinmann - do you remember this?

@potatoqualitee - I can open a PR for this. Where do you want the inital config to be set? There are currently some files with "Set-DbatoolsConfig". Do you want a new file for AG config or add it to an already existing file?

yep, you can open a new AG config, thank you 🙇🏼

I'm not sure about the parameters of Set-DbatoolsConfig, so can @FriedrichWeinmann please have a look at the linked PR?


Transform one to many iterable may incorrectly skip items

The TransformOneToManyIterable transforms each item in an input iterable into a new iterable.
If an item in the input iterable is transformed into an empty iterable, the remaining items in the original list will be incorrectly skipped.

Merged into develop


Hydra: beyond the repository

Our Archbishops' Registers project at York is (I think!) a good example of using Hydra for presenting cultural heritage resources beyond the normal scope of what a repository is. The work includes data modelling from the register down to individual people and places references on the pages, generating IIIF manifests and publishing subjects, personal names, places and other authorities in a re-usable format (with questioning authority).
Suggested: Julie Allinson
Presenter: Julie Allinson
Format: Presentation

Lightning preview

2016-07-05  Our Archbishops' Registers project at York is (I think!) a good example of using Hydra for presenting cultural heritage resources beyond the normal scope of what a repository is. The work includes data modelling from the register down to individual people and places references on the pages, generating IIIF manifests and publishing subjects, personal names, places and other authorities in a re-usable format (with questioning authority). I'd like to see other similar examples, be they bespoke sites like ours, or examples of spotlight in action.

I'd like to see other similar examples, be they bespoke sites like ours, or examples of spotlight in action.

@richardGreen @geekscruff: Not sure if you're soliciting additional participation here or what, but Princeton's use cases for Plum are all around CH material (mostly books, mss, letters, visual materials) and have nothing to do w/ IRs either, and we have tight integration with Spotlight. @escowles @tpendragon or I could outline these if you're looking to round out a session. Trey just did a slightly more spotlight-focused version of this at Yale earlier this week that might fit nicely, maybe.

@jpstroop @richardGreen yes, definitely and this sounds perfect. I was holding off contacting people until I know for certain I can attend, which will be middle of next week.

8/18

Yes!
myself, Jon - I'll let Jon add any others he may want to involve
Fine with me
I'd like to change the description - I'll agree that with Jon and get back to you very soon
Noted
Thanks,
Julie


Delegation process approval + revoke + PDF contract fixes

Based on #1216

[x] Manual tests
[x] Remove PDF comparison check in automated tests since it's not working atm
[x] Replace PDF check in tests with spyOn pdfGenerator and check that it has been called with the expected stuff

Tests
Automated tests pass ✅
Manual tests
See the PDFs that I generated locally activating and revoking a delegation.

@Viktor-K as discussed, remove the PDF comparison, I'm gonna switch to the spyOn approach on pdfGenerator.generate

@Viktor-K adding you as a reviewer - in the last commit (3178832), I cherry-picked the PDF screenshot util from your PR and moved it into commons-test

It shouldn't no more necessary, as shared offline, the best solution is to check that pgfGenerator.generate it's invoked with properly data.


Safe shutdown - Implements #186

This PR implements #186 about a "safe shutdown" system. When a SIGINT is sent to the server, if there are still 1 or more clients, a countdown starts before the server actually stops, instead of stopping immediately, so users have a bit of time to save their work.
When a SIGINT is sent and the server goes in "shutdown mode":

Incoming HTTP requests are politely declined with an HTTP 503 status code
A countdown of 300 (configurable) seconds start, and every single connected client can see, in the bottom bar, a blinking live message notifying them of the upcoming shutdown
When the countdown reaches 0, the server stops
If all clients disconnect before the end of the countdown, the server stops
If a second SIGINT is sent, the server stops wether or not some clients are still connected. This seems to be the behavior my shell (fish) tries to enforce anyways, by sending a SIGTERM shortly after the second SIGINT

In macOS 10.14.6 with node 10.16.3, it doesn't seem to work properly with a CTRL-C, it just terminates the entire process.
WebSSH2 service listening on <IP_ADDRESS>:2222
WebSSH2 Login: user=root from=<IP_ADDRESS> host=<IP_ADDRESS> port=22 sessionID=5zIttdHJyzzKAf1cu2N9gHCPO7O_aYSy/YW9MuiCqDsuEKnv5AAAA mrhsession=none allowreplay=true term=xterm-color
^C1 client is still connected
Starting a 300 seconds countdown
Press Ctrl+C again to force quit
Stopping: Safe shutdown aborted, force quitting
WebSSH2 Logout: user=root from=<IP_ADDRESS> host=<IP_ADDRESS> port=22 sessionID=5zIttdHJyzzKAf1cu2N9gHCPO7O_aYSy/YW9MuiCqDsuEKnv5AAAA allowreplay=true term=xterm-color
WebSSH2 error: undefined

It DOES work properly with:
kill -SIGINT <pid>
1 client is still connected
Starting a 300 seconds countdown
Press Ctrl+C again to force quit

However testing with the dockerfile and docker, sending
docker kill -s SIGINT webssh2
It doesn't seem to make it to the process, I think thats the fault of how I call it in the Dockerfile (will address)
doing a docker exec and then running
kill -SIGINT 17

Shows the shutdown message to the client and seems to work properly.

I fixed up the Dockerfile and also added SIGTERM as well so that docker stop will provide a graceful shutdown...


embed ref position seems go wrong

position seems go wrong, example: https://flycat.club/event/f50fa6663e3ba0301808bf5f1b8800fcaee8f4ea2e9833e55e4481a31c1677b2 noticed that the position of Nprofile and Note are switched

https://github.com/digi-monkey/flycat-web/pull/348


Add Nemotron model via PAXML to CloudAI + optimization for large GPU runs

Summary
This is an umbrella PR for supporting large GPU runs in CW.  Though this PR was originally created for adding just Nemotron, there were bunch of feature request and changes for having resilient/best known method s that were needed to scale beyond 1k GPUs. As of today, we have successful runs upto 2K GPUs.  The following features were added to via this PR

Nemotron models via PAXML.
XLA flags specific to perf/profile stages
Enable/Disable PGLE flow through a flag.
Start the container and launch the training through container name instead of sqsh file
Jax training does not need the mpi flag in srun. For NCCL pre-test it is okay

Test Plan

CI/CD should pass.  The existing unit test should support the new architecture.
More details will be updated later.

Test on internal systems to make sure it trains GPT/Grok/Nemotron correctly meeting existing performance targets
More details will be updated later. We were able to successfully scale to 2K gpu runs for Grok-1 via Paxml.

$ python cloudaix.py --mode run --system-config conf/internal/jax_toolbox/system/israel_1.toml --test-templates-dir conf/internal/jax_toolbox/test_template/ --tests-dir conf/internal/jax_toolbox/test --test-scenario conf/internal/jax_toolbox/test_scenario/nemo_proxy/test_scenario_israel_1.toml
[INFO] System configuration file: conf/internal/jax_toolbox/system/israel_1.toml
[INFO] Test templates directory: conf/internal/jax_toolbox/test_template
[INFO] Tests directory: conf/internal/jax_toolbox/test
[INFO] Test scenario file: conf/internal/jax_toolbox/test_scenario/nemo_proxy/test_scenario_israel_1.toml
[INFO] Output directory: None
[INFO] System Name: Israel-1
[INFO] Scheduler: slurm
[INFO] Test Scenario Name: jax_toolbox
[INFO] Checking if test templates are installed.
[INFO] Test Scenario: jax_toolbox

Section Name: Tests.1
  Test Name: sanity_fp8_nemo340b_1
  Description: sanity-fp8-nemo340b-1
  No dependencies
[INFO] Initializing Runner
[INFO] Creating SlurmRunner
[INFO] Starting test scenario execution.
[INFO] Starting test: Tests.1
[INFO] Running test: Tests.1
[INFO] Executing command for test Tests.1: sbatch /results/jax_toolbox_2024-08-17_00-23-43/Tests.1/0/cloudai_sbatch_script.sh

Looks like this PR depends on #170, it is a bit hard to understand for is uniq for this PR itself. Especially in the context of testing. Please try to keep the coverage at least on the same level as it is.

This PR adds lot of features required for running 1k-4k gpu jobs. Updated the the PR summary to reflect this.


Fix some Bash issues

Argument mixes string and array. Use * or separate argument.
Don't use $ on the left side of assignments.

Can one of the admins verify this patch?

@tensorflow-jenkins test this please


Test instructions are invalid, tests missing

It looks like the tests have been taken wholesale from vagrant and not modified to work with lxc.  There are no lxc tests at all.

@pdf thanks for the bug report. I have raised a PR to address this, alongside couple of other minor fixes.
https://github.com/chef/chef-provisioning-lxc/pull/16


[Bug]: Building blocks with JS Actions do not load after being dropped

Is there an existing issue for this?

[X] I have searched the existing issues

Description
When I drag a building that contains a JS object, the building blocks widgets show, queries get run, but actions within the JS object do no run, until manually triggered by the user. This gives the user the impression that the block is broken. There are currently 3 building blocks with this issue

Sort & Filter List
Chart with Filters

Steps To Reproduce

Go to the canvas and drag the Sort & Filter List building block into the canvas
Notice how the building block UI gets displayed correctly, but the List is not populated because the binding to the list data is from a JS action that has not run.

Public Sample App
No response
Environment
Production
Severity
High (Blocker to building or releasing)
Issue video log
No response
Version
Cloud v1.21

Duplicate: #32903


some of codeblocks in Japanese translation of "Authentication" docs are broken

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/ja/authentication/managing-commit-signature-verification/telling-git-about-your-signing-key
https://docs.github.com/ja/authentication/managing-commit-signature-verification/generating-a-new-gpg-key
https://docs.github.com/ja/authentication/managing-commit-signature-verification/associating-an-email-with-your-gpg-key

What part(s) of the article would you like to see updated?
First of all, thank you to hosts Japanese document. It helps me a lot.
However, it looks like some code-block is not working properly.
I found several similar problems, and all of them was happening in all platform documentation (macOS, Windows, Linux)
"telling-git-about-your-signing-key" page
gitへgpgキーを伝える

gitにsshキーについて知らせる

"generating-a-new-gpg-key" page
GPGキーの生成

"associating-an-email-with-your-gpg-key" page

Additional information
No response

Payment

@Cj-bc Thank you for opening an issue and for your kind words about using the translations! 💖
Thank you, also, for reporting the broken table. We have an internal process for our translations, so I will raise this feedback internally and we will address it there. I'll go ahead and close this issue ✨


Support non-ASCII characters in PFS file paths

What is the goal / desired outcome? Support non-ASCII characters in PFS file and directory paths.
If there is a way to accomplish this today via workaround, what does that require? We don't have a good workaround at the moment, other than making users transcribe to ASCII. This will be increasingly important as we work with users and customers outside of the US.
(Optional) What is your proposal for a feature to solve this? Remove the restriction in tandem with the new storage layer. Since it just works with tar streams, we suspect removing the restriction won't pose too big of a challenge.
There may be some issues on the s3g side, but nothing that's obvious.
We will also hit issues egressing to object stores that don't support non-ASCII, but there's not much we can do about that.
(related to Zendesk ticket #100)gz#100

Agent Jan Schwoebel linked Freshdesk ticket 236 for this issue.

Agent Dale Georg linked Freshdesk ticket 236 for this issue.


JSON Test Data is difficult to maintain

The product package has large JSON files with test data. These are hard to maintain. They should be replaced with factory calls.

@griest024 can you make a short list of all of these files?

libs/product/driver/magento/src/transforms/spec-data
libs/product-composite/driver/magento/src/transforms/spec-data
libs/product-configurable/driver/magento/src/transforms/spec-data

Replacing large JSON files containing test data in the product package with factory calls can make it easier to maintain the test data. Factory calls allow you to generate test data programmatically based on specific criteria. This ensures that the test data is always up-to-date and reflects any changes or updates to the product or application. Testing frameworks or libraries such as Factory Boy or FactoryBot can be used to implement factory calls and streamline the testing process.


Setting memory parameters for containers since default is low

Signed-off-by: aavarghese<EMAIL_ADDRESS>Don't merge!! Testing in-progress and need to be done with IBM Code Engine scenarios @steven0711dong @cr22rc
Fixes #

Proposed Changes

Still draft?

We have not seen any increase in processing/dispatching events with these RAM changes nevertheless this new setting is something we should keep since 25% is too low. Removed draft on this PR.
/cc @pierDipi


[cli] improve display for sui client new-address command

Description
When requesting a new keypair via sui client new-address, display the newly created keypair in a well formatted table.
Test Plan

How did you test the new or updated feature?

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.
Type of Change (Check all that apply)

[ ] protocol change
[x] user-visible impact
[ ] breaking change for a client SDKs
[ ] breaking change for FNs (FN binary must upgrade)
[ ] breaking change for validators or node operators (must upgrade binaries)
[ ] breaking change for on-chain data layout
[ ] necessitate either a data wipe or data migration

Release notes
When requesting a new keypair via sui client new-address, display the newly created keypair in a well formatted table.

nit: In the screenshot, it looks lie the phrase "Created new keypair and saved it to keystore." is clipping through the table output.  Is that being output to stderr or something? It would be worth cleaning that up if you could.

nit: In the screenshot, it looks lie the phrase "Created new keypair and saved it to keystore." is clipping through the table output. Is that being output to stderr or something? It would be worth cleaning that up if you could.

Good observation. It's unfortunately a limitation of the crate we're using. I will try to see if I can manually draw the vertical lines only for rows 1-3.

Fixed!


Support brew installed chruby

Homebrew (on OSX) installs chruby into /usr/opt/local/chruby. These changes allow the chruby and chruby-auto plugins to work with this while retaining support for the usual location for chruby.

Turns out this isn't necessary after all.


Update manifest.yaml

Added Volume size
[x] This is a fix

@keithhubner can you have a look at the PR and see if it is fine.

This PR is because of https://github.com/civo/kubernetes-marketplace/blob/db0f8d35fb510eaa2017ac3ff36ff621b10ed19a/bitwardenrs/app.yaml#L30 having $VOLUME_SIZE while no value is passed for it

Yeah, looks like it's not needed, not sure why I would have put it in there. As long as it's been tested without, i'm happy to remove.

Well @keithhubner what you would prefer adding volume options to manifest file like I did in this PR or something else

@keithhubner can you check if these can be removed?

Hi, yes i'll try and test tonight.

Yup looks fine.

Good to merge right?
On Tue, 23 Mar 2021 at 2:37 PM, keithhubner @.***>
wrote:

Yup looks fine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/civo/kubernetes-marketplace/pull/167#issuecomment-804738895,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB6PRIUWG443JOGOVDAEANDTFBK35ANCNFSM4ZHXN2YA
.

--
https://www.linkedin.com/in/saiyam-pathak-97685a64/

Yes, can't fully test until it's in the marketplace. But if you merge now I can test this morning.


Can't make neofetch --tycat work

What is neofetch looking for that prevents it from displaying png under terminology when tycat works fine?
I installed debian with no desktop and built window manager from source.  So I probably am missing some dependency, because this exact png and neofetch version works if I let debian installer install xfce or kde, etc.

I just needed to rtfm.  imagemagick.

Interestingly enough, I realized I do not have imagemagick installed on my primary machine (bodhi linux 5 = ubuntu 18) but it works there without it.  So must be something else can make it work too or something that comes with imagemagick... I'm curious...  but imagemagick fixed it for debian anyway.

#1392 (also, imagemagick is always needed (the convert command specifically)).
For next time. Provide logs. :)


Process described is not accurate/working anymore

Hello,
It seems there have been changes with the lastest az cli version making this tutorial not working as expected:
The command to reset the password returns an error
`> SP_SECRET=$(az ad sp credential reset --name "$SP_ID" --query password -o tsv)

the following arguments are required: --id`
Which is consistent with az cli documentation (https://docs.microsoft.com/en-us/cli/azure/ad/sp/credential?view=azure-cli-latest#az-ad-sp-credential-reset)

Using the command "az ad sp credential reset --name "$SP_ID" --query password -o tsv" with an older az cli version works and add a client secret visible on the service principal page in the azure portal.
With the lastest az cli version the command "az ad sp credential reset --id "$SP_ID" --query password -o tsv" works, but no secret is added/visible on the service principal page in the portal.
With the lastest version behavior, the command used to check for the expiration date is not returning anything
az ad sp credential list --id "$SP_ID" --query "[].endDate" -o tsv
Please,

update the documentation page
confirm if the new behavior is as expected
if yes, how can we now check for Aks Service Principal credential expiration?

Thank you

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0b842222-f6ac-5266-227f-7e400610e07b
Version Independent ID: 429cd3bf-2691-f633-2e0d-e70e5d8d0e03
Content: Reset the credentials for a cluster - Azure Kubernetes Service
Content Source: articles/aks/update-credentials.md
Service: container-service
GitHub Login: @zr-msft
Microsoft Alias: zarhoads

Thank you for reaching out. At this time we are reviewing the ask and will provide an update as appropriate

Thanks for reporting this. I have assigned the issue to the content author to investigate further and update the document as appropriate.

Hi @jeromesubs, the doc has since been updated to address your feedback. Please let me know if any issues persist. Thank you for your feedback! I will now close the issue.
#please-close


Don't make notebook copies under datalab path.

If the user makes a copy of a notebook, and that notebook is in the
datalab folder, make the copy in the root.

Change LGTM. We should make it possible to specify name + path in our override in a separate CL.