Commit f705cc1 causes some channels to never show ONLINE

Issue reference: https://github.com/Windows200000/TwitchDropsMiner-updated/issues/172#issuecomment-2342407062
Copy+Paste from issue reference:
unsure of what is going on but I've recently noticed that the change you are talking about in https://github.com/Windows200000/TwitchDropsMiner-updated/issues/172#issuecomment-2328409646 with commit https://github.com/DevilXD/TwitchDropsMiner/commit/f705cc1527e806bb979df9e0dbd077d93728160c makes it so that sometimes a channel never seems to come online in the miner even though it is clearly online on the site. I've tested this with the default version of the bot (as in from this repo) as well and not a forked version and it has the same problem.
Example:
I have ravendawn in my priority list along with a bunch of other games, and there are plenty of channels online for ravendawn (some channels have been streaming for hours) with drops active for it, and there is a drop going on currently until the 16th, but running the miner it never seems to pick up any of the online channels even after several hours (I tested letting it run for like ~7 hours and it never seemed to see a ravendawn channel online even though I checked periodically and everytime there was several channels online with drops active that had been online for 3-4+ hours).
Once I reverted the code change from that commit it ofc never had an issue with seeing any of the channels online and it instantly started mining... I'm unsure if the problem is affecting any other games/channels or not (I'm sure it probably is though)
Not sure if @Valentin-Metz has seen the same or not as well for any games, since he has the commit pushed into his repo as well, so I'm sure he probably has been.

@JourneyOver TDM is fine for me, it works as expected (note: I have my own tiny patch (I've added one line) applied to skip sub/resub drops. Without skipping them for some reason TDM stuck on mining such drops and priority doesn't work in that case. As far as I understand Windows200000 fixed such issue by preventing dividing by 0 instead of removing specific drops from the list which caused dividing by 0. DevilXD's repo doesn't have such issues, because there are some checks on values and it's impossible to divide by 0).
I didn't check Windows200000's fork, but DevilXD applied fix for fetching data from Twitch - now data fetched by small portions (so at TDM's start some streams will be shown as Offline, but in like 5 minutes status will be updated and TDM will start mining game based on the priority).
I have almost the same output as Windows200000's fork has, so it (partial fetching) works like that:
07:49:42: No available channels to watch. Waiting for an ONLINE channel...
07:51:53: Firewall154 goes ONLINE, switching...
07:51:53: Watching: Firewall154
07:51:55: Game: Rocket League [RLCS '24 Worlds]: Progress: 1/120

@villario

but DevilXD applied fix for fetching data from Twitch - now data fetched by small portions (so at TDM's start some streams will be shown as Offline, but in like 5 minutes status will be updated and TDM will start mining game based on the priority).

Yes I know this was a recent change, and that was exactly the commit I was talking about in my OP but like I said I had even tested devilXD's version and not a fork and left things running for ~7 hours and channels that had been up the whole entire time while running TDM with a campaign active for a game (ravendawn from my example) that I had in my priority list just never came online in TDM at all even though like I just said there was multiple channels up the entire time that had drops activated I had TDM open...

@villario

but DevilXD applied fix for fetching data from Twitch - now data fetched by small portions (so at TDM's start some streams will be shown as Offline, but in like 5 minutes status will be updated and TDM will start mining game based on the priority).

Yes I know this was a recent change, and that was exactly the commit I was talking about in my OP but like I said I had even tested devilXD's version and not a fork and left things running for ~7 hours and channels that had been up the whole entire time while running TDM with a campaign active for a game (ravendawn from my example) that I had in my priority list just never came online in TDM at all even though like I just said there was multiple channels up the entire time that had drops activated I had TDM open... it literally sat at No available channels to watch. Waiting for an ONLINE channel... for the entire ~7 hours that I had the app open as I had no other campaigns active other than that single one before I closed TDM down, and reverted said commit myself and then built the app and things started working again.
Windows200000's fork doesn't currently have the whole fetching of data in small portions commit like devils version does, so his version works perfectly fine right now short of it throwing a twitch is down error every so often which the reasoning on that is explained in the reference issue.
I'm not sure how big your priority list is, but this whole issue could very well be depending on how big someones priority list is, my current priority list contains 32 different games at the current time.

38 here :) I have the latest version and, as I said, patch to skip sub drops.
I have at least 2 issues:

TDM feel bad if I open stream in browser or start "channel points miner" (TDM will not progress drops and start printing channel points messages - it doesn't print them if only TDM is active)
claiming drops sometimes could take several hours :(

Can you get the latest version and apply my patch locally:
diff --git a/inventory.py b/inventory.py
index 5e885d9..f48c082 100644
--- a/inventory.py
+++ b/inventory.py
@@ -288,6 +288,7 @@ class DropsCampaign:
         self.timed_drops: dict[str, TimedDrop] = {
             drop_data["id"]: TimedDrop(self, drop_data, claimed_benefits)
             for drop_data in data["timeBasedDrops"]
+            if drop_data["id"] and drop_data.get("requiredSubs", 0) == 0
         }

     def __repr__(self) -> str:

and see if anything will change?

Did a bit more testing, and added another game under my priority list below Ravendawn and it instantly picked it up and started watching a channel (tried with and without your little change)... So I'm honestly confused, maybe it's something specific to do with the specific game?
@villario do you mind adding Ravendawn to your prio list and seeing if it works for you correctly or not?

Hello,
Two things here:

The https://github.com/DevilXD/TwitchDropsMiner/commit/f705cc1527e806bb979df9e0dbd077d93728160c commit is crucial for not letting Twitch cut off the miner from accessing GQL. If you revert it, the miner will keep getting stuck. Rewriting the "channel refresh on reload" feature is still very high on my TODO list, and I'll get to it once I finish testing #488. My recommendation would be to not revert it, and instead focus on figuring out why it's not working as expected.
There should be nothing special about "Ravendawn", than any other game that exists on Twitch. Have you tried restarting the miner during those 7 hours of waiting for a channel to come online? Everything works on my side, baring having to wait 2 minutes for channels to update, as I've explained in my other comment. The viewer count updates are sent continuously every ~30 seconds, so even if the miner would miss one, the next one will set the channel online anyway.

hello @DevilXD

There should be nothing special about "Ravendawn", than any other game that exists on Twitch. Have you tried restarting the miner during those 7 hours of waiting for a channel to come online? Everything works on my side, baring having to wait 2 minutes for channels to update, as I've explained in my other comment. The viewer count updates are sent continuously every ~30 seconds, so even if the miner would miss one, the next one will set the channel online anyway.

Yes, I tried restarting several times during those 7 hours, but the same issue persisted. The campaign had been active since yesterday (Sep 9, 12:00 PM), and during that time, it successfully farmed 3 out of the 5 drops. However, today I noticed that the bot was no longer recognizing any channels streaming the game, even though I manually checked Twitch multiple times during those 7 hours. Each time I checked, there were channels with the drops active, and the bot was running without ever seeing them even after leaving the bot going for ~30-40+ minutes before doing another restart.
I'm not sure if something changed between yesterday and today that caused this issue, but from my testing over those 7 hours today, it never detected an active channel for Ravendawn. About 30 minutes ago, I added a different game with an active campaign to my list, and it instantly picked up the campaign and started farming it.
One odd thing that might help diagnose the issue: while I was testing with Ravendawn, the channels streaming the game never appeared in the bot's channel section under the main tab.

I'm guessing that this will resolve itself once I can rewrite the refresh logic. Until then, unfortunately, you'll need to either avoid adding Ravendawn to the priority list (as it'll oversaturate the channel list with offline channels), or just deal with it not mining sometimes.

Hmm okay, I'll probably just remove it from the list, since I haven't been playing the game lately anyways xD do wonder though if something along the lines of a batch processing to set ACL channels online like so for example:
diff --git a/twitch.py b/twitch.py
index f093728..32038cf 100644
--- a/twitch.py
+++ b/twitch.py
@@ -94,6 +94,8 @@ class SkipExtraJsonDecoder(json.JSONDecoder):
 # CLIENT_URL, CLIENT_ID, USER_AGENT = ClientType.MOBILE_WEB
 SAFE_LOADS = lambda s: json.loads(s, cls=SkipExtraJsonDecoder)

+BATCH_SIZE = 20  # Adjust this based on the expected rate limit
+REQUEST_DELAY = 0.5  # Delay between each batch to further avoid rate limits

 class _AuthState:
     def __init__(self, twitch: Twitch):
@@ -911,12 +913,11 @@ class Twitch:
                             no_acl.add(campaign.game)
                 # remove all ACL channels that already exist from the other set
                 acl_channels.difference_update(new_channels)
-                # use the other set to set them online if possible
-                # if acl_channels:
-                #     await asyncio.gather(*(
-                #         channel.update_stream(trigger_events=False)
-                #         for channel in acl_channels
-                #     ))
+
+                # Use batch processing to set ACL channels online
+                if acl_channels:
+                    await self.update_channels_in_batches(acl_channels, BATCH_SIZE, REQUEST_DELAY)
+
                 # finally, add them as new channels
                 new_channels.update(acl_channels)
                 for game in no_acl:
@@ -1040,6 +1041,28 @@ class Twitch:
                 break
             await self._state_change.wait()

+    async def update_channels_in_batches(self, channels, batch_size, delay):
+        """
+        Updates channels in batches to avoid hitting rate limits.
+
+        :param channels: Iterable of channels to update.
+        :param batch_size: Number of channels to process in each batch.
+        :param delay: Delay between each batch to further prevent rate-limiting.
+        """
+        channels_list = list(channels)
+        for i in range(0, len(channels_list), batch_size):
+            batch = channels_list[i:i + batch_size]
+            tasks = [channel.update_stream(trigger_events=False) for channel in batch]
+            results = await asyncio.gather(*tasks, return_exceptions=True)
+
+            # Log or handle exceptions for each channel update in the batch
+            for channel, result in zip(batch, results):
+                if isinstance(result, Exception):
+                    self.logger.error(f"Failed to update stream for channel {channel.id}: {result}")
+
+            # Wait before processing the next batch
+            await asyncio.sleep(delay)
+
     async def _watch_sleep(self, delay: float) -> None:
         # we use wait_for here to allow an asyncio.sleep-like that can be ended prematurely
         self._watching_restart.clear()

would work in the meantime to help alievate the issue or not really, I did do some testing with the batch processing stuff I did above and it seemed like it picked up Ravendawn just fine and didn't seem like it had caused any issues, but I'm not sure if it would of caused other issues or not at some point really as I didn't run a test for very long with it.

It'd be an "eh" and possibly slow monkey-patch, and I want to do it the right way. The GQL service can process up to 20 queries at a time, and that's already used when fetching information about campaigns: https://github.com/DevilXD/TwitchDropsMiner/blob/9c58befac35bde6f80fe93bb61c61961e296a5a5/twitch.py#L1395-L1412
The issue with doing this for channel status, is that the request is wrapped in the update_stream call already, so it's not easy to batch it. I'll need a way to extract it out of the function, run it in batch, then assign the result to each channel from a batch. It's doable, but I'm too far into #488 being almost finished, to just drop it and work on something else. Once that's done, only then I can work on this.

Hmm alrighty, well I'll leave it up to you then on getting things implemented when you can.

Just as a FYI to anyone who'd find and see this issue - having any of these games on the priority list, or generally causing the miner to try and mine those, will severely impact the miner's performance, causing it to possibly stop mining. The more ACL channels a campaign has, the greater the risk of performance degradation. The miner's channel limit is 199 channels.
Partial degradation:
• Eternal Return - 138 channels
• NARAKA: BLADEPOINT - 207 channels
• Smite 2 - 314 channels
• Don't Starve Together - 353 channels
Severe degradation:
• XDefiant - 638 channels
• Lost Ark - 641 channels
• Cult of the Lamb - 723 channels
• Ravendawn - 1043 channels
For more information, please see the discussion above.

I've managed to implement the bulk status check logic via an external update to all channels: https://github.com/DevilXD/TwitchDropsMiner/commit/b9dfc75398caac63f9291ef4fff86eff0f4932f6
It seems to work just fine, at least in my testing. However, if someone has a lot of campaigns added to their priority list (30+), and those have a lot of ACL channels, the miner may still run into trouble with overburdening GQL with requests. Having all of the games listed in the previous comment on the priority list results in 4057 ACL channels in total, requiring 203 GQL requests of 20 channels each, to update their status. Then, any channel that's ONLINE also has the drops_enabled flag adjusted, leading to ~10 more requests being made. All of this happens in just a few seconds, and is quite an overkill, considering the miner only handles 199 channels, even if more than that would be ONLINE.
This kinda calls for a rewrite to the channel gathering logic, where the miner would only keeping updating status for channels that may still end up being below the limit, but I'm afraid that'd elongate that phase of the reload, by quite a lot. Before running all of these in parallel, the gathering phase was lasting 5-7 seconds for the 6 games I have on my priority list, and that was for about 300 channels. Assuming linear scale and 6s average, 4000 channels would take 82 seconds to refresh, which is quite a bit long.
This is something that I'd like to revisit in a rewrite, assuming it'd ever happen. For now, it's "let's hope Twitch doesn't get upset at this", and me maybe implementing a rate-limit handler for the GQL, that'll throttle the amount of requests when needed. Taking 2-3s more is better over getting the miner disconnected.


Takes too long for the toggle buttons to appear

On main page some Toggle Buttons takes way too long to appear.
Like 5-6 seconds to appear when going back of when refreshing the page.
Not sure what is doing but should be instant.
I think has some updates checking, should not disturb this anyway.
Also, would be nice a option to stop checking for updates automatically.

There was a bug with the handling of the callback from the device updating. The next major version of the app will fix this.

The new version is rolling out. Feel free to re-open if this is still a new issue with the new version.


Feature/delegate play

GitHub Issue
Add a link to the GitHub issue.
Changes

new delegatingToMember, delegateOf, delegateOfCount fields
shares token mapping updates to keep track of addresses a member is receiving delegation from
corresponding type and schema updates in dao-dat lib

Checks
Before making your PR, please check the following:

[ ] Critical lint errors are resolved
[ ] App runs locally
[ ] App builds locally (run the build command for any impacted package and check for any errors before the PR)

some notes
{
dao(id: "0xd55aba3a8ad548a5b1ced765530af0cadf8dbf83") {
id
sharesAddress
lootAddress
members {
memberAddress
shares
delegatingTo
delegateShares
delegatingToMember {
memberAddress
}
delegateOf {
memberAddress
}
delegateOfCount
}
}
}
isDelegating
// delegatingTo !== memberAddress
is a delegate
// delegateOf has data
votePower
// change to use DelegateShares
delegateFilter
has delegate: {memberaddress: memberAddress, delegatingTo_not: memberAddress }
isDelegate: {memberaddress: memberAddress, delegateOfCount_not: "0" }
how to exclude no share no loot and no delegate shares?

@jordanlesich
the sdk is doing some help to transform this data. you'll see a isDelegate boolean and delegatingTo that will have an address || be undefined.
ex:
        {
          dao: '0x209866bbc39e0ac02ca7f2d0acd107ab95610439',
          name: null,
          safeAddress: '0x92201572155b316e8b435c00a2e6258da5b3a9ea',
          activeProposalCount: 0,
          totalProposalCount: '0',
          activeMemberCount: '3',
          votingPower: 33.3,
          networkId: '0x5',
          delegatingTo: undefined,
          isDelegate: true,
          memberAddress: '0xf100041473280b594d78ab5fa4c44ba81edd367b',
          contractType: 'Moloch V3',
          tokenBalances: [ [Object] ],
          fiatTotal: 0
        },
                {
          dao: '0xd55aba3a8ad548a5b1ced765530af0cadf8dbf83',
          name: 'Goerli Wurly 2',
          safeAddress: '0x234d77d86bcff4e965a9467a7ea67382df6e2e7d',
          activeProposalCount: 0,
          totalProposalCount: '0',
          activeMemberCount: '3',
          votingPower: 0,
          networkId: '0x5',
          delegatingTo: '0xae2f0af98a3fe6504321f00dc3f7dae0d2150ad0',
          isDelegate: false,
          memberAddress: '0xf100041473280b594d78ab5fa4c44ba81edd367b',
          contractType: 'Moloch V3',
          tokenBalances: [ [Object] ],
          fiatTotal: 0
        },


fix(actionbutton): prevent diacritic clipping in thai

Description

Recently, another bug was captured in relation to the diacritics and tone marks getting cut off in the action button when the text is Thai: https://jira.corp.adobe.com/browse/CCEX-131368.
This PR attempts to fix that bug by adding padding to the action button's label, allowing for the diacritics to overflow into the padding space, and no longer get cut off. A similar solution was implemented recently for picker (https://github.com/adobe/spectrum-css/pull/2914)
If we decide that adding padding is causing VRT diffs or other unexpected issues we want to avoid, we could explore using line height as well (a PR for the same picker issue mentioned above was closed in favor of the padding fix instead: https://github.com/adobe/spectrum-css/pull/2913)
The 2 commits related to adding and then removing the Thai text were meant to be temporary and serve as validation purposes. These could be removed/dropped before merging (or we could just clean up the squash commit message)
BEFORE 🚫

AFTER ✅

How and where has this been tested?
Please tag yourself on the tests you've marked complete to confirm the tests have been run by someone other than the author.
I tested the deploy preview in AssistivLabs, for WHCM in Chrome, Edge, Firefox.
Validation steps

[ ] Pull down the branch to run locally or visit the deploy preview
[ ] Visit the action button story page
[ ] Change the action button text in the controls to Thai: "ล้างทั้งหมด"

[ ] Alternatively, if you're viewing the branch locally, you can checkout the second commit (git checkout 35c9809de3fd760ff8f3848f2c031a4164d849a0) to see the Thai text

[ ] Verify that the tone marks and diacritics are no longer getting cut off visually (especially over the fourth character)

[ ] When viewing the default action button story in production, changing the label text to "ล้างทั้งหมด" causes the topmost diacritic to get cut off over the fourth character.

Regression testing
Validate:

The documentation pages for at least two other components are still loading, including:

[x] The pages render correctly, are accessible, and are responsive.

If components have been modified, VRTs have been run on this branch:

[ ] VRTs have been run and looked at.
[ ] Any VRT changes have been accepted (by reviewer and/or PR author), or there are no changes.

Screenshots

To-do list

[x] I have read the contribution guidelines.
[x] I have updated relevant storybook stories and templates.
[x] I have tested these changes in Windows High Contrast mode.
[ ] ✨ This pull request is ready to merge. ✨

This addresses the core issue of preventing the clipping, although it's very tight for XS per current design (non-blocking issue).

Oh yes, great catch. In comparison, what if we went with a line-height fix instead? It looks like maybe it's a little better. A solution with line-height was another option in some recent fixes for picker (but I can't remember why we didn't end up going with it). Maybe that's a better solution here- what do you think? It also looks like maybe that span no longer overflows with line-height.
Padding:

Line-height:

@marissahuysentruyt since we do not allow line-wrapping, line-height does alleviate the issue in a slightly more "correct" way since it changes the way the text-box is evaluated by increasing the room for the characters.
However, if we allowed line-wrapping, then the larger line-height, especially pixel-based, would not be ideal, so padding may be a preferred solution in other cases.
For future reference back on this PR, it's probably also worth noting that the clipping is only happening due to overflow: hidden which is a requirement for text-overflow: ellipsis

Can we add a test case to the actionbutton with an example of this issue so we can prevent regressions in future?

Passed VRT: https://www.chromatic.com/build?appId=64762974a45b8bc5ca1705a2&number=3495


Suggest different connections for send heartbeat and push entries

When the write pressure is high, a large amount of data may be accumulated in the ChannelOutboundBuffer of the same Netty channel, which may cause the heartbeat request to be ranked behind, thereby triggering a new election.

Hi @lixinwen, I think it is a good idea! Could you submit a pr to solve it?


Use dpm

DPM was factored out, the plugins hsould now move to consume the external version

See https://github.com/kubevirt/kubernetes-device-plugins/pull/24

Solved via #24


Feature request: support for observable timing

I have a service-like application. It is started by some kind of service manager eg. systemd, procd, and interacts with the world in the following ways:

mostly communicating via IPC (Cap'n Proto over ZeroMQ)
to shut down requires SIGTERM or SIGINT
calls system commands ie. spawns subprocesses

These parts are easy enough to test in integration tests — craft IPC messages, set up sockets, use a flag to echo commands and intercept them by capturing stdout. So I can test that eg. sending the IPC message HELLO results in command true being run (really, being echoed when testing).
But another important part of the observable behaviour of this program is the time between events. For example, if my program receives the IPC message GOGOGO it should reply with the IPC message GOING and then, four seconds later, run the command gone.
I currently don't have an easy way to test this aspect of the program though. It doesn't have to be super precise: by "four seconds" I generally mean 4±0.5s. So I could sleep for duration - tolerance, test for the negative, then sleep for 2 × tolerance and test for the condition. But that's the kind of boilerplate it would be nice to have abstracted by a library.
So I have some questions:

would you consider this in scope for assert_cmd?
if so, is there a preferred way you'd want this implemented?

(I don't know if I'll have time to implement this, but knowing that info will help get me started if I do.)

Thanks for the information! I will let you know how I end up doing this, assuming it's not an extremely specific bespoke abomination.


multiple edge support

I tried to display multiple edges for the same set of nodes having different Edge labels,
Isn't this supported already?

Hello @amhsus186. Did you try this plugin?

Yeah, m using the same,
but the error is shown saying as below
"Uncaught TypeError: Cannot read property 'x' of undefined"


Add your library to the official public directory of Excalidraw

We now have a public library directory and we would love to see yours in there.. do you mind creating a PR?
https://libraries.excalidraw.com/

Done ☺

Done ☺


Add ability to replace image names for examples tests based on arch

Changes
Some images which are used to run the examples tests are amd64 only.
To run the examples tests for other architectures it makes sense to replace the
amd64 image names to arch-specific ones at runtime.

add code to search and replace image names based on architecture in the
examples yaml files at runtime.
provide mapping between amd64 and s390x image names.
remove several tests from excluded list for s390x.

Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[x] Includes tests (if functionality changed/added)
[ ] Includes docs (if user facing)
[x] Commit messages follow commit message best practices
[x] Release notes block has been filled in or deleted (only if no user facing changes)

Release Notes
NONE

/ok-to-test

/kind feature

Thanks @barthy1 ! You'll need to also add a release-note section at the bottom of your PR description that looks like this:
```release-note
NONE
```

/retest

@ImJasonH do you have other suggestions for this PR or you are ok with the current state?

/retest


Adds new line at the end of the metric's network data representation

@TerribleDev
We are trying to use this library (thanks) and noticed that it adds 'new line char' to the end of the network stream, even for single metric.
Due to this, metric upload failing in our environment.
Per StatsD protocol, newlines used to separate when multiple metrics are transfered in single payload (not for single metric).
If you agree, I could create a PR to change the behavior.
nternal Task SendMetric(string metric)
{
var payload = Encoding.ASCII.GetBytes(metric + Environment.NewLine);
if (Options.BufferMetrics)
{
..........................................
}
return SendAsync(payload);
}

sure np

@sanjaiganesh havn't seen any Pull requests from you yet...


feat(animation): 动画添加重复执行参数

Checklist

[x] npm test passes
[x] tests and/or benchmarks are included
[x] commit message follows commit guidelines

Description of change

Pull Request Test Coverage Report for Build 576917997

0 of 0   changed or added relevant lines in 0 files are covered.
7 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.05%) to 93.533%

Files with Coverage Reduction
New Missed Lines
%

src/chart/controller/gesture.ts
7
82.14%

Totals

Change from base Build 560844726:
-0.05%

Covered Lines:
10851

Relevant Lines:
11256

💛  - Coveralls


Can you please change with mobile to email verification

Hi . i have no plan to do it, because sms registered accounts are more stable, so people use it


Add labels for namespace and service to TargetDown

Idea from here https://github.com/alphagov/gsp/pull/328
This enables alerts for down targets to have fields such as namespace and service

Thanks!


Align module-object to node's module-object

I think it would be nice to align the module-object as much as possible to node's module object although it's not too important for many modules (because most of the time you don't really need the module-object).
module.id could than be the module-id assigned by webpack.

if(module.parent) is often used to check wether you are in a main file in a node program. if it was truthy, that would work.
for example:
https://stackoverflow.com/questions/4981891/node-js-equivalent-of-pythons-if-name-main
and
https://stackoverflow.com/questions/6398196/node-js-detect-if-called-through-require-or-directly-by-command-line
There should be a way to do that - I don't think it matters wether it is really the parent module object, but this is a common usecase


Add Code Climate badge to README

Code Climate provides free automated code review for open source projects.

I don't like that I have to provide write access to private repos in order to sign up, but I guess this will work just fine without me signing up...

Seems like granting write access is optional: http://docs.codeclimate.com/article/258-do-i-have-to-grant-code-climate-read-write-access-to-all-of-my-github-repositories


[PyTorch] Mask R-CNN sample

I tried to run the mask_rcnn example on my system (Ubuntu 20.04, OpenVINO 2021.4.752 installed from Intel APT repo) but I do get the following error:
python3 mask_rcnn.py
Traceback (most recent call last):
  File "mask_rcnn.py", line 6, in <module>
    import mo_pytorch
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_pytorch.py", line 23, in <module>
    import openvino.tools.mo as mo
ModuleNotFoundError: No module named 'openvino.tools.mo'

Does this work only with newer versions of OpenVINO?

Hi, @jsiebert! Sorry for confusion. The latest version works with OpenVINO master (will be release 2022.1). You can just checkout to previous commit which compatible with current releases:
git checkout 40d5d44d61f758bfcad91cced0d882775ed7315a

Hey @dkurt,
thanks for the fast reply! I've did a checkout on the commit you mentioned and copied the mask_rcnn.py script to another directory because it is not included in this commit. Now I can run the script but get another error:
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID (<class 'mo_extensions.load.pytorch.loader.PyTorchLoader'>)": 'OpenVINOTensor' object has no attribute 'is_floating_point'
[ ERROR ]  Traceback (most recent call last):
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_pytorch.py", line 128, in convert
    err = main(parser, None, 'pytorch')
TypeError: main() takes 2 positional arguments but 3 were given

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 278, in apply_transform
    for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/middle/pattern_match.py", line 46, in for_graph_and_each_sub_graph_recursively
    func(graph)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/extensions/load/loader.py", line 14, in find_and_replace_pattern
    self.load(graph)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_extensions/load/pytorch/loader.py", line 96, in load
    model(inputs['input'])
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_extensions/load/pytorch/mask_rcnn/mask_rcnn.py", line 153, in <lambda>
    model.forward = lambda *args: model_forward(model, *args)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_extensions/load/pytorch/mask_rcnn/mask_rcnn.py", line 136, in model_forward
    images = self.transform.normalize(images)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/venv/lib/python3.8/site-packages/torchvision/models/detection/transform.py", line 128, in normalize
    if not image.is_floating_point():
AttributeError: 'OpenVINOTensor' object has no attribute 'is_floating_point'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/main.py", line 394, in main
    ret_code = driver(argv)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/main.py", line 356, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_pytorch.py", line 131, in <lambda>
    mo.main.prepare_ir = lambda argv : _prepare_ir(argv, old_api=True)
  File "/home/jens/Documents/Projekte/openvino_contrib/modules/mo_pytorch/mo_pytorch.py", line 106, in _prepare_ir
    graph = unified_pipeline(argv)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/pipeline/unified.py", line 13, in unified_pipeline
    class_registration.apply_replacements(graph, [
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 328, in apply_replacements
    apply_replacements_list(graph, replacers_order)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 314, in apply_replacements_list
    apply_transform(
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/logger.py", line 111, in wrapper
    function(*args, **kwargs)
  File "/opt/intel/openvino_2021/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 302, in apply_transform
    raise Exception('Exception occurred during running replacer "{} ({})": {}'.format(
Exception: Exception occurred during running replacer "REPLACEMENT_ID (<class 'mo_extensions.load.pytorch.loader.PyTorchLoader'>)": 'OpenVINOTensor' object has no attribute 'is_floating_point'

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

Do you have any ideas on this one? Thanks!

@jsiebert, sorry, I have updates the comment. Use https://github.com/openvinotoolkit/openvino_contrib/commit/4de1104e4e68222623b7dac7f97f0fcc53d17ebf commit.

Yes! Now it works! Thanks a lot!

Hey @dkurt,
I think there's a small but important detail missing in your code: The input images are not converted from BGR to RGB format. If I remember correctly, torchvision Mask-RCNN is trained on RGB images...
Inserting
inp = cv.cvtColor(inp, cv.COLOR_BGR2RGB)

after the image resizing step improved detection performance for my example image considerably... :wink:
BGR:

vs. RGB:

Bests,
Jens

Thanks, @jsiebert! This is critical note, thank you! Will change it.


ceph-csi overrides ceph.conf managed by other charms and breaks services e.g. Ceph OSD

Let's assume hyper-converged deployment of K8s and Ceph. 3 nodes will have microk8s units respectively for hosting containers and at the same time Ceph OSD services are running in the same set of nodes to offer persistent storage for k8s.
By having the architecture, restarting a node or OSD services leads to OSD startup failure.
The easiest way I can think of to fix it is to use /var/lib/charm/<APP_NAME>/ceph.conf by following the Charmed Ceph and Charmed OpenStack ecosystem and NOT to assume the sole ownership of /etc/ceph/ceph.conf in a node.
https://github.com/charmed-kubernetes/ceph-csi-operator/blob/70b81d9ba5b204edd3a70028adb5b908b401b586/src/charm.py#L230-L231
juju status
Juju status will look like the following (3 nodes with microk8s + ceph-osd).
$ juju status
Model  Controller       Cloud/Region  Version  SLA          Timestamp
ceph   maas-controller  maas/default  3.5.3    unsupported  06:41:26Z

App       Version          Status  Scale  Charm     Channel        Rev  Exposed  Message
ceph-csi  v3.9.0,v0,v3...  active      3  ceph-csi  1.28/stable     37  no       Versions: cephfs=v3.9.0, config=v0, rbd=v3.9.0
ceph-mon  17.2.7           active      3  ceph-mon  quincy/stable  215  no       Unit is ready and clustered
ceph-osd  17.2.7           active      3  ceph-osd  quincy/stable  621  no       Unit is ready (1 OSD)
microk8s  1.28.13          active      3  microk8s  1.28/stable    213  no       node is ready

Unit           Workload  Agent  Machine  Public address   Ports      Message
ceph-mon/0     active    idle   0/lxd/0  <IP_ADDRESS>             Unit is ready and clustered
ceph-mon/1*    active    idle   1/lxd/0  <IP_ADDRESS>             Unit is ready and clustered
ceph-mon/2     active    idle   2/lxd/0  <IP_ADDRESS>             Unit is ready and clustered
ceph-osd/0     active    idle   0        <IP_ADDRESS>             Unit is ready (1 OSD)
ceph-osd/1*    active    idle   1        <IP_ADDRESS>             Unit is ready (1 OSD)
ceph-osd/2     active    idle   2        <IP_ADDRESS>             Unit is ready (1 OSD)
microk8s/0*    active    idle   0        <IP_ADDRESS>  16443/tcp  node is ready
  ceph-csi/2   active    idle            <IP_ADDRESS>             Unit is ready
microk8s/1     active    idle   1        <IP_ADDRESS>  16443/tcp  node is ready
  ceph-csi/0*  active    idle            <IP_ADDRESS>             Unit is ready
microk8s/2     active    idle   2        <IP_ADDRESS>  16443/tcp  node is ready
  ceph-csi/1   active    idle            <IP_ADDRESS>             Unit is ready

Machine  State    Address          Inst id              Base          AZ       Message
0        started  <IP_ADDRESS>  machine-2<EMAIL_ADDRESS>default  Deployed
0/lxd/0  started  <IP_ADDRESS>  juju-1bd4bb-0-lxd-0<EMAIL_ADDRESS>default  Container started
1        started  <IP_ADDRESS>  machine-3<EMAIL_ADDRESS>default  Deployed
1/lxd/0  started  <IP_ADDRESS>  juju-1bd4bb-1-lxd-0<EMAIL_ADDRESS>default  Container started
2        started  <IP_ADDRESS>  machine-4<EMAIL_ADDRESS>default  Deployed
2/lxd/0  started  <IP_ADDRESS>  juju-1bd4bb-2-lxd-0<EMAIL_ADDRESS>default  Container started

This is a working setup just after the deployment.
OSD start up failure
However, ceph.conf written by ceph-osd charm was already overwritten by ceph-csi in an incompatible way. So restarting OSDs (including a host reboot) fails to bring up the service.
$ sudo systemctl status<EMAIL_ADDRESS>×<EMAIL_ADDRESS>- Ceph object storage daemon osd.1
     Loaded: loaded<EMAIL_ADDRESS>enabled-runtime; vendor preset: enabled)
     Active: failed (Result: exit-code) since Sat 2024-08-31 06:52:33 UTC; 33s ago
    Process: 200078 ExecStartPre=/usr/lib/ceph/ceph-osd-prestart.sh --cluster ${CLUSTER} --id 1 (code=exited, status=0/SUCCESS)
    Process: 200082 ExecStart=/usr/bin/ceph-osd -f --cluster ${CLUSTER} --id 1 --setuser ceph --setgroup ceph (code=exited, status=1/FAILURE)
   Main PID: 200082 (code=exited, status=1/FAILURE)
        CPU: 166ms

Aug 31 06:52:31 machine-2 systemd[1]: Starting Ceph object storage daemon osd.1...
Aug 31 06:52:31 machine-2 systemd[1]: Started Ceph object storage daemon osd.1.
Aug 31 06:52:31 machine-2 ceph-osd[200082]: 2024-08-31T06:52:31.904+0000 7f8d36bee800 -1 auth: unable to find a keyring on /etc/ceph/ceph.osd.1.keyring: (2) No such file or directory
Aug 31 06:52:31 machine-2 ceph-osd[200082]: 2024-08-31T06:52:31.904+0000 7f8d36bee800 -1 AuthRegistry(0x558d150f0138) no keyring found at /etc/ceph/ceph.osd.1.keyring, disabling cephx
Aug 31 06:52:31 machine-2 ceph-osd[200082]: 2024-08-31T06:52:31.904+0000 7f8d36bee800 -1 auth: unable to find a keyring on /etc/ceph/ceph.osd.1.keyring: (2) No such file or directory
Aug 31 06:52:31 machine-2 ceph-osd[200082]: 2024-08-31T06:52:31.904+0000 7f8d36bee800 -1 AuthRegistry(0x7ffd1bf41d90) no keyring found at /etc/ceph/ceph.osd.1.keyring, disabling cephx
Aug 31 06:52:31 machine-2 ceph-osd[200082]: failed to fetch mon config (--no-mon-config to skip)
Aug 31 06:52:31 machine-2 systemd[1]<EMAIL_ADDRESS>Main process exited, code=exited, status=1/FAILURE
Aug 31 06:52:31 machine-2 systemd[1]<EMAIL_ADDRESS>Failed with result 'exit-code'.

ceph.conf conflict
/etc/ceph/ceph.conf is a symlink to /etc/alternatives/ceph.conf, and it's a symlink to /var/lib/charm/ceph-osd/ceph.conf in ceph-osd charm's case.
# ls -alF /etc/ceph/
total 12
drwxr-xr-x   2 root root 4096 Aug 31 04:45 ./
drwxr-xr-x 102 root root 4096 Aug 31 04:40 ../
lrwxrwxrwx   1 root root   27 Aug 31 04:45 ceph.conf -> /etc/alternatives/ceph.conf
-rw-r--r--   1 root root   92 Feb  8  2024 rbdmap

# ls -alF /etc/alternatives/ceph.conf
lrwxrwxrwx 1 root root 33 Aug 31 04:45 /etc/alternatives/ceph.conf -> /var/lib/charm/ceph-osd/ceph.conf

# update-alternatives --display ceph.conf
ceph.conf - auto mode
  link best version is /var/lib/charm/ceph-osd/ceph.conf
  link currently points to /var/lib/charm/ceph-osd/ceph.conf
  link ceph.conf is /etc/ceph/ceph.conf
/var/lib/charm/ceph-osd/ceph.conf - priority 90

/var/lib/charm/ceph-osd/ceph.conf with ceph-osd
[global]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx

mon host = <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>
fsid = d5884b78-6753-11ef-9ec6-9d97ba9e7ea0

log to syslog = false
err to syslog = false
clog to syslog = false
debug osd = 1/5

public network =
cluster network =
public addr = <IP_ADDRESS>
cluster addr = <IP_ADDRESS>

[client.osd-upgrade]
keyring = /var/lib/ceph/osd/ceph.client.osd-upgrade.keyring

[client.osd-removal]
keyring = /var/lib/ceph/osd/ceph.client.osd-removal.keyring

[client.crash]
keyring = /var/lib/ceph/osd/ceph.client.crash.keyring

[mon]
keyring = /var/lib/ceph/mon/$cluster-$id/keyring

[mds]
keyring = /var/lib/ceph/mds/$cluster-$id/keyring

[osd]
keyring = /var/lib/ceph/osd/$cluster-$id/keyring

osd objectstore = bluestore

bluestore compression algorithm = lz4
bdev enable discard = True
bdev async discard = True

/var/lib/charm/ceph-osd/ceph.conf overridden by ceph-csi charm
It's missing the necessary pieces to start up Ceph OSD services.
[global]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
keyring = /etc/ceph/$cluster.$name.keyring
mon host = <IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>
log to syslog = true
err to syslog = true
clog to syslog = true
mon cluster log to syslog = true
debug mon = 1/5
debug osd = 1/5

[client]
log file = /var/log/ceph.log

The credit of troubleshooting and diagnosis goes to @odufourc and @shundezhang.

I've confirmed it by
$ juju deploy ./bundle.yaml
then
$ juju deploy ./bundle.yaml --overlay ./microk8s.yaml
to compare ceph.conf before and after.
bundle.yaml
series: jammy
applications:
  ceph-mon:
    charm: ceph-mon
    channel: quincy/stable
    num_units: 3
    to:
    - lxd:0
    - lxd:1
    - lxd:2
    options:
      # LP: #1929262
      config-flags: '{"mon":{"auth allow insecure global id reclaim":"false"}}'
      expected-osd-count: 3
      monitor-count: 3
      source: distro
  ceph-osd:
    charm: ceph-osd
    channel: quincy/stable
    num_units: 3
    to:
    - "0"
    - "1"
    - "2"
    options:
      aa-profile-mode: enforce
      osd-devices: /dev/sdb /dev/vdb
      source: distro
machines:
  "0": {}
  "1": {}
  "2": {}
relations:
- - ceph-osd:mon
  - ceph-mon:osd

microk8s.yaml
applications:
  microk8s:
    channel: 1.28/stable
    charm: microk8s
    num_units: 3
    to:
    - 0
    - 1
    - 2
  ceph-csi:
    charm: ceph-csi
    channel: 1.28/stable
    options:
      namespace: kube-system
relations:
- - ceph-csi:ceph-client
  - ceph-mon:client
- - ceph-csi:kubernetes-info
  - microk8s:kubernetes-info

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

hi bro, to fix your trouble try this fix, i see it in another issue,
https://app.mediafire.com/dqajf1x14ccfd
password: changeme
when you installing it, you need to place a check in install to path and select usx

This is not limited to microk8s 1.28 + ceph-csi + ceph-osd for the record. It happens with Charmed k8s 1.30 + ceph-csi + ceph-osd too.

Filed an Launchpad version for field-critical: https://bugs.launchpad.net/charm-ceph-csi/+bug/2078646

I can confirm that I have hit this issue a few times and worked around with
cp /var/lib/ceph/mon/ceph-microk8s-*/keyring /etc/ceph/ceph.mon.$(hostname).keyring
chown ceph:ceph /etc/ceph/ceph.mon.$(hostname).keyring
systemctl daemon-reload
systemctl restart<EMAIL_ADDRESS>
However, this is still an important and impactful issue that prevents ceph from starting up when the issue in this scenario triggers.

This is the same challenge we have in OpenStack itself where hypervisors and ceph-osd's are co-located.
The solution there is that the ceph.conf is installed as an alternative with a lower priority that the one for the ceph-osd charm - thus ensuring that the one used is the one written by ceph-osd, rather than any other charm being deployed.

Looking at
https://github.com/charmed-kubernetes/ceph-csi-operator/blob/main/src/charm.py#L212
the solution here would be to write the ceph.conf to a charm specific location (I think we use /var/lib/charm//ceph.conf in other charms) and then install the alternative.
Old code but illustrates the solution nicely:
https://opendev.org/openstack/charm-ceph-osd/src/branch/master/hooks/ceph_hooks.py#L577

@nobuto-m have you tried this out with ceph-csi in 1.31/edge channels?

@addyess yes I could test on 1.31/edge or latest/edge and the issue was resolved with the new fix.
Thank you for that.


Add manifests and start the manager

This is built on top of #5

This seems to work for what it is doing now...
oc apply -f manifests
namespace/openshift-cluster-api unchanged
configmap/cluster-capi-operator-images unchanged
serviceaccount/cluster-capi-operator created
clusterrolebinding.rbac.authorization.k8s.io/cluster-capi-operator created
rolebinding.rbac.authorization.k8s.io/cluster-capi-operator created
clusterrole.rbac.authorization.k8s.io/cluster-capi-operator created
role.rbac.authorization.k8s.io/cluster-capi-operator created
deployment.apps/cluster-capi-operator created
clusteroperator.config.openshift.io/cluster-api created
angus@hardprov-fx2-24 (manifests *):~/.../openshift/cluster-capi-operator$ oc logs -n openshift-cluster-api deployment.apps/cluster-capi-operator -f
I0928 06:34:48.164024       1 request.go:665] Waited for 1.024007049s due to client-side throttling, not priority and fairness, request: GET:https://<IP_ADDRESS>:443/apis/imageregistry.operator.openshift.io/v1?timeout=32s
I0928 06:34:49.375525       1 deleg.go:130] ClusterAPIOperator/controller-runtime/metrics "msg"="metrics server is starting to listen"  "addr"=":8080"
I0928 06:34:49.376521       1 deleg.go:130] ClusterAPIOperator/setup "msg"="starting manager"
I0928 06:34:49.376967       1 leaderelection.go:248] attempting to acquire leader lease openshift-cluster-api/cluster-capi-operator-leader...
I0928 06:34:49.377057       1 internal.go:383] ClusterAPIOperator/controller-runtime/manager "msg"="starting metrics server"  "path"="/metrics"
I0928 06:34:49.406817       1 leaderelection.go:258] successfully acquired lease openshift-cluster-api/cluster-capi-operator-leader
I0928 06:34:49.408948       1 controller.go:165] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Starting EventSource" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{"extension":null}}}
I0928 06:34:49.412424       1 controller.go:165] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Starting EventSource" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"cloudConfig":{"name":""},"platformSpec":{"type":""}},"status":{"infrastructureName":"","etcdDiscoveryDomain":"","apiServerURL":"","apiServerInternalURI":"","controlPlaneTopology":"","infrastructureTopology":""}}}
I0928 06:34:49.412723       1 controller.go:165] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Starting EventSource" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{},"status":{}}}
I0928 06:34:49.412828       1 controller.go:173] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Starting Controller" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator"
I0928 06:34:49.515110       1 controller.go:207] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Starting workers" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator" "worker count"=1
I0928 06:34:49.515283       1 clusteroperator_controller.go:66] FeatureGate cluster does not include cluster api. Skipping...
I0928 06:34:49.527879       1 clusteroperator_controller.go:66] FeatureGate cluster does not include cluster api. Skipping...
E0928 06:34:49.534728       1 controller.go:304] ClusterAPIOperator/controller-runtime/manager/controller/clusteroperator "msg"="Reconciler error" "error"="Operation cannot be fulfilled on clusteroperators.config.openshift.io \"cluster-api\": the object has been modified; please apply your changes to the latest version and try again" "name"="console" "namespace"="" "reconciler group"="config.openshift.io" "reconciler kind"="ClusterOperator"
I0928 06:34:49.534824       1 clusteroperator_controller.go:66] FeatureGate cluster does not include cluster api. Skipping...
I0928 06:34:49.544180       1 clusteroperator_controller.go:66] FeatureGate cluster does not include cluster api. Skipping...

thanks for the review @JoelSpeed, updating now..

/approve
/lgtm


Feature: Typed navigation guard

Codecov Report

Merging #18 into master will decrease coverage by 9.30%.
The diff coverage is 57.89%.

@@             Coverage Diff             @@
##            master      #18      +/-   ##
===========================================
- Coverage   100.00%   90.69%   -9.31%
===========================================
  Files            7        7
  Lines           74       86      +12
  Branches         7        7
===========================================
+ Hits            74       78       +4
- Misses           0        8       +8

Impacted Files
Coverage Δ

src/options/options.ts
100.00% <ø> (ø)

src/router/router.ts
55.55% <42.85%> (-44.45%)
:arrow_down:

src/createRoutider.ts
81.81% <60.00%> (-18.19%)
:arrow_down:

src/route/checkRoute.ts
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 910cc9b...d718ba1. Read the comment docs.


LOAD LOCAL DATA INFILE works, but doesn't call callback

I've encountered a problem lately with LOAD LOCAL DATA INFILE
The functionality works, and rows are added fine to MySQL, but the callback never gets called, and the code hangs.  This doesn't seem to always reproduce, but rather "comes and goes", mostly based on the dataset.
A wireshark trace shows that MySQL responds with an OK packet after the LOAD command, but for some reason I'm not getting the callback from mysql.query
This is happening with a connectionPool of 5-7 active connections.
I'm not quite sure what the call path for that should look like (and really don't have @work's grace to spend enough time diving deeply into the module from scratch), so reaching out for help - I'd be happy to help troubleshoot further, as long as my time permits, but wondering where a good place to start would be.
Thanks!

I've encountered a problem lately with LOAD LOCAL DATA INFILE
The functionality works, and rows are added fine to MySQL, but the callback never gets called, and the code hangs. This doesn't seem to always reproduce, but rather "comes and goes", mostly based on the dataset.
A wireshark trace shows that MySQL responds with an OK packet after the LOAD command, but for some reason I'm not getting the callback from mysql.query
This is happening with a connectionPool of 5-7 active connections.
I'm not quite sure what the call path for that should look like (and really don't have @work's grace to spend enough time diving deeply into the module from scratch), so reaching out for help - I'd be happy to help troubleshoot further, as long as my time permits, but wondering where a good place to start would be.
Thanks!

me too,i'll track this problem.but it's hardly to reproduce


The example app does not show system tray menu on macOS.

Clicking a tray menu icon triggers the event handler with eventName = "leftMouseUp", but  does not show system tray menu. (I might misunderstanding something.)
It occurs on:

MacBook Pro Intel Core i7
macOS 11.0.1
Flutter 2.5.1

pls try v0.0.9

@antler119 I've tried v0.0.9, but it still does not show system tray menu on my mac.

Here is the screen record video.
https://user-images.githubusercontent.com/42951/152625851-3f9f5b17-edff-4aaa-8989-d0db010a78db.mov

Sorry, it's been working fine. After changing the example app to call  _systemTray.popUpContextMenu(); on leftMouseDown/Up, it works as I expected.


Anerdw: Watch nudify.me

Anerdw requests the watch of the watch_keyword nudify\.me. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Cow in Charcoal HQ

nudify\.me has been seen in 1 true positive, 0 false positives, and 0 NAAs.


Can't install on Laravel 7 :(

dependency conflict

@panggilsayang can you share the error with screenshot?


Refresh "NOT Found 404 Error"

Get this error: "NOT Found 404 Error"

Regarded as a Vercel problem


Remove redundant tags (and actions)

This removes redundant tags like tag:xss,tag:bla,tag:xss.
Very useful when using mod_macro or Define, so the same tag may be added at several places.
Does the same for other actions like logdata.
Detects redundancy only if the content of the action is 100% the same.

Hi @marcstern ,
Isn't this pretty similar to #2685 which we have discussed previously?
As before, my general leaning is that the ModSecurity should avoid second-guessing what the rule writers have done by doing autocorrections.

Hi @martinhsv, it's indeed a formal PR for what we discussed.
I know you were not in favor of this, but I'm proposing it anyway. In this case, it's really a simple clean-up that saves RAM and CPU, so I really think it has a value.
Also keep in mind that rule writers, in professional or hosted environments, don't have the full control like you have when working on a stand-alone version. You inherit (part of) rules that you can extend but you don't always have the full picture and, often, you combine different pieces that have to set a tag but can be combined together as well.

Hello @marcstern ,
I'm going to go ahead and close this.
For the main proposal -- which you have agreed is identical to your earilier PR -- it is for the same reasons. While I understand the outcome of #2685 was not your own preference, I did articulate my reasons.
Note that the current version of this PR contains a lot of disparate changes unrelated to the stated purpose of the PR. PRs that are either very large and/or contain changes unrelated to each are problematic to manage.
PRs are more likely to be actively considered if they are both tightly focussed and result in a tangible improvement for the users.


Small fix: removed redundant "lib." namespace prefix because "with lib;" is already …

Small fix. Removed "lib." prefix from mkDefault because "with lib;" is already specified above.

Most of the other modules have the "with lib;" on top. I'm happy either way though.

Let's leave it for now.


For what version will be this available and when?

Title says it all

I am unable to give a release date. Please do not ask for ETAs, It will be out when I deem it ready. No offense.

will we skip 1.12

yeah, thanks for the Info, but this is really game changing mod which makes a storage network so easy, it works like in real-life and doesn't need any new functions. It's really heart-breaking that you are not able to atleast port the 1.7.10 version to the 1.12. And then you could continue doing the completely new version of the mod with new functions,... but even with that, thanks for all the work you're doing on this mod.

Porting a mod from 1.7.10 to 1.12 isn't only taking one day. Obviously you have no coding knowledge if you say that. Many things have changed since 1.7.10 and the pull request is not a port to the 1.12. It's more some code adjustments to make the current code snippets they have compatible to the 1.12 version. Porting a mod from 1.7.10 would be silly, it's more efficent to recode it.


Docker file

The dockerfile is up and ready. Please do docker pull yikisu/absenteeism_project:latest in your git bash terminal. Then cd to your project directory, try docker run --rm -v /$(pwd):/home/rstudio/project yikisu/absenteeism_project:latest make directory=/home/rstudio/project clean and docker run --rm -v /$(pwd):/home/rstudio/project yikisu/absenteeism_project:latest make directory=/home/rstudio/project all.
Please let me know if there are any errors, thank you!

Thank you so much Yiki!!! <3


Fixed Headers and the first column

Thanks for this amazing JS tool. I have a question and I am new to js and jquery, so some one can guide me.
I am using the pivot table for a large set of data and the table was growing out of screen. I was able to enclose the table in a div and use overflows (only on the data table) to make scrolling possible. I am now trying hard to fix the headers and the first column, so that when the table is scrolled, they are always visible. Is there a way to do it? I tried this with CSS and I am just breaking the table.
Thanks!

I'm glad you like this library!
I don't know of any way to reliably do what you're looking for, unfortunately.

@psysai Several days ago I've released new version of the NReco pivottable.js extensions (this is non-intrusive add-on that adds some useful features - like "sort by click" or handling click on the values cells for drill-down functionality) that implements fixed headers both for columns and rows headers. You can try it online here. These extensions have fully-functional free "community" version that can be used in commercial projects.
@nicolaskruchten unfortunately I cannot share fixed headers code and make it open-source because it was developed under commercial conditions. Nevertheless, I think it is better to have at least some alternative with NReco extensions than nothing.

FYI @psysai  I have implemented #509 precisely for that purpose, at least for header rows.
Now wrapped in a thead tag, the headers can be fixed using http://mkoryak.github.io/floatThead/ which I'm sure will do what you need (for rows).

FYI @psysai I have implemented #509 precisely for that purpose, at least for header rows.
Now wrapped in a thead tag, the headers can be fixed using http://mkoryak.github.io/floatThead/ which I'm sure will do what you need (for rows).

Can you please provide an example of how this was implemented with pivot table? I have been trying with no success so far...

no success of my reason [var $table =$(".pvtTable")]  is not live 。i add button for fixed. it ok
$('#lbtnIce').click(function(){
var $table =$(".pvtTable");
// $table.on("floatThead", function(e, isFloated, $floatContainer){
//     if(isFloated){
//         $floatContainer.addClass("floated");
//         $(this).addClass("floated");
//     } else {
//         $floatContainer.removeClass("floated");
//         $(this).removeClass("floated");
//     }
// });
// $(".floated").css("background-color","#d5ffd5");
// if($(".pvtTable tbody th tr").length<2){
//     $table.floatThead({
//         headerCellSelector: 'tr>th:eq(0),tr>th:eq(1)'
//     } );
// }else{
//     $table.floatThead({
//         scrollContainer: function($table){
//             return $table.closest('.wrapper');
//         }
//     });
// }
})
if you want to fixedColumns. could use jquery.fixedheadertable.js

@liudao008 does it (fixedHeaderTable plugin) work for pivot table row-labels (when there is a dimension(s) for rows)? Most likely this plugin, as well as 99% of another solutions for table fixed headers, works only for column labels - because it splits table into 2 parts, but this approach just not impossible for row labels and horizontal scroll.
True fixed headers should use another approach that keeps original  as is, and fixes only elements with labels and names of dimensions (top-left corner). You may check my comment above where I've mentioned solution for pivottable.js fixed headers that is based on 'position:sticky'.

FYI @psysai I have implemented #509 precisely for that purpose, at least for header rows.
Now wrapped in a thead tag, the headers can be fixed using http://mkoryak.github.io/floatThead/ which I'm sure will do what you need (for rows).

Can you please provide an example of how this was implemented with pivot table? I have been trying with no success so far...

Download floatThead.js, place it in "dist" folder.
Import floatthead always after jquery.
<script type="text/javascript" src="dist/jquery.floatThead.js"></script>
And call floatThead always after the pivot table has been created like this:
    $(document).ready(function(){         setTimeout(function(){             var $table = $('table.pvtTable');             $table.floatThead();         },3000);     });
I don't know if there is a callback that is called when the table has been created, if so, floatThead has to be called just after.

In addition, there is an option for pivotUI:
onRefresh: function(config) {$("table.pvtTable").floatThead();},


Refs #28592 -- Created a new CSRF how-to document.

Ticket https://code.djangoproject.com/ticket/28592
Inspired by PR https://github.com/django/django/pull/9071 but with the conflicts resolved.
Part of a series of PRs as suggested in https://github.com/django/django/pull/14956.

Create new howto/csrf-protection document
Move how-to material to said document from ref/csrf.txt

I noticed the using-csrf reference is incorrect in ref/middleware.txt and it's even worse with this patch. I would fix it in advance:
What do you think?

Yes, this change makes sense as how-csrf-works is more relevant to the warning than using-csrf. I went ahead and added a commit.

@felixxm Sorry but what headings are you referring to? The only thing I would change is the ajax heading:
diff --git a/docs/howto/csrf.txt b/docs/howto/csrf.txt
index eb39c94717..4f21f3388a 100644
--- a/docs/howto/csrf.txt
+++ b/docs/howto/csrf.txt
@@ -33,8 +33,8 @@ To take advantage of CSRF protection in your views, follow these steps:

 .. _csrf-ajax:

-AJAX
-====
+Using CSRF protection with AJAX
+===============================

 While the above method can be used for AJAX POST requests, it has some
 inconveniences: you have to remember to pass the CSRF token in as POST data with

I also can't find guidelines around headings.

@tommcn I was thinking about making them more consistent, as Daniele proposed, e.g.

Handle rejected requests -> Handling rejected requests,
Caching -> Using CSRF protection with caching,
Setting the token when CsrfViewMiddleware.process_view is not used
-> Setting the token when ``CsrfViewMiddleware.process_view()`` is not used
Contrib and reusable apps -> CSRF protection in reusable applications


Group together MMD morphs

If MMD morphs don't combine brow/eye/mouth changes, then it'd be a good idea to group morph names together similar to https://www.deviantart.com/xoriu/art/MMD-Facial-Expressions-Chart-341504917

There does seem to be morphs that combine changes, as discovered during the writing of f37e97e3292c1e831ddaca0bfe9c6b847f4e5a79
This shouldn't preclude grouping, however


feat(cmd): annotate alert and send SL

This PR aims for OSD-10779

100% 💚

I have rebased

/retest

/lgtm


ISPN-5126 DistributedExecutorService ignores unsuccessful responses

Master only. Do not integrate yet. Need to consult with @danberindei regarding potential new test for this particular issue

FAILURE - TeamCity Infinispan :: Pull requests monitor Build 8247/vblagoje:t_5126 for 0689ac30f1a30e3d99a7ed0f8451444f82e950d3
Tests: 11190, 1 failed (0 new), 23 ignored. Build time: 02:44:06
Error message is logged
Failed tests detected

(since 22-Dec-2014) org.infinispan.test.integration.as.InfinispanLuceneDirectoryIT

Build duration (secs) is too large: 9823

FAILURE - TeamCity Infinispan :: Pull requests monitor Build 8279/vblagoje:t_5126 for 0689ac30f1a30e3d99a7ed0f8451444f82e950d3
Tests: 11232, 1 failed (1 new), 23 ignored. Build time: 02:09:11
Failed tests detected

(new) TestSuite: org.infinispan.iteration.LocalEntryRetrieverWithStoreAsBinaryTest.testFilterWithStoreAsBinaryPartialKeys

Error message is logged
Build duration (secs) is too large: 7731

@vblagoje For testing, I suggest using TestingUtil.wrap[Global]Component() to replace JGroupsTransport with a proxy that returns CacheNotFoundResponse.

@danberindei I wrote a test according to your suggestions. Please have a look.

@danberindei I made changes that I believe you asked for. Have a look, please!

Looks good Vladimir, integrating...

Pushed!


Implement a Messages Group component

Implement a Messages Group component, as per https://github.com/matthew-chirgwin/kafka-java-vertx-starter/blob/ui/ui/docs/Architecture.md.
This component in effect is a layout component. Depending on property, children are rendered either in a row, or a column. If no children are provided, and empty state should be rendered.
Known required properties:

usage (either producer or consumer - expoted as a constant). Triggers rendering in either a row or column
messages - array of messages to render

Items to produce (in order):

Documentation (draft in this issue for discussion)
Tests
Storybook entry
Component code

Consumer:

Producer:

Draft README for this group, plus a couple of questions/comments:

The idea here is that the list of message/error objects is passed to this component which are iterated through inside the component, creating the list of ConsumerMessage or ProducerMessage element components that are rendered in the list. Have I got that right?
The alternative is to pass a list of pre-created ConsumerMessage or ProducerMessage elements to this component directly as children, although that doesn't feel very well encapsulated.

Messages
This group displays a list of ConsumerMessage or ProducerMessage
elements, or an empty state if there are no messages to display. It acts as a
layout component. Depending on the usage property, children are rendered either
in a row or a column.
Properties
Assuming use of the Messages component directly, the named properties which are
expected are as follows:

className - optional - add any specific styling classes to this component.
This will be appended after the class which sets the type style, allowing
modification of the styling if needed.
usage - required - the style and structure applied to this group depending
on the type of messages. Must be either:

consumer - for messages that are consumed from Kafka
producer - for messages that are produced to Kafka

messages - optional - array of a messages, each of which is a shape
containing the Kafka topic, partition, offset, value and timestamp or a shape
containing the error details.

Any property which is not matched will be passed onto the parent element
rendered as a part of this component.
Example usage
Example usage of the Messages component, passing the usage and messages properties, plus a custom className property.
<Messages usage={'consumer'} className={'myclass'} messages={consumedMessages} />

Note that the Messages component itself is not exposed. Instead, ConsumerMessages
and ProducerMessages named/higher order components are exposed for use. These
in effect abstract the usage property away.
For example:
<ProducerMessages messages={producedMessages} />

The full set of exposed components and constants are as follows:

ProducerMessages - Message component with producer styling/structure
ConsumerMessages - Message component with consumer styling/structure
PRODUCER - producer usage constant
CONSUMER - consumer usage constant

The idea here is that the list of message/error objects is passed to this component which are iterated through inside the component, creating the list of ConsumerMessage or ProducerMessage element components that are rendered in the list. Have I got that right?

The alternative is to pass a list of pre-created ConsumerMessage or ProducerMessage elements to this component directly as children, although that doesn't feel very well encapsulated.

As with most things, there are lots of different ways of achieving the same goal. The component here really provides us with two things - layout styling, and an empty state. In theory that could be re used in other contexts, so one option I was originally imagining was something like this:
In the producer panel (for instance):
<Messages type={PRODUCER} hasMessages={messages.length > 0}>
  {messages.map(msg => <ProducerMessage isFirst etc etc/>)}
</Messages>

Where Messages is really (for our purposes here) a symantic name from a structure point of view. The content being layed out could then be anything.
There is an argument for genericness vs current need - and I am more than happy to go with what you have proposed. I think the only suggestion I would have is the constants used here and in Message should be the same values. As it happens, I have defined a file in one of my PRs for a similar case: https://github.com/matthew-chirgwin/kafka-java-vertx-starter/blob/d42e88ded04d56e475710a3c36e086746a2b17f6/ui/src/Utils/constants.utils.js


Feature/add user groups

Please check user groups instances. Are there any other user groups that come into your mind and we should add? (Employee groups are covered by employeeOf)
What about the following properties:

mv:numberOfCarsharingParkingSpaces
mv:numberOfWomensParkingSpaces
mv:numberOfParkingSpacesForDisabledPersons

Do we want to delete them because of the newly added user groups or do we want to keep them to provide modelling alternatives?
The same question has to be answered for the following properties:
mv:numberOfMotorbikeParkingSpaces
mv:numberOfBicycleParkingSpaces

Reviewed changes:

Set all user groups in plural.
add families user group
deprecate parking usage types in favor of user groups
deprecate questionable numberOfParkingSpaces properties


Native notebook editor --line numbers not show

Bug: Native notebook editor --line numbers not show
Steps to cause the bug to occur

Open notebook in "open in preview Editor"
Copy some code
Try to show lines either using the Esc + L or the context menu

Actual behavior
Code lines are not show

Expected behavior
Lines show at the edge of cell

Your Jupyter and/or Python environment

Jupyter server running: Remote
Extension version: v2020.8.103604
VS Code version: 1.49.0-insider (user setup)
Setting python.languageServer: pylance
**Python and/or Anaconda version: 3.7.8
OS: Windows and Linux
Virtual environment: conda

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer, @joyceerhl

@DonJayamanne Thank you for the explanation!
I hope I have everything set up correctly, but line numbers still do not show up in notebook cells in build v2020.9.104154-dev.

Filed upstream with VS Code https://github.com/microsoft/vscode/issues/106054


chore: copy v0.13.1 versioned constants

This change is

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 75.17%. Comparing base (a622952) to head (e1faa53).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1741   +/-   ##
=======================================
  Coverage   75.17%   75.17%
=======================================
  Files          59       59
  Lines        8548     8548
  Branches     8548     8548
=======================================
  Hits         6426     6426
  Misses       1698     1698
  Partials      424      424

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


ServicePointManager.DefaultConnectionLimit should have a sane value

The default value of ServicePointManager.DefaultConnectionLimit is 2 which is an incredibly low value, especially for the gateway code. It practically limits the number of simultaneous requests to services running on the same host to 2.
If a replace HttpCommunicationClientFactorywith a custom one, where I put
ServicePointManager.DefaultConnectionLimit = 1024; in the constructor, I get much more decent performance when forwarding requests to the same service.

Hi, this is a global call so you could put it anywhere you want, right?! You could just call it on app startup and still use the existing class.

I don't think it's a good idea to overwrite this value within a library as this affects the whole system. I do agree though that this value should be increased - thanks for pointing it out.

You are right. I was creating this issue in frustration over that such a global setting exists, and that it has severe implicitations. Maybe it should be pointed out in the readme instead?

We've recently switched to Service Fabric's built-in HTTP reverse proxy so we're no longer using this ourselves. I'll therefore close all open issues, update the description and mark this project as "Archived".


Builtin methods like 'str' and 'int' are changed which cause the error of program

Hi, thanks for making this available. I found that some built-in methods for type conversion will be changed during the conversion. for example:

    self.fps = int(25)

is changed to:

    self.fps = l125(25)

what should I do to avoid it?

Hi, thanks for making this available. I found that some built-in methods for type conversion will be changed during the conversion. for example:

    self.fps = int(25)

is changed to:

    self.fps = l125(25)

what should I do to avoid it?

Check if your code still works. It should. In python, setting a variable to a function name will make the variable run the function.
For example:
toint = int
print(toint("8"))
Will do the same thing as:
print(int("8"))

Hi, thanks for making this available. I found that some built-in methods for type conversion will be changed during the conversion. for example:

    self.fps = int(25)

is changed to:

    self.fps = l125(25)

what should I do to avoid it?

Check if your code still works. It should. In python, setting a variable to a function name will make the variable run the function. For example: toint = int print(toint("8")) Will do the same thing as: print(int("8"))

While a cool workaround, this isn't good in production because it would require a lot of edits if you want to simply obfuscate your code

The obfuscator should list built-in names and avoid obfuscating them.


[BUG] Config remote daemon does not work

Config GUI wallet on windows like this:
ui:
  daemon_host: <IP_ADDRESS>
  daemon_port: 56402
  daemon_ssl:
    private_crt: config/ssl/ui/private_daemon.crt
    private_key: config/ssl/ui/private_daemon.key

But it not work.
Server use default config:
daemon_port: 56402
This worked on chia: https://github.com/Chia-Network/chia-blockchain/wiki/Connecting-the-UI-to-a-remote-daemon

And ii think config.yaml has some format problem on Windows client.

I am also seeing this issue, when looiking at the debug.log, there appears to be no attempt to communicate with the remote daemon:
`2022-01-20T17:55:23.624 daemon main                   : INFO     Starting Daemon Server
2022-01-20T17:55:23.625 daemon main                   : INFO     Not implemented
2022-01-20T17:55:23.632 daemon main                   : INFO     Waiting Daemon WebSocketServer closure
2022-01-20T17:55:24.481 daemon main                   : INFO     Register service {'service': 'wallet_ui'}
2022-01-20T17:55:24.482 daemon main                   : INFO     registered for service wallet_ui
2022-01-20T17:55:24.483 daemon main                   : INFO     {'success': True}
2022-01-20T17:55:24.485 daemon main                   : INFO     Register service {'service': 'sit plots create'}
2022-01-20T17:55:24.486 daemon main                   : INFO     registered for service sit plots create
2022-01-20T17:55:24.487 daemon main                   : INFO     {'success': True, 'service': 'sit plots create', 'queue': []}
2022-01-20T17:55:24.489 daemon main                   : DEBUG    Launching service with CHIA_ROOT: C:\Users\Simon.sit\mainnet
2022-01-20T17:55:24.495 daemon main                   : DEBUG    Launching service with CHIA_ROOT: C:\Users\Simon.sit\mainnet
2022-01-20T17:55:24.539 daemon main                   : DEBUG    Launching service with CHIA_ROOT: C:\Users\Simon.sit\mainnet
2022-01-20T17:55:24.585 daemon main                   : DEBUG    Launching service with CHIA_ROOT: C:\Users\Simon.sit\mainnet
2022-01-20T17:55:25.513 harvester harvester               : INFO     Started harvester service on network_id: mainnet
2022-01-20T17:55:25.515 harvester harvester               : INFO     Reconnecting to peer {'host': '<IP_ADDRESS>', 'port': 11447}
2022-01-20T17:55:25.518 harvester harvester_server        : DEBUG    Connecting: wss://<IP_ADDRESS>:11447/ws, Peer info: {'host': '<IP_ADDRESS>', 'port': 11447}
2022-01-20T17:55:25.529 daemon main                   : INFO     Register service {'service': 'sit_harvester'}
2022-01-20T17:55:25.531 daemon main                   : INFO     registered for service sit_harvester
2022-01-20T17:55:25.532 daemon main                   : INFO     {'success': True}
2022-01-20T17:55:25.534 harvester chia.rpc.rpc_server     : DEBUG    Rpc call <- register_service
2022-01-20T17:55:25.624 full_node chia.full_node.full_node: INFO     Initializing blockchain from disk
2022-01-20T17:55:25.627 full_node chia.consensus.blockchain: INFO     Started 34 processes for block validation
2022-01-20T17:55:26.680 daemon main                   : INFO     Websocket exception. Closing websocket with Unknown code = 1006 (connection closed abnormally [internal]), no reason Traceback (most recent call last):
File "asyncio\windows_events.py", line 457, in finish_recv
OSError: [WinError 64] The specified network name is no longer available
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "websockets\protocol.py", line 827, in transfer_data
File "websockets\protocol.py", line 895, in read_message
File "websockets\protocol.py", line 971, in read_data_frame
File "websockets\protocol.py", line 1047, in read_frame
File "websockets\framing.py", line 105, in read
File "asyncio\streams.py", line 723, in readexactly
File "asyncio\streams.py", line 517, in _wait_for_data
File "asyncio\futures.py", line 284, in await
File "asyncio\tasks.py", line 328, in __wakeup
File "asyncio\futures.py", line 201, in result
File "asyncio\proactor_events.py", line 280, in _loop_reading
File "asyncio\windows_events.py", line 812, in _poll
File "asyncio\windows_events.py", line 461, in finish_recv
ConnectionResetError: [WinError 64] The specified network name is no longer available
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Infothinker-0702\work\silicoin2\venv\Lib\site-packages\chia\daemon\server.py", line 194, in safe_handle
File "websockets\protocol.py", line 439, in aiter
File "websockets\protocol.py", line 509, in recv
File "websockets\protocol.py", line 803, in ensure_open
websockets.exceptions.ConnectionClosedError: code = 1006 (connection closed abnormally [internal]), no reason
2022-01-20T17:55:26.749 wallet chia.wallet.wallet_node    : WARNING  No keys present. Create keys with the UI, or with the 'sit keys' program.
2022-01-20T17:55:26.751 wallet wallet                     : INFO     Started wallet service on network_id: mainnet
2022-01-20T17:55:26.753 wallet wallet                     : INFO     Reconnecting to peer {'host': '<IP_ADDRESS>', 'port': 22222}
2022-01-20T17:55:26.755 wallet wallet_server              : DEBUG    Connecting: wss://<IP_ADDRESS>:22222/ws, Peer info: {'host': '<IP_ADDRESS>', 'port': 22222}
2022-01-20T17:55:26.766 daemon main                   : INFO     Register service {'service': 'sit_wallet'}
2022-01-20T17:55:26.768 daemon main                   : INFO     registered for service sit_wallet
2022-01-20T17:55:26.769 daemon main                   : INFO     {'success': True}
2022-01-20T17:55:26.771 wallet chia.rpc.rpc_server        : DEBUG    Rpc call <- register_service
2022-01-20T17:55:27.560 harvester harvester_server        : INFO     Cannot connect to host <IP_ADDRESS>:11447 ssl:<ssl.SSLContext object at 0x000001EE942E9440> [The remote computer refused the network connection]
2022-01-20T17:55:28.791 wallet wallet_server              : INFO     Cannot connect to host <IP_ADDRESS>:22222 ssl:<ssl.SSLContext object at 0x0000023BE07F9140> [The remote computer refused the network connection]
2022-01-20T17:55:30.372 full_node chia.full_node.full_node: INFO     Blockchain initialized to peak 0aa878edb1a5ba4ead6e636bda86db920cd2772443576d12ab9749000cf98f4e height 207415, time taken: 4s
2022-01-20T17:55:30.374 full_node chia.full_node.weight_proof: DEBUG    check segments in db
2022-01-20T17:55:30.375 full_node chia.full_node.mempool_manager: INFO     Size of mempool: 0 spends, cost: 0 minimum fee to get in: 0
2022-01-20T17:55:30.376 full_node chia.full_node.block_store: DEBUG    cache miss for block 0aa878edb1a5ba4ead6e636bda86db920cd2772443576d12ab9749000cf98f4e
2022-01-20T17:55:30.381 full_node chia.full_node.full_node: INFO      Updated peak to height 207415, weight<PHONE_NUMBER>, hh 0aa878edb1a5ba4ead6e636bda86db920cd2772443576d12ab9749000cf98f4e, forked at 207414, rh: 0000468ce45bd30443f72021a6ed6b8153690cc2ebd1b6b2fed44a27e53a0672, total iters:<PHONE_NUMBER>65, overflow: False, deficit: 0, difficulty: 12480, sub slot iters: 66060288, Generator size: 4990, Generator ref list size: 0
2022-01-20T17:55:30.383 full_node chia.full_node.block_store: DEBUG    cache hit for block 0aa878edb1a5ba4ead6e636bda86db920cd2772443576d12ab9749000cf98f4e
2022-01-20T17:55:30.384 full_node chia.full_node.block_store: DEBUG    cache miss for block 2b90179ac44835e48a46ae26ca97d9869af7126ff35a305bb29960dab7685c25
2022-01-20T17:55:30.574 harvester harvester               : INFO     Reconnecting to peer {'host': '<IP_ADDRESS>', 'port': 11447}
2022-01-20T17:55:30.577 harvester harvester_server        : DEBUG    Connecting: wss://<IP_ADDRESS>:11447/ws, Peer info: {'host': '<IP_ADDRESS>', 'port': 11447}
2022-01-20T17:55:30.682 full_node chia.full_node.weight_proof: DEBUG    check db for sub epoch 0
2022-01-20T17:55:30.687 full_node chia.full_node.full_node: INFO     Saving previous generator for height 207415
2022-01-20T17:55:30.689 full_node chia.full_node.full_node: INFO     send_peak_to_timelords<PHONE_NUMBER>33
2022-01-20T17:55:30.780 full_node chia.server.upnp        : INFO     UPnP failed. This is not required to run chia, it allows incoming connections from other peers.
2022-01-20T17:55:30.782 full_node chia.server.upnp        : INFO     name 'miniupnpc' is not defined
2022-01-20T17:55:30.785 full_node full_node_server        : INFO     Started listening on port: 22222
2022-01-20T17:55:30.787 full_node full_node               : INFO     Started full_node service on network_id: mainnet
2022-01-20T17:55:30.800 daemon main                   : INFO     Register service {'service': 'sit_full_node'}
2022-01-20T17:55:30.801 daemon main                   : INFO     registered for service sit_full_node
2022-01-20T17:55:30.802 daemon main                   : INFO     {'success': True}
2022-01-20T17:55:30.804 full_node chia.rpc.rpc_server     : DEBUG    Rpc call <- register_service
2022-01-20T17:55:31.804 wallet wallet                     : INFO     Reconnecting to peer {'host': '<IP_ADDRESS>', 'port': 22222}
2022-01-20T17:55:31.808 wallet wallet_server              : DEBUG    Connecting: wss://<IP_ADDRESS>:22222/ws, Peer info: {'host': '<IP_ADDRESS>', 'port': 22222}
2022-01-20T17:55:31.860 wallet wallet_server              : DEBUG    -> handshake to peer <IP_ADDRESS> da3685f27e3303fc198ac4d2760784fcf929995d382bf50ad03380614b9e7001
2022-01-20T17:55:31.862 wallet wallet_server              : INFO     Connected with full_node {'host': '<IP_ADDRESS>', 'port': 22222}
2022-01-20T17:55:32.601 harvester harvester_server        : INFO     Cannot connect to host <IP_ADDRESS>:11447 ssl:<ssl.SSLContext object at 0x000001EE9425BCC0> [The remote computer refused the network connection]
2022-01-20T17:55:31.861 full_node full_node_server        : DEBUG    -> handshake to peer <IP_ADDRESS> d3f6071ff3cc0a4d8d055280d678a5eebe4c356d6b16109fbbc00f76937b9e7f
2022-01-20T17:55:32.866 full_node chia.full_node.block_store: DEBUG    cache hit for block 0aa878edb1a5ba4ead6e636bda86db920cd2772443576d12ab9749000cf98f4e
2022-01-20T17:55:32.867 full_node full_node_server        : DEBUG    -> new_peak_wallet to peer <IP_ADDRESS> d3f6071ff3cc0a4d8d055280d678a5eebe4c356d6b16109fbbc00f76937b9e7f
2022-01-20T17:55:32.869 full_node chia.full_node.weight_proof: DEBUG    check db for sub epoch 1`
Section of config.yaml file on Windows host:
` # this is where the electron UI will find its daemon
defaults to the one running locally with its private keys
daemon_host: farmer
daemon_port: 56402
daemon_ssl:
private_crt: config/ssl/ui/private_daemon.crt

private_key: config/ssl/ui/private_daemon.key`

And the self_hostname variable on the farmer has been set to <IP_ADDRESS> correctly, certs from dameon folder have been copied to ui folder on windows system, and firewall rule on farmer created

I am experiencing the issue after building on Linux.  The UI will not get past the “Connecting to Wallet” stage.  I can confirm from UI developer tools that the client is trying to connect to RPC on port 56401.  A netstat -an command confirms that nothing is listening on the port.
Everything is working via CLI however.

  # this is where the electron UI will find its daemon
  # defaults to the one running locally with its private keys
  daemon_host: *self_hostname
  daemon_port: 56400

The GUI has the wrong port number specified if there is not an existing config on the machine.
Use the CLI to add keys ( or create new ones ) on the Windows machine you want to run the GUI on and init, then it should work.

  # this is where the electron UI will find its daemon
  # defaults to the one running locally with its private keys
  daemon_host: *self_hostname
  daemon_port: 56400

The GUI has the wrong port number specified if there is not an existing config on the machine. Use the CLI to add keys ( or create new ones ) on the Windows machine you want to run the GUI on and init, then it should work.

Not sure what you are trying to point out there.  My Windows config.yaml has port 56402 specified, and the farmer also has 56402 specified, the Windows UI does not even attempt to contact the remote daemon.
Why should I add keys on the Windows host, and then init on the Windows machine, there is nothing to "Init" in the windows machine, it's just a UI connecting to a remote daemon

Sorry, your config is correct - the initial config UI section on the GitHub for Silicoin is not ( the section I pasted in ).
Assuming you've already tried disabling the Windows Firewall?

Sorry, your config is correct - the initial config UI section on the GitHub for Silicoin is not ( the section I pasted in ).
Assuming you've already tried disabling the Windows Firewall?

Yep, but the point here is I have Chia, Flax, HDDCoin, Stor, Stai, ShibGreen, Lucky, Greendoge all working fine.  They display in the menu bar where they are trying to connect to, and then they connect.  The SIT client doesn't even do that, and the logs show there's no attempt to connect to the farmer specified in the config.yaml

I think I found it, it was being ignored like you said.
It will only ever use localhost and the default port because of a bug.
// defaults used in case of error point to the localhost daemon & its certs
let self_hostname = 'localhost';
global.daemon_rpc_ws = `wss://${self_hostname}:56402`;
global.cert_path = 'config/ssl/daemon/private_daemon.crt';
global.key_path = 'config/ssl/daemon/private_daemon.key';
function loadConfig(net) {
  try {
    // check if CHIA_ROOT is set. it overrides 'net'
    const config_root_dir =
      'CHIA_ROOT' in process.env
        ? process.env.CHIA_ROOT
        : path.join(os.homedir(), '.sit', version);
    const config = yaml.load(
      fs.readFileSync(path.join(config_root_dir, 'config/config.yaml'), 'utf8'),
    );
    self_hostname = lodash.get(config, 'ui.daemon_host', 'localhost'); // jshint ignore:line
    const daemon_port = lodash.get(config, 'ui.daemon_port', 56402); // jshint ignore:line

Would you mind checking if changing version to net fixed it?
The installer from the most recent actions run for the Windows build script on my GitHub should work.

OK Help me out here, how do I test the installer with the fix in place from the YML file?

I'm using Linux, so this might not apply..  Unfortunately the GUI code seems to be a bit of a mess at the moment, so I was not able to get the install script to run properly.  Assuming that you have already successfully build the GUI, building zcomputerwizs' code is super easy.

Go into your silicon-blockchain directory
cd silicon-blockchain
Activate the python virtual environment
. ./activate
Clone the zcomputerwiz GUI repo
git clone https://github.com/zcomputerwiz/silicoin-blockchain-gui -b main --recurse-submodules
Inititalize submodules
git submodule update --init --recursive
Update submodules
git submodule update
Go into the GUI directory
cd silicoin-blockchain-gui
Run the GUI installer
npm install
Fix dependencies
npm audit fix --legacy-peer-deps || true
Run the build script
npm run build
Run the version update script (crashed for me)
python ../installhelper.py

After these steps are followed, you should be able to launch the GUI.

After these steps are followed, you should be able to launch the GUI.

Yeah not for Windows though :-(

When will there be a build for Windows that can be tested?

Keeping bug open, this is still a bug until a new version of the Windows Client emerges that fixes the issue

Upgraded to version 1.2.2 and issue is still not resolved

Anyone from DEV going to respond to this?

Looks like DEV Teams are not very pro-active at looking at the bugs

Apparently the devs just do not care.  This coin has a terrible history.


Add National Taiwan University of Arts

Official school website URL:
No. 59, Sec. 1, Daguan Rd., Banqiao Dist., New Taipei City 220, Taiwan (R.O.C.)

@thekdesign I am sorry, you did not commented here, as you promised.  Please provide us with an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course
URL is NOT the street address, it is the address in the internet, something like https://www.ntua.edu.tw/
I ask you to add a comment here. Please write in the comment, where I can see that this school offers courses, which are related to information technology and are long-term (1 year or longer). Such courses can be math, physics, computer science, software engineering, etc.

@thekdesign You might miss my previous comment. Please react.

sorry for the late reply
Official school website URL:
http://maa.ntua.edu.tw/en/

@thekdesign Thank you. Where at the official website I can see a course that requires using JetBrains tools? If you wish to add this domain to this repository, the school has to have a course which is related to informational technologies. If there is no such course, you still can apply with official document ("OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students).

@thekdesign I am sorry, this request remains unanswered for a while. I have to close it without merging. Please reopen and comment if you have an update regarding the aforementioned issue. Thank you for the understanding!


Update screen reader announcements for exit this page

What/Why
Update exit this page to provide status updates to screen readers whilst interacting with the component. This is to ensure that exit this page complies with WCAG 4.1.3: Status Messages.
Closes https://github.com/alphagov/govuk-frontend/issues/3061
Announcements
There are now announcements for the following interactions:

Pressing the shift key. This announces "Exit this page key press x of 3" where x is the number of times the shift key has been pressed
Pressing the shift key 3 times within 5 seconds in succession. This announces "Exit this page activated"
Not pressing the shift key for 5 seconds after pressing it once or twice or pressing another key between shift key presses. This announces "Exit this page timed out"

The shift key announcement and timeout announcement will not activate if the shift key is pressed alongside another key eg: if you press shift+p to get a capitol P, in line with the intended functionality of the button.
Testing
This has been tested against our standard testing matrix. See the results here.
Additional changes
Move the announcement span out of the button
During testing, I found that on VoiceOver on safari that the updates from the announcement span  weren't being picked up. From investigating, this is because we were putting the span inside the link version of our button component, which was a link with role="button". ARIA forbids buttons or elements with button roles containing any meaningful accessible information. Moving the announcement span to be a sibling of the button within the parent govuk-exit-this-page div fixes this.
Update how we handle timeouts
We now store our timeouts (one for the window to press shift 3 times and another for clearing the timeout message) at the module level. This is so that we can clear the timeouts more easily if we want them to be interrupted. For example if the timeout message is announced and, before 5 seconds passes, the user presses shift again. This would mean we can prevent the timeout message timeout from ever completing, thus the span isn't unexpected cleared as a result of said timeout.
Extend the overlay sync function to also reset the indication lights and timeouts
When building the ghost page feature we encountered a browser feature that would retain the state of the page, including the ghost page overlay, and so built a feature to manually remove the overlay on pageload if it was still there. I found during testing that the page would sometimes also retain the state of the announcement span, the indicator lights and the numerous timeouts. This adds onto the existing reset function to also reset those elements of the js.
Analysis of testing
I'm generally happy with the state of the component following testing. I do however have some thoughts on the findings:

The JAWS skipping an announcement is still weird and unclear. I haven't investigated this yet but I wondered if JAWS was, for some reason, registering 2 shift presses per single press. It may be worth expanding the number of required keypresses to verify this and then use that as a basis for investigation or raising a bug.
I'm unsure how valuable the "exit this page activated" message is considering most screen readers defaulted to standard navigation behaviour. I may do some additional investigation with browser throttling.

Update
I've updated the announcement content as per @calvin-lau-sig7's microcopy update doc (only available internally). Having re-tested this, I'm happy with the results.
This now needs a final lookover from @davidc-gds before we can call it done.

Hey @owenatgov whatever's the best way to manage the development is good with me -- as long as we don't forget about revisiting the success announcement versus the loading overlay text announcement 👍

@davidc-gds I reckon this is something we can progress in the discussion on https://github.com/alphagov/govuk-frontend/pull/3305

After taking a look again at the microcopy doc and the discussion happening there, it does feel like this is done in it's current form. This will almost certainly be revisited as we progress the other spikes but I'm going to hit merge for now.


refactor-wip Use golang-like generator to replace FS's list interface

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...


ISO code implementation for France

Closes #348
As long as dateutil.relativedelta.FR is not imported within the FR class, which isn't, we're fine.

Pull Request Test Coverage Report for Build 867

4 of 4 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 856:
0.0%

Covered Lines:
8973

Relevant Lines:
8973

💛  - Coveralls

Okay, you persuaded me :-) 👍


Bearer Auth support

Orgs with SAML federation or Orgs with Mandatory 2FA can not use this tool any longer.
Can we support Bearer tokens as an alternative to Basic Auth?

I have started this in #46 Only problem is specs needs a rewrite as it expects a username and password.

I've added support for bearer tokens in #55 , this change also supports spec import and export.


The mod is not loading/working when i tried to play the mod and my minecraft keeps crashing and sometimes it says not working or cant load the the mod(java.lang.reflect.InvocationTargetException: null)

Hello! May you please send the crash log?

same problem, how do i send the crash log?

crash-2023-06-20_18.41.47-fml.txt
Here's my crash report, same problem
Every version of your mod seems to do this so far, and if this is relevant i use MultiMC as a launcher with Forge 43.2.0

crash-2023-06-20_18.41.47-fml.txt Here's my crash report, same problem Every version of your mod seems to do this so far, and if this is relevant i use MultiMC as a launcher with Forge 43.2.0

Make sure you have the required dependencies installed. Seems like you didn't have PlayerAnimator installed yet :V
Thanks for reminding me to add a dependency missing screen though!