Support for testnet4

This is a good exercise of our Network changes. testnet4 is not yet entirely nailed down, though there is a Core PR that defines some parameters https://github.com/bitcoin/bitcoin/pull/29775. Notably this is a new network which introduces new difficulty-changing logic.
We should sort out our Network stuff and then afterward see how minimally-breaking a change we can make to add Testnet4.
cc https://github.com/bitcoindevkit/bdk/issues/1429

This seems to be an argument for Network being non-exhaustive since this suggests adding another network, Testnet 4?

No, it is an argument for our functions taking impl<Params>, and maybe for Params being non-exhaustive to cover new consensus logic flags.
If it were #[non_exhaustive] how would any existing code have prepared for a new network that had a new difficulty adjustment algorithm that nobody had conceived of until a month ago? They couldn't have, so the best they could've done is add a _ => panic!() branch to all of their matches, meaning that they have untestable panic paths (that suddenly become active due in rust-bitcoin "minor releases" no less).
This has all been litigated to death in the "make Network #[non_exhaustive]" issue. We decided to replace Network with impl Params everywhere that we accept it as a parameter, so that the type is then never directly used. Then users can just recreate it themselves so it doesn't matter how useless we make our version.

@apoelstra thanks, I didn't see #2541 which I guess makes sense.  So it sounds like downstream consumers may implement networks however they want roughly speaking.  Maybe I'm missing it, but some type of doc update to describe the use of Param and the reasoning would be helpful for implementers imo.  Anyway, it seems that testnet4 would deprecate testnet3 and so some of the fields in  pub const TESTNET: Params = Params may need to be changed (which could be a breaking change if testnet4 is radically different).

Disclaimer: I'm still low on time but I saw some people are seriously doing tetnet4 and I felt compelled to join the discussion.
It's a big question what should the Testnet variant mean. Since testnet4 has a different genesis block it already cannot mean "all testnets".
I see these possibilities:

Testnet means testnet3, we add a variant for each subsequent testnet.
Testnet means whichever testnet the latest released Core supports at the time of rust-bitcoin release. If you used testnet3 some future version of our library will rug you and change it to testnet4
Resetting testnet is a breaking change, so if shitcoinists trigger a testnet reset they'll also trigger the whole rust-bitcoin ecosystem breakage.
Stop supporting testnet in the enum entirely because it's too messy and testnet is impractical anyway.
Have a numeric version inside the variant or #[non_exhaustive] enum TestnetVersion { ... } though this is basically a form of a useless variants, just swept under the rug (the practical difference is that matching on "any testnet" is easier")
Remove the whole Network enum, everyone should define their own.

Honestly I think all options are quite terrible. I kinda want option 4 but I guess 5 is more realistic. If testnet4 had infinite inflation (doubling instead of halving) then it would be unusable for scams and I'd be more confident it won't reset again in a very long time and perhaps argue that 2 is fine.

Of all of those I prefer option 1 and I think that's what we'll wind up having by default. I agree that it may result in "useless" variants. And it could be that we have a new testnet every few years, as some people are suggesting on-list as a way to pre-empt testnet coins having a market value.
So option 5 is very interesting, where we sweep all the test networks under a single variant with extra fields to disambiguate.. at some point if we have too much detail we have just recreated Params. But having a numeric version (probably for type safety we want a newtype that can only represent known testnets ... but it should support comparison opcodes etc so people can do ranged comparisons, because I expect most networks will differ in their genesis blockhash and nothing else).
The cost for us to support this isn't too bad so I don't think it's worth throwing stuff away (your options 4 and 6). Nor do I think there's any need to say "ecosystem-wide breakage!" and push major revs for every testnet (option 3). And I definitely don't think we should say "Testnet means latest testnet" and rug people (option 2).
So in my view it comes down to a choice between 1 and 5. Which is more of an aesthetic decision than a technical one.

Funny, I was thinking more along the lines of option 2.  And instead of "rug" I had more imagined deprecation period so implementers would have a chance to update.  I mean, why support old version of testnet once testnet4 is in place other then to give implementations a time to shift.

Option 1 seems like it would make for easier pattern matching (and having to care less about testnet specifics).

@tcharding depends on which code does the matching. If you care about which testnet it is then yes, it is shorter but I don't think it's a huge difference:
match network {
    Network::Testnet3 => ...,
    Network::Testnet4 => ...,
    Network::Bitcoin => ...,
    _ => ...,
}

Vs
use bitcoin::network::Testnet;

match network {
    Network::Testnet(Testnet::V3) => ...,
    Network::Testnet(Testnet::V4) => ...,
    Network::Bitcoin => ...,
    _ => ...,
}

OTOH, if you don't care about version you need to write Network::Testnet3 | Network::Testnet4 and update the code whenever a new version is added.

It also lets you say "any testnet, even yet-unknown future ones" by matching on Network::Testnet(...), which is impossible to do if all the testnets have separate variants.
Doing so will make your code more fragile since it means any future testnet that invalidates your assumptions will break you ... but ok, breaking testnet-only code is not the end of the world. And you'll have plenty of warning in any realistic scenario.
I am warming up to option 5.

- Have a numeric version inside the variant or #[non_exhaustive] enum TestnetVersion { ... } though this is basically a form of a useless variants, just swept under the rug (the practical difference is that matching on "any testnet" is easier")

Option 5 :+1:


after update, how does the inferface change to the high level API?

Hi, developer, i notice that vearch has a update recently, and many API inferfaces are changed. The high level API readme became blank as: https://github.com/vearch/vearch/blob/master/docs/APIVisualSearch.md
How does the inferface change to the high level API? is there any document update?

Previously, we used high-level APIs to encapsulate low-level APIs. Later, we found that this would lead users to learn two sets of APIs, so in the new iteration, we combined the two APIs together. You can refer to low-level APIs, which will meet most of your needs。


ZC8A_005: Option not to pass table name in parameters, but only data

https://blogs.sap.com/2022/08/21/database-update-with-utility-anytab-updatetask/comment-page-1/#comment-635980
The need to pass the tablename as a parameter along with the data. It's not good to make ABAPers care about a type name during both the declaration and runtime. You can get rid of it, though, using RTTI to get the table name out of the type of passed data making only nessessary that there always should be a type directly connected with DB table name.
Target Solution proposed: provide option not to pass tablename in parameters. In case it is impossible to find target table - raise exception.

Description provided in the blog.


No option to "import Jupyter notebook" in file-explorer context menu

Pretty sure we used to have a context menu item when the user right clicked on an ipynb file in the VS Code file explorer to "import Jupyter Notebook as Python script". Seems to have gotten lost (or did we somehow decide to remove it?)

Should rename the menu item too to be more specific about what it's doing.

Verification. ipynb files in Explorer should have a Right Click option on them to import them as Scripts. This command had gone missing for a bit and is now restored.


[fix] Layout doc : missing code

Fix #216

Hi @jtouzy
Thank you again for your PR. I already added this code to the repo. It was ready but due to a wrong merge here the code was deleted. So I will decline this PR.
Thank you again!


python311Packages.omemo-dr: 1.0.0 -> 1.0.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python311Packages.omemo-dr is: OMEMO Double Ratchet
meta.homepage for python311Packages.omemo-dr is: https://dev.gajim.org/gajim/omemo-dr/
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 1.0.1 with grep in /nix/store/29wxiz26797g8y3nkx0dnhq70xn4wnj6-python3.11-omemo-dr-1.0.1
found 1.0.1 in filename of file in /nix/store/29wxiz26797g8y3nkx0dnhq70xn4wnj6-python3.11-omemo-dr-1.0.1

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath
gajim
python310Packages.omemo-dr
python311Packages.omemo-dr

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/29wxiz26797g8y3nkx0dnhq70xn4wnj6-python3.11-omemo-dr-1.0.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python311Packages.omemo-dr https://github.com/r-ryantm/nixpkgs/archive/69cfd52eb3620aea7293719512b245c87d40d700.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/69cfd52eb3620aea7293719512b245c87d40d700#python311Packages.omemo-dr

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/29wxiz26797g8y3nkx0dnhq70xn4wnj6-python3.11-omemo-dr-1.0.1
ls -la /nix/store/29wxiz26797g8y3nkx0dnhq70xn4wnj6-python3.11-omemo-dr-1.0.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

3 packages built:

gajim
python310Packages.omemo-dr
python311Packages.omemo-dr

Maintainer pings

Result of nixpkgs-review pr 266615 run on x86_64-linux 1

6 packages built:

gajim
gajim.dist
python310Packages.omemo-dr
python310Packages.omemo-dr.dist
python311Packages.omemo-dr
python311Packages.omemo-dr.dist


Improve modules to provide a combination of contains() and empty() checks on input parameter object elements

Discussed in https://github.com/Azure/ResourceModules/discussions/2214

Originally posted by maxime-roy October 13, 2022
Hi!
I have a quick question for you, is there a particular reason the contains() function is used in the Microsoft.Network/VirtualNetwork module? With the current implementation, the module doesn't allow us to feed an empty string for a subnet property (like a NSG or a RT id ). This scenario could be useful where we send an array of subnet configuration to the module where not all subnets have a NSG, a route table or a NAT gateway linked to them.
It's something that I could submit a fix for, but I want to know if there was a particular reason it was done this way before working on it :) !
Here's an example for what the module prevents us to do:
module virtualNetworkModule './modules/Microsoft.Network/virtualNetworks/deploy.bicep' = {
  name: 'deploy-virtualnetwork-${uniqueString(resourceGroup().name)}'
  scope: resourceGroup(networkResourceGroupName)
  params: {
    name: virtualNetworkName
    location: networkResourceGroupLocation
    lock: ''
    addressPrefixes: virtualNetworkAddressPrefixes
    subnets: [for subnet in virtualNetworkSubnets: {
      name: subnet.name
      addressPrefix: subnet.addressPrefix
      networkSecurityGroupId: (empty(subnet.networkSecurityGroupName) ?  '' : resourceId(resourceGroup().name, 'Microsoft.Network/networkSecurityGroups' , subnet.networkSecurityGroupName))
      routeTableId: (empty(subnet.routeTableName) ? '' : resourceId(resourceGroup().name, 'Microsoft.Network/routeTables' , subnet.routeTableName))
      serviceEndpoints: subnet.serviceEndpoints
      delegations: subnet.delegations
      privateEndpointNetworkPolicies: subnet.privateEndpointNetworkPolicies
      privateLinkServiceNetworkPolicies: subnet.privateLinkServiceNetworkPolicies
    }]
    tags: virtualNetworkTags
  }
  dependsOn: [
    networkSecurityGroupModule
    routeTableModule
  ]
}

Thank you!

Besides the discussion started as a feature request for the Vnet module, for consistency that would make sense to be implemented for all input parameter objects throughout the library.

Current implementation checks if the input object contains the property. Example:
networkSecurityGroup: contains(subnet, 'networkSecurityGroupId') ? {
          'id': subnet.networkSecurityGroupId
        } : null

Proposed improvement checks both if the input object contains the property and if the property value is not empty. Example:
networkSecurityGroup: contains(subnet, 'networkSecurityGroupId') && !empty(subnet.networkSecurityGroupId) ? {
          'id': subnet.networkSecurityGroupId
        } : null

The team is not sure, if idempotency is always possible with all different resource types. Needs further investigation. @AlexanderSehr , can you do some tests (starting with the vnet)? Thanks!

@eriqua , @rahalan , @AlexanderSehr Thanks for looking into this! If you need more information or want to discuss about this, don't hesitate to contact me! I'll be glad to help you.

Just also commented on the discussion. Fundamentally, the proposal makes sense to me - but I see a potential for errors when it commes to the default value, that is, the one to use if

a property was not provided OR
it was provided, but is empty.

I could imaging that simply using null can lead to a "I don't do anything" behavior, which is definitely not what you want if you want to e.g. nullify a property and cannot set it to empty anymore.

Needs to be tested: how the module will react (try with the Vnet). Check with the new implementation, if making it empty get rid of the object that was set before.

Are there any examples of how people implement this module? It doesn't seem practical to hard-code the nsg resource id in the parameter file, and haven't found a way to dynamically generate properties in the bicep template. AzureBastionSubnet, GatewaySubnet, and AzureFirewallSubnet all break this code. thank you!

We should test if this could be resolved with user-defined-types and their ability to specify something as 'optional'.

It's for this reason, I haven't used the virtualNetwork module yet. I'm looking at lambda functions as the savior, but nothing to post yet regarding a solution.

Are there any examples of how people implement this module? It doesn't seem practical to hard-code the nsg resource id in the parameter file, and haven't found a way to dynamically generate properties in the bicep template. AzureBastionSubnet, GatewaySubnet, and AzureFirewallSubnet all break this code. thank you!

There are examples like the 'simple' deployment I did for a Self-Hosted Agent solution. Based on what you said @ThojoUno I strongly suspect you are refering to something a bit more elaborate than that ;)
Could you provide me with an example of what you'd like to achieve when you're referring to 'dynamically generating properties'?
Is it like you get an array of subnets from somewhere and would like to filter them / add / remove properties, etc. and then pass them into the module?

With AVM, this responsibility was delegated to the module owners. Will archive this for now, but may be covered by spec in the future.


BigQuery: FieldValue.getTimestamp() use of Double causes precision loss in some cases.

Environment details

Specify the API at the beginning of the title (for example, "BigQuery: ...")
General, Core, and Other are also allowed as types
OS type and version: any
Java version: any
google-cloud-java version(s): 1.96.0

Steps to reproduce
Create a row with this timestamp: 1337-09-02 11:43:21.622894 UTC, query and use fieldValue.getTimestamp(). Microseconds wouldn't be equal.
Code example
@Test
public void testFloatingPointPrecisionLoss() {
FieldValue fieldValue = FieldValue.of(Attribute.PRIMITIVE, "-1.9954383398377106E10");
long received = fieldValue.getTimestampValue();
long expected = -19954383398377106L;
assertEquals(expected, received);
}
Any additional information below
Cause - floating point conversation causes it; need to use BigDecimal (or equivalent) to perform lossless conversion.

Sorry, but it's not a question, it's a bug...

退订

This is a bug that necessitates changes in the API backend.  There's work in progress to address this.

I think it's much simpler than that: in FieldValue.getTimestamp() replace
new Double(Double.valueOf(getStringValue()) * MICROSECONDS).longValue()
with:
BigDecimal secondsWithMicro = new BigDecimal(getStringValue());
BigDecimal scaled = secondsWithMicro.scaleByPowerOfTen(6);
return scaled.longValue();
Obviously, can be made faster with a bit of parsing by hand rather than using BigDecimal.


Initializing attributes by default

Is it possible to get virtus to do something like:
require 'virtus'

class Address
  include Virtus
  attribute :street,      String
  attribute :city,  String
end

class User
  include Virtus
  attribute :name, String
  attribute :address, Address
end

u = User.new
fail unless "" == u.name
fail unless "" == u.address.city

I'm building a form handling library based on virtus, and not having to deal with nil's would make this a lot easier.

What about specifying the default as an empty string, eg:
attribute :name, String, :default => ''

Hm, so
attribute :address, Address, default => Address.new
?

You could do:
attribute :address, Address, :default => proc { Address.new }

Although I wouldn't be surprised if your approach works too. Virtus will dup the default object if it can, so that anything specified as a default won't be a singleton and shared across multiple instances.

@dkubb if ´Addressewas aVirtus::ValueObject` wouldn't it make sense to reference the same in-memory object since it can't be modified anyways?

@senny I wonder if we should make it so value objects are also Singleton instances? That way there is only ever one copy in memory at a time. That way we're leveraging something from stdlib rather than inventing our own semantics for the objects.

oh wait, nevermind that won't work. Singleton instances are 1:1 with the class names, while value objects only need to have one instance per distinct value.

@senny so yeah, to answer your question, yes, I think we should do that :) What I'm thinking is we should make it so that #dup and #clone return self so that inside Virtus::Attribute::DefaultValue when it clones the object it just gets back the same instance. For a value object in other contexts I think that's valid behaviour.

@dkubb Thanks!  This is what I've got so far for my form library.  https://img.skitch.com/20120209-pewtuuecpkubktpnts2iinbbu1.png (same idea as https://github.com/joshsusser/informal, but better)


Action packs.uninstall does not return valid JSON

SUMMARY
Running st2 run packs.uninstall packs=examples -j does not return well-formed JSON data.
ISSUE TYPE
Bug Report

STACKSTORM VERSION
Paste the output of st2 --version:
st2 3.0dev (2f5fa9b), on Python 2.7.12
OS / ENVIRONMENT / INSTALL METHOD

Installed on Ubuntu 16.04 with st2vagrant.
STEPS TO REPRODUCE

Fire up Ubuntu 16.04 VM with st2vagrant.
Run:# sudo cp -r /usr/share/doc/st2/examples/ /opt/stackstorm/packs/
# st2 run packs.setup_virtualenv packs=examples -j
# sudo st2ctl reload --register-all
# st2 run packs.uninstall packs=examples -j

EXPECTED RESULTS

Valid JSON.
ACTUAL RESULTS

Invalid JSON:
{
    "action": {
        "ref": "packs.uninstall"
    },
    "end_timestamp": "2019-04-18T02:52:47.720613Z",
    "id": "5cb7e67ca08f813f457b8a25",
    "parameters": {
        "packs": [
            "examples"
        ]
    },
    "result": {
        "exit_code": 0,
        "result": null,
        "stderr": "st2.actions.python.UninstallPackAction: DEBUG    Deleting pack directory \"/opt/stackstorm/packs/examples\"\nst2.actions.python.UninstallPackAction: DEBUG    Deleting virtualenv \"/opt/stackstorm/virtualenvs/examples\" for pack \"examples\"\n",
        "stdout": ""
    },
    "result_task": "delete packs",
    "start_timestamp": "2019-04-18T02:52:44.274245Z",
    "status": "succeeded"
}
[
    {
        "action": "packs.unload",
        "id": "5cb7e67ca08f813fa9f006a3",
        "start_timestamp": "2019-04-18T02:52:44.527055Z",
        "status": "succeeded (1s elapsed)",
        "task": "unregister packs"
    },
    {
        "action": "packs.delete",
        "id": "5cb7e67ea08f813fa9f006a5",
        "start_timestamp": "2019-04-18T02:52:46.645818Z",
        "status": "succeeded (1s elapsed)",
        "task": "delete packs"
    }
]

The object and the array are both well-formed JSON, but simply concatenating them together does not constitute result in valid JSON.

I ran into this when refactoring the BATS ChatOps tests to use jq.
Related to but distinct from #4218, because in this issue the command completes successfully.

I'm just checking issues reported for Ubuntu Xenial and tried to reproduce this issue.
It's is still around with st2 v3.5.0. Tested on U18 and U20. So I won't close it.


Question on how to best handle sBIT=10 data

Hi,
Thanks for the nice library. I am trying to use it for 10bit images and am wondering what the recommended procedure is. If I read it with read, the data I get is in np.uint16 and rescaled which it seems is the intended default behaviour (http://www.libpng.org/pub/png/book/chapter11.html#png.ch11.div.7). I am interested in getting the raw values though. If I read it with asDirect, the values seem to be the original ones but the type is apparently int64. What am I missing?
Below is a code snippet demonstrating this behaviour:
import png
import numpy as np
import imageio

# Create simple image
bitdepth = 10
im = np.zeros((5,5),dtype=np.uint16)
im[2,2] = (1<<bitdepth)-1
#print(im)

# Write png
metadata = { "bitdepth": bitdepth, }
filename = 'gray10-test.png'
png.from_array(im,'L',metadata).save(filename)

# Write png as 16bits
filename_as16 = 'gray10-test-as16.png'
png.from_array(im,'L').save(filename_as16)

# Read it back
r = png.Reader(filename)
w, h, pixels, metadata = r.read()
print('read',w, h, metadata)
im = np.vstack(list(map(np.asarray, pixels)))
print('read',np.min(im),np.max(im),im.shape,im.dtype)

# Read it back using asDirect
r = png.Reader(filename)
w, h, pixels, metadata = r.asDirect()
print('asDirect',w, h, metadata)
im = np.vstack(list(map(np.asarray, pixels)))
print('asDirect',np.min(im),np.max(im),im.shape,im.dtype)

# Check with imageio
im_io = imageio.imread(filename)
print('imageio',np.min(im_io),np.max(im_io),im_io.shape,im_io.dtype)

im_io16 = imageio.imread(filename_as16)
print('imageio with as16',np.min(im_io16),np.max(im_io16),im_io.shape,im_io.dtype)

im_io_scaled = (im_io >> 16-bitdepth)
print('scaled from imageio',np.min(im_io_scaled),np.max(im_io_scaled),im_io_scaled.shape,im_io_scaled.dtype)

with corresponding output
read 5 5 {'greyscale': True, 'alpha': False, 'planes': 1, 'bitdepth': 16, 'interlace': 0, 'size': (5, 5)}
read 0 65535 (5, 5) uint16
asDirect 5 5 {'greyscale': True, 'alpha': False, 'planes': 1, 'bitdepth': 10, 'interlace': 0, 'size': (5, 5)}
asDirect 0 1023 (5, 5) int64
imageio 0 65535 (5, 5) uint16
imageio with as16 0 1023 (5, 5) uint16
scaled from imageio 0 1023 (5, 5) uint16

On a related note, as a first time user of the library, I looked at the code examples here but stumbled onto some deprecation warning which I silenced by introducing a call to list in between vstack and map when creating the numpy array. Not sure if this is the best way but it might be good to have a fix for this warning in the documentation.
Best wishes,
Tom

One point worth knowing is that in PNG format, the "raw values" as you say are either 8-bit or 16-bit (or, in grayscale cases, 1-, 2-, or 4-). I guess that is raw from the perspective of the PNG format, not from the perspective of whatever the originating imaging device is. So when there is 10-bit data, it's actually 16-bit data with an sBIT that says interpret as 10-bit.
From what you've said it looks like everything is working as document from PyPNG's side, although i agree it may be less than helpful for people using numpy.
It is worth bearing in mind that PyPNG has nothing to do with numpy and as the numpy module offends my sensibilities, it is the result of a delicate compromise that the documentation mentions numpy at all. So we should clear up one misconception: read() is not returning numpy.unit16 values, it is returning ... well a type i would rather not discuss. But see below.
I've now run your code example, very useful. What is happening:

read() returns each row as a Python array.array instance (specifically type H, 16 bit unsigned);
asDirect() has scaled each pixel value, returning each row as a plain Python list of int.

In the first case numpy "cleverly" notices that the values are a Python array.array instance and therefore translates those array values to numpy 16-bit values.
In the second case numpy only sees a list of plain int, it therefore cannot know what size the actual numeric values are (Python ints have infinite range and prevision), it therefore conservatively chooses its default int type, which is a 64-bit type.
PyPNG (me, really) is under no obligation to make any guarantees about the exact type of the rows it returns, other than it will be a Python sequence of integer values. Therefore if it choose to return an array.array in some cases and a plain Python list in other cases, then that is fine.
You should not rely on any particular behaviour. So, moving on to how we should deal with this in numpy.
Probably the best thing to do is fix the type on the numpy side. That looks like:
im = np.vstack([np.asarray(row, dtype=np.uint16) for row in pixels])

A couple of further observations:
On memory. it seems foolish of numpy to deprecate using vstack on iterables, it would have been the natural way to efficiently stream values into a numpy array. Converting the rows into a list, as numpy will soon require, means that the entire image will be stored as a list of list of Python ints, even if only temporarily. This will be seriously memory intensive (roughly 2 64-bit words per pixel on a 64-bit platform).
On speed (and memory). It will be faster and more memory efficient (as it happens) to do the scaling in numpy.
row >> 6

will rescale a row.
Thanks for the heads-up on numpy.vstack why numpy keeps deprecating perfectly good clear code i do not know, but it is a minor source of annoyance every couple of years or so.

Thanks for the detailed and clear answer.
Is there a simple way to use the faster (as far as I understood your response) read + scaling back in numpy but know the underpinning bitdepth? Indeed, I would need to know the original bitdepth to know what the scaling factor is. Yet using read led to a bitdepth of 16 in the returned metadata. Using read for the data and asDirect for the metadata seems a bit counter-intuitive to me.
Anyway, it looks like if speed is not really a concern, using asDirect and providing the expected dtype to np.asarray would be a suitable approach.


Feature: Add control socket

Currently, the control socket via fuse is limited:

fuse layer caches results
can not predict response size without running command
can not use size 0 on macos, as then nothing can be read

Add unix socket support, outside fuse mount
Server:
$ FUSE_PYTHON_API=0.1 PYTHONPATH=. python tests/test_socket_control.py

Client:
$ echo status | socat - UNIX-CONNECT:/tmp/ControlListener.sock
$ echo reload | socat - UNIX-CONNECT:/tmp/ControlListener.sock

is there memory leak or memory overwrite? why it would try close on urllib3 object?

^C
Exception in thread Thread-1 (handle):
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 1045, in _bootstrap_inner
    self.run()
  File<EMAIL_ADDRESS>line 982, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/glen/scm/plex/fuse/plex-fuse/plexfuse/control/ControlListener.py", line 61, in handle
    conn, addr = self.socket.accept()
                 ^^^^^^^^^^^^^^^^^^^^
  File<EMAIL_ADDRESS>line 294, in accept
    fd, addr = self._accept()
               ^^^^^^^^^^^^^^
ConnectionAbortedError: [Errno 53] Software caused connection abort
Main thread exiting
Traceback (most recent call last):
  File<EMAIL_ADDRESS>line 666, in _exitfunc
    f()
  File<EMAIL_ADDRESS>line 590, in __call__
    return info.func(*info.args, **(info.kwargs or {}))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/glen/scm/plex/fuse/plex-fuse/.direnv/python-3.11/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1184, in _close_pool_connections
    conn.close_socket()
    ^^^^^^^^^^^^^^^^^
AttributeError: 'HTTPSConnection' object has no attribute 'close_socket'
➜


Partials are missing from the guide

I just discovered them in the API, but they are missing in the guide.

I wonder what their advantage over components are
Seemingly they are not available in 2.0 anymore: https://github.com/vuejs/vue/issues/2873

There are no real advantages to partials. They're a fallback for components without a root element in 1.x and offer only a subset of the features normally available to a component. A warning about their limitations could be a good addition to the components guide.

Whoops! Not sure what I was thinking when I wrote this earlier. It seems "partials" were translated to "fragment instances" in my mind somehow. 😅  I think I'd prefer to leave partials out of the guide for now, since they deprecated in 2.0 and can be abused sometimes. Sticking to the component model is a better way to go almost all the time.


JSONDecodeError when getting Bad Request (400) response from API

Describe the bug
A clear and concise description of what the bug is.
When I make a call to /products/fees/v0/listings/Some_SKU/feesEstimate via get_product_fees_estimate_for_sku with a SKU containing a space, the API can't handle the call (I think there is still no workaround) and answers with a 400 response with no content, so in
def _check_response(self, res) -> ApiResponse:
    if self.method == 'DELETE' and 200 <= res.status_code < 300:
        try:
            js = res.json() or {}
        except JSONDecodeError:
            js = {'status_code': res.status_code}
    else:
        js = res.json() or {}

in the else branch a JSONDecodeError is thrown.
Is there any need for the if / else ? Why not always catch the Exception?

This was solved in https://github.com/saleweaver/python-amazon-sp-api/pull/845
We're waiting for a release 😃

Really sorry, totally forgot. @rmanganiello 😀
Released now


How get Google Calender date event(s)?

Description of the Issue and Steps to Reproduce:
Please include the values of all variables used.
Environment:
*Firefox 56
*Arch Linux x64
Both the browser and the OS are important to us, particularly if you have an unsual environment like an IOT application.
Other information that may be helpful:
*Timezone: Europe/Amsterdam
*30-10-2017 11:14
If you are reporting an issue, please run the following code in the environment you are using and include the output:
start:  app.js:6252:5
Object { _isAMomentObject: true, _i: "2017-10-30T00:00:00+02:00", _f: "YYYY-MM-DDTHH:mm:ssZ", _tzm: 120, _isUTC: false, _pf: Object, _locale: Object, _d: Date 2017-10-24T22:00:00.000Z, _isValid: true }  app.js:6253:5
end:  app.js:6255:5
Object { _isAMomentObject: true, _i: "2017-10-31T23:59:59+02:00", _f: "YYYY-MM-DDTHH:mm:ssZ", _tzm: 120, _isUTC: false, _pf: Object, _locale: Object, _d: Date 2017-10-26T21:59:59.000Z, _isValid: true }

This would be a date event for 30-10-2017.
Ensure your issue is isolated to moment. Issues involving third party tools will be closed unless submitted by the tool's author/maintainer.

Found solution:
if (startDate.calendar() === 'Today') {
        return 'Ends Today';
}

Thanks for creating moment! :)


Stop setting public/admin_workers in keystone.conf

Keystone is running under uwsgi, so setting public_workers or
admin_workers in keystone.conf has no effect and is confusing
since someone might think that changing this value will have
some effect.

Can one of the admins verify this patch?

"ok to test" to accept this pull request for testing
"test this please" for a one time test run
"add to whitelist" to add the author to the whitelist

ok to test

retest


Stringtie --merge

So I compared the new --merge option with cuffmerge on the same set of GTF files.
Here is what I get:
Hisat 2.0.0 + Stringtie + Stringtie --merge
parsed genome node DAGs: 20343
sequence regions: 804 (total length: 235278766)
genes: 19537
mRNAs: 52981
exons: 310477
Hisat 2.0.0 + Stringtie + Cuffmerge
parsed genome node DAGs: 18122
sequence regions: 815 (total length: 235906968)
genes: 17307
mRNAs: 61860
exons: 460638
Any idea what is going on? Aren't they supposed to be equivalent? is there a reason to assume that the output from Stringtie is superior?

note: this was without the --dta option in hisat2

If you used annotation, Cufflinks will put in the merge file all genes from the annotation file, while StringTie will not add the annotated genes in the samples for which there is no gene expression in any of the samples.
Other than that, Cufflinks and StringTie both rely on their own algorithms to merge transcripts, and they both have their own filtering methods to eliminate assembled transcripts that they believe to be spurious. At this time, we didn't do any testing to compare which one is superior.

Thanks @mpertea,
It would be sweet to have an option to include all of the genes from the annotation file. We use cuffmerge to generate a GTF file with the known assembly augmented with the newly assembled transcripts and having missing transcripts from the known assembly will lead to endless questions from people.

Yes, I understand. I just talked to the other developers of StringTie and we will include all annotation in v1.2.2 (coming soon).

Thanks. By the way, great work guys Stringtie (& co) are awesome tools.
Prime examples of the awesomeness of the General Intellect !

This was added in v1.2.2, if -G was used with --merge, all reference transcripts which did not "merge" with any assembled transcripts will be appended to the output file, with the original transcript and gene IDs.
This also happens now in without --merge, if the -e option ("estimate only") was given.
Similarly, the -A gene abundance file should always list all the genes from the annotation file (with 0.0 cov/FPKM/TPM if not expressed).

This was added in v1.2.2, if -G was used with --merge, all reference transcripts which did not "merge" with any assembled transcripts will be appended to the output file, with the original transcript and gene IDs. This also happens now in without --merge, if the -e option ("estimate only") was given. Similarly, the -A gene abundance file should always list all the genes from the annotation file (with 0.0 cov/FPKM/TPM if not expressed).

@gpertea Thank you very much! I was wondering how to create a cell-type-specific GTF annotation, and now I understand that I should not include the '-G' option in the 'stringtie --merge' command.

This was added in v1.2.2, if -G was used with --merge, all reference transcripts which did not "merge" with any assembled transcripts will be appended to the output file, with the original transcript and gene IDs. This also happens now in without --merge, if the -e option ("estimate only") was given. Similarly, the -A gene abundance file should always list all the genes from the annotation file (with 0.0 cov/FPKM/TPM if not expressed).

@gpertea Thank you very much! I was wondering how to create a cell-type-specific GTF annotation, and now I understand that I should not include the '-G' option in the 'stringtie --merge' command.

@gpertea Dear Gpertea, I just realized that without the '-G' option in the 'stringtie --merge' command, the output merged GTF file only contains "MSTRG.number" annotations instead of the standard gene names or gene symbols. I understand that "MSTRG.number" annotations are used for novel transcripts, but for known transcripts, is there an option to use gene names or gene symbols instead of the "MSTRG.number" annotations?


"Open in Editor" button .md file is missing response text at the end

What happened?
Hello, the generated .md file does not have all the response from the original response.

I'm using this because of #225
Relevant log output or stack trace
No response
Steps to reproduce

Start a new chat.
Ask for a long text 600 lines long.
Click on the "Open in Editor" button.
See the end of the .md

CodeGPT version
2.2.6
Operating System
macOS

Hmm, this issue might be related to the concurrency issues introduced following the chat tool window UI update.
The completion events that triggered set of UI updates were not running on the Event Dispatch Thread (EDT).
Please upgrade the plugin to v2.2.8, this version should be more stable.

Sorry mate, v2.2.8 does not fix it 😢


Split docs/advanced_usage.rst into several files

Move each section from advanced_usage.rst into its own .rst file.
Add each .rst file to docs/index.rst
Build the docs locally to ensure that every doc shows up properly

Is there any preference as to filesystem layout or naming of these new files?

No preferences, just do what you think makes the most sense. We can always change names later if needed.
I could see them either going directly into docs/ or into docs/advanced_usage/, but again no real preference here.

Closed via #749.


Integration with Music APIs

Integrate your music player with external music APIs (e.g., Spotify, Last.fm) for additional features and data.
Difficulty: Advanced

@Tejas242 Assign this to me please.

@tsongtheng you are working on this issue or not ?? if not please assign this issue of i am interested

Hi @Rohit-Sharmaa , I am working on this project right now. I will let you know if I'm stuck at some point. You can work on this issue after that.

Hey, I was thinking, could you incorporate a feature that allows users to switch to locally stored songs(automatically or manually) in case the app can't connect to the API?

Ok, I will try it if the integration of music APIs is successful.

Hey @Tejas242, can you assign this to @Rohit-Sharmaa. I am unable to complete the implementation. I will be working on other issues.

@tsongtheng assign this to me


Supporting more Frameworks

We want every framework to have a quality component library, but our resourced limited. Community support is vital for Ugnis, so we will work on adding the most upvoted frameworks first.
Thumb up the ones you use and feel free to add additional new entries.

Vue

Angular 1.x

Angular >2

Ember

Cycle

Elm

Preact

not a framework, but coffeescript

Mithril


New autocomplete mode doesn't suggest system tables

Description
New autocomplete mode doesn't suggest system tables, for example, pg_class.
Start typing: select * from pg_c
press Ctrl + Enter
Expected: "pg_class" is in a list of suggestions
Actual: No suggestions at all
DBeaver Version
Community Edition <IP_ADDRESS>407122322
Operating System
Windows 10
Database and driver
PostgreSQL 16
PostgreSQL JDBC Driver 42.7.2
Steps to reproduce
No response
Additional context
No response

Thank you for bug report.

We have the same request from our customers. Ticket number on our internal support system is 387162.


[ASTPrinter] Print parameter type attributes more often

This makes type attributes print properly in usage like typealias T = (@autoclosure () -> Int) -> Void which affects Xcode quick help for example.
Existing tests pass locally (except a couple that fail better, see diffs,) new tests pass.
Toolchain in Xcode shows right thing in quick help.
@milseman I’m concerned this is undoing some of your changes from #4905 / 8923a12585581324dfbd5e6770866fb3e9e7c5b4 to apparently no ill affect.  Maybe other code has changed, or you can point me towards scenarios that introduce the unwanted @noescapes that #4905 was masking out from TypeRepr printing?
Resolves SR-6321.

Why nuke the flag instead of setting turning it off when performing this request?

Well, I thought because it was only added for this purpose.  You prefer to leave it in for use later on?

polite ping (I forgot about this...)

Now that I read this again, if the goal was to remove the effects of printing @noescape everywhere, now that it’s the default this should be good to take as-is.
@swift-ci please smoke test

My naive goal was to make quick-help + derived work; you're right though, I think, the history of @noescape-printing is how the code got here.

@CodaFi could you review?


Get messageIDs for ValidationMessages form fhir.core

Updated fhir.core version to 5.6.43
hapi now populates OO.issue.details with the messageID from the RI java validator
fixed failing tests and test resources (introduced by newer fhir.core version being stricter on some data type setters)
removed duplicated commons-codec dependency from hapi-fhir-structures-dstu3/pom.xml
modified message filtering in ValidatorWrapper, replaced Stringmatchers with messageId based Issue filtering

This PR is ready to be merged.
I had to remove the 1.1 commons-codec dependency which was added to also check the android build.
fhir.core now also uses Base64.isBase64() (introduced in commons 1.5) from common-codecs - which breaks the android build during runtime.
fyi @tadgh

FYI @patrick-werner this failed with a duplicate message code, but I resolved it and pushed a fix to your branch.

thanks @jamesagnew
I'm back from vacation, merged latest master & fixing duplicated Msg.codes now

FYI I suspect this is going to have to be bumped back to post 6.1.0 - We're currently prevented from upgrading core because the R5 structures are missing the search parameter annotations. Grahame is investigating that, but I think it's unlikely that will happen before 6.1.0 hits code freeze.
I have a separate branch also going that fixes the unexpected strictness with date/time parsing. I really want to avoid merging in the "watering down" of the unit tests, since they represent cases that may be invalid but also represent things we know people do in the real world.

FYI I suspect this is going to have to be bumped back to post 6.1.0 - We're currently prevented from upgrading core because the R5 structures are missing the search parameter annotations. Grahame is investigating that, but I think it's unlikely that will happen before 6.1.0 hits code freeze.
I have a separate branch also going that fixes the unexpected strictness with date/time parsing. I really want to avoid merging in the "watering down" of the unit tests, since they represent cases that may be invalid but also represent things we know people do in the real world.

@jamesagnew that's understandable.
I had to adapt another test because of the parser strictness. Build should work now.
I just disabled the auto-merge you enabled to not merge the new core into master.
Is there anything i can help to resolve the parser issues in core @grahamegrieve ?

A fix to the parser issues in core just merged as well yesterday. I think this should all be good to go as soon as the next core build is pushed to central.
@dotasek can you comment here when the next core release is pushed to central? This can merge once that's happened.

A fix to the parser issues in core just merged as well yesterday. I think this should all be good to go as soon as the next core build is pushed to central.
@dotasek can you comment here when the next core release is pushed to central? This can merge once that's happened.
This may make the 6.1.0 release after all!

Great News!
I'm currently reverting the changed Tests/ adapting latest R5 changes into this PR. Will rename this PR.

@jamesagnew i tried the latest core snapshot as dependency. Now some tests which expected parsing errors are failing:
dt.setValueAsString("2013-02-03T11:22Z"); was expected to fail: "Invalid date/time string (datatype DateTimeType does not support MINUTE precision): 2013-02-03T11:22Z"
Do you want to remove/disable these tests? Or was the recent core change too lenient in this use-case?

upgraded to latest R5 structures, identified a bug in fhir.core (can't handle "5.0.0-snapshot2"), created a fixing PR: https://github.com/hapifhir/org.hl7.fhir.core/pull/885


One level too much indirection

Hi!
I just now realized that there is one level too much when my mods are installed via CKAN.
Example:
GameData
. CIT
.. ActiveStruts <- too much
... ActiveStruts
.... [ModContent]
For the CIT-Utils the structure should be:
GameData
. CIT
.. [ModContent]
as well.
This is true for all my CIT packages and breaks the mods which expect a certain path. I don't have access to my dev machine until the weekend so it would be really great if someone could fix the files.
Very sorry for the inconvenience!

Ugh, sure, on it.

Ugh, sure, on it.

Should be fixed now, thanks for notifying us so quickly :)

Should be fixed now, thanks for notifying us so quickly :)


Reject on high timestamp difference for blocks - why and why isn't it consistent

Description
When I run a test case with a large difference in timestamps between two consecutive blocks, I get this error in the log:
2021-07-17 17:06:42.628+00:00 | vert.x-worker-thread-19 | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER>0 is greater than the timestamp margin<PHONE_NUMBER>

And the block is rejected with this RPC reply:
{
  "jsonrpc" : "2.0",
  "id" : 13,
  "error" : {
    "code" : -32000,
    "message" : "Could not import Block"
  }
}

This seems weird, for two reasons:

I can't find in the Yellow Paper any criteria for the block timestamp except for inequality (49) on page 7 (see https://ethereum.github.io/yellowpaper/paper.pdf). So if we ever had a lull of about thirty minutes during which no blocks were created, it would break consensus.

Even if there is such a limit, shouldn't it be consistent? I ran the same test twice with a minute between them, and the limit seems to grow by one microsecond every second (see the first and last lines):

2021-07-17 17:15:27.477+00:00 | vert.x-worker-thread-5 | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER>0 is greater than the timestamp margin<PHONE_NUMBER>
2021-07-17 17:16:29.222+00:00 | vert.x-worker-thread-6 | INFO  | ProtocolScheduleBuilder | Protocol schedule created with milestones: [London: 0]
2021-07-17 17:16:29.247+00:00 | vert.x-worker-thread-13 | INFO  | CalculatedDifficultyValidationRule | Invalid block header: difficulty 131072 does not equal expected difficulty 131136
2021-07-17 17:16:29.254+00:00 | vert.x-worker-thread-16 | INFO  | CalculatedDifficultyValidationRule | Invalid block header: difficulty 131072 does not equal expected difficulty 3686656
2021-07-17 17:16:29.257+00:00 | vert.x-worker-thread-17 | INFO  | CalculatedDifficultyValidationRule | Invalid block header: difficulty 131072 does not equal expected difficulty 131136
2021-07-17 17:16:29.260+00:00 | vert.x-worker-thread-18 | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER>0 is greater than the timestamp margin<PHONE_NUMBER>

Hey @qbzzt, thanks for raising an issue! This seems like unexpected behavior to me, we'll investigate further.

Hi @qbzzt - which test(s) are you getting this behavior for?
It looks like the header validation rule is comparing against the system clock rather than the retesteth clock:
https://github.com/hyperledger/besu/blob/master/ethereum/core/src/main/java/org/hyperledger/besu/ethereum/mainnet/headervalidationrules/TimestampBoundedByFutureParameter.java#L47-L57
Are you getting this when trying to fill tests or when you are running against existing filled tests?

I get this when using the filled test (see badTimestamp.json) and when filling the test (see badTimestampFiller.yml).
The timestamp margin changes between different runs:
2021-07-27 17:02:12.440+00:00 | vert.x-worker-thread-8 | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER> is greater than the timestamp margin<PHONE_NUMBER>
2021-07-27 17:02:49.092+00:00 | vert.x-worker-thread-9 | INFO  | ProtocolScheduleBuilder | Protocol schedule created with milestones: [London: 0]
2021-07-27 17:02:49.115+00:00 | vert.x-worker-thread-12 | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER> is greater than the timestamp margin<PHONE_NUMBER>

These failing difficulty test cases were due to block timestamps in the future, well past the blockchain's system clock.
The PR above used the retesteth clock in the Future Timestamp rule check so that we could import future blocks, but the test engine would have to explicitly advance the clock in order for these blocks to successfully import during a test.  This also broke other existing test assumptions.
After discussion, it was deemed that the difficulty tests could be written with past block times without need for blocks with future timestamps.  I am closing this issue for that reason.  @qbzzt, feel free to re-open if this functionality is necessary.

Hi guys, I'm having this same issue when I spin up my cluster of TX nodes in k8s. I have the first transaction node that is fully synced and I spawned the second node and put reference to first one via static node. The sync of the entire ledger went well. However right after second node caught up with all blocks it fired an error:
2022-06-10 06:49:12.787+00:00 | EthScheduler-Services-9 (validateHeadersJoin) | INFO  | TimestampBoundedByFutureParameter | Invalid block header: timestamp<PHONE_NUMBER> is greater than the timestamp margin<PHONE_NUMBER>
2022-06-10 06:49:12.788+00:00 | EthScheduler-Services-9 (validateHeadersJoin) | WARN  | PipelineChainDownloader | Invalid block detected. Disconnecting from sync target. Invalid checkpoint headers.  Headers downloaded between #5942477 (0x22cba4047db02efad0dc03b1c13fdc8a6c895dd56ce04155813e4780db25d2a8) and chain head do not connect at #5942478 (0x49098e013eca9e519f47b3ba73e6ab92396952ba5a915710be7a1337e8e55543): Invalid block at #5942478 (0x49098e013eca9e519f47b3ba73e6ab92396952ba5a915710be7a1337e8e55543)
2022-06-10 06:49:12.789+00:00 | EthScheduler-Services-9 (validateHeadersJoin) | WARN  | PipelineChainDownloader | Chain download failed. Restarting after short delay.
java.util.concurrent.CompletionException: org.hyperledger.besu.ethereum.eth.sync.tasks.exceptions.InvalidBlockException: Invalid checkpoint headers.  Headers downloaded between #5942477 (0x22cba4047db02efad0dc03b1c13fdc8a6c895dd56ce04155813e4780db25d2a8) and chain head do not connect at #5942478 (0x49098e013eca9e519f47b3ba73e6ab92396952ba5a915710be7a1337e8e55543): Invalid block at #5942478 (0x49098e013eca9e519f47b3ba73e6ab92396952ba5a915710be7a1337e8e55543)

I tried to restart the second node, tried to remove static node file and play with different versions of Besu, but issue still persists.
Any idea how can I fix it?

Ok, sorry for messing with this - my issue was is that time didn't sync properly between two k8s nodes. Setting up NTP on both of them to work properly solved the issue.


⚠ Bofrost.be has degraded performance

In ee64df6, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1545 ms

Resolved: Bofrost.be performance has improved in e742132.


Wrong Query for "Control: Azure DDoS Protection Standard should be enabled"

Describe the bug
The site refers to another query.
To reproduce
Visit https://hub.steampipe.io/mods/turbot/azure_compliance/controls/control.network_security_group_udp_service_restricted?context=benchmark.nist_sp_800_53_rev_5/benchmark.nist_sp_800_53_rev_5_sc/benchmark.nist_sp_800_53_rev_5_sc_5
the query is pointed to https://github.com/turbot/steampipe-mod-azure-compliance/blob/v0.8/regulatory_compliance/network.sp#L90-L98
Expected behavior
A query for DDOS Protection Standard should be used.

Thanks @fgomesz for using Steampipe 👍 , hope you are having a good experience !!
The concern mentioned in this issue does look valid. Thanks for highlighting this, we will dig in a bit more and add the correct query to the control 👍

@fgomesz Appreciate this catch. Thanks
Definitely a few observations while reviewing this issue, let me know if it explains
The link for control Azure DDoS Protection Standard should be enabled
in Steampipe  Azure Mods holds the control for NIST, however, tagged wrongly to hipaa_hitrust_v92 = true > we will correct this
In mods, we re-use some of these queries to avoid re-work.
The same query (network_security_group_udp_service_restricted.sql) is used in the below controls

Cis_v130_6_6  >>  (Cis > 6.6 Ensure that UDP Services are restricted from the Internet (Automated)
Nist_sp_800_53_rev_5_sc_5  >>   DDoS protection standard should be enabled for all virtual networks with a subnet that is part of an application gateway with a public IP. (Ref)

The reason, why the same query was referenced
Neither CIS nor Nist steps are mentioned in their respective content with specific steps.

CIS has stressed on to use Azure NSG in the audit steps.
When we analyzed Azure in its policy, there is no direct validation steps included
Considering the CIS audit steps more reliable, we developed the query in the same model.

But, we understand it might be confusing, we are checking this internally to separate this as well.
The table azure_virtual_network inherently supports enable_ddos_protection attribute as mentioned here, we can bring this part of the NIST control validation
PR reference -  Still we are checking further to re-validate before release next week.
Share your feedback.

@fgomesz Definitely not a waste of time!
You found a bug in our tagging of the control 👍 and we really appreciate people working through these in detail - it's critical to keeping them as well described and accurate as possible!
Please keep those issues (and PRs) coming!


Add a command to link to the levels leaderboard for the bot

Summary
A command to link users to the levels leaderboard
What is the feature request for?
Other exisiting features
The Problem
Currently, users have to search of a message in an infrequently visited channel to get a link to the levels leaderabord if they don't have the link memorized/mookmarked.
The Ideal Solution
A command like /levels or /leaderboard that will give the invoking user an ephemeral message linking to the leaderboard.
The Current Solution
None
Additional Context
Has been requested multiple times by multiple different people for the past few months on the discord.

cc @CasparN


[AvatarGroup] Allow specifying total number of avatars

[x] I have followed (at least) the PR section of the contributing guide.

Closes #29649
Allow the developer to specify a total number of avatars. If a total is provided, all AvatarGroup calculations will be based on this number, rather than relying solely on children.length.
This is useful in use cases where the avatars are based on paginated lists and the server exposes the total count. There shouldn't be a need to request the whole list from the server (or to generate a list with that length) just to fulfil the requirement of the AvatarGroup.

@eduardomcv Can you check my comment before we make a decision about this new prop? https://github.com/mui-org/material-ui/issues/29649#issuecomment-981559908

@eduardomcv Sorry that I change your code because I added more tests and they failed. Can you review it again?
Also added one demo for the total prop.


TLS support?

Just checking to see if there's any desire for TLS support? I'm currently working on securing a swarm cluster with TLS.

The docker_service resource can use TLS by giving it the appropriate options (there's a test-kitchen suite that'll handle that). The docker_image and docker_container resources support TLS through Chef and the underlying docker-api gem


Suggestion: Automatic Zen mode

Hi,
i found a video about another notetaking app (i dont even know its name..) and i like the way that it automatically blends out everything when you start typing in the editor!
This is how i found your plugin.
Could you add an option to trigger zen mode whenever you start typing in the editor?
I attached a clip below
https://github.com/Maxymillion/zen/assets/128899868/2e7f06a9-1ff5-4b92-97ef-f492f1578a23

i just came here to write this and voila.


Removing unused fields from Web Referrer

Link to the issue - PDCL-10206

Details from the ticket:
We have agreed with the Analytics team to work on a new field group to send the buffered events from the previous page, to be more Platform specific. Since these fields will not be used to send previous link interaction with the page-view event, it's better to remove them to keep the schema as clean as possible.

@ninaceban , we don't allow breaking changes in our schemas. If you no longer need some properties, please deprecate those properties rather than delete them.

closing because it has been open from very long.


Undefined sample flag in the subcommand compile

Describe the bug
I tried to generate code using
./packages/cicero-cli/index.js compile --template<EMAIL_ADDRESS>--target Typescript

But unfortunately, it throws an error error: A text/sample.md file is required. Try the --sample flag or create a text/sample.md in your template.
But reading the [documentation](A text/sample.md file is required. Try the --sample flag or create a text/sample.md in your template) and the code, I noticed that this subcommand doesn't have any sample flag defined.
To Reproduce
Steps to reproduce the behavior:

Run ./packages/cicero-cli/index.js compile --template<EMAIL_ADDRESS>
Expected behavior
It shouldn't require a sample.md file.
Screenshots

Desktop (please complete the following information):

OS: Ubuntu 18.04
Version: On master branch

Additional context
Add any other context about the problem here.

Good catch. Thanks @algomaster99

I am not sure why we aren't using the sample in the template when passing an archive (TODO: double check whether the sample is mandatory in a template).

@jeromesimeon Logically it shouldn't be, right?

@algomaster99 This should now be fixed in Cicero 0.21.2 https://github.com/accordproject/cicero/releases/tag/v0.21.2
Let me know if this works and if we can close.

@jeromesimeon This issue still exists in this version as well so shall I start to work upon this issue.


Minimal RGB light support (no CT)

I bought a cheap lightstrip to add to HA and realised I could control it from HE but it would not report back its status. Hence this update.

Is this at line ~255 not needed for some reason?  Maybe @ogiewon has an opinion on why it was there originally?
if (!level) mapping.event.remove(1)

If I added that line, I don't recall my reason for doing so - other than the comment explaining the rationale.


No response after x seconds

Hello Nizarii,
actually i'm using your Rcon Class in a queue daemon where I only opens the connection once and everytime I need new information there will be a new command issued.
The first command gives me successful information back. But all others after that are give me an "" empty response.
I thought maybe the heartbeat is wrong implemented. As it only get called once at the start.
My understanding of a heartbeat is that you call it every x seconds to tell the server i'm still here.
Do you had this issue? I want to keep connection alive and not reconnect every 5 seconds. You know?
Regards

Ah. Found this one. https://www.battleye.com/downloads/BERConProtocol.txt

An empty 2-byte command packet (without actual command string) has to be sent every 45 seconds (or less)
to keep the connection/login alive, if there are no other command packets being sent.
If there are no command packets coming from the client for more than 45 seconds, it will be removed from BE RCon's
list of authenticated clients and will no longer be able to issue any commands.

I never had this issue. I'll make the method sendHearbeat() public, so you can call it in your daemon loop in order to keep the connection alive.

Wait. It doesn't seem to work that way. Actually my workaround is asking for information and if the response is '' I reconnect. But this is very server intense I guess.
Even if I send heartbeat every 5 seconds there is a result drop.

I'll check the connection/socket. Maybe the timeout thing doesn't work right.

Alright, as I understand it, the client has to respond in 10 seconds to every message sent by the server . Since PHP isn't the best language for daemons and socket.io like "socket.on(...)", a solution might be calling a method every 10 seconds, which checks for unresponded messages. This way the connection won't be closed by the server.

Yep. That would be a solution. Do you have an idea how to do check for unresponded messages?

Roughly, I'll test if just checking the socket for a packet is a smart idea ;)

I did a lot of tests and came to the result, that it is not smart to respond to every message sent from the server. Because the in-game chat, as well as other things, are sent to all connected clients, you have to respond to all messages. This wouldn't be a problem, but we are using PHP, so you don't know how many messages are sent to the client. So instead of checking multiple times for a message, which is server intense, it's better to just reconnect. In a future update, you will also get an exception if the connection timed out (due to not responding to server messages).


Add option to disable server certificate verification.

The option name mirrors the 'wget' option name (also --no-check-certificate). The cURL equivalent is called --insecure, which is a bit unclear.
Put in the "developers" section in documentation with proper warnings.
Fixes #168

Nice one Klaus!  Looks good to me :-) I'll merge in a bit.


chore: added classnames to wrapper

chore: added classnames to wrapper

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@2ab1362). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #148   +/-   ##
=======================================
  Coverage        ?   77.86%
=======================================
  Files           ?       64
  Lines           ?     1599
  Branches        ?      630
=======================================
  Hits            ?     1245
  Misses          ?      344
  Partials        ?       10

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2ab1362...7220dbb. Read the comment docs.


i18n: Add Turkish Translation

Oasis Web Wallet Turkish translation by Batuhan Tastekin (Turkish Ambassador)

Hi @batuhantastekin .
I'd like Turkish translation to be finally merged.
Can you please:

pull and rebase to latest develop branch
run yarn extract-messages to sync your translation with latest terms
update your translation for the missing terms and push it
I will then take a final look and approve the PR

Thanks!

Also, don't forget to put Turkish language to the sidebar here https://github.com/oasisprotocol/oasis-wallet-web/blob/1b059720f94aad0c3bec69599b1d19e03543f3b4/src/app/components/Sidebar/index.tsx#L134-L137

I cleaned up this branch in https://github.com/oasisprotocol/oasis-wallet-web/pull/808. Closing.


Fix for dlwp_healpix dataloader not including coupled variables

Modulus Pull Request
Description

Checklist

[X] I am familiar with the Contributing Guidelines.
[X] New or existing tests cover these changes.
[X] The documentation is up to date with these changes.
[X] The CHANGELOG.md is up to date with these changes.
[ ] An issue is linked to this pull request.

Dependencies

/blossom-ci

/blossom-ci

/blossom-ci


Migrate patch type for SyncSets with patches

Migrates previous patchType strings to the new short format

@abutcher I've now tested this manually. It should be ready for review

/lgtm


V3.17.5 disable params scale point values

PR for https://github.com/G-Research/fasttrackml/issues/840.
Internally, this PR tries to centralize the scale state of the params, so that both the ParamsScalePopover and the HighPlot chart can use it. The main issue was that the scale states in selectedParams are originally undefined, whereas the HighPlot needs to be aware of the new states in order to render the axes for each param. I made a clear distinction between the two and made sure that the selectedParams are appropriately updated from the beginning.
I also solved some other styling issues by abridging long param names and allowing the tooltip to show the full name.
Changelog

Fixed issue with Point scale parameters (like strings) allowing Linear/Log button options to be changed
Fixed issue with long param names messing up the button styles
Added support for tooltip overriding for ToggleButton to see full param name on ParamScalePopover when it is too long

Closing this PR as I messed up the branch comparison. Good PR is over here: https://github.com/G-Research/fasttrackml-ui-aim/pull/39


Unit transforms don't seem to apply

Hi!
It seems units aren't being added as expected. As an example, the build/scss file uses the transformGroup SCSS https://amzn.github.io/style-dictionary/#/transform_groups?id=scss, which includes size/rem, which in turn should append the unit rem to relevant variables, but it doesn't seem to be doing that.

Hey @mwargan this is easy to fix by using custom conversions. I updated this package to use the transfomers for px and percent.


Remove Hover Effect

Remove Hover effect from unwanted button and divs

can you assign me this issue??

can you assign me this issue??

this issues created by me please create your new issues


INT B-21940

Agility ticket
Summary
This PR has quite a bit of setup so prepare yer butts. Unfortunately the only way to test this thoroughly is to create a metric butt ton of moves.
Per the BL in Agility, we want to add a new "Destination Requests" queue for TOO. This queue will contain moves that are in Approvals Requested status and have the following criteria based on the SHIPMENT'S DESTINATION ADDRESS GBLOC, not the destination duty location on the orders:

destination SIT requests (in SUBMITTED status)
shipment destination address requests (in REQUESTED status)
move is in APPROVALS REQUESTED status

There are these exceptions for USMC customers (service_members.affiliation)

all USMC moves go into the USMC GBLOC

There are these exceptions for USAF and Space Force customers (service_members.affiliation)

if the destination address is in Alaska Zone II (re_rate_areas.name) it needs to go to the MBFL GBLOC
if it is NOT in Alaska Zone II (re_rate_areas.name), it needs to go to the destination address' respective GBLOC
if the destination address is in Alaska Zone II & the customer is in any other branch (excluding USAF/SF/USMC), the move should be in the JEAT GBLOC

I would highly advise you to set up a tracker doc (I just used the Mac Notes app) so you can mark which GBLOC the moves should be in, as well as move codes, etc. - here's what I did.
USMC:

Move: B4H6BB
GBLOC: USMC
1 Cold St.
Anchorage, AK 99703
Zone II rate area us_post_region_cities_id: 66768964-e0de-41f3-b9be-7ef32e4ae2b4

Move: PKKT69
GBLOC: USMC
2 Cold St.
Anchorage, AK 99703
Zone IV rate area us_post_region_cities_id: 78a6f230-9a3a-46ed-aa48-2e3decfe70ff

Move: 3JTHJF
GBLOC: USMC
3 Cold St.
Los Angeles, CA 90210
CONUS us_post_region_cities_id: 3b9f0ae6-3b2b-44a6-9fcd-8ead346648c4

USAF:

Move: CCPMTW
GBLOC: MBFL
4 Cold St.
Anchorage, AK 99703
Zone II rate area us_post_region_cities_id: 66768964-e0de-41f3-b9be-7ef32e4ae2b4

Move: F4WRMQ
GBLOC: JEAT
5 Cold St.
Anchorage, AK 99703
Zone IV rate area us_post_region_cities_id: 78a6f230-9a3a-46ed-aa48-2e3decfe70ff

Move: CCT8MH
GBLOC: KKFA
6 Cold St.
Los Angeles, CA 90210
CONUS us_post_region_cities_id: 3b9f0ae6-3b2b-44a6-9fcd-8ead346648c4

SF:

Move: 7G8KFB
GBLOC: MBFL
7 Cold St.
Anchorage, AK 99703
Zone II rate area us_post_region_cities_id: 66768964-e0de-41f3-b9be-7ef32e4ae2b4

Move: 9DKMJJ
GBLOC: JEAT
8 Cold St.
Anchorage, AK 99703
Zone IV rate area us_post_region_cities_id: 78a6f230-9a3a-46ed-aa48-2e3decfe70ff

Move: GF6T74
GBLOC: KKFA
9 Cold St.
Los Angeles, CA 90210
CONUS us_post_region_cities_id: 3b9f0ae6-3b2b-44a6-9fcd-8ead346648c4

ARMY:

Move: MFQGV9
GBLOC: JEAT
10 Cold St.
Anchorage, AK 99703
Zone II rate area us_post_region_cities_id: 66768964-e0de-41f3-b9be-7ef32e4ae2b4

Move: YWWKHJ
GBLOC: JEAT
11 Cold St.
Anchorage, AK 99703
Zone IV rate area us_post_region_cities_id: 78a6f230-9a3a-46ed-aa48-2e3decfe70ff

Move: B6XRJ6
GBLOC: MBFL
12 Cold St.
Los Angeles, CA 99510
CONUS us_post_region_cities_id: 3b9f0ae6-3b2b-44a6-9fcd-8ead346648c4

How to test USMC

Log in as a customer, set up with a USMC affiliation
Create a move with orders that land somewhere in Alaska
Add a shipment(s) (I've just been doing an HHG shipment) - you'll need to ensure you have shipment destination address with us_post_region_cities_id that lands in Alaska Zone II (you can manually update to be 66768964-e0de-41f3-b9be-7ef32e4ae2b4)
Sign and submit
Create another move - this time we'll test a move that's NOT in Alaska Zone II
Follow steps 2-4, update us_post_region_cities_id to be 78a6f230-9a3a-46ed-aa48-2e3decfe70ff for Alaska Zone IV
Create another move - this time just do a CONUS move, can be whatever, sign & submit
You should now have 3 USMC moves
Log in as a SC and approve all 3 USMC moves
Log in as TOO and approve all 3 USMC moves
Go in as Prime and request destination SIT on one or two moves
Also ensure you do a destination address request (CONUS might be easiest - ran into bugs for OCONUS stuff)
Go back in as TOO and view the Destination Requests tab
You should see all three moves that need destination actions
Find your ZIPs for the shipment destination addresses & find the GBLOC they fall under in postal_code_to_gblocs

postal_code = ':zip'

Verify that you can't see the USMC moves in those respective GBLOCs (you'll have to sign in as other GBLOC users - I tend to use Safari for that while keeping Chrome up for the other)

How to test USAF/SF in Alaska Zone II & non-Alaska Zone II

Log in as a customer, set up one with USAF affiliation & one with SF affiliation (You'll have to run through these steps twice)
Create a move with orders that land somewhere in Alaska
Add a shipment(s) (I've just been doing an HHG shipment) - you'll need to ensure you have a shipment destination address with us_post_region_cities_id that lands in Alaska Zone II (you can manually update to be 66768964-e0de-41f3-b9be-7ef32e4ae2b4)
Sign and submit
Create another move - this time we'll test a move that's NOT in Alaska Zone II
Follow steps 2-4, update us_post_region_cities_id to be 78a6f230-9a3a-46ed-aa48-2e3decfe70ff for Alaska Zone IV
Create another move - this time just do a CONUS move, can be whatever, sign & submit
You should now have 3 moves for each USAF & Space Force customer (Zone II, !Zone II, CONUS)
Log in as a SC and approve all 6 moves
Log in as TOO and approve all 6 moves
Go in as Prime and request destination SIT on the OCONUS moves
Also ensure you do a destination address request (CONUS might be easiest - ran into bugs for OCONUS stuff)
Go back in as TOO and view the Destination Requests tab
When logged in as a MBFL TOO, you should see AT A MINIMUM, the Alaska Zone II moves (1 for USAF, 1 for SF)
Find your ZIPs for the shipment destination addresses for the other moves & find the GBLOC they fall under in postal_code_to_gblocs

postal_code = ':zip'

Verify that you can see the moves in the destination queues of the GBLOCs you just looked up

How to test remaining queues

Log in as a customer, set up with whatever remaining affiliation
Create a move with orders that land somewhere in Alaska (testing Zone II for JEAT GBLOC)
Add a shipment(s) (I've just been doing an HHG shipment) - you'll need to ensure you have a shipment destination address with us_post_region_cities_id that lands in Alaska Zone II (you can manually update to be 66768964-e0de-41f3-b9be-7ef32e4ae2b4)
Sign and submit
Create another move - this time we'll test a move that's NOT in Alaska Zone II
Follow steps 2-4, update us_post_region_cities_id to be 78a6f230-9a3a-46ed-aa48-2e3decfe70ff for Alaska Zone IV
Create another move - this time just do a CONUS move, can be whatever, sign & submit
You should now have 3 moves (Zone II, !Zone II, CONUS)
Log in as a SC and approve all 3 moves
Log in as TOO and approve all 3 moves
Go in as Prime and request destination SIT on the OCONUS moves
Also ensure you do a destination address request (CONUS might be easiest - ran into bugs for OCONUS stuff)
Go back in as a JEAT TOO and view the Destination Requests tab
You should AT A MINIMUM, see the Alaska Zone II move (since it is not USAF/SF/USMC and in Alaska Zone II)
Find your ZIPs for the shipment destination addresses for the other moves & find the GBLOC they fall under in postal_code_to_gblocs

postal_code = ':zip'

Verify that you can see the moves in the destination queues of the GBLOCs you just looked up

Screenshots

Test coverage on server failed.
Last run: Wed 11 Dec 2024 11:33:59 PM UTC
Refer to this report to see coverage details for the server app.
Documentation for test coverage can be found at this link.

Tested USMC, noticed the "Assigned to" column doesn't seem to function. Refresh has it blank/null again.

Tested USMC, noticed the "Assigned to" column doesn't seem to function. Refresh has it blank/null again.

Does seem to get save for the moves queue though

I believe there's a follow on BL item for destination queue assigned functionality

Tested USMC, noticed the "Assigned to" column doesn't seem to function. Refresh has it blank/null again.

I believe there's a follow on BL item for destination queue assigned functionality

I believe that's correct - E-06095

Tested USMC, noticed the "Assigned to" column doesn't seem to function. Refresh has it blank/null again.

Does seem to get save for the moves queue though

I believe there's a follow on BL item for destination queue assigned functionality

I added in the privilege check which loads the assigned user info in this commit

Tested USMC, noticed the "Assigned to" column doesn't seem to function. Refresh has it blank/null again.

Does seem to get save for the moves queue though

I believe there's a follow on BL item for destination queue assigned functionality

I added in the privilege check which loads the assigned user info in this commit

@ryan-mchugh should be able to see the assigned user now


zipper: implement exercise

Specification

@ferhatelmas would love to work on this issue could you guys guide me a bit on what is required :)

@ferhatelmas Can you guys guide me on how should I proceed, would love to contribute.

@sladyn98, We appreciate your interest to help out!
The main README.md is the best starting point.    The Exercise Files Overview section explains the different files for implementing an exercise.
Are there any specific questions you have on how to start?


Create mytwu.txt

This is the student domain. The teacher domain, twu.ca, already exists.

@marshalljanzen Pull request merged. Thank you!


Add matomo snippet to base.html

Per https://mailarchive.ietf.org/arch/msg/ietf/oxrawLxvirOtB1C8Dd3qdgOY0oo/, add the datatracker to the ietf analytics ingester.
See how this is done for the wagtail-website.

long obe.


Certs Owner Reference on secret from issuer

Is your feature request related to a problem? Please describe.
I have few issuers in my kubernetes cluster, one for my lets-encrypt certificates and 3~4 for my vault certificates. I want to configure the kubernetes' attribute OwnerReferences on my vault issuers but not on the lets-encrypt issuer.
Describe the solution you'd like
I think we could have a boolean field in the issuer's description to enable OwnerReference on the secrets created by the issuer. It would be set to False by default but would be overridable.
i.e secretOwnerReferences, set to true on my Vault issuers so their secrets can be destroyed when the issuer is deleted, but set to false on my lets-encrypt issuer as I want it to keep the default behavior.
Describe alternatives you've considered
I think it's also possible to configure 2 controller in my cluster but this would be overly complex for this purpose.
Environment details (remove if not applicable):

Kubernetes version: 1.20
Cloud-provider/provisioner: bare-metal
cert-manager version: 1.3
Install method: static manifest

/kind feature

Hello @irbekrm,
I'm contacting you just to no if someone in contributors/owners think it's a good idea ?
If you think it's a good idea or not?
Then I could start development on it if it's approbe :)
Regard

Hi, sorry for the delay.
It should be currently possible to set an ownerReference pointing at the creating Certificate on the Secret using --enable-certificate-owner-ref flag on controller- so that Secrets get deleted when the corresponging Certificate gets deleted.
If you were to delete the Secret without deleting the actual Certificate (i.e after the Issuer has been deleted) cert-manager would attempt to re-issue the cert.
You might want to have something that deletes the Certificates when an Issuer has been deleted + enable the owner reference for Certficate -> Secret.
I am not entirely sure if it might make sense for there to be an Issuer -> Certificate owner reference since Certificates aren't actually created by Issuers as such.
In this particular case, it might make more sense to have some custom functionality that deletes those Certificates.
Perhaps there is something more we could do to make that easier, such as labelling the Secrets or allowing for custom hooks to be run when resources are created/deleted- we have been having discussions about implementing that kind of functionality.

Hi,
We are currently investigating the addition of such a field on the Certificate resource in https://github.com/cert-manager/cert-manager/pull/5158.
@lperdereau Could you describe why the certificates issued by your Vault issuer shouldn't add the owner reference to the Secret, but the certificates created by the Let's Encrypt issuer should?

Hi,
We are currently investigating the addition of such a field on the Certificate resource in #5158.
@lperdereau Could you describe why the certificates issued by your Vault issuer shouldn't add the owner reference to the Secret, but the certificates created by the Let's Encrypt issuer should?

Hey 👋 ,
It's been a long time since I've worked on it. But I remembered, I had Vault issuers automatically created and removed, and I wanted to remove Certificate automatically when Issuer is removed.
It should work for every kind Issuer not specially for LE or Vault.

Was it because these Vault issuers were ephemeral?

Yes

If that is the case, would it have worked to rely on removing the ephemeral namespace to get the same effect?
Hmmm, you're right, I didn't think of that, it might do the trick.

In my mind at the time, if a certificate could be issued, it could be revoked by the issuer and the deletion action would be the revocation action.
But if you namespaced an Issuer it can't deliver certs for another namespace, you have to multiply one Issuer over a set of namespace?
Moreover, to my mind the problem would be the same for a clusterissuer, if I remove a cluster issuer I want it to remove all my certs automatically.


Implement MTTR

Two use cases:

Automated: Pods go into crash loops and other failures observable from kubernetes
Human defined: Some sort of incident ticket getting open or closed. For now we could use GitHub issues to simulate this.

Thoughts on this?
Automated 1 (failure based): as defined above: check observable failures / unhealthy states
Automated 2 (success based): check observable healthy states / HTTP 200, e.g. use Prometheus health checks to validate recovered state
*May require human definition (per deployment) of healthy state / when to return 200 response, though once declared, is considered automated

@jaycoon yeah, that sounds reasonable, but I think we need to get more specific, and include example commands showing how we'd gather that data. From the above, it looks like "Automated 1" is meant to define the start of the failure, and "Automated 2" is defining end of failure. So let's call it that, and fill in the HOW:
FAILURE START:
# Fill in commands to capture

FAILURE END:
# Fill in commands to capture

Would like to hear how @AlyIbrahim is doing this now, as he is actively working this task.

 How Might We Measure Failure?

@etsauer @jaycoon
Currently I am trying to capture when a pod is Deleted/Crashed as the failure start and failure end when the number of pods available are equal to the the requested ones.
Not the most accurate, but I am trying to have a baseline.
More accurate one to me is to use the readiness probe

@AlyIbrahim @etsauer
The base case makes sense, and I think this approach is safer, implementing with possible false negatives (e.g. a pod failed, though the Service may still be online because there are other Ready pods). We may even decide not to call this a false negative, but rather a a very good MTTR, and as such could be the target MTTR for the app (i.e. how fast a single pod can restart, maintaining Service availability with no impact to the user).
One thought on this, is to consider Pods and Deployments separately to calculate our MTTR. Right now we are looking a Pod to failure (pod crashed) AND looking to the Deployment config for failure end (desired Replicas). Perhaps we separate Pod MTTR and Deployment MTTR? Then we can address Pod MTTR as our first phase, and Deployment MTTR as the next phase.
Adding in the readinessProbe will let us be more accurate, as you said. We can also consider a livenessProbe. Both probes will default to Success if not defined, though if defined, we can utilize them in our MTTR definition. Some other thoughts on this for the Pod MTTR:
Base case (BC)
Failure start = Desired state
Failure end = Ready pods reaches desired state
Liveness case (LC)
Failure start = livenessProbe failure
Failure end = livenessProbe success
Readiness case (RC)
Failure start = readinessProbe failure
Failure end = readinessProbe success
The more probes a pod has defined, the more accurate the MTTR will be.
To determine failure start and end, it is likely an AND function: (BC & LC & RC)
This will detect specific app failures, when the pods may be running and appear fine, for example.
If LC and RC are not defined, they will default to passing, so the function still works with checking only BC.

depends on #35


A few openblas_utest_ext tests failed for cmake builds of OpenBLAS

Hello,
I tested current develop code  of OpenBLAS on different platforms with different architectures [Loongson 3A6000 (loongarch64), AMD R9 7940HS (x86_64) ] and found the same 8 tests in openblas_utest_ext failed for cmake builds.
tested Operating system: Arch Linux, AOSC OS,...
Building commands:
CC=gcc FC=gfortran cmake -B build
cmake --build build
cd build/utest
./openblas_utest_ext

Testing results on AMD R9 7940HS:
...

TEST 1526/1526 zsbmv:xerbla_lda_invalid [OK]
RESULTS: 1526 tests (1518 ok, 8 failed, 0 skipped) ran in 179 ms

on Loongson 3A6000:
...

If building with CC=gcc FC=gfortran make, all tests in openblas_utest_ext are OK.

I don't recall seeing this, will investigate. Maybe it is the difference in optimization options (-O2 for make vs  either nothing or -O3 for cmake, depending on whether you set CMAKE_BUILD_TYPE). Which gcc version did you use ?

I don't recall seeing this, will investigate. Maybe it is the difference in optimization options (-O2 for make vs either nothing or -O3 for cmake, depending on whether you set CMAKE_BUILD_TYPE). Which gcc version did you use ?

It seems that gcc version does not matter. For AMD R9 7940HS gcc version is 14.1.1 (Arch Linux), for Loongson 3A6000 gcc version is 13.2.0 (AOSC OS).  And I just tested on Debian 12 (gcc version 12.2.0) for AMD R9 7940HS,  the result is the same. I did not set CMAKE_BUILD_TYPE.

On Debian 12 I just tried different CMAKE_BUILD_TYPE with Debug, RelWithDebInfo or MinSizeRel.  The 8 tests in openblas_utest_ext still failed.

Reproduced now but have not identified any differences in build options for the relevant files yet. Curiously the executable sizes for openblas_utest_ext differ considerably between the two builds, so my current assumption is that the problem mainly/only affects the test itself

Well, it had to be a dumb mistake I made in #4470 (affecting "only" 0.3.27)


Fix buidling safe area demo project

What does the pull request do?
Fixes Avalonia.Headless.Vnc reference in SafeAreaDemo project
What is the current behavior?

What is the updated/expected behavior with this PR?

How was the solution implemented (if it's not obvious)?

Checklist

[ ] Added unit tests (if possible)?
[ ] Added XML documentation to any related classes?
[ ] Consider submitting a PR to https://github.com/AvaloniaUI/Documentation with user documentation

Breaking changes

Obsoletions / Deprecations

Fixed issues

You can test this PR using the following package version. 11.0.999-cibuild0034742-beta. (feed url: https://pkgs.dev.azure.com/AvaloniaUI/AvaloniaUI/_packaging/avalonia-all/nuget/v3/index.json) [PRBUILDID]


Missing headers (build 8005.2.0)

Latest available build 8005.2.0 is missing RTC* headers. Decided to rollback to the build 7972.2.0 which has headers.

Experiencing the same issue.

Still trying to work this out. no idea why cocoapods isn't picking up the files


rewrite with Templating and ES6 standards

See the result:

I basically rewritten everything, some fields from the GitHub API was inconsistent with the original code

I use single quote and no semicolon in my coding standard you can change it easily with something like prettier or eslint if you don't like it

I use single quote and no semicolon in my coding standard you can change it easily with something like prettier or eslint if you don't like it

Please check the span tags in the output, it has commas in-between and the third tag is null in the O/P

Done

                Mohas

                JavaScript C# CSS

                15 Repositories

                18 Contributions

1 Followers

6 Following


server: reduce memcopy when record the statistics

What have you changed? (required)

     flat  flat%   sum%        cum   cum%
     0.04s  0.11%  0.16%     23.11s 63.04%  github.com/pingcap/pd/server.(*RaftCluster).HandleRegionHeartbeat
     0.13s  0.35%  0.52%     23.06s 62.90%  github.com/pingcap/pd/server.(*clusterInfo).handleRegionHeartbeat
     0.05s  0.14%  0.65%      8.96s 24.44%  github.com/pingcap/pd/vendor/github.com/gogo/protobuf/proto.Clone
     0.55s  1.50%  2.15%      8.21s 22.39%  github.com/pingcap/pd/vendor/github.com/gogo/protobuf/proto.mergeStruct
     0.13s  0.35%  2.51%      6.51s 17.76%  github.com/pingcap/pd/server/core.(*StoreInfo).Clone
     0.01s 0.027%  2.54%      6.35s 17.32%  github.com/pingcap/pd/server/core.(*RegionsInfo).SetRegion
     0.02s 0.055%  2.59%      6.32s 17.24%  github.com/pingcap/pd/server/core.(*StoresInfo).GetStore
         0     0%  2.59%      6.24s 17.02%  github.com/pingcap/pd/server/core.(*RegionsInfo).AddRegion
     0.05s  0.14%  2.73%      6.05s 16.50%  github.com/pingcap/pd/server.(*clusterInfo).getRegionStoresLocked
     0.03s 0.082%  2.81%      5.52s 15.06%  runtime.systemstack
     0.01s 0.027%  2.84%      5.22s 14.24%  github.com/pingcap/pd/server/core.(*regionTree).update
     0.64s  1.75%  4.58%      5.08s 13.86%  github.com/pingcap/pd/vendor/github.com/google/btree.(*node).iterate
     0.02s 0.055%  4.64%      4.78s 13.04%  github.com/pingcap/pd/vendor/github.com/sirupsen/logrus.Infof

Reduce the store clone.
What are the type of the changes (required)?

Improvement (non-breaking change which is an improvement to an existing feature)

How has this PR been tested (required)?
Manual.

LGTM

/rebuild

/rebuild


Speed varies a lot - my fault or?

Which node-red-contrib-modbus version are you using?
5.23.3
What happened?
When I changed my modbus device (heat pump controller) from connecting over WiFi to ethernet the speed of reading Modbus increase a lot (a factor 10-20 I estimate). That worked well for a long time. When I upgraded to version 5.26 the speed fell drastically, so I reverted to version 5.23 and speed was up again. Now, after a few changes to my flow .. not anything really serious and nothing that to me explains the difference, speed is now down to very slow again. What influences the speed of Modbus performance that much? I have removed all uneccesary nodes and removed a writer.

And then 2 hours later it suddenly runs super fast:

Between the two, I tried to install the v8-profiler node to see if it could reveal anything. It did not work and I uninstalled it and now suddenly Modbus runs super fast.
Now and then (every half hour) I get a lot of these:
12 Nov 15:35:09 - [warn] [modbus-flex-getter:b28b0f8bdeef434b] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-getter:8ab743dfec4860f7] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-write:270db2d2537ce663] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-getter:87d4f075a94f5e75] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-getter:b28b0f8bdeef434b] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-getter:8ab743dfec4860f7] Modbus Failure On State sending Get More About It By Logging
12 Nov 15:35:09 - [warn] [modbus-flex-getter:87d4f075a94f5e75] Modbus Failure On State sending Get More About It By Logging

Not sure why and not sure how to get more logging info / but does not seem to be related.
Server
Other/External server
How can this be reproduced?
Not really sure - sorry
What did you expect to happen?
I would expect reading items would be 10 times as fast.
Other Information
No response

@biancode Ok - will try that next time I get to the location


Error executing main.py

Traceback (most recent call last):
File "C:\apps\routeros-scanner-main\main.py", line 62, in
main(args)
File "C:\apps\routeros-scanner-main\main.py", line 31, in main
res = command.run_ssh(ssh_client)
File "C:\apps\routeros-scanner-main\commands\dns.py", line 16, in run_ssh
sus_dns, recommendation = self.check_results_ssh(res, enabled)
File "C:\apps\routeros-scanner-main\commands\dns.py", line 27, in check_results_ssh
if int(item['ttl'].partition('s')[0]) > 200000:
ValueError: invalid literal for int() with base 10: '2h11m36'

same here:
...(truncated)...
    if int(item['ttl'].partition('s')[0]) > 200000:
ValueError: invalid literal for int() with base 10: '1d'

@salacpavel  see workaround in #14

That did it, thank you.

Hi, we uploaded several in-house fixes, one of them for this issue. You can see the updates here: https://github.com/microsoft/routeros-scanner/pull/24


Permission Issues Resolving Previous Findings in Security Hub

I'm running Prowler via an ECS task in Fargate, using an assumed task role with all the right permissions. New findings are getting added to Security Hub fine, via the native SH support (i.e. -S), but existing findings that have since been resolved are failing to update with a permissions error:
An error occurred (AccessDeniedException) when calling the BatchImportFindings operation: User: <redacted> is not authorized to perform: securityhub:BatchImportFindings on resource: arn:aws:securityhub:ap-southeast-2::product/prowler/prowler
Even though that role definitely has that permission. After some debug, it looks like the resolveSecurityHubPreviousFails() function in include/securityhub_integration is using batch-import-findings, rather than the batch-update-findings I would expect. The permissions error seems to be a red herring, and the issue is you cannot update existing findings in this way, using the batch import. I'm not sure if this recently changed with AWS?
Has anyone else experienced this issue? I'm playing with a patch to move this function over to batch-update-findings, so if this is not just something silly I'm doing and is actually a bug, happy to cut a PR for the fix.

Not sure if it is related to this same issue here https://github.com/toniblyx/prowler/issues/705 that I'm working on it. But also, since a month or so, Prowler is an official integrated product to Security Hub so you have to enable it first on each account region you need. See updated documentation here https://github.com/toniblyx/prowler#security-hub-integration. Do you think your issue is related to that?

Thanks for the response! I did have a look at #705 , but my issue is with a single account, so don't believe it's related. Security Hub appears to be setup properly, I can get findings into SH from Prowler without issue, it just won't update (Archive) existing issues if they are fixed after initially being raised. After patching the code per my original post, everything is working fine (using batch-update-findings instead) - I'm out of the woods now, but if others are also experiencing the same issues I wanted to share my fixes in case it helps others.

Do you mean changing call here https://github.com/toniblyx/prowler/blob/master/include/securityhub_integration#L60 from import to update?

For some reason seems like the fargate execution role is not being assigned properly and it can't do BatchImportFindings. Can you debug that in CloudTrail to see what is it trying to do? it is suspicious to get that AccessDeniedException.

Do you mean changing call here https://github.com/toniblyx/prowler/blob/master/include/securityhub_integration#L60 from import to update?

Yes, exactly. I created #712 to explain in code. May not be perfect, but is working for me in my environment. I agree the access denied error is suspicious, but I've not been able to work out why using CloudTrail etc., though I have put a fair bit of time into trying to understand why it's happening. I did try opening permissions for that role right up, to allow all Security Hub actions, but that didn't help - seems to suggest that there is not a missing perm that I could just add to fix it. Weird.

Hi,
I think we stumbled upon the same error when running Prowler. We run it in CodeBuild which is deployed via Terraform and in some accounts the build fails while in other it runs perfectly well.
We thus contacted AWS and got a very interesting answer. I am not allowed to share the full answer but the outline is this:

This permission error (xx is not authorized to perform: securityhub:BatchImportFindings on resource yy) happens to occur when there is some invalid value in the finding. This invalid value could be a wrong region, account ID or ARNs. So even with administrative access, one could get a similar error

So an input formatting error is reported as a permission error. Is there some way to print the finding JSON in prowler upon import? Maybe it really is malformed in certain edge cases.
I will try to get more information, but I don't have access to the accounts where this happens, so I am currently not sure how to debug it.

Thanks @joerg, can you confirm the same issue with latest version? and if you can share the finding json that is failing in order to figure out what is wrong.

I ran the whole thing again with version 2.5 but the error remains. I also did a lot of debugging and tried finding out what could be the error here but so far the input JSON seems correct all the time. We will keep rolling out Prowler to our production accounts and see if more errors pop up and maybe I will be able to see a pattern.
One reason I could think of is that we also run Prowler in the central SecurityHub account that all other accounts in the organization deliver their findings to and there the error pops up the most. In the end, the fact that the SecurityHub API seems to produce unreasonable error messages is not really helping.

@toniblyx I was able to find the underlying error/bug and it seems that it is not related to the issue. I opened a new issue to work on it #867 .

Just to make this public in order to help @gyrospectre as well.
One way to debug this is running it manually from the same account and region to replicate your codebuild/fargate etc with something like:
bash -x ./prowler -A accountid -R roletoassume -r region -f region -M csv,json,html,json-asff > prowler.log 2>&1
That won't send anything to Security Hub but will generate a large log file with all what it does and the json asff to see what would be trying to send to Security Hub.

This one was fixed here https://github.com/toniblyx/prowler/commit/254cb0cf6353729528587ab131271865571f8ef8 and it is available in branch 2.5.1


async-http-client version 2.10.4 security issues

Describe the bug
current async-http-client version 2.10.4 have 2 High Severity security issues
Can you please bump it the the latest version?
https://snyk.io/test/github/AsyncHttpClient/async-http-client
Cheers,
Paolo

Hi @pmologni
AFAICS 2.10.4 is the latest release available and deployed on Maven central cc @slandelle
My understanding of https://snyk.io/test/github/cloudbees/zendesk-java-client is that the issue is in netty and not async-http-client

https://snyk.io/vuln/SNYK-JAVA-IONETTY-543669

https://snyk.io/vuln/SNYK-JAVA-IONETTY-543490
I proposed an update of the dependency in async-http-client : https://github.com/AsyncHttpClient/async-http-client/pull/1694

Thanks @aheritier!

@slandelle merged the PR but it is useless in our context. These security issues are on the server side of Netty. async-http-client is using the client side.