Quality Problems with Mipmaps

I have a BC3 image that I decoded with my own decoders. I'm not sure how the original BC3 image was created (maybe with nvidia texture tools?). Once I decoded the main mip from the image, I encoded it again to BC3 using your library. Then I decoded the newly encoded image again (with my own decoder), and I noticed that there are issues/differences in how the smaller mip levels look.
I have attached the original images in the expected folder, and the new images using your encoder in the actual folder. I used the following code to encode the images:
BcEncoder encoder = new BcEncoder();
encoder.OutputOptions.generateMipMaps = true;
encoder.OutputOptions.quality = CompressionQuality.BestQuality;
encoder.OutputOptions.fileFormat = OutputFileFormat.Dds;
encoder.OutputOptions.format = CompressionFormat.BC3;
encoder.OutputOptions.ddsBc1WriteAlphaFlag = false;
encoder.OutputOptions.maxMipMapLevel = 5;

Image<Rgba32> faceImage = ...;
var encodedData = encoder.EncodeToRawBytes(faceImage);

mipmap.zip

I'll look into it.

Hey,
Sorry that this took so long, but I've looked into it a little more closely now and it seems that the difference in mipmaps stems from the resizing algorithm used. Most likely whatever program encoded the original image was using a nearest-neighbor-style resizing algorithm, while mine uses ImageSharp's default resizing function (which probably uses some kinda blending of pixels).
For what it's worth, the mip3 in the expected output folder has way more beige and brown colors than the original image in the corresponding spots, while the colors look more accurate proportionately in BcnEncoder's output.
I think adding an option in the encoder for which resizing algorithm to use could be something beneficial, but it doesn't seem like a too big of a problem for now.

Hey,
Sorry that this took so long, but I've looked into it a little more closely now and it seems that the difference in mipmaps stems from the resizing algorithm used. Most likely whatever program encoded the original image was using a nearest-neighbor-style resizing algorithm, while mine uses ImageSharp's default resizing function (which probably uses some kinda blending of pixels).
For what it's worth, the mip3 in the expected output folder has way more beige and brown colors than the original image in the corresponding spots, while the colors look more accurate proportionately in BcnEncoder's output.
I think adding an option in the encoder for which resizing algorithm to use could be something beneficial, but it doesn't seem like a too big of a problem for now.

No worries, thanks for looking into this.  I guess it's just odd to me that there's such huge black bands after a resize from 64x64 to 32x32.  I see now that it is due to ImageSharp.
I guess it's easy enough to try my own resizing techniques and feed each mip separately into your encoder for now.

No worries, thanks for looking into this.  I guess it's just odd to me that there's such huge black bands after a resize from 64x64 to 32x32.  I see now that it is due to ImageSharp.
I guess it's easy enough to try my own resizing techniques and feed each mip separately into your encoder for now.

Seems like the ImageSharp team might add the option for resizing the image with non-premultiplied alpha based on your issue. I can add the option to the encoder once they update the package.

Seems like the ImageSharp team might add the option for resizing the image with non-premultiplied alpha based on your issue. I can add the option to the encoder once they update the package.

Hey, since ImageSharp is not used for resizing in 2.0.0, would you like to try and see if this is still an issue in the new version?

There's a bug in MipMapper.cs in loop at line 85.  If the maxNumMipMaps is a number such that mipWidth or mipHeight do not become one, then the output remains 0. This in turn causes result at line 37 to be an empty array.
Also, I'm wondering if the statement mipWidth == 1 && mipHeight == 1, it should use || not &&. Although this may be a moot point if the method has to be rewritten in a different way.

Ah, you're correct. I'll fix it today.
I'm not sure if it should be ||, since that would mean the last mip might be something else than 1x1. I guess I could check how other encoders handle tall or wide textures.
A bit of a shame that we switched off of ImageSharp right when they added this, but at least they now have the feature if someone needs it.

New version is up in nuget with a fix.

Looks like the problem has been fixed. Thanks!


Switch to a non-memory-retaining API for password testing

TLDR;

Loads the dictionaries and adjacency graph on demand (reducing retained memory)
Adds a Zxcvbn::Tester for performance-sensitive use cases (for old behaviour)
Changes to add_word_lists(key=>val) from add_word_list(key,val)

At the moment we're loading all the dictionary and adjacency graph data when the library is required, and storing it in a constant. This makes for slow load times for Rails applications, and a huge chunk of retained memory. For our Rails application it's the biggest retained memory by gem after ActiveSupport.
The API stays the same:
Zxcvbn::Tester.test("password")

The change is that Zxcvbn.test now loads the dictionary on demand, and doesn't retain it in memory afterwards. This has the advantage of not adding overhead to applications when they add gem "zxcvbn" to their Gemfile, and that the base memory usage for the application is much lower. The disadvantage is that is will take longer to test an individual password.
For performance sensitive use cases where retained memory is not an issue, I've added a Zxcvbn::Tester class that you can instantiate once (on application initialisation if you wished), for example:
tester = Zxcvbn::Tester.new
tester.add_word_lists("custom" => ["words"])
tester.test("password 1")
tester.test("password 2")
tester.test("password 3")

For the basic case of Zxcvbn.test("password") it will simply load the dictionaries, test the password, and release the memory.
Before this change:

require allocates 264,315 objects (21MB of memory), retains 87120 objects (6.8MB of memory)
password test doesn't allocate many objects or retain memory (all happens on require)

$ time ruby -I./lib -e ''
real	0m0.176s
user	0m0.090s
sys	0m0.081s

$ time ruby -I./lib -e 'require "zxcvbn"'
real	0m0.325s
user	0m0.219s
sys	0m0.100s

$ ruby -I./lib -rmemory_profiler -e 'puts MemoryProfiler.report { require "zxcvbn" }.pretty_print'
Total allocated 264315
Total retained 87120

allocated memory by gem
-----------------------------------
zxcvbn-ruby/lib x 21126753
rubygems x 602449
json-1.8.2 x 348926
2.2.0/lib x 34868
2.2.0 x 18275
other x 80

$ time ruby -I./lib -rzxcvbn -e 'Zxcvbn.test("password")'
real	0m0.329s
user	0m0.221s
sys	0m0.101s

$ ruby -I./lib -rmemory_profiler -e 'puts MemoryProfiler.report { require "zxcvbn"; Zxcvbn.test("password") }.pretty_print'
Total allocated 265702
Total retained 87123

allocated memory by gem
-----------------------------------
zxcvbn-ruby/lib x 21198553
rubygems x 602369
json-1.8.2 x 348926
2.2.0/lib x 77508
2.2.0 x 18275
other x 160

retained memory by gem
-----------------------------------
zxcvbn-ruby/lib x 6813690
json-1.8.2 x 261837
rubygems x 57182
2.2.0 x 4921
2.2.0/lib x 1649

retained objects by gem
-----------------------------------
zxcvbn-ruby/lib x 85166
json-1.8.2 x 1261
rubygems x 637
2.2.0 x 42
2.2.0/lib x 19

After this change:

require allocates 127 objects (0.01MB of memory), retains 0.006MB
password test allocates 256,268 objects (21MB of memory), retains 79 objects (0.006MB of memory)

$ time ruby -I./lib -e ''
real	0m0.176s
user	0m0.090s
sys	0m0.081s

$ time ruby -I./lib -e 'require "zxcvbn"'
real	0m0.214s
user	0m0.121s
sys	0m0.090s

$ ruby -I./lib -rmemory_profiler -e 'puts MemoryProfiler.report { require "zxcvbn"; }.pretty_print'
Total allocated 7579
Total retained 806

allocated memory by gem
-----------------------------------
rubygems x 621383
json-1.8.2 x 70463
2.2.0 x 18275
2.2.0/lib x 14985
zxcvbn-ruby/lib x 10570
other x 80

allocated objects by gem
-----------------------------------
rubygems x 6233
json-1.8.2 x 878
2.2.0 x 178
2.2.0/lib x 162
zxcvbn-ruby/lib x 127
other x 1

retained memory by gem
-----------------------------------
rubygems x 59752
zxcvbn-ruby/lib x 6490
2.2.0 x 4921
json-1.8.2 x 1214
2.2.0/lib x 880

$ time ruby -I./lib -rzxcvbn -e 'Zxcvbn.test("password")'
real	0m0.324s
user	0m0.221s
sys	0m0.098s

$ ruby -I./lib -rmemory_profiler -e 'puts MemoryProfiler.report { require "zxcvbn"; Zxcvbn.test("password") }.pretty_print'
Total allocated 266098
Total retained 803

allocated memory by gem
-----------------------------------
zxcvbn-ruby/lib x 21200073
rubygems x 621303
json-1.8.2 x 348926
2.2.0/lib x 91801
2.2.0 x 18275
other x 160

allocated objects by gem
-----------------------------------
zxcvbn-ruby/lib x 256268
rubygems x 6232
json-1.8.2 x 2348
2.2.0/lib x 1070
2.2.0 x 178
other x 2

retained memory by gem
-----------------------------------
rubygems x 58070
zxcvbn-ruby/lib x 6730
2.2.0 x 4921
json-1.8.2 x 1841
2.2.0/lib x 1360

retained objects by gem
-----------------------------------
rubygems x 650
zxcvbn-ruby/lib x 79
2.2.0 x 42
2.2.0/lib x 17
json-1.8.2 x 17

This is probably a good time to bump to v1.0 as well?

These changes look good to me.
Ping @stevehodgkiss...

We've been running it in production for a few weeks. Did you want to see a before and after memory footprint for our app?
Seeing as most people aren't "password-strength-as-a-service" and their bottlenecks lie elsewhere I think the default is better to be reduced memory at the price of slightly slower slower performance.

And if you are "password-strength-as-a-service" you can always:
module MyApplication
  PASSWORD_TESTER = Zxcvbn::Tester.new
end

MyApplication::PASSWORD_TESTER.test("password 1")
MyApplication::PASSWORD_TESTER.test("password 2")
MyApplication::PASSWORD_TESTER.test("password 3")

@toolmantim we :heart: numbers, so before/after stats would be cool :smile:

Hey Tim, thanks for the PR!
The code looks good - we should definitely support both options. I'm not sure which should be the default though... there is a big performance difference between the 2 options - 0.002/password vs 0.095/password on my machine. https://gist.github.com/stevehodgkiss/0183a4f4880472366ea9

To be clear my main optimisation here was the performance of requiring the gem (allocating memory, and loading the YML), and memory overhead… it wasn't optimising the speed of testing a password. The benchmark was the memory footprint of the app that used it, and the time to require "zxcvbn".
For us, on a daily basis across all our systems and apps we're doing many more fresh require "zxcvbn" calls than we are Zxcvbn.test calls, and for processes like Sidekiq, cron, etc that will never even invoke Zxcvbn they get a memory reduction and a load speed boost.
I used this similar pattern with the UserAgentParser gem, and just let people load it and assign it to a constant if they wanted… rather than perform lots of expensive stuff at require time.

Yep, I see where you're coming from... I guess we just need to make it easy for users to choose their tradeoff.
Do you prefer the manual approach over something built into the gem to preload or not?
# where sidekiq? indicates whether this process is for running sidekiq...
PASSWORD_TESTER = sidekiq? ? Zxcvbn : Zxcvbn::Tester.new
# vs
Zxcvbn.preload! if sidekiq?

For a typical Rails application I wouldn't suggest to preload it at all… keep the memory for other things and let the GC do it's thing (especially 2.2 and the way Ruby GC is heading). I'm not retaining it in our app (we're on 2.2).
Personally I'm happy with "By default it's threadsafe, doesn't retain memory, etc" along with a Readme and code doc saying "If you're calling it multiple times you can keep the dictionaries around in memory like so" as a Readme/code-doc addition (also, that's why I put such a large comment at the top of the Zxcvbn::Tester class).
Before this change:
$ ruby -rmemory_profiler -I. -e 'MemoryProfiler.report { require "config/environment" }.pretty_print'
Total allocated 2312988
Total retained 476229

allocated memory by gem
-----------------------------------
activesupport-4.2.1 x 60094221
sprockets-3.0.1 x 56626099
2.2.0/lib x 28225732
zxcvbn-ruby-0.0.3 x 21126729
rubygems x 13918784
json-1.8.2 x 9353860
mime-types-2.4.3 x 7098487
erubis-2.7.0 x 5427205
actionpack-4.2.1 x 5290384
parser-<IP_ADDRESS> x 5268090
...
retained memory by gem
-----------------------------------
sprockets-3.0.1 x 14024689
2.2.0/lib x 7369496
zxcvbn-ruby-0.0.3 x 6813346
activesupport-4.2.1 x 5282673
json-1.8.2 x 3402509
...
retained objects by gem
-----------------------------------
sprockets-3.0.1 x 132076
2.2.0/lib x 85716
zxcvbn-ruby-0.0.3 x 85161
activesupport-4.2.1 x 57325
json-1.8.2 x 22427
mime-types-2.4.3 x 21063

After this change:
$ ruby -rmemory_profiler -I. -e 'MemoryProfiler.report { require "config/environment" }.pretty_print'
Total allocated 2057572
Total retained 390073

allocated memory by gem
-----------------------------------
activesupport-4.2.1 x 60149050
sprockets-3.0.1 x 56626099
2.2.0/lib x 28229776
rubygems x 13925583
json-1.8.2 x 9075397
mime-types-2.4.3 x 7098487
erubis-2.7.0 x 5427205
actionpack-4.2.1 x 5290384
parser-<IP_ADDRESS> x 5268090
activerecord-4.2.1 x 4941444
...
retained memory by gem
-----------------------------------
sprockets-3.0.1 x 14024689
2.2.0/lib x 7370079
activesupport-4.2.1 x 5285833
mime-types-2.4.3 x 3216794
json-1.8.2 x 3141886
rubygems x 1008626
...
retained objects by gem
-----------------------------------
sprockets-3.0.1 x 132076
2.2.0/lib x 85720
activesupport-4.2.1 x 57352
json-1.8.2 x 21180
mime-types-2.4.3 x 21063
rubygems x 10237
actionpack-4.2.1 x 7414

Perhaps @schneems or @samsaffron have a suggestion too?

I agree, let's conserve memory by default and add a note to the readme about the tradeoff and how to optimise further for performance.

Cool, I'll update the Readme with the note etc.

Have updated the Readme with an example

@stevehodgkiss :+1: :tada:

Release 0.1.0 with these changes. Thanks @toolmantim!

@stevehodgkiss it's been 2.5 years. Reckon it's time for a 1.0? ;)

@toolmantim maybe someday :)


Switch build system to hatch

This is a first attempt at switching to hatch as build system.
(This pull request can wait until the intended renaming of the project is done.)
The goal is to:

simplify project configuration in a single pyproject.toml (removing setup.cfg, requirements.txt, and tox.ini )
simplify building, testing and publishing (locally and in CI) with hatch build, hatch run cov and hatch publish
build on Linux, Windows and macOS.

Remaining tasks:

[ ] set up deployment to PyPI using hatch publish and a HATCH_INDEX_AUTH repository secret.

@imagejan do you mind if I rebase this onto the renaming branch and merge it into the renaming?
I think it would be double the work to deploy to pip and conda with faim-ipa and then change to hatch and pyproject.toml.

@tibuch sorry for the delayed reply. Sure, go ahead! I think you did so anyway, right? I'll have a look at your changes later.

@imagejan I think this is ready.
I have tagged and released current main as v0.2.5.
Once this is squash-merged we can rename the repo and test the publishing to pypi. Maybe we want to change the versioning to date-based versioning (see #113).
Merging this would close:

#96
#20
#38

And pave the way for:

#40
#113

set up deployment to PyPI using hatch publish and a HATCH_INDEX_AUTH repository secret.

This seems to be outdated and 'Trusted Publisher Management' from pypi should be used instead. The current publish workflow makes use of that.

set up deployment to PyPI using hatch publish and a HATCH_INDEX_AUTH repository secret.

This seems to be outdated and 'Trusted Publisher Management' from pypi should be used instead. The current publish workflow makes use of that.

Where did you find this information?
Prominent projects are using hatch publish which seems to be a bit more concise (i.e. less verbose) to me.
See for example: https://github.com/mkdocs/mkdocs/blob/65c24c21f0057ec4717d20d14d5fb7af22fe8caf/.github/workflows/deploy-release.yml

set up deployment to PyPI using hatch publish and a HATCH_INDEX_AUTH repository secret.

This seems to be outdated and 'Trusted Publisher Management' from pypi should be used instead. The current publish workflow makes use of that.

Where did you find this information?
Prominent projects are using hatch publish which seems to be a bit more concise (i.e. less verbose) to me. See for example: https://github.com/mkdocs/mkdocs/blob/65c24c21f0057ec4717d20d14d5fb7af22fe8caf/.github/workflows/deploy-release.yml

Some tutorial which refers back to this post: https://blog.pypi.org/posts/2023-04-20-introducing-trusted-publishers/

@imagejan should I move forward with this? We leave mobie-utils as runtime dependency.
If you agree I will squash/merge this, do the repo renaming and see how deploying to pypi works.

Agreed. Please go ahead 🚀 😄


Arun/questions/list

refactors Modal component to be resusable
adds Modal windows to add Question button

appruwuved

Thanks guise


There is no documentation on how to contribute

We need documentation telling folks how to contribute.
ie, gerrit workflow and github issues like this for bugs/RFEs

fixed by https://github.com/redhat-openstack/tripleo-quickstart/commit/cb309fe2ff16c8a9f02be119b4fbec390198fe6a


Consistent window corners on W10

Description of the new feature / enhancement
In most cases the UI is right-angled (classic Metro style), while in some cases it's rounded corner(modernized Fluent UI).

That means the UI of PowerToys is not very coordinated.
I use Windows 10 now and prefer the classic Metro style, but for Windows 11 users, they may prefer modernized Fluent UI,
adding a button to switch UI mode should make PowerToys looks more coordinated and will be very much appreciated by both Windows 10 and Windows 11 users !!
also sorry for my poor English.
Scenario when this would be used?
Scenario when this would be used : Every PowerToys user's daily use.
We will be happy to change the UI mode according to our prefererces.
Supporting information
No response

@Mj65536 Thanks for the feedback - the plan is to move as much of these PowerToys experience to a Windows 11 look and feel because of WinUI doing the same thing (that we use for e.g. Settings).
I think what you are referring to here is the rounded corners that are not always consistent (in e.g. Text Extractor) - that is something we could have a look at!

@Mj65536 Thanks for the feedback - the plan is to move as much of these PowerToys experience to a Windows 11 look and feel because of WinUI doing the same thing (that we use for e.g. Settings).
I think what you are referring to here is the rounded corners that are not always consistent (in e.g. Text Extractor) - that is something we could have a look at!

Okay, I got it.

I say we should use the system setting for the radius. I agree that the appearance on Windows 10 and Windows 11 can (and should) be different from each other, matching the OS.

I say we should use the system setting for the radius. I agree that the appearance on Windows 10 and Windows 11 can (and should) be different from each other, matching the OS.

Yeah, you are quite right. But the developers seem to just insist on following Windows11 and  design trending.


add axis parameter to pnorm(x, 2)

Hi Steven,
This makes it much easier to write sum-of-norms regularization e.g. without unrolling the sum (for example, network lasso paper you dont need |E| terms, http://web.stanford.edu/~hallac/Network_Lasso.pdf)
This is consistent with how linalg.norm() works, although i just added the code for p=2 for now.
Cheers,
Matt

Looks good to me. I've been meaning to add this for a while. Does it require NumPy 1.10? There was a NumPy error in the Travis tests (which use NumPy 1.9).
At some point we should add a version of SOC_Elemwise that can take a matrix for X and an axis argument, so everything is vectorized.

looks like keepdims was added to linalg.norm in 1.10 so i think its needed
(i believe weird things happen if numeric() collapses its return value to a
1D array)
On Sat, Jan 16, 2016 at 3:29 PM, Steven Diamond<EMAIL_ADDRESS>wrote:

Looks good to me. I've been meaning to add this for a while. Does it
require NumPy 1.10? There was a NumPy error in the Travis tests (which use
NumPy 1.9).
At some point we should add a version of SOC_Elemwise that can take a
matrix for X and an axis argument, so everything is vectorized.
—
Reply to this email directly or view it on GitHub
https://github.com/cvxgrp/cvxpy/pull/256#issuecomment-172271963.

let me know how you want me to resolve this... if 1.9 support is important than i think we could do something manually reshape to the right dimension

1.9 support isn't that important, but this seems like too small a thing to
require 1.10 for. Could you see if there's an easy fix for 1.9?
On Sat, Jan 16, 2016 at 3:34 PM Matt Wytock<EMAIL_ADDRESS>wrote:

let me know how you want me to resolve this... if 1.9 support is important
than i think we could do something manually reshape to the right dimension
—
Reply to this email directly or view it on GitHub
https://github.com/cvxgrp/cvxpy/pull/256#issuecomment-172272501.

sure thing, thats fixed now ... fixing one more python 3 bug (i typed xrange for some reason...)
agree re: SOC_Elemwise taking matrix args that would be nice here

fyi - green now :)

Great, merging now.


Use ScreenHandler#getType method instead of using mixin accessor

Fix issues with modded ScreenHandlers extending vanilla that overrides getType to return its own.

There are more screen handlers that have null type on vanilla though, like the creative and horse inventory.
Maybe mixin-ing getType to not throw an error when an injected field is true? Something like this:
public ScreenHandlerType<?> getType() {
    if (emi$bypassError) return this.type;
    if (this.type == null) {
        throw new UnsupportedOperationException("Unable to construct this menu by type");
    }
    return this.type;
}

Mixining ScreenHandler seems to work.

Ah, I think this mixin is overkill here. While other screen handlers may use null, it's fine to catch the exception in that case, because they're not going to have recipe handlers, so performance (for updating craftables etc) is not going to be a concern. And, truthfully, it's just kinda a nasty mixin to have to maintain. I'd be fine with the initial implementation provided there's just an extra case in the try block that skips the call only if it's a PlayerScreenHandler, sorry if I didn't convey that properly.

Will these changes fix integration with slotlink once the new emi update comes out with the recent commits? Even with deirn's update to slotlink fixing it, I'm still unable to grab items from the storage with EMI interface enabled. I can use the Fill Recipe button now as intended, but the game still treats the storage as a recipe viewer, so I can never take anything out of it to put into my inventory.


Builds are moved: git => GitHub Actions artifacts

Now builds are generated automatically on push and placed as artifacts
on corresponding GitHub actions. Unless you add [skip ci] prefix.
Merge after #78 merged

Should be recreated from scratch because files were changed


Add X2RGB10 format mapping for DMA-BUFs

Requires<EMAIL_ADDRESS>And for mesa driver it also requires https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/24353
Tested with WLR_RENDERER=vulkan sway + output * render_bit_depth 10.

Which version of ffmpeg is required for this code to compile? Is this also a case of a format available since ages, but simply not supported in the ffmpeg backend?


Install/Uninstall Kite makes plot pane uncloseable.

Issue Report Checklist

[x ] Searched the issues page for similar reports
[x ] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[x ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[ x] Could not reproduce inside jupyter qtconsole (if console-related)
[ x] Tried basic troubleshooting (if a bug/error)

[x ] Restarted Spyder
[ x] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[ x] Tried the other applicable steps from the Troubleshooting Guide

[ x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Since installing and removing (with difficulty) Kite the Plots pane will not close.
The option to uncheck it is greyed out.
I have uninstalled and reinstalled Spyder 5 and I cannot close the plots pane.
What steps reproduce the problem?

Install Kite
Uninstall Kite
Try to close plots pane

What is the expected output? What do you see instead?
Plots pane checkmark greyed out.
Paste Traceback/Error Below (if applicable)

PASTE TRACEBACK HERE

Versions

Spyder version: 5.0.0
Python version: 3.9.2
Qt version:
PyQt version:
Operating System name/version: Mac OS Big Sur 11.2.3

Dependencies

PASTE DEPENDENCIES HERE

Managed to close it with "close current pane" after all other panes closed. That is the only way

This will be fixed in 5.0.1.

Thanks @ccordoba12


FD_SETSIZE redefined

The macro FD_SETSIZE is defined in c:/devkitPro/wut/include/nsysnet/socket.h and in c:\devkitpro\devkitppc\powerpc-eabi\include\sys\select.h with a different value each. Is this considered a bug in wut?
In file included from c:/devkitPro/wut/include/utils/TCPServer.hpp:6:0,
                 from d:/Consoles/WiiU/Applications/HBL/tcpgecko_plugin/src/tcpgecko/TCPGeckoServer.hpp:4,
                 from d:/Consoles/WiiU/Applications/HBL/tcpgecko_plugin/src/tcpgecko/TCPGeckoServer.cpp:1:
c:/devkitPro/wut/include/nsysnet/socket.h:5:0: warning: "FD_SETSIZE" redefined
 #define FD_SETSIZE 32

In file included from c:/devkitPro/wut/include/utils/TCPServer.hpp:4:0,
                 from d:/Consoles/WiiU/Applications/HBL/tcpgecko_plugin/src/tcpgecko/TCPGeckoServer.hpp:4,
                 from d:/Consoles/WiiU/Applications/HBL/tcpgecko_plugin/src/tcpgecko/TCPGeckoServer.cpp:1:
c:\devkitpro\devkitppc\powerpc-eabi\include\sys\select.h:42:0: note: this is the location of the previous definition
 # define FD_SETSIZE 64

If yes, I would appreciate some fix or suggestion to fix it. Maybe something like:
#  ifndef	FD_SETSIZE
#	define	FD_SETSIZE	32
#  endif

If I change the include order, the warning is fixed:
#include <nsysnet/socket.h>
#include <stdio.h>

This would be dealt with by #136, which shims the network layer and runs everything through devkitPPC (with a setsize of 64).

Since this issue has been open for so long, why don't we just comment out/delete the #define FD_SETSIZE 32 and call it a day? It seems to work for suppressing the warning and the constant doesn't seem to serve any useful purpose within wut or a homebrew application's code.

This issue was fixed by the addition of the devoptab network socket layer introduced in wut 1.0.0-beta12.


[c++] Fix bug in nnz of variant-indexed dataframes

Issue and/or context: Pre-existing bug found while working on issue #2407 / [sc-51048].
I ran sdf.count() on a variant-indexed dataframe which has dim0 being a string (which the spec allows even though most users don't take advantage of this). I got the error
[TileDB::FragmentInfo] Error: Cannot get non-empty domain; Dimension is variable-sized

There is a repro now built into the unit-test cases on this PR.
Changes:
Only use the code that assumes dim0 is int64 soma_joinid when that is in fact the case.
Notes for Reviewer:

Leaving in draft while I address a unit-test failure


fine-tuned issues

When I fine-tuned the ip-adapter-plus-face_sdxl_vit-h.bin model, I exchang the ImageProjModel to Resampler in the tutorial_train_sdxl.py , and then reported this error. I use https://huggingface.co/h94/IP-Adapter/tree/main/models/image_encoder as image_encoder. Could you please help me find out what the problem is?
RuntimeError: Error(s) in loading state_dict for Resampler:
size mismatch for latents: copying a param with shape torch.Size([1, 16, 1280]) from checkpoint, the shape in current model is torch.Size([1, 16, 2048]).
size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 1280]).
size mismatch for proj_in.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for proj_out.weight: copying a param with shape torch.Size([2048, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 2048]).
size mismatch for layers.0.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
size mismatch for layers.0.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
size mismatch for layers.0.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
size mismatch for layers.0.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.0.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
size mismatch for layers.0.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
size mismatch for layers.1.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
size mismatch for layers.1.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
size mismatch for layers.1.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
size mismatch for layers.1.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.1.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
size mismatch for layers.1.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
size mismatch for layers.2.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
size mismatch for layers.2.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
size mismatch for layers.2.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
size mismatch for layers.2.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.2.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
size mismatch for layers.2.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).
size mismatch for layers.3.0.norm1.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.0.norm1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.0.norm2.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.0.norm2.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.0.to_q.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([768, 2048]).
size mismatch for layers.3.0.to_kv.weight: copying a param with shape torch.Size([2560, 1280]) from checkpoint, the shape in current model is torch.Size([1536, 2048]).
size mismatch for layers.3.0.to_out.weight: copying a param with shape torch.Size([1280, 1280]) from checkpoint, the shape in current model is torch.Size([2048, 768]).
size mismatch for layers.3.1.0.weight: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.1.0.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([2048]).
size mismatch for layers.3.1.1.weight: copying a param with shape torch.Size([5120, 1280]) from checkpoint, the shape in current model is torch.Size([8192, 2048]).
size mismatch for layers.3.1.3.weight: copying a param with shape torch.Size([1280, 5120]) from checkpoint, the shape in current model is torch.Size([2048, 8192]).

I solved it by following ：
image_proj_model = Resampler(
dim=1280,
depth=4,
dim_head=64,
heads=20,
num_queries=args.num_tokens,
embedding_dim=image_encoder.config.hidden_size,
output_dim=unet.config.cross_attention_dim,
ff_mult=4,
)


This card action is disabled because you've blocked

The following error message comes up when trying to resume or pause matches:
"This card action is disabled because you've blocked"
This is not the only issue that we are having. Since a while there are no new matches being made. Icebreaker is added, but it does nothing. Any clue what to do?

Hi @AronPGGM ,
Thanks for raising the query.

There is possibility that app service might got failed. If yes, please sync the app service.

If still facing the issue,  please share the app insight logs.

Hi @gsv022, thanks for reaching out. We can not find the screen that you communicated. I checked as Teams admin/Sharepoint admin and a colleague checked as Global Reader.

Replying so the case will not be closed.

Hi @AronPGGM ,
Could you please confirm that the deployment was done by you or by some other.
To reach the app service. Go to Azure Portal-->Select the resource group (where the icebreaker resources get created)-->Go to App Service-->Click Deployment Center-->Check for success state in the logs section


[composer] error when sharing halo

Error when joining HALO on new profile:
NOTE: No error is visible to the user.

see for #5568 for another version of this with different errors on latest 0.4.2-ish dev


ASR supports ADE v1 or ADE v2 or both ?

Please could you clearly confirm which version of Azure Disk Encryption (ADE) is currently supported by ASR replication feature ?
Note section at the top of the page says "Azure Site Recovery currently supports only Azure Disk Encryption-enabled VMs that run a Windows OS. Azure Disk Encryption-enabled VMs without Azure AD app are supported only if they are using managed disks. "
Azure Disk Encryption versions:
v1: requires an Azure Active Directory (Azure AD) application parameter to enable VM disk encryption
v2: not require an Azure Active Directory (Azure AD) application parameter to enable VM disk encryption
Is it correct to say that both versions are supported ?
Thanks in advance

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: de2b1249-5247-938e-cc2a-bb5ba92a910d
Version Independent ID: fc999d69-7ce6-dacc-5add-7c591680bf7c
Content: Configure replication for Azure Disk Encryption-enabled VMs in Azure Site Recovery
Content Source: articles/site-recovery/azure-to-azure-how-to-enable-replication-ade-vms.md
Service: site-recovery
GitHub Login: @asgang
Microsoft Alias: sutalasi

Thanks for your comment. The feedback has been shared with the content owner for further review. Thanks for your patience.

Thanks for your feedback. Yes, supported. For ADE without AAD only Windows VMs with managed disks are supported.  I clarified in a couple of articles at https://docs.microsoft.com/en-us/azure/site-recovery/azure-to-azure-how-to-enable-replication-ade-vms and https://docs.microsoft.com/en-us/azure/site-recovery/azure-to-azure-common-questions. The changes should be live later today.
@please-close


IB Designable error

Main.storyboard: warning: IB Designables: Using class UIView for object with custom class because the class CosmosView does not exist
--
If I am missing a setting or checking off something, please let me know. I am a new user to this control.
-Kevin.

Hi @KMessina1, thanks for reporting. Yes, I saw this annoying error many times myself. It could be fixed in Identity Inspector:

Set the Class to CosmosView.
Set the Module to Cosmos if you used CocoaPods or Carthage setup method. Leave it blank if you used file setup method.
Clean the build.
Restart Xcode.

Let me know if it works for you. :)

Yes it did. I have had this same issue with my own IBDesignable's periodically in the past and stopped using them. I had hoped by now Apple would have solved the issue. Maybe xCode 9's new IB Designer will address it better. Great control by the way. Have it in a Tableview and works very nicely. Good job creating it and thank you for making it open to others.
-Kevin.

I am glad it worked. I think we can leave this ticket open, since it can be useful to others.

Maybe xCode 9's new IB Designer will address it better.

I hope so too :).

Hi,
On XCode 9b5, I have a "Designables Build failed" in the Interface builder right Panel. Module is set to Cosmos, I've cleaned the project, started XCode again but still have this error. (Using Cocoapods with 11.0.3).
Thanks !

@wasappi oh ... good! :) Let me know if you still have any problems in Xcode 9.

Closing the issue. Feel free to reopen if needed.

I also have the  error: IB Designables: Failed to render and update auto layout status for ViewController (BYZ-38-t0r): dlopen(Cosmos.framework, 1): no suitable image found.  Did find:
Cosmos.framework: required code signature missing for 'Cosmos.framework'
I did follow your instructions, but it doesn't work.

sarah

Thanks for reporting @SarahNys. It looks like in #105 people reported similar errors, so their solution may help.


datahub frontend does not show the avro schema in table tab

Describe the bug
I was ingesting some kafka topics, and so, using: "platformSchema":{"documentSchema": schema}, as written inside the kafka-etl.py.
Everything works as expected during the ingestion, but the frontend looks weird.
I am using latest master from https://github.com/linkedin/datahub. Nothing too crazy appeared in the javascript console (just some 404)
Am I doing something wrong / is it the intended behaviour?
I was expecting to have all first level fields visible when viewing as a table. (image present laters)
To Reproduce
Steps to reproduce the behavior:

Ingest this snippet into kafka MetadataChangeEvent_v4 topic:
{"auditHeader":null,"proposedSnapshot":{"com.linkedin.pegasus2avro.metadata.snapshot.DatasetSnapshot":{"urn":"urn:li:dataset:(urn:li:dataPlatform:kafka,fxt-data-store-sf-example,dev)","aspects":[{"com.linkedin.pegasus2avro.dataset.DatasetProperties":{"description":{"string":"some description"},"uri":null,"tags":["exampletag1","exampletag2"],"customProperties":{"producer":"fxt-data-store","consumer":"fxt-data-bi"}}},{"com.linkedin.pegasus2avro.schema.SchemaMetadata":{"schemaName":"fxt-data-store-sf-example","platform":"urn:li:dataPlatform:kafka","version":1,"created":{"time":1599118463,"actor":"urn:li:corpuser:","impersonator":null},"lastModified":{"time":1599118463,"actor":"urn:li:corpuser:","impersonator":null},"deleted":null,"dataset":null,"cluster":null,"hash":"","platformSchema":{"com.linkedin.pegasus2avro.schema.KafkaSchema":{"documentSchema":"{\"type\": \"record\", \"name\": \"test1\", \"namespace\": \"fxt.data.store\", \"fields\": [{\"type\": \"string\", \"name\": \"Status\"}, {\"type\": {\"type\": \"long\", \"logicalType\": \"timestamp-millis\"}, \"name\": \"LastModifiedDate\"}, {\"type\": \"string\", \"name\": \"MPP_QualityCaseTopic__c\"}, {\"type\": {\"type\": \"long\", \"logicalType\": \"timestamp-millis\"}, \"name\": \"Last_Status_Change__c\"}, {\"type\": \"string\", \"name\": \"MPP_IncidentCategory__c\"}, {\"type\": \"string\", \"name\": \"MPP_IncidentSubCategory__c\"}], \"doc\": \"\"}"}},"fields":[{"fieldPath":"","jsonPath":null,"nullable":false,"description":{"string":""},"type":{"type":{"com.linkedin.pegasus2avro.schema.BooleanType":{}}},"nativeDataType":"string","recursive":false}],"primaryKeys":null,"foreignKeysSpecs":null}}]}},"proposedDelta":null}
Go to the description inside datahub
The images are as mentioned below.

Expected behavior
I was expecting all columns of first level to appear inside hte table view.
Screenshots

Desktop (please complete the following information):

OS: macOS
Browser chrome
85

Additional context
NA

Hm, solved on my own.
The issue was coming from the field section that need to be extracted on top of it:

You need to provide both the schema and the list of fields in order for the frontend to work correctly.


Provide integration with Github actions

Provide an integration with Github actions.
The desired solution should kick-off a CI workflow with Github actions (see https://docs.github.com/en/free-pro-team@latest/actions/guides/about-continuous-integration for reference) and once the container image is built, the integration should trigger Keptn to continue kick-off the deployment, testing, evaluation, and potential promotion of the built image.
Therefore, the Github integration has to send a sh.keptn.event.configuration.change to Keptn.
Definition of Done

[ ] repo created in the keptn-sandbox organization and code contributed to this repo
[ ] depending on the implementation: have a structured and detailed README.md how to achieve this or have an implementation that can be used out-of-the-box (some research might be needed what can be done here)
[ ] integration sends a sh.keptn.event.configuration.change after the image is built from the Github action integration, which triggers a deployment with Keptn

@jetzlstorfer would this be something that is covered by https://github.com/keptn/keptn/issues/2797 ? Or is this a separate idea?

@jetzlstorfer would this be something that is covered by https://github.com/keptn/keptn/issues/2797 ? Or is this a separate idea?

@christian-kreuzberger-dtx  My idea was to provide an integration for Keptn users so they can kick off Keptn sequences from Github actions. Sounds like this is not proposed in #2797
I'm fine with marking this one as duplicate and track progress in the other issue.

My idea was to provide an integration for Keptn users so they can kick off Keptn sequences from Github actions. Sounds like this is not proposed in #2797

That's exactly what is proposed in #2797, but more generic. I've modified the issue description a little bit to make it clearer.

Hi @oleg-nenashev , can you take care of this?

I'm interested in working on it as part of GSOC.


Make the CLI output useful [if only for prettier debugging]

responding to https://github.com/joshbduncan/word-search-generator/pull/26#issuecomment-1341718692

Yes, I should do just that.  How I'd think of handling numbers greater than 9 is by drawing a grid, similar to https://github.com/duck57/LightbulbSharkCipher/blob/master/draw.py
That said, that's also some clunky terminal display for a crossword of significant size.

A simple solution would just be to make each "grid" section 2 characters (wide) so you can handle up to 99 words.
-----------------------
       CROSSWORD
-----------------------
▓▓ 12       ▓▓ ▓▓ ▓▓ ▓▓
15                   ▓▓
13 ▓▓ ▓▓ ▓▓ ▓▓ ▓▓ ▓▓ 11
   ▓▓ ▓▓ ▓▓ ▓▓ ▓▓ ▓▓
   ▓▓ ▓▓ ▓▓ ▓▓ ▓▓ ▓▓
   ▓▓ 16
   ▓▓ 17
▓▓ ▓▓ 14          ▓▓ ▓▓


DataLoader2 Feature Documentation

📚 The doc issue
As we further develop DataLoader2, we should add a separate page under ("API Reference") for DataLoader2 documentation. The page should cover topics such as adapters, linter, reading service, and etc, as well as information related to writing custom adapters and reading service.

Closing as the most of docs have been updated.


Refactor code (and some other changes)

This is following discussions on #26 and during the meeting that now that we are going toward a stable version, it would be worth refactoring the code to make it a bit more future-proof.
This is most likely not final at all, and it'd be great to hear everyone's thoughts to refine it. And also to check that it still works as expected (in particular if I have inadvertently deleted something important!). And we can open dedicated issues for anything arising for discussions here.
Here is a schema of the draft new structure:

The guiding principles in refactoring were to try to:

keep each step separate, with a different class/script
minimised chained imports (file A imports B which imports C, file D also imports B etc.)
ensure that the main steps can be easily followed from cats.run
make it easy (when possible) to add more API/different optimisation tools in the future

As a reminder, the main steps of CATS are:

Read data/information from the command line and the config file
Pull forecast data from a carbon intensity (CI) API
Find the window of time (start time + end time) that would minimise average carbon intensity within the known forecast
(optional) Estimate the total energy used and carbon footprint by starting or not at the best time

An overview of some changes beyond structure
I aimed at keeping the functionalities as they were as much as possible, just trying to enhance when I could. Here are some changed:

Output as been changed a bit, here is the new one for now:

Using config.yml found in current directory
Using carbonintensity.org.uk for carbon intensity forecasts.
Using location provided: PE29

Carbon intensity forecast loaded for between 29/05/2023-20:30 UTC and 31/05/2023-20:30 UTC.

Best start time: 31/05/2023-12:00 UTC (in 39.1 hours)
	 Expected end time: 31/05/2023-17:50 UTC
	 Expected average carbon intensity: 70.03 gCO2e/kWh

Estimated carbon footprint of running job at best time: 10.68 gCO2e
	vs running it now: 13.26 gCO2e (2.58 gCO2e saved)
	vs running at worst time (31/05/2023-01:00 UTC): 17.54 gCO2e (6.87 gCO2e saved)
	- Estimated energy usage: 0.15 kWh

Regarding how to find the best start time:

the "simple" method has been removed, as it doesn't really hold to minimise carbon intensity
more discussion is needed  regarding what data the carbon intensity.org.uk API sends back. More on this on issue #42.

Now all carbon intensity values with timestamps (from the API or computed internally) are stored as a CarbonIntensityEstimate (defined in CI_api_interface.py but could be defined more centrally). It is a simple adaptation of the previously existing CarbonIntensityAverageEstimate and  has properties such as value, start, end and timedelta and can be sorted by CI value.

Added some progress update/error messages throughout (more needed!)

How this release addresses existing open issues

#32 : as it stands, it only plans job within the forecast window (latest start time is end_of_forecast - duration). How to remove this constraints probably needs to be discussed further as it's not trivial.
#22 : this new structure should make it much easier to add new countries, directly in CI_api_interface.py (e.g. here)
It should finish closing #3 as carbon footprint estimates are now fully functional

Open questions:

Why some things that are not errors are being printed on stderr and others on stdout? As a temporary measure, all updates/messages are sent to stdout and only errors/warnings to stderr, but would be great to hear some thoughts on that.
I suspect I may have broken the pipeline with the at linux scheduler with the different output, any thoughts on the best way to pass information on start time?

What is still needed
Not necessarily before merging, but here is a list non-exhaustive of what still needs to be done (the idea of this PR is to discuss this new structure while polishing the details).

Write more tests
Comment the code/functions more thoroughly
Check how it deals with edge cases, absurd values, etc. (and add error messages)
Rewrite README
Check how the new structure works with the at linux command

Thanks for these @tlestang

I don't see the benefit of introducing a new plain class for the API interface, compared to the currently implemented APIInterface namedtuple. It provides the same access to the request url and response parsing functions through dot notation and a lot less code.

More on this below after the second comment, but regarding class vs namedtuple, I don't mind as long as it's easy to switch between APIs (so we would need a master function to pick the right namedtuple in case it's not provided directly by the user). I can draft a new commit reversing to namedtuple.

More problematic to me is that the API-specific logic of forming the query URL and parsing the response is coupled back to the API interface code. Somebody wishing to interface cats with a new web API would have to modify the implementation of CI_API_interface itself, adding a new statememt if self.choice_CI_API == 'myapi.org' and hardcoding API-specific code in there. I feel like this is a step back from the current implementation (introduced in #37) in which interfacing against a different API is only a matter of providing two functions (possibly grouping them into a new APIInterface object) without having to touch the code of cats itself at any point.

This is an interesting point, and probably needs more thinking. However not sure about not having to touch the code, I see two different use cases here:

We or other contributors will want to add other CI APIs (e.g. for other countries), and we ideally want to make them part of CATS so that these new APIs are available to the whole community. In this case, it would be good to have all the URL/parsing codes in the same place and api_inferface is a good place for that (it also makes it easier to add things by copy-pasting). And in terms of how much hassle it is to add it, it's equivalent now and with the new code (api_interface needs to be modified either way, and current code requires messing with __init__.py as well), but the existing code doesn't allow user to easily pick an API, this is what the new argument --api-carbonintensity introduces.

Second use case is if users want to pass their own API wrapper directly to CATS without having to modify the code. And in this case I agree, it would be good to make it possible in an easier way. But how would that work in practice? It would be good to have an idea of how the user would do it if we want to implement it.

I opened issue #44 to have a space to discuss (2) further.

Lastly, do we really want to rename api_query.py and api_interface.py ?

The motivation behind that is to clarify that the code in there is related to pulling carbon intensities, in case more APIs for other parts of the code are included later. But at the moment, this is only future-proofing so I don't mind removing the CI_ in the names if it's an issue for now.

Are we good to merge that into main? From what I can see, it now covers everything already present in main (thanks for the feedback! and please do highlight if anything's still missing for users :) ), plus it fixes bugs and edge cases (and hopefully allows more modularity for future-proofing). windowed_forecast in particular has been changed compared to first PR here to better build on what the existing module did, and also combine both sum and trapezoidal integration with the right bounds (its exact implementation can of course be revised in another PR if people think it should be coded differently, suggestions much welcome, but at least outputs is accurate for both methods for now)
There are quite a few things still needed, e.g. more extensive tests and checks of arguments for edge cases (but I believe these are still outstanding in main anyway, so merging would make it easier to submit future PR)

I'm not sure exactly where we are with this one - does it make sense to discuss at the meeting on Wednesday. One possibility would be to try and chop the changes into smaller sets but I think some wider discussion around the architecture could be useful.
My two cents is that a useful approach to thinking about this is to consider other future uses of (maybe bits of) the code base and think about how they would work. So, for cats, I imagine a user who has some big python package providing a service that runs some regular processing, and it's the processing that they want to schedule (no need to integrate with a scheduler). Or, somebody wants to post-process job scheduler logs.
Some (partial) answers to the questions above:

Why some things that are not errors are being printed on stderr and others on stdout? As a temporary measure, all updates/messages are sent to stdout and only errors/warnings to stderr, but would be great to hear some thoughts on that.

The way the python currently interfaces with at is via a shell backtick expansion - stuff that comes out of stdout from cats goes into an argument to at and (from the point of view of at) everything else is ignored and goes to the terminal. We could sensibly use python's subprocess module to do this instead. This would give us more control but could limit what the user could do.

I'm closing this for now as we discussed it won't be merged as-in. But let's keep the branch around for reference.


elementary os no icon in the system tray

In elementary os does not display an icon in the system tray. Notifications are displayed.

Elementary OS Loki removed AppIndicators in their 0.3/4 beta preventing many normal tray icons from being displayed.
You can look at these three pages:
https://github.com/electron/electron/issues/6479
https://github.com/electron/electron/issues/5050
https://github.com/electron/electron/pull/7319

Just an update, apparently Electron 1.4.2 fixes the missing icon in Elementary you just need to change the version in the get_prerequisites.sh script located in /usr/bin/spotifywebplayer and then run the script with sudo sh /usr/bin/spotifywebplayer/get_prerequisites.sh

Thank you, it works perfectly.


Update twitter-the-algorithm-analysis-deeplake.ipynb

Changed model to model_name
Updated the ChatOpenAI parameter to model_name

Fixes # (issue)
Before submitting

Who can review?
Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

thanks @prateekkeshari!


Improve article 1910.13912

Exact location of issue
Many places in this paper, eg https://ar5iv.labs.arxiv.org/html/1910.13912#S1.p1.14
Also https://ar5iv.labs.arxiv.org/html/1910.13912#bib.L1
Problem details
The references do not render, and the links to them appear as they are mentioned.
Desktop (please complete the following information)

OS: Ubuntu
Browser: firefox, chromium

(Optional) Screenshots

Thank you for the report!
It appears that we have a regression in certain uses of amsrefs.sty, as latexmlc fails to recognize any of the bib keys. This may be specific to the way we use latexml for bibliography processing in arXiv - I'll investigate.
example snippet from the paper source:
\begin{bibdiv}
\begin{biblist}

\bib{alon_spencer2016book}{book}{
      author={Alon, N.},
      author={Spencer, J.},
       title={{The Probabilistic Method}},
     edition={4},
   publisher={Wiley},
        date={2016},
        ISBN={9781119061953},
}
%...


Removing deprecated endpoints

Description
Relates to https://github.com/chanzuckerberg/napari-hub/issues/1202
Changes introduced:

Removes /plugins and /plugins/excluded api and related components
Updates bdd-test to remove tests for the deprecated endpoints
Cleans up bdd-test to better follow pep8 guidelines for formatting and typing hints
Removes read permission to the plugin_blocked table for the backend module
Removes unused API key from backend lambda env variable and update cloudwatch event rule that is no longer needed.

Notes

Do not merge this PR until the adoption of /plugin/index/all is confirmed by internal users.

I'm still not set up to run the BDD tests

Hopefully, the following helps. Please let me know if you run into any issues with this. 😸
Setting up to run BDD tests
You might have to run pip install on bdd-test-requirements.txt to ensure you have all the dependencies needed for the BDD tests.
pip install -r bdd-test-requirements.txt

Running BDD tests locally
If you want to run the tests against your local environment, ensure you have the backend application running locally in another terminal, and run the following command from the backend folder:
PORT=12345 python -m pytest bdd_tests

12345 is the default port; you don't need to specify the PORT variable if you run your application on the default port. But you can use that if you run your Flask application on a different port.
Running BDD tests against staging and prod
The prefix variable is helpful in cases where you are running against non-local environments. For staging and production, it can be run by setting the PREFIX value to staging and prod, respectively. For example, to run the BDD tests against the prod-environment, use the following command:
PREFIX=prod python -m pytest bdd_tests

Running BDD tests against dev environments
The PREFIX variable also comes to the rescue in this case. The dev environment can have any prefix, so you would have to specify the dev-branch name that acts as the prefix for the environment. For example, to run the BDD tests against the dev-shared dev-environment, use the following command:
PREFIX=dev-shared python -m pytest bdd_tests

I'm assuming we'll wait until @seankmartin has confirmed their team has migrated away from these endpoints before merging in?

Yes, that is correct.

Just to confirm, are we keeping /plugins/<plugin>/versions/<version> , /manifest/<plugin> and /manifest/<plugin>/versions/<version> around for internal testing/utility purposes?

Yes, retained these two endpoints as they provide functionality that is not covered in the current public endpoints and could prove useful in internal testing/debugging.

Hey @manasaV3, we are using the endpoint plugin\index\all for our projects now, thanks!

@manasaV3 Would you want to merge this before regression testing starts today?


reset_offset

Sorry, this is not really cluster-specific functionality. You should instead create a sarama.Client and use NewConsumerFromClient to instantiate the cluster, then perform partition-specific operations with plain sarama.Client.


Problem with watchify recipe

It seems that if you create a task similar to the "watch" task in watchify recipe gulp.dest is not emitting "end" event in some cases. This prevents you from continuing the pipeline for example with gulp-livereload.
Everything seems to be working fine when you run the task for the first time, or if you save a file that's required by the entry file. When you save the entry file watchify emits the "update" event, the rebundle function is called and bundle file is written, but the pipeline seems to get stuck to gulp.dest and everything after that is ignored.
Also everything seems to be working fine if you have no require calls on your entry file.
I was able to track it down to vinyl-fs library's writeStream method, where for some reason the "finish" listener is not triggered on line 15.
Here's my task

The watchify recipe was redone, let me know if this is still an issue and I will reopen


Resizing chart causes axis label to overwrite tick labels

I created a chart of size 284 x 284.  It looks like this:

I then resized the chart (in JavaScript) to be 398 x 398.  It looks like this:

Notice the "Predicted" label is on top of the X-axis tick labels.

I have made a fix to the axis layout code which I suspect will help with this. I'm going to mark this as "code pending" and when I commit the 2.6 version with all fixes, we should re-test this and see how it works


[Snyk] Security upgrade nginx from 1.19 to 1.24

This PR was automatically created by Snyk using the credentials of a real user.Keeping your Docker base image up-to-date means you’ll benefit from security fixes in the latest version of your chosen image.
Changes included in this PR

Dockerfile-old

We recommend upgrading to nginx:1.24, as this image has only 108 known vulnerabilities. To do this, merge this pull request, then verify your application still works as expected.
Some of the most important vulnerabilities in your base image include:

Severity
Priority Score / 1000
Issue
Exploit Maturity

714
Exposure of Resource to Wrong Sphere SNYK-DEBIAN10-CURL-3065760
No Known Exploit

714
Directory Traversal SNYK-DEBIAN10-DPKG-2847944
No Known Exploit

714
Out-of-bounds Read SNYK-DEBIAN10-LIBTASN16-3061094
No Known Exploit

714
Buffer Overflow SNYK-DEBIAN10-OPENSSL-1569403
No Known Exploit

714
OS Command Injection SNYK-DEBIAN10-OPENSSL-2807585
No Known Exploit

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Directory Traversal

Closing as redundant.


Windows Remote Desktop not starting with user name containing a '.'

Guac spooler is failing to retrieve the user password from keyvault when the username contains a '.' (dot)
2022-07-19 08:45:10,659 ERROR: (BadParameter) The request URI contains an invalid name: john.john-password
Code: BadParameter
Message: The request URI contains an invalid name: praveen.routhu-password
Traceback (most recent call last):
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/guac/spooler.py", line 107, in get_user_password
    retrieved_secret = client.get_secret(f"{username}-password")
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/azure/keyvault/secrets/_client.py", line 73, in get_secret
    **kwargs
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/azure/keyvault/secrets/_generated/_operations_mixin.py", line 1574, in get_secret
    return mixin_instance.get_secret(vault_base_url, secret_name, secret_version, **kwargs)
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/opt/cycle/guac/venv/lib/python3.6/site-packages/azure/keyvault/secrets/_generated/v7_3/operations/_key_vault_client_operations.py", line 704, in get_secret
    raise HttpResponseError(response=response, model=error)
azure.core.exceptions.HttpResponseError: (BadParameter) The request URI contains an invalid name: john.john-password
Code: BadParameter
Message: The request URI contains an invalid name: john.john-password

https://github.com/xpillons/azhop-guac/issues/10


Different base model and lora model in prolificdreamer config

Thanks for your impressive re-implementation of prolificdreamer. I noticed that you use stable-diffusion-2-1-base for the pre-trained sd model and stable-diffusion-2-1 to initialize the Lora model, and these two models use schedulers with different prediction types. I don't know if such a setting is from the original prolificdreamer and whether it improves the performance. I would be grateful if you could please provide some explanation about that.

Thank you! Yes the authors claim that they use v-prediction to train the LoRA model:

In my experiments, v-prediction performs better than epsilon prediction especially after t annealing, where using epsilon prediction often causes over-saturated color. You can also try to compare the two choices yourself :)

Exactly, I have tried using epsilon prediction for both models, and it leads to over-saturated color. Thanks for your help!

could you explain why do these operator on noise_pred by lora and then calculate loss with pretrain sd noise?

Hi @silence401 ,
I think in the v-prediction setting, the output of the lora model should be considered as v, and we have to convert it back to the meaning of noise_pred and then calculate loss with pre-train sd noise.


Re-enable no debug no debuginfo while building to reduce build time

Hello again,
While troubleshooting the issue with the patch files, I had noticed that running without debug appeared to be working correctly again. I haven't dug into why, but we also never figured out why exactly it stopped working in the past.
This PR adds back in the parameters for removing debug and debuginfo from the build. This cuts the build time almost in half compared to building everything. It also appears to produce ever so slightly smaller files (an MB or 2 at most).
Current build from my fork here https://github.com/evanjarrett/fedora-acs-override/actions/runs/4080002541
The main concern with merging this is if it will ever break again and start creating larger files. I'm assuming that is not the intention and must have been a bug in the rpmbuild system.

Thanks again @evanjarrett ✨ ✨ ✨


Feature suggestion: process output of terraformer so that terraform can be applied directly to it...

(If this is already done, I could not find a reference to it)
I have two use immediate use cases for terraformer:

Take an existing installation (e.g., an entire GCP Project or an entire AWS (sub)account) that has been changed manually too many times and turn it into something that can be cloned in a different project/subaccount.

Again, take an existing installation, create a usable configuration as with the previous case, and use it with a tool such as https://github.com/fugue/regula for security auditing.

I volunteer to do this if nobody is working on it, but I hate doing duplicate work!

Hi @symbiont-ji ,
I'm thinking about doing something similar and happy to collaborate.
What do you think is the best way to achieve your objective #1 ?


Flutter isn't connecting to my device after updating sdk

I tried doing flutter run but it says device not connected when actually my android emulator is opened, if i use my mobile device it works but if i use android emulator it doesn't
I ran flutter doctor -v and this is the result

flutter doctor -v
[√] Flutter (Channel stable, 1.22.1, on Microsoft Windows [Version 6.3.9600], locale en-US)
    • Flutter version 1.22.1 at C:\src\flutter
    • Framework revision f30b7f4db9 (2 months ago), 2020-10-08 10:06:30 -0700
    • Engine revision 75bef9f6c8
    • Dart version 2.10.1

[√] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at C:\Users\brightisaac\AppData\Local\Android\Sdk
    • Platform android-30, build-tools 30.0.3
    • Java binary at: C:\Program Files\Android\Android Studio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)
    • All Android licenses accepted.

[√] Android Studio (version 3.5)
    • Android Studio at C:\Program Files\Android\Android Studio
    • Flutter plugin version 44.0.1
    • Dart plugin version 191.8593
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[√] VS Code (version 1.44.2)
    • VS Code at C:\Users\brightisaac\AppData\Local\Programs\Microsoft VS Code
    • Flutter extension version 3.12.2

[!] Connected device
    ! Device emulator-5554 is offline.

! Doctor found issues in 1 category.`

My android sdk path is already updated which is at android-sdk: C:\Users\brightisaac\AppData\Local\Android\sdk

Please what do i do to connect it to my android device

Hi @BrightCode1
I cannot reproduce this issue on the latest stable channel.
Can you please upgrade to the latest  dev and try again
flutter channel stable
flutter upgrade --force
flutter doctor -v
If the problem persists, Can you please provide your flutter doctor -v with the upgrade and flutter emulators --verbose logs,
Thank you

I changed my command shell from git bash to terminal and ran flutter run and it worked


No GUI

Installed the container (-latest) on a Qnap NAS to day, install run OK but I don't get the "open Link" sign, after I start the container, to get the GUI in the browser.
I have your jdownloader-2 and filezilla container installed and with both of them I have no problems to open the GUI's.
Any idea ?

What is the container's configuration ?  Did you mapped ports ?  Are you able to manually open the GUI in the browser (http://<qnap IP>:<mapped port>) ?

For beginning I have 2 other container installed from you, filezilla and jdownloader2. Both work fine and I have not mapped any ports on this two.
However what I have noticed is that the CPU shows 0% activity on this container.
Find the configuration attached.
config.txt

If you don't map ports yourself, maybe qnap is doing it automatically.
What is the URL of filezilla/jdownloader2 when you access them?
Could you provide the output of the following command:
docker inspect <name of the container>

URL for
filrzilla  http://IP of NAS :49157/
jdownloader  http://IP of NAS:49154/
Output of the docker inspect command attached.
docker inspect.txt

Ok so there is no port mapping configured.  I cannot tell why QNAP did not create mapping automatically like your other containers...  Maybe try to re-create the container or add the mapping yourself ?

How do I find out which container ports I should be mapping ?

I got it, mapped the ports 5800 and 5900 and its working now.
Thanks for you help.

Mapping only port 5800 is enough.  Port 5900 is used only when you want to access the UI via a VNC client.

@jlesage would you like me to add a section for QNAP config tips to the readme? If yes, let me know what section to nest under.


Complex aliases does not appear to be highlighted properly

➜ ~ zimfw info
zimfw version: 1.1.1 (previous commit is 6129062)
ZIM_HOME:      /Users/mark.dziewit/.zim
Zsh version:   5.7.1
System info:   Darwin SpaceRocketLauncher 19.3.0 Darwin Kernel Version 19.3.0: Thu Jan 9 20:58:23 PST 2020; root:xnu-6153.81.5~1/RELEASE_X86_64 x86_64

gist of .zimrc:
https://github.com/marek2901/config_files/blob/master/.zimrc
gist of .zshrc:
https://github.com/marek2901/config_files/blob/master/.zshrc

Description
Some of my aliases appear to be broken when they keep working. Autocomplete works too. This issue seems to be only with highlighting.
Steps to reproduce
Start zsh
make an alias like:
➜ ~ alias upmymaster="gco master && g pull origin master && gco -"

then type without exec
➜ ~ upmymaster

Notice the color of upmymaster command. It works perfectly fine but appear to be highlighted as broken for some reason.
Images or other information

So syntax highlighting has a bug ? maybe its then zsh-users/zsh-syntax-highlighting module

Thank you soo much for pointing out. So sorry for causing unnecessary problems. Closing that issue as it appears to be related to https://github.com/zsh-users/zsh-syntax-highlighting/pull/696 (there seem to bo lotsa issues in zsh-users/zsh-syntax-highlighting module with alias highlighting).
Thanks for the help :).

Hi, z-sy-h maintainer here.  Sorry to raise a long-inactive thread but the referenced issue, https://github.com/zsh-users/zsh-syntax-highlighting/pull/696, only affects aliases with literal newlines or literal semicolons in their definitions' right-hand sides.  Your example wouldn't be affected unless either g or gco is itself such an alias.  Feel free to open an issue over at z-sy-h to discuss further, if needed.  Cheers!


adaption of the install script to run tunnelsats on openvz/lxc virtualised linux systems  (like strato.de)

Problem:
Standard WireGuard installation is not working on a v-linux server based on openVZ/lxc virtualisation platform (like strato.de), because the VM system does not  allow the installation of additional Linux kernel modules, which is essential for WireGuard to work.
Solution:
However, the use of wireguard-go allows to run WireGuard on a user level without the need of kernel modules. It is a bit slower but fast enough for lightning node. Wireguard-go is written by the same team which writes WireGuard.
Here is a link https://d.sb/2019/07/wireguard-on-openvz-lxc how to install the newest version of wireguard-go.
The command to check for modules with the command modprobe cls_group in virtualized environments should be removed, cause it produced an error, therefore aborting the loading of wireguard.
Recommendation to correct two small minor bugs of the script:

check for the correct path of bash and adapt the bash path in the script automatically (/usr/bin/bash was not linked on the virtualized environment, instead it was installed to /bin/bash)
if the command ip route reports an empty response for subnets, then the wireguard configuration tunneslsatsv2.conf should omit ip daddr != {$localNetworks} in the command line for  PostUp = nft add chain ip %i nat ... setting up the NAT. If the variable $localNetworks is empty then wireguard reports a syntax error and stops loading.

also add an information to the install script for raspibolt users to change the listen directive in the lnd.conf to <IP_ADDRESS>

Many big thanks to ziggie1984 to get this little tiny piece of information out of the big system!!!!


لله ولي التوفيق

مع خالص الشكر والتقدير

Hello! Can you try translating that to English? Unfortunately, we only provide support in English. Thank you!

Translation implies that the user is happy with our product. Thank you for using App Center!


Query engine (WIP)

Overview
This pull request refactors the query engine from using a two-level map/reduce system to an n-level iterator-based system. The previous system had a number of issues that warranted a refactor:

Map/reduce limits the types of queries that can be done (e.g. nested queries)

Used a cursor-based system with very loose types which caused excessive memory allocation because it needed to type convert to interface{}.

Was very complicated after several iterations of multiple developers adding numerous features.

This refactor addresses these issues:

Uses n-level nested iterators to stream, merge, and join point data. This allows more complicated queries such as nested queries and also allows aggregate functions to be developed more easily.

All points have a specific data type and iterators can use buffering and flyweights to reduce or eliminate memory allocations altogether.

Simplifies the query system by making everything an "iterator" with well defined levels of execution.

Iterators
The core concept in the new engine is the influxql.Iterator. There are separate iterator interfaces for each type (Float, Integer, String, Boolean) and each one produces a stream of typed points (FloatPoint, IntegerPoint, StringPoint, BooleanPoint).
When a *SelectStatement comes into the system, it is passed to influxql.Select() which generates a slice of iterators -- one per field. For each field, the engine generates an IteratorOptions to hold the expression, conditions, time range, etc and passes that to the IteratorCreator (which at the top level is implemented by the tsdb.Store).
The IteratorCreator is an interface that takes an IteratorOptions and returns an Iterator. The store will pass off to the shard and the shard will pass off to the engine. Each of these levels is an IteratorCreator and handles creating child iterators and appropriately merging them together. The final level (the engine) receives the options and has the ability to decide how to implement it. Most of the time it will take an expression such as sum(value) and create an iterator for the value field and wrap it in a SumIterator, however, optimizations can be added such as returning a precomputed count() instead of iterating over all the values.
Once the iterators are returned from the iterator they are joined (via influxql.Join()) and passed to the influxql.Emitter which emits model.Row objects with appropriate grouping of name and tags.
TODO
This pull request is a work in progress, however, it is large so I wanted to get it pushed up for review as soon as possible. The following are a list of known issues that still need to be resolved. They are all relatively minor.

[ ] LIMIT/OFFSET
[ ] SLIMIT/SOFFSET
[ ] Integer type
[ ] Convert derivative() to iterator form
[ ] Convert top()/bottom() to iterator form
[ ] Binary expression evaluation
[ ] Fills
[ ] Remote execution

Unfortunately github won't show us the entire diff. :-(

@otoolep Diffs are for the weak. :)

Yeah, I can that! Time for file-level review, methinks.

Looks like the cluster (remote iterator) stuff also still needs to be wired up?

@pauldix Yes, I was going off my list based on the test suite and I missed that. I added it to the PR's TODO.

So one thing I haven't been able to quite understand yet is the map phase for the map/reduce iterators.
For many map/reduce iterator the map and reduce steps are the same; count for example,
but for others it is not i.e. mean. I see that there are ReduceIterators and ReduceSliceIterators and it seems like the functions that don't really need a map phase just use the ReduceIterator interface. The ReduceIterator step is applied as many times are necessary to arrive at a final result. The functions that do need a map phase use the ReduceSliceIterator. Does a ReduceSliceIterator guarantee that all needed values are present in the slice or is the ReduceSlice step applied multiple times too? In other words, will the output of a ReduceSliceIterator every be passed back as input to the next iteration of a ReduceSliceIterator?
If it is guaranteed that all needed values are present in the slice how and why? Meaning why not have an explicit map phase so that only intermediate results need be passed to the reduce phase. Is this an artifact of the storage layer where all needed values are guaranteed to live in the same shard etc.

@nathanielc The ReduceSliceIterator guarantees that all points in a group (name + tags + window) are present before the aggregate is computed. This is required for a few aggregate functions (e.g. median(), stddev(), & percentile()).
Some of the aggregate functions such as mean() & distinct() are computed using ReduceSliceIterator right now because it's easier. They actually need their own specific iterator types to implement an optimized map/reduce. It's not a difficult change to make but I wanted to get all the tests passing before I move on to optimizations.

@gunnaraasen I just pushed up a working copy of the code: c9a2eb3.

@benbjohnson That makes sense, so taking mean for example to optimize it would you create a new point type called MeanPoint or something that contains both the sum and count of a subset of the group, and then MeanPoint would be reduced into the final FloatPoint result of the mean? Or would you not bother with a new type and somehow make it fit into the existing types?

@benbjohnson Could you tell me when will you finish the PR?  Thanks and Happy New Year!

@benbjohnson Could you tell me when will you finish the PR?  Thanks and Happy New Year!

@jsternberg I went through the test suite again to see where we're at and what needs to be done. I compiled the full list here:
https://gist.github.com/benbjohnson/02c8504e7c5e307f81c8

Query engine is complete (except for remote execution). Please review. Remote exec will be done in a separate PR.

Circle is failing due to go vet errors:
Go vet failed. Please run 'go vet ./...' and fix any errors.

tsdb/query_executor.go:567: influxql.Fields composite literal uses unkeyed fields
tsdb/query_executor.go:570: influxql.Sources composite literal uses unkeyed fields
tsdb/query_executor.go:624: influxql.Fields composite literal uses unkeyed fields

Looked through PR. Looks good from what I can see. Couple of small typo-type nits:

iterators.gen.go.tml:129: anymore
call_iterator.go:25: auxilary
iterator.go:556: Series could have a comment (also the ID and Combine methods)
iterator.go:578: SeriesList could have a comment.

I think someone mentioned that SHOW TAG KEYS should be implemented for a cluster, in this branch, but local testing of this branch indicates that it doesn't fix #5591. This is a note to myself, more than anything :smile:

Auxiliary is misspelled in most comments. I'll make a follow up commit after this gets merged for those, but since they're comments, I think they're immaterial for initial approval so others can start building on this.
Anymore is a US spelling which is probably why you're not familiar with it (http://alt-usage-english.org/anymore.html).
I'll add documentation comments to the series stuff. I might remove those later since it was primarily used for an implementation of fill that doesn't exist anymore. Now it is used for determining the types of auxiliary iterators, but there's a much more efficient implementation that should probably be used instead.
Any cluster commands won't work until we have remote exec which isn't part of the initial PR.

:+1:

SWEEEEEET!


New mods for watcher's eye is missing

Hi there, the new mods on watcher's eye for Malevolence and Zealotry are missing (it works if you do it manually for those I have tried)
Malevolence

(20–30)% increased Skill Effect Duration while affected by Malevolence
+(18–22)% to Non-Ailment Chaos Damage over Time Multiplier while affected by Malevolence
+(18–22)% to Cold Damage over Time Multiplier while affected by Malevolence
(15–20)% increased Recovery rate of Life and Energy Shield while affected by Malevolence
Unaffected by Poison while affected by Malevolence
Unaffected by Bleeding while affected by Malevolence
Damaging Ailments you inflict deal Damage (10–15)% faster while affected by Malevolence

Zealotry

Critical Strikes Penetrate (8–10)% of Enemy Elemental Resistances while affected by Zealotry
(10–15)% increased Cast Speed while affected by Zealotry
Effects of Consecrated Ground you create while affected by Zealotry Linger for 2 seconds
Consecrated Ground you create while affected by Zealotry causes enemies to take (8–10)% increased Damage
Gain Arcane Surge for 4 seconds when you create Consecrated Ground while affected by Zealotry
180% increased Maximum total Recovery per second from Energy Shield Leech while affected by Zealotry
(100–120)% increased Critical Strike Chance against Enemies on Consecrated Ground while affected by Zealotry

It's not just these auras unfortunately. #935 is a PR for all of the missing mods.  I can probably add these to that branch

Ah, I see, I will close this issue then.


Swift 3 Support.

Is this library support Swift 3?

Check out branch swift3.

Is this library add buttons in the navigation bar?

Yes, that is its main feature.

Okay, Thanks.
By the way, where are you from?

Czech Republic 🇨🇿  Hope the module will serve you well, hit me up if you stumble upon any problem.

Thank you!


Fix for integrated speakers not working

My speakers weren't working on Lenovo Legion 7i Pro, have been looking for a solution for two days and finally found one. Have to share it with the community

All thanks to the glorious @edkirin

Nope, wasn't that.
This seems to have helped though: https://www.reddit.com/r/LenovoLegion/comments/12752y7/comment/kg2uv27/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

All thanks to the glorious @edkirin

Btw, it wasn't me who found the solution. I referenced to source in my post.

@edkirin I wonder in what cases does power saving fix works, and in what cases you just need to update the firmware.
And it still doesn't fix missing sound on Windows with every part of its sound system working fine. I guess Windows installs its own broken firmware on boot?

Thank you! 🧡

This ultimately didn't work for me, still without speakers sound, but this solution is better:
https://github.com/trgeiger/tas2781-fix-16IRX8H
It doesn't automate the sound issue fix, but at least I have consistent success with its manual activaton.
Some people report that linux kernel 6.12-rc fixes the sound issue, but it didn't work for me either.


Import-Module posh-gvm

I have followed all the steps to set up the posh-gvm. When I get to the Import-Module posh-gvm I get an offline error. I have not found a fix for this and I need to learn groovy so I can create and troubleshoot ScriptRunner scripts. Is there a fix for this or a workaround? My JAVA_HOME is set so that is not the issue. I am stuck.

@mzbullock  I had the same offline issues and it's because this repo uses deprecated sdkman apis.
A quick fix would be to go to
/WindowsPowerShell/Modules/posh-gvm/posh-gvm.psm1 line 22,23
Replace it with
$Script:PGVM_SERVICE = 'https://api.sdkman.io'
$Script:PGVM_BROADCAST_SERVICE = 'https://api.sdkman.io/2'

restart powershell and try again.

@srimajji
Hi,  I had tried ,but it did not work.


Session sensitive data store?

So here is my problem, I am using this tool as part of my selenium testing. For each browser, I would like a separate data store so that actions performed by one browser do not affect the other.
I did not see anything in the documentation to support such a use case. If it's not supported out of the box, then I'd like to wonder out loud how it can be done.
Perhaps it's possible to assign a unique cookie, have all keys in the data store be prefixed by the value of that cookie (transparently)?

Good idea!
I had never thought of that use-case before, but it shouldn't be too hard to create a session-specific data store class.  I'll add it to the to-do list.  In the meantime though, you can probably whip-up something that meets your needs pretty easily by inheriting from the DataStore abstract class and implementing somethign similar to the MemoryStore class, but with a bit of extra logic to separate the data by session.

well, browsing through the source, I spotted this:

      // Get the current DataStore (this can be changed at any time by third-party code)
      var db = util.isExpressApp(router) ? router.get('mock data store') || dataStore : dataStore;

I'm not sure this feature is documented anywhere, so I hope it's an intentional feature because it's allowed me to do this:
    app.use(function(req, res, next) {
        var cookie = req.cookies.__express_session_cookie;
        if (cookie == undefined) {
            console.log('CREATING NEW SESSION');
            var randomNumber = Math.random().toString();
            res.cookie('__express_session_cookie', randomNumber);
            console.log('creating new data store for SESSIONID:' + randomNumber);

            sessionDataStores[randomNumber] = prepareData();
            app.set('mock data store', sessionDataStores[randomNumber]);
        } else {
            console.log('USING EXISTING SESSION');
            if (!sessionDataStores[cookie]) {
                // eg: you restart the server but use the same browser which kept
                // the session cookie
                sessionDataStores[cookie] = prepareData();
            }
            app.set('mock data store', sessionDataStores[cookie]);
        }

        next();
    });

very very simple multi-session data storage. prepareData() is just returning

here is something cleaner:
function createSessionStore(req, res, next) {
    var cookie = req.cookies.__express_session_cookie;

    if (cookie == undefined) {
        var randomNumber = Math.random().toString();
        res.cookie('__express_session_cookie', randomNumber);

        sessionDataStores[randomNumber] = prepareDataStore();
        app.set('mock data store', sessionDataStores[randomNumber]);
    } else {
        if (!sessionDataStores[randomNumber]) {
            sessionDataStores[cookie] = prepareDataStore();
        }
    }
    next();
}

Then you can use it like so:
app.use(cookieParser());
app.use(createSessionStore);

and my prepareDataStore() looks like:
function prepareDataStore() {
    var myDB = new swagger.MemoryDataStore();
    myDB.save(require('./dataEndpoint'));
    return myDB;
}

and dataEndpoint.js would look like:
var swagger = require('swagger-express-middleware');

module.exports = new swagger.Resource('/endpoint', {
    someKey: 'someValue'
});

Actually my little function had a bug.
function createSessionStore(req, res, next) {
    var cookie = req.cookies.__express_session_cookie;

    if (cookie == undefined) {
        var randomNumber = Math.random().toString();
        res.cookie('__express_session_cookie', randomNumber);

        sessionDataStores[randomNumber] = prepareDataStore();
        app.set('mock data store', sessionDataStores[randomNumber]);
    } else {
        if (!sessionDataStores[cookie]) {
            sessionDataStores[cookie] = prepareDataStore();
        }
        app.set('mock data store', sessionDataStores[cookie]);
    }
    next();
}

This should be much better. I'll see if I can turn this into its own npm installable module (and of course, have some tests).
But I would need confirmation that the approach I'm taking is leveraging an intentional feature that will continue to exist.


Destructured env vars not replaced in --modern builds

Version
4.0.0-rc.7
Reproduction link
https://github.com/runarberg/vue-cli-modern-process-env-fail
Environment info
$ npx @vue/cli@next info

Environment Info:

  System:
    OS: Linux 5.0 Ubuntu 19.04 (Disco Dingo)
    CPU: (4) x64 Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
  Binaries:
    Node: 12.11.1 - ~/.nvm/versions/node/v12.11.1/bin/node
    Yarn: Not Found
    npm: 6.11.3 - ~/.nvm/versions/node/v12.11.1/bin/npm
  Browsers:
    Chrome: Not Found
    Firefox: 69.0
  npmPackages:
    @vue/babel-helper-vue-jsx-merge-props:  1.0.0
    @vue/babel-plugin-transform-vue-jsx:  1.0.0
    @vue/babel-preset-app:  4.0.0-rc.7
    @vue/babel-preset-jsx:  1.1.0
    @vue/babel-sugar-functional-vue:  1.0.0
    @vue/babel-sugar-inject-h:  1.0.0
    @vue/babel-sugar-v-model:  1.0.0
    @vue/babel-sugar-v-on:  1.1.0
    @vue/cli-overlay:  4.0.0-rc.7
    @vue/cli-plugin-babel: ^4.0.0-rc.7 => 4.0.0-rc.7
    @vue/cli-plugin-eslint: ^4.0.0-rc.7 => 4.0.0-rc.7
    @vue/cli-service: ^4.0.0-rc.7 => 4.0.0-rc.7
    @vue/cli-shared-utils:  4.0.0-rc.7
    @vue/component-compiler-utils:  3.0.0
    @vue/preload-webpack-plugin:  1.1.1
    @vue/web-component-wrapper:  1.2.0
    eslint-plugin-vue: ^5.0.0 => 5.2.3
    vue: ^2.6.10 => 2.6.10
    vue-eslint-parser:  5.0.0
    vue-hot-reload-api:  2.3.4
    vue-loader:  15.7.1
    vue-style-loader:  4.1.2
    vue-template-compiler: ^2.6.10 => 2.6.10
    vue-template-es2015-compiler:  1.9.1
  npmGlobalPackages:
    @vue/cli: Not Found

Steps to reproduce
Destructure process.env somewhere in your code:
<template>
  <HelloWorld :msg="msg" />
</template>

<script>
const { VUE_APP_MSG: msg } = process.env;

export default {
  data() {
    return {
      msg,
    };
  },
};
</script>

Add the env var to your .env:
 VUE_APP_MSG="Hello, Bug!"

Then build with the --modern flag:
npm run build -- --modern

What is expected?
The build output should have the destructured env vars replaced. That is msg should be set to "Hello, Bug!".
What is actually happening?
This only happens in the legacy build. In the modern build the variable is undefined.

I think this is a regression. I think this worked as expected in vue-cli 3.

Caused by https://github.com/vuejs/vue-cli/pull/3782. See the reasoning at https://github.com/vuejs/vue-cli/issues/3579
I'm wondering if we can revert it though. Let me have a test first.


Progress of each step is not shown till all steps are completed for local repo

Describe the bug
In local repo, Progress of each step is not shown till all steps are completed, then all steps show up as completed along with the code all at once instead of streaming like bolt.new
Link to the Bolt URL that caused the error
N/A
Steps to reproduce
Clone the repo, submit prompt, progress of each step not shown, shows steps all at once after all completed
Expected behavior
should show progress of each in real time with code streaming in files like bolt.new
Screen Recording / Screenshot
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

the progress of each step seems to work for firefox but not chome & safari, also the code is not being streamed in file even in firefox

Hi! 👋 Thanks for the comments!
Some features are missing from the locally run version of the Bolt codebase currently, we're tracking this internally and will keep you posted!

@kirjavascript can I work on this issue? Or are you guys still speculating steps to go about it?


Tests timing out since latest rustup

Eg https://github.com/bjorn3/rustc_codegen_cranelift/runs/2066221503?check_suite_focus=true had been running for 3h34m before I canceled it.
test slice::test_binary_search has been running for over 60 seconds
test slice::test_binary_search_by_overflow has been running for over 60 seconds

#[test]
fn test_binary_search() {
    let b = [(); usize::MAX];
    assert_eq!(b.binary_search(&()), Ok(usize::MAX / 2));
}

goes into an infinite loop.

Works fine when compiling the sysroot with -Zmir-opt-level=1.

Standalone test:
use core::cmp::Ordering::{self, *};

#[inline]
fn binary_search_by<'a, F, T>(this: &'a [T], mut f: F) -> Result<usize, usize>
where
    F: FnMut(&'a T) -> Ordering,
{
    let mut size = this.len();
    let mut left = 0;
    let mut right = size;
    while left < right {
        let mid = left + size / 2;

        // SAFETY: the call is made safe by the following invariants:
        // - `mid >= 0`
        // - `mid < size`: `mid` is limited by `[left; right)` bound.
        let cmp = f(unsafe { this.get_unchecked(mid) });

        // The reason why we use if/else control flow rather than match
        // is because match reorders comparison operations, which is perf sensitive.
        // This is x86 asm for u8: https://rust.godbolt.org/z/8Y8Pra.
        if cmp == Less {
            left = mid + 1;
        } else if cmp == Greater {
            right = mid;
        } else {
            return Ok(mid);
        }

        size = right - left;
    }
    Err(left)
}

#[test]
fn test_binary_search() {
    let b = [(); usize::MAX];
    assert_eq!(binary_search_by(&b, |p| p.cmp(&())), Ok(usize::MAX / 2));
}


Create filtering system

Streams should be filterable by game being played. This will most likely require a rework of the storage layer to something queryable (mongo?).

I experimented with filtering stream by language, but I wasn't able to achieve it in a way, that would work in production enviromen, since Redis is not made for queries. Database where they are possible would help a lot, with adding features such as filtering, giving us random stream from selected platform, or filtered by any other available parameter

I've also thought about just using hash sets within Redis. I'm still not totally set that filtering is something worthwhile -- the vast majority of hits to the site are not logged-in twitch users so I'm guessing we're seeing a more casual crowd (which more configuration options might not thrill).
I'm thinking on it.

Hey Benny! Thanks for the interest and support. Things have been insane with work for the last couple months so I haven't had a chance to shake the dust out of this project, but I'll take a look this evening and see how realistic this is to implement.

Wow I am absolutely impressed by how quickly you out of that filtering system. Can’t wait to see it!

Still getting the kinks out; never had a good deployment system lol. Standby...