[DOCS]: How to select your model in the program for Windows?

Description
I only found this instruction for docker.
https://github.com/Mintplex-Labs/anything-llm/blob/master/server/storage/models/README.md#text-generation-llm-selection
How to add your model to the program? Creating a downloaded folder in AppData\Roaming\anythingllm-desktop\storage\models\ doesn't work.
Before this, I wanted to connect the model via koboldcpp, but the application does not support this backend.
I need a model with q8, I don’t like ollama, but the worst thing is that it downloads 4q versions of neural networks.

If that is the case, then this issue needs to be worked #513 to get full support for you then

Otherwise, you may certainly get a sub-par performance. Some people have said that using the LocalAI connector but pointed at your Kobold endpoint works? Oddly enough

@timothycarambat I tried every available local menu item, nothing wanted to work with kobaldcpp.
Judging by the fact that that task has been hanging for 4 months, adding another local backend is of no interest to anyone. Which seems very strange. Since there is no llama.cpp server, no kobaldcpp, no text-generation-webui.
But there are several server part models that are cumbersome and not easy to launch, as well as adding the llm model you need with the launch parameters you need. impossible or inconvenient.

@intulint It's been in the backlog because we don't use it and have only seen the request 2, maybe 3 times now. If we prioritize every single provider that gets added we will never move on past that set of features since there are an unlimited amount of tools for running LLMs. Everyone has a preference and it's unreasonable to expect us to add them instantly if the demand is not high enough. Looking now, that request has 4 votes on it.
That's just the way it is. Also, we are OSS, so if you really need it today, you can add it. This strategy we have taken thus far is not impossible or inconvenient for those who use the tool in its current state, just for you and others who need XYZ integration.
In an ideal world, every LLM provider is supported so nobody has any requests, but we are a small core team and contribution is optional, not required. So at the end of the day, we are responsible for doing most integrations.
Since that is what this issue is about - I'll close this issue and it can be tracked on #513. Until then.

@timothycarambat
But the problem for which I opened this ticket is not in the launch method and it has not yet been resolved. I don't agree with the closure.
I did not find how to add my already downloaded model to the program that is installed by the installer. This is not in the documentation, that's all. There is a mention that you can add your own model in the Docker version, but it is not clear how in the application.
I have a feeling that everyone is trying to make users depend on some servers from which models will be downloaded. Neither here, nor in lmstudio, nor most likely in ollama, you can simply add your pre-downloaded model.
This, guys, already looks like some kind of conspiracy, heh.


Use external DittoPresenceViewer dependency

Updates to use the presence viewer from its own repo / package as part of the path to a single source of truth for the presence viewer.
Closes CXTOOLS-358
Part of CXTOOLS-337

This will now depend on https://github.com/getditto/DittoPresenceViewer/pull/5 and a matching new release of the new package to avoid undoing the work in https://github.com/getditto/DittoSwiftTools/pull/167.
Marking as a draft for now.


Support import file from other folder?

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
d-----         2023/4/29      8:17                alib
-a----         2023/4/29      8:30             78 test.jou

alib/func.jou
import "stdlib/io.jou"

def helloworld() -> void:
    printf("Hello World!")

test.jou
import "alib/func.jou"

def main() -> int:
    helloworld()
    return 0

I want:
Hello World!

But:
compiler error in file "test.jou", line 1: import path must start with 'stdlib/' (standard-library import) or a dot (relative import)

Think should work on parse_import

Jou's syntax is explicit, so you need import "./alib/func.jou" where . means the current directory.


[mysql] Fix the restoration fail when checkpoint happened in a mysql transaction

Aims to fix up #276 #411
Add test in BinlogSplitReaderTest#testReadBinlogCrossTransaction

The test is failed.

The CI passed, merging...

cdc 2.2.0 still has this problem...

Aims to fix up #276 #411 Add test in BinlogSplitReaderTest#testReadBinlogCrossTransaction

cdc 2.2.0 still has this problem...

+1 cdc 2.2.0 still has this problem....

+1 cdc 2.2.0 still has this problem....

upgrade the version of mysql-binlog-connector-java to v0.25.3

目前 cdc 2.4.1 依旧存在这个问题

flink1.17.1+cdc 2.4.1 还是会出现这个问题

flink 1.17 + cdc 3.1 依然存在此问题。


Make a better way to access our "static data".

Our current way of doing this is using a function and adding the data to the global environment with:
# A function for accessing variables.
get_static_data <- function(...) {
  data <- list()
  # Create a list for validating metacoder observation data
  basic_names <- c("otu_abundance", "otu_annotations")
  analyzed_names <- c("taxa_abundance", "otu_proportions", "taxa_proportions")
  all_names <- c(basic_names, analyzed_names)
  valid_list <- list(basic = basic_names, analyzed = analyzed_names, all = all_names)
  # Create various rank based lists
  ranks <- list(c("Kingdom"), c("Phylum"), c("Class"), c("Order"), c("Family"), c("Genus"), c("Species"))
  rank_index <- list("Kingdom" = 1, "Phylum" = 2, "Class" = 3, "Order" = 4, "Family" = 5, "Genus" = 6, "Species" = 7)
  mc_df_rank_list <- list("Kingdom" = 4, "Phylum" = 5, "Class" = 6, "Order" = 7, "Family" = 8, "Genus" = 9, "Species" = 10)
  format_list <- list("phyloseq_format" = 0, "raw_format" = 1, "basic_format" = 2, "analyzed_format" = 3)
  data <- list(valid_list = valid_list, ranks = ranks, rank_index = rank_index, mc_df_rank_list = mc_df_rank_list)
  list2env(data[...], envir = .GlobalEnv)
}

However a more elegant solution might be to create a "package environment" (link so that this data does not easily disrupt the end users experience.

Duplicate of #8


Displayed products don't adjust to filter selection

Steps to reproduce:

Standard installation with demo products.
Remove these overrides from the theme to use the actual module code:

themes/community-theme-default/modules/blocklayered/blocklayered.tpl
themes/community-theme-default/js/modules/blocklayered/blocklayered.js

View a category page to see layered navigation.
Move the upper price slider knob downwards, towards a lower price.

Expected result:

As one releases the knob, presented products not matching the filter get removed.

Actual result:

Presented products don't change.

Hint: pagination buttons and product count numbers get adjusted just fine.

Fixed together with #5.


Return async compute() outputs

To align with latest spec: https://github.com/webmachinelearning/webnn/pull/323

@Honry I just landed the patch of https://github.com/webmachinelearning/webnn-polyfill/pull/204, it would impact WebNN-Samples, please help fix this issue, thanks.


Feature/get psc details

https://companieshouse.atlassian.net/browse/IDVA3-2028

CI: Security warnings found!

CI: Security warnings found!


Update path for RN-app built with ignite-andross

I haven't used the ignite cli or the ignite-andross boilerplate before and I need to update a React Native app that is currently running on RN 0.55.4.
What does the update path for apps built with this boilerplate looks like? I see that there was a 3.2.0 release yesterday that adds support for RN 0.59. Can I update my existing project with the CLI or should I use something like this and do a manual update?
Thanks in advance for the help!

@timkuilman Ignite doesn't handle dependency upgrades so using https://pvinis.github.io/purge-web/ and updating manually is your best bet.


Heating / Off button inversion if thermostat is of "over thermostat" type

I have multiple instances of Versatile Thermostats, and I use a Versatile-thermostat-ui-card on all of them.
I have noticed a small diplay bug : on the only "thermostat-over-thermostat" I have, the "Heat" and "Off" button are reversed versus the other thermostats.
As you can see on screenshot below, the "Séjour" thermostat Heat/Off buttons are in reverse order.

The main difference between this thermostat and the others (3 shown but I have 4 more), is the "Séjour" is of type "Thermostat over thermostat" while the others are "over switch".
I would expect the UI renders identically, whatever the underlying Versatile Thermostat type.
I take the opportunity to thank you very warmly (pun intended) for this wonderful work you are doing !

Hello, the order for VTherm over_climate is given by the underlying climate. If you have more hvac_mode you have something like that :

But you are right I will try to always put the turn-off button at the end. Thank you for your report.

I have the same problem, with my "over thermostat" device using a ZigBee physical thermostat:

Yes I will reorder the hvac-modes coming from the underlying.


Space Leak in deployed nodes

We're seeing a space leak in deployed nodes. The nodes get OOM killed as can be seen in the memory graph here: https://monitoring.awstest.iohkdev.io/grafana/d/Oe0reiHef/cardano-application-metrics-v2?orgId=1&refresh=1m

Staging snapshot shared on slack https://files.slack.com/files-pri/T0N639Z4N-FR64QTEJH/staging-gc-bytes.png 2 Dec 2019

I have looked into this and have found the following.

I can reproduce the space leak by running just a node that connects to a proxies. Just run scripts/mainnet.sh, no need to run the byron-proxy or cardano-explorer yourself.
By limiting the stack size to 2MB, I get a stack overflow after ~1 min on my
machine. Exact commit: https://github.com/input-output-hk/cardano-node/commit/3c7387d657d6afe0fae000116cebbad9e902d0a4
When I rip out logging & monitoring, I no longer get a stack overflow even after running + 5min. @karknu also ran it for 13 min without a stack overflow. Exact commit: https://github.com/input-output-hk/cardano-node/commit/3406ffe29beae64ab5c5e97a936fa29ac7ca7f66

My conclusion: the memory leak is in the logging/monitoring code, not network or consensus.
I think the right people from the logging/monitoring should investigate this further. For example, if the leak is inside a tracer, they could start by disabling some tracers until they find the one leaking memory.

I guess that the memory leak is related to some decoding into CBOR.
Please, investigate further.

Not that the problem DevOps diagnosed is that we're leaking stack space which should not be confused with memory space (the former is a subset of the latter, the latter also includes heap space). This measurement is too short to say anything about the slow but steady leaking of stack space.


VICON_POSITION_ESTIMATE vs ATT_POS_MOCAP

Right now I think we have two messages that serve the same purpose. If one wants to add support for one of this messages (I'm aware that vicon_position_estimate can already be handled in the mavlink receiver but att_pos_mocap can't) in the estimators, for which one should one move to? Shouldn't one of those be removed?
According to the current implementation, att_pos_mocap should fit better the purpose, as attitude is sent in quaternion representation. (also the name is more adequate, as VICON is not the only existing mocap system).

Please use att_pos_mocap as suggested and remove vicon message support in the process.


NEW MAINTAINER - active fork, new repo name, & plans

What
Hello all (to anyone still actively using sequelize-auto-migrations). To all who have contributed PRs over the years - THANK YOU! Your PR is not in vain. Hence, we've created a fork that we're actively maintaining.
It's available here:
https://github.com/matmar10/sequelize-lazy-migrations
The new npm name is sequelize-lazy-migrations and you can install it via that name:
npm install --save-dev sequelize-lazy-migrations

Plans

Add a baseline unit test coverage
Merge as many PR contributions as possible (anything without breaking changes will be considered)
Fix high priority bugs (e.g. phantom migrations being generated due to validations)

Why?
My company is still actively utilizing and relying on this library. Requests to add contributors have gone unanswered. Hence this was the best course of action.

@matmar10 did you check this fork: https://github.com/Scimonster/sequelize-auto-migrations?
FYI this other fork has some other features that your fork does not have.

Thanks @bigLucas! Are you using the features in that fork? We could merge them in at sequelize-lazy-migrations if so.

I'm trying to use the features for the first time because I'm kind of new to Sequelize (I was a TypeORM boy and now I'm trying to use sequelize in a new project at work).
I found that fork through this StackOverflow answer which has some PR links about why the owner of the fork has created it.
So I guess it could be useful for your purpose to know this other fork, and in my case perhaps I might go back to TypeORM.


meta ^1.8.0

flutter 3.0.5-stable depende do meta: 1.7.0
Because every version of flutter from sdk depends on meta 1.7.0 and hasura_connect >=4.0.0+3 depends on meta ^1.8.0, flutter from sdk is incompatible with hasura_connect >=4.0.0+3.
So, because pay depends on both hasura_connect 4.0.0+3 and flutter from sdk, version solving failed.

@tauamendonca as you can see in your picture, in MASTER flutter depends on meta 1.8.0, however if you check the link tag 3.0.5, it depends on 1.7.0
https://github.com/flutter/flutter/blob/3.0.5/packages/flutter/pubspec.yaml
@Bwolfs2 please verify

Fixed, i reduce to 1.7.0
Thanks o/


This pull request enables extra infos added by monolog processors to be logged to db to

Merged it, sorry for the very long delay of more than one year...


require polkit package for rpm install

we should do another PR on the release-0.21.x branch

https://github.com/openziti/ziti-tunnel-sdk-c/pull/694


Make config read only

Add new ReadOnlyContainer types and use them for the config. This solves #370.

I now added some more unittests for the container types with and without the experiment.
I didn't know that UserDict and UserList exist. What is the benefit of using them?

Also, currently test_double_nested_config fails due to the modifications because it is doing _config.pop('seed'). Should we include some mechanism to create a writable copy of the read-only containers? Maybe implement __copy__ and __deepcopy__ to return a mutable dict?

I didn't know that UserDict and UserList exist. What is the benefit of using them?

These classes where designed for inheritance, so all the methods that access some internal state  go through the dunder methods (__setitem__, __delitem__). This would mean that you only have to overwrite these methods to get a functioning read only container. Other public methods like clear should than work as expected right away. You can read more about it here https://treyhunner.com/2019/04/why-you-shouldnt-inherit-from-list-and-dict-in-python/

That makes sense. But it is apparently not correct that all methods that access the internal state go through the dunder methods, at least not for UserList. Also for UserDict, setdefault does not use __setitem__ when there is already a value set for that key and clear only calls __delitem__ when the dict is not empty (which makes sense), so those two methods must again be overwritten explicitly. I made the containers inherit from the user types, but it did not make the code cleaner, and certainly not faster.
When used in interactive mode (ipyhton, jupyter notebook), it may make sense to overwrite all methods that can potentially change the contents to provide the user with a corresponding docstring.
Another possibility would be to completely drop the inheritance from any dict or list type and write a wrapper around it that delegates the method calls to the wrapped object.

The approach to drop the inheritance to all list/dict types would look something like this:
class ReadOnlyList:
    def __init__(self, *args, **kwargs):
        self.data = list(*args, **kwargs)

    def __repr__(self):
        return repr(self.data)

    def __lt__(self, other):
        return self.data < self.__cast(other)

    def __le__(self, other):
        return self.data <= self.__cast(other)

    def __eq__(self, other):
        return self.data == self.__cast(other)

    def __gt__(self, other):
        return self.data > self.__cast(other)

    def __ge__(self, other):
        return self.data >= self.__cast(other)

    def __cast(self, other):
        return other.data if isinstance(other, self.__class__) else other

    def __contains__(self, item):
        return item in self.data

    def __len__(self):
        return len(self.data)

    def __getitem__(self, i):
        return self.data[i]

    def __add__(self, other):
        if isinstance(other, self.__class__):
            return self.__class__(self.data + other.data)
        elif isinstance(other, type(self.data)):
            return self.__class__(self.data + other)
        return self.__class__(self.data + list(other))

    def __radd__(self, other):
        if isinstance(other, self.__class__):
            return self.__class__(other.data + self.data)
        elif isinstance(other, type(self.data)):
            return self.__class__(other + self.data)
        return self.__class__(list(other) + self.data)

    def __iadd__(self, other):
        if isinstance(other, self.__class__):
            self.data += other.data
        elif isinstance(other, type(self.data)):
            self.data += other
        else:
            self.data += list(other)
        return self

    def __mul__(self, n):
        return self.__class__(self.data * n)

    __rmul__ = __mul__

    def __imul__(self, n):
        self.data *= n
        return self

    def copy(self):
        return self.__class__(self)

    def count(self, item):
        return self.data.count(item)

    def index(self, item, *args):
        return self.data.index(item, *args)

which is basically a copy of UserList without the modifying methods.

Maybe two alternatives should be considered:

Use tuple and types.MappingProxyType instead of list and dict
Use deepcopy for all arguments

Advantages of the second:

no captured function can change the global config. This is guaranteed.
the datatype is preserved
isinstance checks work

When the ReadOnlyList does not inherit from list, why not using tuple? Tuple is an immutable (ReadOnly) list.

Use tuple and types.MappingProxyType instead of list and dict

Then, isinstance checks don't work anymore. They neither work with UserList and UserDict though.

When the ReadOnlyList does not inherit from list, why not using tuple? Tuple is an immutable (ReadOnly) list.

Correct. Does not make sense to make ReadOnlyList not inherit from list.

Use deepcopy for all arguments

There are two possibilities: copy when (a) the configuration is initialized or (b) when the function is called.

a) This makes sure that the global configuration is not changed. But still, the (captured function) local configuration could accidentally be changed and the changes would persist between successive calls of the same captured function, again, not recorded.
b) This solves the problems of (a) and lets the user do anything he wants inside the captured function (which is nice), but performing a deepcopy operation every time a captured function is called could drastically increase the overhead, depending on the size of the configuration.

If we assume that the only container types in the configuration are tuple, list and dict (which is obviously not guaranteed), I would prefer to subclass from those types and raise an exception if there is an attempt to change something. Then, isinstance checks still work and the user gets feedback where exactly he tries to change the configuration.
We could also save a copy of the configuration and check after each function call if any value was changed. This probably introduces less overhead than doing a deepcopy on every call, and it works without any new types.
I'm now completely unsure about what is the best solution for this. @JarnoRFB what is your opinion on this?

This seems more complex than I initially thought. To throw something else in the arena we might consider using freeze for converting everything to pyristent data structures https://github.com/tobgu/pyrsistent. However, I have not personally worked with it, but maybe it is good for our usecase

This seems more complex than I initially thought.

Definitely.
I listed all options that currently are under debate below. What we might want to achieve:

keep isinstance checks intact
give user feedback where the write to the config values happens
provide a way to get a mutable copy of the config
low overhead

No custom types

types are preserved -> isinstance and similar things still work as expected
works for any type, even user-defined classes, not just the built-in collection types

deepcopy on every call

no feedback, user can do anything in a captured function
high overhead

deepcopy on initialization

does not prevent changes of local config values that persist between function calls
no feedback, invalid writes to the config are not detected

deepcopy on initialization and equals checks after each function call

low-resolution feedback, invalid writes are detected
a "mutable" copy can be obtained in the captured function doing a deepcopy again
lower overhead than deepcopy on every call

Special Types

Everything listed here introduces new types for dict and list, and only prevents writes to those collections. Any write to user-defined types is not prevented.
have a low overhead during runtime (for the call of a captured function), only introduce overhead during initialization

pyrsistent

isinstance checks don't work
new dependency, maybe too complicated for our use case
no feedback
a modified copy can be created from the config

subclass of UserDict/UserList

isinstance does not work as expected
feedback is possible on a very fine-grained level

subclass of list/dict

isinstance intact
feedback on very fine-grained level possible
probably faster than UserDict/UserList

tuple/types.MappingProxyType

breaks isinstance
uses the same type for list and tuple config values, which may be confusing or break existing code

I think we shouldn't break isinstance checks, so from the list of special types only the subclasses of list/dict are possible. But those still only prevent writes to lists and dicts and don't work with user defined types. At the moment, I think we should keep a copy of the original config and check if the config passed to the function was modified. This works with all types, does not require a deepcopy on every call of a captured function and keeps the types intact while still detecting invalid writes and giving the user feedback.

What about something along these lines? (look into captured_function.py)

Thanks for the detailed and structured overview! After thinking a bit about it I would say that it is more or less impossible to guarantee immutability to data that is intrinsically mutable. The only solution that disables changes that persist between function calls is really deepcopy on every call. But as you said that might lead to large overheads. I believe deepcopy on initialization and equals checks after each function call that you implemented in your last PR will only work for custom types that implement __eq__. For everything else, comparisons against deep copied versions will fail. Therefore I would go with the following.

subclass of list/dict so we do at least prevent obvious mistakes and keep isinstance checks intact
provide an optional setting for deepcopy on every call in a separate PR. I think this would be useful for everyone that does not put custom types in their configs
As an idea to achieve something close to equality checks we might consider comparing hash values of configs as proposed in  #467 .

I believe deepcopy on initialization and equals checks after each function call that you implemented in your last PR will only work for custom types that implement eq. For everything else, comparisons against deep copied versions will fail.

You are right. But also computing a hash of the configuration only works for types that implement __hash__, and btw, dict is not hashable. It is a completely different discussion of how to compute a hash value for the configuration, and I assume that it involves some serious thinking about it.
I'll then implement your suggestions and squash the commits of this pull request in the following week.

Thanks for the change! Time to get this merged. I think the CI error could just be fixed by explicitly setting the seed in config.
@ex.config
def config():
    a = 1
    seed = 42

...

@ex.main
def main(_config):
    assert _config == {
        'a': 1,
        'seed': 42,
        'sub_sub_ing': {'d': 3},
        'sub_ing': {'c': 2},
        'ing': {'b': 1}
    }, _config

    ing_main()
    sub_ing_main()
    sub_sub_ing_main()

Then just fix the flake8 errors and I think this is good to go.

Coverage increased (+0.2%) to 84.617% when pulling 585153f922d98fe9f914644012bdc846f21ca170 on thequilo:read_only_config into fe3a95a3215c16e59685662a93353af312428f51 on IDSIA:master.

Python 2.7 is driving me crazy. And Codacity is complaining about unused variables in config scopes, intentional uses of assert in testcases and intentionally using type==... instead of isinstance.

@thequilo thanks! Looks good now. I hope to get rid of  py 27 asap, but I can't make a release myself. If it takes to long, we could also just switch without a release...  Merging then!

Wow, thanks @JarnoRFB  and @thequilo. This is very well thought through feature. Thanks a lot for this!
And I agree about codacy: I thought it was a useful tool to improve code quality at the time. But it has mostly produced false alarms, so I am removing it again.

Now overwriting entries in the config does not work, but when someone did this in the right way, this also does not work:
import copy
def foo(_config):
     _config = copy.deepcopy(_config)
     _config['something'] = ...

How about changing the datatype back to the original datatype when using copy?
import copy

class ReadOnlyDict(dict):
    ...
    def __copy__(self):
        return {**self}

    def __deepcopy__(self, memo):
        d = dict(self)
        return copy.deepcopy(d, memo=memo)

>>> type(copy.copy(ReadOnlyDict()))
dict
>>> type(copy.deepcopy(ReadOnlyDict()))
dict

@boeddeker sounds very reasonable to me. PR welcome!

Hmm, this actually was in the list of features that the read-only contains should have ("provide a way to get a mutable copy of the config"), but it somehow didn't get implemented

Another thing, is it intended to lose the original datatype in the config?
make_read_only makes an isinstance check but does not preserve the original datatype.
>>> class MyDict(dict):
...     pass
>>> isinstance(MyDict(), dict)
True

Solutions:

Make a type check: type(o, dict)
Use a dynamic type: type('ReadOnlyDict', (ReadOnlyDict, *inspect.getmro(MyDict)), {})()

I prefer the first.


Reenables now-working tests

Microsoft pushed update 16.4.3 today and google mock no longer crashes.
Note: This will continue to crash on AppVeyor because they are still using 16.4.2. Since we have two Windows developers local to the project that can manually review windows changes, this can be safely ignored.

Tagging issue #327


Слава Україні!

Great crate! (pun intended)
This PR fixes some typos...

Героям Слава! =)
@Sajjon Ah thanks! You have veeeeeery good eyes!

@greyblake actually I cheated! Using this crate:
https://github.com/crate-ci/typos which is really really fast and a great tool which flags very few false positives.
I'm using it with precommit in my crate identified_vec, like so: https://github.com/Sajjon/identified_vec/blob/main/.pre-commit-config.yaml#L5
Pretty neat! And can be added to github worklflows on PRs.
It can be customized with a _typos.toml file.

Cool, I did not know about typos. I will investigate it, it looks like something I should add into every project I work on :)


nan values while training

Hello again,
I've a question related to the training process. I have been able to train the network successfully until NAN values of PPL appear. In my case, they start to appear in epoch 47, and after that epoch every value of PPL and validation PPL is NAN.
Gonna paste some lines of the output while training (I've print some additional values):
`Epoch = 46
Training with learning rate = 0.02
Total neg log lik = 1292.63056993
PPL = exp(0.00103047717629)
PPL: 1.0010; Speed: 12549.65 sps
Total neg log lik = 2790.23252964
PPL = exp(0.00111217814479)
PPL: 1.0011; Speed: 15146.28 sps
Total neg log lik = 4349.67130947
PPL = exp(0.00115584377909)
PPL: 1.0012; Speed: 16240.85 sps
Total neg log lik = 6142.70772409
PPL = exp(0.00122423224731)
PPL: 1.0012; Speed: 16856.70 sps
Total neg log lik = 7894.18972039
PPL = exp(0.00125863994266)
PPL: 1.0013; Speed: 17253.00 sps
Total neg log lik = 9953.48264909
PPL = exp(0.00132247590469)
PPL: 1.0013; Speed: 17521.59 sps
Total neg log lik = 12641.4703844
PPL = exp(0.00143967182766)
PPL: 1.0014; Speed: 17718.59 sps
Total neg log lik = 14958.8259408
PPL = exp(0.00149063555692)
PPL: 1.0015; Speed: 17859.47 sps
Total number of training labels: 10436608
Net saved
Total number of validation labels: 14513408
Validation perplexity is 6.6061
Epoch = 47
Training with learning rate = 0.02
Total neg log lik = 1606.82540393
PPL = exp(0.00128095137431)
PPL: 1.0013; Speed: 12486.81 sps
Total neg log lik = 3401.88553357
PPL = exp(0.00135598115975)
PPL: 1.0014; Speed: 15087.08 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 16219.43 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 16852.34 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17255.23 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17535.15 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17743.79 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17898.87 sps
Total number of training labels: 10436608
Net saved
Total number of validation labels: 14513408
Validation perplexity is nan
Epoch = 48
Training with learning rate = 0.02
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 12501.37 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 15115.35 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 16232.72 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 16846.52 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17248.92 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17529.18 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17734.80 sps
Total neg log lik = nan
PPL = exp(nan)
PPL: nan; Speed: 17893.42 sps
Total number of training labels: 10436608
Net saved
Total number of validation labels: 14513408
Validation perplexity is nan`
I don't understand what is happening.
May I am doing something wrong? How can I fix that?
Thank you again!

Another question related to this, in that case what can we consider good values of perplexity?
Because I've noticed that my validation ppl doesn't decrease after 2 epoch, being the best value 1.28. However, as you can see in the previous output, training ppl decreases and reaches a value of 1.00. Am I overfitting? My dataset is quite large, about 29M of sentences, and I am using a batch size of 256 as you suggested, so it seems to me that training only 2 epochs is not enough. Any advice?
Thanks again, I really appreciate it

Perfect perplexity is 1.0, so you are definitely overfitting :)
1.28 on validation set is quite good. Normally the script should stop once the validation PPL stops decreasing. Did you modify the source code (elif best_ppl not in validation_ppl_history[-PATIENCE_EPOCHS:]: part maybe)?
My experience with this model has shown that 2-5 epochs is usually enough. 2 epochs means that the model has seen all 29M sentences twice.

Hello @ottokart
thanks for the quick response! I appreciate it. I finally get 1.13 of validation perplexity just training two epochs and the results I am getting are quite good.
Answering your question, I've changed a bit the script main.py to have more training epochs and more patience epochs, and I am saving the model each epoch to use the one which gives me better validation perplexity. I prefer selecting the model using that approach (just a matter of taste). The thing is that I thought I'd have need more than two epochs to get the optimal model.
Thanks again and I close this issue!
Ana


WatchStream Exception When Handling AsyncSnapshot Instead of Type

watchStream might be broken because it tries to return a AsyncSnapshot instead of the
I am trying to watch the Firebase Auth.
final User user = watchStream((AuthService auth) => auth.user, null).data;
The Stream in AuthService is
Stream<User> get user => _auth.authStateChanges();
The following error is thrown:
type 'AsyncSnapshot<User>' is not a subtype of type 'User'
Error location:

Dart Unhandled Exception: type 'AsyncSnapshot' is not a subtype of type 'User' of 'value', stack trace: #0      _WatchEntry.lastValue= (package:get_it_mixin/src/mix_in.dart)
_MixinState.watchStream. (package:get_it_mixin/src/mix_in.dart:793:15)

No, that's the corrrect behaviour as an async snapshot contains more information than just the data. In case of an error it contains the error. And it contains the current connection state.

Will add test project to demo that there is an internal error that throws the exception.

This error looks to be Firebase specific.
For some reason, Firebase returns an AsyncSnapshot from the Stream instead of a User.
Stream<User> get user => _auth.authStateChanges();

Firebase Code
import 'dart:async';

import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:get_it/get_it.dart';
import 'package:get_it_mixin/get_it_mixin.dart';

class GetItStreamTypeIssueModel extends ChangeNotifier {
  final FirebaseAuth _auth = FirebaseAuth.instance;
  Stream<User> get user => _auth.authStateChanges();
}

final GetIt g = GetIt.instance;

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();
  registerViewModel();
  runApp(MyApp());
}

void registerViewModel() {
  g.registerSingleton<GetItStreamTypeIssueModel>(GetItStreamTypeIssueModel());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'GetIt Stream Type Issues',
      home: Scaffold(body: SafeArea(child: StreamTypeIssue())),
    );
  }
}

class StreamTypeIssue extends StatefulWidget with GetItStatefulWidgetMixin {
  @override
  _StreamTypeIssueState createState() => _StreamTypeIssueState();
}

class _StreamTypeIssueState extends State<StreamTypeIssue>
    with GetItStateMixin {
  @override
  void initState() {
    super.initState();
  }

  @override
  void dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    watchStream((GetItStreamTypeIssueModel model) => model.user, null);
    return Column(
      children: [
        Text('Type Name: '),
      ],
    );
  }
}

Ah, now I get it. you mean the x is already an AsyncSnapshot and I try to wrap it in another snapshot?
Could you show me the part were you get your Stream from firebase?

Thanks for the quick reply.

hmm, strange. could you do me a favor and try if the same thing happens if you use get_it_hooks instead of the mixin. It is implemented a bit different and would help me getting a better idea. I would have to setup a firebase auth which I can do tomorrow. I wonder why the stream tests worked

I can try tomorrow.
Thanks for offering to setup firebase_auth. I was unable to replicate their implementation because it looks like the type safety doesn't work because they are passing back the result from a platform channel. Could be totally off base there.

I tried it and it works without problems.
class AuthManager {
  Stream<User> _userStream;
  Stream<User> get userUpdates {
    if (_userStream == null) {
      _userStream = FirebaseAuth.instance.authStateChanges();
    }

    /// I don't want the getter to produce a new stream on every call;
    return _userStream;
  }

  Widget build(BuildContext context) {
    final user = watchStream((AuthManager a) => a.userUpdates, null);
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'User s',
            ),
            Text(
              user?.data?.email ?? 'No user',
              style: Theme.of(context).textTheme.headline4,
            ),
            FloatingActionButton(
              child: Text('-'),
              onPressed: () async => await FirebaseAuth.instance.signOut(),
            )
          ],
        ),

Thank you for testing. Let me test with your code.
The problem is most likely my implementation which is good news.

There was the multiple listener issue that you fixed in v1.5.1 but it is still popping up with watchStream I believe. It is fixed with watch but not watchStream.
Please try running the following code in Flutter web and you will see a max call stack size exceeded error.
Firebase Auth Code
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:get_it/get_it.dart';
import 'package:get_it_mixin/get_it_mixin.dart';

final GetIt g = GetIt.instance;

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();
  initManagers();
  runApp(MyApp());
}

void initManagers() {
  g.registerSingleton<AuthManager>(AuthManager());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Firebase Test',
      home: Scaffold(body: SafeArea(child: FirebaseTest())),
    );
  }
}

class FirebaseTest extends StatefulWidget with GetItStatefulWidgetMixin {
  @override
  _FirebaseTestState createState() => _FirebaseTestState();
}

class _FirebaseTestState extends State<FirebaseTest> with GetItStateMixin {
  @override
  void initState() {
    super.initState();
  }

  @override
  void dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final user = watchStream((AuthManager a) => a.userUpdates, null);
    return Column(
      children: [],
    );
  }
}

class AuthManager {
  Stream<User> _userStream;

  Stream<User> get userUpdates {
    if (_userStream == null) {
      _userStream = FirebaseAuth.instance.authStateChanges();
    }

    return _userStream;
  }
}

I just ran my example app which is almost the same on web without any problems.

Thanks, let me test again tomorrow with a new project and hopefully close this issue.

Did it work?

Hi Thomas, sorry for the delay in responding.
I switched to a new computer but am still running into an issue.
Whenever I run the code with the GetItMixin more than once in the web browser, it throws a maximum call stack exceeded error.

════════ Exception caught by widgets library ═══════════════════════════════════════════════════════
Invalid argument: Maximum call stack size exceeded
The relevant error-causing widget was:
  SafeArea file:///F:/Development/FlutterCoffeeCalendar/lib/main_firebase_auth_test.dart:26:28
════════════════════════════════════════════════════════════════════════════════════════════════════

I switched computers and redownloaded my entire dev environment.
Can you try running the code below using the latest beta branch?
Firebase Auth Code
import 'package:firebase_auth/firebase_auth.dart';
import 'package:firebase_core/firebase_core.dart';
import 'package:flutter/cupertino.dart';
import 'package:flutter/material.dart';
import 'package:get_it/get_it.dart';
import 'package:get_it_mixin/get_it_mixin.dart';

final GetIt g = GetIt.instance;

void main() async {
  WidgetsFlutterBinding.ensureInitialized();
  await Firebase.initializeApp();
  initManagers();
  runApp(MyApp());
}

void initManagers() {
  g.registerSingleton<AuthManager>(AuthManager());
}

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Firebase Test',
      home: Scaffold(body: SafeArea(child: FirebaseTest())),
    );
  }
}

class FirebaseTest extends StatefulWidget with GetItStatefulWidgetMixin {
  @override
  _FirebaseTestState createState() => _FirebaseTestState();
}

class _FirebaseTestState extends State<FirebaseTest> with GetItStateMixin {
  @override
  void initState() {
    super.initState();
    FirebaseAuth.instance.signInAnonymously();
  }

  @override
  void dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    final user = watchStream((AuthManager a) => a.userUpdates, null);
    return Column(
      children: [],
    );
  }
}

class AuthManager {
  Stream<User> _userStream;

  Stream<User> get userUpdates {
    if (_userStream == null) {
      _userStream = FirebaseAuth.instance.authStateChanges();
    }

    return _userStream;
  }
}

If everything works on your end, then let's chalk this up to something wrong with my system and close the issue.

What do you mean by running more than onces?

If I press play to launch the code, anytime I rerun either via hot reload or close and press play again, it will throw the error.
The only way to get rid of the error is to close Android Studio.
Then, it only works once before getting the error again.

A, thanks. Have you tried if it happens with vs code too?Also have you tried it with my Firebase test project too? Am 28.10.2020 14:53 schrieb Ray Li<EMAIL_ADDRESS>If I press play to launch the code, anytime I rerun either via hot reload or close and press play again, it will throw the error.
The only way to get rid of the error is to close Android Studio.
Then, it only works once before getting the error again.
—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.

I have not tried with VS Code.
Just tested with your code and I run into the same issue.

I am open to scheduling a meeting if you think that would help.

I'm just trying to reproduce it

Just tried it with latest master. I add you to my sample project

I will try with beta now. Btw are you in the Flutter Studygroup Slack?

I can reproduce on latest beta

I tried again with master and it works.

Great, thank you for your investigation!

I am not in the Flutter Studygroup


Move Control::accept_event to Node

Viewport::gui.key_input_accepted seems could be replaced by set_input_as_handled and is_input_handled, since all detections gui.key_event_accepted end up being set_input_as_handled

what...? I'm not closing it...

have a look at my other comments, which seem to be easily resolvable.

Alrady fixed. Just careless mistakes.

While the Control::accept_event is become the same as Viweport::set_input_as_handled, I want to do some trimmer in this PR. Move Control::accept_event to Node::accept_event. Makes all nodes available to receive input events more briefly like accept_event previous control nodes. This will not lead to compatibility issues.

Moving a method up in the hierarchy can be a bit complicated and might need compatibility code, we'll see when CI has finished

You will need to register compatibility methods, see here for instructions

Remove Useless Viewport::gui.key_input_accepted

OK. And why the PR's title is changed? move this method is not the main thing this PR do.

My recommendation would be to keep the two changes in two different pull requests.
The removal of Viewport::gui.key_input_accepted is enough of a change for a single pull request.
Since the behavior change could cause irritations with users, debugging or reverting it that way is much cleaner.
Also: the smaller a PR, the easier it is to review.
The change of moving accept_event from Control to Node would need to be discussed and evaluated separately - a proposal for that change could potentially be a good starting point, in order to evaluate if there is sufficient support in the community for that change.

The move is the method is by far the most impactful change of this

Thanks! And congrats for your first merged Godot contribution :tada:

Yay!


Update dependencies

Changed

Updated dependencies.

Codecov Report

Merging #105 (3f52ac8) into master (33f05d8) will decrease coverage by 2.68%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #105      +/-   ##
==========================================
- Coverage   95.41%   92.72%   -2.69%
==========================================
  Files          89        7      -82
  Lines        1395      110    -1285
  Branches      162       13     -149
==========================================
- Hits         1331      102    -1229
+ Misses         61        8      -53
+ Partials        3        0       -3

Flag
Coverage Δ

cuaktask
92.72% <ø> (ø)

iocuak
?

porygon
?

porygon-typeorm
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

.../iocuak/src/classMetadata/decorators/injectFrom.ts

.../domain/ContainerSingletonServiceImplementation.ts

...es/domain/ContainerRequestServiceImplementation.ts

...ask/modules/ContainerModuleTaskDependencyEngine.ts

...rud/modules/iocuak/TypeOrmCreateContainerModule.ts

packages/iocuak/src/task/utils/isTaskKind.ts

.../services/api/ContainerServiceApiImplementation.ts

...data/utils/api/convertToContainerModuleMetadata.ts

...ing/models/api/bindingScopeApiToBindingScopeMap.ts

packages/iocuak/src/common/utils/isFunction.ts

... and 72 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f63d5f2...3f52ac8. Read the comment docs.


Workflows are referencing vulnerable actions

Hello, there!
As part of the university research we are currently doing regarding the security of Github Actions, we noticed that one or many of the workflows that are part of this repository are referencing vulnerable versions of the third-party actions. As part of a disclosure process, we decided to open issues to notify GitHub Community.
Please note that there are could be some false positives in our methodology, thus not all of the open issues could be valid. If that is the case, please let us know, so that we can improve on our approach. You can contact me directly using an email: ikoishy [at] ncsu.edu
Thanks in advance

The workflow Release.yml is referencing action gittools/actions/gitversion/setup using references v0.9.7. However this reference is missing the commit 90150b4 which may contain fix to the vulnerability.
The workflow Release.yml is referencing action gittools/actions/gitversion/execute using references v0.9.7. However this reference is missing the commit 90150b4 which may contain fix to the vulnerability.

The vulnerability fix that is missing by actions' versions could be related to:
(1) CVE fix
(2) upgrade of vulnerable dependency
(3) fix to secret leak and others.
Please consider updating the reference to the action.
If you end up updating the reference, please let us know. We need the stats for the paper :-)

Fixed, thanks! I enabled dependabot to keep on top of this in the future.


FPS drop

I dont want to be a whiner but in my Mekanism room i drop to like 13-16 fps while i normaly have 50-60 fps. Purely when i look @ Mekanism i already see a FPS drop. The server has no TPS drop so its purely Graphical related. I dont know if it can be optimized or something.
If i have to guess its due to the Gasses pressurized pipes that are the cause. since that part lags the most.
I wish the FPS drop would be less severe cause i love the mod its just a bit of a nausance to walk arround with such low FPS.
Mekanism <IP_ADDRESS>

Makanism causes huge FPS lag with the 5x ore process . It goes down to 10FPS around the chemical crystallizer . Its only when the MEK machines that are special for the 5x is running . When I stop them all with the high Redstone it stops the lag . So this makes me think it is makanism , and its machines . I am using enderio for all the power , gas , water , and item .

I was just discussing this problem with a few on Reddit and it seems most put their Mekanism machinery in another world or far-off chunk with a chunk loader to keep it active because of the rendering taking a lot of frames. The solution is to basically let the CPU do the work but not to draw it.  I normally get the debug overlay's calculation of 180-200fps, which drops to about 40fps when I enter my base with a Tier 4 (5x) processing area. Running i5-4670, GTX 760, 8GB DDR3-1600 (3GB dedicated to Java).

TBH man all the extra work required , and fps drop associated with the 5x its not worth it, and beside 4x is plenty for me . :)

@crunkazcanbe Few things...

Turn off the fancy pipes in the config, or...
Use EnderIO conduits (they are all compatible, just tested these before I started rebuilding my base).
Load your Mekanism machinery elsewhere so you don't have the rendering problem and framerate drop. Mystcraft, Compact Machines, there are many solutions out there apparently.
5x ores is worth it when you start running low on gold and you need more electrum to finish off the walls in your base.

By the way, don't blame "bad rendering code" on Mekanism alone, regardless if you have super video cards or not. Minecraft is still just a Java game and from the word go has been poorly optimized.
Also, your rig is overkill, especially if you think that the human eye can't distinguish more than 35 FPS. Humans on average can distinguish 150 FPS, which is why higher frequency displays exist.

https://www.dropbox.com/sh/kaqgbrr3tg6o7uz/AABjZwuQLhvWK4nfyJSV9yXCa?dl=0
ive noticed that the energy conduits cause a major fps drop, and have also been replacing them with enderio conduits, though i must have missed a few because it still sucks in my base, 6 fps or less

@unpairedbracket @aidancbrady Energy cubes causes the most laggy experience. If you have multiplies of them the FPS drops pretty much.

@cleverca22 Yeah, the fancy cable made a small difference in my case as well, but I am starting to realize that it is something more than that. Unless this mod can be optimized further, building a factory in a remote chunk (same/different world) and using Tesseracts to transfer items in and out is the only solution (since the framerate problem goes away when I leave my base). I haven't finished rebuilding with EnderIO conduits, but so far so good. I will see what happens when I finish building my sulfuric acid room.
@halvors I have heard this before on other discussion boards, but before I started moving to EnderIO Capacitor Banks, I had 2 Ultimate and 16 Elite Energy Cubes all collecting energy from multiple Mekanism generators and sending them forward to machines. As I mentioned above to cleverca22, I didn't see the drop until sulfuric acid processing.

I have never used the mekanism battery's or cables or pipes . I use enderio for pipes wires and battery's. . The only thing i have from mekanism is the machines needed for ore processing .  It has to be the rendering on those new machines that cause it . Also btw to those it my concern i know the human eye can not see much over 100fps , and my pc is just a hobby . Every month i buy the new and latest hardware . I build gamming pcs for a living and fps does count when your playing on one or more 4k screens. Any game that plays under 100fps is unexpectable to me.
Also no im not building more ore processing in another location . I like watching it and like it in my base.

I've tested just the 5x machines alone in a superflat world. This setup:

Causes my fps to drop from 600 to around 270, but turning it on only causes a further drop to 260. Is it the ore processing machines causing the issue, or the sulfuric acid production? Does the fps recover if you look away from the machines? (If not, it's more than a simple TESR rendering code inefficiency)

@unpairedbracket I see it with sulfuric acid production. When they are running, framerate is significantly lower, regardless if I am looking at them or not. Once I move away from them and they are unloaded visually (they are still chunk-loaded), framerate resumes normally. I have tested this multiple times by turning off different parts of the production line. (I had each processing tier in a separate room; they weren't all clustered together. Framerate would be bad regardless if I was in the same room or not.)

What kind of numbers are you getting for chunk updates (next to framerate on the debug overlay) when they are and aren't running?

I have completely disassembled my base a few days ago to move said machinery into a remotely chuck-loaded area, so I wouldn't be able to get the numbers in production. I will try testing it later tonight in a creative world to see if I can duplicate my original setup.

I rebuilt my entire setup in creative, stage by stage, all the way to T4 crystal production. Imgur album here. Please see the attached descriptions for each image (F3 debug visible on all screenshots).

EnderIO conduits for item/gas/liquid distribution (as a test versus Mekanism pipes).
Mekanism cables for power distribution.
Energy is collected via 36 wind turbines (high up in the sky, not visible here).
Extra Utilities' Reservoirs (33) are providing water. They don't even touch the framerate interestingly.

Results:

Still suffering frame rate issues (most noticeably when the Electrolytic Separators, 11 of them, were added for HCl production), decreasing more as I added onto the production line.
When I look away from the entire setup, my framerate jumps back up and stabilizes around 160 fps with average chunk updates. So basically, it's a rendering issue.

In my opinion, I really think T4 production needs a revamp. After rebuilding this setup, I can't say I am going to bother going past HCl production for a T3 (4x) processing.

Cuts into a visible factory's framerate. (I like to look at my hard work while it is in production, not have it mindlessly churning away out of sight.)
Machines are resource-intensive (energy, materials, space required for setup, etc) for only a 25% increase in ore production (over 4x). Some balance is needed here.
I already get boatloads of ore from the Digital Miner (which is already a cakewalk to setup with TE's Tesseracts). I already have three of these bad boys (one for the Overworld, one for the Nether, and one I am saving for the right Mystcraft age).

Also, as a side note, The Chemical Washer is slow as molasses on a January morning, even setting up an array of them, to feed a Chemical Crystallizer (I might create a separate ticket if this is not addressed). This machine needs upgrades in the worst way and is the bottleneck of the entire 5x ore production system.
Hope you enjoyed (loathed?) reading this, I took an evening to research this in lieu of building my new base. :)

Yep the chemical washer is the bottle neck. I am glad i wasn't the only one haveing a huge fps hit with the 5x ore processing. I am with you to im not going past 4x ore processing anymore with mekanism , and your right who in there right mind does not want to see all there hard work process! Mekanisms 5x ore processing rendering code needs looked at . Also would be nice if we could put upgrades in the chemical washer since its a huge bottle neck.


Updates testcontainers dependency and updates version constraints for other dependencies.
Why it does that
Briefly state why the change was necessary.

testcontainers had some API changes, to keep up-to-date the version was upgraded and the code appropriately changed.
Other dependencies had also some updates and the version constraints in Cargo.toml were updated.
In Rust Edition 2024 (not yet released) working with the functions std::env::set_var and std::env::remove_var becomes unsafe (see this tracking issue: https://github.com/rust-lang/rust/issues/124866). To prepare for this, the calls to these functions were put into unsafe blocks and the lint level has been changed from forbid to deny, excluding these specific calls with allow directives.

Further notes
List other areas of code that have changed but are not necessarily linked to the main feature. This could be method signature changes, package declarations, bugs that were encountered and were fixed inline, etc.
Linked Issue(s)
Closes # <-- insert Issue number if one exists

During the bumping of the dependencies, it was discovered that the MongoDB implementation is not working anymore. Therefore it was decided, to remove this implementation.

In Rust Edition 2024 (not yet released) working with the functions std::env::set_var and std::env::remove_var becomes unsafe (see this tracking issue:

The new unsafe blocks could use some hint as to why they were deemed safe. E.g. because the program is single-threaded at that point. Note that tests are executed in multiple threads by default.


docs: improve our recommended pattern for amending the npm depgraph

This will also allow renovate to work correctly with our repo :tada:

Thanks to https://github.com/renovatebot/renovate/discussions/16063#discussioncomment-2959346 for pointing me here


Does FlexAttention Support torch.vmap?

I know the scaled_dot_product_attention of torch.nn.functional does not and it is a big problem for me. Does FlexAttention support batched mode?

Cc @zou3519

@MiladInk We haven't added support yet but it shouldn't be too hard. I've also been meaning to add vmap support for scaled_dot_product_attention... might do it this weekend.

btw, we just added vmap support for scaled_dot_product_attention: https://github.com/pytorch/pytorch/pull/133964
We'll also look into doing it for FlexAttention soon.

I am getting
RuntimeError: vmap: It looks like you're attempting to use a Tensor in some data-dependent control flow. We don't support that yet, please shout over at https://github.com/pytorch/functorch/issues/257 .

error when trying out document masking style approach:
    def get_block_mask(self, idx: Tensor) -> BlockMask:
        batch_size, seq_length = idx.shape

        not_pad_mask: BoolTensor = (idx != self.pad_token)  # noqa

        block_end_mask: BoolTensor = (idx == self.block_end_token)  # noqa
        block_ids = torch.cumsum(block_end_mask, dim=-1, dtype=torch.int)

        # determine which blocks to mask
        unique_block_ids = torch.unique(block_ids)
        keep_blocks = torch.ones_like(unique_block_ids, dtype=torch.bool)
        keep_blocks.bernoulli_(
            # enable random masking only during training (similar to dropout)
            p=self.config.mask_block_prob if self.training else 0.0
        )

        document_end_mask: BoolTensor = (idx == self.tokenizer.eot_token)  # noqa
        document_ids = torch.cumsum(document_end_mask, dim=-1, dtype=torch.int)

        def causal(b: Tensor, h: Tensor, q_idx: Tensor, kv_idx: Tensor) -> Tensor:
            return q_idx >= kv_idx

        def padding(b: Tensor, h: Tensor, q_idx: Tensor, kv_idx: Tensor) -> Tensor:
            # do not attend to padding and padding should not attend to anything
            return not_pad_mask[b, q_idx] and not_pad_mask[b, kv_idx]

        def block_masking(b: Tensor, h: Tensor, q_idx: Tensor, kv_idx: Tensor) -> Tensor:
            q_block_id = block_ids[q_idx]
            kv_block_id = block_ids[kv_idx]

            # freely attend to own block and any other unmasked block and any block end token of other blocks
            return ((q_block_id == kv_block_id)
                    or keep_blocks[kv_block_id]
                    or block_end_mask[kv_block_id]
                    or (kv_block_id == 0))

        def document_masking(b: Tensor, h: Tensor, q_idx: Tensor, kv_idx: Tensor) -> Tensor:
            # attend only within same document
            return document_ids[q_idx] == document_ids[kv_idx]

        return create_block_mask(
            and_masks(causal, padding, block_masking, document_masking),
            B=batch_size,
            H=None,  # broadcast over heads
            Q_LEN=seq_length,
            KV_LEN=seq_length,
            BLOCK_SIZE=self.block_size,
        )

Is it related?


Fix conflicting dependencies between upstream JavaModules

Fixes https://github.com/com-lihaoyi/mill/issues/2732
We introduce a new localCompileClasspath, which differs from compileClasspath in that it leaves out the transitiveCompileClasspath from upstream modules and resolvedIvyDeps from third party dependencies.
Also tried to do some cleanup of JavaModule, refactoring out a transitiveModuleCompileModuleDeps helper to DRY things up, re-arranging things so all the fooModuleDeps helpers are all together.
Tested via a new unit test mill.scalalib.HelloWorldTests.multiModuleClasspaths, which fails on master and passes on this PR. This test case asserts the overall "shape" of the classpaths, which should help rule out an entire class of mis-configuration w.r.t. how the classpaths are wired up

I'm going to merge this as is for now: with compileIvyDeps and compileModuleDeps not transitive, but ivyDeps and moduleDeps being transitive, even if depended on from a compileModuleDeps, though in that case only from the compileClasspath.
That should mostly match how provided works in SBT: adding a provided dependency in SBT pulls in all its transitive non-provided dependencies as well. One difference is that this applies to the fullClasspath, because SBT doesn't have a distinction between compile and run classpaths. This should also mostly match the current behavior, at least before the regression described in https://github.com/com-lihaoyi/mill/issues/2732

Just for completeness, I'd like to mention, there is also PR #2736, which also fixes issue #2732 and is more minimal. Just in case we later find this approach too invasive in behavior.

Also, this PR changes the order of dependencies and puts all elements from ivy sources before others. This is a behavioral change. Was this intentional?

Somewhat. The status quo was broken, so of course we'd need to change things, so I while changing things I thought we might as well put things in a consistent order. With this PR, the order is always ivy deps, then upstream modules, then the current module. IIRC the previous orderings weren't as consistent between the various *ClassPaths

Classpath ordering is definitely a risk of breakage, it happens when a user has multiple different files at the same path on the classpath in different jars or classfile folders.
But I don't think that's something we should guarantee too strongly. Even listing things on the filesystem can vary in order, so if a user deploys jars in a folder and does java -cp * classpath ordering can break things if deployed on different machines or even different folders on the same machine (we've seen bugs like this before at work). So in some ways code that depends on classpath ordering is already broken


The Start/Stop VMs solution during off-hours: bug with Resource Group naming

[Enter feedback here]
It seems that the automation is bugging with resource group with a . in the name Error message
XXX.Production is not a valid ResourceGroup Name. Please verify your input!
Thanks
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 225c9d05-83dd-b006-0025-3753f5ab25bf
Version Independent ID: 9eecef0c-b1cb-1136-faf7-542214492096
Content: Azure Automation Start/Stop VMs during off-hours overview
Content Source: articles/automation/automation-solution-vm-management.md
Service: automation
Sub-service: process-automation
GitHub Login: @MGoedtel
Microsoft Alias: magoedte

@ChristopheH-Ekonoo Thanks for your comment! We will review and provide an update as appropriate.

@ChristopheH-Ekonoo Can you please provide the resource group name and also make sure you are using latest version of the solution ? Also,  can you share at what step are you prompted with error ?
Here are few similar issues which can help you in fixing the issue.

https://github.com/MicrosoftDocs/azure-docs/issues/12579
https://github.com/MicrosoftDocs/azure-docs/issues/19360

Kindly through the issue links and if you are still facing the issue , provide the requested data.

Hello Swathi
It is exactly the same as #12579 and the last comment after the closure is from me
The solution was deployed just 2 days before the error
Here are the error on ScheduledStartStop_Parent
Logging into Azure subscription using Az cmdlets...
Environments
{[AzureChinaCloud, AzureChinaCloud], [AzureCloud, AzureCloud], [AzureGermanCloud, AzureGermanCloud], [AzureUSGovernme...
Successfully logged into Azure subscription using Az cmdlets...
Runbook (ScheduledStartStop_Parent) Execution Started...
Validating the resource group name (RG_Espon_Production)
RG_Espon_Production is not a valid ResourceGroup Name. Please verify your input!
Kind Regards
Christophe Humbert
Infrastructure Director
<EMAIL_ADDRESS>62 rue de la Montagne, Luxembourg
21 rue Glesener, SilverSquare, Luxembourg
+352 691 391 974
<EMAIL_ADDRESS>www.allset.lu
From: SwathiDhanwada-MSFT<EMAIL_ADDRESS>Sent: Tuesday, July 28, 2020 07:04
To: MicrosoftDocs/azure-docs<EMAIL_ADDRESS>Cc: Christophe Humbert (AllSet<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [MicrosoftDocs/azure-docs] The Start/Stop VMs solution during off-hours: bug with Resource Group naming (#59484)
@ChristopheH-Ekonoohttps://github.com/ChristopheH-Ekonoo Can you please provide the resource group name and also make sure you are using latest version of the solution ? Also, can you share at what step are you prompted with error ?
Here are few similar issues which can help you in fixing the issue.

#12579https://github.com/MicrosoftDocs/azure-docs/issues/12579
#19360https://github.com/MicrosoftDocs/azure-docs/issues/19360

Kindly through the issue links and if you are still facing the issue , provide the requested data.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/issues/59484#issuecomment-664778284, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AM6S7XC5RIMVUCFNL7VX5HTR5ZL3PANCNFSM4PEM4DHA.

@ChristopheH-Ekonoo Kindly note that you cannot manage VM across subscriptions.If that is not the case, I recommend engaging our technical support team to investigate further. Please file a support case using these steps. If you do not have a support plan, send mail to<EMAIL_ADDRESS>include your subscription ID and a link to this GitHub thread (for context).
As there are no further documentation changes, we will close the issue for now. If there are further questions , please revert and we will be glad to assist you.

Just wanted to make a note for any future sufferers. I had this issue today which was caused by the Automation Run As account being granted permissions to a resource in another subscription. It appears it was connecting to the other subscription first so didn't find the Resource Group, and the command doesn't try other Subscriptions.  Removing the erroneous permissions resolved my issue.
Adding it to the list of things for others to check.


contact me or accounts like connect me on linkedin missing.

should link LinkedIn account

@BamaCharanChhandogi
@devanshar

this issue is solved


fix(ffe-layout-legacy): Remove screen type from media queries, see #719

Coverage remained the same at 81.459% when pulling 00ca0adb15a7fb4e8d61776633e4e8b8e63014d3 on layout-legacy-remove-screen-media into e5b885de1323a8f19028ef7f2e72d866f387fc82 on develop.


Data region

There is an issue in LCS for Self-service environments. The data region is from now on kept as a secret. For "Microsoft Managed" environments you still can see the information. Example: Primary region (Active) West Europe
So one cannot be sure in which data center your Self-Service environment  are located. This would be an very interesting information to present in 2LCS.

Unfortunately, this is no longer exposed in LCS app internal requests. Nothing I can do in 2LCS.

Unfortunately, this is no longer exposed in LCS app internal requests. Nothing I can do in 2LCS.


[Feature] Improve key-pair auth performance

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature
@VersusFacit's analysis:
Params:
Unix time command -- acceptable imprecision for the orders of magnitude we're dealing with here
5000 node project
average 2 runs each for each authentication method
dbt snowflake - user pass: 427.83s user 42.43s system 16% cpu 46:49.12 total
dbt snowflake - key pair: 1011.76s user 44.96s system 32% cpu 53:42.99 total

400 vs 1000 is quite a dramatic difference!
Avenues for investigation:

We want to first try caching the keyfile contents
Redo timing analysis to see if there's any benefit
Look at any extra calls being made to snowflake
If that fails, create a ticket for further investigation

Useful docs I used for testing: https://docs.snowflake.com/en/user-guide/key-pair-auth
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt
openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub
ALTER USER <you> SET RSA_PUBLIC_KEY=''; -- yes this will require admin help; Don't include header/footer strings

Hey team! I found this issue after switching from a 2048-bit key-pair to a 4096-bit one, and found my dbt run times increasing from ~5 minutes to ~15 minutes.
I had a bit of a dig on this, and figured I'd share some findings (and can put together some suggestions for changes as well if there's a preference on how this is handled in the project).
The core issue seems to be that the private key is being read (and validated - I'll get to that) on every dbt node, which eats up the time.
First - this took me too long to find, but the easiest solution seems to just be using the reuse_connections profile config. Maybe it could be a suggestion in the key-pair authentication section of the docs to use that config?
Anyway - in terms of testing what was going on, I did a few tests using the Snowflake Connector for Python and found that (with a fixed private key) execution times were virtually the same across either password, 2048-bit key, or 4096-bit key options.
I had a look at dbt-snowflake and found the above, i.e. that the private key was being read each on each node. Adding a bit of caching somewhat resolved the issue and substantially reduced run times.
I was a bit surprised by this, so I decided to check how long loading keys actually took. My test script looked like this:
def benchmark_key_import(key: str, unsafe_skip_rsa_key_validation: bool = False, n_tries: int = 1000):
    start = time.time()
    for _ in range(0, n_tries):
        private_key = serialization.load_pem_private_key(
            data=bytes(key, 'utf-8'),
            password=None,
            unsafe_skip_rsa_key_validation=unsafe_skip_rsa_key_validation
        )
        key_bytes = private_key.private_bytes(
            encoding=serialization.Encoding.DER,
            format=serialization.PrivateFormat.PKCS8,
            encryption_algorithm=serialization.NoEncryption(),
        )
    end = time.time()
    print(end - start)

Some results from that:

Loading a 2048-bit key 1000 times with validation (which is enabled by default) took 73s
Loading a 4096-bit key 1000 times with validation took 375s
Loading a 2048-bit key 1000 times with no validation took 0.15s

That validation is pretty substantial, which is why I didn't want to immediately put a PR together.
The cryptography docs don't provide much detail on exactly what's unsafe about skipping validation, and I don't know nearly enough about the security elements to say for sure. However, the Snowflake Connector for Python is also using cryptography, and either requires bytes (which it reads with validation) or an instance of RSAPrivateKey (which would already be validated). Essentially, this means that dbt-snowflake can (and perhaps should) skip validation since it's already being done later by the Snowflake Connector and there's no value in doing it twice.
Caching would of course help as well; I imagine there aren't any cases where data changes during an execution such that a cached result became invalid (and if it did, then it could probably be stored in a dictionary instead to maintain it) but my sense based on the above is that skipping validation would be a more sensible solution and would effectively invalidate the need for caching.
Beyond that, and as mentioned at the top, I think enabling the reuse_connections config is the ideal option, since it also means the re-validation that is happening in the Snowflake Connector is also skipped. Enabling this config did result in the shortest run-times in my testing (and run-times that were largely equal across password/private key auth methods). This might be academic, but I'd be interested to know if there's any particular reason for this being disabled by default, and if there's any telemetry on how often it's enabled?

Thanks for the thorough analysis @amardatar! I agree that updating the default behavior of reuse_connections looks like the correct approach. It also improves performance for user/password auth as well. I made that update and translated your performance testing into a dbt-snowflake test.

Hi @amardatar, thanks for the thorough analysis; it is much appreciated!
The Problem
What you've stumbled into looks like a known issue in cryptography 😞 (link):

The cause of this is almost certainly our upgrade to OpenSSL 3.0, which has a new algorithm for checking the validity of RSA private keys, which is known to be much slower for valid RSA keys.
Unfortunately we're in between a rock and a hard place and don't have a way to avoid this cost without silently accepting invalid keys which can have all sorts of negative consequences.
I don't have a better suggestion than "try to avoid loading the same key multiple times".

The description from the PR that implemented the unsafe_skip_rsa_key_validation has this note, which doesn't inspire confidence.

This is a significant performance improvement but is only safe if you know the key is valid. If you use this when the key is invalid OpenSSL makes no guarantees about what might happen. Infinite loops, crashes, and all manner of terrible things become possible if that occurs. Beware, beware, beware.

Given that this keypair is user-configurable from YAML, I don't feel comfortable bypassing this check.
A Potential Solution

Caching would of course help as well; I imagine there aren't any cases where data changes during an execution such that a cached result became invalid (and if it did, then it could probably be stored in a dictionary instead to maintain it).

This is a great observation, so I took the liberty of modifying your script into a reprex to investigate the performance impact.
# benchmark.py
import time
from cryptography.hazmat.primitives import serialization
from functools import cache

@cache
def cached_load_pem_private_key(data, password, unsafe_skip_rsa_key_validation):
    return serialization.load_pem_private_key(
        data=data,
        password=password,
        unsafe_skip_rsa_key_validation=unsafe_skip_rsa_key_validation,
    )

def benchmark_key_import(
    key: str,
    unsafe_skip_rsa_key_validation: bool = False,
    n_tries: int = 1000,
    method=serialization.load_pem_private_key,
):
    print(
        f"unsafe_skip_rsa_key_validation={unsafe_skip_rsa_key_validation}, n_tries={n_tries}, method={method}"
    )
    cached_load_pem_private_key.cache_clear()

    start = time.time()
    for _ in range(0, n_tries):
        private_key = method(
            data=bytes(key, "utf-8"),
            password=None,
            unsafe_skip_rsa_key_validation=unsafe_skip_rsa_key_validation,
        )
        key_bytes = private_key.private_bytes(
            encoding=serialization.Encoding.DER,
            format=serialization.PrivateFormat.PKCS8,
            encryption_algorithm=serialization.NoEncryption(),
        )
    end = time.time()

    print(end - start)
    print(cached_load_pem_private_key.cache_info())
    print()

# openssl genrsa -out key.pem 4096
with open("key.pem", "r") as f:
    key = f.read()

    benchmark_key_import(key)
    benchmark_key_import(key, unsafe_skip_rsa_key_validation=True)

    benchmark_key_import(key, method=cached_load_pem_private_key)
    benchmark_key_import(
        key, method=cached_load_pem_private_key, unsafe_skip_rsa_key_validation=True
    )

❯ openssl genrsa -out key.pem 4096
❯ python benchmark.py
unsafe_skip_rsa_key_validation=False, n_tries=1000, method=<built-in function load_pem_private_key>
345.70546793937683
CacheInfo(hits=0, misses=0, maxsize=None, currsize=0)

unsafe_skip_rsa_key_validation=True, n_tries=1000, method=<built-in function load_pem_private_key>
0.16813015937805176
CacheInfo(hits=0, misses=0, maxsize=None, currsize=0)

unsafe_skip_rsa_key_validation=False, n_tries=1000, method=<functools._lru_cache_wrapper object at 0x102551170>
0.4363410472869873
CacheInfo(hits=999, misses=1, maxsize=None, currsize=1)

unsafe_skip_rsa_key_validation=True, n_tries=1000, method=<functools._lru_cache_wrapper object at 0x102551170>
0.09387397766113281
CacheInfo(hits=999, misses=1, maxsize=None, currsize=1)

From the results above, we can get most of the runtime improvement from caching serialization.load_pem_private_key (and maybe even private_key.private_bytes?), so I'd prefer to go down this route instead.
Thanks again for starting the conversation @amardatar, would love to hear your perspective.

Hey @aranke - no worries, and thanks for the context - I didn't know that, but makes sense!
So my suggestion on using unsafe_skip_rsa_key_validation is really just predicated on the fact that the Snowflake Connector is already doing validation - which means it's being done twice at the moment, and arguably can be skipped once. That's not necessarily a good reason to do this though - I wouldn't expect the behaviour to change in the Snowflake Connector, but perhaps relying on that behaviour isn't ideal either.
In terms of caching - I did try a very basic implementation in dbt-snowflake, and didn't leave enough info in the comment above, but it's probably useful so I'll just run through it here.
Just summarising the below:

seralization.load_pem_private_key returns an instance of PrivateKeyTypes, and if this is passed to the Snowflake Connector, then the Snowflake Connector doesn't need to re-validate the key.
Caching on _get_private_key, which currently returns bytes via PrivateKeyTypes.private_bytes(), is only partially effective since the Snowflake Connector ends up re-validating the key.
Attempting to change _get_private_key to return an instance of PrivateKeyTypes is not trivial (although it's not necessarily difficult either) as the class is serialised at some point.
An option may be to cache the bytes (which are serialisable), but return PrivateKeyTypes from _get_private_key to avoid re-validation in the Snowflake Connector - and to avoid re-validation in dbt-snowflake, it can be skipped using the unsafe flag if cached key bytes are present (so initially setting the cache runs without the unsafe flag).

In more detail:
My first attempt with caching was just a straight-forward cache of the result of _get_private_key - and while that did roughly halve the time it took for a run, it didn't save as much time as it should. I believe the reason for this is because _get_private_key returns bytes, and providing that to the Snowflake Connector results in it re-validating the key. This makes sense - half the time because half the number of key validations.
I tried instead returning and using the response of serialization.load_pem_private_key - which is an instance of PrivateKeyTypes - but instead got the error TypeError: expected bytes-like object, not RSAPrivateKey with the stack trace pointing to this line.
I don't know enough about how this is all being used to know if it's okay to avoid serialising the key in some way - if that can be done, then I think the performance benefits will be realised since the key won't need to be re-validated (by dbt-snowflake or by the Snowflake Connector). Without doing that, caching the bytes still leaves a lot of performance on the table.
Maybe another option though would be to skip validation after the first time (and return an instance of PrivateKeyTypes from _get_private_key) - this means the very first usage by dbt-snowflake is validated, but subsequent usages are not. The main reason for caution that I can think of here is that I'm not sure how any serialised results of the class are being used, and it's possible that at some point the private key could be changed in the serialised form, and used (without validation) in a subsequent call.

Opened a new issue in dbt-labs/docs.getdbt.com: https://github.com/dbt-labs/docs.getdbt.com/issues/5779


Reader for CSV files

🚀 Feature
I would love to make drag&drop for CSV files work in napari.
Motivation
One of my plugins supports loading a CSV file into the properties/features of a [labels] layer.
Pitch
For visualizing archived measurements, we need to load CSV files (and/or other file formats). The tricky part is that the current reader-interface does not support loading something into an existing layer (such as the currently selected layer).
Alternatives
The current implementation is a magicgui-like dock widget, which is quite cumbersome for this purpose.
Additional context
There are multiple plugins that load CSV files the one or the other way. It might make sense to explore what they do and see if code can be reused to implement this feature request:
https://www.napari-hub.org/?search=csv&sort=relevance&page=1

Why creating a reader that will read labels (from tiff for example) and csv at once and produce a new layer from it is not a proper solution?

Great idea! In the future, with NGFF, we could store label images together with tabular data as meta data.
How would you write a loader for pair of image+table data? Must they come together as a zip file? Or must they have similar names?

How would you write a loader for pair of image+table data? Must they come together as a zip file? Or must they have similar names?

Both are options. You could provide data in single bundle (zip, tar, h5, etc), or you could use the name that could be guessed based on the label file name.
From my point of view, both solutions reduce the probability of user error.
My example from PartSeg is checking if next to the image file there is a file with the _mask suffix to load the masking layer.
Next to such reader should be wrttter that will be saving all files in one action (labels layer data, CSV, etc).


How to prevent tfjob from Running while there are still pods in Pending status?

In our production environment users run into this situation occasionally: some pods started running while some others were still pending due to insufficient resources or other reasons. This will leading to a problem that tfjob was starving because of absent partners.
I understand this may be not a problem to tf-operator framework. Since pod's command is launching tensorflow job, and if this pod is running already, it is actually running tensorflow.
Is there any suggestion on this anyway? My initial thoughts were adding some check all pod status and if not all running then wait command before the python runtf.py command manually?

Disclaimer: I am not a contributor of tf-operator.
That's what gang-scheduling was invented for. You basically need to setup another scheduler e.g. kube-batch and enable gang-scheduling in tf-operator. The new scheduler will run whole group of pods at a time, or leave them all pending, so you would not encounter partial running group.

More customization is discussed in this thread: https://github.com/kubeflow/tf-operator/issues/916

Many thanks to your reply, which helps a lot! @pskindel @johnugeorge


fix numeric status not displaying by ensuring it's a string

[X] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

Proposed changes

ensures text status is actually a string so that it will be displayed.
Checklist

[x] I have read the contribution guidelines
[ ] For non-bugfix PRs, I have discussed this change on the forum/slack team.
[x] I have run grunt to verify the unit tests pass
[ ] I have added suitable unit tests to cover the new/changed functionality

Coverage increased (+0.007%) to 68.423% when pulling e86f6a841ab6b3f5856937e942ab7b06e811cd70 on fix-numeric-status into fad8dcd304728014551da41186d25db9f7320abd on master.


Scrolling in element catalog not functioning.

From @arcio on October 23, 2015 22:1
I'm working in chrome Version 46.0.2490.71 (64-bit)
When viewing the catalog the left-hand column won't scroll.
For instance:
https://elements.polymer-project.org/elements/neon-animation
The page will scroll but the left column will not. You have to zoom out to be able to see the entire list.
Copied from original issue: Polymer/polymer#2620

From @abdonrd on October 24, 2015 9:25
I confirm the problem.

Easy fix is to add this to the :host rule in app-sidebar:
overflow-y: auto;
-webkit-overflow-scrolling: touch;

Seems like a regression from this commit: https://github.com/Polymer/polymer-element-catalog/commit/0326093fe68c1da4a3885551ef74a8df5c1592f9.  @blasten, we probably need to add back the overflow-y: auto in to the right place.

That's right, I can fix this one


support for passing dataset keys for Zarr files

Hi!
I am wondering if it is possible to allow passing specific dataset keys for zarr files (changes anywhere load_stack/smart_load gets called). Right now, the data to be processed needs to be under '/' (root group). However, often pmaps can be stored like 'volumes/pmaps'. The zarr could also contain other keys ('volumes/raw') and I feel plantseg should be able to handle that and just load the dataset specified for processing.
Best,
Samia

Hey @lorenzocerrone I think you fixed this by #212?

This is supported in v2.0.0aX


python312.tensorflow: remove distutils

Remove distutils reference using an archlinux patches. Fixes #305827 while waiting for upstream fix.
Description of changes
tensorflow does not build with python3.12 as it still dependens on dist utils. Archlinux folks got a fix for that.
This build is RAM-expensive, I had to use a "beefy" machine to compile the derivation !
So I could not get a proper nixpkgs-review.
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.
 After the PR, I corrected the URL of the patch (not sure why it managed to compile with the wrong URL). All the tests are failing with the proper patch URL then ?

Thanks for the quick feedback ! After the PR, I corrected the URL of the patch (not sure why it managed to compile with the wrong URL). All the tests are failing with the proper patch URL then ?

No worry ! Let me check.

Result of nixpkgs-review pr 325224 run on x86_64-linux 1

16 packages marked as broken and skipped:

python312Packages.bambi
python312Packages.bambi.dist
python312Packages.distrax
python312Packages.distrax.dist
python312Packages.elegy
python312Packages.elegy.dist
python312Packages.n3fit
python312Packages.n3fit.dist
python312Packages.rlax
python312Packages.rlax.dist
python312Packages.tf2onnx
python312Packages.tf2onnx.dist
python312Packages.treex
python312Packages.treex.dist
python312Packages.trfl
python312Packages.trfl.dist

52 packages failed to build:

frigate
gpt2tc
python312Packages.baselines
python312Packages.baselines.dist
python312Packages.cleanlab
python312Packages.cleanlab.dist
python312Packages.dm-haiku
python312Packages.dm-haiku.dist
python312Packages.edward
python312Packages.edward.dist
python312Packages.flax
python312Packages.flax.dist
python312Packages.gpt-2-simple
python312Packages.gpt-2-simple.dist
python312Packages.imgcat
python312Packages.imgcat.dist
python312Packages.islpy
python312Packages.islpy.dist
python312Packages.keras
python312Packages.keras.dist
python312Packages.loopy
python312Packages.loopy.dist
python312Packages.nanobind
python312Packages.nanobind.dist
python312Packages.objax
python312Packages.objax.dist
python312Packages.pymc
python312Packages.pymc.dist
python312Packages.pyopencl
python312Packages.pyopencl.dist
python312Packages.pytensor
python312Packages.pytensor.dist
python312Packages.reikna
python312Packages.reikna.dist
python312Packages.sasmodels
python312Packages.sasmodels.dist
python312Packages.tensorflow (python312Packages.tensorflow-build ,python312Packages.tensorflowWithoutCuda)
python312Packages.tensorflow.dist (python312Packages.tensorflow-build.dist ,python312Packages.tensorflowWithoutCuda.dist)
python312Packages.tensorflow-probability
python312Packages.tensorflow-probability.dist
python312Packages.tensorflowWithCuda
python312Packages.tensorflowWithCuda.dist
python312Packages.tf-keras
python312Packages.tf-keras.dist
python312Packages.tflearn
python312Packages.tflearn.dist
python312Packages.vqgan-jax
python312Packages.vqgan-jax.dist
witnessme
witnessme.dist
wyoming-openwakeword
wyoming-openwakeword.dist

2 packages built:

python312Packages.pot
python312Packages.pot.dist

It seems that the patch does not apply:
Running phase: installCheckPhase
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/build/tensorflow-2.13.0-python/tensorflow/__init__.py", line 29, in <module>
    import distutils as _distutils
ModuleNotFoundError: No module named 'distutils'

That would explain the errors I've got afterwards too.  The error is during the installCheckPhase but not sure why the patch doesn't apply :/

The patch has been merged upstream: https://github.com/tensorflow/tensorflow/pull/70959

There are still references to distutils https://github.com/search?q=repo%3Atensorflow%2Ftensorflow distutils&type=code
https://cs.opensource.google/search?q=distutils&ss=tensorflow%2Ftensorflow:tensorflow%2F

What's the status here?

I have an untested patch that I'll add here very soon.
On Monday, August 5th, 2024 at 22:24, Pol Dellaiera @.***> wrote:

What's the status here?

—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.

Merged upstream : https://github.com/tensorflow/tensorflow/pull/74090#pullrequestreview-2340129419

Sadly it's slightly after the branch cut for the next release :( I forgot to check that in time


Update Flight Hub to include latest 20H1 builds

Added build 19033 which flighted on 11/26/2019 for both - the Insider Fast & Insider Slow rings
Added SDK for build 19028 which was published on 11/27/2019
Added build 19025 which flighted on 11/20/2019 for the Insider Slow ring

cc @bleblancMSFT


added get started in into

Related Issue

#551

Work Done

Added get started section

A big thank you @khaledyoussef24 for all the PR reviews!!


not equal operator isn't parsed correctly

This query
find {webChannel: != null} return .webChannel limit 100

leads to that error:
{"error": "Error: Expected comparison operator"}

I figured out that the "not equal" operator is !==. Can we change that to simply !=. That would be more consistent. Only JavaScript does !== and we also don't do ===.

I'm closing this one in favour of #31 which should be more clear.


[Error] PDF cover cannot be shown

Platform:  Debian GNU/Linux 8.10 (jessie)
1、[error] PDF cover cannot be shown correctly. See the Picture1 below.
2、[enhancement] Can you add a go-back-to-cover button on epub browser. See the Picture2 below.
Thank you!
Picture1

Picture2

Please make high priority to issue #47 which is a big performance improvement for huge books like mine. I don't think the projects I listed are mature enough for use.
And I think I will try a workaround for the thumbnail issue of PDF, ie, use ghost script to generate thumbnails for BookBrowser. Anyway, the book should not update too often.

Yep. I was thinking of something like that. I'll probably make it call imagemagick if it is present.
As for #47, I'm working on it when I have time. I've figured out how to do an efficient checksum of the epub without reading the whole thing, and now, I just need to figure out how I'm going to store metadata in a forwards compatible way.

Thanks for your hard word!

Another candidate in go
https://github.com/hhrutter/pdfcpu

@LiMinggang That looks almost perfect! I'll see how it works later today.

There is another JS version which supports PDF thumbnail. No idea if it's OK for us.
https://github.com/oguzhaninan/Buka

No, that won't work. It uses PDF.js to render it. Last time I tried, I couldn't get it running in otto (a js runtime for go).


Upgraded to .NET 6

Upgraded dependencies and upgraded .NET Framework to whats supported by Microsoft.

This is a PR to just show this functionality is done. We are going to upgrade the actions and stuff in another ticket.


OneToMany relationship with User entity is ignored

OneToMany relationship with the User entity is ignored.
The following jdl
entity AnEntity {
    aField String
}

relationship OneToMany {
    User{anEntity} to AnEntity{user}
}

...generates the follwing jhipster entity.json without the User relationship
{
  "relationships": [],
  "fields": [
    {
      "fieldId": 1,
      "fieldName": "aField",
      "fieldType": "String"
    }
  ],
  "changelogDate": "20151020210726",
  "dto": "no",
  "pagination": "no"
}

Here is the above example in a jhipster sample app if needed.
Just run jhipster-uml entities.jh

By the way I tried to flip the relationship to
relationship ManyToOne {
     AnEntity{user} to User{anEntity}
}

without any success.

I'm still trying to fix another issue. You'll have to be patient.

I tested your case with the fix and it seems to work.

Great! Looking forward of using it.