Automatic batch sizing

PyTorch Lightning supports a mode where it automatically computes the maximum batch size on your accelerator (by collecting gradients over a few batches and then using binary search to find a max that doesn't give OOM errors). Documentation is here. It appears straightforward to implement (see "Note" on docs.)
Having this enabled would be very useful for computing the maximum batch size possible. From this one could factor the desired batch size and accumulate gradients across multiple batches per optimizer step, so this is related to #132 in the obvious way.
This will also combine nicely with a move to LightningCLI, IMO, since that uses a subcommand interface and we could do something like yoyodyne scale_batch_size --arg1 --arg2 to enable this.

I think I am on the hook to finally do this. Current thoughts after looking at previous discussions:

A user specifies a --batch_size
Optionally, a user can specify a --max_batch_size, in which case we recompute something like batch_size_prime and a number of gradient accumulation steps k so that we do not put > max_batch_size on GPU at a time. With these + PTL features, we can ask the trainer to accumulate gradients thus simulating an effective batch size of batch_size automatically.
- I guess we can use your implementation here: https://gist.github.com/kylebgorman/178f6fe9a7b83286d0d30c8136575c81 for that (I have not looked at it closely FYI).
Instead of --max_batch_size a user can specify something like --find_max_batch_size. Then, we set max_batch_size automatically using the PTL feature, and if max_batch_size > batch_size, we compute batch_size_prime and k to automatically accumulate gradients.

This means we add two new args: --max_batch_size and --find_max_batch_size, where --max_batch_size should not be set if  --find_max_batch_size is requested. I think supporting setting --max_batch_size manually makes sense as the finding step can be skipped if I have already found it for my exp + GPU, and I just want to enter it manually instead of using compute.
Does this seem right? I think this could go in a combinaiton of util.py and train.py but we can make another module too if need be.

Let's define two things: "notional batch size" is the number of elements per training step, and "effective batch size" is the size of the minibatches of which there is at least one per step. Then:
(A) effective batch size x N = notional batch size
where N is a positive counting number >= 1. Users need to be able to:

train by specifying effective batch size and N
find the largest batch size possible, and
compute, for a given notional batch size, the largest effective batch size (and the smallest N) that satisfies (A).

Currently, effective batch size should be specified with --batch_size and N with --accumulate_grad_batches. That seems fine and satisfies (1).
In a perfect world, I think (2) would be done with a separate CLI tool called yoyodyne-tune maybe. It would inherit all, or nearly all, the flags from the trainer, and just call the tuner instead. (If this is done I can also move the --auto_lr-find tuner into this CLI tool too.) One issue in the design of (2) is how it can be integrated into the sweeping code. It is my contention that we would want to sample --max_batch_size rather than batch size during sweeps, apply (2) and (3), and then train with that. And that needs to be feasible. Any thoughts on this issue?
As I understand it you're proposing putting (3) in (1) which makes it more general; i.e., you can use it without using (2), e.g. if you think you know what the answer to (2) is, and I think I agree. So then you'd just add --max_batch_size as a flag as you propose.
I think yes, you'd stash this all in train.py unless you're creating a new binary, in which case you'd import a lot of the functionality from train.py (and probably lightly refactor it).
My implementation of the solver for (A) appears to work. I don't know if the brute force or elegant solver is better---but I can do a timing experiment.
One last thing, you may want to look at the PTL 2 interface to make sure it's not radically different.

Timing results below.
Elegant solution
$ python -m timeit -r 10 -s 'import size; import random; random.seed(1985)' 'size.size_elegant(random.randrange(2 ** 7, 2 ** 12, 2 ** 7), random.randrange(16, 1000))'
50000 loops, best of 10: 6.37 usec per loop

Brute force solution
$ python -m timeit -r 10 -s 'import size; import random; random.seed(1985)' 'size.size_brute_f[Downloads]$ python -m timeit -r 10 -s 'import size; import random; random.seed(1985)' 'size.size_brute_force(random.randrange(2 ** 7, 2 ** 12, 2 ** 7), random.randrange(16, 1000))'
50000 loops, best of 10: 4.09 usec per loop```

# Conclusions

Brute force is always a little faster, even when I try different ranges of values and not accounting for the time spent importing `sympy`. (I believe it is already an indirect dependency of our system, since I see it getting installed when I install Yoyodyne on a fresh system.) So I would be fine just using it if you want.

We can write that one can solve this elegantly by computing all divisors of the notional batch size, but we find that a simple brute force solution from the top to bottom is slightly faster.

Let's define two things: "notional batch size" is the number of elements per training step, and "effective batch size" is the size of the minibatches of which there is at least one per step. Then:
(A) effective batch size x N = notional batch size

I think for consistency we may want to define tehm in the opposite, in general. See here in PTL docs: https://lightning.ai/docs/pytorch/stable/advanced/training_tricks.html "Accumulated gradients run K small batches of size N before doing a backward pass. The effect is a large effective batch size of size KxN, where N is the batch size.", I have this this usage of "effective batch size" in HF argument names too I think.

where N is a positive counting number >= 1. Users need to be able to:

train by specifying effective batch size and N
find the largest batch size possible, and
compute, for a given notional batch size, the largest effective batch size (and the smallest N) that satisfies (A).

Currently, effective batch size should be specified with --batch_size and N with --accumulate_grad_batches. That seems fine and satisfies (1).
In a perfect world, I think (2) would be done with a separate CLI tool called yoyodyne-tune maybe. It would inherit all, or nearly all, the flags from the trainer, and just call the tuner instead. (If this is done I can also move the --auto_lr-find tuner into this CLI tool too.) One issue in the design of (2) is how it can be integrated into the sweeping code. It is my contention that we would want to sample --max_batch_size rather than batch size during sweeps, apply (2) and (3), and then train with that. And that needs to be feasible. Any thoughts on this issue?

Yeah previously I had a script for running sweeps that does 3) when converting sweep hyperparameters to yoyodyne training. But to do this I set a --max_batch_size manually. To combine this with 2) don't we just need to call the lightning batch_size_finder and presumably it returns max_batch_size for us? Basically we would need to init a trainer with the batch size finder callback, cal fit on it, and then figure out how to parse the max batch size out of that interface (probably trivial though I did not see it in the docs).

As I understand it you're proposing putting (3) in (1) which makes it more general; i.e., you can use it without using (2), e.g. if you think you know what the answer to (2) is, and I think I agree. So then you'd just add --max_batch_size as a flag as you propose.

Yes!

I think yes, you'd stash this all in train.py unless you're creating a new binary, in which case you'd import a lot of the functionality from train.py (and probably lightly refactor it).
My implementation of the solver for (A) appears to work. I don't know if the brute force or elegant solver is better---but I can do a timing experiment.
One last thing, you may want to look at the PTL 2 interface to make sure it's not radically different.

For the batch size finder?

I think for consistency we may want to define tehm in the opposite, in general.

Yea, I realize these are not great terms. Redo them whatever way you think is best.

Yeah previously I had a script for running sweeps that does 3) when converting sweep hyperparameters to yoyodyne training. But to do this I set a --max_batch_size manually. To combine this with 2) don't we just need to call the lightning batch_size_finder and presumably it returns max_batch_size for us? Basically we would need to init a trainer with the batch size finder callback, cal fit on it, and then figure out how to parse the max batch size out of that interface (probably trivial though I did not see it in the docs).

Yeah, the question is whether you want a batch size finder and then parse its output, or pack both of them in the trainer.

One last thing, you may want to look at the PTL 2 interface to make sure it's not radically different.

For the batch size finder?

Yes.

Yeah previously I had a script for running sweeps that does 3) when converting sweep hyperparameters to yoyodyne training. But to do this I set a --max_batch_size manually. To combine this with 2) don't we just need to call the lightning batch_size_finder and presumably it returns max_batch_size for us? Basically we would need to init a trainer with the batch size finder callback, cal fit on it, and then figure out how to parse the max batch size out of that interface (probably trivial though I did not see it in the docs).

Yeah, the question is whether you want a batch size finder and then parse its output, or pack both of them in the trainer.

Sorry so you mean should we combine 2) and 3) by subclassing the PTL trainer? I guess this factors into your comments about a separate cli yoyodyne-tune. Is this coupled to a particular trainer class? I can look into that. I had been thinking of this as simply adding a python function into something like train.py, which first calls the trainer to get max_batch_size (or optionally does not if we have requested a max_batch_size), and then sets args to the "real" trainer for accumulating gradients.

One last thing, you may want to look at the PTL 2 interface to make sure it's not radically different.

For the batch size finder?

Yes.

Ok will do.

Sorry so you mean should we combine 2) and 3) by subclassing the PTL trainer? I guess this factors into your comments about a separate cli yoyodyne-tune. Is this coupled to a particular trainer class? I can look into that. I had been thinking of this as simply adding a python function into something like train.py, which first calls the trainer to get max_batch_size (or optionally does not if we have requested a max_batch_size), and then sets args to the "real" trainer for accumulating gradients.

I don't think this needs to be done with subclassing. In PTL 2 it's now a callback passed to the trainer. In PTL 1.9 it is done with an argument to the trainer constructor. So I think either way you want to construct the trainer object before you find the size...

Sorry so you mean should we combine 2) and 3) by subclassing the PTL trainer? I guess this factors into your comments about a separate cli yoyodyne-tune. Is this coupled to a particular trainer class? I can look into that. I had been thinking of this as simply adding a python function into something like train.py, which first calls the trainer to get max_batch_size (or optionally does not if we have requested a max_batch_size), and then sets args to the "real" trainer for accumulating gradients.

I don't think this needs to be done with subclassing. In PTL 2 it's now a callback passed to the trainer. In PTL 1.9 it is done with an argument to the trainer constructor. So I think either way you want to construct the trainer object before you find the size...

Right, I meant that I thought you were saying to combine 2) and 3) inside the trainer, which they do not already do as far I can tell. I can just try to add this to train.py by first calling the trainer with the batch size finder callback/arg, then setting batch_size and accumulation steps for the real trainer, and we can discuss more in a PR?

SGTM. I don't know what the right solution is yet.

STill catching up on PR, useful code may be how we implement transducers for asr. (We need to mini batch for the grad accumulation since it's such a memory hog.) https://github.com/NVIDIA/NeMo/blob/9218c3aab7af7c2d7f3d6e45c0b027bafe25eba8/nemo/collections/asr/modules/rnnt.py#L1420

Summarizing an offline conversation I had with @bonham79, I think the following is true:

It would be nice to have a way to compute max batch size without also training.
However, it is essential to have an ability to compute batch size within the trainer. If this isn't inside the trainer, it won't be possible to do this (repeatedly, as is necessary) during a sweep as the size of the model and/or desired batch size changes on each iteration of the sweep.
One obvious synthesis of this is to put the automatic batch size computation into the trainer; this will first be computed and logged. Then, if actual training isn't disabled (one might set --max_epochs 0 or someting like that if you don't really want to train), this will then, in combination with the desired batch size flag, compute and set up the gradient accumulation stuff, and then train under those settings.

Hmm, thinking about this. I think this may be a case of needing to add a wrapper to the default PTL trainer instead of stashing in train.py or subclassing. This should give more control during the Wandb sweeps and allow us to alias stuff. (Like, can pass a new flag --find-batch-size that just sets internal --max_epochs 0.)  We'd probably would have to do this anyway. (Libraries I've been seeing keep needing to rewrite PTL anyhow.)

What are the explicit advantages to doing this with a "wrapper" as opposed to just rolling it into our train methods?

Most PTL code interactions use the trainer function as interface. So it would give us more  control on how downstream apis and libraries engage with our models


locale: Text labels for some options

I don't know enough about HEVC tiers and "maximum bitrate" in VBR, so I left some "placeholder text" commented out. Also, not sure about line 158.

Description
Added some explanations where I could, in hopes of them being useful. Also added commented-out placeholder texts for missing labels.
Motivation and Context
Issue #11
How Has This Been Tested?
Works on my machine
Types of changes

Documentation (a change to documentation pages)
Checklist:

[ ] My code has been run through clang-format.
[x] I have read the contributing document.
[x] My code is not on the master branch.
[x] The code has been tested.
[ ] All commit messages are properly formatted and commits squashed where appropriate.

Should be good now, I hope.

Updated again.


[Revamp] Add comparison operators

Equal to
Not equal to
Less than
More than
Less than or equal to
More than or equal to

`void fibonacci(n) {
if (n == 0) return();
if (n <= 2) {
putbeeper();
return():
}
fibonacci(pred(n, 1));
fibonacci(pred(n, 2));
}
`

Comparison operators together with the backpack and floor globals allow for solving some traditional Karel tasks with much shorter code. But they don't really add new functionality or drastically change the "Karel-spirit".  Is a small change that can leverage Karel into new interesting areas.
The valid operators should be ==, <, <=.  (!=, >, >= can be achieved with the "not" operator)


Install file directory location

What feature or improvement do you think would benefit Files?
I'm trying to locate the install directory of this application in Windows 10. I wish it was a reveal button under settings.
Requirements

This proposal will accomplish X
This proposal will accomplish Y
This proposal will accomplish Z

Files Version
v2.4.40
Windows Version
win 10
Comments
n/a

Type files.exe into the start menu and click open file location


[Bug]:  MacOS slow infererence

Project Version
3.2.6
Platform and OS Version
MacOS
Affected Devices
M1 M2
Existing Issues
No response
What happened?
dura mucho para hacer una inferencia de un audio de 5 o 10 segundos y no utilizar mps backend para acelerar cpu
Steps to reproduce

Expected behavior
idk
Attachments
nothing

Screenshots or Videos
No response
Additional Information
No response

#836 on Windows too

answered on the other issue


Update MODS schema

Is your feature request related to a problem? Please describe.
I am not very familiar with MODS. We recently had a request to improve the MODS crosswalk in our repository in order for us to be harvested more thoroughly and I noticed that DSpace 6.4/7.6 currently uses MODS version 3.1, which was released in 2005.
Describe the solution you'd like
Update the MODS schema to a more modern version. From what I understand the newer versions are much more expressive.
Describe alternatives or workarounds you've considered
Additional context
The MODS website shows that version 3.8 has been released on 2022-05-22.

Agreed, but the priority is unclear to me.  I'm not sure how much the MODS metadata format is used via OAI-PMH (as that's the only place DSpace uses MODS).  For now, I'm flagging as "help wanted" but leaving it unscheduled.  I'm glad to pull this over to a board though if we find a volunteer who wants to work on this.

Thanks @tdonohue. Agree the priority is very low and needs a volunteer to take it forward (will probably be me). We had a request by FAO for us to improve the MODs mappings in our repository and I noticed that the DSpace implementation essentially hasn't been touched in twenty years. I wanted to get an issue filed to at least get it on the record here.


Create API review only when running CI on master branch

Automatic API reviews are created when CI runs on internal pipeline and this causes issue when its triggered against any feature branch. Feature branch may have different set of API changes than in master and this creates a new revision of API review in pending status and causes issue when next CI runs from master branch. API tool finds a difference in API surface level since comparison is against feature branch instead of previously approved master branch.
This PR is to temporarily disable automatic API review creation from feature branch. Long term fix will be to maintain only one revision in pending status which will only have delta against approved version. So irrespective of source of CI trigger, this will always match against last approved revision.

The following pipelines have been queued for testing:java - templatejava - template - testsjs - templatenet - templatenet - template - testspython - templatepython - template - tests You can sign off on the approval gate to test the release stage of each pipeline. See eng/common workflow


Dynamically set store name with data from backend

Hello,
I want to load different stores from localStorage depending on the logged in user.
So I wanted to call createStore a little bit later, in an useEffect, after I get from backend the username, so that I can set the name of the store containing the username. This way depending on which user is logged in, the correct store is retrieved.
I pasted an example of what I need and I wasn't able to make it work.
I was able to create new stores with the correct names and have them populated with the state I wanted, but if I do a Refresh in Browser, the created stores are then ignored and data is not retrieved from them, but from some kind of an empty store.
Is this supported? Have I done something wrong? Should I do it differently?
I felt like this way was the least invasive way of doing it, otherwise if I add 'username' as a root inside the state object, I will have to do more refactor of our code.
`
export default function App() {
  const { username, isLoading } = useUsername();
  const [storeCreated, setStoreCreated] = React.useState<boolean>(false);

  React.useEffect(() => {
    if (!isLoading && username) {
      console.log('createStore CALLED');
      createStore({
        yourDetails: {
          firstname: "bill",
          lastname: "luo"
        },
        {
          name: `myName_${username}`,
          middleWares: [],
          storageType: localStorage,
        }
      });
      setStoreCreated(true);
    }
  }, [username, isLoading]);

  return (
    <StateMachineProvider>

Little State Machine

        <Form />

    </StateMachineProvider>
  );
}

`

I was able to solve it like this, so this can be closed.
`
function StateMachineProviderWrapper({ children }: { children: React.ReactNode }): JSX.Element {

  const { username, isLoading } = useUsername();
  const [storeCreated, setStoreCreated] = React.useState<boolean>(false);

  React.useEffect(() => {
    if (!isLoading && username) {
      console.log('createStore CALLED');
      createStore({
        yourDetails: {
          firstname: "bill",
          lastname: "luo"
        },
        {
          name: `myName_${username}`,
          middleWares: [],
          storageType: localStorage,
        }
      });
      setStoreCreated(true);
    }
  }, [username, isLoading]);
  if (storeCreated) {
    return <StateMachineProvider>{children}</StateMachineProvider>
  }
    return <>{children}</>
  }
}

export default function App() {
  return (
    <StateMachineProviderWrapper>

Little State Machine

        <Form />

    </StateMachineProviderWrapper>
  );
}

`


feat port: Algebra.DirectSum.Decomposition

A @[simps (config := { fullyApplied := false })] on line 133 is causing a maximum recursion depth so I commented it out with a porting note

bors r+


retropie install gpi.sh removes audio for rpi zero 2w

Sorry i am not very famialiar with the audio system, when i run this script i lose audio capability
wget -O - "https://raw.githubusercontent.com/RetroFlag/GPiCase2-Script/main/retropie_install_gpi2.sh" | sudo bash

+1.  Verified that using retropie 4.8 + the safe shutdown script makes me lose access to audio too.

I've encountered the same issue running retropie 4.8 on the rpi zero 2w in the GPi Case 2W once installing the safe shutdown script for the GPi Case 2. I've gotten audio working again and here's what I found:
Modifying the /etc/rc.local to comment out the lines added by the script disables safe shutdown and audio returns. The lcdfirst.sh and lcdnext.sh scripts appear to be the culprit as they appear to me to be what's used to detect when the GPi Case 2 is connected and disconnected to the dock in order to switch between HDMI and LCD output. I was able to disable both of them from running and keeping SafeShutdown.py by doing the following:

Comment out the sh /opt/RetroFlag/lcdfirst.sh& and sleep 1& lines in /etc/rc.local to disable the lcdfirst.sh script from running.
Comment out lcdrunProcess = Process(target = lcdrun), lcdrunProcess.start(),  and lcdrunProcess.join() lines in /opt/RetroFlag/SafeShutdown.py


Payment channel in LND

Hi,
I have created a new channel and send some LND to another channel. The payment transaction was successful.
It will not reflect in another channel, But amount will be detected in first channel.
Can anyone provide me a payment channel document for LND?
Your environment

version of lnd
lnd version 0.12.1-beta commit=v0.12.1-beta

which operating system (uname -a on *Nix)/

version of btcd, bitcoind, or other backend
bitcoind: Bitcoin Core version v22.99.0-08bcfa27675d

any other relevant environment details

You can check this link,
https://docs.lightning.engineering/lightning-network-tools/lnd

It will not reflect in another channel

What do you mean here? What command did you use to check the balance?

You can check this link, https://docs.lightning.engineering/lightning-network-tools/lnd

It will not reflect in another channel

What do you mean here? What command did you use to check the balance?

Hi,
Using below commands to send LND to another channel.
Command for checking balance: ./lncli --network=testnet walletbalance
connect 1st with 2nd server
open a channel with local amount in server 1
./lncli --network=testnet openchannel --push_amt=10 --local_amt 20000
It will added in list channels
Add invoice in server2
./lncli --network=testnet addinvoice --amt=10 --memo="test"
send payment in server1
./lncli --network=testnet sendpayment --pay_req=lnt........
check lookupinvoice
./lncli --network=testnet lookupinvoice ""
close channel

If you pay from one node to another, then on one node you have an invoice (the node getting paid) and on the other node you have a payment (the node paying). So I'm not sure if that is your question, but you should see the same payment with lncli listinvoices on the payee node and with lncli listpayments on the payer node.
Be aware that --push_amt is equal to sending a payment. So you're basically giving free money to the other node!

If you pay from one node to another, then on one node you have an invoice (the node getting paid) and on the other node you have a payment (the node paying). So I'm not sure if that is your question, but you should see the same payment with lncli listinvoices on the payee node and with lncli listpayments on the payer node.
Be aware that --push_amt is equal to sending a payment. So you're basically giving free money to the other node!

Hi @guggero
Sending lnd to another channel. The same payment can be visible in lncli listinvoices on the payee node and with lncli listpayments on the payer node. Then, I have to check my walletbalance [./lncli --netowrk=testnet walletbalance] but it is not credited.
If i check with ./lncli --netowrk=testnet channelbalance, It shows both local and remote balance.
Where we can check the updated channel balance in lnd?

The lncli walletbalance shows you the (L1) on-chain balance. The lncli channelbalance shows you the (L2) off-chain balance. If you send a payment, that only affects the off-chain balance. You can use lncli listchannels to see the available balances of each individual channel.
You can familiarize yourself with how the Lightning Network works with resources like lnbook. That should help you understand the difference between local and remote balance.
You can also sign up to our community Slack server for help with basic questions.
The issue tracker should be used for bugs and production issues.


After adding self-implemented Layer-Normalization, the backward time of gradient_penalty became large

My implementation of layer-normalization is:
class Layer_Norm(nn.Module):

    def __init__(self, dim):
        super(Layer_Norm, self).__init__()
        self.dim = dim
        self.g = Parameter(torch.zeros(1, dim))
        self.b = Parameter(torch.zeros(1, dim))
        self.init_weights()

    def forward(self, input):
        miu = torch.sum(input, 1).unsqueeze(1)/self.dim
        input_minus_miu = input - miu.expand_as(input)
        sigma = (torch.sum((input_minus_miu).pow(2), 1)/self.dim).sqrt().unsqueeze(1)
        input = input_minus_miu*self.g.expand(input.size())/sigma.expand_as(input) + self.b.expand(input.size())

        return input

    def init_weights(self):
        self.g.data.fill_(1)
        self.b.data.fill_(0)

After plugging in this before ReLU, the backward of gradient_penalty became large 0.1149s compared to the former value 0.0075s.
I compiled the source code from master branch, commit deb0aef30cdaa78f9840bfa4a919ad206e8e73a7 and also modified the ReLU source code before compiling according to your instruction.
I am wondering if it is because the my implementation of layer-normalization contains something not suitable for double backward?

Do you confirm that the value is wrong? Maybe you can refer to the gradgradcheck to test your double backward instead of just using the value. The gradgradcheck will be soon added in PR https://github.com/pytorch/pytorch/pull/1643. You can write a gradgradcheck similar to the PR.

@caogang The result seems reasonable but the time seems a bit too long. I'll test using gradgradcheck later. Thanks for your reply.

Oh, your problem is the bigger time cost after pluggin your Layer_Norm module? From my intuition, it may because of the expand or expand_as. Maybe you can test it.

@caogang what is the other way to complete similar behavior here as expand and expand_as. Broadcasting mechanism seems to still have some problems. https://github.com/pytorch/pytorch/issues/1787. And repeat costs more about 3s.

Yeah, maybe it is the best methods using expand in current branch. Just waiting for the broadcasting feature to be merged, or you can contrubute to the above PR. :)


Segmentation fault when passing tf.string tensor arguments in TFLite

1. System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 20.04
TensorFlow installation (pip package or built from source): pip
TensorFlow library (version, if pip package or github SHA, if built from source): at least TF 2.7 and tf-nightly are affected

2. Code
import tensorflow as tf

class TestModel(tf.keras.models.Model):

  def __init__(self, **kwargs):
    super().__init__(**kwargs)
    self._hash = tf.lookup.StaticHashTable(
        tf.lookup.KeyValueTensorInitializer(
            tf.constant(['testing', 'this', 'thing']),
            tf.constant([1, 2, 3])),
        default_value=-1)

  @tf.function
  def test(self, word):
    return self._hash.lookup(word)

test_model = TestModel()
signatures = [test_model.test.get_concrete_function(tf.TensorSpec([None], tf.string))]

converter = tf.lite.TFLiteConverter.from_concrete_functions(signatures, test_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS,
  tf.lite.OpsSet.SELECT_TF_OPS
]

tflite_model = converter.convert()
interpreter = tf.lite.Interpreter(model_content=tflite_model)

# Causes segmentation fault. Running test_model.test directly works fine.
result = interpreter.get_signature_runner()(word=tf.constant(['testing', 'that', 'thing']))

3. Failure after conversion
Conversion raises the following warning but completes.
2021-11-18 22:35:48.110926: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1880] Graph contains the following resource op(s), that use(s) resource type. Currently, the resource type is not natively supported in TFLite. Please consider not using the resource type if there are issues with either TFLite converter or TFLite runtime:
Resource ops: HashTableV2, LookupTableFindV2, LookupTableImportV2
Details:
	tf.HashTableV2() -> (tensor<!tf_type.resource>) : {container = "", device = "", key_dtype = !tf_type.string, shared_name = "13", use_node_name_sharing = false, value_dtype = i32}
	tf.LookupTableFindV2(tensor<!tf_type.resource>, tensor<?x!tf_type.string>, tensor<i32>) -> (tensor<*xi32>) : {device = ""}
	tf.LookupTableImportV2(tensor<!tf_type.resource>, tensor<3x!tf_type.string>, tensor<3xi32>) -> () : {device = ""}

However, trying to run the signature fails with a segmentation fault.
I thought that static hash table lookups were supported in TFLite since many versions ago. Any possible temporary workarounds until this is fixed would be very much appreciated.

Hi @sanatmpa1! Could you look into this issue. It is replicating in TF 2.5,2.6 and nightly . Attaching Gist for Reference.

@leandro-gracia-gil,
I tried reproducing the issue, where I can see that session getting crashed after running the code.But I don't see the warning you mentioned above. Here's the gist and can you clarify on the same? Thanks!

@sanatmpa1
I'm happy to try providing more info, but I'm not sure what you need. I can confirm you that for me running the exact code I pasted above in TF 2.7 does show that warning among other outputs, and also the keras warning present in the gist. Could it be that the gist is simply less verbose?
Here's a more detailed output:  (I've omitted the initial lines including hostnames and available CUDA devices and versions)
WARNING:tensorflow:Skipping full serialization of Keras layer <__main__.TestModel object at 0x7f4fa1c49790>, because it is not built.
2021-11-25 21:17:45.790533: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2021-11-25 21:17:46.010478: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:363] Ignored output_format.
2021-11-25 21:17:46.010513: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:366] Ignored drop_control_dependency.
2021-11-25 21:17:46.011638: I tensorflow/cc/saved_model/reader.cc:43] Reading SavedModel from: /tmp/tmp4h39x40o
2021-11-25 21:17:46.012182: I tensorflow/cc/saved_model/reader.cc:107] Reading meta graph with tags { serve }
2021-11-25 21:17:46.012213: I tensorflow/cc/saved_model/reader.cc:148] Reading SavedModel debug info (if present) from: /tmp/tmp4h39x40o
2021-11-25 21:17:46.015451: I tensorflow/cc/saved_model/loader.cc:210] Restoring SavedModel bundle.
2021-11-25 21:17:46.026026: I tensorflow/cc/saved_model/loader.cc:194] Running initialization op on SavedModel bundle at path: /tmp/tmp4h39x40o
2021-11-25 21:17:46.042800: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 31164 microseconds.
2021-11-25 21:17:46.053337: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:237] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2021-11-25 21:17:46.089494: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1880] Graph contains the following resource op(s), that use(s) resource type. Currently, the resource type is not natively supported in TFLite. Please consider not using the resource type if there are issues with either TFLite converter or TFLite runtime:
Resource ops: HashTableV2, LookupTableFindV2, LookupTableImportV2
Details:
        tf.HashTableV2() -> (tensor<!tf_type.resource>) : {container = "", device = "", key_dtype = !tf_type.string, shared_name = "13", use_node_name_sharing = false, value_dtype = i32}
        tf.LookupTableFindV2(tensor<!tf_type.resource>, tensor<?x!tf_type.string>, tensor<i32>) -> (tensor<*xi32>) : {device = ""}
        tf.LookupTableImportV2(tensor<!tf_type.resource>, tensor<3x!tf_type.string>, tensor<3xi32>) -> () : {device = ""}
2021-11-25 21:17:46.089591: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1891] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):
Flex ops: FlexHashTableV2, FlexLookupTableFindV2, FlexLookupTableImportV2
Details:
        tf.HashTableV2() -> (tensor<!tf_type.resource>) : {container = "", device = "", key_dtype = !tf_type.string, shared_name = "13", use_node_name_sharing = false, value_dtype = i32}
        tf.LookupTableFindV2(tensor<!tf_type.resource>, tensor<?x!tf_type.string>, tensor<i32>) -> (tensor<*xi32>) : {device = ""}
        tf.LookupTableImportV2(tensor<!tf_type.resource>, tensor<3x!tf_type.string>, tensor<3xi32>) -> () : {device = ""}
See instructions: https://www.tensorflow.org/lite/guide/ops_select
2021-11-25 21:17:46.089635: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1962] Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs

Estimated count of arithmetic ops: 0  ops, equivalently 0  MACs
INFO: Created TensorFlow Lite delegate for select TF ops.
INFO: TfLiteFlexDelegate delegate: 2 nodes delegated out of 3 nodes with 1 partitions.

Segmentation fault (core dumped)

Important update: this does not seem to be specifically a StaticHashTable problem, but rather a problem of passing string tensor arguments to exported functions in TFLite!
This version of the function causes a segmentation fault just the same:
  @tf.function
  def test(self, word):
    return word

The crash does not happen if changing the argument type in the signature from string to something else like tf.int32.

Can confirm that the problem seems solved in TF 2.11. Closing the issue. Thanks!


Part of the screen is missing when you set android_disable_fullscreen to true, in recent app overview.

Like tihs.

Go to "android/app/main/res/values/styles.xml" file on your project and change the line
<item name="android:windowFullscreen">true</item>
as
<item name="android:windowFullscreen">false</item>

Should be resolved by 48981828e831768c4ad89a35475e8f9f24e0aca5.

Should be resolved by 48981828e831768c4ad89a35475e8f9f24e0aca5.


[WIP] Add dropna and sort, ascending to sort_values

Blocked on https://github.com/pandas-dev/pandas/pull/30584, but I wanted to capture what I think this would look  like since that PR seems close.

[ ] Tests added / passed
[x] Passes black dask / flake8 dask
[x] Closes #5306

Thanks for this.
Is the test failure at https://travis-ci.org/dask/dask/jobs/648525814#L1265 related to the changes here?
Looking forward, will we need some compatibility code to avoid passing through these kwargs to older versions of pandas not expecting it?

Hmm I think that test failure is because I am setting a default for sort. I'll change that default to None.

will we need some compatibility code to avoid passing through these kwargs to older versions of pandas not expecting it?

Yes I'll add a version check.

Thanks, this looks nice.
Let's wait a bit to see how https://github.com/pandas-dev/pandas/pull/30584 goes.

Yep agree. Not sure if there is typical way of making that clear.

Just to note that the upstream work is in pandas master now. So presumably if we pin the version above latest we could merge this? Not sure if that is a good idea or not... :)

I think this is finally done @TomAugspurger. Thanks for your review!

Thanks for sticking with this!


Fix RCTBridgeModule import to work with newer react-native version

The current library breaks on newer RN version.
This patch fixes it by checking what kind of library it has to import, keeping backward-compatibility.

@farmisen  I think this PR should be merged. I just upgraded my RN project to 0.48.3 and this fixed my failing build cc @lexor90


mgmt: mcumgr: grp: img_mgmt: Fix using signed values

Fixes wrongly using signed values for slot and image number when
listing images.

Just to clarify since I'm unfamiliar with cbor, is there any chances that this causes incompatibilities with whatever is on the other side of this api? (mcumgr tools I suppose?)

Just to clarify since I'm unfamiliar with cbor, is there any chances that this causes incompatibilities with whatever is on the other side of this api? (mcumgr tools I suppose?)

I gave it a try with the go mcumgr client and nordic device management phone app and both worked, I guess if it causes an issue or not would be down to the libraries being used e.g. I don't think a typeless language like python is going to care but some super strict language might.

Though I will say mcuboot's serial recovery has the correct types which this PR changes mcumgr to and I've not heard of any incompatibilities with mcuboot's serial recovery

Though I will say mcuboot's serial recovery has the correct types which this PR changes mcumgr to and I've not heard of any incompatibilities with mcuboot's serial recovery

Fair, looks like they've always been int in the go client too: https://github.com/apache/mynewt-newtmgr/blob/master/nmxact/nmp/image.go#L91-L92


Transcribe functionality for Spanish Language

Hello,
I currently have a AWS Connect application that supports EN-US and ES-419/ES-US , In some cases I can have voicemails that are in Spanish language.
Currently all the transcripts I am seeing are for English, if there is a Spanish voicemail then the transcripts are empty. So is there a way to add functionality for Spanish transcripts?
Thanks,
Monika

I was able to resolve this

I was able to resolve this

Good morning,
I am interested in how you were able to solve it for Spanish.
Best regards.

I was able to resolve this

Good morning,
I am interested in how you were able to solve it for Spanish. Best regards.

Add a attribute languageCode with Spanish language code
Same is mentioned in the Appendix C: of the implementation guide.


Search bar not working

Whenever I type a keyword it wont show a preview like it should and it infinitely loads when I hit enter.
I am on Ubuntu 16.04 and using apache2

Have you made sure you've done both of these:

You've archived galleries correctly
You've re-indexed your Sphinx recently

I'm going to need a ton more info than this.

Turns out it was just my sphinx install that was the problem. I followed the tutorial on a fresh debian server instead of ubuntu and everything works flawless. thank you


Issue connecting to surveymonkey connector

Describe the Bug with repro steps
After selectin required trigger and click sign a pop-up window is getting opened with survey monkey login window but in the original window we end up getting error "browser closed"
What type of Logic App Is this happening in?
Standard (Portal)
Are you using new designer or old designer
New Designer
Did you refer to the TSG before filing this issue? https://aka.ms/lauxtsg
Yes
Workflow JSON
No response
Screenshots or Videos
No response
Browser
Tried with edge, firefox and chrome
Additional context
No response

It like on the SurveyMonkey connector specifically, our popup window is showing as closed for a second during initialization when it's still open, which closes our login flow and you have to restart.  Not sure why but I'm still investigating.

We released a mitigation for this issue and you should be able to create SurveyMonkey connections again.
The issue is still present, but until the root issue is fixed this will unblock customers.


On refreshing any page, it is redirecting to login.

Route or role guard might be causing this issue


[sourcekitd] Add support for transferring raw data

We will need this to transfer a binary encoded syntax tree in the future

I think @akyrtzi or @benlangmuir should review this.

@swift-ci Please smoke test

@swift-ci Please smoke test


Fix spelling and grammar

Hi @jbampton !
Thanks for your contribution 👍
I'm not English speaker. We will be grad if you would make more better English.


[Bug]: /bin/sh: 1: next: not found

Steps to reproduce
See repo https://github.com/the5starz/next-sample
create new nextjs 12 application in a front folder (sample monorepo) using
npx create-next-app@latest --ts front
go to netlifty and create new site from github
Setup

Result

A link to a reproduction repository
https://github.com/the5starz/next-sample
Plugin version
<EMAIL_ADDRESS>More information about your build

[ ] I am building using the CLI
[X] I am building using file-based configuration (netlify.toml)

What OS are you using?
No response
Your netlify.toml file
[build]
command = "yarn build:front"
publish = "front/.next"
[[plugins]]
package = "@netlify/plugin-nextjs"
Relevant log output (or link to your logs)
1:23:32 PM: $ yarn build:front
1:23:32 PM: yarn run v1.22.10
1:23:33 PM: $ cd front && yarn build
1:23:33 PM: $ next build
1:23:33 PM: /bin/sh: 1: next: not found
1:23:33 PM: error Command failed with exit code 127.

Hi. This implies that next hasn't been installed. What sort of monorepo is it, and where is the package.json that includes next? If it's all installed in front, could you set the baseDir to front and run the build command in there instead?

I created a clean sample that demonstrates the problem
https://github.com/the5starz/next-sample
I cannot set baseDir to front as I want to use other packages from the same
monorepo in front (say common)
based on
https://github.com/netlify/netlify-plugin-nextjs/blob/v3/docs/monorepos.md
and yarn monorepo I should not set baseDir
Muly
On Mon, Dec 13, 2021 at 1:46 PM Matt Kane @.***> wrote:

Hi. This implies that next hasn't been installed. What sort of monorepo
is it, and where is the package.json that includes next? If it's all
installed in front, could you set the baseDir to front and run the build
command in there instead?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/netlify/netlify-plugin-nextjs/issues/976#issuecomment-992382232,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AART6EYWVZWKO4E6M6JHB73UQXMJRANCNFSM5J54E4LA
.

Ah, the issue there is that your yarn workspace is setup incorrectly. Change the root package.json workspaces key to this:
  "workspaces": {
    "packages": [
      "front"
    ]
  },

...and add a version number to the child package.json. You can also change the build command to yarn workspace front build instead of including the cd front part

Yes, that was the problem, so solved now, thanks


Add icon to exe file in windows export

add version_info and icon sections in "export to windows platform".
add version_info and icon to godot exe file (editor & template exe).
fix an problem in image class.
change all default icons to android export icon (a little more rounded).
create an python script for convert file to cpp byte array for use in
'splash.h'.
export to windows:

icon:

Wow, amazing! Is this portable? Can I export from linux and it will add the
Icon too?
Also I believe the right way to add the library is in /tools instead of
drivers/
So when you compile without tools it's not included.

Do we know the authors of this "pe_bliss" library? can we get them to
give us the library with a more standard license, like MIT, BSD, etc?
the statement "Free to use, modify, and distribute." on their readme
might not be enough.
On 24 September 2015 at 00:53, Juan Linietsky<EMAIL_ADDRESS>wrote:

Yeah it's probably not very clear, for reference freetype (which is used
for the font importer) is in tools/freetype, so I guess simply move it over
to tools/pe_bliss
On Wed, Sep 23, 2015 at 7:51 PM, Masoud BH<EMAIL_ADDRESS>wrote:

but I thought all libraries there In /drivers .
i add this code in drivers SCsub :
if (env["tools"]=="yes"):
SConscript('pe_bliss/SCsub');
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-142753607.

—
Reply to this email directly or view it on GitHub.

i found pe_bliss library from http://code.google.com/p/portable-executable-library

change all default icons to android export icon (a little more rounded).

Will it be possible to disable this? Personally when I see a desktop game with a rounded icon I think "damn yet another mobile game ported to desktop, not interested". Please give us the possibility to keep our good old square icons on desktop :D

@masoudbh3 where does the README file come from? I don't see it on the google code repository

From https://code.google.com/p/portable-executable-library/ :
"Library is free to use in both commertial and non-commertial projects. You can also modify and redistribute it. If you are using it, please, do not forget to specify the name or other copyright of PE Bliss somewhere in the description of your project."
There is a blog post in russian about the project here : http://kaimi.ru/2012/09/portable-executable-library/

"Library is free to use in both commertial and non-commertial projects. You can also modify and redistribute it. If you are using it, please, do not forget to specify the name or other copyright of PE Bliss somewhere in the description of your project."

The rights granted seem to be good, but it's still a custom license, and mentioned only on a Google Code page that is already archived (and who knows how long Google will keep the archives up), so I would advise against using this. We probably should avoid "tainted" Godot's MIT code with external libraries using non-standard licenses.

Can't the author be contacted?
On Fri, Sep 25, 2015 at 12:23 PM, Rémi Verschelde<EMAIL_ADDRESS>wrote:

"Library is free to use in both commertial and non-commertial projects.
You can also modify and redistribute it. If you are using it, please, do
not forget to specify the name or other copyright of PE Bliss somewhere in
the description of your project."
The rights granted seem to be good, but it's still a custom license, and
mentioned only on a Google Code page that is already archived (and who
knows how long Google will keep the archives up), so I would advise against
using this. We probably should avoid "tainted" Godot's MIT code with
external libraries using non-standard licenses.
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-143251072.

Google says his mail is<EMAIL_ADDRESS>We could ask him to relicense it under MIT
On Fri, Sep 25, 2015 at 12:51 PM, Juan Linietsky<EMAIL_ADDRESS>wrote:

Can't the author be contacted?
On Fri, Sep 25, 2015 at 12:23 PM, Rémi Verschelde <
<EMAIL_ADDRESS>wrote:

"Library is free to use in both commertial and non-commertial projects.
You can also modify and redistribute it. If you are using it, please, do
not forget to specify the name or other copyright of PE Bliss somewhere in
the description of your project."
The rights granted seem to be good, but it's still a custom license, and
mentioned only on a Google Code page that is already archived (and who
knows how long Google will keep the archives up), so I would advise against
using this. We probably should avoid "tainted" Godot's MIT code with
external libraries using non-standard licenses.
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-143251072.

Can't the author be contacted?
We could ask him to relicense it under MIT

That's always a good idea IMO :)

I wrote to him last night..
On 25 September 2015 at 17:52, Juan Linietsky<EMAIL_ADDRESS>wrote:

Google says his mail is<EMAIL_ADDRESS>We could ask him to relicense it under MIT
On Fri, Sep 25, 2015 at 12:51 PM, Juan Linietsky<EMAIL_ADDRESS>wrote:

Can't the author be contacted?
On Fri, Sep 25, 2015 at 12:23 PM, Rémi Verschelde <
<EMAIL_ADDRESS>wrote:

"Library is free to use in both commertial and non-commertial projects.
You can also modify and redistribute it. If you are using it, please, do
not forget to specify the name or other copyright of PE Bliss somewhere
in
the description of your project."
The rights granted seem to be good, but it's still a custom license, and
mentioned only on a Google Code page that is already archived (and who
knows how long Google will keep the archives up), so I would advise
against
using this. We probably should avoid "tainted" Godot's MIT code with
external libraries using non-standard licenses.
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-143251072.

—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-143259436.

@punto- there is an russian readme.txt file in /pe_lib direcotory.
@akien-mga no problem you can change your game icon when export that.
@reduz i wrote to kaimi when post this commit, but not given answer.
other contact info for kaimi:
ICQ: 1003875
Jabber<EMAIL_ADDRESS>from readme file i found author is DX not kaimi.
he/she is another kaimi.ru blog author.
but does not exist any contact info except send PM to this :
http://forum.kaimi.ru/profile.php?id=3
https://translate.google.com/translate?hl=en&sl=ru&tl=en&u=https%3A%2F%2Fforum.kaimi.ru%2Fprofile.php%3Fid%3D3&sandbox=1

I think there's 2 questions here:

does the license text apply to the whole library? I think yes, that
is obviously the intention, but some more strict people might
interpret it as no, unless it's stated on every source file or a
COPYING or LICENSE file or something like that (but I don't know what
exactly). It also doesn't help that we find this "licence-y" text in 3
different places (the README, the header pe_base.h and the Google Code
page), and on each the text is slightly different.
if we assume that 1) is good, the second question is, is the
license compatible with MIT? I think yes, the intention seems to be
there, but given my experience[1] I agree with Rémi that it might not
be good enough.

This sucks because we really appreciate this work, and we've been
trying to get these features for a while, but ideally we'd have to get
the author of the library to give us a standard license for it.
[1] I used to maintain the tolua++ library, which was a fork of tolua,
which had a similar custom license, and we used to get requests all
the time from medium and big game dev studios to give them a standard
license or a custom license (ie a signed contract by me saying I won't
sue them :p ). Eventually we switched to MIT with permission from the
original author. The game development industry is kinda iffy about
custom licenses, but they're perfectly fine with MIT, etc.
On 25 September 2015 at 17:23, Rémi Verschelde<EMAIL_ADDRESS>wrote:

"Library is free to use in both commertial and non-commertial projects. You
can also modify and redistribute it. If you are using it, please, do not
forget to specify the name or other copyright of PE Bliss somewhere in the
description of your project."
The rights granted seem to be good, but it's still a custom license, and
mentioned only on a Google Code page that is already archived (and who knows
how long Google will keep the archives up), so I would advise against using
this. We probably should avoid "tainted" Godot's MIT code with external
libraries using non-standard licenses.
—
Reply to this email directly or view it on GitHub.

On 25 September 2015 at 12:23, Rémi Verschelde<EMAIL_ADDRESS>wrote:

"Library is free to use in both commertial and non-commertial projects. You
can also modify and redistribute it. If you are using it, please, do not
forget to specify the name or other copyright of PE Bliss somewhere in the
description of your project."
The rights granted seem to be good, but it's still a custom license, and
mentioned only on a Google Code page that is already archived (and who knows
how long Google will keep the archives up), so I would advise against using
this. We probably should avoid "tainted" Godot's MIT code with external
libraries using non-standard licenses.
—
Reply to this email directly or view it on GitHub.

Do we have any news about the state of this?

Here reply. But who asked i dont know.

Who has the answer?

I got permission from the author to add the MIT license text to all the
files in the library, so let's do that on this PR for the copy that we
would integrate on our repo. This is the text:
--
Copyright (c) 2015 dx, http://kaimi.ru
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
This needs to go on all the files that belong to the pe_bliss library.
Then we wait to hear from the legal people to see if this is acceptable.
On 19 October 2015 at 17:25, Masoud BH<EMAIL_ADDRESS>wrote:

Who has the answer?
—
Reply to this email directly or view it on GitHub
https://github.com/okamstudio/godot/pull/2518#issuecomment-149335957.

OK since we got permission from the author of pe_bliss, is this ready to be merged?

@punto- got permission and i add license text to library files.
maybe ready for merge now. i don't know.


incorrect arrangement of modules

If the problem is that everything is squashed in the middle, this is probably because the width of the whole bar is set to 10px. Removing the line "width": 10, should then solve the issue.

ty


[BitFinex] EOS lending only 5.50% of all available

I am using the default config and haven't set up any max or mins, and somehow EOS is the only currency were I see a limit of how much I can lend:

EOS1 EOS = 1.41 USD
Lent 91.0584 of 1655.4278 (5.50%) TotalLent 91.0584 of 91.0584 (100.00%) Lendable
0.00900% Day Avg.0.00042% Day Eff.0.15% Year0.16% Year Comp.

As the code is the same for all coins, I suspect this something coming from BitFinex. Do you have the bot running in a screen/tmux so you can see all the debug output?

Updated the code and restarted the program a few hours ago... now it shows like the rest of the coins so not sure what happened and can't reproduce it now. If it happens again I'll post the console output here.

Was there a recent transfer of coins in to the account? I noticed a lag in the bot updating when I transfered in some new coins but it eventually updated itself and now lends 100% of all my coins.

not in the case of EOS. I have only deposited ETH, BTC and Dash. All the EOS I bought on the platform and I was lending them already.

I am going to close it as it hasn't happened again.


Alternative name - mocha-watch is taken on npm

There's already a package named mocha-watch on npm. This package is released as @gustavnikolaj/mocha-watch and has already caused some unfortunate confusion (#11).
I'm very much open to suggestions for a new name :-)
Please make each suggestion in a separate comment so we can use reaction-emojis to vote!
/cc @alexjeffburke @papandreou

mocha-glare
mocha-watcher
watch-mocha

I'll leave the boring option here:
mocha-watch
Or, in other words: Reach out to the owner of the mocha-watch package and convince him to handover the package name :smile:
https://github.com/paulpflug/mocha-watch

mocha-big-brother
mocha-nsa 😜
mocha-goggles
mocha-blinded
Sorry I bad at naming 😂

Liking this thread already, it was bound to get interesting :) More thoughts:
mocha-peek
imocha (watching inodes or something?)
mocha-seek (another filesystem pun I guess but it does seek out your tests)
And of course, there is always the boring option. I also like mocha-tower btw, either it's towering over mocha or it's short for watchtower. PS I feel terrible at this too!

mocha-mon
actually this is the fav of mine so far - executing it reads "mocha monitor "

After intense discussions on the unexpected gitter channel we landed on imocha. The rename has been done and the old package is deprecated. I'll move the repo over shortly.

Thanks for your help guys! :rocket:


Skip the redirect on git clone

The Problem/Issue/Bug
I noticed this:

warning: redirecting to https://git.drupalcode.org/project/drupal.git/

So might as well fix it.

Summary by CodeRabbit

Chore: Updated the Drupal core repository cloning command in .gitpod/drupal/envs-prep/set-default-environment.sh. The updated command now includes ".git" at the end of the URL, ensuring that the full repository, including all branches and tags, is cloned. This change enhances the development environment setup process by providing complete access to the Drupal core repository.

Thank you @rfay
I added this update to https://github.com/shaal/DrupalPod/pull/123


[BUG] Unable to use Managed Identity when calling Set-PnPAdaptiveScopeProperty

Reporting an Issue or Missing Feature
I am trying to build an Azure Function using Managed Identity that will receive a set of Values for some Adaptive Scope Properties we are trying to set as part of our workflow, however when running the Script we are seeing the error Unable to connect to the SharePoint Online Admin Center at 'https://{tenant}-admin.sharepoint.com' to run this cmdlet
Expected behavior
Set Adapative Scope Property to MyNewValue = Value1
Actual behavior
Unable to connect to the SharePoint Online Admin Center at 'https://{tenant}-admin.sharepoint.com' to run this cmdlet. If this URL is incorrect for your tenant, you can pass in the correct Admin Center URL using Connect-PnPOnline -TenantAdminUrl. If you are using Privileged Identity Management (PIM) on your tenant, please ensure you have activated at least the SharePoint Administrator role and allowed some time for it to activate. Error message: The remote server returned an error: (401) Unauthorized.
Steps to reproduce behavior
Connect-PnPOnline -Url $siteAddress -ManagedIdentity
Set-PnPAdaptiveScopeProperty -Key MyNewValue -Value Value1
What is the version of the Cmdlet module you are running?
'PnP.PowerShell' = '2.4.0'
Which operating system/environment are you running PnP PowerShell on?

[x] Windows
[x] Azure Functions

In your connect-pnponline cmdlet, can you please use https://tenant-admin.sharepoint.com site URL and try ?

Hi @gautamdsheth,
Can you confirm how this will attach to an individual site when running the Set-PnPAdaptiveScopeProperty as this command doesn't have any way of setting which site we are editing?

Hi @gautamdsheth,
I have tried your mentioned option however as queried in my original response this tries to make the change against the Tenant Admin portal not the site I am trying to apply the item to. Do you have any further suggestions?

hey @hbland-cjc , can you please try with tomorrow's or later nightly builds and let us know if this still persists ? We have made quite a few changes related to managed identity auth, maybe that helps with your issue ?

Closing this due to lack of response . If the issue is still present with the latest builds, please do reopen .


Update flask version

Urgent change: Due to changes in the itsdangerous package, we need to use the most updated version of Flask.

#sign-off


Extractor: insert statements

Extractor: insert statements

ac642719053de7975bea4bff121b576619da162e

Verified that we're extracting the table name from insert statements.


Deepcopy output rather than copy

@gdalle could you see if this branch sorts your problem out? If so, I'll merge it, as I don't think it does any harm on my end.

It probably slows things down and makes them type unstable. Is it really a deep copy that you need? Why even copy?

Related:

https://github.com/JuliaLang/julia/issues/31995
https://github.com/JuliaLang/julia/issues/45143

Probably worth closing for now? Maybe in favor of a custom _copy that handles the nothing case as suggested in https://github.com/gdalle/DifferentiationInterface.jl/pull/126#issuecomment-2033173761?
I'm usually not a fan of these manual patches, but I think it's justified by the relevance of DifferentiationInterface for testing Tapir


chore: add missing package deps & upgrade to webpack-bundle-analyzer@3

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly. (PR: #)
[ ] I have added tests to cover my changes (if not applicable, please state why)
[x] All new and existing tests passed.

Good point @clarkdo!
Will add yarn.lock changes when I come home.


Does not search for deleted

HELPPP
Hello I have several tables in my system with deletedAt, but the typeorm does not search if it has a defined date, it will only bring the data if it is null. But there is a flag { withDeleted: true } which will return deleted and non-deleted records, how do I do it by nestjs-paginate

You cannot currently.
I would accept a PR adding withDeleted: true to PaginateConfig, and then extend the query builder with .withDeleted()
https://typeorm.io/select-query-builder#querying-deleted-rows

I would like to give you the Pull Request, could you give me permission to do so?

no need to give permissions. you can open pull requests on this repo from your fork. it's a default settings for public github repos ;)

Ok friend, I created the pull request. =)

closed with #219


condition method with weights

It would be nice to have a new method for condition that accepts weights from StatsBase; FrequencyWeights in particular would be very useful for things like minibatching or repeat observations.

Maybe one should just use a special set of weighted observations, i.e., condition(model, observations) where the weights are included in observations, to keep the API consistent.
Related: https://github.com/TuringLang/DynamicPPL.jl/issues/208

Maybe one should just use a special set of weighted observations, i.e., condition(model, observations) where the weights are included in observations, to keep the API consistent.
Related: TuringLang/DynamicPPL.jl#208

That could work, but wouldn't that require adding another struct or using something like a tuple for the observations argument? Not sure if that's any cleaner than adding a new argument.

Wouldn't this functionality be a generalization of the MiniBatchContext? With a vector instead of loglike_scalar?

Wouldn't this functionality be a generalization of the MiniBatchContext? With a vector instead of loglike_scalar, and a names field for the weighted variables?

It’s similar, although I’m not sure it would need a names field? The weights are on observations, rather than variables. Mostly I want something I can use with AbstractPPL instead of just DynamicPPL.

It’s similar, although I’m not sure it would need a names field? The weights are on observations, rather than variables.

Oh, I have no idea, that was just spontaneous generalization. It wouldn't hurt to implement the generalized case I guess.

Mostly I want something I can use with AbstractPPL instead of just DynamicPPL.

True, that's critical. I really have no better idea, but my hesitation against an extra argument is: it becomes part of the interface and thus priviledged. How far should be go with other special cases?
The alternative idea of using a special type sounds more generalizable to me. I have previously been thinking of allowing other extensible expressions the speculative "probability expressions" used in the design document, something like
condition(m, @P(do(X = ...)))

where the model is responsible of accepting such things or not. A weighting scheme, like
condition(m, @P(weighted(X = ..., weights)))

fits into that scheme.

Just to add my thoughts to the discussion:

Adding weights as a keyword is a no-no IMO, for the same reasons as @phipsgabler mentioned. Weighting the observations (or the computation of any random variable) should be a separate thing as it's more general than just the use-cases it would have in condition.
Should it not be an action on the Model, e.g. weight(model, syms...) or something along those lines? In DPPL we would just implement this in a similar manner as condition, i.e. using contexts under the hood but a nicer user-facing function.


Top level dashboard shows incorrect score, counts inactive products

Summary
The top level score in the main screen dashboard shows an incorrect
average when assets are added and then deleted.
Root Cause
The score itself continues to calculate using scores from assets that
have been deleted.
The Fix
In app/models/score.rb it pulls all scores, even ones from products that
are inactive in its summary scope.  The product model, as part of its
default scope, specifically only works with products that are active
(is_active).  In score.rb i did a join to this model to only consider
scores with products returned from the product model.

thanks @eallenvii! 👋


[ENH] Add model checkpoint for the rest of the deep learning classifiers

Reference Issues/PRs
Fix #321

What does this implement/fix? Explain your changes.
Adds the model checkpoint callback to save the best model for all of the deep learning classifiers

Any other comments?
I did not touch the tapenet classifier given that it was proposed for multivariate time series (to the best of my knowledge) and I did not want to make any changes that I shouldn't do, if you want I can add the changes, PS: am not a code author of tapenet, but for all the others I grantee the changes I am doing.

do these changes mean it always saves the model? If so, could we make it a parameter option? These saved models can be quite big, our first run of InceptionTime on the UCR greated 5+GB of saved models we didnt need.

@TonyBagnall no the model checkpoint saves the best model during training, at the end of training, i load it and delete it from file. I created an issue where i mention that we should have a parameter option whether or not to delete it after training.

do these changes mean it always saves the model? If so, could we make it a parameter option? These saved models can be quite big, our first run of InceptionTime on the UCR greated 5+GB of saved models we didnt need.

no the model checkpoint saves the best model during training, at the end of training, i load it and delete it from file. I created an issue where i mention that we should have a parameter option whether or not to delete it after training. Check #407 , once this PR is in will take care of that issue


ref: tests now use SyntheticNode

Quite a large PR. Bulk of it is refactoring of tests to use the SyntheticNode, along with some API improvements.
I will still be doing some refactoring to simplify some of it.

Closing, will open new simpler PR


just test commit to test release pipeline builds

Description
this is just test pr to test release pipeline builds which are happening twice
Related Issue

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Chore (non-breaking change that does not add functionality or fix an issue)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] I have run the pre-merge tests locally and they pass.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] If Gemfile.lock has changed, I have used --conservative to do it and included the full output in the Description above.
[ ] All new and existing tests passed.
[ ] All commits have been signed-off for the Developer Certificate of Origin.

test pr, closing this


Improved Documentation

CI \ CD
More Example Usages
Begin Remediation Guide

I agree with more applied example usage, particularly for first time usage.
running terraform init, followed by terraform plan, before running the terrascan seems to make a big difference and only found that through other bug reports.
The new 'remote module' support for TF seems to be in place, but isn't clear unless you happened to have check tickets/merge requests.
Remote module support may have an:
"invalid active developer path....missing xcrun" on mac, fix would be:
xcode-select --install
sudo xcode-select --reset

We've updated our README.md and our docs site since this issue was posted.
There are definitely more improvements we can make.
Anything in particular we should focus on? @dhartford, and anyone else!

We have improved our documentation since this issue was reported. Closing this one for now.
But if there is anything specific that is required from documentation perspective, please feel free to open a new issue.
Thank you!


[BUG] LTIME is not translated properly

We are off by two orders of magnitude

/cib

Branch 170-_BUG_LTIME_is_not_translated_properly created!


[Impeller] Improve performance of the glyph atlas pipeline

Right now the glyph atlas pipeline computes the UVs for glyph sampling in the fragment shader. This is probably much more expensive than needed, we should make this shader look more like the Skia shader.
I experimented with pulling the geometry computations into the text contents and it did significantly improve performance. There is probably more low hanging fruit here if we work at it a bit.
From the editorial page of wonderous with lots of text
Before

After

This was fixed by the above PR


inclusion / exclusion picker - performance issue

Changed so only 100 suggestions are displayed (max) when the user starts to type into the picker


EG-4 fix-ening

About the PR

Updated the EG-4 (and X-01 by proxy):

Fixed a spelling mistake
Buffed the auto-recharge on the EG-4
added locales for examining.

Why / Balance

The EG-4 now has a recharge on par with Captain's laser pistol, due to feedback on it not being fast enough.
Technical details

Media

Requirements

[ ] I have read and am following the Pull Request and Changelog Guidelines.
[ ] I have added media to this PR or it does not require an ingame showcase.

Changelog

:cl:

fix: the X-01 Multiphase and the EG-4 energy revolver now show their fire mode when examined.
tweak: Buffed the Auto-recharge rate of the EG-4 energy revolver.
fix: Fixed spelling mistake when examining the EG-4 energy revolver.

Summary by CodeRabbit

New Features

Introduced a new string entry for examining the fire mode of the energy gun, allowing for dynamic display of the current mode.

Bug Fixes

Corrected a typographical error in the description of the EG-4 energy revolver.
Updated the fire mode name from "disable" to "disabling" for clarity.

Improvements

Increased the auto-recharge rate of the BatterySelfRecharger component for better performance.

god damnit


Tesla @0x11DFE thank you for this suggestion.
Alexey @pukpuker спасибо за это.
@Gusenkov7 пожалуйста, будь милым. Мы все здесь хорошие люди.
Я позабочусь об этом.

The malware was updated, And i am being attacked by Russians. Oh i remember the malware has timezone based checks...
SHA256: bc981589e3b87e8ff9292af7e40662b33933fa896dc214677b2bbf1b3d4fabe4

They also have a mirror hosted at antihohol.cloud

The malware was updated, And i am being attacked by Russians. Oh i remember the malware has timezone based checks... SHA256: a4fbd5dfa976d2526590065d16e166ae2ba5b58a17bdcc8d1efbaca35ae55cdd

How u can reverse throw dnspy, if file c++ with vmp xD

The malware was updated, And i am being attacked by Russians. Oh i remember the malware has timezone based checks... SHA256: a4fbd5dfa976d2526590065d16e166ae2ba5b58a17bdcc8d1efbaca35ae55cdd

How u can reverse throw dnspy, if file c++ with vmp xD

It is packed and no this malware is made in C#.

Thank you to everyone for their input on this.
I've decided to list domain en.xachinaproject.fun.
Closing.

They have a mirror running the website is still up!
https://antihohol.cloud/

http://asfamilystyle.ru/

http://asfamilystyle.ru/

рагулих, че по сайту? Вроде все норм

http://asfamilystyle.ru/

рагулих, че по сайту? Вроде все норм

Sure yes it only spreads malware...

@StevenBlack Please add the domain to the list.


jam is wrong for cells

At some point in writing #60, my implementation of jam for cells became wrong (I could have sworn it was correct at some point...)
For context, +jinx-gate takes in a gate and sample and sends it to Ares, which runs the interpreter and returns a jammed noun to Vere, which Vere then attempts to +cue. I've put some printfs from Ares in curly braces.
(some 1)
[0 1]

> (jam (some 1))
201

>  (jinx-gate:f some 1)
{ares}: result  [0 1]
{ares}: jammed result  9
dojo: hoon expression failed

So basically Ares thinks the jam of [0 1] is 9 instead of 201.

cue also has an issue that I haven't quite figured out what it is yet. If I try to cue (shax 0x0) then it crashes, but it works fine on some other things. Since both cue and jam use backrefs, and I never felt confident about them, its possible its an issue with backrefs that's affecting both.

Whatever the issue is with cue, it is separate. I'll make a different issue.


Minimize dockerfile

Pull Request Test Coverage Report for Build 559

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 52.424%

Totals

Change from base Build 558:
0.0%

Covered Lines:
638

Relevant Lines:
1217

💛  - Coveralls


Does Redisson support BlockingQueue ?

I just read the documentation, but in the documentation there is no mention of BlockingQueue.
Does redisson support this ? I'm wondering because redis supports natively blocking operation on queues. Am I missing something ?

Currently, reddison doesn't support this feature.

Hi @mrniko Accepting PR for this? I have an implementation of a RedissonBlockingQueue working.

@pdeschen yea, i'll review it

+1

Done


[WIP] Save allocations (?) by using symbols for JSON keys

The JSONs we parse have the same keys repeating in many places.
Currently JSON parsing returns string keys, which allocates a new string every time:
[18] pry(main)> a1, a2 = JSON.parse('[{"a": 1}, {"a": 2}]'); a1.keys.first.equal? a2.keys.first
=> false

This returns symbols instead, so each key is allocated at most once.

[ ] TODO: is this an incompatible API change in any way?
I'm a bit surprised that all tests passed :-).  Manageiq tests pass too against this.  The reason is that RecursiveOpenStruct implements "indifferent access" — x.foo, x[:foo], x["foo"] all work, whether you constructed with symbol or string keys:
https://github.com/aetherknight/recursive-open-struct/blob/v1.0.4/lib/recursive_open_struct.rb#L131-L135

[ ] TODO: verify that this produces "mortal" symbols that can be GC'd. That's only possible since MRI 2.2.

Nice theory, but needs evidence of win
~I see total allocations slightly actually go up :-(~ Fluctuates randomly from run to run.
Measuring on kubeclient test suite by appending this to test/test_helper.rb:
at_exit do
  # happens shortly before Minitest.run
  GC.start
  $gc_stat_before_tests = GC.stat
end
Minitest.after_run do
  puts '>'*80
  printf "%40s\t%9s\t%9s\t%9s\t%s\n", 'GC.stat key', 'before', 'after', 'diff', 'ratio'
  GC.stat.each do |k, after_v|
    GC.start
    before_v = $gc_stat_before_tests[k]
    printf "%40s\t%9d\t%9d\t%9d\t%.3f\n", "#{k}:", before_v, after_v, after_v - before_v, after_v / before_v.to_f
  end
  puts '<'*80
end

I see on master 738077 total_allocated_objects:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
                             GC.stat key	   before	    after	     diff	ratio
                                  count:	       43	       61	       18	1.419
                   heap_allocated_pages:	      413	      413	        0	1.000
                     heap_sorted_length:	      413	      413	        0	1.000
                 heap_allocatable_pages:	        0	        0	        0	NaN
                   heap_available_slots:	   168348	   168348	        0	1.000
                        heap_live_slots:	   122390	   168306	    45916	1.375
                        heap_free_slots:	    45958	       42	   -45916	0.001
                       heap_final_slots:	        0	        0	        0	NaN
                      heap_marked_slots:	   122307	   131317	     9010	1.074
                        heap_eden_pages:	      413	      413	        0	1.000
                        heap_tomb_pages:	        0	        0	        0	NaN
                  total_allocated_pages:	      413	      413	        0	1.000
                      total_freed_pages:	        0	        0	        0	NaN
                total_allocated_objects:	   817165	  1555242	   738077	1.903
                    total_freed_objects:	   694775	  1386936	   692161	1.996
                  malloc_increase_bytes:	     1176	    41976	    40800	35.694
            malloc_increase_bytes_limit:	 16777216	 16777216	        0	1.000
                         minor_gc_count:	       36	       53	       17	1.472
                         major_gc_count:	        7	        8	        1	1.143
      remembered_wb_unprotected_objects:	      505	      580	       75	1.149
remembered_wb_unprotected_objects_limit:	     1010	     1150	      140	1.139
                            old_objects:	   120807	   130029	     9222	1.076
                      old_objects_limit:	   241614	   258582	    16968	1.070
               oldmalloc_increase_bytes:	  4124968	    64056	 -4060912	0.016
         oldmalloc_increase_bytes_limit:	 16777216	 16777216	        0	1.000
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

and with this change 775103 total_allocated_objects:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
                             GC.stat key	   before	    after	     diff	ratio
                                  count:	       43	       62	       19	1.442
                   heap_allocated_pages:	      413	      413	        0	1.000
                     heap_sorted_length:	      413	      413	        0	1.000
                 heap_allocatable_pages:	        0	        0	        0	NaN
                   heap_available_slots:	   168333	   168333	        0	1.000
                        heap_live_slots:	   122389	   168201	    45812	1.374
                        heap_free_slots:	    45944	      132	   -45812	0.003
                       heap_final_slots:	        0	        0	        0	NaN
                      heap_marked_slots:	   122306	   128777	     6471	1.053
                        heap_eden_pages:	      413	      413	        0	1.000
                        heap_tomb_pages:	        0	        0	        0	NaN
                  total_allocated_pages:	      413	      413	        0	1.000
                      total_freed_pages:	        0	        0	        0	NaN
                total_allocated_objects:	   817162	  1592265	   775103	1.949
                    total_freed_objects:	   694773	  1424064	   729291	2.050
                  malloc_increase_bytes:	     1176	    24840	    23664	21.122
            malloc_increase_bytes_limit:	 16777216	 16777216	        0	1.000
                         minor_gc_count:	       36	       54	       18	1.500
                         major_gc_count:	        7	        8	        1	1.143
      remembered_wb_unprotected_objects:	      505	      562	       57	1.113
remembered_wb_unprotected_objects_limit:	     1010	     1100	       90	1.089
                            old_objects:	   120807	   127895	     7088	1.059
                      old_objects_limit:	   241614	   252694	    11080	1.046
               oldmalloc_increase_bytes:	  4125736	   158312	 -3967424	0.038
         oldmalloc_increase_bytes_limit:	 16777216	 16777216	        0	1.000
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

I suspect RecursiveOpenStruct's _get_key_from_table_ trying name.to_s before name.to_sym (see link above) means we still get a string allocated for each symbol.  (:a.to_s returns new string every time.  Darn you Ruby and your mutable strings!)  I'll try swapping these 2 ROS lines.
I think this PR is orthogonal to other ROS problems listed in #251.
~OTOH malloc_increase_bytes is down from 40800 to 23664 :-)~ Fluctuates randomly from run to run.
I'm not sure what I should look at.  I actually want peak memory usage on big JSON, don't think test suite does that.
@ilackarms can you help measuring this? (low priority, probably low payoff compared to #250.)
With your #250 involving a ruby JSON parser, I suppose we'll get lots of unavoidable string allocations during parsing,  That's fine, but if you can make it free the strings and use symbols in the final data structure, could help with total RAM.

@cben the payoff may in fact be better from this change because the memory allocation for parsing gets garbage collected once it's finished. the ROS objects remain in memory; if we can reduce their footprint that might produce a better memory reduction

Closing for now until I revisit it and get more data (if anybody else wants, I'm glad to help).
I still believe this could be a win, but there is a larger space to explore.

Is this backward-compatible?  Seems it is currently, thanks to RecursiveOpenStruct allowing indifferent [:symbol] / ["string"] access, even after .to_h.
Ensure this produces "mortal" symbols to prevent memory leak DOS — MRI version dependent.
There are a few RecursiveOpenStruct options I wanted to tweak to avoid deep-copying the hashes JSON.parse just created, but these seem to break the indifferent access :-(
JSON.parse can be told which array/hash classes to use.  Direct parsing to say OpenStruct may be more efficient than to hashes and then deep-copying, and then wrapping on demand...
Disabling/bypassing RecursiveOpenStruct is most promising.
This would need to be implemented separately with streaming parsing.

Hopefully we'll measure various directions in ManageIQ and come back with data...


Input background should be white instead of transparent

instead of

Fixed by https://github.com/containous/faency/pull/27


build: include "cstdint" for the types uint8_t and uint16_t

This fixes a build problem on Fedora 38 with clang 16. (there are other build problems in external libraries)
Commit Message:
Additional Description:
Risk Level: Low
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:

/retest

/retest


An $or-operator query with an $and-operator fails to construct properly.

Describe the bug
If you query like:
const items = [{ prop1: "value1", prop2: "value2" }, { prop1: "value3", prop2: "value4" }];
const connection = await Database.getConnection();
return connection.em.getRepository(Entity).find({
       $or: items.map(i => ({
           $and: [{ prop1: i.prop1, prop2: i.prop2}]
      }))
});

... it will construct a query like:
select `e0`.* from `entity` as `e0` where (((`e0`.`prop1` = 'value1' and `e0`.`prop2` = 'value2')) and ((`e0`.`prop1` = 'value3' and `e0`.`prop2` = 'value4')))
... instead of the expected:
select `e0`.* from `entity` as `e0` where (((`e0`.`prop1` = 'value1' and `e0`.`prop2` = 'value2')) or ((`e0`.`prop1` = 'value3' and `e0`.`prop2` = 'value4')))
Stack trace
No error, so no stack trace.
To Reproduce
Steps to reproduce the behavior:
See above.
Expected behavior
Create this query:
select `e0`.* from `entity` as `e0` where (((`e0`.`prop1` = 'value1' and `e0`.`prop2` = 'value2')) or ((`e0`.`prop1` = 'value3' and `e0`.`prop2` = 'value4')))
Additional context
It is related with the querybuilding using the incorrect synthax to append the group. I've created an additional repo where the query is constructed correctly, but I am not sure what effect the removal of that if has.
https://github.com/mikro-orm/mikro-orm/compare/dev...jsprw:patch-2
Versions

Dependency
Version

node
v12.9.1

typescript
v3.6.4

mikro-orm
v3.0.0-rc

your-driver
mysql2 v2.0.1

A bug indeed, this should be working as you expected, confirmed and replicated.
But you are overcomplicating it a bit, you can pass the items array directly and it will work as you want:
const items = [{ prop1: "value1", prop2: "value2" }, { prop1: "value3", prop2: "value4" }];
const connection = await Database.getConnection();
return connection.em.getRepository(Entity).find({
       $or: items
});

Multiple object keys are considered as ANDs, so prop1: { $and: [1,2] } is the same as prop1: [1,2].

Multiple object keys are considered as ANDs, so prop1: { $and: [1,2] } is the same as prop1: [1,2].

Thank you! It will make the code easier to read!

Fixed in 3.0.0-rc.1


"sudo pip3.6 install --upgrade" won't work

Hi,
I am trying to run the following command:
sudo pip3.6 install --upgrade git+https://github.com/Xilinx/PYNQ.git
But I got the following error:

Collecting git+https://github.com/Xilinx/PYNQ.git
Cloning https://github.com/Xilinx/PYNQ.git to /tmp/pip-5j6ud6cd-build
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/pynq.egg-info
writing pip-egg-info/pynq.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pynq.egg-info/top_level.txt
writing entry points to pip-egg-info/pynq.egg-info/entry_points.txt
writing dependency_links to pip-egg-info/pynq.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pynq.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'pip-egg-info/pynq.egg-info/SOURCES.txt'
writing manifest file 'pip-egg-info/pynq.egg-info/SOURCES.txt'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-5j6ud6cd-build/setup.py", line 243, in <module>
    subprocess.run(['systemctl' ,'restart', 'pl_server'])
AttributeError: 'module' object has no attribute 'run'
Restarting PL server

----------------------------------------

Am I missing something?
Best Regards,
Cong

Thanks. I think the problem might be the SD card image is not up to date. Only upgrading PYNQ package from image v1.5 probably won't work.


Re-using YATE ?

This is a pretty useful resource for teaching, thank you !
For the Turtle syntax coloring, automatic namespace addition, and autocompletion, Would you consider reusing YATE ?
https://perfectkb.github.io/yate/

For sure! It would be fantastic to incorporate. And also YASGUI itself would be great for SPARQL.
Bastián is currently finishing up his undergrad thesis but we hope to continue extending and improving the tool with other students (or indeed if there are volunteers).

I would have loved to be of some help here, however I am afraid I don't know much about vue.js to be efficient.
At least I can explain here what needs to be done on the html:

add the following script at the bottom of the HTML page:

<script src="https://perfectkb.github.io/yate/dist/yate.bundled.min.js"></script>
<script src="https://perfectkb.github.io/yate/doc/doc.min.js"></script>
<script type="text/javascript">
var yate = YATE(document.getElementById("the_id_of_the_textarea"));
</script>

use functions of the yate object to get or set the data in the textarea. The API is forked from YASQE: https://triply.cc/docs/yasgui-api#yasqe-api

That would help a lot!
I would say that this is a high priority in terms of possible short-term enhancements. I will try to find someone to work on this, but if not maybe I will try myself when I have the time. Might be fun to learn some Javascript. :)

It would be fantastic to incorporate YATE, although I developed the frontend on Vue.js I'm far from an expert, specially when it comes to webpack components.
The file where our text areas are defined is this one; I think this shouldn't be hard, but there's no way to know for sure until we give it a shot ! :D


Support for editable installs

Currently, we cannot convert a pip install -e .... to Pipfile's format.

Done!

Is there a way to use it with pipenv install?

@abdulhaq-e: It appears this feature has regressed.
See https://github.com/kennethreitz/pipenv/issues/313

working on master

What about installing the project you're working on as editable, like pip install --editable . would do?  Is this possible?

python setup.py develop appears to work for me?

@wkretzsch, would you run that as pipenv run python setup.py develop?

Right.  I run it from inside a pipenv shell
pipenv shell
# wait for virtualenv to load
python setup.py develop

I think the develop command is only supported by setuptools.
Here is the head of my setup.py
#!/usr/bin/env python
from setuptools import setup, find_packages

setup (
# ...

I have version 9.1.0 and  -e option is not there, it does not appear in pip install -h and it does not work. So @techalchemy I do not understand is it there or not ?

It is not an argument to the command line any more than the name of a package would be considered an argument, which is why it doesn’t appear in help. It is passed as a part of a requirement, not as an option to pipenv. So yes, as stated, it remains supported, please attempt to use it and file an issue if you find a problem with it.


List of appropriate metrics to test

Starting pitch:

BLEU (baseline)
ROUGE-S / any other skip-gram or conventional metric of interest that may improve on BLEU
BLASER 2.0
CometKiwi (reference-based variant) / other model-based/text translation-oriented metric of choice (e.g. critical error detection?)

and

the above with different variants of pre-processing etc.

It doesn't include BLASER as it's from 2023 but this paper also has a nice taxonomy of metrics which we could re-use.

Also sentence similarity metrics e.g. these models: https://huggingface.co/models?pipeline_tag=sentence-similarity&sort=trending
SPICE is using one of these as part of: https://arxiv.org/pdf/2405.13845


bugs when running the training code

2023-09-11 10:21:49,337	ERROR tune_controller.py:911 -- Trial task failed for trial PPO_meltingpot_fcb07_00000
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ray/air/execution/_internal/event_manager.py", line 110, in resolve_future
result = ray.get(future)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/auto_init_hook.py", line 24, in auto_init_wrapper
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/ray/_private/worker.py", line 2495, in get
raise value
File "python/ray/_raylet.pyx", line 1787, in ray._raylet.task_execution_handler
File "python/ray/_raylet.pyx", line 1684, in ray._raylet.execute_task_with_cancellation_handler
File "python/ray/_raylet.pyx", line 1366, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 1367, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 1583, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 864, in ray._raylet.store_task_errors
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::PPO.init() (pid=13902, ip=<IP_ADDRESS>, actor_id=7e027fca141b6dc2cdd8f15501000000, repr=PPO)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/algorithm.py", line 517, in init
super().init(
File "/usr/local/lib/python3.10/dist-packages/ray/tune/trainable/trainable.py", line 169, in init
self.setup(copy.deepcopy(self.config))
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/algorithm.py", line 639, in setup
self.workers = WorkerSet(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/worker_set.py", line 179, in init
raise e.args[0].args[2]
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 525, in init
self._update_policy_map(policy_dict=self.policy_dict)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 1727, in _update_policy_map
self._build_policy_map(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 1838, in _build_policy_map
new_policy = create_policy_for_framework(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/policy.py", line 142, in create_policy_for_framework
return policy_class(observation_space, action_space, merged_config)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/algorithms/ppo/ppo_torch_policy.py", line 64, in init
self._initialize_loss_from_dummy_batch()
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/policy/policy.py", line 1418, in _initialize_loss_from_dummy_batch
actions, state_outs, extra_outs = self.compute_actions_from_input_dict(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/policy/torch_policy_v2.py", line 571, in compute_actions_from_input_dict
return self._compute_action_helper(
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/utils/threading.py", line 24, in wrapper
return func(self, *a, **k)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/policy/torch_policy_v2.py", line 1291, in _compute_action_helper
dist_inputs, state_out = self.model(input_dict, state_batches, seq_lens)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/models/modelv2.py", line 259, in call
res = self.forward(restored, state or [], seq_lens)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/models/torch/recurrent_net.py", line 259, in forward
return super().forward(input_dict, state, seq_lens)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/models/torch/recurrent_net.py", line 98, in forward
output, new_state = self.forward_rnn(inputs, state, seq_lens)
File "/usr/local/lib/python3.10/dist-packages/ray/rllib/models/torch/recurrent_net.py", line 274, in forward_rnn
self._features, [h, c] = self.lstm(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/rnn.py", line 810, in forward
self.check_forward_args(input, hx, batch_sizes)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/rnn.py", line 730, in check_forward_args
self.check_input(input, batch_sizes)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/rnn.py", line 218, in check_input
raise RuntimeError(
RuntimeError: input.size(-1) must be equal to input_size. Expected 148, got 24

Does this caused by the version of the dependencies

Not sure but this might help in the meanwhile, https://github.com/rstrivedi/Melting-Pot-Contest-2023/issues/5#issuecomment-1705718568

When I install the packages it gives me the following error (I run this in colab):
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
numba 0.56.4 requires numpy<1.24,>=1.18, but you have numpy 1.24.3 which is incompatible.
tensorflow-datasets 4.9.2 requires protobuf>=3.20, but you have protobuf 3.19.6 which is incompatible.
tensorflow-metadata 1.14.0 requires protobuf<4.21,>=3.20.3, but you have protobuf 3.19.6 which is incompatible.

These problems are not relevant to the installation of the provided dependencies. They seem to be stemming from other things installed in yoru environment. Could you try to use a virtual env instead of using system python or try to create a fresh virtual environment and check if it works?


Remove description from Module & Add tests

Closing #96

Codecov Report

Merging #107 into master will increase coverage by 1.37%.
The diff coverage is 47.05%.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5860351...a42d89b. Read the comment docs.