Error in training encoder decoder in Parametric UMAP with 1D vector data.

Hello,
I am trying to fit transform data with Parametric UMAP using Keras architecture for the encoder and decoder as described in docs. The data is of the shape (74096,1) and I have 100 samples for testing. I want to embed to 2D. My architecture is as follows
Setup
    dims = (74096,1)
    latent_dim = 2

Encoder
    encoder_inputs = keras.Input(shape=dims)
    xe = layers.Conv1D(32, 3, activation="relu", strides=2, padding="same",name='conv1')(encoder_inputs)
    xe = layers.Conv1D(64, 3, activation="relu", strides=2, padding="same",name='conv2')(xe)
    xe = layers.Flatten(name='flatten')(xe)
    xe = layers.Dense(16, activation="relu")(xe)
    z_mean = layers.Dense(latent_dim, name="z_mean")(xe)
    z_log_var = layers.Dense(latent_dim, name="z_log_var")(xe)
    z = Sampling()([z_mean, z_log_var])
    encoder = keras.Model(encoder_inputs, [z_mean, z_log_var, z], name="encoder")

 Layer (type)                   Output Shape         Param #     Connected to
==================================================================================================
 input_94 (InputLayer)          [(None, 74093, 1)]   0           []

 conv1 (Conv1D)                 (None, 37047, 32)    128         ['input_94[0][0]']

 conv2 (Conv1D)                 (None, 18524, 64)    6208        ['conv1[0][0]']

 flatten (Flatten)              (None, 1185536)      0           ['conv2[0][0]']

 dense_139 (Dense)              (None, 16)           18968592    ['flatten[0][0]']

 z_mean (Dense)                 (None, 2)            34          ['dense_139[0][0]']

 z_log_var (Dense)              (None, 2)            34          ['dense_139[0][0]']

 sampling_37 (Sampling)         (None, 2)            0           ['z_mean[0][0]',
                                                                  'z_log_var[0][0]']

Decoder
# This simply gets the mirror of the encoder set up
reshape_dims = (encoder.get_layer('conv2').output_shape[1], encoder.get_layer('conv2').output_shape[2])

latent_inputs = keras.Input(shape=(latent_dim))
xd = layers.Dense(16 , activation="relu")(latent_inputs)
xd = layers.Dense(encoder.get_layer('flatten').output_shape[1], activation="relu")(xd)
xd = layers.Reshape(reshape_dims)(xd)
xd = layers.Conv1DTranspose(64, 3, activation="relu", strides=2, padding="same")(xd)
xd = layers.Conv1DTranspose(32, 3, activation="relu", strides=2, padding="same")(xd)
decoder_outputs = layers.Conv1DTranspose(1, 3, activation="sigmoid", padding="same")(xd)
decoder = keras.Model(latent_inputs, decoder_outputs, name="decoder")

Model: "decoder"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 input_95 (InputLayer)       [(None, 2)]               0

 dense_140 (Dense)           (None, 16)                48

 dense_141 (Dense)           (None, 1185536)           20154112

 reshape_55 (Reshape)        (None, 18524, 64)         0

 conv1d_transpose_118 (Conv1  (None, 37048, 64)        12352
 DTranspose)

 conv1d_transpose_119 (Conv1  (None, 74096, 32)        6176
 DTranspose)

 conv1d_transpose_120 (Conv1  (None, 74096, 1)         97
 DTranspose)

=================================================================
Total params: 20,172,785
Trainable params: 20,172,785
Non-trainable params: 0

With this defined I set up my Parametric UMAP to fit to data called alignment_onehot (shape: (100, 74093)) using validation data val_onehot (shape: (10, 74093)).
    embedder = ParametricUMAP(
        encoder=encoder,
         decoder=decoder,
        dims=dims,
        parametric_reconstruction= True,
         reconstruction_validation=val_onehot,
        verbose=True,
        random_state=42
    )
embedding = embedder.fit_transform(alignment_onehot)

but when I try to fit the data I get the following Errors:
WARNING:tensorflow:Model was constructed with shape (None, 2) for input KerasTensor(<input_95 info>), but it was called on an input with incompatible shape (3, None, 2)
Input 0 of layer "decoder" is incompatible with the layer: expected shape=(None, 2), found shape=(3, 100, 2)
I'm not sure where the shape 3 comes from?? There are 100 embedded data points with latent dimension 2 but I'm confused on where how the 3 is included..

Wed Oct  5 13:14:13 2022 Construct fuzzy simplicial set
Wed Oct  5 13:14:14 2022 Finding Nearest Neighbors
Wed Oct  5 13:14:14 2022 Finished Nearest Neighbor Search
Wed Oct  5 13:14:14 2022 Construct embedding
WARNING:tensorflow:Model was constructed with shape (None, 2) for input KerasTensor(type_spec=TensorSpec(shape=(None, 2), dtype=tf.float32, name='input_95'), name='input_95', description="created by layer 'input_95'"), but it was called on an input with incompatible shape (3, None, 2).
Epoch 1/10

Error
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_119266/1148489687.py in <module>
      5
      6
----> 7 embedding = embedder.fit_transform(alignment_onehot)

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/parametric_umap.py in fit_transform(self, X, y, precomputed_distances)
    215             return super().fit_transform(precomputed_distances, y)
    216         else:
--> 217             return super().fit_transform(X, y)
    218
    219     def transform(self, X):

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/umap_.py in fit_transform(self, X, y)
   2770             Local radii of data points in the embedding (log-transformed).
   2771         """
-> 2772         self.fit(X, y)
   2773         if self.transform_mode == "embedding":
   2774             if self.output_dens:

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/parametric_umap.py in fit(self, X, y, precomputed_distances)
    200             return super().fit(precomputed_distances, y)
    201         else:
--> 202             return super().fit(X, y)
    203
    204     def fit_transform(self, X, y=None, precomputed_distances=None):

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/umap_.py in fit(self, X, y)
   2682
   2683         if self.transform_mode == "embedding":
-> 2684             self.embedding_, aux_data = self._fit_embed_data(
   2685                 self._raw_data[index],
   2686                 self.n_epochs,

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/parametric_umap.py in _fit_embed_data(self, X, n_epochs, init, random_state)
    460
    461         # create embedding
--> 462         history = self.parametric_model.fit(
    463             edge_dataset,
    464             epochs=self.loss_report_frequency * self.n_training_epochs,

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
     68             # To get the full stack trace, call:
     69             # `tf.debugging.disable_traceback_filtering()`
---> 70             raise e.with_traceback(filtered_tb) from None
     71         finally:
     72             del filtered_tb

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/engine/training.py in tf__train_function(iterator)
     13                 try:
     14                     do_return = True
---> 15                     retval_ = ag__.converted_call(ag__.ld(step_function), (ag__.ld(self), ag__.ld(iterator)), None, fscope)
     16                 except:
     17                     do_return = False

/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/parametric_umap.py in train_step(self, data)
   1133
   1134         with tf.GradientTape() as tape:
-> 1135             y_pred = self(x, training=True)  # Forward pass
   1136             # Compute the loss value
   1137             # (the loss function is configured in `compile()`)

ValueError: in user code:

    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/engine/training.py", line 1160, in train_function  *
        return step_function(self, iterator)
    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/engine/training.py", line 1146, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/engine/training.py", line 1135, in run_step  **
        outputs = model.train_step(data)
    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/umap/parametric_umap.py", line 1135, in train_step
        y_pred = self(x, training=True)  # Forward pass
    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
        raise e.with_traceback(filtered_tb) from None
    File "/data/cresswellclayec/conda/envs/DCA_ER/lib/python3.9/site-packages/keras/engine/input_spec.py", line 295, in assert_input_compatibility
        raise ValueError(

    ValueError: Exception encountered when calling layer "gradient_clipped_model_8" "                 f"(type GradientClippedModel).

    Input 0 of layer "decoder" is incompatible with the layer: expected shape=(None, 2), found shape=(3, 100, 2)

    Call arguments received by layer "gradient_clipped_model_8" "                 f"(type GradientClippedModel):
      • inputs=('tf.Tensor(shape=(100, 74093, 1), dtype=float32)', 'tf.Tensor(shape=(100, 74093, 1), dtype=float32)')
      • training=True
      • mask=None

I should say that I believe, maybe naively, that this is happening because Parametric umap is expecting 2D data (ie image data, (28, 28,1) ) and not the 1D data I'm using.. and that the embedded data is being re-cast weirdly inside umap before it is submitted to the decoder (hence the transition from the output (None, 2) --> (3, None, 2) )..

Hi Evan,
Can you try using the default encoder and decoder, rather than the convolutional networks?

Hello Tim,
Thanks for your quick response! It works with the default encoder and decoder.
However the resulting embedding isn't very good. I was hoping a convolutional network would solve the issue given the nature of the sequence data. For illustration here is an example embedding with the 100 sample data points (I ran the same analysis for 10000 data points with little difference in embedded grouping).

You could try an rnn encoder and decoder. You also will probably see more structure if you crop the outliers in your plot.

I’m going to mark this as closed since the original error is resolved but feel free to respond.

Looking through the network you defined, it looks like you took that architecture from a vae?

I did use vae as inspiration given that it has been effective in analyzing this type of data before (genome sequence data) and it seemed a natural fit with umap.
Just for my own understanding though: is the error I was having on the umap end? Is it just simply that parametric umap was not meant to handle 1D data? The example from the website used 2D convolutions.
WARNING:tensorflow:Model was constructed with shape (None, 2) for input KerasTensor(type_spec=TensorSpec(shape=(None, 2), dtype=tf.float32, name='input_2'), name='input_2', description="created by layer 'input_2'"), but it was called on an input with incompatible shape (3, None, 2).

Given the error it just seems that umap is passing something weird into the decoder.. I assumed that as long as my networks took the correct input shape and gave the correct output shape I should be fine regardless of the internals..

You also will probably see more structure if you crop the outliers in your plot.

Also those outliers persevere, when more data is included, very strange. I really want to get this architecture to work but I will try an rnn for the encoder and decoder. Thanks!

Hey, no, the issue is not on the UMAP end. If you can write an auto encoder in tensorflow with your dataset you should be able to use the same encoder here


Bulk edit for request Query parameters seems broken.

Describe the bug
Bulk edit for request Query parameters is failing (silently not responding).
To Reproduce
From the request Query tab, click 'Bulk Edit'. Nothing happens in the UI.
Expected behavior
Clicking 'Bulk Edit' would presumably enter the bulk edit editor (exactly like when clicking 'Bulk Edit' in the Header tab). Instead, nothing seems to happen and in the Insomnia DevTools, this error message is visible: "Uncaught TypeError: updateSettingsUseBulkParametersEditor is not a function".
Desktop (please complete the following information):

OS: Ubuntu 20.04
Installation Method: snap
App Version: 2020.3.3

same hare!
Environment:
Windows 10 PRO (build 1909)
Installation method: EXE file
App Version: 2020.3.3


[Issue] Declarative Setup - Helmchart installation with GitOps pattern and ArgoCD is not possible currently

To be able to use ArgoCD for deployment it is necessary for a helm-chart to be deployed declaratively. This is needed for our productive deployment as well later on.
In the case of btp-service-operator helm chart the mandatory parameters are confidential this hinders a declarative setup. In the past there were ways to circumvent this issue but these are not available anymore!

Workaround -> not working anymore
Using the --values parameter is also not going to work since these values are secret values and cant be stored for our declarative deployment in this way. (we use bitnami sealed secrets for this purposes)

The cleanest solution is to allow the operator to consume an already existing secret if the property is given!
In this case a setup similar to this example with some optional parameters fixes the helm chart installation:
# usual way (will not be changed)
helm upgrade --install <release-name> sap-btp-operator/sap-btp-operator \
    --create-namespace \
    --namespace=sap-btp-operator \
    --set manager.secret.clientid=<clientid> \
    --set manager.secret.clientsecret=<clientsecret> \
    --set manager.secret.url=<sm_url> \
    --set manager.secret.tokenurl=<url>

# optional way (actually IMHO recommended way):
helm upgrade --install <release-name> sap-btp-operator/sap-btp-operator \
    --create-namespace \
    --namespace=sap-btp-operator \
    --set manager.secret.create=< true (default) | false > \
    --set manager.secret.name=<default-secret-name>

This way users can point to an existing secret and allow argoCD to deploy the helmchart and the operator can still get all of its needed information since it would be in the same namespace anyways.
Supporting this feature would allow adoption of the btp-service-operator in a simpler and more future proven way there is demand for this and it is also a better design for k8s deployments if there are confidential parameters (this is just my honest opinion here :) ) .
https://github.com/SAP/sap-btp-service-operator/issues/115 https://github.com/SAP/sap-btp-service-operator/issues/87
Implementation on operator side is simple:
if manager.secret.create == true
  helm values are filled
  normal behaviour as before
else
  retrieve values from the secret that is provided in manager.secret.name
  continue normal behaviour as before

Best regards,
Rudolf

I was looking through the code and saw he following property:
.Values.manager.secret.enabled for the secret here.
This means we can replace the secret with our own secret just have to name it accordingly:
sap-btp-service-operator like here.
It would be nice if you could add this usage as an example in your documentation! We will try that.

@rufreakde .Values.manager.secret.enabled set to false is a way to go,
you can create the secret by yourself or helm will do it for you.
Basically this is the approach for multi tenant clusters you put sap-btp-service-operator secret in your namespace and the operator will find it.
BTW we're planning to drop the naming convention and find the secrets with label selectors.


Update MD file on new branch 2017-11-28_11-32-09

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestRef1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


It seems that this library isn't able to use in GAE.

Hi.
I want to deploy my app in Google App Engine.
so I ran the code below...
goapp serve

Then this error has occured
go-app-builder: Failed parsing input: parser: bad import "unsafe" in github.com/mattn/go-sqlite3/doc.go from GOPATH

I think this error won't be resolved as long as this library import "unsafe".
Do you have any ideas?

go-sqlite3 is cgo package. so you can't use this.


Error: failed to parse contract artifact

Gettting error when trying to declare contract:
starkli declare target/dev/inference.sierra --compiler-version 2.5.3
Error: failed to parse contract artifact
Scarb project: https://github.com/Vid201/zkml_onchain/tree/main/2l_relu_sigmoid/orion

hi @Vid201 , i had the same error, is the .env file in your src folder?

hi @Vid201 , does your Scarb.toml contain [[target.starknet-contract]]?
...
[dependencies]
starknet = ">=2.5.3"

[[target.starknet-contract]]

Ahh, yeah, that was the problem. I changed the Scarb.toml with that and added #[starknet::contract] to Cairo file. Totally missed that.
Thanks for the help!


chore: Code consistency

Adding suggestions based on Reference Architecture code consistency document:
https://github.com/nodeshift/nodejs-reference-architecture/blob/master/docs/development/code-consistency.md#code-consistency

@alexalikiotis when we run "git commit -m "foobar", husky will trigger the tests (which will start the lint) and if that fails, the git commit is aborted. The .husky directory and its contents were added by this command:
npm install husky --save-dev && npx husky init


Hybrid app - Getting duplicate notifications for the Expensify statement

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number:
Reproducible in staging?: Needs Reproduction
Reproducible in production?: Needs Reproduction
If this was caught on HybridApp, is this reproducible on New Expensify Standalone?:
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL: https://expensify.slack.com/archives/C049HHMV9SM/p1730491509556909
Issue reported by: @AndrewGable
Slack conversation (hyperlinked to channel name): #expensify-bugs
Action Performed:

Login to the HybridApp.
Set up and enable Expensify statement notifications.

Expected Result:
One notification from Concierge regarding statement
Actual Result:
Duplicate notifications received for same month
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Standalone
[ ] Android: HybridApp
[ ] Android: mWeb Chrome
[ ] iOS: Standalone
[X] iOS: HybridApp
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

View all open jobs on GitHub

This has been labelled "Needs Reproduction". Follow the steps here: https://stackoverflowteams.com/c/expensify/questions/16989

Sorry, the weekend then was OOO so I missed this. I don't have an iOS device so going to post in the #quality (?) channel. I'm thinking it might be Internal.

Posted in #quality

Not overdue, just added labels

Waiting on an internal engineer

Waiting for an internal engineer.

Does this happen when you're logged into NewApp and HybridApp at the same time? Or just HybridApp?

HybridApp only

Internal, waiting on an engineer

Still happening BTW

I'll look into this when I get some cycles

I spent some time on this yesterday. I should be able to reproduce this in dev by just running the ExpensifyWalletRegulatoryEmails:
CreateJob
name: www-prod/ExpensifyWalletRegulatoryEmails
data: {"debugEmail"<EMAIL_ADDRESS>
I wasn't able to reproduce with NewDot so I may need to test with the HybridApp specifically.

Looks like this is affecting most old push notifications:

Agree - I think I saw it on most of them

No progress on this. HybridApp and push notification dev aren't in a good spot right now so this probably has to wait until that gets better.


Spans are not associated into a single trace

Disclaimer - I may just be doing something wrong....

I have 3 Spring Boot web services. Date, Time, and DateTime.
DateTime uses Apache Camel to call the Date and Time services.
All three of these services are instrumented with the default opentracing-spring-jaeger-cloud-starter dependency.

I have Jaeger running in minikube. I also have the three services above running in minikube using the Jaeger sidecar agent. Each service is in a separate deployment/pod. The orchestration is using native service discovery.

The Jaeger Query UI does register the services in its drop down list.
When I click "Find Traces" nothing is found.
If I grab the spanId from any of the three services, I can search for it and it is found.
The spanIds do not seem to be associated in any way. Even when I use the DateTime spanId (which I would expect to see the calls to Date and Time).

Am I doing something wrong here?
Thanks!

Hi @danmikita
Unfortunately I don't have much experience with Apache Camel so I'm not really sure what might be going on here.
Have you checked if the traces are the anticipated ones if you replace Camel with the RestTemplate?
@pavolloffay Is Camel instrumented by OpenTracing?

@geoand Thank you for such a quick response. Trying RestTemplate was a great idea! I wasn't heading down the path of thinking the client mattered. I thought by instrumenting all the servers it would match up by using a uuid.
I tried switching to RestTemplate - but unfortunately I get the exact same result. I can search for each call individually by span id, but doing a trace search fails to find anything.
Any other ideas?

I wish it had worked 😁.
Just to be 100% clear, the RestTemplate you used was a spring bean, correct?

Nope - let me make that change. Thank you again for your help.

No problem, let's hope it works 🤞
On Tue, Aug 21, 2018, 00:22 Dan Mikita<EMAIL_ADDRESS>wrote:

Nope - let me make that change. Thank you again for your help.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opentracing-contrib/java-spring-jaeger/issues/14#issuecomment-414467629,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AELBv0eXKeHt44Arwy0eGi1LO3TxeIb0ks5uSyiXgaJpZM4WEm9c
.

Just to close out this conversation: I found that the reason I wasn't able to search for my traces is because all the pods running in minishift are behind by 3 days in regards to their date/time. Once I expanded my search range I was able to find them all.
Thank you!

That's what I was about to suggest!
Sometimes the filters have weird values preventing you from seeing the data.
Glad to hear it worked!
On Tue, Aug 21, 2018, 04:50 Dan Mikita<EMAIL_ADDRESS>wrote:

Just to close out this conversation: I found that the reason I wasn't able
to search for my traces is because all the pods running in minishift are
behind by 3 days in regards to their date/time. Once I expanded my search
range I was able to find them all.
Thank you!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opentracing-contrib/java-spring-jaeger/issues/14#issuecomment-414523303,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AELBv1d909i37bAZEipV3qA_J1smw0yiks5uS2dzgaJpZM4WEm9c
.

Hey @danmikita and @geoand
I have a very similar problem. I have 2 Microservices calling each other by REST, but cannot see the dependency within the trace. I've posted in Stackoverflow (with minimalistic Github repo) as I think it's not a bug. It's a very simple Spring Boot example and able to run with docker-compose build and docker-compose up
It would be great if you guys can help me out there! It would be really appreciated.
Best regards

@derXear Answered here


Streaming to kafka

I know it's a long shot, but asking anyway incase someone has already solved this.
Can I pump the delta to kafka instead of keeping it in memory and waiting for an agent to poll the endpoint? is it possible with some tweaks?
My usecase - An Open source GoldenGate equivalent which will stream all the DML operation in the source Oracle to a compacted kafka topic.

It would require pretty substantial "tweaks" to Databus to do this, but the basic idea of using Kafka as the transport layer is sound and is, in fact, exactly what LinkedIn's follow-on project Brooklin does:  https://github.com/linkedin/Brooklin/
Big caveat:  currently no Oracle connector is available.  A trigger-based one (as in Databus) is under development and may be open-sourced eventually, but there's no timetable for this.  You could consider adapting Databus's trigger-based source code to create a Brooklin connector, however; it wouldn't be a small job, but it's also not nearly as large a job as converting Databus to use Kafka.  This approach would still require the same kind of "databusification" of your Oracle DB (sy$txlog table, coalesce job, etc.) as Databus itself does.

From: jaisonpjohn<EMAIL_ADDRESS>Sent: Thursday, August 29, 2019 11:52 AM
To: linkedin/databus<EMAIL_ADDRESS>Cc: Subscribed<EMAIL_ADDRESS>Subject: [linkedin/databus] Streaming to kafka (#123)
I know it's a longcshot, but asking anyway incase someone has alreay solved this.
Can I pump the delta to kafka instead of keeping it in memory and waiting for an agent to poll the endpoint? is it possible with some tweaks?
My usecase - Opensource GoldenGate equivalent which will stream all the DML operation in the source Oracle to a compacted kafka topic.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flinkedin%2Fdatabus%2Fissues%2F123%3Femail_source%3Dnotifications%26email_token%3DAAXBMP3ZT4YSES7EJSJEVALQHALHJA5CNFSM4ISFV2NKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HII4ENQ&data=02|01|groelofs%40linkedin.com|c44841b6ad4145b66f5508d72cb210f7|72f988bf86f141af91ab2d7cd011db47|1|0|637027015611260481&sdata=vqmIjPa9XMbKpCRo1uavo%2FGSPgr4GwK58U2JgkYMu3I%3D&reserved=0, or mute the threadhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAXBMP4OP3SX7FTMBQADV53QHALHJANCNFSM4ISFV2NA&data=02|01|groelofs%40linkedin.com|c44841b6ad4145b66f5508d72cb210f7|72f988bf86f141af91ab2d7cd011db47|1|0|637027015611260481&sdata=CMi0BiaHkWeia0C5IDdK96HZSKqGU3DYFqCbsPoVoFw%3D&reserved=0.


State what python version is used in README

Based on your description in #185, I've created this PR for you to review to close #185
We now have an additional place in the README that needs to be updated manually when Python version is upgraded

We now have an additional place in the README that needs to be updated manually when Python version is upgraded

We could use the permalink https://github.com/cpp-linter/cpp-linter-action/blob/bb0610e1deae636c111f86d8839c04b916ba67ef/action.yml#L120 which github would render as:
https://github.com/cpp-linter/cpp-linter-action/blob/bb0610e1deae636c111f86d8839c04b916ba67ef/action.yml#L120
But we would still have to update the permalink if future changes to action.yml changes the line number.

I'm going to push some changes...

docker images not maintained by GitHub

This seems like it is not a precise description (that's my mistake). I think "third-party docker images" might be better. Or maybe a direct link to the github docs about specifying a "job's container image" (which is different from the job's runner image).

The placement of the note breaks the flow of paragraphs. I'm moving it up under the warning at the top.

Please hold off from merging this
The info provided isn't completely accurate to the current latest cpp-linter-action release.

We should merge this only when we're ready to make a new release. We also have to satisfy #182 (with another round of testing).


as removes bases styles for functional/class components

Environment
System:

OS: macOS 10.14.2
Shell: 5.3 - /bin/zsh

Binaries:

Node: 8.9.0
Yarn: 1.12.3

npmPackages:

babel-plugin-styled-components: 1.5.1 => 1.5.1
styled-components: 4.1.2 => 4.1.2

Reproduction
https://codesandbox.io/s/30k0n99jv5
Steps to reproduce

Create Styled-Component
Use the Styled-Component in a functional component (be sure to pass down your props to the styled-component)
Extend functional component with some additional styles
Use your extended version of the component with the as as prop

Expected Behavior

Base styles should still be applied
Extended styles should be applied overtop of base styles
New tag is used

Actual Behavior

Base styles are dropped, and only the extended styles are applied
New tag is still properly applied.

I'm seeing this on 4.1.3 as well.
Headline.js:
import styled from 'styled-components';

const Headline = styled.h1`
  color: orange;
  display: block;
  font-weight: 300;
`;

export default Headline;

StyledHeadline.js:
import { Headline } from 'components';
import styled from 'styled-components';

const StyledHeadline = styled(Headline)`
  color: pink;
`;

When I use StyledHeadline with the as prop, the only styles applied are color: pink;.

I'm seeing the same thing when a base component has some default props...those aren't taken into consideration either when using the as prop on a child component.

I too am seeing this issue on 4.1.3

The reason this happens is because as switches out the underlying component being rendered. If the target wrapped in styled is not another styled component, passing as will retain deeper styles since there is a custom component layer between, breaking the ability to "fold" the components.

If you do want to retain the ability to fold the styles as deeply as possible, I'd suggest wrapping your custom components in styled() and exporting that.
function FooBase(props) {
  return <div {...props} />;
}

export const Foo = styled(FooBase)`
  color: red;
`;

vs
export function Foo(props) {
  return <Wrapper {...props} />;
}

const Wrapper = styled.div`
  color: red;
`;

@probablyup In my above Headline example, the target wrapped in styled is a styled component. I am passing the return value from styled.h1 into the next styled call with no changes and still the base styles are mssing. Is there something else going on?

@AustP Yours looks like an actual bug. If you can reproduce in a sandbox, can take a look

Hello,
I did a little bit of digging. A possible reason why this was reported as a bug (the original issue, not @AustP's variant) is because it worked in the 4.0.0-beta versions of SC, and stopped working at some point.
Here's a reproduction with<EMAIL_ADDRESS>where the bug doesn't happen - styled props of the inner component are kept and the outer component is a button.
With<EMAIL_ADDRESS>though, the styles of the inner component are lost.
So it happened somewhere in that diff
I will try to keep doing some digging - from my side, it prevents me from moving away from the beta version of SC because there is no clean way around.

Another example: https://codesandbox.io/s/r7kjpyjwq
Related issue: https://github.com/styled-components/styled-components/issues/1981

What is the best workaround in the meantime?

I encountered this issue too.
In case of use styled-component for the UI-library that has internally gives className, and using as option will behave exactly as the pasted by guys the examples.
This could not keep the appearance as enough UI component.
I read styled-component's source code a little.
For now, I just found behavior is below.
const elementToBeCreated = this.props.as || this.attrs.as || target;

https://github.com/styled-components/styled-components/blob/master/packages/styled-components/src/models/StyledComponent.js#L135
return createElement(elementToBeCreated, propsForElement);

https://github.com/styled-components/styled-components/blob/master/packages/styled-components/src/models/StyledComponent.js#L172
Is it generated another element when use as?
Which means, Even if can be passed enough props object by propsForElement, the internal logic by the component itself that is originally passed as a target is not executed.
If the classNames within a component are internally generated by different architectures such as classnames, the component will be destroyed when use with as.
Is as use case only use to the primitive element?
This is might become a stall point when use styled-component with some UI library.
Also, this may be breaking if the UI library has props of the same name (as).
If my understanding is not wrong, the best workaround is ... "Take care when use as option" 🤕
Can I know the contributor's opinion about this?
cc: @probablyup
@leerob thanks for your example, It is easy to understand.


Unclear documentation

Hi!
I'm trying to setup a multi logger configuration
let stdout = ConsoleAppender::builder().build();
let logfile = FileAppender::builder()
        .encoder(Box::new(PatternEncoder::new("[{d(%Y-%m-%d %H:%M:%S)}] {l} - {m}\n")))
        .build("exceptions.log")?;
let config = Config::builder()
        .appender(Appender::builder().filter(Box::new(ThresholdFilter::new(LevelFilter::Error)))
            .build("logfile", Box::new(logfile)))
        .appender(Appender::builder()
            .build("stdout", Box::new(stdout)))
        .build(Root::builder().appender("logfile").appender("stdout").build(LevelFilter::Info))?;

and after reading the docs, still unclear what is purpose of this
Root::builder()...build(LevelFilter::Info)

Why is the level filter provided in the root builder, if we configure it with the appender beforehand?

The reason is that we may have multiple loggers pointed at a single appender. What would happen then if we wanted to change just the root log level? Does this clear the question up?

Yes, it clears up. Thanks.


Send a few messages w/ welcome to contribution subject

http://spin.atomicobject.com/2014/07/01/google-analytics-web-apps/
http://discuss.emberjs.com

moved to the internal issues sheet


improve README

add background of this couchapp
how to build this?
how to develop and test?

done in 53b4a11e7a51fdbca6d0f1665c50e6d78b78ad0d


fix: Fix avm unit test with proving by passing the public_inputs

Benchmark results
Metrics with a significant change:

proof_construction_time_sha256_30_ms (16): 1,797 (+26%)
protocol_circuit_witness_generation_time_in_ms (private-kernel-init): 594 (+29%)
protocol_circuit_witness_generation_time_in_ms (private-kernel-reset-small): 2,315 (+21%)
protocol_circuit_witness_generation_time_in_ms (private-kernel-tail-to-public): 10,533 (+45%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_30_ms
11,707 (-1%)
3,187 (+1%)
:warning: 1,797 (+26%)
1,664 (+2%)
1,543 (-2%)

proof_construction_time_sha256_100_ms
43,745
11,721
5,433
5,446 (+2%)
5,344

proof_construction_time_poseidon_hash_ms
79.0
34.0
35.0 (+3%)
58.0
86.0 (-2%)

proof_construction_time_poseidon_hash_30_ms
1,514 (-1%)
417
201
229 (+2%)
271 (+2%)

proof_construction_time_poseidon_hash_100_ms
5,743
1,559 (-1%)
718
771 (-3%)
789 (-2%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
1,412
1,412
1,412

l1_rollup_calldata_gas
9,452
9,472
9,476

l1_rollup_execution_gas
610,273
610,293
610,297

l2_block_processing_time_in_ms
750
1,403
2,691

l2_block_building_time_in_ms
25,739 (+2%)
50,803
100,932 (+1%)

l2_block_rollup_simulation_time_in_ms
25,644 (+2%)
50,502
100,577 (+2%)

l2_block_public_tx_process_time_in_ms
22,046 (+2%)
46,705
96,831 (+2%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
6,940 (-1%)
9,800

node_database_size_in_bytes
12,173,392
16,052,304

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Function
time_ms
bytecode_size_in_bytes

GasToken:_increase_public_balance
66.0 (-2%)
13,873

GasToken:set_portal
16.6 (-7%)
3,495

Token:constructor
93.7 (+3%)
24,207

FPC:constructor
61.7 (+2%)
13,893

GasToken:mint_public
51.7 (-2%)
10,241

Token:mint_public
593 (-5%)
19,216

Token:assert_minter_and_mint
63.8 (-12%)
13,034

AuthRegistry:set_authorized
30.7 (-7%)
7,869

FPC:prepare_fee
77.2 (-4%)
15,187

Token:transfer_public
48.2 (-12%)
31,425

FPC:pay_refund
57.5 (-2%)
20,080

Benchmarking:increment_balance
2,675
15,465

Token:_increase_public_balance
58.8 (-2%)
15,089

FPC:pay_refund_with_shielded_rebate
51.0 (-15%)
21,167

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.177 (-1%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
10.3
16.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.593
0.511
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
47.9
75.6
131
244
469

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.490
0.464
0.449
0.442
0.438

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
58.9 (-1%)
111
182
351
691

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.499
0.499
0.481
0.475
0.475

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
72.2
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
133
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.514
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
85,672
670,983

Transaction size based on fee payment method
| Metric |  |
| - |  |


Make mem::size_of::<T>() == 0, TableLayout::new::<T>() and mem::needs_drop::<T>() const

Just preparation for the upcoming for "Add new MIR constant propagation based on dataflow analysis" https://github.com/rust-lang/rust/pull/101168, but this change is also useful on its own.

@bors r+

:pushpin: Commit 4e39290492a2831474c3c505b523e7f209c829d6 has been approved by Amanieu
It is now in the queue for this repository.

:hourglass: Testing commit 4e39290492a2831474c3c505b523e7f209c829d6 with merge e39d245d31db1a0a2ff2a7dfe29d76279e555dec...

:sunny: Test successful - checks-actions
Approved by: Amanieu
Pushing e39d245d31db1a0a2ff2a7dfe29d76279e555dec to master...


Ability to sort and search by all/most fields in tables

Some tables contain fields that are not sortable or searchable, but language in the tool suggests that you should be able to. We should also implement sort across all fields/tables for consistency. I recall this being a complex issue and maybe out of scope, but we should assess which of the fields are sortable/searchable and add them.
Here is an itemized list of tables and fields that need filters/searchability added:
Public DMPs: Org, Owner
My dashboard main plans table: Role, Visibility, Shared
My dashboard Org plans table: Owner
Users: Plans, Current privileges
Plans: Org, Owner (high priority - multiple requests), Visibility (required for #278 )
Guidance: Themes (alphabetical by first word of first theme)

Refer to Users page is working for search on User name

This issue was also raised during the DMPonline Usability Testing exercise in 2020

Some fields may be straightforward to make visible and sortable, and so those can be added in due course.
Other fields will not be straightforward to add, and so they can be ignored.


.emitted() returns object by reference

I'm not sure whether this behaviour is expected or not, but I think it should be clearer in the documentation:
const wrapper = mount(...);

wrapper.trigger('emit-event');
expect(wrapper.emitted().myEvent.length).to.equal(1);

wrapper.trigger('emit-event');
expect(wrapper.emitted().myEvent.length).to.equal(2);

// BUT, when saved to a variable:

wrapper.trigger('emit-event');
const emitted = wrapper.emitted();
expect(emitted.myEvent.length).to.equal(1);

wrapper.trigger('emit-event');
expect(emitted.myEvent.length).to.equal(2);

I expected the object returned by wrapper.emitted() to not change, but it did.

This is the intended behavior.
You're right, we should make this clearer in the docs. Are you able to make a PR?

Will do this afternoon :+1:


Why not Anchor Lib?

If a script is dealing with generating contextual mark positioning and the information was inside the anchor lib not in its name then it would be more convenient to interpret the anchor while generating the feature. Also differentiating the cursive attachment anchor and mark positioning anchor is primitive atm. Right now the method is to have a certain naming scheme for the anchors but if the information that is going to be interpreted while generating the mark positioning is larger then differentiating the anchors just by name seems inconvenient and messy. If there was an anchor lib, I suppose I could add a string 'type' key to the anchor lib and explicitly say it's a cursive attachment anchor by putting a string 'cursive' as the value of the key. Also using 'context' and then putting some other information there. Imagine if the context is large, then naming the anchor to indicate this is not the way to go. Why not adding a lib or some sort of mechanism to take care of this in the anchors?

There is an issue on the topic of ligature anchor which expended to better anchors in general on the ufo-spec: https://github.com/unified-font-object/ufo-spec/issues/32

Currently, there isn't an anchor Lib because RoboFab didn't have it, but it's worth considering and discussing

The lib would need to be stored somewhere and the UFO doesn't support that. We considered adding a <lib> element to everything long ago (specifically, I wanted point.lib), but decided that the complexity wasn't worth the functionality. But, I agree that using naming schemes is not idea. That was part of the reason we added persistent identifiers to various UFO objects. Here's what you can do:

Establish a storage location for anchor data in the glyph lib.
Get the identifier for a given anchor.
Use that identifier in the glyph lib to reference the anchor.

Something like this:
# storage location
libKey = "design.bahman.anchorstuff.something.etc"
if libKey not in glyph.lib:
    glyph.lib[libKey] = {}

# get the anchor's identifier
identifier = anchor.generateIdentifier()

# store the anchor's data
glyph.lib[libKey][identifier] = dict(foo=1, bar="yes", hello="world")

You can then reference that as needed.
Does that help?

Thank you very much for your input. What you suggested Tal is a good solution for now and it helps. But it also opens a room for unexpected things to happen. This way if the anchor gets deleted then the information still remains in the UFO:
glyph = CurrentGlyph()
anchors = []
libKey = "design.bahman.anchorstuff.something.etc"
if libKey not in glyph.lib:
    glyph.lib[libKey] = {}

for anchor in glyph.anchors:
    identifier = anchor.getIdentifier()
    glyph.lib[libKey][identifier] = dict(foo=1, bar="yes", hello="world")
    anchors.append(anchor)

for anchor in anchors:
    glyph.removeAnchor(anchor)

print(glyph.lib[libKey])

Also I think adding the anchor lib would be cleaner to interpret and also more explicit in terms of being pythonic.


Significant performance loss with Results on 0.20.7

Describe the bug
There has been a significant performance loss with the Results page between 0.20.6 and 0.20.7.  With 20.6 it took about 8 seconds to change sorts, go to next page, etc.; with 20.7 these tasks take 60-70 seconds.
To Reproduce
For reference, my database has 5323 records.
0.20.6
Launch Speedtest Tracker
Click Results = 7s
Change Per Page to 100; seems to remember this between sessions most of the time
Click Server (sort ascending) = 8s
Click Server (sort descending) = 8s
Click Page 4 = 8s
Click ID (sort by ID) = 8s
0.20.7
Launch Speedtest Tracker
Click Results = 7s
Change Per Page to 100; seems to remember this between sessions most of the time
Click Server (sort ascending) = 60s
Click Server (sort descending) = 68s
Click Page 4 = 66s
Click ID (sort by ID) = 64s
I often get several "page unresponsive" alerts from my browser during these long waits.
Expected behavior
The 8-second waits on 20.6 were a bit annoying, but the 60+ seconds with 20.7 make the Results page pretty unusable.
I have noticed other areas that don't perform very well (including with 20.6 so probably not related to this issue).  As an example it typically takes about 8 seconds to open a detailed results page.  This seems excessively long to me to display a single page of detailed results, but I don't use it very often so I hadn't reported it.
For now I have rolled back to 20.6.
Environment
Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-113-generic x86_64)
Docker version 27.0.3, build 7d4bcd8
Speedtest Tracker v0.20.7
Browser = Brave Version 1.67.123 Chromium: 126.0.6478.126 (Official Build) (64-bit)

SQLite.  I'm not sure what qualifies as "large", but I do delete the records associated with a consistently slow speedtest server every few days.  I haven't really kept track of how many records each time, and it varies from day to day depending on how often the bad server is selected by speedtest.  When I originally did the initial purge of these records several weeks ago they made up about 15% of the total record count.

SQLite performance can be impacted if you have other IO operations happening on the same disk. Start by double checking there first, it's also possible the indexes might need to be rebuilt in your database to help with performance.
I haven't felt any performance issues with my install but I'm running a MySQL DB.

Just to clarify, I'm running in Docker (lscr.io/linuxserver/speedtest-tracker:latest) so I can easily switch between 20.6 and 20.7 using the exact same config and database.  I don't think the issue is related to the database, disk i/o, or the server as I can switch back and forth between the two versions within a few seconds and 20.6 consistently produces 8-second-ish results while 20.7 consistently produces 60+ second results.  The only difference between the two is which docker image is being used.
That said, I did run a vacuum and a reindex on the database which reduced the database size by about 780k, but performance was unchanged (still about 8 seconds for 20.6 vs 60+ seconds for 20.7).  I also tried deleting my "latest" (i.e., 20.7) docker image and repulling it, but performance remains unchanged.

Perhaps this is an issue with the Docker image?

lscr.io/linuxserver/speedtest-tracker:0.20.6

1	12.9 MB	COPY /root-out/ / # buildkit
2	0 B	ARG BUILD_DATE
3	0 B	ARG VERSION
4	0 B	ARG MODS_VERSION=v3
5	0 B	ARG PKG_INST_VERSION=v1
6	0 B	LABEL build_version=Linuxserver.io version:- 6cd44267-ls1 Build-date:- 2024-05-22T19:21:09+00:00
7	0 B	LABEL maintainer=TheLamer
8	23.6 kB	ADD --chmod=744 https://raw.githubusercontent.com/linuxserver/docker-mods/mod-scripts/docker-mods.v3 /docker-mods # buildkit
9	4.1 kB	ADD --chmod=744 https://raw.githubusercontent.com/linuxserver/docker-mods/mod-scripts/package-install.v1 /etc/s6-overlay/s6-rc....
10	0 B	ENV PS1=$(whoami)@$(hostname):$(pwd)\$ HOME=/root TERM=xterm S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 S6_VERBOSITY=1 S6_STAGE2_HOOK=...
11	13.8 MB	RUN |4 BUILD_DATE=2024-05-22T19:21:09+00:00 VERSION=6cd44267-ls1 MODS_VERSION=v3 PKG_INST_VERSION=v1 RUN echo "**** install run...
12	6.8 kB	COPY root/ / # buildkit
13	0 B	ENTRYPOINT ["/init"]
14	0 B	ARG BUILD_DATE
15	0 B	ARG VERSION
16	0 B	LABEL build_version=Linuxserver.io version:- 1.26.1-r0_8.3.8-r0-ls5 Build-date:- 2024-06-06T18:32:04+00:00
17	0 B	LABEL maintainer=nemchik
18	42.6 MB	RUN |2 BUILD_DATE=2024-06-06T18:32:04+00:00 VERSION=1.26.1-r0_8.3.8-r0-ls5 RUN echo "**** install build packages ****" && apk...
19	17.9 kB	COPY root/ / # buildkit
20	0 B	EXPOSE map[443/tcp:{} 80/tcp:{}]
21	0 B	ENV LSIO_FIRST_PARTY=true
22	0 B	ARG BUILD_DATE
23	0 B	ARG VERSION
24	0 B	ARG SPEEDTEST_TRACKER_VERSION
25	0 B	LABEL build_version=Linuxserver.io version:- v0.20.6-ls29 Build-date:- 2024-06-13T01:54:56+00:00
26	0 B	LABEL maintainer=thespad
27	0 B	ENV HOME=/config
28	91.5 MB	RUN |3 BUILD_DATE=2024-06-13T01:54:56+00:00 VERSION=v0.20.6-ls29 SPEEDTEST_TRACKER_VERSION=v0.20.6 RUN apk add --no-cache g...
29	4.5 kB	COPY root/ / # buildkit
30	0 B	VOLUME [/config]

lscr.io/linuxserver/speedtest-tracker:latest

1	12.9 MB	COPY /root-out/ / # buildkit
2	0 B	ARG BUILD_DATE=2024-07-02T14:32:49+00:00
3	0 B	ARG VERSION=5f4013a7-ls6
4	0 B	ARG MODS_VERSION=v3
5	0 B	ARG PKG_INST_VERSION=v1
6	0 B	ARG LSIOWN_VERSION=v1
7	0 B	LABEL build_version=Linuxserver.io version:- 5f4013a7-ls6 Build-date:- 2024-07-02T14:32:49+00:00
8	0 B	LABEL maintainer=TheLamer
9	25.3 kB	ADD --chmod=755 https://raw.githubusercontent.com/linuxserver/docker-mods/mod-scripts/docker-mods.v3 /docker-mods # buildkit
10	4.2 kB	ADD --chmod=755 https://raw.githubusercontent.com/linuxserver/docker-mods/mod-scripts/package-install.v1 /etc/s6-overlay/s6-rc....
11	945 B	ADD --chmod=755 https://raw.githubusercontent.com/linuxserver/docker-mods/mod-scripts/lsiown.v1 /usr/bin/lsiown # buildkit
12	0 B	ENV PS1=$(whoami)@$(hostname):$(pwd)\$ HOME=/root TERM=xterm S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 S6_VERBOSITY=1 S6_STAGE2_HOOK=...
13	13.9 MB	RUN |5 BUILD_DATE=2024-07-02T14:32:49+00:00 VERSION=5f4013a7-ls6 MODS_VERSION=v3 PKG_INST_VERSION=v1 LSIOWN_VERSION=v1 RUN echo...
14	7.5 kB	COPY root/ / # buildkit
15	0 B	ENTRYPOINT ["/init"]
16	0 B	ARG BUILD_DATE=2024-07-11T18:49:50+00:00
17	0 B	ARG VERSION=1.26.1-r0_8.3.9-r0-ls10
18	0 B	LABEL build_version=Linuxserver.io version:- 1.26.1-r0_8.3.9-r0-ls10 Build-date:- 2024-07-11T18:49:50+00:00
19	0 B	LABEL maintainer=nemchik
20	48.3 MB	RUN |2 BUILD_DATE=2024-07-11T18:49:50+00:00 VERSION=1.26.1-r0_8.3.9-r0-ls10 RUN echo "**** install build packages ****" && ap...
21	17.9 kB	COPY root/ / # buildkit
22	0 B	EXPOSE map[443/tcp:{} 80/tcp:{}]
23	0 B	ENV LSIO_FIRST_PARTY=true
24	0 B	ARG BUILD_DATE=2024-07-13T21:24:55+00:00
25	0 B	ARG VERSION=v0.20.7-ls34
26	0 B	ARG SPEEDTEST_TRACKER_VERSION=v0.20.7
27	0 B	LABEL build_version=Linuxserver.io version:- v0.20.7-ls34 Build-date:- 2024-07-13T21:24:55+00:00
28	0 B	LABEL maintainer=thespad
29	0 B	ENV HOME=/config
30	91.8 MB	RUN |3 BUILD_DATE=2024-07-13T21:24:55+00:00 VERSION=v0.20.7-ls34 SPEEDTEST_TRACKER_VERSION=v0.20.7 RUN apk add --no-cache g...
31	4.5 kB	COPY root/ / # buildkit
32	0 B	VOLUME [/config]

I'll do a little digging on this later in the week.

So, I "think" I narrowed this down to referencing json data in sqlite. Can you give me some details on the server specs your Docker instance is running on? My servers might be too fast which is why I'm not seeing much of a slow down.

Hardware is dual Xeon X5550 @ 2.67GHz (8 cores, 16 threads), 48GB RAM
Docker is running in an Ubuntu 22.04 VM with 8 cores, 6GB RAM (about 53% in use)
Neither are heavily loaded (load avg typically <1)
I wondered if the json structures within sqlite might be contributing to some of the general performance I mentioned (e.g., taking 8 seconds to open a detail record).  That said, were any changes made to json-related code between 20.6 and 20.7?  The reason I ask is that the significant performance hit that my original post referred to seems to be very specific to 20.7, and possibly specific to the docker image.  As mentioned, if I simply switch back to the 20.6 docker image (using the same config/data), the significant performance hit goes away, although opening detail records still takes 8 seconds.

Between 0.20.6 and 0.20.7 nothing I did, but that being said it could of been dependency related.

Oh, OK.  Didn't think about that.  I did see some differences in the docker images as shown above, but I don't understand that side of docker enough to know if they could be related.

Just a quick update...I pulled the new 0.20.8 docker image this morning.  I'm experiencing the same performance issues with it as with 0.20.7.  As soon as I revert to the 0.20.6 image, performance returns to normal:
Task                             0.20.8     0.20.6
--------------------------------------------------
Open Results page (100 items)     7.30s      7.57s
Sort Server Ascending            63.22s      7.41s
Sort Server Descending           68.12s      7.53s

A few days ago you mentioned the referencing of json data in sqlite, so I did some timing comparisons between 0.20.6 and 0.20.8.  In both scenarios, my Per Page is set to 100.
Results Table (0.20.6)
Sort by ID		 7.71s
Sort by Server ID	 7.90s
Sort by Server		 7.73s
Sort by Download	 7.59s
Sort by Upload		 7.85s
Sort by Ping		 7.78s
Sort by Status		 7.74s
Sort by Created at	 7.98s

Results Table (0.20.8)*
Sort by ID		62.55s
Sort by Server ID	73.26s
Sort by Server		70.22s
Sort by Download	70.23s
Sort by Upload		73.50s
Sort by Ping		71.89s
Sort by Status		74.14s
Sort by Created at	Initial page load 7.93s / Subsequent sort 72.51s

*The URL in the address bar updates at about 8 seconds, but the results table isn't updated until the times indicated above.
*Of interest is that when I initially click Results, which appears to be a descending Created At sort, the table displays in about 7.93s; for all sort selections after that the times are as indicated above (i.e., the performance of the initial display of the Results page does not seem to be affected; only if I select a different sort order and then it doesn't matter which sort order I select).

If it's related to the database or database indexes regardless of version I'd expect the results to be the same(ish). This is leading me down the Docker and/or code path now. Looks like there is a new Filament release and it references performance improvement for table checkboxes so this could be related.
I'm going to tag and release v0.20.9 shortly and I'll include only dependency updates to see if it fixes it.

Both v0.20.9 and v0.21.0 appear to have returned to the old (v0.20.6) performance levels.
It takes about 8 seconds to switch sort orders with these two versions rather than the 60-75 seconds I was seeing with v0.20.7 and v0.20.8.
Thank you!

Not a clue what it was but sounds like a win!


Wire up<EMAIL_ADDRESS>to go to the team

The team decided on<EMAIL_ADDRESS>as the general website email address. This can just be a receive-only email list that forwards to multiple people on the team.

All done


Make consul based dynamic configuration possible

Jenkins build failed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2094/

Jenkins build failed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2095/

Missing jira id in commit messages

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2096/

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2097/

@keyki @doktoric  please

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2098/

@doktoric dependency syntax is fixed

Jenkins build failed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2103/

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/2104/


Stress Detection Project added [ML Projects]

Stress Detection Project

Close #3897

Checklist:

[✅] I have mentioned the issue number in my Pull Request.
[✅] I have commented my code, particularly in hard-to-understand areas
[✅] I have created a helpful and easy to understand README.md

@MohitGupta121  @TusharKesarwani  Kindly, review this PR


SOLR-15891: Remove bloat in Solr docker layers

https://issues.apache.org/jira/browse/SOLR-15891
This removes the bloat in the first non-header docker layer, specifically this line:
find /opt/solr/ -type f -print0 | xargs -0 chmod 0644; \

That line increased the size of that docker layer to the complete size of the solr installation, meaning that two layers (in both local and official) were the full size of the solr installation.
By moving the 0644 chmod to the COPY and download sections, in local and official, the only thing we have to do in the referenced layer is give executables and directories the correct permissions, which in practice takes 250kB instead of 200mB. This achieves what we want.
One caveat is that COPY --chmod requires buildkit enabled, and Docker v20 installed. I think this is fine given this is a new offering, so we can assume users are using new Docker versions. (much better than having the docker image be 1.5-2x the size)
Some other small changes made:

The tests now re-run when the shared.sh file is changed
The test official image is tagged when it is built. We can change this in the future if we want to. (Maybe we re-think this in the future, and have a flag for the docker build to be "official" or "local", and keep the same commands (disabling push for "official".))
I fixed the labels for the docker image
The test directories that use setfacl on linux, now use chmod a+rwx if setfacl is not available. This is unfortunate, but there really is no other work around if we want the tests to run on Macs. Willing to have discussion here if y'all don't think this is acceptable.

@janhoy I decided to go ahead and move to the system of just using the permissions from the TGZ, and removing the overwriting in the dockerfile. This works well, and I just had to fix the permissions for the docker scripts.


gst-plugins-bad vulkan build failure

Hitting
ERROR: gstreamer1.0-plugins-bad-1.14.4-r0 do_compile: oe_runmake failed
Perhaps I need to override the package config for gstreamer1.0-plugins-bad.
../../../gst-plugins-bad-1.14.4/ext/vulkan/vkerror.c: In function '_vk_result_to_string':
../../../gst-plugins-bad-1.14.4/ext/vulkan/vkerror.c:66:16: error: 'VK_RESULT_BEGIN_RANGE' undeclared (first use in this function); did you mean 'VK_RESULT_MAX_ENUM'?
../../../gst-plugins-bad-1.14.4/ext/vulkan/vkerror.c:66:16: note: each undeclared identifier is reported only once for each function it appears in
Makefile:994: recipe for target 'libgstvulkan_la-vkerror.lo' failed

| ../../aarch64-oe4t-linux-libtool  --tag=CC   --mode=compile aarch64-oe4t-linux-gcc  -march=armv8-a+crc -fstack-protector-strong  -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot -DHAVE_CONFIG_H -I. -I../../../gst-plugins-bad-1.14.4/ext/vulkan -I../..    -I../../../gst-plugins-bad-1.14.4/gst-libs -I../../gst-libs -I../../ext/vulkan -pthread -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/gstreamer-1.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/glib-2.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/lib/glib-2.0/include  -DGST_USE_UNSTABLE_API -fno-strict-aliasing  -DG_THREADS_MANDATORY -DG_DISABLE_CAST_CHECKS -Wall -Wdeclaration-after-statement -Wvla -Wpointer-arith -Wmissing-declarations -Wmissing-prototypes -Wredundant-decls -Wwrite-strings -Wformat-security -Wold-style-definition -Winit-self -Wmissing-include-dirs -Waddress -Wno-multichar -Wnested-externs       -fvisibility=hidden -pthread -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/gstreamer-1.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/glib-2.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/lib/glib-2.0/include -pthread -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/gstreamer-1.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/include/glib-2.0 -I/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot/usr/lib/glib-2.0/include  -O2 -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0=/usr/src/debug/gstreamer1.0-plugins-bad/1.14.4-r0                      -fdebug-prefix-map=/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0=/usr/src/debug/gstreamer1.0-plugins-bad/1.14.4-r0                      -fdebug-prefix-map=/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot=                      -fdebug-prefix-map=/__w/meta-flutter/jetson-nano-devkit/tegra-demo-distro/build/tmp/work/armv8a_tegra-oe4t-linux/gstreamer1.0-plugins-bad/1.14.4-r0/recipe-sysroot-native=  -c -o libgstvulkan_la-vkerror.lo `test -f 'vkerror.c' || echo '../../../gst-plugins-bad-1.14.4/ext/vulkan/'`vkerror.c

Is there an expected Vulkan header version required for GST 1.14.4?  Perhaps I'm running something too new.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1316

Do you need the Vulkan gstreamer plugins?  If not, it might be simpler to disable it altogether with PACKAGECONFIG:remove = "vulkan" in a bbappend for the recipe.
If you do need them, I'd be happy to take a patch over at the meta-tegra repo.

Actually, we already have a patch for this, added with this commit.

I do need Vulkan gstreamer plugins.  I'm targeting dunfell-l4t-r32.5.0.
I see the patch it in master, but not in :
https://github.com/OE4T/meta-tegra/tree/4af18120863c97a04795f294b542a8cf036347e1/contrib/recipes-multimedia/gstreamer-1.14/gstreamer1.0-plugins-bad
https://github.com/OE4T/meta-tegra/tree/dunfell-l4t-r32.5.0/contrib/recipes-multimedia/gstreamer-1.14/gstreamer1.0-plugins-bad
Then of course there are many updates to gst vulkan since 14.  What's the limiting factor for 14?

vulkan in OE-Core dunfell is at version 1.1, so you shouldn't need the patch in dunfell-based branches unless you've got another layer where you're overriding with a newer version... in which case, you should be able to apply the patch there via a bbappend.
You don't have to use gstreamer 1.14, but we have the recipes, and default to using them in the demo distro, because they're more compatible with NVIDIA-provided binaries (their prebuilt gstreamer plugins and the gstreamer users in their prebuilt containers).

@madisongh
I'm working around it using
echo 'PACKAGECONFIG:remove:pn-gstreamer1.0-plugins-bad = "vulkan"' >> ./conf/local.conf

This had no impact
echo 'PREFERRED_VERSION:gstreamer1.0 = "1.16%"' >> ./conf/local.conf
echo 'PREFERRED_VERSION:gstreamer1.0-libav = "1.16%"' >> ./conf/local.conf
echo 'PREFERRED_VERSION:gstreamer1.0-plugins-bad = "1.16%"' >> ./conf/local.conf
echo 'PREFERRED_VERSION:gstreamer1.0-plugins-good = "1.16%"' >> ./conf/local.conf
echo 'PREFERRED_VERSION:gstreamer1.0-plugins-ugly = "1.16%"' >> ./conf/local.conf
echo 'PREFERRED_VERSION:gstreamer1.0-plugins-base = "1.16%"' >> ./conf/local.conf

1.18 is the desired version.

The recipe name in the PREFERRED_VERSION variables is not an override, so you should use an underscore as separator there, not a colon.
The config file that is being included by the distro config file honors a variable called GST_VERSION to make it easier to change the preferred version.


[Question/Feedback from Discussions] Extend Databricks module

Discussed in https://github.com/Azure/ResourceModules/discussions/3088

Originally posted by Ulveno April  4, 2023
Hello,
Awesome work with all the modules. I was testing the Databricks module but it looks like there are missing properties params.

CC: @Ulveno

Hey @clintgrove,
please note that we're currently working on migration the still relevant issues from the original CARML modules over to AVM. This one is for a module that's owned by you.
There is not much in the description, but at least a few parameter seems to be missing (unless they're read-only). Please triage when you have a moment :)

Noted, I will have a look as soon as I can.

I am getting answers from Product group about these parameters. I suspect that none of them will add value to our current deployment code.
authorizations
uiDefinitionUri
And these below might just be read only and CANNOT be set at deployment
createdBy
managedDiskIdentity
storageAccountIdentity
updatedBy

thanks for checking @clintgrove .
As chatted about, I suspect the best course of action will be to add them as properties (not parameters), but outcommented like
{
  (...)
  // createdBy: createdBy // Read-only


systemverilog-plugin: fix unnamed packed array

Now vpiPackedArrayTypespec can contain another, named type.

UHDM-integration-test: https://github.com/chipsalliance/UHDM-integration-tests/pull/734


Remove restrictions for aliases from versions and plugins for Version catalog

Removed restrictions for plugins and versions since they are accessed with:
libs.plugins.<name-of-plugin> and libs.versions.<name of version>.
For dependencies:
We still apply restriction but we apply them only for first part of alias for:

bundle
bundles
version
versions
dependency
dependencies
plugin
plugins

So for example:
findDependency-my, dependencyAlias-my are forbidden. But my-findDependency is allowed.
Additionally we don't do a check with contains anymore, but we split first part of alias by camel case. So finddependency or dependencyalias are also valid now.
Restrictions stayed here mainly because dependencies are accessed directly like: libs.<name of dependency> and we have accessors like libs.plugins and also methods like libs.dependencyAliases.

I also have to update documentation, will do this after https://github.com/gradle/gradle/pull/18207 is merged. And since I added word dependency and dependencies I will add this to potential breaking changes
Fixes #18167

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

Thinking about it, maybe would make sense to move all optional API methods to some "namespace", like api, e.g. libs.api.findDependency().
And then restricted words would be api, bundles, plugins, versions without any need to do any of this contains('version') magic. WDYT?

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

@bot-gradle test QFL

OK, I've already triggered QuickFeedbackLinux build for you.

@bot-gradle test and merge

Your PR is queued. See the queue page for details.

OK, I've already triggered a build for you.


Update genienlp

We need the genienlp library to be the same version as the trained
model, otherwise it crashes with an obscure error.

Tests won't pass because the staging NLP server is down.


Network refactor for the new protocol v37

Plugged new Serializer ORecordSerializerNetworkV37
Added executors for Open/Connect 37
Added ORecordSerializerNetworkFactory for matching the right serializer for the given protocol
Small fix on OSBTreeRidBag on serialization
Changed tests from ORecordSerializerNetwork to ORecordSerializerNetworkV37

What are the positive impacts of these changes?
Scott

hi @smolinari,
This is part of the plan of the review of the binary protocol, we are trying to unify the record serialization on the network to the network protocol itself, having a single on network versioning system and the ability to evolve network and disc independently.
Bye

Thank you for the explanation. 👍
Scott


Correct Gower distance bouds inconsistency & EGO for mixed integer

There was both an inconsistency for the data normalization within Gower distance and a problem as the data were not in [0,1] as intended.
This PR correct this error by using directly the bounds of the variables (xlimits) and by doing a proper job to project the data into [0,1]^n.

The  Acquisition criterion was not maximized at all with Mixed-integer variables. Now, it is maximized from random starting points by CObyLA.

Fix several bugs.

Hi, @relf ,
I have found several bugs recently while working on hierarchical variables Kriging models.
This PR fixes the latter problems. ( I can't ask for review prior to 1st review. )
Paul


Bild.de Extractor not working

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
Germany
Provide a description that is worded well enough to be understood
Today i tried downloading a video from bild.de i have never realy downloaded from there. But after the download starts the file is done. The content of the file is nothing just a black screen.
The m3u8 file i get out looks like this. I already tried updating im on the<EMAIL_ADDRESS>now.
#EXTM3U #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-VERSION:5 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-TARGETDURATION:6 #EXTINF:6.0, segment_0_playlist_0.ts #EXTINF:6.0, segment_0_playlist_1.ts #EXTINF:6.0, segment_0_playlist_2.ts #EXTINF:6.0, segment_0_playlist_3.ts #EXTINF:6.0, segment_0_playlist_4.ts #EXTINF:6.0, segment_0_playlist_5.ts #EXTINF:6.0, segment_0_playlist_6.ts #EXTINF:6.0, segment_0_playlist_7.ts #EXTINF:6.0, segment_0_playlist_8.ts #EXTINF:6.0, segment_0_playlist_9.ts #EXTINF:6.0, segment_0_playlist_10.ts #EXTINF:3.96, segment_0_playlist_11.ts #EXT-X-ENDLIST #EXTM3U #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-VERSION:5 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-TARGETDURATION:6 #EXTINF:6.0, segment_1_playlist_0.ts #EXTINF:6.0, segment_1_playlist_1.ts #EXTINF:6.0, segment_1_playlist_2.ts #EXTINF:6.0, segment_1_playlist_3.ts #EXTINF:6.0, segment_1_playlist_4.ts #EXTINF:6.0, segment_1_playlist_5.ts #EXTINF:6.0, segment_1_playlist_6.ts #EXTINF:6.0, segment_1_playlist_7.ts #EXTINF:6.0, segment_1_playlist_8.ts #EXTINF:6.0, segment_1_playlist_9.ts #EXTINF:6.0, segment_1_playlist_10.ts #EXTINF:3.96, segment_1_playlist_11.ts #EXT-X-ENDLIST #EXTM3U #EXT-X-PLAYLIST-TYPE:VOD #EXT-X-VERSION:5 #EXT-X-MEDIA-SEQUENCE:0 #EXT-X-TARGETDURATION:6 #EXTINF:6.0, segment_2_playlist_0.ts #EXTINF:6.0, segment_2_playlist_1.ts #EXTINF:6.0, segment_2_playlist_2.ts #EXTINF:6.0, segment_2_playlist_3.ts #EXTINF:6.0, segment_2_playlist_4.ts #EXTINF:6.0, segment_2_playlist_5.ts #EXTINF:6.0, segment_2_playlist_6.ts #EXTINF:6.0, segment_2_playlist_7.ts #EXTINF:6.0, segment_2_playlist_8.ts #EXTINF:6.0, segment_2_playlist_9.ts #EXTINF:6.0, segment_2_playlist_10.ts #EXTINF:3.96, segment_2_playlist_11.ts #EXT-X-ENDLIST
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp.exe -vU https://www.bild.de/video/clip/news-ausland/deftiger-abgang-vom-10m-turm-bademeister-sorgt-fuer-skandal-85158620.bild.html
[debug] Command-line config: ['-vU', 'https://www.bild.de/video/clip/news-ausland/deftiger-abgang-vom-10m-turm-bademeister-sorgt-fuer-skandal-85158620.bild.html']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[7cccab79e] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 2023-06-11-git-09621fd7d9-essentials_build-www.gyan.dev (setts), ffprobe 2023-06-11-git-09621fd7d9-essentials_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.18.0, brotli-1.0.9, certifi-2023.07.22, mutagen-1.46.0, sqlite3-2.6.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1864 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: 71c3f327a60b9cbbaa27ff92ed08abe76e600010298e392b98a41e2cdac634f8
yt-dlp is up to date<EMAIL_ADDRESS>[Bild] Extracting URL: https://www.bild.de/video/clip/news-ausland/deftiger-abgang-vom-10m-turm-bademeister-sorgt-fuer-skandal-85158620.bild.html
[Bild] 85158620: Downloading JSON metadata
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 85158620: Downloading 1 format(s): 0
[debug] Invoking hlsnative downloader on "https://hds.ak.token.bild.de/85158620,delivery=hls.m3u8"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 3
[download] Destination: Bademeister sorgt für Skandal [85158620].m3u8
[debug] File locking is not supported. Proceeding without locking
[download] 100% of    1.64KiB in 00:00:00 at 6.58KiB/s

The bild.de extractor isn't aware of pages which deliver a choice of an MP4 file and an M3U8, and doesn't parse the M3U8 (instead downloads it as a file).
A quick hack to ignore the M3U8:
diff --git a/yt_dlp/extractor/bild.py b/yt_dlp/extractor/bild.py
index f3dea33c4..426a78b60 100644
--- a/yt_dlp/extractor/bild.py
+++ b/yt_dlp/extractor/bild.py
@@ -27,11 +27,18 @@ def _real_extract(self, url):
         video_data = self._download_json(
             url.split('.bild.html')[0] + ',view=json.bild.html', video_id)

+        url = None
+        for src in video_data['clipList'][0]['srces']:
+            if src['type'] == 'video/mp4':
+                url = src['src']
+                break
+
         return {
             'id': video_id,
             'title': unescapeHTML(video_data['title']).strip(),
             'description': unescapeHTML(video_data.get('description')),
-            'url': video_data['clipList'][0]['srces'][0]['src'],
+            'url': url,
             'thumbnail': video_data.get('poster'),
             'duration': int_or_none(video_data.get('durationSec')),
         }

(Yes - there's no error checking, no fallbacks, and a lot of assumptions in this simple hack.)
In a proper solution, the type of each provided URL would need to be checked, and the M3U8 parsed in order to produce further formats and proper URLs.
A workaround for you right now is to throw the downloaded M3U8 at yt-dlp again:
yt-dlp "Bademeister sorgt für Skandal [85158620].m3u8"

As background information, here's part of the JSON this extractor expects and assumes:
    "clipList": [
        {
            "type": "main",
            "srces": [
                {
                    "src": "https://videos-world.ak.token.bild.de/BILD/38/18/41/46/38184146,property=Video.mp4",
                    "type": "video/mp4"
                }
            ]
        }
    ],

and here's what it's getting for the clip mentioned in this ticket:
    "clipList": [
        {
            "type": "main",
            "srces": [
                {
                    "src": "https://hds.ak.token.bild.de/85158620,delivery=hls.m3u8",
                    "type": "application/x-mpegURL"
                },
                {
                    "src": "https://hds.ak.token.bild.de/85158620,delivery=pmd.mp4",
                    "type": "video/mp4"
                }
            ]
        }
    ],

While I personally "hate" that website, I thought it might be easy to contribute a fix, so I posted a pull request.

I understand what you mean. I kinda dont like this publisher too. But i need it for work.


docs: Add Arabic translations

On which page do you see this issue?
No response
Describe the issue

This is a continuation of the discussion #2471

Please assign the issue to me
What browsers are you seeing the problem on?
No response

Thanks for the PR @souhaib-benbouzid

you're welcome, I'm happy to give back to the daisyUI. @saadeghi


Add configs for generating Blueprint files

[ ] Ability to configure certain endpoint groups from being excluded from generation
[ ] Generate blueprints for public or private endpoints

For configuring endpoint groups to be excluded from generation, we can go with something like this for the config file:
<generators>
  <blueprint>
    <excludes>
      <exclude group="SomeGroup" />
    </excludes>
  </blueprint>
</generators>

And for generating public or private endpoints, we can just do that as part of the generate command. Passing in --public should generate public, --private private, and nothing should do everything.


Is it possible to use chatgpt's API instead of gpt4's?

Dear author,
in performing the second step, I don't have the API for gpt4, can I use the API for chatgpt instead?

Yes, sure. In our experiments, when compared against ChatGPT4, ChatGPT3.5 is slightly less accurate as a judge so I would suggest manually verifying a few annotations before running it for large-scale experiments.


docs: update Ruby version, tutorial notes for multi-containers

Changes:

Update Ruby version from 2.7.4 (EOL) to 3.2.2 (latest)
Add explicit rack server (probably due to major Ruby version update)

Doc updates:

Added optional note around using Secret Manager rather than hardcoding password as an environment variable, linking to docs
Added note around how the default service account will have permission to connect to Cloud SQL, but explicit permission required for custom service accounts
Fixed filename on gcloud run command

Mirror of changes to https://github.com/GoogleCloudPlatform/cloud-sql-proxy/pull/1813 (also includes other potential improvements)

Thanks once again @glasnt, appreciate this change greatly! 😄 👏


Make toggles internal in default and local value providers

Summary
This PR makes toggles internal in DefaultValueProvider & LocalValueProvider.
This change is needed so the property can be accessed when using @testable import to iterate over all toggles for testing purposes.

The PR description states that the internal access control is needed for testing purposes but no tests have been added. I suspect that you need this because you intend to write some tests externally to the Toggles' test target, i.e. you want to use @testable import Toggles in an unrelated test suite breaking incapsulation?

That's a fair point, yes! I also realised we only need a list of all the variables and not the toggles dictionary. Do you think it's a reasonable change to expose that list (as in public let variables)?
I think it could be helpful whenever you want to iterate over variables, whether it's for testing purpose or not.


Updating the database client

Would it be possible to update the database client so we can use MySQL 8? Currently, it does not work due to unsupported authentication mode.

Has this been implemented? I can't seem to find any documentation or dev versions that have this.


chore(): update readme

Updates the readme to be more specific regarding the purpose of the project, rather than how it works.

LGTM


[Bug] Undefined emit when stopping record in recorder

Refs: https://github.com/medooze/media-server-node/blob/master/lib/Recorder.js#L132

thank you for reporting it and for the patch!

fixed on v0.81.0


Add workflow to create infra-deployments PRs

https://issues.redhat.com/browse/HACBS-1773

Requires:

https://github.com/hacbs-contract/ec-cli/pull/413
https://github.com/hacbs-contract/ec-policies/pull/330
https://github.com/hacbs-contract/enterprise-contract-controller/pull/66

Requires:

https://github.com/hacbs-contract/ec-cli/pull/413
https://github.com/hacbs-contract/ec-policies/pull/330
https://github.com/hacbs-contract/enterprise-contract-controller/pull/66

I just realized that the secrets are set at the organization level. So no action is needed here. Marked those items as complete.

All requirements met. Going to merge it so I can try it out.


2.0.2

I missed adding a tag and bumping the version in #62
Adding that here plus publish and release this commit

looks good! meant to be a draft?

looks good! meant to be a draft?

whoops, nope.  ready for review!


Double Labeling Problem

objects being detected    multi-label。一个对象有多标签，如何处理，比如Cascade R-CNN

@fcqfcq You may need to rewrite some parts of the pipeline and model head.


Consolidate bundling

Combines external modules into a single compatibility layer bundle at convert time. Only bundles modules used in the output script. This should eliminate redundant code from shared dependencies of external modules.
Runs tinyify on the final bundle to optimize.
Closes #25.

Looks good overall. I run into an issue though when I have collections with scripts that are using the shim'ed libs, eg. expect.js (https://github.com/bookmoons/postman-to-k6/blob/bookmoons/consolidate-bundling/lib/shim/expect.js). Then I get this error:
GoError: stat /<removed>/postman-to-k6/libs/chai.js: no such file or directory
The output script is piped to /<removed>/postman-to-k6/. Looks like the libs in https://github.com/bookmoons/postman-to-k6/blob/bookmoons/consolidate-bundling/lib/shim/ are trying to load unbundled files.

Shoot, will look into this.

This will remove the need for k6 scripts to import the library shims / not have those in the same repo correct?

@johnhiggs No, this PR is for all shims and external libs to be put into one JS file rather than several different ones. It will also only include the external libs actually needed to run the output script, rather than all of them as is currently the case. Removing the need for external libs will require adding native support in k6 for these libs/APIs or by adding the relevant libs as curated libs to https://jslib.k6.io/ (can be used as remote modules or downloaded for use as local modules). The latter being the path we're currently aiming for.

Awesome thanks for clarifying
On Fri, Sep 20, 2019 at 12:02 PM Robin Gustafsson<EMAIL_ADDRESS>wrote:

@johnhiggs https://github.com/johnhiggs No, this PR is for all shims
and external libs to be put into one JS file rather than several different
ones. It will also only include the external libs actually needed to run
the output script, rather than all of them as is currently the case.
Removing the need for external libs will require adding native support in
k6 for these libs/APIs or by adding the relevant libs as curated libs to
https://jslib.k6.io/ (can be used as remote modules or downloaded for use
as local modules). The latter being the path we're currently aiming for.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/loadimpact/postman-to-k6/pull/41?email_source=notifications&email_token=ACTIN6OB4PV5KIV4AHGJ45DQKTXXTA5CNFSM4IO56TI2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7HE5VI#issuecomment-533614293,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACTIN6IZYKPDR3Q46YM2V2LQKTXXTANCNFSM4IO56TIQ
.

Went over this more carefully. It has the problem you found and it was also not quite bundling everything. Pushed fixes for all of this.
But this has revealed an issue. One of the dependencies uses Object.setPrototypeOf(), which seems to be unavailable in k6. Going to look into this.

Great, now it works better! 🎉
Regarding Object.setPrototypeOf(), maybe this would need an update of the core-js version we're using in k6. We've been trying to update core-js (and Babel.js) for some time now in k6, but I believe we ran into some issues with it. Let me check with the k6 team and I'll get back to you. If the update of core-js in k6 will take some time maybe we could polyfill ([1] or [2]) Object.setPrototypeOf() in postman-to-k6?
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/setPrototypeOf#Polyfill
[2] https://github.com/wesleytodd/setprototypeof

Nice, yes. Maybe that could also eliminate that rough edge in the shim, where it couldn't quite get a clean prototype.

Opened #43 for the prototype issue.

Made a submission that should resolve the prototype issue #44. After that's settled I can rebase this patch onto it.

Rebased this onto the prototype fix. Tests pass. The polyfill now ends up bundled in the compat layer. This collection that loads all shim components converts and executes successfully. I think it's in good shape.
{
	"info": {
		"_postman_id": "9b0cc8d3-04d1-4196-87a0-3af92541f760",
		"name": "Depends",
		"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
	},
	"item": [
		{
			"name": "TestDepends",
			"event": [
				{
					"listen": "prerequest",
					"script": {
						"id": "f3c1e91d-0c0c-481f-b841-a5d067d40a48",
						"exec": [
							""
						],
						"type": "text/javascript"
					}
				},
				{
					"listen": "test",
					"script": {
						"id": "d7a05bac-89c9-4c48-aadc-472bc6631e6d",
						"exec": [
							"const cheerio = require('cheerio')",
							"const _ = require('lodash')",
							"const cryptoJs = require('crypto-js')",
							"",
							"const json = xml2Json(\"<html></html>\")",
							"pm.test('test body', function () {",
							"    pm.expect('Response body').to.include('Response')",
							"    pm.response.to.have.jsonSchema({})",
							"})",
							""
						],
						"type": "text/javascript"
					}
				}
			],
			"request": {
				"auth": {
					"type": "awsv4",
					"awsv4": [
						{
							"key": "secretKey",
							"value": "secret",
							"type": "string"
						},
						{
							"key": "accessKey",
							"value": "key",
							"type": "string"
						}
					]
				},
				"method": "GET",
				"header": [],
				"url": {
					"raw": "{{protocol}}://example.com?Action=List",
					"protocol": "{{protocol}}",
					"host": [
						"example",
						"com"
					],
					"query": [
						{
							"key": "Action",
							"value": "List"
						}
					]
				}
			},
			"response": []
		}
	],
	"variable": [
		{
			"id": "6148a217-db3c-4c2d-9807-992aab6ec48b",
			"key": "protocol",
			"value": "https",
			"type": "string"
		}
	],
	"protocolProfileBehavior": {}
}

Yup, this works now. I have tested it as well 👍
I'd like to see one improvement though. Right now this is what the import list looks like in the output script from the Postman collection you posted above:
import "./libs/shim/core.js";
import "./libs/shim/cheerio.js";
import "./libs/shim/crypto-js.js";
import "./libs/shim/lodash.js";
import "./libs/shim/expect.js";
import "./libs/shim/jsonSchema.js";
import "./libs/shim/xml2Json.js";
import "./libs/shim/urijs.js";
import { aws4, URI } from "./libs/compat.js";

and all the shims (well, except libs/shim/core.js) follow this pattern:
import { URI } from '../compat.js'
const Extend = Symbol.for('extend')
postman[Extend].module.urijs = URI

could we instead inject these 3 line glue code snippets into libs/shim/core.js (an renaming the shim layer libs/shim.js)? So injecting something like this:
// inject as imports
import { cheerio, cryptoJS, lodash, expect, jsonSchema, xml2json, URI } from '../compat.js'

// this line would be redundant in libs/shim/core.js
const Extend = Symbol.for('extend')

// inject these at an appropriate place after Extend is defined
postman[Extend].module.cheerio = cheerio
postman[Extend].module.cryptoJS = cryptoJS
postman[Extend].module.lodash = lodash
postman[Extend].module.expect = expect
postman[Extend].module.jsonSchema = jsonSchema
postman[Extend].module.xml2json = xml2json
postman[Extend].module.urijs = URI

This would mean the libs output would only be 2 files, libs/shim.js and libs/compat.js, rather than all the small libs/shim/*.js files that are being output right now.

Will have a look at this.

As discussed offthread, I'll consolidate the shim to get all external deps into a single file.

I'm hitting issues here that are not yielding easily. There's some kind of interaction that seems to hit a memory leak. I'll have to put this aside for now.


L0 cover decision

Oriented to postal code base16
Initial choice, with no "ghost cell"

Optimized choice, using 3 "ghost cells":  small fragment  of 1J, 4J and  5K (design on base16h).

Oriented to postal code base32
(perhaps non-valid for scientific codes - need squares on hexadecimals).
Design on base16h... Using 3 "ghost cells": small fragment  of  3Z, 5R and  4P.

Dicision on meeting of 2022-08-24 with UY:

postal codes base32
country cover oriented to base16,  so  the cover with 16 square cells + 3   "ghost cells"


EPIC Beta 1:  chatgpt-ui

Make the development version running of chatgpt-ui both server and ui part up and running on a container ready to be tested out.

YES success!


UI: Custom-role (rules) listVirtualMachines is not enforced

ISSUE TYPE

BUG

COMPONENT NAME
Component: API, UI
CLOUDSTACK VERSION
Cloudstack 4.18
SUMMARY
UI: Custom-role  (rules) listVirtualMachines is not enforced
Steps to reproduce the issue

Create a account with the default role (user)

screenshot

Create a custom role

screenshot

Login with the normal user account  and deploy a vm

With admin user credentials  change the account role from user to custom role

(cmk) > update account roleid=e2251d81-7e2b-4f1b-bcd0-e404de8426ce  account=kiran domainid=489f928d-26c7-11ee-b6ac-1e00a8000339

With admin user crdentials  Modify the rules in custom role ( for example  Deny )

screenshot

For example :
I have updated the listvirtualmachine permission to deny for the custom-role
(cmk) > update rolepermission roleid=e2251d81-7e2b-4f1b-bcd0-e404de8426ce ruleid=1c55b696-0f22-4302-93fe-da44aadcf282 permission=deny
{
"success": true
}

With normal user credentials Execute the list virtual machines

Expected behaviour
(cmk) > list virtualmachines
Error: (HTTP 432, error code 9999) The API [listVirtualMachines] does not exist or is not available for the account Account [{"accountName":"kiran","id":5,"uuid":"25753675-308f-4c19-b659-51482966201a"}

Login to the Ui with ( normal user)

Able to see the virtual machines
Actual behavior
Normal user is able to see the virtual machines , even though the listVirtualMachines api is not working
Expected behavior
Normal user should not be able to see the virtual machines as the listVirtualMachines api is not working

@kiranchavala - from what I can tell, this is because your role doesn't have a deny all at the end. The UI doesn't use listVirtualMachines API to generate this page. It uses listVirtualMachinesMetrics API.
One solution could be to deny listVirtualMachines* in your role. Or add a Deny * at the end (which would block all API calls not listed).

@mlsorensen thanks Adding a deny rule for listVirtualMachinesMetrics solved the issue


Switch subscription resumption test app to lit-icd-app

https://github.com/project-chip/connectedhomeip/pull/32496 accidentally disable lit-icd-app tests for subscription resumption. We should switch the subscription resumption test to use lit-icd-app.

@wqx6 @yunhanw-google I'm not sure this is a good idea. LIT ICD app doesn't enable the timeout resumption feature since it conflicts with the Check-In Protocol.

@wqx6 @yunhanw-google I'm not sure this is a good idea. LIT ICD app doesn't enable the timeout resumption feature since it conflicts with the Check-In Protocol.

Agree. Thanks
Then we just merge the change without replacing all cluster app with lit example app, since the rest of change are still valuable for later lit test later. Thanks

Done


Implement leak detection and troubleshooting tool for Dart and Flutter.

Peer issue in DevTools: https://github.com/flutter/devtools/issues/3951

@polina-c is this still a P1 issue? It hasn't been updated in a while.

Yes, it is. There are many artifacts around it that are updated. For example, original issue in DevTools.

Closing, as it is adopted for Flutter Framework.


Redis

Hello,
First of all, thanks for your repo, it makes learning Docker with Symfony a bliss :)
I just have a question concerning the Redis part.
When using the line redis_host: redis I have the following error :
Invalid configuration for path "snc_redis.clients.doctrine.dsns.0": The redis DSN "redis" is invalid.
I had to use redis_host: redis://redis instead.
I'm still learning how to use Docker, am I missing something ?
I'm using Symfony3 and SncRedisBundle.
Thanks

Hello @benjaminjonard ,
If you are in the PHP container (docker-compose exec php bash), try to ping the redis container.
Here, is my result:
root@597b06d48146:/var/www/symfony# ping redis
PING redis (<IP_ADDRESS>): 56 data bytes
64 bytes from <IP_ADDRESS>: icmp_seq=0 ttl=64 time=0.106 ms
64 bytes from <IP_ADDRESS>: icmp_seq=1 ttl=64 time=0.070 ms
64 bytes from <IP_ADDRESS>: icmp_seq=2 ttl=64 time=0.067 ms
64 bytes from <IP_ADDRESS>: icmp_seq=3 ttl=64 time=0.077 ms
^C--- redis ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.067/0.080/0.106/0.000 ms

So, redis host is reachable.
If you are using the SncRedisBundle, you have to fill the configuration file like this:
#config.yml
snc_redis:
    clients:
        default:
            type: predis
            alias: default
            dsn: redis://localhost

In my README.md, I use something like that (in the parameters.yml file):
# path/to/your/symfony-project/app/config/parameters.yml
parameters:
    redis_host: redis

So, you need to adapt the config file like this:
snc_redis:
    clients:
        default:
            # ...
            dsn: redis://%redis_host%

In my humble opinion, it's not a good idea to put the full IP address/dsn in a config file. This is why I was using the parameters.yml.
Kind regards,
Maxence
Ps: close if everything is ok for you!


issue when running bugscanner

~ $ bugscanner
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/bin/bugscanner", line 33, in
sys.exit(load_entry_point('bugscanner==0.1.8', 'console_scripts', 'bugscanner')())
File "/data/data/com.termux/files/usr/bin/bugscanner", line 25, in importlib_load_entry_point
return next(matches).load()
File "/data/data/com.termux/files/usr/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/data/data/com.termux/files/usr/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 664, in _load_unlocked
File "", line 627, in _load_backward_compatible
File "", line 259, in load_module
File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/bugscanner-0.1.8-py3.9.egg/bugscanner/init.py", line 1, in
File "", line 259, in load_module
File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/bugscanner-0.1.8-py3.9.egg/bugscanner/bug_scanner.py", line 1, in
File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/multithreading/init.py", line 1, in
from multithreading.logger import Logger
File "/data/data/com.termux/files/usr/lib/python3.9/site-packages/multithreading/logger.py", line 6, in
from loguru import logger
ModuleNotFoundError: No module named 'loguru'

install loguru module first,
$ pip install loguru

and install module again if something like this happend.

Another error occurred
~/bugscanner/bugscanner $ python direct_scanner.py
Traceback (most recent call last):
File "/data/data/com.termux/files/home/bugscanner/bugscanner/direct_scanner.py", line 1, in
from .bug_scanner import BugScanner
ImportError: attempted relative import with no known parent package

Not like that. First, you need to install this script, don't direct run like that
pip install bugscanner

and then use command like in this tutorial here


Peer to Peer Code Review - 08/17/2023

You have done really a great job while designing and coding the whole wireframe. Great!

I Appreciate your feedback and review of my awesome-book App 🚀.
THANK YOU 😀.


Add provisions to protect form a large number of instances started simultaneously

This might be in the form of touching a temporary/hidden file and making sure there is a grace period between processes sending data

Released in Chronicle-Analytics-2.23ea0, BOM-2.23ea16

Released in Chronicle-Analytics-2.23ea0, BOM-2.23ea17