[SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

What changes were proposed in this pull request?
This pr fix push foldable into CaseWhen branches if elseValue is empty. For example:
spark.sql("CREATE TABLE t1(bool boolean, number int) using parquet")
spark.sql("INSERT INTO t1 VALUES(false, 1)")

spark.sql(
  """
    |SELECT *
    |FROM   (SELECT *,
    |               CASE
    |                 WHEN bool THEN 'I am not null'
    |               END AS conditions
    |        FROM   t1) t
    |WHERE  conditions IS NULL
    |""".stripMargin).show
}

How do we optimize the filter conditions before this pr:
Filter isnull(CASE WHEN bool#7 THEN I am not null END) -> Filter CASE WHEN bool#7 THEN isnull(I am not null) END -> Filter (bool#7 AND isnull(I am not null)) -> Filter (bool#7 AND false)

How do we optimize the filter conditions after this pr:
Filter isnull(CASE WHEN bool#7 THEN I am not null END) -> Filter CASE WHEN bool#7 THEN isnull(I am not null) ELSE isnull(null) END -> Filter CASE WHEN bool#7 THEN false ELSE isnull(null) END -> Filter CASE WHEN bool#7 THEN false ELSE true END -> Filter NOT (bool#7 <=> true)

Why are the changes needed?
Fix correctness issue.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Unit test.

Test build #145185 has started for PR 34580 at commit 9acd0a9.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49654/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49654/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49654/

Test build #145187 has started for PR 34580 at commit 76982eb.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49656/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49656/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49656/

Test build #145185 has finished for PR 34580 at commit 9acd0a9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145185/

Test build #145189 has started for PR 34580 at commit d66a76d.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49658/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49658/

Test build #145187 has finished for PR 34580 at commit 76982eb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145187/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49658/

Test build #145189 has finished for PR 34580 at commit d66a76d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145189/

Test build #145196 has started for PR 34580 at commit e2a9c43.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49665/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49665/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49665/

Test build #145196 has finished for PR 34580 at commit e2a9c43.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145196/

Test build #145227 has started for PR 34580 at commit 1e47f6b.

Test build #145227 has finished for PR 34580 at commit 1e47f6b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49697/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145227/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49697/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49697/

Test build #145234 has started for PR 34580 at commit f01d931.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49704/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49704/

Test build #145240 has started for PR 34580 at commit e77ffb4.

Test build #145234 has finished for PR 34580 at commit f01d931.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145234/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49710/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49710/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49710/

Test build #145242 has started for PR 34580 at commit 26436d2.

Test build #145242 has finished for PR 34580 at commit 26436d2.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49712/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49712/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49712/

Test build #145240 has finished for PR 34580 at commit e77ffb4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145240/

Test build #145258 has started for PR 34580 at commit 31f5a50.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49728/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49728/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49728/

Test build #145258 has finished for PR 34580 at commit 31f5a50.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145258/

Make sense.

Test build #145297 has started for PR 34580 at commit 237f8a8.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49767/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49767/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49767/

thanks, merging to master/3.2!

@wangyum can you open a backport PR for 3.2?

OK. PR for 3.2: https://github.com/apache/spark/pull/34627

Test build #145297 has finished for PR 34580 at commit 237f8a8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145297/


Authentication error when using Tag Build task

I am getting the following error when I try to tag a Build from a Release definition.  I am not sure where to enable OAuth in the Build/Release options.
##[error]Could not find token for autheniticating. Please enable OAuth token in Build/Release Options.
I do have the security set as mentioned in this link (https://github.com/colindembovsky/cols-agent-tasks/tree/master/Tasks/TagBuild)

I did find the "Allow Scripts to Access OAuth Token" on the build definition and it was already checked.
Is there a similar option on the Release definition, because I don't see an Options tab.

Found it: https://stackoverflow.com/questions/41004090/vsts-allow-scripts-to-access-oauth-token-in-release-managment

Hi @mayeager22 - glad you got it sorted!


generalize cmwell-formats

Currently, we can render all Formattables to Strings. This is not enough.
E.g: To support streaming end to end, we might want to render output as Source[ByteString,_].
Also, input doesn't have to be a strict value. we can also wrap compound objects.
I suggest we use a type class based approach, similar to akka.http.impl.util.{Rendering,Renderable}, and thus enable to render to anything else we might want in the future.
quick draft of what I have in mind:
// instead of:
trait Formattable

case class SearchResults(
  fromDate: Option[DateTime],
  toDate: Option[DateTime],
  total: Long,
  offset: Long,
  length: Long,
  infotons: Seq[Infoton],
  debugInfo: Option[String] = None) extends Jsonable with Formattable {
    // ...
}

// maybe we can do something along the lines of:
trait Formattable[S] {
  def inner(): List[Formattable[S]] = Nil
  def intersperseSupport(): (Option[S],S,Option[S])
  def format(implicit rend: Rendering[S,R]): R = {
    val (head,sep,tail) = intersperseSupport()
    val h: R = head.fold(rend.empty[R])(rend.render)
    val m: R = inner().foldLeft(h)((r,f) => rend.append(r,sep,f))
    tail.fold(m)(t => rend.append(m,sep,t))
  }
}

case class SearchResults[S : Renderable](
  fromDate: Option[DateTime],
  toDate: Option[DateTime],
  total: Long,
  offset: Long,
  length: Long,
  infotons: List[Infoton],
  debugInfo: Option[String] = None) extends Jsonable with Formattable[S] {
    override def inner() = infotons
    override def intersperseSupport() = {
      val rend = implicitly[Renderable[s]]
      val sep = rend.separator
      val l = List(total,offset,length).map(rend.mkRenderable)
      val t = toDate.fold(l)(d => rend.mkRenderable(d) :: l)
      val f = fromDate.fold(t)(d => rend.mkRenderable(d) :: t)
      val s = f.foldRight(List.empty[S])((s,l) => sep :: s :: l).tail
      (Some(rend.fromList(s)),sep,debugInfo.map(rend.mkRenderable))
    }
    // ...
}

So, here S can be a jena RDF statement, and Formattable.format could take all the statements and either pass it through a regular jena model to output a String or Array[Byte], or pass it through some streaming engine (I'm looking at you alpakka: https://github.com/akka/alpakka/issues/159 ) to output a Source[ByteString,_], according to the implicitly passed Rendering instance.
Another option, is treat S as JsValue, which will output a json, either fully structured, or streamed.
This approach, of breaking the render process into 2 unrelated tasks of preparing the input and then rendering it, can be very powerful, and will enable wide range of usages. For example, infotons does not need to be a List, it can be a Source... we can abstract over the container, as long as we have the correct Renderable in scope...

Wow. +1. Sounds like a plan. @YoniMataraso, if I may, I suggest to prioritize it.

Seen too late - so opened this one https://github.com/thomsonreuters/CM-Well/issues/547


discoverReaders() returns empty list

Summary
terminal.discoverReaders() always returns an empty list. Upon further debugging, I figured out that the value of one of the query parameters is causing the issue.

Any value for compatibile_sdk_version that contains a PATCH number (the third one) makes the API to return an empty data array. It appears that the SDK sends version 1.3.2 to the API which does not work as mentioned.

If you use a different version number such as 1.3, you will get the complete list of readers.

To reproduce the bug:

Create a connection token.
Initialize the SDK.
Discover readers (with simulated: false):

const config = {simulated: false};
const discoverResult = await terminal.discoverReaders(config);
if (discoverResult.error) {
  console.log('Failed to discover: ', discoverResult.error);
} else {
  console.log(discoverResult.discoveredReaders);  // Bug: always empty
}

Hey @mahmoudsaeed thanks for the feedback, that reader being returning the simulated_wisepos_e is not intended to be used with the JS SDK and will only show up for our api based integrations. If you order and register hardware to your account you will see it returned in the list when simulated is false.


Service bus Module

Servicebus module inclusive of :

servicebus namespaces, auth rules and network rule set
servicebus topics, subscription and topic auth rules
servicebus queues, auth rules

Future enhancement: enhancement for queue and subscription forward_to and forward_dead_lettered_messages_to parameters currently only take in full string value, instead of key map reference.

hi @jorseng, this looks great, can you please have a look at https://github.com/aztfmod/terraform-azurerm-caf/pull/731 to get best of both worlds?


Append relative path when using local tag

I want to use this package in a mono-repo, but I have run into a problem: when using the [local] tag, the output code does not have the relative path appended. Example:
Input .prettierignore-sync:
[local]
.gitignore
packages/benchmarks/.gitignore

Output .prettierignore:
############################################
# GENERATED BY IGNORE-SYNC, DO NOT EDIT!!! #
# https://github.com/foray1010/ignore-sync #
############################################

node_modules/
lerna-debug.log

generated/
generator/build/
generator/generated/

But I expect this output:
############################################
# GENERATED BY IGNORE-SYNC, DO NOT EDIT!!! #
# https://github.com/foray1010/ignore-sync #
############################################

node_modules/
lerna-debug.log

packages/benchmarks/generated/
packages/benchmarks/generator/build/
packages/benchmarks/generator/generated/

Otherwise it gives me conflicts with other packages that I don't want to ignore.

Why you need to copy the package's .gitignore to the root .gitignore? Both your /.gitignore and /packages/*/.gitignore will be resolved by git

It's for Prettier & ESLint, not for git.

The reason of this behavior is because ignore-sync treat those dependencies as template that are for copy and paste, so the path inside is not relative to the destination file
For example, /packages/a/.npmignore can copy the /.gitignore directly, without messing up the files with prefix ../../.
And user can define file structure like the following code, so that the ignore files can be stored in different directory
[local]
.config/commonignore

It works the same for remote config
[github/gitignore]
Node.gitignore
Global/macOS.gitignore

Therefore, it does not work the way that solve your problem.
With that being said, we can introduce a new tag like [relative] for this different behavior, if you wanna help coding it.

Without having read your comment yet, I was just submitting this proposal: #301

@foray1010 PR updated with your comments!

3.0.0 released🎉


setminus and smallsetminus

unicode-math defines the following mappings:
0x2216 -> \smallsetminus
0x29F5 -> \setminus

This appears to a bit strange to me. Unicode defines 2216 as "set minus", so shouldn't 2216 map to \setminus? This is pointed out in Will's tugboat article, but I do not understand the rational behind the choice made my unicode-math.

We've just bumped up against this incompatibility during development of the STIX Two fonts: https://github.com/stipub/stixfonts/issues/179. (I'm not sure what took us so long.)
Would anyone from here like to comment on that proposal?

Another unexpected interaction I found (clearly a font issue) is with the New CM fonts where both character are actually the small set minus, so I actually have to redefine \setminus back to the normal definition of \mathbin{\backslash}.

Thanks for the decades of commentary here. I'm sorry that I dug my heels in so much about this -- in hindsight, it would have been better to make the change originally proposed to correspond to Unicode as much as possible.
My solution to this issue is going to be exactly that -- use "2216 for \setminus, fake \smallsetminus, and anyone who needs something different can use some one-line redefinitions (such as \renewcommand\setminus{\reversesolidus} ).

Thanks for the decades of commentary here. I'm sorry that I dug my heels in so much about this -- in hindsight, it would have been better to make the change originally proposed to correspond to Unicode as much as possible.
My solution to this issue is going to be exactly that -- use "2216 for \setminus, fake \smallsetminus, and anyone who needs something different can use some one-line redefinitions (such as \renewcommand\setminus{\reversesolidus} ).

I know it's generally preferred to use GitHub reactions to express gratitude, but they easily get lost so I want to take the short time to say thank you for coming around with a (simple) solution—as well as all the other maintenance work.
I'm looking forward to deleting a lot of code from my personal class to make my documents compile w/o missing symbols 0:-)
Thanks again and have a great day!


#111 - registerHandlerClass:forRoute: interface is generated for Swift 3

Looks like specialized Class is not interfacing with Swift and declaration for
- (void)registerHandlerClass:(Class <DPLRouteHandler>)handlerClass forRoute:(NSString *)route;

can't be generated.

Once it's green, LGTM

looks like travis is running on CocoaPods 1.0 and the lock is on 0.39.0 so the Podfile is out of date.

From travis logs:
$ pod --version
0.34.4

I'll take care of this.


文章发表时间会随着文章更新而更新为最新时间怎么解决

如题

将大特固定就好

换了一台电脑，现在正常了

NexT is rebased into organization repo.
If you want new feature, fix, or support, create new in NexT v6.x repo (desirable in English).
There is instructions on English or Chinese how to update from v5.1.x to v 6.x
You also may read this for details.


feat: disable collab feature when running in iframe

This will disable starting collaboration sessions and hide the collaboration button in the UI. This button will also be hidden until the collabAPI is initialized.

This is a comparison of UI in Notion notebook using current production (top one) with this branch (bottom one)

👍
As discussed on call:

let's disable rendering the <Collab/> component at all when inside iframe (thus removing the checks inside the Collab component itself). This will also make it so that collabAPI is not set, so we'll need to account for that in the functions and components that depend on it.

This is updated and ready for review, @dwelle

Shipping, thanks! ❤

hey there,
i was using excalidraw inside of workadventure - an open source, virtual office-thingy
https://github.com/workadventure/workadventure
In there you can load different tools as iframes - excalidraw was a perfekt fit for that!

I started seeing issues, when two or more people wanted to work together -> i guess this is because of this PR.
I was using the direct collabobration links (#room=...)
Is there any chance this feature comes back, maybe behind a feature flag or by self-hosting?
regards and thanks for this nice tool :)


Add missing GPIO pins

Adds (all?) previously missing GPIO pins
Set all of the ESP32-C3's GPIO's as NOTOUCH, as it does not have a touch controller
Mention in the README which pins are not recommended for use

Closes #8

If you feel like it- please merge and publish a new patch release whenever convenient for you.

Version 0.29.4 has been published on crates.io.


Fix logo filename and add a feature to rename new image with his MD5 hash

Hi, I'm not introducing more changes to the branch master because I'm working on Bludit v4 in branch v4.0.
Why do you want to change the filename to a hash name?

Hi, I'm not introducing more changes to the branch master because I'm working on Bludit v4 in branch v4.0.
Why do you want to change the filename to a hash name?

This is a need for my users who dot not pay attention to the filename before upload it. Sometimes the filename contains critical information which could help someone to identify a place or a person. I need to avoid that. Also, I'm working to delete exif metadata from each uploaded pictures to enforce the security/anonymity. This feature is a default feature for some web applications/cms/website/blog. I believe, if an author/publisher want to tell something about a place or a person, this have to be intentionally, not accidentally. I will only allow that on Bludit.

This is a need for my users who dot not pay attention to the filename before upload it. Sometimes the filename contains critical information which could help someone to identify a place or a person. I need to avoid that. Also, I'm working to delete exif metadata from each uploaded pictures to enforce the security/anonymity. This feature is a default feature for some web applications/cms/website/blog. I believe, if an author/publisher want to tell something about a place or a person, this have to be intentionally, not accidentally. I will only allow that on Bludit.

Hello, with the new version of Bludit v4.0 rc1, I would like to close the old Github issues. If you feel that your issue is not resolved in the latest version, create a new ticket.

Help and Support use the Forum https://forum.bludit.org
Bugs and new requests here in Github https://github.com/bludit/bludit/issues


[Bug] Write Reusable JavaScript with Functions

Needs to update description and test
Link: https://learn.freecodecamp.one/javascript-algorithms-and-data-structures/basic-javascript/write-reusable-javascript-with-functions
Scope: https://github.com/huluoyang/learn/blob/master/curriculum/challenges/02-javascript-algorithms-and-data-structures/basic-javascript.json#L2330-L2356
Original: https://github.com/freeCodeCamp/curriculum/blob/dev/challenges/02-javascript-algorithms-and-data-structures/basic-javascript.json#L2993

Thanks for pointed out this. I've fixed it.


[Bug]:  ssm_managed_compliant_patching: value of FindingUniqueId changes but the resource is the same

Steps to Reproduce
Just run Prowler scan that includes this check
Expected behavior
I expect that for the same resource I will get the same id (item["ResourceId"]).
Actual Result with Screenshots or Logs
resource_id seems to be changing for any new query for resource summary, even if the resource remains the same. That breaks the fingerprint.
Relevant code:
def __list_resource_compliance_summaries__(self, regional_client):
        logger.info("SSM - List Resources Compliance Summaries...")
        try:
            list_resource_compliance_summaries_paginator = (
                regional_client.get_paginator("list_resource_compliance_summaries")
            )
            for page in list_resource_compliance_summaries_paginator.paginate():
                for item in page["ResourceComplianceSummaryItems"]:
                    resource_id = item["ResourceId"]
                    resource_status = item["Status"]

                    self.compliance_resources[resource_id] = ComplianceResource(
                        id=resource_id,
                        status=resource_status,
                        region=regional_client.region,

For each new summary query, item["ResourceId"] refers to to the new summary id and not to the resource id. When it’s saved to resource_id, it breaks the fingerprint, since it practically points to other resource, while it’s the same one.
How did you install Prowler?
RUN pip3 install prowler==$CONTROL_VERSION
Environment Resource
Fargate batch
OS used
linux debian slim bookworm image (https://github.com/docker-library/python/blob/81b6e5f0643965618d633cd6b811bf0879dee360/3.9/slim-bookworm/Dockerfile#L7)
Prowler version
3.8.2
Pip version
23.0.1
Context
No response

Hi @aviramshm, I can't reproduce the issue on our side. What is the ResourceType for the items whose ResourceId changes?
For the ManagedInstance type the ResourceId is always the same with the EC2 Instance Id.

Hi @aviramshm, do you have any updates on this? Thanks!

Hi @jfagoagas thanks for responding and sorry for the delay on our side,
the ResourceType is AwsEc2Instance as declared in the check's json link.
The scenario was that we ran the tool once a day, and for each run we got different ResourceId  about the same instance.
Thanks

Hi @aviramshm @avichayJit we cannot reproduce this on our side. Please try again with Prowler 3.11.2 which is the latest release and feel free to reopen it if you notice the same behaviour.
Thanks for using Prowler 🚀


support for SQLite as an embedded store

Gazette currently supports RocksDB and simple JSON files as embedded / in-process stores backed by recovery logs. Scope of this ticket is to enable SQLite as a "core" store. Goals:

Native support for the full range of SQLite engine capabilities: multiple database (ATTACH), multi-DB transactions (supporting SQLite's master / subordinate journal scheme), WAL journaling mode, etc.
Use of the canonical https://github.com/mattn/go-sqlite3 driver.
Fully backed by a recovery logs, with a tight hinted recovery bound.

Not a current goal, but interesting follow-up work: the ability to express additional streaming concepts as SQL using SQLite vtables (eg, INSERT INTO my_output_topic SELECT msg.id, foo.enriched_value FROM foo WHERE foo.id = msg.id).
The primary challenge is that SQLite uses very long-lived database files which are randomly written and read (though always on page-aligned boundaries). This doesn't play well with recovery logs, as a recovering process must begin reading from the very first DB file write, and play back all operations thereafter, even if all portions of the DB file have been over-written many many times since then. We must somehow "compact" the representation of DB files within the recovery log to bound the amount of log a recovering process must read.
The strategy used by this store implementation is the following:

A Sqlite VFS with cgo bindings implements appropriate hooks to record SQLite DB changes as they're made.
Transaction logs (SQLite journals and WALs) are recorded as write-once, append-only files. The implementation will account for the various strategies SQLite uses with journals (PERSIST/TRUNCATE vs DELETE, re-writing the WAL from byte zero), mapping each to delete-and-recreate operations.
Each attached SQLite DB is implemented as a ColumnFamily of a recorded RocksDB "page DB", which stores pages of the DB file keyed on offset (recall all DB reads and writes are page-aligned).

Open question: how much will use of RocksDB impact the performance of SQLite (as compared to a regular DB file)? This will need to be assessed. The "page" DB can be optimized for point-lookups, in-place memtable updates, memory mapped reads, and other tricks to minimize copying, etc. The binding will be in C++ such that SQLite pager does not need to traverse the cgo barrier to access pages from the DB. My suspicion is that, with tuning, the overhead will be completely acceptable for almost all use cases. As a fallback, the store could employ a strategy of writing both into the page DB and also the raw DB file. Reads / mmap fetches would be directed to the raw file. The downside is this doubles node disk usage and slows recovery (as the DB file would need to be re-built on shard startup).

done!


[x64] clean up internal uses of result_type()

Why? For X64 deprecation (#8178) I am working on some guardrails around dtype canonicalization, so we want to avoid extra calls to canonicalize_dtype that will generate noise.
This change is intended to be a pure refactoring, with no user-visible impact. It relies on two facts:

dtypes.result_type canonicalizes the output, so there is no need to call canonicalize_dtype again
dtypes.result_type(*args, return_weak_type_flag) is essentially equivalent to dtypes._lattice_result_type(*args) followed by dtypes.canonicalize_dtype on the output, so we can replace this common pattern with a single dtypes.result_type call.

Closing, as this has gone stale.


[WIP]docs: update gzip docs

What type of PR is this?

docs
What this PR does / why we need it (en: English/zh: Chinese):

en: update gzip docs
zh: 更新 gzip 文档
Which issue(s) this PR fixes:

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

参考 https://github.com/cloudwego/cloudwego.github.io/pull/365 完善 example 示例代码保证 简单示例的main函数是全的


Error in the GREL partition documentation

There is an error in the GREL partition documentation on docs.openrefine.org
Reported on the mailing list https://groups.google.com/g/openrefine/c/KwUyRM7VBB4/m/FVkX6T2bAAAJ

This is a piece of text of partition() documentation:
partition(s, s or p fragment, b omitFragment (optional))
Returns an array of strings [ a, fragment, z ] where a is the substring within s before the first ..............................The expresion "abcdefgh".partition(/c.e/) will output [“abc”, "cde", defgh” ].
I think that:
"abcdefgh".partition(/c.e/)
[ "ab", "cde", "fgh" ]
and not:
[“abc”, "cde", defgh” ]

Change made by #5372 was still incorrect....


EXPIRE/TTL and primaryKey issues

Like several others have posted before me, it looks like EXPIRE isn't properly supported by this adapter, which is a major factor in selecting Redis over other data stores. I'm using Redis for caching, and need the keys to expire automatically.
My initial workaround which used the native connection to set the EXPIRE value on the created keys worked great, until the key expired and was flushed from Redis, however the keys created by Waterline which contain the primary keys of the data are not cleaned up. Thus, when attempting to locate something in Redis, it does not exist and I would try to create the key. This results in a ERROR Error (E_UNKNOWN) :: Encountered an unexpected error AdapterError: Record does not satisfy unique constraints error because the primary key is already in the Waterline keys waterline:cache:_indicies and waterline:cache:id (cache being my model and id being my primaryKey.)
Simply setting EXPIRE on the Waterline keys at the beginning of the program won't work, because Waterline will recreate these keys if they are not present (already expired), and the EXPIRE timeout will not be set, and you'll be back at square one.
The typical user may not notice this if they are using autoPk: true, because the IDs will be incrementing and a conflict should not happen (although you'll see the Waterline keys will contain references to expired keys anyway, and keep growing after the data is long gone). Since I'm using a custom primaryKey, there is always the chance for conflict.
I think we need to look into this and come up with a solution that allows setting the TTL on keys and also manages the indices Waterline creates. I'm relatively new to Redis, so I don't pretend to have the answers, but I think I identified a problem many Redis users will run into.

+1

+1 for this. Expiry is pretty integral for me to use Redis.

+1.

+1

Since Expire isn't really a valid waterline value it's kinda hard to keep the consistencies between all the waterline adapters and keep the api's the same saying that, i do believe the ttl/expires value would be super awesome to add, and it's one of the coolest  features in Redis.
@jonathanwiesel Looking at the other guys commit it shouldn't be too hard to add into the current branch with a pull request to bring up to the master branch.
Lets see if i can do that sometime tomorrow huh, since i have fixed all the other problems haha

So had a talk with the waterline guy's it look's like we are just going to put it in the waterline and then these adapters will get updated to support ttl

:+1:

:+1:
1 - I believe there is room for a TTL interface in waterline given several datastores support this.
2 - Even for datastores/adapters that don't support it I think waterline could provide a shim as it's not incredible difficult to do. Simple example: https://github.com/dmarcelino/connect-waterline/blob/master/connect-waterline.js#L120
cc: @devinivy

+1
any update on this?

@camsjams not really haha, we have talked about but apart from that not really.

@sailsbot, I would say this one of the definitive features that sails-redis should support, so I vote for reopening this one.

Yeah agreed re-opened

This behaviour just gave me a good headache.
1 we can set expire ttl ourselves
2 but the primaryKey is stored and won't disappear, leading to a sails-thrown uniqueness error.
The problem is that in most use cases involving a redis DB, we don't want sails to manage uniqueness for us, AT ALL. Uniqueness is already managed by the primaryKey ! We write there, or should be anyway.
I am currently reading index.js, and it's clear that the uniqueness and primaryKey checks should be optionnal, instead of throwing. The key string, ie 'waterline:collectionname:pkname:pkvalue' should also be under our control instead of imposed by the adapter. Redis is meant to be shared between instances, and not all of them are sails. Even if we had only sails server, it would force us to use the same tablename and pkname between our instances !
Redis is a key value database and repeatedly finding an item with a constant 'pseudo-key' (like uid, or email) wich is hidden inside the object, and an index beeing used as the key instead is a major anti-pattern. We ask redis to FIND{where} instead of asking redis to GET key.
It's sad, because to use Redis at least half correctly we have to shortcut waterline and sails-redis almost completely and lose a lot of useful standardization, model definition, type checking, and lifecycle callbacks.

@Kallikrein agreed....
i've been looking into this problem for a bit now.... I think i have a good way of doing it but it requires quite a bit of change not only on the redis adapter side...
I'm hopefully going to share it with everyone this weekend, i might get super busy, but I'll try....

@Ryanc1256 Thanks.
Is there a communication channel like slack, discord or irc where we might chat together about participating and some todo's ?
For the time beeing I identified a set of points that contradict my vision of an ergonomic redis orm.
A lot of tests define a valid waterline adapter.
I don't know wich tests concern waterline compliance or only redis behaviour. The problem I have with waterline know is that I don't know if I do agree with the opinionated default adapter behaviour, I would appreciate insight on what is possible and what is not before trying to fit square features in a round loader...
in ./test/unit/adapter.create.js
it('should return error on non-auto incrementing primary key'

It should not.
it('should not create record with non-unique attributes'

It should.
Almost all of ./lib/database/sequence.js should be optionnal
  // Build a NoSQL Key name for this sequence
  this.keyName = 'waterline:' + collectionName.toLowerCase() + ':_sequences:' + this.name;

./lib/database/schema.js
Schema.prototype.indexKey = function(collectionName, index) {
  var name = collectionName.toLowerCase();
  return 'waterline:' + name + ':' + index;
};

./lib/database/indice.js
  // Build a NoSQL Key name for this sequence
  this.keyName = 'waterline:' + collectionName.toLowerCase() + ':_indicies:' + this.name;

We need control on these strings
As far as my opinion goes, I think the expected behaviour is the following for most use cases :

It should use primaryKey as a redis GET key
If defined in attributes, it should execute a keyGen() instance method function as a lifecycle callback to generate the key. keyGen : function() { return ['myKeyPrefixIWroteMyself', this.primaryKey].join(':')}
If defined as a class method, it should execute a keyGen class method function during adapter loading and store the return value as a prefix for future key generation.
It should remove the primaryKey property before outputing JSON object to store (redundant with key)
It should accept an expire value in the model definition and set expire property
If defined in attributes, it should execute a expire() instance method function as a lifecycle callback at each creation
If defined as a class method, executing Model.expire(int) should set the expire value for the model.
Create, save or update should be able to have ttl as an argument

I fear a lot of these features would contradict waterline dogmas.
The problem is that waterline wants us to be able to swap db from one to another, but we don't want to do that. It's looking for the smallest common denominator, alas clearly opinionated from a sql point of view. It is a very good thing when dealing with sql database, but it's poorly hacking how mongo or redis have a different perspective on storing data.
Maybe a better solution would be to create a concurrent adapter, focused on servicing a redis database ?
I would also gladly give up almost all of aggregate.js, since waterline grouping syntax is as complicated as native query - but way slower because of the agnostic layer. Plus, the key:value is what we want redis for, we don't want complex queries. Even if we do, we won't use waterline standard api since it's focused on doing good queries for sql and will most likely not expose the correct redis native functionality, since it's not conforming to sql standards.
I wonder, before I start doing another adapter, how waterline will let me in control of my custom adapter. I would like to be sure I won't mistake brick walls for doors - since I open doors with the head :)
I hope those interested in maintaining or forking redis support would gather and talk together to share knowledge and experience, and eventually plan for future dev...
See ya

@Kallikrein I'll let @Ryanc1256 chime in but I have always seen Redis as a very different adapter than all the others.
Just curious though at what point do you actually need Waterline for the redis stuff you mentioned? Why not just require("redis") and set and get the keys you defined?

@particlebanana i find that waterline just makes everything nicer, yes you can do everything with your normal get a set, but if you want to do some relationship stuff, thats when it gets hard I must admit...
@Kallikrein
well, you can email me about it, since im guess its very different time zone where you are to me :p and i think my email is on my profile...
If you have a google hangouts account, i can im you when im at work ( for the next 8 hours ish )
I'm going to write up a changelog, on my redis or in a PR idk fork that i propose to change in this adapter to bring the speed and bring it back to normal redis usage, but backed by how waterline works...
So people can comment on it... I would love to have people's input on it... but i'll let everyone know when it's up there.
Before you do something/rewrite something, just message me... i've been thinking about this quite a bit and could you any help really :P

@particlebanana
A very good question !
I like the model / attribute definition, type checking, and standardized lifecycle callbacks.
I give you my use case :
A user signs up with an email.
Validation :

beforeValidate  I check this email is not already used (in postgres) and I generate a random token
I check that it is a valid email
I check the password is 8 char long

Creation :

beforeCreate I hash the password
I create a temporary key:value entry (KEY email {password: password, token: token}) expires 2 hrs
afterCreate I set TTL, then send an email to my user.

In another route I receive the token/email through url contained in the email.
I query REDIS for the email as the key, check token validity.
If it's fine, I create the user in my postrgres.
I could use my own factory/decorator logic but that would mean getting it out of alpha limbos :), and I'm kinda in a rush, trying to lift an app as the only dev and with no salary until we launch.
I finally opted to keep using waterline for my model definition. It keeps things standards and tidy, it's much more developed, there is a community around it and I like it.
I just needed to set ttl, once I scaned my redis I checked for the key syntax, I then log'd the model instance, then wrote my own expire() method that works pefectly fine. Since I use the id returned by create(), the access is O1.
This solution is stable and will scale pretty well, since this process is not frequent.
BUT it is totally suboptimal, as far as I understand clearly.

I let vanilla sails-redis manage the primary key. Since the autoincrement is up, I will never have a uniqueness thrown at my face.
I let vanilla sails-redis manage the find process. It's a pity, since the find({email: email}) is much slower than a get(key). I can't get the key, because I don't know it, it's an autoincremented int :'(
There is a lot of uneeded data generated : Indices and unique tables, index key (instead of the primaryKey). I don't know how find() is working yet in detail, but I'm afraid it's not O1 and most likely On with a lot of potential ram and proc waste.

For the time beeing I can stick to sails-redis. If I need more ooomph I will have to adress sails-redis or making a key-focused custom adapter (which will not be waterline canon).
PS : I wish to thank everyone involved in sails / waterline / adpater dev and support. I greatly appreciate the time saved and the robustness of this framework. Let's notice the irony that you question using waterline and adapters :)
@Ryanc1256 It's the evening here. (France) I sent you a hangout request (whatever that means :D)

@Kallikrein I totally agree with everything you just put down! :P
and Its early morning here (NZ) and I think i added your to hangouts, you just gotta accept it :P

Edit : I can send the id returned by sails-redis create() in the email instead of ... the email. I indeed have the key. (thanks for rubberducking my backend sir)
So except for a little (but growing) waste of ram in the unique key collection, I don't have any good reason not to use waterline. Since I don't think millions of user are about to saturate my redis of wasted id storage... Safe for work.
I will most probably track down and kick uniqueness tables creation in my own fork when I have the time.
The ideal micro-fix would be to create sails-redis specific class methods (keyGen(), expire, get(), set()), as proposed above.
I need to understand how waterline factory is producing ModelClass to properly access my custom model options. It might be shorter than we think, as long as it's considered valid to customize an adapter at such length.

@Kallikrein here ya go buddy!
@particlebanana could use your input too... see what you think.. :)
https://github.com/balderdashy/sails-redis/pull/90

As a related note to anyone using Sails, waterline and Redis, I've released a library that caches Waterline responses from a DB etc into Redis as a caching layer:
https://www.npmjs.com/package/treasury

@camsjams awesome! I always thought Redis would work great for this.

Hi guys, any update on the current status of this? Or is this feature more of a long term goal?
Many thanks guys, appreciate all your work on this awesome adapter :)

@tyronedougherty it's defiantly a long term thing due to the fact waterline needs to support it before we can implement it...

@Ryanc1256 @tyronedougherty @particlebanana @camsjams @Kallikrein @dmarcelino @richdunajewski @jonathanwiesel @timestep @jnorberg @lukasreichart Hey everyone, we're moving all the redis issues to the main Sails repo (balderdashy/sails/issues) to gather more exposure to the community. Best of luck, thanks for all the input so far!


NotImplementedError: in the line structured_llm = llm.with_structured_output(schema, include_raw=True)

Hello,
I'm not running the whole app.
I want to build a knowledge graph, run query on the knowledge graph and retrieve result from the LLM.
While executing this line of code
    # Define chain
    schema = create_simple_model(allowed_nodes, allowed_relationships)
    structured_llm = llm.with_structured_output(schema, include_raw=True)
    self.chain = prompt  | structured_llm

I encountered a NotImplementedError:

Here's the llm call :
  llm = AzureChatOpenAI(
     deployment_name="gpt-4",
    openai_api_version="2023-05-15",
    temperature = 0
    )

@tomasonjo do you know why AzureChatOpenAI would not support structured output in LC?

Make sure to use the latest version from the partner library:
from langchain_openai import AzureChatOpenAI

AzureChatOpenAI(
    azure_deployment="35-turbo-dev",
    openai_api_version="2023-05-15",
)


Needs deployment instructions

Currently there are no instructions for deploying to production. These should be included, preferably for multiple different environments (heroku, AWS, etc)

I would add that the development setup instructions should also be enhanced by adding information about how to setup Google API keys and OpBeat keys. I still can not get the project to run on my local machine.

all help will be appreciated. @caskuplich @smiller171

@jrogatis Do you have any direction on this? What are the current procedures among the core developers?

@smiller171 this app was designed to run at digital ocean FCC servers. But at the development cicle we use heroku. So the runs on both. We don't have any directions on that, what give you a open path for ideias ! ;-)


log variables used when starting Vagrant deployment

Signed-off-by: samuel.elias<EMAIL_ADDRESS>

Coverage remained the same at 69.457% when pulling 025f8a29b137e72fd17d123dcb908dbd2ac16baf on samelias:vagrant into 408fd28c90cb681115cfd322b6158bab0351fd51 on contiv:master.


Rendering tabs based on authentication and user roles

Version
Tell us which versions you are using:
react-native 0.50.3

react-native-router-flux v4.0.0-beta.22

Expected behaviour
Should render tabs depending on condition
Actual behaviour
There is no route defined for key0. Must be one of: "key2"
I have to implement tabs based on user roles after login. In some case, there may be 2 tabs and in other case, there may be 3 or more tabs. It wil be determined only after logged in.
Code:
renderTabs() {
const autho = []; //['admin', 'non-admin'];//this is dynamic array
return (
<Tabs lazy hideNavBar swipeEnabled={false} tabBarStyle={styles.tabBarStyle} showLabel={false} tabBarPosition={"bottom"}>
{auth0.map((accessRight, index) => {
if(accessRight==='admin')
return <Scene key="Admin" title="Admin" titleStyle={{ color: '#949494'}} titleDesc="Admin" component={Admin} icon={TabIcon('thermometer-full')} />;
if(accessRight==='non-admin)
return <Scene component={NonAdmin} key='non-admin' title="Non-Admin" titleDesc="Non-Admin" titleStyle={{ color: '#949494'}} icon={TabIcon('wrench')} headerMode="float"/>;
})}

);
}
Steps to reproduce
For non-obvious bugs, please fork this component, modify Example project to reproduce your issue and include link here.

create tabs with some condition
make it rerender using redux state change
basically have no tabs when not logged in and after logged in have tabs based on condition

Duplicate of #2349
hope to be useful to you


Metro Exodus Enhanced seems to work fine

Needs OverrideAppId = true to respect AI model.
Works with both the developer DLL and the original 2.1.55 DLL.
I only played it for a few minutes but didn't notice any artifacts.

For me Metro exodus enhanced crashes with the dxgi.dll, doesn't work with XInput1_3.dll, but works fine if I use XInput9_1_0.dll.
I'm using dlss ver 3.1.1 from techpowerup's page.

Yes, I forgot that part. I have updated the first post.

Thanks for the info, added it to the readme, guess maybe the DLL names depends which store version you own? Not sure, I mentioned them both anyway.


Question: Is the ConsoleLogger safe to use in production?

I was configuring the logging for a new ASP.NET Core app, and I had the vague memory that the ConsoleLogger was not supposed to be used in production.
Searching around, all I could find was a couple of comments on this issue that suggested that a global lock in the ConsoleLogger library could cause performance issues.
Comparing the ConsoleLogger  code between 1.0 and 2.0, it looks like a previous lock() usage has been removed.
Does that mean the ConsoleLogger is now safe to use in production, or were these comments referring to a different global lock?
Thanks

I don't think it has ever been unsafe. It's just you will probably take a performance hit if you enable it.

Sure, that's what I meant by unsafe though - you were advised not to use it in production due to the performance impact (as mentioned a couple of times in this issue). Are those performance concerns still there, or have they been alleviated by the rewrite to remove the lock?

Perf has been improved and now the logging happens on a background thread so individual calls to log should be fairly fast. They do still access a lock, but it is just to add the log to a queue for later processing.
I haven't seen any perf numbers of before and after though so I can't recommend for or against it.

Same question I have. The performance warning has been removed from the docs for 2.x,
bit is still there for 1.x. So my guess is that if there's still an impact it would be negligible.
Would be great to have a definite answer though.

@glennc

Having looked through the code, the answer given here, (and I vaguely remember hearing something mentioned on the community standup) I'm pretty sure this is safe for production now.

@andrewlock console logger still uses BlockingCollection with size limit of 1024 so Log calls would start blocking when it gets reached. So very detailed console logs should be enabled with caution.

Thanks for the update @pakrym, that's good to know. Definitely something to bear in mind, especially as logging to the console is the preferred logging mechanism in e.g. Docker.
I wonder if there should still be a mention of this somewhere in the docs, given that it could be difficult to diagnose slow requests, if you're not aware of this potential bottleneck.


Idle timeout of outbound broker connection may not trigger reconnection

Describe the bug
Internally EnMasse uses the broker plugin to establish connections from the broker to the route network. It configures an idle timeout as part of the outbound open performative in order to achieve connectuvity robustness.
If the broker detects an idle timeout the broker should close the connection and proceed to reestablish a new replacement connection.  However, there is a defect. For the reconnection to occur, the peer (router) needs to shutdown the socket.  If this shutdown does not occur or is not detected (as might be the case under unusual network connections), the connection remains unmade.
Here's a trace where the peer responds to the Close by replying Close and closing shutting down the socket.
2020-12-23 12:44:24,543 INFO [org.apache.activemq.artemis.integration.amqp.ProtonClientConnectionManager] Connection localhost/<IP_ADDRESS>:15672 created
[322200002:0] -> AMQP
[322200002:0] -> Open{ containerId='foo', hostname='null', maxFrameSize=131072, channelMax=65535, idleTimeOut=8000, outgoingLocales=null, incomingLocales=null, offeredCapabilities=null, desiredCapabilities=null, properties={product=apache-activemq-artemis, version=2.13.0, qd.route-container-group=cluster}}
[322200002:0] <- AMQP
[322200002:0] <- Open{ containerId='test', hostname='null', maxFrameSize=4294967295, channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null, offeredCapabilities=null, desiredCapabilities=null, properties=null}[322200002:0] -> Close{error=Error{condition=amqp:resource-limit-exceeded, description='local-idle-timeout expired', info=null}}
2020-12-23 12:44:40,544 INFO [org.apache.activemq.artemis.integration.amqp.ProtonClientConnectionManager] Connection localhost/<IP_ADDRESS>:15672 destroyed
2020-12-23 12:44:40,544 INFO [org.apache.activemq.artemis.integration.amqp] connectionDestroyed for connector amp-connector

and for comparison, one where the peer consumes the close but fails to respond or shutdown the connection.
2020-12-23 12:47:07,610 INFO [org.apache.activemq.artemis.integration.amqp] Starting connector amqp-connector
2020-12-23 12:47:07,611 INFO [org.apache.activemq.artemis.integration.amqp] connectionCreated for connector amqp-connector
2020-12-23 12:47:07,611 INFO [org.apache.activemq.artemis.integration.amqp.ProtonClientConnectionManager] Connection localhost/<IP_ADDRESS>:15672 created
[1421049339:0] -> AMQP
[1421049339:0] -> Open{ containerId='foo', hostname='null', maxFrameSize=131072, channelMax=65535, idleTimeOut=8000, outgoingLocales=null, incomingLocales=null, offeredCapabilities=null, desiredCapabilities=null, properties={product=apache-activemq-artemis, version=2.13.0, qd.route-container-group=cluster}}
[1421049339:0] <- AMQP
[1421049339:0] <- Open{ containerId='test', hostname='null', maxFrameSize=4294967295, channelMax=65535, idleTimeOut=null, outgoingLocales=null, incomingLocales=null, offeredCapabilities=null, desiredCapabilities=null, properties=null}
[1421049339:0] -> Close{error=Error{condition=amqp:resource-limit-exceeded, description='local-idle-timeout expired', info=null}}

This defect does not affect inbound AMQP connections to Artemis so this defect does not affect brokered.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

This issue represents a robustness issue.  The broker should try to restablish its outbound connection regardless of the what the peer does or doesn't do.


add support for the entire mitre group metadata

When --enrichments are enabled, we get a lot of data back about each mitre group.
There is now a new field which includes all of this data, where we previously just included the name
of the group. An example of this data is included below.
This is parsed with pydantic so that fields can be accessed.
Note that if this data structure changes, we will need to update the definition.
 {'created': '2017-05-31T21:31:45.629Z',
  'created_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
  'external_references': [{'external_id': 'G0001',
                           'source_name': 'mitre-attack',
                           'url': 'https://attack.mitre.org/groups/G0001'},
                          {'description': '(Citation: Cisco Group 72)',
                           'source_name': 'Group 72'},
                          {'description': '(Citation: Novetta-Axiom)',
                           'source_name': 'Axiom'},
                          {'description': 'Esler, J., Lee, M., and Williams, '
                                          'C. (2014, October 14). Threat '
                                          'Spotlight: Group 72. Retrieved '
                                          'January 14, 2016.',
                           'source_name': 'Cisco Group 72',
                           'url': 'http://blogs.cisco.com/security/talos/threat-spotlight-group-72'},
                          {'description': "Kaspersky Lab's Global Research and "
                                          'Analysis Team. (2013, April 11). '
                                          'Winnti. More than just a game. '
                                          'Retrieved February 8, 2017.',
                           'source_name': 'Kaspersky Winnti April 2013',
                           'url': 'https://securelist.com/winnti-more-than-just-a-game/37029/'},
                          {'description': 'Novetta Threat Research Group. '
                                          '(2015, April 7). Winnti Analysis. '
                                          'Retrieved February 8, 2017.',
                           'source_name': 'Novetta Winnti April 2015',
                           'url': 'https://web.archive.org/web/20150412223949/http://www.novetta.com/wp-content/uploads/2015/04/novetta_winntianalysis.pdf'},
                          {'description': 'Novetta. (n.d.). Operation SMN: '
                                          'Axiom Threat Actor Group Report. '
                                          'Retrieved November 12, 2014.',
                           'source_name': 'Novetta-Axiom',
                           'url': 'https://web.archive.org/web/20230115144216/http://www.novetta.com/wp-content/uploads/2014/11/Executive_Summary-Final_1.pdf'},
                          {'description': 'Tarakanov, D. (2015, June 22). '
                                          'Games are over: Winnti is now '
                                          'targeting pharmaceutical companies. '
                                          'Retrieved January 14, 2016.',
                           'source_name': 'Kaspersky Winnti June 2015',
                           'url': 'https://securelist.com/games-are-over/70991/'}],
  'group': 'Axiom',
  'group_aliases': ['Axiom', 'Group 72'],
  'group_description': '[Axiom](https://attack.mitre.org/groups/G0001) is a '
                       'suspected Chinese cyber espionage group that has '
                       'targeted the aerospace, defense, government, '
                       'manufacturing, and media sectors since at least 2008. '
                       'Some reporting suggests a degree of overlap between '
                       '[Axiom](https://attack.mitre.org/groups/G0001) and '
                       '[Winnti Group](https://attack.mitre.org/groups/G0044) '
                       'but the two groups appear to be distinct based on '
                       'differences in reporting on TTPs and '
                       'targeting.(Citation: Kaspersky Winnti April '
                       '2013)(Citation: Kaspersky Winnti June 2015)(Citation: '
                       'Novetta Winnti April 2015)',
  'id': 'intrusion-set--a0cb9370-e39b-44d5-9f50-ef78e412b973',
  'matrix': 'mitre-attack',
  'modified': '2023-03-20T22:03:44.661Z',
  'object_marking_refs': ['marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168'],
  'type': 'intrusion-set',
  'url': 'https://attack.mitre.org/groups/G0001',
  'x_mitre_attack_spec_version': '3.1.0',
  'x_mitre_deprecated': False,
  'x_mitre_domains': ['enterprise-attack'],
  'x_mitre_modified_by_ref': 'identity--c78cb6e5-0c4b-4611-8297-d1b8b55e40b5',
  'x_mitre_version': '2.0'}

Note - this branch required a rework of the MitreAttackGroup object AND a bump of the attackcti library dependency to ^0.4.0.  There were structural changes in the objects it returns between 0.3 and 0.4 that would cause errors during parsing.


Some issues with Logitech GPW 2

OS
macOS 13.3.1 (a) (Build 22E772610a)
LinearMouse
v0.9.1
Describe the bug

Ctrl + scrolling not work, Cmd/Opt + scrolling is OK.
Buttons / Assign actions to mouse buttons can not record the side buttons of GPW 2.

But, these functions are working well with Logitech GPW 1, not sure if it's a problem only with GPW 2.
To reproduce
No response
Expected behavior
No response
Anything else?
No response

Thanks for your feedback!
Would you mind letting me know if you have Logi Options installed? I've heard from some other users that similar software can cause conflicts.
Also, can you confirm whether both GPW 1 and GPW 2 are using the same settings?

Thanks for your reply!
Regarding the first question: I have installed G HUB, but I have tested the functions both with and without G HUB installed, and it didn't work in either case.
As for the second question: I am certain that GPW 1 and GPW 2 are using the same settings.
If you need any log files for debugging, pls feel free to ask for them!

What confuses me the most is why the LinearMouse app works well with the GPW 1, but not with the GPW 2. I use the same laptop with different mouses. The GPW 1 is used at the office and the GPW 2 is used at home. Currently, the office experience is perfect! (I love the new function: Assign actions to mouse buttons). However, the home experience is not as good as the office T_T

我怀疑日志里没有有效的信息，不过还是可以提供一下。因为如果录制按键没有反应，说明按键没有传递到 LinearMouse。可能是被其他软件劫持了（显然 G HUB 已经被你排除），也可能是鼠标本身没有产生对应的事件。
我之前用过一款小米的鼠标，如果用 2.4GHz 接收器连接，侧键在 macOS 下无法使用，但是使用蓝牙连接就没问题，罗技鼠标还没有遇到类似的情况，或许你可以尝试一下 GPW 2 不同的连接方式有没有区别。
至于 control+滚动不工作，倒是更像是被其他软件劫持了，因为理论上跟 cmd/option+滚动没什么区别。


[Feature Request] Adding Localization in the picker

Hi,
If it possible to add a locality for the picker so it can support multi languages ?

Hi, currently default localization values is already implemented. When you change your system language the text also changes if your localization is set such as:
return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      localizationsDelegates: const [
        GlobalMaterialLocalizations.delegate,
        GlobalWidgetsLocalizations.delegate,
        GlobalCupertinoLocalizations.delegate,
      ],
      supportedLocales: const [
        Locale('en', ''), // English, no country code
        Locale('zh', ''), // Chinese, no country code
      ],
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );

This is an example when using the Chinese language.


⚠ Jenkins Plugins Artifactory has degraded performance

In 2995d1c, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1411 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 34e7b3f after 2 hours, 41 minutes.


[macOS] IME fails to react to backspace

During an IME composition, pressing backspace affects the precompose text, but not the IME's candidates.

Target app: a text field
Step:

Use macOS's Chinese Pinyin IME
Press a few keys (in the GIF I pressed a few W). It looks correct for now.
Press backspace.

Expected: The candidate list is shortened accordingly.
Actual: The candidate list does not change.

Further proof:

Press a few letter keys again.

Actual: The new keys are appended to the candidate list before backspace presses.

Select a candidate using digit key 1.

Actual: The output text is the unshortened candidate.

Both showing that the IME did not receive the backspace key at all.
Version:

[✓] Flutter (Channel master, 2.6.0-12.0.pre.355, on macOS 11.6 20G165 darwin-x64, locale en)
    • Flutter version 2.6.0-12.0.pre.355 at /Users/tongmu/dev/flutter
    • Upstream repository git@github.com:flutter/flutter.git
    • Framework revision ccc8261495 (30 minutes ago), 2021-10-14 15:33:05 -0500
    • Engine revision 8a18794648
    • Dart version 2.15.0 (build 2.15.0-217.0.dev)
[!] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
• Android SDK at /Users/tongmu/Library/Android/sdk
✗ cmdline-tools component is missing
Run path/to/sdkmanager --install "cmdline-tools;latest"
See https://developer.android.com/studio/command-line for more details.
✗ Android license status unknown.
Run flutter doctor --android-licenses to accept the SDK licenses.
See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[!] Xcode - develop for iOS and macOS (Xcode 13.0)
• Xcode at /Applications/Xcode.app/Contents/Developer
! CocoaPods 1.9.1 out of date (1.10.0 is recommended).
CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
Without CocoaPods, plugins will not work on iOS or macOS.
For more info, see https://flutter.dev/platform-plugins
To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.
[✓] Chrome - develop for the web
• Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
[✓] Android Studio (version 3.3)
• Android Studio at /Applications/Android Studio.app/Contents
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart
• Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
[✓] VS Code (version 1.61.0)
• VS Code at /Applications/Visual Studio Code.app/Contents
• Flutter extension version 3.27.0
[✓] Connected device (2 available)
• macOS (desktop) • macos  • darwin-x64     • macOS 11.6 20G165 darwin-x64
• Chrome (web)    • chrome • web-javascript • Google Chrome 94.0.4606.81
! Doctor found issues in 2 categories.

We might want to test it on other platforms because I haven't.
cc @cbracken

Update: It does not react to left/right keys either (try pressing left during composing then insert a different key.) In fact, doing this for multiple times results in the follow assertion error, crashing the entire keyboard system:
2021-10-14 16:56:25.270 testapp[15277:1027028] *** Assertion failure in -[FlutterEmbedderKeyResponder handleUpEvent:callback:], ../../flutter/shell/platform/darwin/macos/framework/Source/FlutterEmbedderKeyResponder.mm:677

I'm going to test this out momentarily to see if I can determine whether this is a desktop stable blocker. I'm making the assumption now that it is.

Assigning myself while I investigate this.
Related: https://github.com/flutter/flutter/issues/92060
Maybe related: https://github.com/flutter/flutter/issues/85328

I confirmed that this is not happening on Windows, but I filed a more subtle bug I noticed there, unrelated to this one: https://github.com/flutter/flutter/issues/92132

This will probably require solving https://github.com/flutter/flutter/issues/30725 to solve this the right way.  See discussion over there.

I believe this is also a problem on iOS when a physical keyboard is used.  See https://github.com/flutter/flutter/issues/30725#issuecomment-948855904.


FeatureProposal: Support Java8 Completionstage in API

I assume Awaitility is often used in conjunction with Java8 CompletionStage. It would be nice if there was a more writable way to await for completion of those, such as:
CompletionStage<String> stage = CompletableFuture.completedFuture("foo");
// current
await().until(() -> stage.toCompletableFuture().isDone());
// wish to have
await().untilDone(stage);

@tkruse  The next version of Awaitility will require Java 8 so I'm revisting old issues. Is this something that you still would find valuable? If so could you please explain why
await().untilDone(stage);

is better than:
 stage.toCompletableFuture().get(10, TimeUnit.SECONDS);

I am not sure. It's been too long since I wrote this kind of code. SO feel free to close, I can reopen if I remember.


bugfix: Collapse not working in the menu

onClick Collapse not working in the menu (Setup)
Expected behavior
It should collapse on clicking
Screenshots

This issue was fixed and merged to development pull req #104

this issue is fixed, henceforth the issue is no longer valid.


Cloning entire PDF along with catalog properties

Hello,
Is it possible to clone an entire PDF and include all of the original PDF's properties such as the catalog with acroForms, etc? I'm attempting to reconstruct a new PDF from an old PDF only with less pages. At present I've just got a little test function:
const includePages = async originalDoc => {
  // Create a new document to house the pages 'still' wanted
  const modifiedDoc = await PDFDocument.create()

  // Copy in the properties from the original (catalog, etc)
  // ... ¯\_(ツ)_/¯

  // Copy pages 0 and 1 only for example (say our original PDF has like 5 pages)
  const pages = await modifiedDoc.copyPages(originalDoc, [0, 1])
  pages.forEach(page => modifiedDoc.addPage(page))

  // Return the modified document with pages 0 and 1 only
  return modifiedDoc
}

At present the modified document only carries across the catalog with 'type' and 'pages' in its dictionary, but it's missing a few other ones such as the 'acroForm', so when I go to modify some of the interactive fields later on, it produces an error since it no longer exists.
Is this a thing anyone's come across and successfully implemented?
Thanks very much :)

Hello @sdfereday!
I think that copying specific structures from a donor PDF makes sense for certain use cases (e.g. #218). But this can be tricky, because automatically copying all the catalog structures will probably result in undesired/unexpected behavior most of the time (e.g. nonsensically merged outlines, conflicting AcroFields, etc...). So this would need to be implemented carefully, perhaps on a case-by-base basis.
However, for the specific use case you're asking about here (constructing a new PDF from an old PDF only with less pages), it seems to me that you could just load the PDF and remove pages with PDFDocument.removePage. For example:
const pdfDoc = await PDFDocument.load(pdfBytes);
pdfDoc.removePage(0);
pdfDoc.removePage(2);
const newPdf = await pdfDoc.save();

I hope this helps. Please let me know if you have any additional questions! And my apologies for the (very) delayed response.


Bower support

Please add bower support. Thanks.

Can't you do bower install shprink/ionic-native-transitions ?

Included in documentation of beta3


Logo proposal

I would like to contribute to the development of the project. I am a designer. I can design a logo for you. What is your decision ?

Hi @ritalinn, thanks for the offer but I am currently happy without a logo.


Wrong color information for dust particles

The forth element is the size of the particle. However, currently it's treated as the alpha channel.

:tada: This issue has been resolved in version 1.13.6 :tada:
The release is available on:

npm package (@latest dist-tag)
Visual Studio Marketplace
GitHub release

Your semantic-release bot :package::rocket:


user12986714: Watch<PHONE_NUMBER>

user12986714 requests the watch of the watch_number<PHONE_NUMBER>. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Eliah Kagan in Charcoal HQ


Instructions in readme are incorrect for checking out

Following the instructions I get:

git clone git@github.com:StephenGrider/ReduxSimpleStarter.git
Cloning into 'ReduxSimpleStarter'...
Warning: Permanently added the RSA host key for IP address '<IP_ADDRESS>' to the list of known hosts.
Permission denied (publickey).

You should have:

git clone https://github.com/StephenGrider/ReduxSimpleStarter.git

Using "git@" implies a login that only you possess.

Fixed!