[Discuss] Ballista Future Direction

No that Ballista is it’s own top-level project, I want to extend a previous discussion around what exactly Ballista is (previous discussion https://github.com/apache/arrow-datafusion/issues/1916)
I believe the consensus from that discussion was that Ballista should be a standalone system but in practice I think we have adopted somewhat of a “both and” approach in the sense that we have added several extension point to Ballista (while also providing default implementations that allow you to run Ballista out-of-the-box). I think this is a reasonable direction and agree that it is important that Ballista be something you can use “as is”.
That raises the question of what are the use cases for Ballista that we would like to optimize for?
To take a concrete example, I think the current architecture is optimized for batch processing using a straightforward map-reduce implementation. This has many advantages:

Scheduling is relatively straightforward.
Cluster utilization is efficient since the unit of schedulable work is a single task, so whenever a task slot frees up you can just schedule a pending task on it.
It is resilient to task failures since all intermediate results are serialized to disk so recovering from a spurious task failure is just a matter of rescheduling the task.

However, for non-batch oriented work it has some serious drawbacks

Results can only be returned once the entire query finishes. We can’t start streaming results as soon as they are available.
Stages are scheduled sequentially so each stage is bound by its slowest partition.
Queries that return large resultsets can be quite resource intensive as each partition must serialize its entire shuffle result to disk

There has already been some excellent proof-of-concept work done on a different scheduling paradigm in https://github.com/apache/arrow-datafusion/pull/1842 (something my team hopes to help push forward in the near future) but this raises some questions of it’s own, Namely, when do we use streaming vs map-reduce execution? In the PoC it is a very simple heuristic but in real uses cases I’m not sure there is a one-size-fits-all solution. In some cases you may want to use only streaming execution and use auto-scaling or dynamic partitioning to make sure each query can be scheduled promptly. Or you may only care about resource utilization and want to disable streaming execution entirely.
This is one particular example but you can imagine many others.
To bring this back around to some concrete options, I think there are a few different ways we can go:

Ballista is a distributed computing framework which can be customized for many different use cases. There are default implementations available which allow you to use Ballista as a standalone system but the internal implementation is defined in terms of interfaces which allow for customized implementations as needed.
Ballista is a standalone system with limited customization capabilities and is highly optimized for its intended use case. You can plugin certain things (ObjectStore implementations, UDFs, UDAFs, etc) but the core functionality (scheduler, state management, etc) is what it is and if it doesn’t work for your use case then this is not the right solution for you.
Ballista is a standalone system which is highly configurable but not highly extensible. It ships with, for instance, schedulers optimized for different use cases which can be enabled through runtime configuration (or build-time features) but you can’t just plugin your own custom scheduler implementation (without upstreaming it :)).

To be transparent, my team is building a query engine which is sensitive to time-to-first-result latency so we are very interested in fully streaming execution (and hoping to upstream as much as we can) but want to make sure that this is in line for the desired direction of Ballista for the rest of the community.

cc @andygrove @yahoNanJing @realno @Ted-Jiang

@thinkharderdev Thanks for the nice write up. Among the three options I am leaning towards 3 is more realistic to achieve. That is, the community can be more focused on making it work really well with a specific set of use cases first, which will hopefully grow the community further.
For streaming v.s. batch I don't have a strong opinion at the point, I believe whoever can use it in a real use case should try to drive the project forward.
We are not ready yet to do anything serious, though we have two major use cases in mind:

Replace Spark for batch processing - 100s of billions of rows regularly with mega plans (100s of millions of expressions and projections)
High concurrency, medium latency (sub second) queries from data storage of 100s of billions or rows - result sets are not large

Use case 2 seems similar to your use case. I am curious when you said fully streaming execution did you mean like Flink? I think there is value to support operators/algorithms that needs to see the entire dataset/partition multiple times (for example ML), so a hybrid model would be good. For example, if the compiler can analyze the query and turn part of it "fully streaming" if possible.
Some other requirements for us are:

easier and cheaper than Spark to operate
natively support k8s - delicate resource and cluster management entirely to k8s

@thinkharderdev  Thanks for writing it up ❤.

To be transparent, my team is building a query engine which is sensitive to time-to-first-result latency so we are very interested in fully streaming execution (and hoping to upstream as much as we can) but want to make sure that this is in line for the desired direction of Ballista for the rest of the community.

We have the same purpose: use ballista to build a latency sensitive query system. So we are also very interested in fully streaming execution (avoid shuffle to disk).

To bring this back around to some concrete options, I think there are a few different ways we can go:

For now, I have no strong opinion about these choices.

I am curious when you said fully streaming execution did you mean like Flink?

Exactly, the current execution model is basically Flink Batch execution, but what we ultimately want (for our use case) is streaming execution with RecordBatch as the event type.

Somewhat of a tangent for this discussion, but my project (https://github.com/influxdata/influxdb_iox) is also interested in time to first result (aka streaming) execution, as well as the lower resource (memory) usage such execution provides; While we will likely not use ballista directly, we intend to ensure DataFusion can be used for this type of system.
One concrete example is the work @tustvold  is working on recently to get parquet decoding / IO better integrated (so the parquet decoding work can be done in a streaming fashion as well)

Thanks for starting this discussion @thinkharderdev :heart:
With Ballista moving to this new repository I think it is an excellent time to "reboot" the project and assess what we are trying to build here.
I'd like to provide some historical context for how we ended up where we are today:
The original goal with Ballista was essentially "rewrite Apache Spark in Rust" but avoiding an architecture that heavily favors a particular programming language (Scala, in Spark's case). This is why we serialize plans to protobuf format rather than just using Rust's "serde" crate, which would have been much easier. I now hope that we can eventually adopt https://substrait.io/ as the serialization format to make it easier for Ballista to leverage query engines other than DataFusion.
Quite early in the development process, I discovered Apache Arrow and made that a core part of the design as well. This in my mind was another clear advantage over Apache Spark, which is largely row-based.
Obviously, the choice of Rust is another major differentiator with its unique approach to memory management and safety.
I designed Ballista based on my experience of using Apache Spark for SQL/ETL batch jobs.
I would fully support seeing Ballista support both batch and streaming and I think it would be fine for a user to pick one or the other when executing a query and use different APIs for each case. That said, I have not looked into this so there are likely complications that I am not even aware of here. I will start learning more about streaming in Spark and Flink so that I can better contribute to the discussion.

To be transparent, my team is building a query engine which is sensitive to time-to-first-result latency so we are very interested in fully streaming execution (and hoping to upstream as much as we can) but want to make sure that this is in line for the desired direction of Ballista for the rest of the community.

I also have major usecases for latency-sensitive, potentially-multisource queries.
It boils down to being able to use it for end-user/interactive applications
One of the biggest bummers to me about Spark is that its architecture cripples it for latency-sensitive workloads
I wanted to see what the latency was like to do a basic, two-DB join query between in-memory databases:

https://github.com/GavinRay97/spark-playground/blob/44a756acaee676a9b0c128466e4ab231a7df8d46/src/main/scala/Application.scala#L80-L112

Something like:
SELECT ... FROM db1.foo JOIN db2.bar ON ... LIMIT 1

Using the latest Spark nightly snapshot, this takes 150-200ms on my personal machine.
A significant portion of this is spent on things relevant to multi-node computation but not required for doing in-memory on a single node (serialization, broadcasts, scheduling/coordination)
The codegen + execution time isn't that bad
Understandably Spark isn't tailored for this. But there's a lot of great technology in there (Catalyst, Tungsten) that are state-of-the-art for query optimization and performance, and it's a bummer that you can't configure Spark (to my knowledge) for a "local" mode or directly interact with just the pieces you need to manually evaluate expressions/do query optimization.
Would be great if the future of Ballista accommodated for this. Opens up interesting possibilities.

Great, thanks for the feedback everyone!
I would summarize the key points as:

Based on this and previous discussions, the goal is not to create yet another general "distributed compute framework" and that Ballista should be something that works out of the box.
There is interest from the community for both batch processing (ala Spark) and a latency-sensitive distributed query engine with streaming execution.
Ballista shouldn't be inextricably tied to DataFusion as the underlying compute engine. This isn't the case right now really but Substrait can be an "engine agnostic" representation of a compute plan in the future and we shouldn't foreclose that future work with design decision in the more immediate term.

Please chime in if anyone came to different conclusions :)

I have a PR open (https://github.com/apache/arrow-ballista/pull/33) to improve the top-level README to better describe the current state of the project and the future direction, linking to this discussion.
I have also created a PR against "This Week In Rust" (https://github.com/rust-lang/this-week-in-rust/pull/3276) to promote the new repo and hopefully direct more people to contribute to the discussion here.

Hi @thinkharderdev, for the point 2 of whether to use ballista for batch processing or latency-sensitive query processing, there should be a few things to be clarified:

How the ballista cluster to be deployed
Should the ballista cluster work in a long running way
Should the tasks work in a long running way
Which way to do the data exchange, push-based or pull-based
Should the exchanged data be flushed to disk

How the ballista cluster to be deployed

Since k8s is so common and popular today, I think we should support it natively. Like Spark for batch processing, we can provide a way to deploy the ballista by k8s on demand.

Should the ballista cluster work in a long running way

I prefer the cluster as a standalone system and runs in a long running way. For a long running system, we have to pay much attention to several aspects, like avoiding memory leaks, managing historical states, etc. While for Spark, it previously focuses on batch processing rather than long running system. Then its cluster always be destroyed after the batch processing finishes and it doesn't need to pay much attention to the long running related aspects.

Should the tasks work in a long running way

For streaming engines, like Flink, the tasks always work in a long running way. It will bring other challenges. The interests of my teams are not on this. We will focus on latency-sensitive interactive queries.

Which way to do the data exchange, push-based or pull-based

The pull-based way may not be as efficient as the push-based way. However, the push-based way needs the whole task pipeline topology to be determined before task execution. While for the pull-based way, like the Spark employs, AQE can be introduced to make is possible to change the whole query plan adaptively during query execution. One coin has two sides. Therefore, I propose to implement both.

Pull-based good for ad-hoc queries
Push-based good for latency-sensitive interactive queries or tasks

Should the exchanged data be flushed to disk

It also depends. Flushing to disk will be good for error recovering and easy memory management. However, it's not efficient. Therefore, I also propose to implement both.

Flushing to disk, good for batch processing with pull-based data change
Not flushing to disk, good for latency-sensitive queries with push-based data change.

I agree with @yahoNanJing on his answers above. The only thing I would add is that with respect to deployment we should be able to support different auto-scaling behaviors for long running clusters. Push-based, latency sensitive query execution probably will have more strict demands for auto-scaling since we need to schedule a lot of tasks all at once. I know that this is something my team is very interested in.

Regarding k8s support mentioned by @yahoNanJing - this is already supported and documented in the user guide (which needs updating to remove the DataFusion parts and we also need to get this published).
https://github.com/apache/arrow-ballista/blob/master/docs/source/user-guide/distributed/deployment/kubernetes.md

Hi everyone, first thanks for the great work on datafusion and ballista i am currently on @andygrove  book for processing engine and it's pretty interesting as it help us to learn more about how processing engine works and take an extra steps on learning engine such spark.
I am not an expert in this area, but i have few questions in mind as  i am willing to explore how ballista answer these and help in production grade data processing engine.
One of the things that i find interesting, is spark/redis integration that bump up spark performance due to the in memory nature of redis. However, i am not sure if that's already true in the context of ballista/datafusion due to the nature of Arrow an espcially with the integration of Plasma project in Arrow, made by Ray team, so i wonder guys if you can shed some lights on this.
One more think, how does ballista compare to engine such as Ray and Dask in general, and does potentially ballista could be direct competitor of those frameworks.
Finally as Ballista compares directly to spark, how do we see the implmentation of BALLISTA ML, Graph.
Thanks everyone for the great works.

For spark integration, check out https://github.com/blaze-init/blaze by @yjshen
In terms of the current status of the Plasma project I think you may have to ask on the<EMAIL_ADDRESS>list -- I remember some discussions about it previously but not sure

Hi @ziedbouf. I'm glad you are finding the book helpful! Let me try and answer some of your questions.
I have not used Plasma and there is no support for it in DataFusion/Ballista. I recall a discussion about it being unmaintained but I could be wrong. Ballista doesn't provide an in-memory cache like redis, although it would be possible to implement a custom datasource to connect to redis.
Ballista is similar to Spark SQL and Dask-SQL in terms of architecture, so yes, could be seen as a competitor although it is not as mature yet. Ballista does not have any stream, graph, or ML capabilities yet though. As mentioned in this discussion, some contributors are planning on working on streaming.
My personal view is that we need to get Ballista to the point of maturity where it can run industry-standard benchmarks at scale with performance and scalability at least as good as Spark. With that, and some better docs, the project hopefully starts to gain more traction and more contributors and that would eventually lead to people building ML features perhaps.
The blaze project is really interesting because it leverages the mature Spark scheduler and uses DataFusion for execution.

What are the downsides of apache spark, why somebody should use ballista?
imo as far as I read posts and watched a talk on yt, the memory consumption of spark is huge. Even a hello world in spark will consume a lot of memory. The reduced memory consumption can be a big advantage for ballista. I always have to increase the memory of our production spark jobs - sometimes to even 32GB per executor.
What can do apache spark good?
Its integration very good with the hadoop distributed file system. This is from my perspective the big advantage: the computation is taken place as close to the data as possible, moving only data when really required. Again, as far as I understand, this is currently not possible with ballista?
And, as I am a data engineer and doing a lot of analytics, I really like the spark-shell to play around with the data.
I just started using rust, so maybe I am not the biggest help for implementing features, but I know spark from a developer perspective quite well as I am using it daily.
What could be a direction: Have the data mostly/only in ram in the cluster, reducing the (slow) HDD/SSD reads and running the computation than on the arrow In-memory data frames. That maybe a niche to fit into.

What can do apache spark good?

One major advantage is that it is very mature (so has many feature, documentation, integrations, etc -- such as spark-shell)

@andygrove what benefits are you expecting from migrating to Substrait?

I think that Substrait will become the defacto standard for serializing relational algebra and has lots of smart people working on so I think we will benefit from this work and not have to build all new features ourselves and it will also potentially open up interesting integrations with the rest of the ecosystem.

I created a Google doc where we can collaborate more on this discussion and define the Ballista architecture that we are aiming for. This will provide documentation that we can put in the repo to help everyone understand how everything works and the direction we are moving towards. Contributors welcome as always.
https://docs.google.com/document/d/1Fd44vVmjSD6NuSGaFjHeI9pFSRxOIWmZ6Rv7aZ3S5Sw/edit?usp=sharing

open up interesting integrations with the rest of the ecosystem.

e.g.:

substrait test suites that show correct before and after optimization plans that we could get for free
substrait fuzzers for testing
databases in other languages wanting to optimize query plans, being able to send them to datafusion as substrait and get back optimized results as substrate
a double check that we didn't implement anything not in substrait in our model, and vice versa (i.e. apply nodes)

I also have some thoughts about a unified execution engine, welcome to take a look and comment:
https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845
cc @andygrove @mingmwang

I also have some thoughts about a unified execution engine, welcome to take a look and comment:
https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845
cc @andygrove @mingmwang

Thanks @liurenjie1024. I recently read the F1 paper (https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41344.pdf) which I found very interesting. It's quite a bit larger in scope than Ballista (it's intended to be more of a full DBMS than a query engine like Ballista), but the query processing section is very interesting. It has some very interesting properties:

The scheduler/planner can dynamically determine whether queries should be distributed or executed centrally on a single executor so you can potentially eliminate the scheduling and shuffle overhead on fast OLTP-style queries.
Distributed queries can be either batched (similar to Ballista now where each stage is fully materialized) or pipelined (where data is streamed between the stages).

Yes,deciding the execution mode(batched or pipelined) at runtime is an interesting topic, and I believe the first step is to     make ballista flexible enough to support complex job graph.

Ideally, we
http://www.vldb.org/pvldb/vol11/p746-yin.pdf

I also have some thoughts about a unified execution engine, welcome to take a look and comment:
https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845
cc @andygrove @mingmwang

How to get access to the doc?

Recently I found there are huge gaps in DataFusion's physical planning phase, I think we need to put more effort to address those gaps first.

I also have some thoughts about a unified execution engine, welcome to take a look and comment:
https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845
cc @andygrove @mingmwang

How can I get access to the doc ?

I also have some thoughts about a unified execution engine, welcome to take a look and comment:
https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845
cc @andygrove @mingmwang

How can I get access to the doc ?

I was able to view it. I think you have to be signed into Notion though (to any account)

@thinkharderdev @liurenjie1024
I think the 'Bubble Execution' model is what exactly you would like to achieve. And as I know, Alibaba's MaxCompute engine had already implemented the  'Bubble Execution' model.
https://alibaba-cloud.medium.com/dag-2-0-the-adaptive-execution-engine-that-supported-150-million-distributed-jobs-in-the-2020-31ffad8bd722
Some questions are still not clear to me as of now, particularly if we plan to bring pipeline based scheduling(
(Morsel-Driven Parallelism) into Ballista,  then what's the relationship between Bubble, Stage and Pipeline,
and how to divide the DAG into Bubbles, how to split the Stage into multiple Pipelines etc.

I think the 'Bubble Execution' model is what exactly you would like to achieve

Yes, I agree 100%.

then what's the relationship between Bubble, Stage and Pipeline,

I think generally a stage and pipeline would be the same thing. A pipeline breaker would be a map/reduce (aka repartition). A bubble would be a layer on top of that which breaks the graph into streaming subgraphs which are fully materialized at the bubble boundary.

cc @mingmwang @thinkharderdev
Sorry for late reply from a long vacation.

Some questions are still not clear to me as of now, particularly if we plan to bring the pipeline based scheduling(
(Morsel-Driven Parallelism) into Ballista, then what's the relationship between Bubble, Stage and Pipeline,
and how to divide the DAG into Bubbles, how to split the Stage into multiple Pipelines etc.

Bubble and stage
My motivation of dividing dag into bubble comes from two cases:

Subplan reuse in complex olap queries, e.g. the spool operator in this paper which materialize results into disk.
The dataflow api which can persist data explicitly.

From the bubble execution paper you mentioned, it takes into account the resource limitation of query exection, and add vertical cut when performing a bubble.
The relationship between stage and bubble is clearly described by this pic from alibaba's article:

Stage and pipeline
In fact in my proposal I didn't mention bringing morsal driven into Ballista. The key problem with morsel driven is that it uses shared state so that each pipeline is paralleled, and it's not practical in distributed execution. Presto uses morsel driven approach to execute each stage, e.g. breaking stage plan fragments into pipelines and executes them in parallel. This way it can implement two level scheduling, e.g. each stage has a parallelism and each pipeline has another parallelism. They claim that this is more flexible, e.g. a query can have more resources when cluster is idle, and less resources when it's busy. I'm not a big fan of this approach and not sure whether it's state of art.

cc @mingmwang @thinkharderdev Sorry for late reply from a long vacation.

Some questions are still not clear to me as of now, particularly if we plan to bring the pipeline based scheduling(
(Morsel-Driven Parallelism) into Ballista, then what's the relationship between Bubble, Stage and Pipeline,
and how to divide the DAG into Bubbles, how to split the Stage into multiple Pipelines etc.

Bubble and stage
My motivation of dividing dag into bubble comes from two cases:

Subplan reuse in complex olap queries, e.g. the spool operator in this paper which materialize results into disk.
The dataflow api which can persist data explicitly.

From the bubble execution paper you mentioned, it takes into account the resource limitation of query exection, and add vertical cut when performing a bubble.
The relationship between stage and bubble is clearly described by this pic from alibaba's article:
Stage and pipeline
In fact in my proposal I didn't mention bringing morsal driven into Ballista. The key problem with morsel driven is that it uses shared state so that each pipeline is paralleled, and it's not practical in distributed execution. Presto uses morsel driven approach to execute each stage, e.g. breaking stage plan fragments into pipelines and executes them in parallel. This way it can implement two level scheduling, e.g. each stage has a parallelism and each pipeline has another parallelism. They claim that this is more flexible, e.g. a query can have more resources when cluster is idle, and less resources when it's busy. I'm not a big fan of this approach and not sure whether it's state of art.

I think pipeline execution is the trend. Presto, Clickhouse, Starrocks, DuckDb, Velox,  all those are in this direction or trying
on.

I think pipeline execution is the trend. Presto, Clickhouse, Starrocks, DuckDb, Velox, all those are in this direction or trying
on.

I'm neutral to this. But if we want to support pipeline, current datafusion operators can't be reused.

In terms of pipeline execution (at least in terms of a push based, pipelined execution model), I wanted to point out that @tustvold  investigated this approach in DataFusion (and figured out a way to reuse the current operators). See https://github.com/apache/arrow-datafusion/pull/2226 which added a scheduler under a feature flag
Our eventual goal is to support running a plan on 100s of parquet files without having to fetch them all before (or concurrently). However, we currently have other things blocking this goal so additional work to the scheduler is on hold for now
You can find more detail on https://github.com/apache/arrow-datafusion/issues/2504

Our eventual goal is to support running a plan on 100s of parquet files without having to fetch them all before (or concurrently). However, we currently have other things blocking this goal so additional work to the scheduler is on hold for now

I'm a little confused here. Avoiding fetching 100s of parquet files is more like an optimizer issue?

I'm a little confused here. Avoiding fetching 100s of parquet files is more like an optimizer issue?

What I mean is that now, if you give a datafusion plan 5000 parquet files (maybe for a SUM() type query) , it will likely try to start reading / decoding all 5000 files concurrently, even if the upstream operators can only consume a small fraction at once . This means the resources (file handles and/or memory buffers) for reading all 5000 are held open during the plan.
It also means if the plan terminates early (e.g. ... LIMIT 10) a large amount of IO will be done / wasted.
What we would like to happen is that a smaller subset are read and fully processed before new ones are opened. Of course, one challenge with doing this is we still need sufficient parallelism to hide IO latencies.

I'm a little confused here. Avoiding fetching 100s of parquet files is more like an optimizer issue?

What I mean is that now, if you give a datafusion plan 5000 parquet files (maybe for a SUM() type query) , it will likely try to start reading / decoding all 5000 files concurrently, even if the downstream operators can only consume a small fraction at once . This means the resources (file handles and/or memory buffers) for reading all 5000 are held open during the plan.
It also means if the plan terminates early (e.g. ... LIMIT 10) a large amount of IO will be done / wasted.
What we would like to happen is that a smaller subset are read and fully processed before new ones are opened. Of course, one challenge with doing this is we still need sufficient parallelism to hide IO latencies.

Within a given partition FileStream will still process them sequentially right?

Within a given partition FileStream will still process them sequentially right?

Yes that is my understanding -- maybe the parquet reader has gotten fancier since I last looked at it 🤔  Sorry if I am causing confusing

I have written a more detailed doc here, welcome to comment.
cc @alamb @mingmwang @thinkharderdev


Adding other open flags

Any thoughts on adding other flags like WAL to the SQLiteOpenFlags file? On the surface it doesn't seem like any changes would have to be made to support WAL (i.e. hidden in sqlite's underlying impl). WAL is a huge step forward for high concurrency cases.

The only argument I can come up with for why not is that it might not be supported on all platforms, but other than that I am all for it :)


chore: whitelist x-client-trace-id header to be logged

What kind of change does this PR introduce?
chore
What is the new behavior?
Whitelist x-client-trace-id to be logged

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 86.667%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/monitoring/logger.ts
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
4870

Relevant Lines:
5597

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 86.667%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/monitoring/logger.ts
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
4870

Relevant Lines:
5597

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 86.667%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/monitoring/logger.ts
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
4870

Relevant Lines:
5597

💛  - Coveralls


Maester appears to be bundling an unofficial  version of Pester 5.6

Install-Module -Name Maester
Untrusted repository
You are installing the modules from an untrusted repository. If you trust this repository, change its
InstallationPolicy value by running the Set-PSRepository cmdlet. Are you sure you want to install the modules from
'PSGallery'?
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "N"): Y
WARNING: Module 'Pester' version '3.4.0' published by 'CN=Microsoft Windows, O=Microsoft Corporation, L=Redmond, S=Washington, C=US' will be superceded by version '5.6.0' published by 'CN=Jakub Jareš, O=Jakub Jareš, L=Praha, C=CZ'. If you do not trust the new publisher, uninstall the module.

This is a known issue on Windows and is because Windows out of the box includes an older version of Pester that was signed by Microsoft. Pester is now open sourced and is maintained by the community. The older version is no longer used.
See the guide at
https://maester.dev/docs/installation
To avoid the warning run this command before installing Maester.
Install-Module Pester -SkipPublisherCheck -Force -Scope CurrentUser

@merill, would it be helpful to add an optional -Prerequisites switch to Install-MaesterTests that includes something like this?
if ( ((Get-Module Pester -ListAvailable).Version | Sort-Object -Descending -Top 1) -lt '5.6.0') {
    Install-Module Pester -SkipPublisherCheck -Force -Scope CurrentUser
}

That's a great idea @SamErde  why didn't I think of that!!
Would you like to submit a PR?

Will do!

Why stop at just Pester?! This just became good inspiration for a function to install any missing required module versions.

Thanks @SamErde the latest version now does the auto-install of Pester during Install-MaesterTests.


Injection warning

class MyBloc extends Bloc<MyBlocEvent, MyBlocState> {
  MyBloc(this._repo) : super(MyBlocState.initial());
final Repository<Todo> _repo;

main
  Directory _dir;
  configureRepositoryLocalStorage(
    baseDirFn: () => _dir.path,
  );
  final _dirName = await getApplicationDocumentsDirectory();
  _dir = await _dirName.create();
  _dir.deleteSync(recursive: true);

   GetIt.instance.registerRepositories(
     baseDirFn: () => _dir.path,
   );
   await GetIt.instance.allReady();

Although injection works fine, I'm getting this warning, Is there any way to get rid of this?

[MyBloc] depends on [Repository] which is not injectable!
Did you forget to annotate the above classe(s) or their implementation with @injectable?

Sorry, this is unrelated to Flutter Data.
It seems that Injectable does not work with classes already registered with GetIt?


“UnicodeDecodeError” when using the function “readshapefile”

When I read a shapefile by using “readshapefile”, the error “UnicodeDecodeError” happened,like this:
“UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 0: invalid continuation byte”
I realized the encoding type of the shapefile isn’t “utf-8”, so I read the documentation of Basemap.
Here: https://matplotlib.org/basemap/api/basemap_api.html
readshapefile(shapefile, name, drawbounds=True, zorder=None, linewidth=0.5, color='k', antialiased=1, ax=None, default_encoding='utf-8')
Then I found the parameter “default_encoding=’utf-8’”, so I changed this parameter as “default_encoding=’ANSI’”. But the error didn’t change, still “UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 0: invalid continuation byte”.
So, this is the problem: I already changed the parameter, if the encoding type of the shapefile isn’t “ANSI”, the error should be “UnicodeDecodeError: 'mbcs' codec can't decode bytes in position 0—1” instead of “utf-8”.
So is the parameter “default_encoding=’utf-8’” fixed?  I can’t change the parameter??

I have a similar error to this, only when I attempt to plot counties, which can also be traced back to shapefile.py. This started when I reinstalled Anaconda last night (using python 3.6, matplotlib 3.0.2 and basemap 1.2.0). I tried changing to basemap 1.1.0 and it still didn't work.
File "plot_all.py", line 164, in
m.drawcounties(linewidth=0.1,color='#222222',zorder=10)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap_init_.py", line 1981, in drawcounties
default_encoding='latin-1',drawbounds=drawbounds)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap_init_.py", line 2148, in readshapefile
for shprec in shf.shapeRecords():
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 999, in shapeRecords
for rec in zip(self.shapes(), self.records())]
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 971, in records
r = self.__record(oid=i)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 946, in __record
value = u(value, self.encoding, self.encodingErrors)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 104, in u
return v.decode(encoding, encodingErrors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 2: invalid continuation byte

I have a similar error to this, only when I attempt to plot counties, which can also be traced back to shapefile.py. This started when I reinstalled Anaconda last night (using python 3.6, matplotlib 3.0.2 and basemap 1.2.0). I tried changing to basemap 1.1.0 and it still didn't work.
File "plot_all.py", line 164, in
m.drawcounties(linewidth=0.1,color='#222222',zorder=10)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py", line 1981, in drawcounties
default_encoding='latin-1',drawbounds=drawbounds)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py", line 2148, in readshapefile
for shprec in shf.shapeRecords():
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 999, in shapeRecords
for rec in zip(self.shapes(), self.records())]
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 971, in records
r = self.__record(oid=i)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 946, in __record
value = u(value, self.encoding, self.encodingErrors)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 104, in u
return v.decode(encoding, encodingErrors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 2: invalid continuation byte

Yes, the parameter "default-encoding" doenn't work, so it can only read the shapefile which is encoding by "utf-8". I already emailed the author. Maybe he is fixing it.

pyshp apparently changed how they accept the encoding parameter, so it is
now broken. I hope to get a fix out later today.
On Thu, Jan 3, 2019 at 2:02 AM Jingwang Li<EMAIL_ADDRESS>wrote:

I have a similar error to this, only when I attempt to plot counties,
which can also be traced back to shapefile.py. This started when I
reinstalled Anaconda last night (using python 3.6, matplotlib 3.0.2 and
basemap 1.2.0). I tried changing to basemap 1.1.0 and it still didn't work.
File "plot_all.py", line 164, in
m.drawcounties(linewidth=0.1,color='#222222',zorder=10)
File
"C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py",
line 1981, in drawcounties
default_encoding='latin-1',drawbounds=drawbounds)
File
"C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py",
line 2148, in readshapefile
for shprec in shf.shapeRecords():
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 999,
in shapeRecords
for rec in zip(self.shapes(), self.records())]
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 971,
in records
r = self.__record(oid=i)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 946,
in __record
value = u(value, self.encoding, self.encodingErrors)
File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 104,
in u
return v.decode(encoding, encodingErrors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 2:
invalid continuation byte
Yes, the parameter "default-encoding" doenn't work, so it can only read
the shapefile which is encoding by "utf-8". I already emailed the author.
Maybe he is fixing it.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/matplotlib/basemap/issues/449#issuecomment-451068081,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AARy-JpxCte2ux7feTrlZxoj5KtS8ayoks5u_asMgaJpZM4ZhnK3
.

pyshp apparently changed how they accept the encoding parameter, so it is now broken. I hope to get a fix out later today.
…
On Thu, Jan 3, 2019 at 2:02 AM Jingwang Li @.***> wrote: I have a similar error to this, only when I attempt to plot counties, which can also be traced back to shapefile.py. This started when I reinstalled Anaconda last night (using python 3.6, matplotlib 3.0.2 and basemap 1.2.0). I tried changing to basemap 1.1.0 and it still didn't work. File "plot_all.py", line 164, in m.drawcounties(linewidth=0.1,color='#222222',zorder=10) File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py", line 1981, in drawcounties default_encoding='latin-1',drawbounds=drawbounds) File "C:\Users\Tomer\Anaconda3\lib\site-packages\mpl_toolkits\basemap__init__.py", line 2148, in readshapefile for shprec in shf.shapeRecords(): File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 999, in shapeRecords for rec in zip(self.shapes(), self.records())] File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 971, in records r = self.__record(oid=i) File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 946, in __record value = u(value, self.encoding, self.encodingErrors) File "C:\Users\Tomer\Anaconda3\lib\site-packages\shapefile.py", line 104, in u return v.decode(encoding, encodingErrors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 2: invalid continuation byte Yes, the parameter "default-encoding" doenn't work, so it can only read the shapefile which is encoding by "utf-8". I already emailed the author. Maybe he is fixing it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#449 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AARy-JpxCte2ux7feTrlZxoj5KtS8ayoks5u_asMgaJpZM4ZhnK3 .

Thanks a lot! I can't wait to use Basemap.

Hello, I seem to have the same problem with pyshp.
Python 3.7.2 (default, Jan 10 2019, 23:51:51)
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import shapefile as shp
>>> shapefile = shp.Reader("Restrictions_for_Drones.shp")
>>> for shape in shapefile.shapeRecords():
...     print(shape)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/noskill/projects/airaid/software/dev/lib/python3.7/site-packages/shapefile.py", line 1039, in shapeRecords
    for rec in zip(self.shapes(), self.records())])
  File "/home/noskill/projects/airaid/software/dev/lib/python3.7/site-packages/shapefile.py", line 1012, in records
    r = self.__record(oid=i)
  File "/home/noskill/projects/airaid/software/dev/lib/python3.7/site-packages/shapefile.py", line 987, in __record
    value = u(value, self.encoding, self.encodingErrors)
  File "/home/noskill/projects/airaid/software/dev/lib/python3.7/site-packages/shapefile.py", line 104, in u
    return v.decode(encoding, encodingErrors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfc in position 28: invalid start byte

Here's the zip from which the shapefile comes:
data.zip

I also have this problem how to fix it ?

Hey @WeatherGod - did you implement the fix? Much appreciated.

Give https://github.com/matplotlib/basemap/pull/459 a try. I haven't had time to make a unit test to verify.

Thanks, I'll take a look.


Support tslib 2.3.0+

Bug Report
Current Behavior
https://github.com/ReactiveX/rxjs/blob/c2b3e88b9c2dbb9965833ca130b60d6466ef9552/package.json#L143
tslib is set to ~2.1.0 which conflicts with ^2.3.0 used in other projects causing duplicate tslib modules
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ ├─┬<EMAIL_ADDRESS>│ │ └──<EMAIL_ADDRESS>deduped
│ ├─┬<EMAIL_ADDRESS>│ │ └──<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ └──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>
This duplication is adds ~0.6 Kb to a compiled project.
Expected behavior
tslib to be versioned such that there are not conflicts with other commonly used packages.
See:
https://github.com/ng-packagr/ng-packagr/issues/2174#issuecomment-979220267
Environment

RxJS version: ~7.4.0

Possible Solution
Perhaps a version of ^2.1.0 would be suitable to continue supporting older versions of packages (like Angular 12) but also supporting Angular 13

Perhaps a version of ^2.1.0 would be suitable to continue supporting older versions of packages (like Angular 12) but also supporting Angular 13

This solution seems reasonable to me on it's face. I'm not sure if there are other implications. Generally TypeScript libraries aren't the best about honoring semver. 🤔

Thank you for the timely resolution!

I have a lot of stuff to go through, so the next release may not be until the end of the week or later. Thank you for your contribution!


add support for multiple TMS read support in Backends

continue work started in #210
TODO

[x] tests
[x] documentation

cc @AndrewAnnex

@AndrewAnnex this is ready for review if you want to try 🙏
(FYI: I will need to merge this before the end of the week)

alright thanks for the review @AndrewAnnex 🙏
Let me know if you want to have another look or maybe try yourself before we merge this

@vincentsarago nope, I think it's good to go, or rather, I don't have time to test it prior to the deadline you mentioned. I'm sure any fixes that are needed would be small


react-static + i18next

Does anyone have experience integrating react-static with i18next or other multilingual libraries?
I can't configure the SSR correctly in any way. I tried to use the gatsby article of 2017, but I couldn't make friends with the configs.
I will be glad of any advice and sorry for my English

Hi @VovaParamonov,
I used https://github.com/woffrek/react-static-plugin-localized plugin for multilingual support. Not ideal, but works

Thanks, @sti-web


Use Webpack bundle tracker v1

This PR adds support for webpack-bundle-tracker v1 and the new stats format.

What else is left on this PR? I can help if there's any actionable items left

Any update on this PR? world be great to have this version

Any update on this PR? world be great to have this version

I think they may be waiting on more feedback for features since this is a breaking change. However, yeah this would be nice to have ASAP!
https://github.com/owais/django-webpack-loader/issues/215

Good point about this being a breaking change.  How about adding a setting to have this disabled by default (thus not breaking), and eventually (several months from now?) the default can change

As far as I understand this a expected breaking change. This is ready to go, but @owais has no merged it. Let's have patience and wait for him.

it works for me thanks!

Hi @owais, it looks like this is ready to go. Any plans to merge this soon? I think it blocks the upgrade to webpack v5.

Hi @owais, it looks like this is ready to go. Any plans to merge this soon? I think it blocks the upgrade to webpack v5.

Thanks for all the contributions and patience everyone. Unfortunately I cannot merge the PR until I can test, verify and understand the changes completely and I really don't have any free time to do that. I know this project has suffered quite a bit because of me moving on to other things and I'd like to pass it on to people that actually use this piece of software every day and are willing to maintain it. I've updated the readme with a message to reflect tihs.
If your organization are heavy users of django-webpack-loader and webpack-bundle-tracker, I'd love to invite you to help maintain the project and transfer both the repositories, pypi and npm packages to you. If it sounds like something your organization would like to do, please get in touch with me over email. Of course, individuals are welcome as well especially with existing track record of maintaining OSS software.

Thanks for all the contributions and patience everyone. Unfortunately I cannot merge the PR until I can test, verify and understand the changes completely and I really don't have any free time to do that. I know this project has suffered quite a bit because of me moving on to other things and I'd like to pass it on to people that actually use this piece of software every day and are willing to maintain it. I've updated the readme with a message to reflect tihs.
If your organization are heavy users of django-webpack-loader and webpack-bundle-tracker, I'd love to invite you to help maintain the project and transfer both the repositories, pypi and npm packages to you. If it sounds like something your organization would like to do, please get in touch with me over email. Of course, individuals are welcome as well especially with existing track record of maintaining OSS software.


fix mobile external login page

Description

fix the font size of the pop-up layer on the mobile external login page
Related Issue :

How Has This Been Tested?

Go to external login page. (See PR #3298 )
Login the client.
Check the font size of the pop-up layer on the mobile external login page and the web page.

Need additional checks?
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have read the CONTRIBUTING document.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have added tests to cover my changes.

Additional Context

deploy to 2


Add "Awesome MXnet" page for collecting all cool mxnet examples and tutorials/blogs

Use example/README.md for collecting mxnet examples and tutorials as suggested by @tqchen
P.S. Please ignore the previous build file change logs which had no file changes but historical commit info. They were old experiments for creating MXnet pip installation and  they have nothing todo with this example/README.md change.

Can one of the admins verify this patch?

LGTM


Fix #210: Add scikit-learn plots to README.

I started reviewing this, not noticing that it's a draft. I was getting some warnings when running generate_plots.ipynb locally, so I updated from main and now it seems to run ok without errors.
I think this is ready to merge, but I'll wait until I hear from you @anevolbap unless I'm missing something :)

@drbenvincent, feel free to merge! The error is likely related to #259, which has already been merged.


Add runn tests for 0.0.4

Currently, we are not the tests for 0.0.4, this PR includes these cases. This will make CI much longer, I wonder to what extent we need to keep running the tests in 0.0.3.
Opinions?
cc: @hayesgb and @ian-r-rose

As long as they are in the test matrix, shouldn't they be parallelized? If that's the case, I wouldn't worry too much about the extra time.

You are right @ian-r-rose , I forgot that we were parallelizing.
The failure we are seeing is related to https://github.com/coiled/coiled-runtime/issues/166 if you are ok with it would you mind merging this one in?

No objections to merging here.


Labeled button

Something went wrong with the labeled buttons: value is hidden by default, you need to hover over the value to be able to see it. I guees it happened with the latest release.
https://semantic-ui.com/elements/button.html#labeled

Duplicate of #6439


Being able to customize the line/circle width

I would like to be able to customize the design of the line and circles (especially changing their size as we can change the color already).
Maybe by adding 2 optional props for line width and circle width and height.
This is what I have:

and this is what I am trying to achieve:

Thanks!

For the moment I just override the css like:
const useStyles = makeStyles(() =>
  createStyles({
    root: {
      '& div[data-testid="tree-leaf-click"] > div': {
        width: '.5rem',
        height: '.5rem',
      },
      '& div[data-testid="tree-leaf"]': {
        '&::after': {
          width: '2px',
        },
      },
      '& #timeline-main-wrapper': {
        '&::-webkit-scrollbar-thumb': {
          backgroundColor: '#dbdbdb',
          outline: '1px solid #dbdbdb',
        },
      },
    },
  })
);

the width of the line can be customized via lineWidth and the circle dimension can be adjusted via timelineCircleDimension


Improve versioned documentation handling

The current way to import documentation from other projects is currently quite cumbersome and doesn't work correctly for multiple project.

[ ] Port to Nanoc 4
[ ] Support multiple repository sources correctly
[ ] Make it simpler to configure documentation sources without the need to copy&paste data after every release

@grobie I guess this https://github.com/prometheus/docs/pull/1383#issuecomment-508498058 refers to "Make it simpler to configure documentation sources without the need to copy&paste data after every release" item in this checklist, correct? Do you have any specific idea in mind to improve it?

@pracucci I do, I'll try to write it down next week.


only denormalize up to support of BlockVector

Also adding changes to utilise the better and more stable recurrence for sum(SemiclassicalJacobiWeight) as done in https://github.com/JuliaApproximation/SemiclassicalOrthogonalPolynomials.jl/pull/90#issue-1899382453


set-string envoy.extraArgs can not use

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
When I try to set the extra thread count parameter for envoy when I use cilium install, cilium does not install successfully.
command:
./cilium install --set-string envoy.enabled=true --set-string envoy.extraArgs=[{"concurrency":"2"}]
result:
Error: Unable to install Cilium: YAML parse error on cilium/templates/cilium-envoy/daemonset.yaml: error converting YAML to JSON: yaml: line 46: could not find expected ':'
Cilium Version
1.14.4
Kernel Version
5.10
Kubernetes Version
1.20.2
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Can you try the below instead ?
cilium install --set-string envoy.enabled=true --set-string envoy.extraArgs={"--concurrency 2"}


Error when running as root on Linux

When running as root, this error pops up and no images were saved.
[0517/064559.081037:ERROR:zygote_host_impl_linux.cc(90)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

Hello, have you tried adding the --no-sandbox flag like mentionned in this comment ?


Fully test Node-API libraries

Client and CLI tests need to pass (both GH Actions and Buildkite)
End to end tests need to pass and actually confirm the Node-API library is being used

E2e still blocked by https://github.com/prisma/e2e-tests/issues/1729


Elrond: change the token symbol

Description
We've changed the token symbol from ERD to eGLD.
Testing instructions

Types of changes

Checklist

[ ] Prefix PR title with [WIP] if necessary.
[ ] Add tests to cover changes as needed.
[ ] Update documentation as needed.
[ ] I have read the guidelines for adding a new blockchain
[ ] If there is a related Issue, mention it in the description (e.g. Fixes #<issue_number> ).

Salut @andreibancioiu : the PR looks OK, but I notice that elrond.com is referring to old symbol ERD.  Is this OK?

Salut @andreibancioiu : the PR looks OK, but I notice that elrond.com is referring to old symbol ERD. Is this OK?

Salut @catenocrypt! We have to update the site as well - thank you for noticing. In the Blockchain Explorer, we've updated the name: https://explorer.elrond.com/validators.

@andreibancioiu , OK.  One more question: the hrp prefix stays "erd", right?

@andreibancioiu , OK. One more question: the hrp prefix stays "erd", right?

Yes, it stays the same. The addresses will still be like erd1l453hd0gt5gzdp7czpuall8ggt2dcv5zwmfdf3sd3lguxseux2fsmsgldz.


Avoiding duplicate "completed" calls

I'm using a SwinjectStoryboard extension to set up injection for my main VC. That's working fine. I added the following to set up injection for another VC in the same storyboard. This view controller is reached by a segue from the initial view controller:
defaultContainer.storyboardInitCompleted(DetailViewController.self) { (r:Resolver, c:DetailViewController) in
    c.bar = "Injected bar"
}

[I'm trying out SwinjectStoryboard with Xcode's default master/detail view app template, in case that's not obvious.]
Here's the thing: With the above code, injection works, but the closure is called twice with different instances of DetailViewController. The first is when setup() is called, and the second is when the DetailViewController appears on the screen. If I set a breakpoint in the closure, c is a different instance in the second call, and it's the second one that appears. I don't know why the first is created or what happens to it.
Why do I get two calls to this closure with different instances, and how would I avoid that?

Does v1.1.2 reproduce the issue?

Closing due to inactivity


External Specification References

Adds a log during render if there are any unused defs or unresolved refs
Allows an external_specs list option in the spec config that is a name/value object where the value is a URL of an external (spec-up) specification
Creates a new x-ref tag to use external spec references [[x-ref: {name}, {ref}]] where {name} is from above external_specs and {ref} is the definition within the specification that is being referred to

It also closes #56

I'll take a look at this over the weekend, thanks for the contribution!

Awesome, thanks!

My only question is does the code try to do external fetch upon every watched change when editing, or is it caching at all? Just wouldn't want render to freeze up based on unavailable external resources when editing or building via automated means.

any chance we can get this PR approved? @csuwildcat

@darrellodonnell will do now!

@brianorwhatever would it be possible to resolve those conflicts? They shouldn't be anything big, I think it's just the auto-gen'd HTML and the lock via the recent dependency update.

I will be sure to merge right after, I didn't see that last update of yours until just now.

@csuwildcat done!


Add getGenericSignature() helper method to ArchetypeBuilder

This is just a small re-factoring moving some code around to make it possible for an ArchetypeBuilder to create a generic signature based on a provided list of generic parameter types and a set of archetype and requirements already known to the builder.

@DougGregor @rjmccall Doug, John, could you have a look at this PR? This is the small of re-factoring we discussed a few days ago.
Let me know if anything is missing.


How to unload children nodes

Hello guys!
I have a problem... I using ajax to load children nodes, but in case when I click option in context menu I have to unload children from the one specific node and make it as jstree-leaf.
I'm refreshing the node with refresh_node function, but it's making load_node, which is downloading first level of children and I can't stop it.
If I make inst.refresh() then all nodes would be refreshed so it works, but if other nodes were having children, then their children also would be unloaded, which I don't want to do.
unload: {
                label: "Unload",

action: function (data) {
                    var inst = $.jstree.reference(node.id);

                    inst.refresh_node(node);

                },

Do you have any ideas how to do it?
Best regards

Hi,
If I understood correctly, calling refresh_node on the node's parent should do the trick.
Best regards,

Unfortunately it's not. In my case I have large tree with many nodes with lazy loading by ajax, and these nodes have option unload in menu. And what I want to do is when one of these nodes have children use this option to "unload" them from this node and make this node jstree-leaf again. Refresh_node() parent not works, exactly do nothing at all. Refresh() tree works, but I want to unload children of just one node, not of all nodes in the tree.

Ok, I already found a solution for this. I'm refreshing this node and returning just one element from the server, then catching loaded.jstree event and after that deleting this useless child node.
I was can't delete child nodes without this, because this is a large tree and deleting all of them was making page freezed.

I have noticed with the suggested solution of refreshing the parent comes with the reloading of all siblings and their open children which is both undesirable in terms of efficiency as well as ugly in terms of animation.
I propose the following solution:
$('#jsree').on("after_close.jstree", function(e, data) {
    for (const child_id of data.node.children.slice()){
        // dictionary to ignore operations that do not need to be forwarded to server
        // useful in logic that forwards tree operations (must remove child_id after ignoring)
        ignore_ajax_request_dict['delete_node'].add(child_id)
        $("#jsree").jstree('delete_node', child_id);
    }

    data.node.state.loaded=false;
    data.node.children = true;
    $("#jsree").jstree('redraw_node', data.node);
});


fmt_number is mutating source data, so multiple calls of same code produces error

I'll look into it..!
from great_tables import GT
from great_tables.data import exibble

gt_ex = GT(exibble)

# works
gt_ex.fmt_number(columns="num", decimals=1, scale_by=1/1000, pattern="{x}K")

# errors because num is now strings
gt_ex.fmt_number(columns="num", decimals=1, scale_by=1/1000, pattern="{x}K")

@machow It still seems to be an issue with the fmt_integer() example at https://posit-dev.github.io/great-tables/examples-qmd/fmt-number.html


username disappears when guard()->attempt()

Laravel Version: 5.6.15
Adldap2-Laravel Version: 8.1.4
PHP Version: 7.1
LDAP Type:  ActiveDirectory

I can import LDAP users with the php artisan adldap:import command without any issues; queries AD, finds the user and inserts it in the local database.
When I try to login using the login.blade.php file, the logincontroller doesn't get the 'username' value - I get the 'A username must be specified.' exception.  The POST Data has a _token, username and password.
I've modified my login.blade.php file to replace 'email' with 'username'.  I've added the public function username override in the LoginController.php file.
I even changed the AuthenticatesUsers username function from 'email' to 'username' to see if that would help and I'm getting the same error in Guard.php.  I even swapped the empty password and empty username check to make sure the password was getting through and it is.
For some reason it just seems like 'username' isn't getting from the login.blade.php to the login controller.
The session is empty in the Whoops Environment and details data.
I just have Auth::routes(); in my web.php file.  Do I need to create a specific post route with username passed to the controller?
I did move my Eloquent User model to a Models folder if that matters; I can register a new user using it though.
Any help would be greatly appreciated.

In AuthenticatesUsers.php, if I dd($request) in the attempLogin(Request, $request) I get then entire $request with the +request: ParameterBag with an array of the the _token, username and password with the correct values.
If I remove that dd and move it to the credentials function and dd($request->only($this->username(), 'password'); I get the correct username value and password value in an array.
Back in the attemptLogin function, the attempt(this->credentials($request), $request->filled('remember')); function is called in the guard object.
in the Guard.php attempt function, if I dd($username, $password, $bindAsUser) before the validateCredentials - the $username is now NULL; the password and $bindAsUser are correct.

I'm not getting the 'remember' POST data from the login form for some reason.

Are you sure you're using a name attribute on you login form and not just an id?
eg.
<input type="text" id="username" name="username" />

No name attribute causes the form not to submit data.

I believe my form is correct:

{{ __('Username') }}

                            <input id="username" type="text" class="form-control{{ $errors->has('username') ? ' is-invalid' : '' }}" name="username" value="{{ old('username') }}" required autofocus>

                            @if ($errors->has('username'))

                                    {{ $errors->first('username') }}

                            @endif

The $request has '_token', 'username', 'password'.  The 'password' field comes though fine.

Thanks Greg! Appreciate you posting your solution. Was away on vacation until today so I wasn't able to assist.
If you have any further issues just create another and I can help you out.


terminology: Use the term "global time" to refer to time widget.

@timabbott @alya: FYI :)

@timabbott: Done! Thanks for the feedback! :)

Merged, thanks @eeshangarg!


How to extract transformation matrix from superposition class

Hi, I am wondering if there is an easy way to extract an all in one transformation matrix from this code https://github.com/arose/ngl/blob/950e09f4f8726a479d4eed128f42c90b2891f6d1/src/align/superposition.ts#L140-L143

Found a way


Improve error handling and properly parse URL on Windows

Two bug fixes:

Properly handle the case when there is an error calling os.Stat and it's not a error because the file doesn't exist (it may be good to log this error some how, but I'm not sure the best way to do that so I left it out).

Properly parse the URL on Windows. On Windows, os.PathSeparator will be \. There are no \ characters in a URL, so parsing would fail and the created path would be weird and contain invalid characters on Windows. This will parse always using a /, so it should work on both Unix & Windows.

@csterwa


feat(v2): Handle fs imports for @mdx-js/runtime usage

Motivation
I want to move mdx parsing to client via @mdx-js/runtime and it requires that fs is somehow mocked for client
https://github.com/mdx-js/mdx/issues/913
Have you read the Contributing Guidelines on pull requests?
yes
Test Plan
Try use import MDX from '@mdx-js/runtime' in any .mdx file. Should build without errors.
Related PRs
Nope

Hi,
To me, the solution is not to hardcode your requirement in core, but to provide this requirement as a local site plugin.
You should be able to add this webpack config through the configureWebpack lifecycle: https://v2.docusaurus.io/docs/lifecycle-apis#configurewebpackconfig-isserver-utils
If it's not the case we'll reopen and look for a solution

BTW I'm curious why do you want to MDX parsing to the client?

To many big markdown files (50 Kb+) which heavily increase memory consumption in build

And thanks for alternative solution! It is better indeed


Update the opam 2.1 support to use the new 2.1 branch instead of master

opam 2.1 just branched from master so this PR is here to update dockerfile-opam to use this new branch instead of master

Given that there's going to be opam-2.0, opam-2.1 and opam-2.2 as soon as there's something meaningful on opam's master branch, would it be worth generalising hash_opam_2_0:string -> hash_opam_2_1:string to opams:(string * string) list so passing in a list of branchname, sha pairs? Otherwise the API changes again in a week or so?

yes indeed it is worth generalizing but I think this is ok if it will change again next week or so. I feel like this is better to split the urgent PR from the more prone to discussion API change.

Agreed, we can make that change to also install opam-2.2 once this is rebuilt.


Valid e-mail recipient specification rejected (special characters)

Checklist

[X] I have used the search function to see if someone else has already submitted the same bug report.
[X] I will describe the problem with as much detail as possible.

App version
6.600
Where did you get the app from?
F-Droid
Android version
11
Device model
QX1050 (FX Technology Pro1x)
Steps to reproduce

Either copy e-mail address together with the sender name containing accented characters from a received e-mail (using the new functionality for displaying sender details) and paste it into the recipient address of a newly composed message, or type such an address anew - as an example, use something like: "Tomáš Marný"<EMAIL_ADDRESS>(starting with the quote mark and finishing with the greater than sign).

Press "Tab" key, or change to a different field.

An exclamation mark appears next to the provided address. Clicking on the exclamation mark results in displaying "Special characters are not supported at the moment" (actually, I see a localized version of this message, thus the real English version of this message may be slightly different).

The point is - the recipient specification written this way is perfectly valid (even copied by K9 from a received message), but K9 refuses it.
Expected behavior
No exclamation mark is displayed, provided address is used for the "To" header of the composed message and sent to the address between the "less than" and "greater than" signs after user sends the message.
Actual behavior
The recipient specification is refused as described above.
Logs
k9mail-logs_230421.txt

Checklist

[x]  I have used the search function to see if someone else has already submitted the same bug report.
[x]  I will describe the problem with as much detail as possible.

App version
6.600
Where did you get the app from?
F-Droid
Android version
11
Device model
QX1050 (FX Technology Pro1x)
Steps to reproduce

Either copy e-mail address together with the sender name containing accented characters from a received e-mail (using the new functionality for displaying sender details) and paste it into the recipient address of a newly composed message, or type such an address anew - as an example, use something like: "Tomáš Marný"<EMAIL_ADDRESS>(starting with the quote mark and finishing with the greater than sign).
Press "Tab" key, or change to a different field.
An exclamation mark appears next to the provided address. Clicking on the exclamation mark results in displaying "Special characters are not supported at the moment" (actually, I see a localized version of this message, thus the real English version of this message may be slightly different).

The point is - the recipient specification written this way is perfectly valid (even copied by K9 from a received message), but K9 refuses it.
Expected behavior
No exclamation mark is displayed, provided address is used for the "To" header of the composed message and sent to the address between the "less than" and "greater than" signs after user sends the message.
Actual behavior
The recipient specification is refused as described above.
Logs
k9mail-logs_230421.txt

Hello @cketti .
We have thoroughly tested and replicated the bug scenario, and it has occurred for us on the following device and software configuration:
Phone Model: Samsung A6
Android Version: 10
K9 App Version: 6.702
App Source: F-Droid

Hello @cketti
My attempt to fix this problem led me to this part of the code below in function defaultObject, class RecipientSelectView:
 if (!CharsetUtil.isASCII(completionText)) {
            setError(getContext().getString(R.string.recipient_error_non_ascii));
            return null;
        }

In merge request #1535, which was for Issue #1328, this code was added to solve that issue.
Of course, it is mentioned there that: "This doesn't fix #1328, but it improves error handling a bit."
In light of these things, what approach do you suggest I take to fix this bug?

Hello @cketti My attempt to fix this problem led me to this part of the code below in function defaultObject, class RecipientSelectView:
 if (!CharsetUtil.isASCII(completionText)) {
            setError(getContext().getString(R.string.recipient_error_non_ascii));
            return null;
        }

In merge request #1535, which was for Issue #1328, this code was added to solve that issue. Of course, it is mentioned there that: "This doesn't fix #1328, but it improves error handling a bit."
In light of these things, what approach do you suggest I take to fix this bug?

As far as I can see, there are two issues:

The check added in MR #1535 ignores the difference between the real address (provided between the "<" and ">" characters) and the display name included in quotes preceding the real address. Only the real address should be used within the SMTP control communication; the display name is just provided in the message header and the SMTP server shouldn't have any issues with it. This part seems to be a regression compared to situation before MR #1535.

The original issue described in issue #2633 related to international domain names (IDN) and international e-mail addresses which requires special handling of such characters within the SMTP communication (e.g. conversion of these addresses to UTF-8 among others).

Solving the first one should be rather simple and can be done independently from the second one. Solving the second one probably requires much more effort and possibly might depend on support within underlying components used in K-9.

Right now we don't have a dedicated parser for a user-entered address list that could be used to make the distinction between the name and the email address. The parser that is currently used doesn't support (unencoded) special characters anywhere in the input. That's why the error message was added.

Thanks for your response. I may be wrong, but it seemed to me that the
original issue (preceding the patch implementing the error message) was
related to a special character within the domain of the e-mail address.
I don't know whether there would be some problem if the special
character appeared in the display name part, but I wouldn't think so -
the message from the mail server administrator mentioned a particular
extension of a SMTP protocol, but I don't think that this extension
would be necessary for handling special characters in the display name
part of the address. Am I wrong?
Tomas

Thank you, I'll test your fix as soon as the change becomes available in the F-Droid repository. :-)


minions option on rating table does not work

the recent option to put the minions to appear is not working, I put in 1 and even then they do not appear as normal players in the leaderboard

Made a quite stupid and simple mistake, should be fixed once #1278 gets merged.


How about the HTML document that rendered by client script?

After an overview of source code.
I found the node-crawler use request to make a http request.
But if the page i crawl that rendered by some client technology such as: AngularJS、ReactJS....
I may can't get the correct DOM of the page?

@JerryC8080 you would want to use Phantom JS for that.

@scott-wyatt
Ok, i am so sorry for this such lower question.
I just like the API of node-crawler.


feat: revert of the revert of feat: dashboard qr code button (#4378)

Description

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

closes #
Risk

Testing

Engineering

Operations

Screenshots (if applicable)

Current dependencies on/for this PR:

develop

PR #4406   👈

This comment was auto-generated by Graphite.


Update schema-directives.md

fix typo found in https://github.com/strawberry-graphql/strawberry/issues/1666
Types of Changes

[ ] Core
[ ] Bugfix
[ ] New feature
[ ] Enhancement/optimization
[x] Documentation

Issues Fixed or Closed by This PR
https://github.com/strawberry-graphql/strawberry/issues/1666

Thanks for contributing to Strawberry! 🎉 You've been invited to join
the Strawberry GraphQL organisation 😊
You can also request a free sticker by filling this form: https://forms.gle/dmnfQUPoY5gZbVT67
And don't forget to join our discord server: https://strawberry.rocks/discord 🔥


Newbie trying to compile for Win64

Hi,
I was over the original page and following the install instructions and got stuck.
I found your repo and cloned it... and can't figure out how to compile it for Windows.
Is there a full description on how to build it?
This old one is what I found:
https://basile.b.gitlab.io/dexed/build.html
Thanks so much!

When you cloned it, did you also check out the submodules? (Meaning, you have all the folders shown here in your copy of the "etc" folder).
If so, what was the issue you were running into exactly? Are you getting a compiler error while building the Lazarus project file?

Basile has now started working on Dexed again here, and provides prebuilt Windows 64 binaries directly.


Added remome-offline-runners script

Removes offline runners caused by bugs/mishaps in the charm. Most of it not written by me, but by Canonical colleague. My modifications are:

page_size=100 on query call
while True loop to run the process until there are no more runners to remove

The test failure I'm guessing is because the runners weren't available in staging, @yhaliaw should be able to get the to pass when he is back

I have rerun the e2e-test.
This PR is ready for merging.


Certificate for deb.goaccess.io expired

The current certificate for deb.goaccess.io (03:40:33:2C:8B:AC:AF:10:1B:CE:4D:9D:65:81:CF:AB:5F:2F) expired today (exactly May 8, 2019, 6:36:27 PM GMT) and is no longer valid.

Thanks a lot for the heads up on this. It should be back up again.

Yes. Looks fine from here. Thanks for the fast response.


BE-106 Increase Enable Updatable Fee

merged in https://github.com/public-awesome/launchpad/pull/601


Fix "notifier" payload inside Context

We send language information (for example "C#/NETCORE"), however we don't send "notifier" payload.
Example payload:
"notifier": {
    "name": "Sharpbrake",
    "url": "https://github.com/airbrake/sharpbrake"
    "version": "3.0.1"
  },

It seems like we are supposed to send it but it appears to be broken. The payload I see in the dashboard is this:
{
  "hostname": "",
  "language": "C#/NETCORE",
  "os": "Microsoft Windows 10.0.14393 ",
  "remoteAddr": "[redacted ;)]"
}

Give me 30 min to check that.

Fixed by https://github.com/airbrake/sharpbrake/pull/46.


iPadOS: cannot move panes around (activity bar, panel)

Refs: #85990
We probably did not hook up drag and drop support for moving views around.

Somewhat related, I cannot drag things around when a mouse is connected to the iPad, e.g. tried to move tabs.

@bpasero what you are seeing is https://github.com/microsoft/vscode/issues/85254#issuecomment-629917537.

Tested mouse/trackpad with latest iPadOS, iPadOS now emits right mousedown, mousemove, and mouseup event so technically we can listen to these events to support drag & drop for panels/views.

The challange here is that in a lot of places like the list view we actually listen to dragover, drop, dragleave and dragend.
I guess the fix would be for the DragAndDropObserver to emulate the dragover events my listening on the mousedown, mousemove and mouseup
Apart from that the List and the AbstractTree would need to adopt and use the DragAndDropObserver. fyi @joaomoreno
I can look into this, assigning to April.


Negatable Optional<Boolean> not recognized when annotation processor is enabled.

I have an option like this:
    @Option(names = "--progress", paramLabel = "<progress>", description = "Show query progress", negatable = true)
    public Optional<Boolean> progress;

and I'm getting this error when building the project with Maven:
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/jwas/src/trino/client/trino-cli/src/main/java/io/trino/cli/ClientOptions.java:[144,30] progress must be a boolean: only boolean options can be negatable.

when I enable the annotation processor in the pom.xml file:
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <annotationProcessorPaths>
                        <path>
                            <groupId>info.picocli</groupId>
                            <artifactId>picocli-codegen</artifactId>
                            <version>4.6.3</version>
                        </path>
                    </annotationProcessorPaths>
                    <compilerArgs>
                        <arg>-Aproject=${project.groupId}/${project.artifactId}</arg>
                    </compilerArgs>
                </configuration>
            </plugin>

It works without the annotation processor.
I can't really avoid using the Optional and I'm trying to build a static binary using GraalVM.

I'd love to try! Any pointers?

I've been looking at the annotation processor tests but there's a lot of different kinds of tests and I can't find a basic one that'd cover most data types.


WIP

Signed-off-by: Michael Gasch<EMAIL_ADDRESS>

Codecov Report

Merging #68 (15c6a45) into main (a4ff1ac) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #68   +/-   ##
=======================================
  Coverage   36.23%   36.23%
=======================================
  Files           1        1
  Lines          69       69
=======================================
  Hits           25       25
  Misses         42       42
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a4ff1ac...15c6a45. Read the comment docs.


Compatibility with https://github.com/LiberTEM/LiberTEM/pull/1207/

On my system I get test failures from access violations, but I assume they are unrelated.
Contributor Checklist:

[x] I have added or updated my entry in the creators.json file
[ ] I have added a changelog entry for my contribution
[N/A] I have added/updated documentation for all user-facing changes
[N/A] I have added/updated test cases

Reviewer Checklist:

[ ] /azp run libertem.libertem-live-data passed

/azp run libertem.libertem-live-data

/azp run libertem.libertem-live-data

/azp run libertem.libertem-live-data

/azp run libertem.libertem-live-data

/azp run libertem.libertem-live-data

/azp run libertem.libertem-live-data


⚠ Bofrost.be has degraded performance

In f59be0d, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1171 ms

Resolved: Bofrost.be performance has improved in 522c1f1 after 20 minutes.


add template of social account and fix some bug

add Client ID and Client secret
add template of social account

good job 👏 👍


Making FncValues IndexMode public

Hello Andreas,
While using your library, I was trying to create a new RecordLayout, but I couldn't use the setIndexMode public method to initialize a function value, because the IndexMode enum wasn't public.

Good point. That might the case for other enums as well. Thanks, I will have a look for these kinds of issues.


nixos/sshd: update kexAlgorithms, fix links

Motivation for this change
The curve25519-sha256 key exchange method is defined in RFC 8731 that is identical to<EMAIL_ADDRESS>OpenSSH supports the method since version 7.4, released on 2016-12-19. It is literally a violation of the "both in Secure Secure Shell and Mozilla guidelines" rule, but it provides essentially the same but a future-proof default.
Also, links to the Mozilla OpenSSH guidelines are updated to refer to the current place.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/254


Unable to run backstage with the snyk plugin using the latest cli (webpack 5)

I'm getting errors like the following when I try to run backstage with the snyk plugin:
[0] ERROR in<EMAIL_ADDRESS>4:15-32
[0] Module not found: Error: Can't resolve 'crypto' in '/Users/iainbillett/code/backstage/node_modules/@snyk/dep-graph/dist/legacy'
[0]
[0] BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
[0] This is no longer the case. Verify if you need this module and configure a polyfill for it.
[0]
[0] If you want to include a polyfill, you need to:
[0]     - add a fallback 'resolve.fallback: { "crypto": require.resolve("crypto-browserify") }'
[0]     - install 'crypto-browserify'
[0] If you don't want to include a polyfill, you can use an empty module like this:
[0]     resolve.fallback: { "crypto": false }

Expected Behavior
I can use the snyk plugin (backstage-plugin-snyk)
Current Behavior
When I start the app (e.g. yarn dev) webpack fails with the error above.
Possible Solution
I've tested setting resolve.fallback.crypto = false in packages/cli/src/lib/bundler/config.ts but presumably this could cause a runtime issue. It does compile though.
Steps to Reproduce

Create a backstage instance (or upgrade an existing one) e.g. npx @backstage/create-app
Follow instructions to install the Snyk plugin
run yarn dev

Context
I've made an issue with snyk also.

Snyk does not expect to remove the dependcy on crypto any time soon so it would make sense to support extending the webpack config to allow for adding the crypto-browserify polyfill again.
https://github.com/snyk-tech-services/backstage-plugin-snyk/issues/29#issuecomment-899443555

I think reintroducing all of the polyfills that we removed is the way to go here, #6871. Sorry about the trouble it caused

Nice, that should work. 🚀


Upgrade golang version in README, and mod

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 55.874%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
3819

Relevant Lines:
6835

💛  - Coveralls


fix: ensure termination signals are forwarded to java process

Add Tini and use it as the default entrypoint. Also, add gnupg2
dependency to downloader layer to enable gpg verification of Tini
executable before copying to final layer.
Fixes #87

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

hm, not sure what's going on with CI build; looks like maybe some env vars aren't properly configured in Travis?
$ git clone https://github.com/Graylog2/graylog-docker.git graylog-docker \
  && cd $_
Cloning into 'graylog-docker'...
remote: Enumerating objects: 25, done.
remote: Counting objects: 100% (25/25), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 506 (delta 9), reused 18 (delta 3), pack-reused 481
Receiving objects: 100% (506/506), 154.99 KiB | 1.57 MiB/s, done.
Resolving deltas: 100% (259/259), done.

$ git fetch origin pull/88/head:pr-88
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (3/3), done.
From https://github.com/Graylog2/graylog-docker
 * [new ref]         refs/pull/88/head -> pr-88

$ git checkout pr-88
Switched to branch 'pr-88'

$ make &>/dev/null

$ echo $?
0

Any update on this from Graylog maintainers? We'd love to have this fix
merged in.
On Mon, Aug 12, 2019, 11:07 AM Kyle Smith<EMAIL_ADDRESS>wrote:

@knksmith57 commented on this pull request.
In Dockerfile
https://github.com/Graylog2/graylog-docker/pull/88#discussion_r313004060
:

@@ -64,6 +77,8 @@ ARG GRAYLOG_UID=1100

ARG GRAYLOG_GROUP=graylog
ARG GRAYLOG_GID=1100
+COPY --from=graylog-downloader /opt/tini /tini
good call-out 👌; I thought about this end ended up intentionally making
/tini the location in the final layer because that's the location in the
Tini docs https://github.com/krallin/tini#using-tini and also the one I
see most often in the wild.
I use /opt/tini in the downloader layer to conform with the existing
convention of using /opt/ for downloaded executables.
I'm 100% open to any location and happy to update to accommodate --just
took my best shot in the dark to remain idiomatic 😄
good feedback!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Graylog2/graylog-docker/pull/88?email_source=notifications&email_token=AAJCXQI7FRGIXSFMU7GB2BDQEGDC7A5CNFSM4IKYCO52YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCBI4REI#discussion_r313004060,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJCXQMQAYDHEOVRESQH45LQEGDC7ANCNFSM4IKYCO5Q
.

definitely let me know if there are any changes you'd like made --I'm happy to do so. Thanks!

@mehrenreich @mariussturm @jalogisch Can one of you take a look at this? Thank you! :smiley:

thanks for taking a look, @jalogisch!
as it turns out, tini is available directly via apt-get :sweat_smile:. this is breaking news to me and very exciting.
I just pushed up a refactor to install from there instead of using the brittle and error prone curl + copy strategy originally proposed.
@jalogisch, I think this addresses all of your outstanding concerns but one: whether or not tini is safe to pull in. IMO, to the extent that it's safe to blindly download and run any software from a stranger on the internet, we should be fine with this one as it's already bundled with docker itself. happy to discuss more though if you feel it appropriate!
Please let me know if there's anything else you'd like addressed!

also, for posterity:
if you'd like to build / run locally, you can lift the example docker-compose.yml from the docs then build + tag the local image as graylog/graylog:3.1 before spinning up the stack:
$ make &>/dev/null \
    && docker tag graylog graylog/graylog:3.1

$ docker-compose up

Do you have that already running in your environment with that change?

I verified this change by standing up a local stack and stopping the containers using various techniques; however, I don't have a durable environment where graylog is constantly running.
I do, however, have countless apps spanning many programming languages and base images running in production environments via tini (as does much of the docker-running world).
If there's anything I can do to help assist with the QA process, please don't hesitate to ask!
Thanks! 😄

thank you @knksmith57


[liblemon] build failure

Host Environment

OS: MacOS Catalina
Compiler: revision (as provided by XCode 11.3.1)

To Reproduce
Steps to reproduce the behavior:
brew install doxygen
./vcpkg install liblemon
Failure logs
There should be no empty directories in /REDACTED/vcpkg/packages/liblemon_x64-osx
The following empty directories were found:
/REDACTED/vcpkg/packages/liblemon_x64-osx/share/doc/lemon/html

If a directory should be populated but is not, this might indicate an error in the portfile.
If the directories are not needed and their creation cannot be disabled, use something like this in the portfile to remove them:
file(REMOVE_RECURSE "${CURRENT_PACKAGES_DIR}/a/dir" "${CURRENT_PACKAGES_DIR}/some/other/dir")

Found 1 error(s). Please correct the portfile:
Additional context
Can work around the problem by uninstalling doxygen:
brew uninstall doxygen
Or by applying this diff:
diff --git a/ports/liblemon/portfile.cmake b/ports/liblemon/portfile.cmake
index 960b09987..76000d9e9 100644
--- a/ports/liblemon/portfile.cmake
+++ b/ports/liblemon/portfile.cmake
@@ -40,6 +40,7 @@ vcpkg_copy_tool_dependencies(${CURRENT_PACKAGES_DIR}/tools/liblemon)
 file(REMOVE_RECURSE ${CURRENT_PACKAGES_DIR}/debug/include)
 file(REMOVE_RECURSE ${CURRENT_PACKAGES_DIR}/debug/share)
 file(REMOVE_RECURSE ${CURRENT_PACKAGES_DIR}/bin ${CURRENT_PACKAGES_DIR}/debug/bin)
+file(REMOVE_RECURSE ${CURRENT_PACKAGES_DIR}/share/doc)

 # Handle copyright
 file(INSTALL ${SOURCE_PATH}/LICENSE DESTINATION ${CURRENT_PACKAGES_DIR}/share/liblemon RENAME copyright)

The current master branch commit reference I'm using at the moment is 1be75a28bfce2084bb0b59ec92a6853f13d7be66.

Hi @spykermj
Thanks for posting this issue.
This issue should be fixed now. Please try to update vcpkg and build it again.


Missing License

Hi,
there is no licensing information in the repository, meaning that this is not useable freely as default copyright laws apply.
Could you please add a license to the project ?
Thanks

Supporting this.
I would like to use this project as well for my project, but I cannot since it lacks a license.  I hope I don't have to reimplement this project.
May I suggest a short and permissive license such as the MIT license?

@albesze "bribe" not necessary, but very much appreciated 😆 Thanks.


Follow the google HTML style guide in the style guide's HTML

The style guide's HTML file does not itself follow it's own rules
From the guide

#Omit optional tags (optional).
For file size optimization and scannability purposes, consider omitting optional tags. The HTML5 specification defines what tags can be omitted.
(This approach may require a grace period to be established as a wider guideline as it’s significantly different from what web developers are typically taught. For consistency and simplicity reasons it’s best served omitting all optional tags, not just a selection.)

   <!DOCTYPE html>

       <title>Spending money, spending bytes</title>

Sic.

   <!DOCTYPE html>
   <title>Saving money, saving bytes</title>

Qed.

then compare to the style guide's own HTML
<!DOCTYPE html>

<meta charset="utf-8">
<title>Google HTML/CSS Style Guide</title>
<link rel="stylesheet" href="javaguide.css">

<link rel="shortcut icon" href="https://www.google.com/favicon.ico">

Google HTML/CSS Style Guide

1 Background

This ...

...

It seems like according to the guide it should be
<!DOCTYPE html>
<meta charset="utf-8">
<title>Google HTML/CSS Style Guide</title>
<link rel="stylesheet" href="javaguide.css">
<link rel="shortcut icon" href="https://www.google.com/favicon.ico">

Google HTML/CSS Style Guide

1 Background

This ...
...

and the script should be either moved to the end or tagged with defer so no need for

Hi Greggman. Apologies for the delay in responding to this!
I agree that all of our HTML-formatted style guides should ideally follow the recommendations provided by our HTML/CSS Style Guide. We have a process for converting our internal style guides into these external ones, so there’s little point updating the HTML files in this repo directly.
I’ll file an internal issue for improving these templates. Thanks for the feedback!

I should probably add that the HTML/CSS Style Guide recommendation to omit optional tags is an optional recommendation, but I think there are other ways in which our style guide markup doesn’t follow our style guide.

i think the big problem with most of our HTML files is that they're generated, so i don't know that styleguide matters as much with those


align API between ML-KEM and Kyber

Fixes #31 by renaming enc and dec to encaps and decaps for Kyber and swapping the output order for Kyber.encaps()

wouldn't it have been better the other way round... retain the Kyber API and align the ML-KEM api to it?
this will break existing users of the library

Going forward, almost no one will use Kyber in favour of the NIST designed ML-KEM. The academic paper was what I used for this repository but when the specs are finalised the FIPS document will be the source of truth.
For current users of this library, the only intended use was to learn about Kyber which can be done easily regardless of API changes.

my point is that for this to be a library it needs to be actually usable by other projects, to be usable the API needs to be stable.
changing method names is breaking API for no benefit what so ever

At some point this can be considered a library to use ML-KEM and Kyber (I believe this is what you want to achieve) but I am only concerned with having a fixed API once the repository is advertised as a library with a fixed API.
While we are refactoring, improving and adding various features I consider everything here something which can be modified to benefit the code in the long term.
FIPS 203 itself is still a draft and will be different once finalised (indeed this repo already diverges from FIPS 203 due to known typos). So until NIST have fixed everything I feel no obligation for an educational python script to be fixed.


fix inline attachments and embedded images

My fix for MessageMask->getHTMLBodyWithEmbeddedBase64Images() - now will working correctly :)
Tested on gmail and outlook.
Additionally I added:

Part->content_type = content type of part
improved id part search from headers
fix bug (non-object on MessageMask::getHtmlBody())

Hi @dwalczyk ,
thanks a lot for your pr :+1:

This change broke message bodies completely. Getting a completely empty bodies array for almost all of the messages in my folder. How did this change even make it to production??

Hi @hadesflames ,
thanks for your feedback. If you are experiencing any problems please feel free to create a new issue.
Please keep in mind that no one is getting paid for their contribution - we are all just doing it for each other. I'm grateful for every pull request. Please take a look at the license and read it carefully (note the limitations). Perhaps your expectations towards open source software are a bit off?
Best regards,