Add metrics in the Driver

Working on the issue CSHARP-687
Implement PoC of the metrics in the driver.
Current public API:
// If you really want, you can implement IDriverMetricsProvider by yourself
Builder.WithMetrics(IDriverMetricsProvider provider)

And additional extension for netstandard2.0 target:
Builder.WithMetrics(App.Metrics.IMetricsBuilder builder)
// which you can use like following:
clusterBuilder.WithAppMetrics(new MetricsBuilder()
                           .Configuration.Configure(options => options.AddEnvTag("Company.Project")
                                                                      .AddAppTag()
                                                                      .AddServerTag())
                           .SampleWith.ForwardDecaying()
                           .Report.ToGraphiteForDriver(graphiteUri))

For simplier usage to the end user there were implemented two extensions for App.Metrics:

CassandraDriverGraphitePointTextWriter - simple writer for graphite paths that disable default App.Metrics reporter behaviour that adds metric-type to the beginning of the path
CassandraDriverAppMetricsReportingExtensions - extensions that nicely glue custom PointTextWriter in the App.Metrics

This PR contains an implementation of the main subset of metrics from the Java driver but also inject some additional metrics that can be useful in some cases.
Global metrics scope can be divided into three main zones: cluster-zone, host-zone, and session-zone.
So, all currently implemented metrics listed here:
Cluster level metrics
clusters.${ClusterName}.connected-sessions: Counter
clusters.${ClusterName}.alive-hosts: Gauge
clusters.${ClusterName}.connected-hosts: Gauge

Node level metrics
Currently ${NodeAddress} has format $"{ipAddress}_port" and maybe we should convert it to readable dns names if we can
${RequestType} represents one of the request types from cql-messaging protocol specification
nodes.${NodeAddress}.pool.in-flight: Gauge
nodes.${NodeAddress}.pool.max-requests-per-connection: Gauge
nodes.${NodeAddress}.pool.open-connections: Gauge
nodes.${NodeAddress}.pool.available-streams: Gauge

nodes.${NodeAddress}.retries.total: Meter
nodes.${NodeAddress}.retries.read-timeout: Meter
nodes.${NodeAddress}.retries.write-timeout: Meter
nodes.${NodeAddress}.retries.unavailable: Meter
nodes.${NodeAddress}.retries.other: Meter

nodes.${NodeAddress}.ignores.total: Meter
nodes.${NodeAddress}.ignores.read-timeout: Meter
nodes.${NodeAddress}.ignores.write-timeout: Meter
nodes.${NodeAddress}.ignores.unavailable: Meter
nodes.${NodeAddress}.ignores.other: Meter

nodes.${NodeAddress}.errors.request.total: Meter
nodes.${NodeAddress}.errors.request.read-timeout: Meter
nodes.${NodeAddress}.errors.request.write-timeout: Meter
nodes.${NodeAddress}.errors.request.unavailable: Meter
nodes.${NodeAddress}.errors.request.other: Meter

nodes.${NodeAddress}.errors.connection.init: Meter
nodes.${NodeAddress}.errors.connection.auth: Meter

nodes.${NodeAddress}.bytes-received: Counter
nodes.${NodeAddress}.bytes-sent: Counter

nodes.${NodeAddress}.speculative-executions: Meter

nodes.${NodeAddress}.messages: Timer
nodes.${NodeAddress}.cql-messages: Timer
nodes.${NodeAddress}.types.${RequestType}: Timer

Session level metrics
Here, ${RequestType} is the CQL statement type (i.e. INSERT, DELETE, SELECT, etc.). Also, I try to correctly handle request binding and separate requests for this case too.
Currently, I simply assign Session object hash to the ${SessionId}, but it leads to very uncomfortable paths to use. Maybe we should assign more meaningful session-id or maybe just drop connected-nodes metrics...
sessions.${SessionId}.connected-nodes: Gauge
sessions.bytes-sent: Counter
sessions.bytes-received: Counter

# [${KeyspaceName}.${TableName}.] is optional and exists only for queries
# for which we know this kind of information
sessions.requests.[${KeyspaceName}.${TableName}.]${RequestType}.cql-request: Timer
sessions.requests.[${KeyspaceName}.${TableName}.]${RequestType}.cql-client-timeouts: Meter
sessions.requests.bound.[${KeyspaceName}.${TableName}.]${RequestType}.cql-request: Timer
sessions.requests.bound.[${KeyspaceName}.${TableName}.]${RequestType}.cql-client-timeouts: Meter

I still think that information about keyspace, table name, and request type can be very useful for analyzing Cassandra server/client performance. It is crucial to hierarchically subdivide requests by these parameters because in a large application there is usually a lot of fast requests that hides problems with other requests.
Implementation details

This implementation of the feature contains separate abstraction for different metrics types for easy testing and easier possible future transition to other metrics library.
Current implementation of these abstractions uses App.Metrics library and exposed only in netstandard2.0 target
All metrics stored in the Registry classes, that have the following structure:

IMetricsRegistry -> IHostLevelMetricsRegistry
                 -> ISessionLevelMetricsRegistry -> IStatementLevelMetricsRegistry
                 -> IClusterLevelMetricsRegistry

This implementation tries to create all necessary metrics node at the construction of main objects (Session, Table, Cluster, etc) to avoid wasteful frequent memory allocation and unnecessary concurrent access to the metrics. There is only one issue (as far as I remember) in the function SessionLevelMetricsRegistry.GetStatementMetrics(IStatement statement) that creates table-specific metrics for every request on the fly. We can simply refactor this method and store the necessary metrics nodes in the Table object.

I understand that there is still a lot of work to do, but I want to hear some guidance and critique from the community about current work and suggestions about further actions to improve it.
TODOs

[ ] Refactor metric path names
[ ] Unit-tests
[ ] Check integration tests with enabled metrics (safety check to avoid NullReferenceException in production code)
[ ] Performance benchmarks
[ ] Flexible API to disable/enable some kinds of metrics

Hi, I will start reviewing this PR this week. Sorry for the delay!

Hi, I will start reviewing this PR this week. Sorry for the delay!

Great! I'll try to quickly react to your comments and questions.

Currently ${NodeAddress} has format $"{ipAddress}_port" and maybe we should convert it to readable dns names if we can

I think we should not convert it to dns names because some complications might come up because of that and the only time the driver resolves DNS names is during startup (in case there are contact points that are hostnames instead of ip addresses). After that, the driver will discover the addresses for the remaining hosts by querying system.local and system.peers tables so we should just use the addresses that the driver are using to connect to these hosts.

Global metrics scope can be divided into three main zones: cluster-zone, host-zone, and session-zone.
So, all currently implemented metrics listed here:

Currently the C# driver has Cluster and Session but the goal is to merge these two into a single class (which will probably be called Session as well but it might get another name like Client). This will be done on the next major version and I think that we can define the metrics names with that in mind.
So what I propose is to have just Cluster level metrics that would contain the Session level metrics of all sessions for the same Cluster instance. We really don't expect users to have more than one Session per Cluster instance and it is usually considered an anti pattern (which is why we are merging them in the next major).
The Node level metrics would be a "sub" level below Cluster level. So it would look like this:
// node level
$"{ClusterName}.nodes.{NodeEndpoint}.{MetricName}"

// cluster level
$"{ClusterName}.{MetricName}"

Examples:
// node level
"c1.nodes.<IP_ADDRESS>:9042.pool.open-connections"

// cluster level
"c1.bytes-sent"

One big plus of this is that it's the same structure the java driver 4 is using and it would make it so much easier for users to reuse the same dashboards (or at least make it easier to create a dashboard for the C# application from an existing dashboard for a java application).
Another big plus is that we can maintain the exact same name structure in the next major, we just have to replace {ClusterName} with {SessionName} or {ClientName} whatever it may be.
The metrics that you've implemented seem to be on point, so big kudos there! Here is a config file that lists all of the metrics that java driver collects in case you haven't seen it yet (you probably have though).
There's some that don't make sense in the C# driver currently like the ones related to throttling. Also, some settings on the file mentioned above that are related to cql-requests and latency might not make sense to implement since we haven't implemented any kind of latency tracking / request tracking component in the C# driver yet. I'm talking about the highest-latency setting under cql-requests for example.
Overall I'm very excited for this feature and it's looking great already 😄

After this first pass review I can tell that many hours were put into this, thanks for this awesome contribution.

Thank you for the review and for the time that you put into investigating my code :-)
I will fix issues you pointed within 1-2 days.

Regarding metrics per request type, I still don't agree with those.

Let's discuss the question about metrics granularity in more details.
Should we inject request type, keyspace/table name in the metric path or not?
Current situation
At the moment metrics feature support implemented in the Java, NodeJs, C++ and Python DataStax client libraries. All these implementations have roughly the same set of reported metrics (for example, here is metrics exposed by Python client). And across all clients, there is only one metric that relates to the request latency: request_timer - latency of all requests performed by the driver.
Problems
I think that high-level metrics about requests (latency, rate, etc) is core part of the feature, but when it aggregated over all requests there are many cases when we can't rely on the metric value or can't interpret it properly. I have the following arguments, that supports this statement:

We need to separate metrics for different request types because the server handles different requests in  different ways. Cassandra is highly optimized for write requests and if we mixup request metrics for SELECT and INSERT queries then metric value only indirectly related to the actual read (or write) request performance.
We also need to separate metrics for different tables, because very often application manipulates with many tables during its lifetime, and very often there is a bunch of fast and lightweight requests that don't impact application performance very much. The problem is that this continuous request flow can greatly influence the statistics of the overall requests and, as a consequence, it can greatly complicate the process of determining the source of performance degradation.
If application operates with different tables there is a problem as in argument 1, because different tables have different workloads and query performance. Mixing metrics values (latency values, rates) for such tables leaves us with statistics that are hard to interpret.
Also, I think that separation of metrics by request type keyspace/table name can be helpful not only in case of request latency or rate, but also there can be added some internal metrics related to requests and current work on the decomposition can simplify the implementation of future features in the Driver.

Proposed solutions
You have mentioned solutions that can help in the investigation of performance degradation in some cases:

Cassandra server metrics

This solution can't give us information about the source of requests to the given table. Usually, we can see the degradation of read latency for some table, but there is no information about what application is struggling from this problem at the moment.

Slow query logger

I think this option can be applied for deep and long investigation of problems. But in many cases, we need a quick way to react and precisely locate the source of the problem and in this case, metrics are more suitable for the task.

Example
I can imagine the following example where server-side logs can't help investigate a problem. Suppose that we store events about different business objects in the single Cassandra table. This table has skewed read pattern - fresh events requested more often than old events. Also, this storage lays at the core of our product and there are many services that access events from this storage. Of course, these services also access many other tables that are also stored in the Cassandra.
Imagine that one day you see unexpectedly high read latency for this table at server-side. This may mean that one of your services actively tries to read old events and here client-side metrics greatly simplify our task because we can investigate metrics from all our services trying to find degradation of read requests for the single table. And it is necessary to have a decomposition of metrics because in another case we can't find the source of the problem (problematic service that reads old events) because our aggregated latency metric will be polluted with many other noisy requests to other tables and we were not able to see performance of requests to our table with events.
Finally
We are extensively using Cassandra C# driver client and very excited to contribute to your project. Because of that, we are so annoying about these implementation detail of the feature - we think it will be very useful to have separated metrics in the Driver.
Also, we implement Thrift client for Cassandra with metrics support and in our argumentation about metrics feature we sometimes rely on the experience with Thrift client. And during its usage, we monitor metrics, separated by a table name and request types and sometimes it helps us to understand the cause of degradation of our application.
I understand that this design decision is unique across all driver implementations, because of that I think we can implement some switch, that enables metrics separation feature only for paranoid users. If this feature is too big we can try to split it, but this must be done very carefully because basic feature design must be ready to injection of different query-specific parameters in the metric path.
What do you think about it?

Thanks @Umqra for submitting the pull request to the driver.
Given that is a patch that adds a core functionality and we will be maintaining this functionality going forward, it's important that we follow these guidelines:

The new API surface should be minimal, we should add the new public interfaces/methods only when needed.
The functionality should be inline with the other drivers.
We should complement and not overlap functionality that is already in the server.
It should not impact overall performance of the driver.

About gathering metrics by table and type of queries, Apache Cassandra already supports that: http://cassandra.apache.org/doc/latest/operating/metrics.html
For example:

ReadLatency: Local read latency for this table.
RangeLatency: Local range scan latency for this table.
WriteLatency: Local write latency for this table.
CoordinatorReadLatency: Coordinator read latency for this table.
CoordinatorWriteLatency: Coordinator write latency for this table.
CoordinatorScanLatency: Coordinator range scan latency for this table.

It's useful to have client metrics per cluster and host, that way a user can troubleshoot comparing server vs client metrics.
Note that we can still provide extension points to make it easier for an user to implement their own metrics, so in this case, it could be a default implementation that publishes a set of metrics (common to other drivers) and an interface that a user can optionally implement to gather their own metrics.

+1 to what @jorgebay said, let me just add my 2 cents.

Cassandra server metrics
This solution can't give us information about the source of requests to the given table. Usually, we can see the degradation of read latency for some table, but there is no information about what application is struggling from this problem at the moment

In this situation you would most likely have your infrastructure monitoring system showing which application instances are having issues.

Slow query logger
I think this option can be applied for deep and long investigation of problems. But in many cases, we need a quick way to react and precisely locate the source of the problem and in this case, metrics are more suitable for the task.

I disagree. A request tracker like the one in the Java driver will allow you to quickly find out which queries are slow because the query is logged. It really feels like a request tracker like this would be the perfect fit for the problem you are describing.

I can understand your concerns, but I'm not ready to give up at this point so I will try to suggest alternative implementation approaches that can simplify the extension of metrics paths on the application side:

We can build a single source of universal events that contains different useful information that driver determine about a request (there can be simple data like table name, keyspace, request type and also there can be information about driver internals related to this request). With this event source, the application can simply subscribe to it and send metrics manually. This approach also simplifies the implementation of Slow Query Logger because it can simply consume universal events and log it appropriately if it needed.
We can provide an API that allows users to define some custom sub-context for the request that will be propagated in the driver.

Actually, the first option is very attractive to me and if you like this idea we can discuss it in more details and I can try to implement it.

We can build a single source of universal events that contains different useful information that driver determine about a request (there can be simple data like table name, keyspace, request type and also there can be information about driver internals related to this request). With this event source, the application can simply subscribe to it and send metrics manually. This approach also simplifies the implementation of Slow Query Logger because it can simply consume universal events and log it appropriately if it needed.

Java's RequestTracker is appropriate for this use case. It can be described as an "event handler" provided by the application that will be invoked by the driver everytime a request has been executed. For your use case you could implement a RequestTracker that would publish custom metrics. But again, it would also be the responsibility of the application to parse the query string in order to extract the keyspace and table information in case it's needed because there are lots of cases where the driver does not have or need that information.
Also, the RequestTracker receives the execution profile with which the request was executed so the application could use different execution profiles for different groups of requests and the request tracker would be able to know which group of request it is. This wasn't the main purpose of execution profiles obviously but it's there and it might be useful.

We can provide an API that allows users to define some custom sub-context for the request that will be propagated in the driver.

I think I understand what you're proposing here but it sounds like a major overhaul of the API for all the components (Session, LINQ and Mapper). Furthermore it would be something that makes the C# driver farther from the other drivers which goes against our goal of consistency.

ReuestTracker looks good, I'll try to inspect its implementation in Java driver and adapt to this PR. Actually, it will be even cooler if the Driver will provide some request specific values which it can understand easily (for example - the driver can easily deduce type and table name of the requests for LinqToCql queries)
But you stand strictly against this extension of RequestTracker?

Thanks for the follow up changes, I'll come back to review the changes later this week or early next week at worst.

ReuestTracker looks good, I'll try to inspect its implementation in Java driver and adapt to this PR.

I would definitely avoid increasing the scope of this PR even further. If you are interested in implementing the RequestTracker that is awesome but it would be best if it was a separate PR where we can separate the discussions and the review process. I understand that in your application's use case you will probably use RequestTracker for additional metrics but from the driver's point of view it's a completely different feature / component.
If you need the App Metrics integration for the Request Tracker implementation and you want to develop both features at the same time then I guess it's fine to add that to this PR...

Actually, it will be even cooler if the Driver will provide some request specific values which it can understand easily (for example - the driver can easily deduce type and table name of the requests for LinqToCql queries)
But you stand strictly against this extension of RequestTracker?

Hmm this is a tricky one... This wouldn't require API changes to existing methods so we should be fine on that front. The only concern is the inconsistent nature of those properties since not all methods of ´LINQandMapper` will actually know those properties. If we wanted to make it consistent then we would have to do lots of API changes so that users can pass those parameters individually...
I'm trying to think about a way to support something like that but I'm out of ideas. Maybe we should move this discussion (related to RequestTracker) to the ticket because this is important enough that it shouldn't get buried under the Metrics discussion. If you decide to implement Request Tracker here then we can still discuss the code level details in this PR, but I'd love to have an "overview" of how it's going to look like on the ticket description / comments.
Again, thank you very much for the contribution and I'll review the changes in a couple of days :+1:

Just a quick update: some things came up so I'll only be able to look at the last changes later this week.

@joao-r-reis, that's okay. I'm also can't continue fixing this PR at the moment, but I hope that soon there will be some time to fix all issues in the code. Also, I want to inspect RequestTracker in the Java Driver and propose some basic ideas of its implementation for C# in the ticket that you mentioned earlier.

Oh that's fine, I'm glad to know that you are thinking of contributing to the RequestTracker feature. Take your time, there's no rush to get this out as we only planned on tackling this later this year anyway.
Maybe it's a bit more efficient timewise to hold off on taking another look at this PR until you had some time to fix it up?

Yes, if we are no hurry then the second pass of your review can wait until I finish fixing issues. Also, I need to remove current support of specific metric prefixes (table name, request type) and understand the way we can inject this information in the RequestTracker. When I'm done with this I surely notice you but if I will be stuck with this until the end of the next week - please, ping me about my progress :-)

Hi, I might be picking up this soon, did you have any progress on this? If I understood correctly, you would be more interested in the Request Tracker than the Metrics feature by itself so it might make sense for me to pick up the development of Metrics and you can focus on the Request Tracker after Metrics are done.

Hi. I'm interested in both features. I pause work on the feature for a moment, but now I think I'm ready to return to the development. This PR lasts for a long time so I think we need to concentrate our efforts to move this feature in the production because protracted development is a bad thing.
At the moment I working on the removal of support of table-specific metrics to simplify code and remove unnecessary abstractions. I think this can take 1 or 2 days. After that I notify you and we can discuss next steps in the development of the feature (maybe something from my initial todo list like unit tests, integration tests, benchmarks etc).

Sounds good to me. Just a heads up, I will be unavailable next week and the following week so I won't be able to do code reviews or participate in the discussion during that time.

After that I notify you and we can discuss next steps in the development of the feature (maybe something from my initial todo list like unit tests, integration tests, benchmarks etc).

I'll make sure I'm available to code review and discuss this as soon as you notify me. I do think that the next step should be unit and integration testing.

Okay, now I'm done with removing burden of the previous implementation of the feature which includes support of table level metrics different kind. I decided to refactor commits in the branch, because there was over 100 commits.
Also, I introduce new abstraction which now called Observers in the code (maybe this was inspired a little bit by this blog post). This abstraction take care of all monitoring facilities in case of different events that happened in the driver. Monitoring might includes logging, sending metrics, request tracking etc. So, this abstraction must help us to develop RequestTracker feature too. In addition, now code for the metrics registries is dead simple and servers as a clear documentation of available metrics in the Driver. Also, I think that Observers can simplify testing and mocking.
Current Observers hierarchy looks like following:
          IClusterObserver
            |          |
            v          v
IRequestObserver     IHostObserver
                            |
                            v
                     IConnectionObserver
                            |
                            v
                     IOperationObserver

But current implementation of observers has a problem: top level Observers carry corresponding MetricsRegistry (HostLevel or ClusterLevel), but sometimes we can't instantiate MetricsRegistry, because not all necessary information are available at the moment. For example, if we initialize ClusterObserver in the constructor of the Cluster class, than we can't instantiate ClusterLevelMetricsRegistry, because ClusterName determined after initializing control connection. This problem leads to really entagled code and I ask for advice in this situation from you.
Current sample dashboard in Grafana with metrics:

I like the Observer approach, 👍
Regarding the initialization issue, I think that we can't have a single solution for all of those issues. We'll have to go on a case by case basis. For the metrics that depend on the Cluster / ControlConnection initialization that one is easy, we should only be initializing the metrics after cluster initialization (which includes the control connection). I think that the way you have it implemented right now will initialize Metrics after cluster initialization so we're good there. For other similar dependencies I think it's best to discuss them when they come up so if you have ran into some of those please list them here and we can discuss them (or post a comment on the relevant lines of code to open a discussion thread).
It's looking good, I can't review everything in detail (due to what I said in my last message, i.e., I'm going to be mostly unavailable until the 26th of August) but in my opinion the next immediate steps should be:
1 - Merge/Rebase this branch with latest changes from master. From the top of my head I know that at least there are some changes in ControlConnection and Cluster initialization that might require some changes to the ClusterObserver initialization, not sure.
2 - Deal with the todo's, I noticed there are still a decent amount of those in the code and if there's something I can help you with those then you can post a comment on the respective line (in this PR) to open a discussion thread and we'll discuss how to move forward on each one of them.
When I get back, I'm thinking of creating a branch so that I can also push some commits / open PRs and contribute to this feature since this is one of the most request features to have on the driver and we have a high interest in getting this released. I'm still not sure how I'm going to do it (where to create the branch, etc.) but I'll deal with it then.

Hey! I'm back and as I said previously my intention is to start contributing to this feature in order to get it out as soon as possible. I will create (this week) a dev branch on this repo with the current state of this PR and any future contributions can be made by opening a PR against the new dev branch.

That's great.
Unfortunately, I couldn't work on the PR during your absence. I hope that at the end of this week I will able to return back to the feature and fix all my todos in the code. I can also rebase this branch on the fresh master, in case if you didn't start working in this direction already.

Yeah that would be great, I haven't picked this up yet so I'm ok with delaying my work on this until next week in order to give you some time to fix whatever you can before I start.

Landed in dev branch metrics_master_ft <4b098f876be3aac32dd65f98edcbe5976ba0a1d5>


not launch

Hello,
I got this error when  launching ...

Thanks

what's your ng version? Does it occur in new ng project?

Try use it in new v14 project. If it still occur error, feedback to me.


Add modules for coordinate system.

[ ] CoordinateSystem module
[ ] Cartesian module
[ ] Screen module

Modules and functions for coordinate system should be in a different solution, so a new library, Fermata.CoordinateSystem has been created.
Close this issue.


make parse_expr_with_alias public

Some very small change, as the title said.
close #1439

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.351%

Totals

Change from base Build<PHONE_NUMBER>0:
0.0%

Covered Lines:
29687

Relevant Lines:
33225

💛  - Coveralls


NIO's Resolver interface pretty much forces thread hops for external Resolvers

https://github.com/apple/swift-nio/blob/5768317b704288e172b2dc927e7927f4c16843be/Sources/NIOPosix/Resolver.swift#L32
NIO's resolver interface doesn't tell the resolver what EventLoop the returned future should ideally be coming from.
That means that a normal resolver will likely return futures of a random EL which is bad for performance and unnecessary.
NIO's own GAIResolver kinda cheats and gets constructed inside ClientBootstrap -- after it has selected the EL for the new Channel. That possibility doesn't exist for outside resolvers because they need to be initialised when added to ClientBootstrap.resolver(instanceHere).
yes, you can manually pick an EL, pass that to the ClientBootstrap as the group and also pass it to your resolver but that's brittle, error-prone, reimplementing NIO functionality and probably also precludes a multi-shot resolver from caching.

@Lukasa should we invent a new label like needs-major that is something like easy-in-major-challenging-in-minor for issues (like this one) that are possible (but ugly/hard) to be fixed in minors (adding new interfaces that overlap with old ones) but would be incredibly easy in a minor.
Then once NIO3 comes around they can just be done.

If you know the EventLoopGroup you can use .any() to get the "current" EventLoop and avoid thread hops without passing the EventLoop to any of the Resolver methods explicitly because they are called while being on the EventLoop.

If you know the EventLoopGroup you can use .any() to get the "current" EventLoop and avoid thread hops without passing the EventLoop to any of the Resolver methods explicitly because they are called while being on the EventLoop.

Sure, but that's not "guaranteed" to work and just unnecessarily ugly :).


Eliminate SHAMapInnerNode lock contention:

The SHAMapInnerNode class had a global mutex to protect the array of node children. Profiling the old std::mutex-based mutex solution, out of 360,000,000 lock operations, 11,349,853 blocked because the mutex was already locked: that is over 4% of the time.
This commit removes that global mutex, and replaces it with a new per-node 16-way spinlock. Profiling with the new per-node 16-way spinlock suggests that the contention was effectively eliminated. Out of 360,000,000 lock operations, only 1,057 failed to acquire the lock on the first 'spin', with 50% of those managing to acquire their requested spinlock on the next spin and in no instance requiring more than 3 spins.
While this patch is unlikely to directly and measurably increase performance, it is a meaningful improvement and does so with no increase in memory usage or additional CPU.
Kudos to @RichardAH who suggested profiling locks and collected the original data that led down this path.

Needs minor testing! Closing for now.


Is it allowed to request assignment for an issue that has not been taken action for a while?

Idea Description
If it is allowed, Is it okay if it's about two weeks after the inactivity?
I apologize if it is inappropriate to ask a question here.
Reference images and links

Thanks for your question.
I believe we have no choice but to assign others to make our conference app great when it seems that they are not working on it. Therefore, I think the answer is 'yes.'


 We need the stable 2.8 branch
Also re-fix nginx config for Airflow (don't know why we lost this changes)

He,  please rebase / sync your branch with the upstream master. There's no need for this PR to contain this many commits.

Also, can you point me to the specific failure in master branch - couldn't find it.

He, please rebase / sync your branch with the upstream master. There's no need for this PR to contain this many commits.

Done @akariv


Storage target updates

Hey @BenFradet, could you please cherry-pick these to https://github.com/snowplow/snowplow/pull/3438?

@chuwy on it :+1:

done, closing


aws distributed training struck !

i use the command
python  /home/ec2-user/incubator-mxnet/tools/launch.py -n 1 -s 1  -H hosts --sync-dst-dir /home/ec2-user/incubator-mxnet/ "source activate mxnet_p36 && python image_classification.py --model vgg11 --gpus 0 --kvstore dist_sync"
and it stuck ,the info is this

018-12-14 02:33:36,036 INFO rsync /home/ec2-user/incubator-mxnet/example/gluon/ -> <IP_ADDRESS>:/home/ec2-user/incubator-mxnet/
2018-12-14 02:33:36,036 INFO rsync /home/ec2-user/incubator-mxnet/example/gluon/ -> <IP_ADDRESS>:/home/ec2-user/incubator-mxnet/
[02:33:37] src/van.cc:290: Bind to role=scheduler, id=1, ip=<IP_ADDRESS>, port=9093, is_recovery=0
INFO:root:Starting new image-classification task:, Namespace(batch_norm=False, batch_size=32, builtin_profiler=0, data_dir='', dataset='cifar10', dtype='float32', epochs=120, gpus='0', kvstore='dist_sync', log_interval=50, lr=0.1, lr_factor=0.1, lr_steps='30,60,90', mode=None, model='vgg11', momentum=0.9, num_workers=4, prefix='', profile=False, resume='', save_frequency=10, seed=123, start_epoch=0, use_pretrained=False, use_thumbnail=False, wd=0.0001)
[02:33:37] src/van.cc:398: ? => 1. Meta: request=0, timestamp=0, control={ cmd=ADD_NODE, node={ role=server, ip=<IP_ADDRESS>, port=46899, is_recovery=0 } }

but if i delete the arg  --kvstore dist_sync ,just run
python  /home/ec2-user/incubator-mxnet/tools/launch.py -n 1 -s 1  -H hosts --sync-dst-dir /home/ec2-user/incubator-mxnet/ "source activate mxnet_p36 && python image_classification.py --model vgg11 --gpus 0 "
it run successfully
and the two instance are on the same Security groups
and rules
All TCP | TCP | 0 - 65535
SSH | TCP | 22 | <IP_ADDRESS>/0

@mxnet-label-bot add [question]

@Davdi, did you manage to find the issue? To me it seems that it should be connected to connectivity, because when you remove --kvstore dist_sync the default value is used, which for this tutorial is device.
I also notice that you set n = 1, but mention "2 instances". Which configuration do you try to achieve? And what is the content of your hosts file?

@mxnet-label-bot add[Distributed]

@mxnet-label-bot add[distributed]

@mxnet-label-bot add [distributed]


Impossible to "fix" the return_immediately deprecation

We are using PubSub heavily in our project, which means lots of long-lived consumers. Since streaming pulls are not a thing in PHP, we resort to an infinite while loop:
# simplified version of the code
while(true) {
  $this->processMessages(
    $subscription->pull()
  );
  sleep(1);
}

Because the return_immediately option has been deprecated some time ago, this produces a deprecation log on every pull request:
[2022-06-17T06:57:33.459907+00:00] deprecation.INFO: User Deprecated: return_immediately is deprecated. {"exception":"[object] (ErrorException(code: 0): User Deprecated: return_immediately is deprecated. at /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php:138)"} []
[2022-06-17T06:57:33.460273+00:00] deprecation.INFO: User Deprecated: return_immediately is deprecated. {"exception":"[object] (ErrorException(code: 0): User Deprecated: return_immediately is deprecated. at /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php:118)"} []
[2022-06-17T06:57:33.460353+00:00] deprecation.INFO: User Deprecated: return_immediately is deprecated. {"exception":"[object] (ErrorException(code: 0): User Deprecated: return_immediately is deprecated. at /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php:118)"} []

I thought that with #4734 this would be solved, but unfortunately the deprecation is triggered in PullRequest::getReturnImmediately(), which does always get called by the internal protobuf Message class in some places.
At first I thought to submit a workaround for this, so that the deprecation is only shown when the option is set to true (false is the default):
    public function getReturnImmediately()
    {
        if ($this->return_immediately === true) {
            @trigger_error('return_immediately is deprecated.', E_USER_DEPRECATED);
        }
        return $this->return_immediately;
    }

The setter should still always trigger the deprecation, so you still see it when you use the option and set it to false.
However, then I noticed that this code is generated. So I don't think this is possible, unless there's a way to customize generated classes like this?
In any case, I would very much like to be able to remove this deprecation from our processes, as it's polluting our logs, making them harder to read. Is there another way to go here?

Thank you for reporting this! I understand why this would be frustrating. I see two potential paths forward here:

Track down what part of the stack calls getReturnImmediately(), and see if we can remove the call.
Submit a PR to protocolbuffers/protobuf to remove the deprecated errors on the getters.

Simply editing the PullRequest.php file will not work, as these are autogenerated, and any change that isn't to the generator itself (e.g. the protobuf repository) would get overwritten.
Between the two options above, 1 is preferable.

Hi @pkruithof
I'm trying to replicated your issue with the following basic push and pull script, but I'm not getting any deprecation message as my code is never reaching the if block in the following script:
$pubsub = new PubSubClient();
$topic = $pubsub->topic('my-topic-name');
$subs = $pubsub->subscriber('my-subs-name');
$topic->publish($message = ['data' => 'Test Message']);
$messages = $subs->pull();
$subs->acknowledge($message[0]);

Can you check if there's a particular scenario in which you're getting the deprecation notice? Like are you getting the notice for a basic script as above too?

Closing due to inactivity. Please reopen if the issue still persists.

Hi @yash30201, the deprecation is suppressed by the @ operator. You can see it when you enable the scream option in xdebug, or by adding this in your script for example:
set_error_handler(function ($errno, $errstr, $errfile, $errline) {
    throw new ErrorException($errstr, $errno, 1, $errfile, $errline);
});

result:
PHP Fatal error:  Uncaught ErrorException: return_immediately is deprecated. in /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php:167
Stack trace:
#0 [internal function]: {closure}(16384, 'return_immediat...', '/app/vendor/goo...', 167)
#1 /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php(167): trigger_error('return_immediat...', 16384)
#2 /app/vendor/google/protobuf/src/Google/Protobuf/Internal/Message.php(1598): Google\Cloud\PubSub\V1\PullRequest->getReturnImmediately()
#3 /app/vendor/google/protobuf/src/Google/Protobuf/Internal/Message.php(1842): Google\Protobuf\Internal\Message->existField(Object(Google\Protobuf\Internal\FieldDescriptor))
#4 /app/vendor/google/protobuf/src/Google/Protobuf/Internal/Message.php(1931): Google\Protobuf\Internal\Message->fieldByteSize(Object(Google\Protobuf\Internal\FieldDescriptor))
#5 /app/vendor/google/protobuf/src/Google/Protobuf/Internal/Message.php(1564): Google\Protobuf\Internal\Message->byteSize()
#6 /app/vendor/grpc/grpc/src/lib/AbstractCall.php(117): Google\Protobuf\Internal\Message->serializeToString()
#7 /app/vendor/grpc/grpc/src/lib/UnaryCall.php(39): Grpc\AbstractCall->_serializeMessage(Object(Google\Cloud\PubSub\V1\PullRequest))
#8 /app/vendor/grpc/grpc/src/lib/BaseStub.php(295): Grpc\UnaryCall->start(Object(Google\Cloud\PubSub\V1\PullRequest), Array, Array)
#9 /app/vendor/grpc/grpc/src/lib/BaseStub.php(545): Grpc\BaseStub->Grpc\{closure}('/google.pubsub....', Object(Google\Cloud\PubSub\V1\PullRequest), Array, Array, Array)
#10 /app/vendor/google/gax/src/Transport/GrpcTransport.php(218): Grpc\BaseStub->_simpleRequest('/google.pubsub....', Object(Google\Cloud\PubSub\V1\PullRequest), Array, Array, Array)
#11 /app/vendor/google/gax/src/GapicClientTrait.php(751): Google\ApiCore\Transport\GrpcTransport->startUnaryCall(Object(Google\ApiCore\Call), Array)
#12 /app/vendor/google/gax/src/Middleware/CredentialsWrapperMiddleware.php(59): Google\Cloud\PubSub\V1\Gapic\SubscriberGapicClient->Google\ApiCore\{closure}(Object(Google\ApiCore\Call), Array)
#13 /app/vendor/google/gax/src/Middleware/FixedHeaderMiddleware.php(68): Google\ApiCore\Middleware\CredentialsWrapperMiddleware->__invoke(Object(Google\ApiCore\Call), Array)
#14 /app/vendor/google/gax/src/Middleware/RetryMiddleware.php(85): Google\ApiCore\Middleware\FixedHeaderMiddleware->__invoke(Object(Google\ApiCore\Call), Array)
#15 /app/vendor/google/gax/src/Middleware/OptionsFilterMiddleware.php(62): Google\ApiCore\Middleware\RetryMiddleware->__invoke(Object(Google\ApiCore\Call), Array)
#16 /app/vendor/google/gax/src/GapicClientTrait.php(725): Google\ApiCore\Middleware\OptionsFilterMiddleware->__invoke(Object(Google\ApiCore\Call), Array)
#17 /app/vendor/google/cloud-pubsub/src/V1/Gapic/SubscriberGapicClient.php(1264): Google\Cloud\PubSub\V1\Gapic\SubscriberGapicClient->startCall('Pull', 'Google\\Cloud\\Pu...', Array, Object(Google\Cloud\PubSub\V1\PullRequest))
#18 [internal function]: Google\Cloud\PubSub\V1\Gapic\SubscriberGapicClient->pull('projects/dev-pk...', 1000, Array)
#19 /app/vendor/google/cloud-core/src/ExponentialBackoff.php(97): call_user_func_array(Array, Array)
#20 /app/vendor/google/cloud-core/src/GrpcRequestWrapper.php(148): Google\Cloud\Core\ExponentialBackoff->execute(Array, Array)
#21 /app/vendor/google/cloud-core/src/GrpcTrait.php(82): Google\Cloud\Core\GrpcRequestWrapper->send(Array, Array, Array)
#22 /app/vendor/google/cloud-pubsub/src/Connection/Grpc.php(411): Google\Cloud\PubSub\Connection\Grpc->send(Array, Array)
#23 /app/vendor/google/cloud-pubsub/src/Subscription.php(688): Google\Cloud\PubSub\Connection\Grpc->pull(Array)
#24 /app/bin/pubsub.php(18): Google\Cloud\PubSub\Subscription->pull()
#25 {main}
  thrown in /app/vendor/google/cloud-pubsub/src/V1/PullRequest.php on line 167

@pkruithof as mentioned, this is an issue in protobuf, and should be handled over there. I filed this issue to hopefully track it: https://github.com/protocolbuffers/protobuf/issues/13428
Have you tried using the protobuf extension? It's possible that this problem won't exist there. Try installing the extension with pecl install protobuf.


Failed to NewDB. Since SchemaVersion is incompatible, delete Database and fetch again

see https://github.com/vulsio/vulsctl/issues/29

This issue was closed because it has been inactive.


Infinite Scroll feature

Hi, it would be a great feature for mobile to allow for infinite scroll. This table works great for mobile already except for the user ease of use in pagination, this is more irritating on mobile than it is on desktop and infinite scroll + this perfect material look would be a killer combo.

You are not forced to use pagination if you do not want to. I'm not completely sure what you are requesting.  Where more content is fetched from the server as the user scrolls down?

Yes that is exactly what I am requesting! For example, initially load 10 items, when the user scrolls and gets near the bottom of the list, start loading 10 more items and append to the list, then continue to do so until all items have been scrolled through. I haven't thought much into it, but I guess really this could be done by adding a function trigger for "ScrolledNearBottom" like the other triggers and then in the controller function just append the next set of results.

This is a pretty good idea, I think I could probably check if the bottom of the table is visible in the viewport and fire off some sort of event.

The table already works pretty well using md-virtual-repeat-container, there are just some issues because it expects ng-repeat instead of md-virtual-repeat. I'm gonna try to make it work and send a PR. BTW great work!
<md-data-table-container flex layout="column">
  <md-virtual-repeat-container flex>

        {{item.name}}

  </md-virtual-repeat-container>
</md-data-table-container>

@daniel81bcn, that would be a great PR!  Virtual repeat is ideal for my use case.

I did the integration but I should do some cleanup and make disabled rows work again before sending the PR.
For a demo: http://daniel81bcn.github.io/md-data-table/
The changes: https://github.com/daniel81bcn/md-data-table/commit/7a04df866b99057f1a22ef333751b70ff6e94949

@daniel81bcn I'm using your branch right now and it's working fine.  The scrollbar behaves strangely once you reach the end of the rows, but that might be a quirk of the md-virtual-repeat feature in general.

@andrea-vega on the demo you can see that I used 2 different tables, one with just the head and the other one with the actual body and infinite scroll, because otherwise the md-virtual-repeat-container doesn't calculate well the height and it does that weird behave reaching the end.
As the md-virtual-repeat needs to add a wrapper and the table doesn't let us add tags in the middle of thead, tbody I think we have to look for some other solution -_-

@daniel81bcn your solution gets most of the way there.  I know it's not an easy feat. @daniel-nagy, have you looked into this at all? :)

Definitely would like to use the infinite scroll feature if @daniel-nagy gets around to looking at this.

There is nothing stoping you from using md-virtual-repeat in version 0.9.x.

    {{item.id}}
    {{item.name}}
    {{item.date}}

and if you add the following to your style sheet
table.md-table {
  position: relative;
}

tbody.md-body.md-virtual-repeat-container {
  display: table-row-group;

  .md-virtual-repeat-scroller {
    top: initial;
    bottom: initial;
  }
}

It will look okay but I'm not sure if the md-virtual-repeat directive is actually working correctly or not. I've never used before.

I don't think Virtual repeat can cover http calls. That's really what I'm after. :S

@mlenser you can use async calls with mdVirtualRepeat, look at the controller for the Infinite Scroll example on https://material.angularjs.org/latest/demo/virtualRepeat
// For demo purposes, we simulate loading more items with a timed
// promise. In real code, this function would likely contain an
// $http request.
if (this.toLoad_ < index) {
  this.toLoad_ += 20;
  $timeout(angular.noop, 300).then(angular.bind(this, function() {
    this.numLoaded_ = this.toLoad_;
  }));
}

@daniel-nagy the problem is that the md-virtual-repeat-container directive adds a couple of div's after tbody which creates a non-valid HTML

I have an algorithm for a fixed table header. I know the people over at #184 want it. I don't want to make it a part of this module but I might offer it as a separate directive and then build off it to implement an infinite scroll feature.

Has anyone gotten md-virtual-repeat to work with an md-data-table in a md-dialog?

Has anyone gotten md-virtual-repeat to work with an md-data-table, period?

@stephengardner I did, using md-on-demand and fixe-head. Take a look:
Html:
<md-virtual-repeat-container class="virtual-repeater-container" md-top-index="topIndex" fix-head>
  <md-table-container>

                      // headers

                       // cells

  </md-table-container
</md-virtual-repeat-container>

@marcomafessolli I'd appreciate a codepen to show how yours is working.
I'm not sure if you were using it within the context of a  md-content and having toolbars/navbar on top of the page or not.
With md-toolbars on top of the md-content and placing your code fragment in the md-content , the layout breaks.

@marcomafessolli +1 please do CodePen

@mikila85 @b3hrooz
Working code pen md-data-table with mdVirtualRepeater
https://codepen.io/skendzicr/pen/GEjEvb


Microk8s stopped listening at <IP_ADDRESS>

I'm running the microk8s snap v1.15.5 on Ubuntu 18.04.3:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:        18.04
Codename:       bionic
$ snap list microk8s
Name      Version  Rev  Tracking  Publisher   Notes
microk8s  v1.15.5  984  1.15      canonical✓  classic

My microk8s configuration is pretty default, and I'm deploying services and ingresses from our deployment-configs repository. I don't think the details of exactly what I'm deploying matter to this issue, but I can walk you through that if it's important.
I frequently find that microk8s stops being accessible on localhost (<IP_ADDRESS>), and I'm experiencing that same issue right now. I think basically this happens after microk8s has been running for a while, and the only reliable fix I've found is to reset microk8s completely with microk8s.reset && snap disable microk8s && snap enable microk8s.
Here's the state of my microk8s cluster:
$ microk8s.kubectl get all,ingress
NAME                                          READY   STATUS    RESTARTS   AGE
pod/default-http-backend-5d5ff5d4f5-l462n     1/1     Running   10         2d2h
pod/nginx-ingress-microk8s-controller-j4hfn   1/1     Running   1          87s
pod/ubuntu-com-79997c7db9-55587               1/1     Running   8          2d2h
pod/ubuntu-com-blog-76467ff4cc-7wn28          1/1     Running   8          2d2h

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/default-http-backend   ClusterIP   <IP_ADDRESS>   <none>        80/TCP    2d2h
service/kubernetes             ClusterIP   <IP_ADDRESS>     <none>        443/TCP   2d2h
service/ubuntu-com             ClusterIP   <IP_ADDRESS>    <none>        80/TCP    2d2h
service/ubuntu-com-blog        ClusterIP   <IP_ADDRESS>   <none>        80/TCP    2d2h

NAME                                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/nginx-ingress-microk8s-controller   1         1         1       1            1           <none>          2d2h

NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/default-http-backend   1/1     1            1           2d2h
deployment.apps/ubuntu-com             1/1     1            1           2d2h
deployment.apps/ubuntu-com-blog        1/1     1            1           2d2h

NAME                                              DESIRED   CURRENT   READY   AGE
replicaset.apps/default-http-backend-5d5ff5d4f5   1         1         1       2d2h
replicaset.apps/ubuntu-com-79997c7db9             1         1         1       2d2h
replicaset.apps/ubuntu-com-blog-76467ff4cc        1         1         1       2d2h

NAME                                    HOSTS                                              ADDRESS     PORTS     AGE
ingress.extensions/staging-ubuntu-com   staging.ubuntu.com,www.staging.ubuntu.com          <IP_ADDRESS>   80, 443   2d2h
ingress.extensions/ubuntu-com           ubuntu.com,www.ubuntu.com,ubuntu.net + 7 more...   <IP_ADDRESS>   80, 443   2d2h

I've noticed that the pod/nginx-ingress-microk8s-controller-j4hfn pod keeps restarting about every minute, and when it does, the <IP_ADDRESS> IP address assigned next to the ingresses disappears for a while and then comes back.
But regardless, whether <IP_ADDRESS> is appearing next to the ingresses or not, the host never seems to be able to access the cluster on that address:
$ curl <IP_ADDRESS>
curl: (7) Failed to connect to <IP_ADDRESS> port 80: Connection refused

I'm sure there's more I could do to debug this, but I don't know what the next step is. Could you give me pointers as to where to look next?

Could you attach the tarball produced by microk8s.inspect?

$ microk8s.inspect
Inspecting services
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-apiserver is running
  Service snap.microk8s.daemon-proxy is running
  Service snap.microk8s.daemon-kubelet is running
  Service snap.microk8s.daemon-scheduler is running
  Service snap.microk8s.daemon-controller-manager is running
  Service snap.microk8s.daemon-etcd is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy openSSL information to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster

WARNING:  Docker is installed.
File "/etc/docker/daemon.json" does not exist.
You should create it and add the following lines:
{
    "insecure-registries" : ["localhost:32000"]
}
and then restart docker with: sudo systemctl restart docker
Building the report tarball
  Report tarball is at /var/snap/microk8s/984/inspection-report-20191108_091134.tar.gz

inspection-report-20191108_091134.tar.gz

Is that Docker warning relevant? Is that what I need to do?

Is that Docker warning relevant? Is that what I need to do?

The docker warning is relevant only if you are using the registry addon (microk8s.enable registry) in which case the docker client needs to trust the addon registry in order to push images there.

Ah, not relevant then.
Let me know if there's anything obvious in that tarball, or if there's a way I can help look into it?

In the logs of the API server I see a lot of error messages like this one:
Error: unable to find suitable network address.error='no default routes found in "/proc/net/route" or "/proc/net/ipv6_route"'. Try to set the AdvertiseAddress directly or provide a valid BindAddress to fix this

The default route is needed to identify the interface where kubernetes should listen to. While the APIserver starts we wait for 2 minutes for the default interface to be set (https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/wrappers/run-with-config-args#L80). Could you verify that when you see this abnormal behavior the apiserver is complaining about the default route? To see the logs do sudo journalctl -u snap.microk8s.daemon-apiserver.service.


unfold-full error if fully applied term is still a function

open nat
definition seq_diagram (A : ℕ → Type) : Type := (Πn, A n → A (succ n))
variables (A : ℕ → Type) (f : seq_diagram A)
include f

definition shift_diag [unfold-full] (k : ℕ) : seq_diagram (λn, A (k + n)) :=
λn a, f (k + n) a

example (n k : ℕ) (b : A (k + n)) : shift_diag A f k n b = sorry :=
begin
  esimp,
end

gives
invalid 'rewrite' tactic, rewrite step produced type incorrect term
function expected at
  shift_diag A f k n b

proof state:
A : ℕ → Type,
f : seq_diagram A,
n k : ℕ,
b : A (k + n)
⊢ shift_diag A f k n b = sorry

Note: I like the fact that shift_diag is considered to be "fully applied" if it has 3 arguments, and doesn't also need the two extra arguments after unfolding seq_diagram

I fixed the problem.

Note: I like the fact that shift_diag is considered to be "fully applied" if it has 3 arguments, and doesn't also need the two extra arguments after unfolding seq_diagram

This is not the semantics of [unfold-full]. In the current implementation, shift_diag will be unfolded only when it is applied to 5 arguments.

This is not the semantics of [unfold-full]. In the current implementation, shift_diag will be unfolded only when it is applied to 5 arguments.

Ok, I think it's fine either way.


LSP: Share module search paths between the TLAPS and SANY.

The TLAPS LSP server should support path sharing both ways:

TLAPS could provide its path to the vscode extension.
Also, the TLAPS has to consider paths configured in the extension.

Maybe related to:

https://github.com/tlaplus/tlaplus/issues/490
https://github.com/tlaplus/tlapm/issues/3

@lemmy, what do you think of this plan:

I build a class in the extension which maintains the paths. The TLC, SANY, and TLAPS then use the paths.

User can add path entries via configuration property. There is no such currently, so I would add one.
TLAPS would supply its stdlib path "automatically" if installed and enabled.
TLA Tools would supply its internal modules as well.
Community modules could be handled in a special way here. It could be downloadable on a user click.

Also, I add a tree view in the side pane to show these paths. Later, we can add modules provided by each path, improving the visibility of existing modules.

I have to solve this for the TLAPS integration, because it will fail if some non-TLAPS-stdlib modules are used in a spec. Also, I have to use these paths if I want to integrate the SANY parser in the TLAPS LSP server for better error messages.

Sounds good to me assuming you are proposing to let the extension be the "facilitator of paths" between the various tools.  How the tools make use of the path information is then up to the tool.  For example, TLAPS adds its path information to the extension, and TLC is given this information when it launches.
What are the precedence rules WRT a tool's default lookup path and the extension's configuration property?

What are the precedence rules WRT a tool's default lookup path and the extension's configuration property?

I think the tool should prioritize its internal library (e.g., the TLC prioritizes the modules contained in tlatools.jar).
Other paths are passed as additional search paths. Hopefully, that will not be a problem when some TLAPS modules are moved to the community modules repo.

This might require tool changes as, IIRC, a library path that is passed to TLC takes precedence over TLC's internal library path.


cant seem to connect to use the docker instance.

Hi,
I followed the docker-compose.yml steps.
I am puzzled how would i connect to the docker hauk server?
I dont intend to use https, so i am just using the docker-compose.yml and the config.php configuration.
But which url i need to access?
Thanks.

The Docker container listens on port 80. If you don't intend to use a reverse proxy, you have to add a ports directive to the compose file for this to be reachable from outside, e.g.
services:
  hauk:
    image: bilde2910/hauk
    container_name: hauk
    volumes:
      - ./config/hauk:/etc/hauk
    ports:
      - 8912:80/tcp

.. for it to be reachable on port 8912, for example. (-p 8912:80/tcp for Docker CLI.) I do strongly recommend HTTPS, however - setting up Let's Encrypt in a reverse proxy is not too difficult, and HTTPS both protects against MITM attacks and eavesdropping, and is required for some functionality like in-browser geolocation on the map, and end-to-end encryption functionality (in some browsers).
When you have added the ports directive in the compose file, you can connect to the Hauk server at http://<docker host IP>:<chosen port>/.

Hi! Can you confirm if this solved the issue?

Assuming resolved, closing

@bilde2910 Sorry for the delayed response.
Thanks for your inputs.
I followed your suggestions, and now i can get the default hawk page on browser.
But when i input same URL in the hawk app, i get failed to connect error:

Assuming the server is on the same network, you typo'd the address (198.168 instead of 192.168).


Example embedded

I have added one example for now, and a set of CMakefile that I have used over the years to compile mlpack and integrate it into any application. I have never seen any problem with these and they always worked as they should.
The CMakeLists.txt is a simplified version of mlpack original CMakeList.txt and removed everything that is not relevant for a standing application.
PS: it might look like one directory is a duplicate of the other, but this is intended, the first one is a fully working example with a random forest, and the second one is just a set of CMake files that can be copied or adapted, and they are standalone.

I will need to push a couple of things to this PR and then we can merge it


Implement buffer copying

See https://www.opengl.org/registry/specs/ARB/copy_buffer.txt

@kvark I think me and @Androcas will simply add the following methods:
command::Buffer::copy_buffer(src: R::Buffer, dst: R::Buffer,
                             src_offset_bytes: usize, dst_offset_bytes: usize,
                             size_bytes: usize);

Encoder::copy_buffer(src: handle::Buffer, dst: handle::Buffer,
                     src_offset: usize, dst_offset: usize,
                     size: usize) -> Result<(), CopyError>;

Using glCopyBufferSubData for OpenGL, vkCmdCopyBuffer for Vulkan (we won't handle synchronization) and DeviceContext::CopySubresourceRegion for D3D11.

@Bastacyclop sounds good!


I wonder where the links to this functionality would live? I guess there could be new buttons on the homepage https://acronymy.net
On the backend, adding this kind of thing would be easier if we moved to a different database. See #7.

Random undefined words would be great. Thinking of new words can be tricky, since readily-guessable words are often taken.

completed in 30f936671ae1e36d464065161239ee0c15ee5a28


Compilig speedup

I we change the extensions from cpp to hpp of source files (except blecker.cpp), then we will achieve some compiling speedup because the compiler will not generate object files for these sources.

Good idea, I did not know it is working. Will try, thx


Multiple mirror servers without geolocation in Fedora Mirror Manager

The following mirrors are listed in the Fedora Mirror Manager mirror server list but are missing from the mirror server location list:

cofractal-sea.mm.fcix.net
mirror.fcix.net

The following mirrors are also missing from the mirror server list:

coresite-atl.mm.fcix.net

The lack of coresite-atl is expected; that server isn't even deployed yet.


linux: Gracefully catch device errors

Right now, the phy-linux panics when its device is not present/disappears/has other errors.  Instead, we should find a way for the PHY implementation to gracefully report what has happened so the application can decide how to deal with the situation.
Most likely, the best solution is to define a PHY error type which bubbles up through the FdlMaster's poll() call.

A related question is how to deal with device disconnects.  For most real-world applications this is not relevant as a hardware RS485 won't ever disappear.  But, for example, a USB device cannot be assumed stable.  Should the PHY implementation auto-reconnect?  Should this be the job of the application?


Error messages from Bro

 0.000000        Reporter::ERROR /opt/bro/share/bro/site/otx/otx.dat/Input::READER_ASCII: could not read first line   (empty)

Seems to be caused by writing the file in place. Should be an easy fix to write it under a temporary name and then move it to the proper name.

Tested and working in my environment.


Fix new DLPack protocol error messages and tests

Follow-up: #5306

Formatted string were broken :smile:
Fix wrong tests. I noticed that stream consistency cannot be tested unless directly calling __dlpack__ because the current stream in consumer (cupy.from_dlpack) and producer (cupy.ndarray__dlpack__) are always the same if both ends are CuPy.
Test failure detected in https://github.com/chainer/chainer-test/pull/673#issuecomment-874877502

pfnCI, test this please.

Jenkins CI test (for commit bc44b16e21605931ba6d3b6d9fe66a4d5a68adf0, target branch master) succeeded!


Version 1.1.0. Added UMD wrapper and unit tests.

Updated to add a UMD wrapper to make the Guac common JS easier to use in CommonJS, AMD, and Browser environments.
Wrote a simple test.
I'm not sure about the version...I suggested a bump to 1.1 since this is a somewhat big change (not breaking but definitely adds more than just bug fixes).
However, perhaps it would be better to align this version number to the actual Guacamole source it's based on (so 0.9.9)? Now sure how bad that would screw things up for others currently using this...or if NPM even allows/likes that.

DON'T MERGE THIS IN YET...I'm trying to get it work in a browser and having some issues...should've done more Integration testing.  I'll post a fix soon...standby. Thanks.

As for versioning, I've had problems with this before. NPM won't allow you to update packages with the same version, you need to always go up. My first version was 0.9.8 followed by some bugfixes, so already 0.9.8 was out of date.
If it is still usable as a global objects after these changes 1.1 is totally fine with me. The Guacamole version this is representing will be noted in the readme, so I hope there won't be much confusion.

@padarom Ah ok...then no worries with the versioning.  The README should suffice.  Let me test it more Today.  I was trying to use browserify with CommonJS format but my "solution" doesn't really work in the browser as of yet since things like "fs" and "eval" are not supported in browser.  The easiest solution would be to just modify the actual guacamole source but I'm trying very hard not to do that so it will be easy to update this project as new versions of guacamole are released.

Did you manage to test it or is it still not working properly?

As there hasn't been any activity in 5 months I'm closing this PR. If you happen to have a working solution I'll gladly take another PR or reopen this one. Issue #2 stays open though, so in case anyone else wants to contribute or I get the time to do it we can still work on it.


primitives: Reduce alloc requirements

Recently we reduced the alloc requirements in units but we did not propagate these changes up to primitives.
Remove a bunch of alloc feature gating from primitives.

hmm I must have botched the API text file stuff, this got through CI but https://github.com/rust-bitcoin/rust-bitcoin/actions/runs/12265170371/job/34220646659?pr=3715 one has all the stuff that was added here.

Oh, its just because CI ran before #3682 was merged.


Implement transaction compiler

The task of a transaction complier would be to facilitate compilation of transaction programs. As a part of this task, the complier would also help with compiling account modules and note scripts. I'm thinking we could even put this component into a separate crate (still under this repo) - maybe something like tx-compiler or tx-executor.
Here is my initial draft of what a transaction compiler could look like:
pub struct TransactionComplier {
    assembler: Assembler,
    account_procedures: BTreeMap<AccountId, Vec<Digest>>,
}

impl TransactionComplier {
    /// Compiles the provided module into [AccountCode] and associates the resulting procedures
    /// with the specified account ID.
    pub fn load_account(
        &mut self,
        account_id: AccountId,
        account_code: &ModuleAst,
    ) -> Result<AccountCode, TransactionError> {
        todo!()
    }

    /// Compiles the provided program into the [NoteScript] and checks (to the extent possible)
    /// if a note could be executed against an account with the specified interface.
    pub fn compile_note_script(
        &mut self,
        note_script: &ProgramAst,
        target_account_proc: Option<Vec<Digest>>,
    ) -> Result<NoteScript, TransactionError> {
        todo!()
    }

    /// Compiles a transaction which executes the provided notes against the specified account.
    ///
    /// The account is assumed to have been previously loaded into this compiler.
    pub fn compile_transaction(
        &mut self,
        account_id: AccountId,
        notes: Vec<Note>,
        tx_script: Option<ProgramAst>,
    ) -> Result<CompiledTransaction, TransactionError> {
        todo!()
    }
}

/// Compiles the provided program into the [NoteScript] and checks (to the extent possible)
/// if a note could be executed against an account with the specified interface.

Hey @bobbinth could you elaborate on how you're thinking about checking note compilation checks.

I was thinking we could do a kind of static analysis checks mentioned in https://github.com/0xPolygonMiden/miden-vm/pull/895#issuecomment-1555763543.
For example, if the note script is something like this:
begin
  <some code here>
  if.true
    call.0x123456789
  else
    call.0x987654321
  end
end

And as target_account_procs we'd pass in vec![0x123456789] it should compile. But if we pass in vec![0x111111111] as the target_account_procs it should return an error.
We'd check similar things for a note script which looks like this:
use.miden::abi::basic_wallet

begin
  call.basic_wallet::recieve_assets
end

In fact, after a program has been compiled into MAST, checking this should be identical to checking the previous example.
In the future, we can also add support for something like this:
begin
  <some code here>
  if.true
    call.0x123456789@0x123
  else
    call.0x987654321@0x456
  end
end

One thing I'm thinking is that maybe the interface I described above my not cover everything that we want. Specifically, we might want to pass interfaces of two or more different accounts to check if the note can be compiled against all of them. So, maybe we change the signature to something like:
pub fn compile_note_script(
    &mut self,
    note_script: &ProgramAst,
    targets: Vec<AccountInterface>,
) -> Result<NoteScript, TransactionError> {
    todo!()
}

Where AccountInterface is something like this:
pub struct AccountInterface(Vec<RpoDigest>);

An in terms of how: once the program is reduced into MAST, I was thinking we could traverse the MAST in DFS order to build up all calls which would have to happen in each possible execution path. For example, for a script like this:
begin
  <some code here>
  if.true
    call.0x123456789
    <some code here>
    if.true
      call.0x111111111
    else
      call.0x222222222
    end
  else
    call.0x987654321
  end
end

There are 3 possible execution paths each resulting in the following calls:

0x123456789, 0x111111111.
0x123456789, 0x222222222.
0x987654321.

So, this note can be compiled against an account which exposes any of these. But, for example, we wouldn't be able to compile it against an account which has interface: 0x123456789, 0x333333333.

Why is the interface defined such that the client provides target_account_proc: Option<Vec<Digest>> as an argument to compile_note_script instead of providing an optional account id (Option<AccountId>) from which the function sources the Option<Vec<Digest>> from internal state (account_procedures.get(account_id)).

/// Compiles the provided program into the [NoteScript] and checks (to the extent possible)
/// if a note could be executed against an account with the specified interface.
pub fn compile_note_script(
    &mut self,
    note_script: &ProgramAst,
    target_account_proc: Option<Vec<Digest>>,
) -> Result<NoteScript, TransactionError> {
    todo!()
}

or

/// Compiles the provided program into the [NoteScript] and checks (to the extent possible)
/// if a note could be executed against an account with the specified interface.
pub fn compile_note_script(
    &mut self,
    note_script: &ProgramAst,
    target_account_proc: Option<AccountId>,
) -> Result<NoteScript, TransactionError> {
    todo!()
}

@frisitano brought up a good point that specifying targets as account IDs may not always work as sometimes we may know the set of interface methods for an account, but won't know its ID. In such situations we would not be able to use load_account_interface (or would have to use it with some dummy account IDs - but this would be quite hacky and probably fragile).
One approach to address this is to have a NoteTarget enum which could look like so:
pub enum NoteTarget {
    AccountId(AccountId),
    Procedures(Vec<RpoDigest>),
}

Then, compile_note_script method could look like so:
pub fn compile_note_script(
    &mut self,
    note_script: &ProgramAst,
    targets: Vec<NoteTarget>,
) -> Result<NoteScript, TransactionError> {
    todo!()
}

We'd still keep the load_account_interface method as described in the previous comment. To make handling of NoteTarget simpler, we could add a helper method which could look like so:
fn get_target_interface(&self, target: NoteTarget) -> Result<Vec<RpoDigest>, TransactionError> {
    match target {
        NoteTarget::AccountId(id) => todo!("lookup account interface"),
        NoteTarget::Procedures(procs) => Ok(procs),
    }
}

Some thoughts on how compile_transaction could work.
First, when TransactionCompiler is created, we should compile prologue, epilogue, and note setup scripts. MAST roots of these should be stored in member variables of TransactionCompiler.  In the future, when we have something like a pcall instruction, we'd need to store only the root of the entire transaction kernel.
When compile_transaction is called, the first thing we need to do is to make sure that the account with the specified ID had been previously loaded into the compiler (loading accounts would be a responsibility of some higher-level component). If the account hadn't been loaded, we'd return an error.
Then, we'd create an AssemblyContext and compile notes one-by-one using this context using (via the compile_in_context() method) and keeping the returned CodeBlock's for each note. If tx_script was provided, we'd need to compile it in the same context as well.
At the end, we need to convert this context into a CallBlockTable. For this, we could add a public method to the assembler (though, there are other options to do this as well). This method could look something like this:
pub fn build_cb_table(&self, context: AssemblyContext) -> Result<CodeBlockTable, AssemblyError> {
    todo!()
}

To the returned code block table, we'd need to add code blocks of all compiled notes and the code block of the tx_script if one was provided.
An assumption I'm making here is that AssemblyContext maintains a unified callset across different invocations of compile_in_context. If it doesn't, we might need to make a few more changes in the assembler.
So, at this point, we should have MAST roots for all notes (and tx_script) and a call block table containing block for all calls made from the notes. The next step is to build a transaction program.
Currently, we could do it by manually building a MAST which would look like so:

To build this MAST, we need to know only the roots of executed note scripts. This MAST would be represented by a CodeBlock. Note that in the future, once we have the pcall instruction, this step would not be necessary.
Then, we'd instantiate the Program struct using its with_kernel constructor passing in:

The CodeBlock we built in the previous step as root,
The CallBlockTable we built in the step before as cb_table.
Transaction kernel as the kernel. This is also probably something we should have as a member variable for the TransactionCompiler.

Then, we finally return this program.
Putting all of the above together, the TransactionCompiler struct would probably have the following fields:
pub struct TransactionComplier {
    assembler: Assembler,
    account_procedures: HashMap<AccountId, Vec<Digest>>,
    prologue: CodeBlock,
    epilogue: CodeBlock,
    note_setup: CodeBlock,
    tx_kernel: Kernel,
}

closed by #147


Arabic localization

Added arabic support, should work automatically, but if you want to fake the website then add this to your local storage:
localStorage.setItem("i18nextLng", "ar")

Need translation for the line of how far are you from the location
-I have tried messing with this and changing the translation but it didn't work out, would be cool if you can take a look and help figure this out :)

Also, as of now the automatic language detection is not working as planned, I am not sure what I did wrong there so maybe you can take a look at that too

@DanLiebeschutz Do you want to take a look?
Another point which related to localization but not this PR:
Location details are language specific and can't be translated. Possible solution is to filter locations that are written in the same language (add this input to the location form). What do you guys think?

Location titles you mean cant be translated? Or the content?
שם אחראי מוקד
או השם עצמו?

i think there's an issue here


e2e,compute: Quarantine very flaky usb redir test

What this PR does
This test is causing a lot of impact and leading to PRs getting stuck in the merge queue. It more than qualifies for quarantining[1]
[1] https://search.ci.kubevirt.io/?search=with+usbredir+support+Should+work+several+times&maxAge=48h&context=1&type=bug%2Bissue%2Bjunit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
The following alternatives were considered:
Links to places where the discussion took place:
Special notes for your reviewer
/cc @dhiller @xpivarc @victortoso
This is blocking up the merge queue a bit so it would be good to get this in as soon as possible.

Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

NONE

Related PR: https://github.com/kubevirt/kubevirt/pull/12977

Related PR: #12977

Ops, different test

Should probably apply to the other two tests too for now.

Should probably apply to the other two tests too for now.

I haven't seen them failing that much but if you think it is best, I can add it to the other two?

Should probably apply to the other two tests too for now.

Applied it to the three tests - https://github.com/kubevirt/kubevirt/compare/000f353e1c0574136527360f457d42466ebce7b8..7a28a3390dec09b15815be22be395161ea728dc9

I haven't seen them failing that much but if you think it is best, I can add it to the other two?

Yes. One of them I actually removed with https://github.com/kubevirt/kubevirt/pull/12977/commits/222b245a2aa18674cec8db20594bf770673bb33c
I just triggered the CI with a quick possible fix... let's see:
https://github.com/kubevirt/kubevirt/pull/12977/commits/b09174a0b6f2be61727bf73b9d4ac316ccad7290
Feel free to quarantine them anyway till we are sure they work as expected. Sorry for the delay on fixing this.

ee to quarantine them anyway till we are sure they work as expected. Sorry for the delay on fixing this.

Thanks @victortoso   - can you add an lgtm if you are happy?

/lgtm

/approve

/override pull-kubevirt-fossa
Not related to this PR.

/override pull-kubevirt-e2e-k8s-1.31-sig-storage
Not related to this change

/sig ci


[BUG] Tensorclass autocast float to tensor

Describe the bug
@tensorclass casts float to a torch.Tensor of type float32 which triggers batch_shape validation errors. Same problem with int.
To Reproduce
from tensordict import tensorclass
import torch

@tensorclass
class MyTensorClass:
    my_float: float
    my_tensor: torch.Tensor

if __name__ == "__main__":
    my_float = 5.0
    my_tensor = torch.tensor([1,2,3])
    # This works
    my_tensor_class = MyTensorClass(my_float, my_tensor)
    # Note: my_float gets casted to a torch.Tensor
    print(my_tensor_class)

    # This fails since batch_shape does not match
    my_tensor_class = MyTensorClass(my_float, my_tensor, batch_size=[3])

prints:
MyTensorClass(
    my_float=Tensor(shape=torch.Size([]), device=cpu, dtype=torch.float32, is_shared=False),
    my_tensor=Tensor(shape=torch.Size([3]), device=cpu, dtype=torch.int64, is_shared=False),
    batch_size=torch.Size([]),
    device=None,
    is_shared=False)

RuntimeError: batch dimension mismatch, got self.batch_size=torch.Size([3]) and value.shape=torch.Size([]).

Expected behavior
Non-tensor values should stay non-tensor values.
System info
Describe the characteristic of your environment:

TensorDict version 0.6.2 (bug exists since 0.5 I think)
Python version: 3.9

Checklist

[x] I have checked that there is no similar issue in the repo (required)
[x] I have read the documentation (required)
[x] I have provided a minimal working example to reproduce the bug (required)

try
@tensorclass(autocast=True)
class MyTensorClass:
    my_float: float
    my_tensor: torch.Tensor

or
@tensorclass(nocast=True)
class MyTensorClass:
    my_float: float
    my_tensor: torch.Tensor

on nightlies.
The first one will ensure that all the type annotations are respected (unless they're not plain types, like Optional or Union)
The second will just not cast anything: if it's not a tensor, its type will not be changed.
LMK if that helps!

Thanks a lot! That helps!
I would still change the default behavior to nocast=True. Casting as the default behavior is unexpected.


Crash when generating report with zero rows other than the header row in a PDFTable

ℹ Please fill out this template when filing an issue.
All lines beginning with an ℹ symbol instruct you with what info we expect. You can delete those lines once you've filled in the info.

What did you do?
I was generating a report that had no table rows other than the header row.
What did you expect to happen?
I expected the main header and header row of the table to get generated.
What happened instead?
I encountered a crash at the following code in PDFTableObject of the calculate method.
var rowIdx = 0
while cells[rowIdx].allSatisfy({ $0.type == .rowHeader || $0.type == .header }) && rowIdx < table.size.rows {
          headerCells += cells[rowIdx]
          rowIdx += 1
 }

TPPDF Environment
TPPDF version: 2.3.5 (master branch commit: 0dbe543c517c94d0bebacdaf1c0894a9a28c4a78)
Xcode version:  14.1
Swift version: 5.7
Demo Code / Project
I think the solution to the crash is to short circuit the while loop by checking the rowIdx first:
var rowIdx = 0
while  rowIdx < table.size.rows && cells[rowIdx].allSatisfy({ $0.type == .rowHeader || $0.type == .header }) {
          headerCells += cells[rowIdx]
          rowIdx += 1
 }

That way the cells[rowIdx] expression won't get executed and won't crash

Or perhaps instead of checking table.size.rows, check against cells.count instead.

If the issue still remains, can you please provide a small reproducible example (incl. the document generation with data that crashes).
Otherwise please close the issue :)

I haven't actually forked the repo just yet. I'm using the Swift Package Manager to include it in my project, which makes the code read-only. I'll fork it and then try out the fix and if it works I'll submit a PR.

I submitted a pull request for this #316

Any chance of taking a look at it?


MAJOR When planning you can now indicate that releases should not be created by default

Additionally there is now an option on the command to opt-in to automatically releasing.

--assume-no-releases is the new flag for not auto-releasing
--release=comma,delimited,composer,packages is the new option for opting in to auto-releasing modules

I don't really understand why this is needed

When you plan a release it automatically assumes a release of every module. This is a flag for the opposite functionality. It will assumes no releases.

Ah right, that makes sense. Maybe rename the “release” terminology to something else- it confuses me. How about “assume patch versions”

Do you still want this? I'm not overly convinced we need it TBH

Na I'll reopen if I change my mind.


PR#2355 Continued + expanded scope

Original PR by @blazeyo
Pull Request Type

[ ] ✨ feat
[x] 🐛 fix
[ ] ♻ refactor
[ ] 💄 style
[ ] 🔨 chore
[ ] 📝 docs

Relevant Issues

resolves #2317
closes #2355
resolves #928
What is in this change?
Reduces the loading time of the documents popup by fetching pinned documents once per folder.
Results on my local instance with ~14k files:

loading time

before: 105 seconds
after: 2 seconds

number of queries for pinned documents:

before: ~14k
after: 8

Before
[0.05ms] Checked/created documentsPath
[1.28ms] Checked DocumentSyncQueue.enabled()
[2.29ms] Read subfolder: custom-documents
[122.83ms] Read subfolder: folder1
[205.92ms] Read subfolder: folder2
[301.90ms] Read subfolder: folder3
[336.27ms] Read subfolder: folder4
[35961.37ms] Read subfolder: folder5
[49814.86ms] Read subfolder: folder6
[51723.69ms] Read subfolder: folder7
[105922.62ms] Finished processing all folders and files

Function call timings summary:
  readFile: 1939.09ms
  parseJSON: 694.86ms
  getPinnedWorkspaces: 100892.30ms
  getWatchedWorkspaces: 6.67ms
  checkCachedVector: 1971.95ms

After
[0.05ms] Checked/created documentsPath
[11.17ms] Checked DocumentSyncQueue.enabled()
[12.08ms] Read subfolder: custom-documents
[54.88ms] Read subfolder: folder1
[65.54ms] Read subfolder: folder2
[74.09ms] Read subfolder: folder3
[90.14ms] Read subfolder: folder4
[648.97ms] Read subfolder: folder5
[781.37ms] Read subfolder: folder6
[817.45ms] Read subfolder: folder7
[1926.20ms] Finished processing all folders and files

Function call timings summary:
  readFile: 734.33ms
  parseJSON: 398.08ms
  getPinnedWorkspaces: 328.55ms
  getWatchedWorkspaces: 1.58ms
  checkCachedVector: 288.89ms

Additional Information
An additional optional parameter select has been added to Document.where.
Developer Validations

[x] I ran yarn lint from the root of the repo & committed changes
[x] Relevant documentation has been updated
[x] I have tested my code functionality
[x] Docker build succeeds locally

@blazeyo Thanks a ton for handling this. Expanded scope to do the same functionality with reporting of watched status on documents, which would also cause issues if thousands were watched. While I'm sure we can improve the speed even further from 2sec - this is still incredible.
Again, thanks a ton.


fix(compatibility): Correct version relations for Mender 3.2 and 3.3

@mender-test-bot cherry-pick to:

3.2.x
3.3.x


remove error handling from counter factual

Signed-off-by: Ke Xu<EMAIL_ADDRESS>

I'm personally in favor of removing error handling like this. It's not very obvious to users how to retrieve error messages from failed analyses. In Python it is more common to raise an exception and let the caller handle the error than have a two step process of checking for a null result and then fetching the error for the failed computation.

I'm personally in favor of removing error handling like this. It's not very obvious to users how to retrieve error messages from failed analyses. In Python it is more common to raise an exception and let the caller handle the error than have a two step process of checking for a null result and then fetching the error for the failed computation.

Why wasn't this a concern in the PR where the error handling was added https://github.com/microsoft/responsible-ai-widgets/pull/460?


Call blocking will prevent re-render

Version
2.6.11
Reproduction link
https://jsfiddle.net/fw89qnoc/
Steps to reproduce
Let's start with simple call blocking function:
https://jsfiddle.net/yd6xr8ea/
The idea here is, that counter should update before myVal , so I thought $forceUpdate should do the trick:
https://jsfiddle.net/0jdhy41g/
But it doesn't.
So according to docs, https://vuejs.org/v2/api/#updated I've put the code in the $nextTick of the updated function.
https://jsfiddle.net/fw89qnoc/
But that still doesn't do the trick. Adding 1ms timeout works, but it feels hackish and doesn't really do justice the docs.
https://jsfiddle.net/bwd9j3sq/
What is expected?
Both values should update a re-render independedly
What is actually happening?
Unless timeout is involved, re-render is blocked, even when used recommendation from the docs.

It's not the render of Vue that is blocked but rather the browser rendering. A PR to docs are welcome but there isn't anything to do on Vue's side

@posva But shouldn't $nextTick ensure that myVal will change before call blocking code executes?


In iOS 10.0.1, window.flutter_inappWebView is undefined

[x] I have read the Get Started - Installation section
[x] I have read and done the Get Started - Setup Android section
[x] I have read and done the Get Started - Setup iOS section
[x] I have already searched for the same problem

Environment

Technology
Version

Flutter version
2.0.4

Plugin version
5.3.2

Android version

iOS version
10.0.1

Xcode version

Device information:
Description
Expected behavior:
Windows. flutter_inappWebView works well on lower versions of iOS
Current behavior:
In lower versions of iOS, window.flutter_inappWebView is undefined
Steps to reproduce

The problem was also found in iOS 11.0.3


Client Modules

In Python, Node.js, the usual suspects. Even cooler would be a node.js/Gremlin bridge that gave you the graph object.

Closed and moved to: https://discourse.cayley.io/t/client-modules/225


Allow loading configuration from a string instead of file

Please allow loading config from a string rather than file.
Use case: framework that needs to do it's own logging but doesn't want to require an nlog or app config file to be brought along with in deployment.

Looks like a duplicate of #2820. Maybe close the other?
If you load the string into a XmlDocument then it should work nicely:
https://github.com/nlog/NLog/wiki/Explicit-NLog-configuration-loading

That doesn't compile on .NET core

That doesn't compile on .NET core. I have an xml reader option and a string file name option as the two parameters...

What is the build-error? Remember I'm not sitting next to you :)

Severity	Code	Description	Project	File	Line	Suppression State
Error	CS1503	Argument 1: cannot convert from 'System.Xml.XmlElement' to 'string'
Error	CS1503	Argument 2: cannot convert from 'string' to 'NLog.LogFactory'

System.Xml.XmlDocument xmlDoc = new System.Xml.XmlDocument();
xmlDoc.LoadXml(Resources.nlog_config); // this is a string
string currentDir = System.IO.Directory.GetCurrentDirectory();
NLog.LogManager.Configuration = new NLog.Config.XmlLoggingConfiguration(xmlDoc.DocumentElement, currentDir);

Looking at the code I am only seeing XmlReader overloads. However those also ask for a filename which seems odd. I tried using the XmlReader methods and wrapping my config string with an xml text reader but that didn't work either.

Sorry my mistake. Redirected you to an internal method. Try this instead:
                System.IO.StringReader sr = new System.IO.StringReader(xmlString);
                System.Xml.XmlReader xr = System.Xml.XmlReader.Create(sr);
                NLog.LogManager.Configuration = new NLog.Config.XmlLoggingConfiguration(xr, null);

Will update the wiki

I would add the last lines needed to create a logger:
LogFactory factory = LogManager.LogFactory;
Logger logger = factory.GetCurrentClassLogger();

we will add a helper to NLog in the future

Maybe one day the XmlConfig will not be part of Nlog-Core
Sent from my Samsung device
-------- Original message --------
From: Julian Verdurmen<EMAIL_ADDRESS>Date: 02/08/2018 11:44 (GMT+01:00)
To: NLog/NLog<EMAIL_ADDRESS>Cc: Rolf Kristensen<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Subject: Re: [NLog/NLog] Allow loading configuration from a string instead of   file (#2819)
Reopened #2819https://github.com/NLog/NLog/issues/2819.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/NLog/NLog/issues/2819#event-1767228489, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AK-fnNLEY7hf01vwIXieX9_GnFbD43n9ks5uMsnsgaJpZM4VmzHs.


not working for android

ı toke this error
MainActivity.kt: (1, 9): Package name must be a '.'-separated identifier list #42519

Hi @azaderdogan have you tried the test version 2.0.4-dev.1 ?

Hi @azaderdogan, ~have you tried the test version 2.0.4-dev.1 ?~ Can you try with the new version 2.0.5 ?

Thanks. work for me.

Glad to hear it :)


BlawxBot Relevance Demonstration

I want to generate an example on the server that shows that BlawxBot is capable of determining that an attribute is relevant for one object and not relevant for another, based on the rules.  This will require fixing #247 .

This was implemented in #274 in the Wills Demo.


Fail to run dotnet run and unable to load page after dotnet run in CodeSpace

Description:
Running dotnet run failed, need to execute dotnet dev-certs https --trust first, but this is not mentioned in the README.md. The error message is as follows:

According to the error message, run dotnet dev-certs https --trust and then run dotnet run successfully. Click the link output in the log, and the page loading fails as follows:

Repro steps:

Run azd auth login.
Run cd ./src/eShopAppHost.
Run azd provision.
Run dotnet dev-certs https --trust.
Run dotnet run.

Environment:

Azd version: azd version 1.11.0 (commit 5b92e0687e1fa96dfc8292f4b900c0c58610b6a5)
OS: CodeSpace

Expected behavior:
dotnet run can be successfully executed and the page can be loaded successfully by clicking the link.
@elbruno, @PabloNunes and @hemarina for notification.

Description: Running dotnet run failed, need to execute dotnet dev-certs https --trust first, but this is not mentioned in the README.md. The error message is as follows:  According to the error message, run dotnet dev-certs https --trust and then run dotnet run successfully. Click the link output in the log, and the page loading fails as follows:
Repro steps:

Run azd auth login.
Run cd ./src/eShopAppHost.
Run azd provision.
Run dotnet dev-certs https --trust.
Run dotnet run.

Environment:

Azd version: azd version 1.11.0 (commit 5b92e0687e1fa96dfc8292f4b900c0c58610b6a5)
OS: CodeSpace

Expected behavior: dotnet run can be successfully executed and the page can be loaded successfully by clicking the link. @elbruno, @PabloNunes and @hemarina for notification.

Thanks @Yionse , working in updating the docs with this with @PabloNunes !
Best


Move react-query-devtools from devDependencies

Otherwise the production build fails.

:facepalm: thanks, closing :smile:


revert: 4.0.0

This reverts commit 4cd73b41826a5854da2d72f162194287b110d275.
It should be a minor version.

Does this work?
We should also delete the release tag.
But we already released the 4.0.0 in npm and it cannot be reverted there. What if in the future we want to bump to 4.0.0?

Unpublish npm version by https://amplitude.atlassian.net/wiki/spaces/GOV/pages/2570715611/incident-2024-07-11-bad-npm-version#Actions
Delete S3 object
Remote tag

@yuhao900914, I delete the 4.0.0 tag, the next release will based on the last version 3.8.0

@Mercy811  Thanks. Good to know. It's much better than maven.

@yuhao900914, to prevent this from happening again I'm also going to dadd a release.yml file to branch v3.x to create a publish v3.x like what we have publish 2.x in the typescript repo https://github.com/amplitude/Amplitude-TypeScript/actions/workflows/publish-v2.yml


Is there a way to disable leaves growing? [1.12.2]

I've tried to change the "growthMultiplier" to zero and I've also tried to use the Potion of Persistance and then tried both, but the leaves keep growing back? Is there a way to prevent that?

The freeze potion will turn the leaves on the applied tree into vanilla leaves.


:robot: Triggering CI on branch 'release-next' after synching to openshift/master

/test variant-x-e2e

OCF Webhook is merging this PR


local index mode for developers

to make it easier to test parser changes without beating up on okc's servers. For safety reasons, username, password, and host have all been filled with dummy values.
Simply save https://www.okcupid.com/messages to a local file (eg. /tmp/messages.html) then run
arrow_fetcher.py -i /tmp/messages -f output.txt

and you'll be able to test the index parser.

Good idea! Maybe one day we could add a separate script that downloaded both the index HTML and all of the message HTML for use when developing.

Sounds like a good idea. I'll think on that.


Add support for v2 endpoint

Version bumped to 0.1.71

@haleybash-optimizely @delikat @aliabbasrizvi @jophde

Remember to update CHANGELOG. Otherwise LGTM.

lgtm now that CHANGELOG has been updated


special-case parsing of "require" in unparseNodeArgs(); closes #4035

this adds a special case when unparseNodeArgs() detects require; it removes the =, which is apparently required for everything else but prohibited in Node.js v8.

if this passes, I'm probably going to cut a release asap since I'm blocking on it

Coverage remained the same at 92.692% when pulling a2fb6d84cdcc20d6ada8d78ef557e5f42a9bf0db on issue/4035 into ec17f6315e0817bfb8e37279d31affc4ec108623 on master.