[NaN check] Add NaN check to support bfloat16.

@ys950902, thanks for helping with this. The problem is a bit more involved and there was a previous attempt that was abandoned. Can you please take a look at https://github.com/microsoft/DeepSpeed/pull/5252
Do you think you can incorporate the learnings into your PR?

@ys950902, thanks for helping with this. The problem is a bit more involved and there was a previous attempt that was abandoned. Can you please take a look at #5252
Do you think you can incorporate the learnings into your PR?

Sorry for later response, I am doing some other work these days, my understanding is nan check is not for float16 but also bfloat16, and won't add some extra log info on deepspeed side, the uses can only use API was_step_applied() to check whether update successfully.

@ys950902 This check only makes the training continues without error but the grad_norm and loss are not descending any more. Would it be better to turn this into an error or provide a way to solve the overflow problem by code/by warning?


Train on custom 2D pose dataset

Hi and thanks for great work !
I would like to finetune the model on some custom 2D pose dataset. From what I read you trained on 3D and 2D pose datasets so that should be possible, right ?
Best
Alexander

Yes, of course. You can refer to this issue to create your own dataset.

Thanks for your answer. That particular issue is about retraining on the model output. My question is more related to how the data in the img_info dictionary is generated and what is the format ?
I have right now 2D pose data in a csv file with the format nose,182,416,img001.jpg,768,800 etc.
How do I go from that data to the img_info dict ?
Any help is very much appreciated.
Best,
Alexander

Crowdpose is a perfect example for your need. It just loads the 2D pose data. You need to replace this line to the 2D pose of your data. Please be sure to set the correct keypoint ordering at this line. To achieve this, you can define a dict in constants.py like this. Then you can replace constants.Crowdpose_14 to the keypoint order dict you defined.
self.kp2d_mapper = constants.joint_mapping(constants.KEYPOINT_ORDER,constants.SMPL_ALL_54)


How to use SD Upscale via API

How to use SD Upscale via API ?
Which config should I pass to the API?

In the next version probably we could support upscaler API.

Understood, it would be great, I'm looking forward to it. I wish you a successful development of the project!


Add expected pass model

A physical model that predicts the likelyhood of a successfull pass given the locations and velocities of all players, the initial ball velocity, and the ball moving angle.

I have started working on the implementation of the model and am currently encountering two major pain points:

Normalized coordinates (wrt attacking direction) are not by default included in the processed data even though they must be used somewhere such as in xG and xT - is there a standardized way to obtain them that I'm missing? At the moment, I would do databallpy.features.add_team_possession to get the possession info and use that to calculate the normalized coordinates myself.
The tabular tracking data format I obtain via match.tracking_data does not make sense to me - currently a row corresponds to an entire frame of tracking data rather than a object-position pair. But this means that I don't have and can't add any meta data about the players (e.g. to identify which team a player belongs to) and also can't join player identities with the event data (e.g. to exclude the passer from potential receivers). Is there a built-in way to get a different table format and to get the missing mapping information between tracking and event data?

Hi @jonas-bischofberger, thanks for your message and great to see that you started!

right now there is not a build in way to normalize coordiantes wrt attacking direction. The playing direction for home team is always from left to right, and for the away team from right to left. I will open an issue to create a build in way to get normalized coordinates wrt attacking direction.

For now, there are two scenarios: if you need only the tracking and event data at the moment of the pass, use the team_id column in the event data to find out whether it is the match.home_team_id or the match.away_team_id. If it is the away team id, you have multiply all _x, _vx (and _ax) columns by -1 in the tracking data, and the start_x, start_y (and end_x, end_y) in the event data. If you need it normalized for all frames, not only the ones where events happen, the approach you use right now is the only solution.

This was a design choice at the beginning of the package. All the metadata about the players can be found in match.home_players and match.away_players. You can use match.player_id_to_column_id() to match player ids to the column id in the tracking data (which is f"{team_side}_{jersey_number}). Also check out the match.home_players_column_ids() or match.away_players_column_ids() to get a list of column ids for an entire team.

Lastly, check out the match.passes_df or the match.pass_events for more info. For instance, match.pass_events is a dict with PassEvents with attributes like team_side, start_x. The PassEvents should work generally, but is still in beta use so some bugs might be in there. On top of that, I have limited access to metrica data so there might be some weir edge cases.
If you have any ideas/updates on how to make the package more intuitive and easier to use, please let me know so I can make some changes to the package and make it easier for anyone to use.

@jonas-bischofberger
For point 2, would you prefer to have a double indexed (frame number, player) pd.DataFrame? Or what would you propose? Adding different types of representations of the data could be considered for future versions.

For the time being, I'm using this function to do the conversion. I didn't test it thoroughly but it seems to do the job for my current purposes!
def per_object_frameify_tracking_data(
    df_tracking, frame_col, x_cols, y_cols, vx_cols, vy_cols, players, player_to_team, new_x_col="x", new_y_col="y",
    new_vx_col="vx", new_vy_col="vy", new_player_col="player_id", new_team_col="team_id", v_cols=None, new_v_col="v",
):
    """ Converts tracking data with '1 row per frame' into '1 row per frame + player' format """
    dfs_player = []
    for player_nr, player in enumerate(players):
        coordinate_cols = [x_cols[player_nr], y_cols[player_nr], vx_cols[player_nr], vy_cols[player_nr]]
        coordinate_mapping = {x_cols[player_nr]: new_x_col, y_cols[player_nr]: new_y_col, vx_cols[player_nr]: new_vx_col, vy_cols[player_nr]: new_vy_col}
        if v_cols is not None:
            coordinate_cols.append(v_cols[player_nr])
            coordinate_mapping[v_cols[player_nr]] = new_v_col
        df_player = df_tracking[[frame_col] + coordinate_cols]
        df_player = df_player.rename(columns=coordinate_mapping)
        df_player[new_player_col] = player
        df_player[new_team_col] = player_to_team.get(player, None)
        dfs_player.append(df_player)

    df_player = pd.concat(dfs_player, axis=0)

    all_coordinate_columns = x_cols + y_cols + vx_cols + vy_cols
    if v_cols is not None:
        all_coordinate_columns += v_cols

    remaining_cols = [col for col in df_tracking.columns if col not in [frame_col] + all_coordinate_columns]

    return df_player.merge(df_tracking[[frame_col] + remaining_cols], on=frame_col, how="left")

I'm also basically done with the implementation of the model, but I decided that I would like to bundle the core functionality of my model into its own library so that it can be used as independently as possible. I currently have what I believe is a relatively efficient backend implementation and an interface that comes with (a) a function that adds a column "xC" (expected completion) to a dataframe containing passes given a supplementary dataframe containing tracking data and (b) a function that adds a column "AS" and a column "DAS" adding (dangerous) accessible space to tracking data.
I would need a little bit of guidance how to proceed from here - I can offer to implement the functionality of (a) and (b) adding the respective columns to databallpy dataframes using my own to-be-published library. Also I could write a dashboard or notebook that walks through + illustrates the computational steps involved in the model by an example, without implementing the vectorized version in databallpy itself. Let me know what you think, @Alek050!

Hi @jonas-bischofberger,
Great to hear you are making progress and are planning on implementing the model in its own package! I am happy to share that the new version of DataBallPy will have a functing to normalize tracking data relative to attacking direction (see develop docs).
Depending a little on how well you are planning to maintain your own package, I think it would be most logical to do as you propose: add a small translator in databallpy between databallpy format to what your package expects as input, and get back the xC, AS, and DAS (depending on what is being asked by the user).
With that, I would still really love to have a notebook explaining how you get to your results in code, text and visualizations. For the precise vectorized implementation we will just refer to your package and paper so its clear where the nerds can find the real code.
I think that the implementation in DataBallPy has to wait untill you released your package, but if you want you can get started on the docs for some explenations in the calculations etc. (should be added in docs/features/{name your passing model}.ipynb)
If you have any questions you know where to find me (we can also plan in a meeting to discuss this).


bam to fastq

adds the bam to fastq functionality using samtools sort, collate, and fastq in fastq_prep.nf

Weird bug for me that might not be the same on other people's setups: samtools collate has a hard time finding the correct /tmp directory, so I have to specify it before submitting the nextflow job. The error associated with this is:
Failed to open file "/home/groups/CEDAR/lancasru/tmp/collatef.0000.bam" : No such file or directory

and the 2 fastq files written will be empty.
resolved by slurm setting TMPDIR to /tmp

adds the bam to fastq functionality using samtools sort, collate, and fastq in fastq_prep.nf

Weird bug for me that might not be the same on other people's setups: samtools collate has a hard time finding the correct /tmp directory, so I have to specify it before submitting the nextflow job. The error associated with this is:
Failed to open file "/home/groups/CEDAR/lancasru/tmp/collatef.0000.bam" : No such file or directory

and the 2 fastq files written will be empty.
resolved by slurm setting TMPDIR to /tmp

I submitted my review and then re-read this comment -- totally fine if TMPDIR needs to be set manually; the tmp directory needing to be set (or else causing error) is not uncommon in Singularity and it sounds like its already been buggy


network overhead

/kind bug
/kind feature
/kind question
What steps did you take and what happened:
When I sent a request with vegeta, I thought had sufficient capacity but not handled properly.
What did you expect to happen:
I draw diagram with network flow.

I think the part marked in red is network overhead. In that part, if the transformer and predictor communicate directly, the load will be reduced by half. Also, you can use a trick to reduce the network load by placing the predictor and transformer on the same node.
Currently I am receiving a 640x640 rgb image as input. In order to reduce external traffic, png/base64 data was received by the transformer, but the load was still large due to the following problems. A 640x640 fp32 image is 3x640x640*4Byte = 4.7MB in bytes. However, when this image is transmitted as png/base64, it is resolved within a few hundred kilobytes. The use of a transformer is inevitable.
However, due to this problem, even if a transformer is used, the bandwidth of the internal network is greatly affected. I tried to solve this problem by putting the transformer and predictor on the same node, but it failed for this reason.
Are there any options to solve this?
Environment:

Istio Version: 1.10
Knative Version: 0.22.3
KFServing Version: 0.7.0
Kubeflow version: X
Kfdef:[k8s_istio/istio_dex/gcp_basic_auth/gcp_iap/aws/aws_cognito/ibm] X
Minikube version: X
Kubernetes version: (use kubectl version): 1.18.5
OS (e.g. from /etc/os-release): ubuntu 20.04.2

@caffeinism You are running the no-mesh mode, if you choose to run the mesh mode by injecting the istio sidecar in transformer and predictor then transformer can directly communicate with the predictor without going through the ingress gateway.

@yuzisun Could it be possible to use a transformer in the model mesh as well? I can't find that information.

Not right now, transformer is not supported for modelmesh but this is something on our roadmap. Transformer and predictor can also be placed in the same pod as multi containers or even same container, which predictor you are using?

@yuzisun I use triton predictor.

@yuzisun How can I use transformer in the same pod as multi containers?

Transformer and Predictor has own Pod. So, In my opinion, In kserve architecture, Can not use transformer and predictor in same pod.
But, To use preprocess and postprocess in triton, it seems that you need to use triton python backend and ensemble.
Please see triton preprocessing example.
So, In order to implement triton ensemble using kfserving, it seems to be necessary to create a total of 4 types: Preprocess TrainedModel, Postprocess TrainedModel, Model TrainedModel, and Ensemble TrainModel.
But it is hard to use and manage them.

@yuzisun How can I use transformer in the same pod as multi containers?

There was a WIP PR you can see the example here using localhost between transformer and predictor.

@yuzisun That's a nice feature! But more clearly, shouldn't it be implemented outside of predictor and transformer? As follows..
apiVersion: serving.kubeflow.org/v1beta1
kind: InferenceService
metadata:
  name: torchserve-transformer
spec:
  collocate: true
  predictor:
    containers:
    - image: kfserving/torchserve-kfs:0.3.0
      name: kfserving-container
         ...
  transformer:
    containers:
    - image: kfserving/torchserve-image-transformer:latest
      name: transformer-container
         ...

It would be good to be able to use a predictor like triton rather than a custom container.

That’s the goal but as you know the current code is structured to create separate pods for predictor and transformer, It will take a bit effort to achieve the above. I am just showing that’s possible with current inference service spec.

Thanks. Some questions have been resolved. I will close this issue.


BERT: tf_SUT runs SingleStream forever?

Running BERT with TensorFlow in the SingleStream scenario:
$ time python3 run.py --backend=tf --scenario=SingleStream

seems to take forever. (With bert.SingleStream.target_latency = 55 (ms), the ONNX runtime finishes in just over 1 minute.)
Weirdly, it prints:
2020-05-02 00:25:38.505377: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate(GHz): 1.7335
pciBusID: 0000:02:00.0
2020-05-02 00:25:38.505409: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.1
2020-05-02 00:25:38.505440: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10
2020-05-02 00:25:38.505458: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10
2020-05-02 00:25:38.505474: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10
2020-05-02 00:25:38.505490: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10
2020-05-02 00:25:38.505506: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10
2020-05-02 00:25:38.505528: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2020-05-02 00:25:38.505630: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-02 00:25:38.506097: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-02 00:25:38.506466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0
2020-05-02 00:25:38.506493: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-05-02 00:25:38.506502: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187]      0
2020-05-02 00:25:38.506509: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0:   N
2020-05-02 00:25:38.506632: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-02 00:25:38.507101: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2020-05-02 00:25:38.507498: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7533 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0, compute capability: 6.1)

again and again (and again).
I thought this could be related to my PR https://github.com/mlperf/inference/pull/593. I tried, however, moving the new code into __init__ and nothing seemed to have changed.

When bert.SingleStream.target_latency = 55555 (ms), ONNX finishes very quickly, but TF keeps going and going...

@psyhtest This is because we are creating a new TF session every time IssueQuery() is called. Maybe I should have somehow keep the session "open", but I need help from TF experts for how to do that.

Hm, tf_SUT.py looks ok to me, don't see yet why it would close the session.

I've managed to run only two queries by placing:
bert.SingleStream.target_latency = 120000
bert.SingleStream.min_query_count = 1

into user.conf. (Interestingly, no matter how much I increase target_latency beyond 120 seconds, LoadGen still issues 2 queries.)
Unlike ONNX and PyTorch, TF does run abysmally slow because of repeated init:
(mlperf) anton@mlperf-inference-bert-anton:/workspace$ time python3 run.py --backend=tf --scenario=SingleStream
...
================================================
Additional Stats
================================================
QPS w/ loadgen overhead         : 0.07
QPS w/o loadgen overhead        : 0.08

@guschmue  please help!

this is because tf.estimator.predict() will always create the model, than takes the batch and yields the results. Since we call this on issue_query() this will happen for each issue_query() and is going to be extra slow.
The fix would be to export an inference interface into a saved_model, than use this one with a regular tf.Session().

@guschmue Could you give me some pointers or docs about that? Thanks

The doc is here
https://www.tensorflow.org/api_docs/python/tf/estimator/Estimator
but the real truth is here:
https://github.com/tensorflow/estimator/blob/master/tensorflow_estimator/python/estimator/estimator.py#L612
Creating a new interface for inference should be straight forward but we'd need to upload the new saved-model.
Switching over use a good old session should not be too much work.

@guschmue  will share with @nvpohanh on a script to covert to a new saved-model. Loadgen does not work with the estimator interface. The decision is to update the reference model to the new format.

@nvpohanh, I think we can export to a saved_model with
run_squad.py --export_triton
https://github.com/NVIDIA/DeepLearningExamples/blob/master/TensorFlow/LanguageModeling/BERT/run_squad.py#L1098

@guschmue Ah, that's great! I am OOTO on 7/9-10, but I will try this next Monday.

TF model is frozen with batch size 1.  The submitter is free to other batch sizes with the provided script.


cgh.depends_on not working with memcpy events

Bug summary
I have a kernel with computation that depends on memcpy operations to multiple device vectors. I added a dependency using cgh.depends_on(std::vector<sycl::event>), but this doesn't seem to have an effect because my computation kernel is still operating on uninitialized input values.
To Reproduce
#include <array>
#include <iostream>
#include <sycl/sycl.hpp>

int main() {
  sycl::queue queue{sycl::gpu_selector{}};
  constexpr int N = 10;
  std::vector<double> x(N, 1);
  std::vector<double> y(N, 10);
  double *a = sycl::malloc_device<double>(N, queue);
  double *b = sycl::malloc_device<double>(N, queue);

  std::vector<sycl::event> depends;
  depends.push_back(queue.memcpy(a, x.data(), x.size() * sizeof(double)));
  depends.push_back(queue.memcpy(b, y.data(), y.size() * sizeof(double)));

  queue.submit([&](sycl::handler &cgh) {
    cgh.depends_on(depends);
    cgh.single_task([=]() {
      printf("a:\n");
      for (int i = 0; i < N; i++) {
        printf("%f ", a[i]);
      }
      printf("\n");

      printf("b:\n");
      for (int i = 0; i < N; i++) {
        printf("%f ", b[i]);
      }
      printf("\n");
    });
  });

  return 0;
}

Expected behavior
The output should be:
a:
1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
b:
10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000

What I get is:
a:
0.000000<PHONE_NUMBER>.193862 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
b:
0.000000<PHONE_NUMBER>.193862 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000

However, if I add a queue.wait() or wait for each copy event individually before the printing kernel, I do get the expected result.
Describe your setup

Built from source in release mode. Commit: c33e8c401f46122f2a5b028a46bbecadfb983612
Describe the dependencies that AdaptiveCpp sits on top of in your setup:

clang version and how clang was installed: 15, installed with APT
host compiler: c++
CUDA version: 11.8

Compiled with CMake, targeting a NVIDIA GeForce RTX2070 GPU with cuda:sm_75

Optional additional diagnostic information
[AdaptiveCpp Info] kernel_cache: Registering kernel ZZ4mainENKUlRN7hipsycl4sycl7handlerEE_clES2_EUlvE_
[AdaptiveCpp Info] backend_loader: Searching path for backend libs: '"/home/jollychen/adaptivecpp/lib/hipSYCL"'
[AdaptiveCpp Info] backend_loader: Successfully opened plugin: "/home/jollychen/adaptivecpp/lib/hipSYCL/librt-backend-ocl.so" for backend 'ocl'
[AdaptiveCpp Info] backend_loader: Successfully opened plugin: "/home/jollychen/adaptivecpp/lib/hipSYCL/librt-backend-cuda.so" for backend 'cuda'
[AdaptiveCpp Info] backend_loader: Successfully opened plugin: "/home/jollychen/adaptivecpp/lib/hipSYCL/librt-backend-omp.so" for backend 'omp'
[AdaptiveCpp Info] Registering backend: 'ocl'...
[AdaptiveCpp Info] ocl_hardware_manager: Hiding platform 'NVIDIA CUDA' because it has known incompatibilities. Set ACPP_RT_OCL_SHOW_ALL_DEVICES=1 to override.
[AdaptiveCpp Info] ocl_hardware_manager: Discovered OpenCL platform Intel(R) OpenCL
[AdaptiveCpp Info] ocl_hardware_manager: Hiding platform 'Intel(R) FPGA Emulation Platform for OpenCL(TM)' because it has known incompatibilities. Set ACPP_RT_OCL_SHOW_ALL_DEVICES=1 to override.
[AdaptiveCpp Info] multi_queue_executor: Spawned for backend OpenCL with configuration:
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     memcpy lane: 0
[AdaptiveCpp Info]     kernel lane: 1
[AdaptiveCpp Info] Registering backend: 'cuda'...
[AdaptiveCpp Info] multi_queue_executor: Spawned for backend CUDA with configuration:
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     memcpy lane: 0
[AdaptiveCpp Info]     memcpy lane: 1
[AdaptiveCpp Info]     kernel lane: 2
[AdaptiveCpp Info]     kernel lane: 3
[AdaptiveCpp Info] Registering backend: 'omp'...
[AdaptiveCpp Info] multi_queue_executor: Spawned for backend OpenMP with configuration:
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     memcpy lane: 0
[AdaptiveCpp Info]     kernel lane: 1
[AdaptiveCpp Info] Discovered devices from backend 'OpenCL':
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     vendor: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
[AdaptiveCpp Info]     name: Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
[AdaptiveCpp Info] Discovered devices from backend 'CUDA':
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     vendor: NVIDIA
[AdaptiveCpp Info]     name: NVIDIA GeForce RTX 2070
[AdaptiveCpp Info] Discovered devices from backend 'OpenMP':
[AdaptiveCpp Info]   device 0:
[AdaptiveCpp Info]     vendor: the hipSYCL project
[AdaptiveCpp Info]     name: hipSYCL OpenMP host device
[AdaptiveCpp Info] dag_manager: DAG manager is alive!
[AdaptiveCpp Info] runtime: ******* rt launch initiated ********
[AdaptiveCpp Info] queue: Constructed queue with node group id 1
[AdaptiveCpp Info] dag_manager: Checking DAG flush opportunity...
[AdaptiveCpp Info] dag_manager: Checking DAG flush opportunity...
[AdaptiveCpp Info] dag_manager: Checking DAG flush opportunity...
[AdaptiveCpp Info] runtime: ******* rt shutdown ********
[AdaptiveCpp Info] dag_manager: Waiting for async worker...
[AdaptiveCpp Info] dag_manager: Submitting asynchronous flush...
[AdaptiveCpp Info] dag_builder: DAG contains operations:
[AdaptiveCpp Info] 0. Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80} @node 0x55593d5dab20
[AdaptiveCpp Info] 1. Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80} @node 0x55593e230430
[AdaptiveCpp Info] 2. kernel: ZZ4mainENKUlRN7hipsycl4sycl7handlerEE_clES2_EUlvE_ @node 0x55593de31c90
[AdaptiveCpp Info]     --> requires node @0x55593d5dab20 Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info]     --> requires node @0x55593e230430 Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info] dag_manager: waiting for async worker...
[AdaptiveCpp Info] dag_manager [async]: Flushing!
[AdaptiveCpp Info] dag_manager [async]: Submitting node to scheduler!
[AdaptiveCpp Info] multi_queue_executor: Processing node 0x55593d5dab20 with 0 non-virtual requirement(s) and 0 direct requirement(s).
[AdaptiveCpp Info] multi_queue_executor: Dispatching to lane 0: Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info] inorder_executor: Processing node 0x55593d5dab20 with 0 non-virtual requirement(s) and 0 direct requirement(s).
[AdaptiveCpp Info] inorder_executor: Dispatching to lane 0x55593d7a7190: Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info] dag_manager [async]: Submitting node to scheduler!
[AdaptiveCpp Info] multi_queue_executor: Processing node 0x55593e230430 with 0 non-virtual requirement(s) and 0 direct requirement(s).
[AdaptiveCpp Info] multi_queue_executor: Dispatching to lane 1: Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info] inorder_executor: Processing node 0x55593e230430 with 0 non-virtual requirement(s) and 0 direct requirement(s).
[AdaptiveCpp Info] inorder_executor: Dispatching to lane 0x55593dff17f0: Memcpy: CPU-Device0 #1 {0, 0, 0}+{1, 1, 80}-->CUDA-Device0 #1 {0, 0, 0}+{1, 1, 80}{1, 1, 80}
[AdaptiveCpp Info] dag_manager [async]: Submitting node to scheduler!
[AdaptiveCpp Info] multi_queue_executor: Processing node 0x55593de31c90 with 2 non-virtual requirement(s) and 2 direct requirement(s).
[AdaptiveCpp Info] multi_queue_executor: Dispatching to lane 2: kernel: ZZ4mainENKUlRN7hipsycl4sycl7handlerEE_clES2_EUlvE_
[AdaptiveCpp Info] inorder_executor: Processing node 0x55593de31c90 with 2 non-virtual requirement(s) and 2 direct requirement(s).
[AdaptiveCpp Info]  --> Synchronizes with other queue for node: 0x55593d5dab20
[AdaptiveCpp Info]  --> Synchronizes with other queue for node: 0x55593e230430
[AdaptiveCpp Info] inorder_executor: Dispatching to lane 0x55593dd3f310: kernel: ZZ4mainENKUlRN7hipsycl4sycl7handlerEE_clES2_EUlvE_
[AdaptiveCpp Info] dag_manager [async]: DAG flush complete.
a:
0.000000<PHONE_NUMBER>.193862 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000
b:
0.000000<PHONE_NUMBER>.193862 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000 10.000000
[AdaptiveCpp Info] dag_manager: Shutdown.

Thanks!

Wow, thanks for the quick reply! You're right, if I add a queue.wait() at the end, I do get the expected result.
In my actual code I do have a synchronization point before exiting and copying back the result so it didn't occur to me that was the problem. But from your explanation, it's clear that I am running into the same issue there because I am queuing up a sequence of copyHtoD->compute operations with the same host buffer so by the time the actual memcpy happens the input values were already overwritten. Thank you for helping me figure that out!


Open Windows Terminal Here

Hope it is possible,
Add "Open Windows Terminal Here" to windows right click context manu, and start the terminal in the current right click location.
Thanks 😊

This is the console docs repo, for the vintage in-box console. The Terminal docs are over at https://github.com/MicrosoftDocs/terminal, and Terminal feature requests should be made over at https://github.com/MicrosoftDocs/terminal.
This is something the Terminal already supports, see https://github.com/microsoft/terminal/issues/1060 and its PR https://github.com/microsoft/terminal/pull/6100, and also see the list of open issues for the context menu entry


Use Grpc.Core.Channel call duplex stream service, the response stream read task status is always "WaitingForActivation"

What version of gRPC and what language are you using?
Grpc Version: 2.23.0
Language C#
What operating system (Linux, Windows,...) and version?
windows 1607.
What runtime / compiler are you using (e.g. python version or version of gcc)
CoreCLR. SDK Version: .Net Core 3.0.100-preview9-014004
What did you do?
If possible, provide a recipe for reproducing the error. Try being specific and include code snippets if helpful.
Code Repository: https://github.com/lv-conner/GrpcLearn.git
What did you expect to see?
the response task should work like in debug mode
the read task normal execute;
What did you see instead?
the read task status always be WaitingForActiviation until the RequestStream call CompleteAsync();
Anything else we should know about your project / environment?
if i use GrpcChannel provide by Grpc.Net.Client eveything is ok,what's different between GrpcChannel and Channel;
Did i make mistake?
the client invoke code as follow:
        static async Task Main(string[] args)
        {
            //when host in Grpc.AspNetCore.Server use this;
            AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true);
            //when use this,it can work;
            //var channel = GrpcChannel.ForAddress("http://localhost:8888", new GrpcChannelOptions()
            //{
            //    Credentials = ChannelCredentials.Insecure
            //});
            //then host in aspnetcore with https; use this;
            //var channel = GrpcChannel.ForAddress("https://localhost:8888");
            //use this will work when service host in aspnetcore.but can not work when service host in Grpc.Core.Server
            var channel = new Channel("localhost:8888", ChannelCredentials.Insecure);
            var client = new HelloMessageService.HelloService.HelloServiceClient(channel);
            var helloRes = client.SayHello(new HelloMessage.HelloMessageRequest() { Name = "tim lv" });

            var res = await client.SayHelloAsync(new HelloMessage.HelloMessageRequest() { Name = "tim lv" });
            Console.WriteLine(res.Message);

            var call = client.SayHelloStreamVersion();
            var readTask = Task.Run(async () =>
            {
                while(await call.ResponseStream.MoveNext())
                {
                    var message = call.ResponseStream.Current;
                    Console.WriteLine(message);
                }
            });
            while (true)
            {
                var result = Console.ReadKey(intercept: true);
                if (result.Key == ConsoleKey.Escape)
                {
                    break;
                }
                await call.RequestStream.WriteAsync(new HelloMessage.HelloMessageRequest() { Name = "tim lv" });
            }
            await call.RequestStream.CompleteAsync();
            await readTask;
            Console.ReadLine();
        }

From your code example, I don't see which task is always in the state "WaitingForActivation" in Grpc.Core (and is not in that state when using grpc-dotnet). Can you clarify that.
From your code example I also don't understand why the behavior you're experiencing is problematic.

From your code example, I don't see which task is always in the state "WaitingForActivation" in Grpc.Core (and is not in that state when using grpc-dotnet). Can you clarify that.
From your code example I also don't understand why the behavior you're experiencing is problematic.

Thank you for your attention。the read task is always in WaitingForActivation status  when i use .net core 3.0.100-preview9-014004 .After i upgrade .net core preview 9 to .net core rc1。the read task work ！ i'm try to uninstall the rc1 and reinstall the preview 9 ,but the rc1 install with vs2019 preview。so i think this may be the .net core preview9 issue and they fix it in rc1。


Presence not working

Presence updated to undefined

Are you running the latest version  of the bot? Your package.json should say "version": "2.0.0". Or, if you run:
git pull origin master

it should get you to the last version.
(This was a bug that happened before but I'm sure that this shouldn't be happening here anymore.)

It says 2.0.0 i manually installed ffmpeg and ffmpeg-static

Do sou have any console logs that you can share? Are you getting any error?

as you can see the song title stays as undefined

I see, can you post here the content of your queue.txt file please so I can check what link is producing this error?

The problem should be fixed with #18, please update your bot by either downloading the zip of the master's branch or by running (if you cloned the repo):
git pull origin master

Also, you'll most likely need to remove the node_modules folder and run:
npm install

before running the bot.
I'm going to close this Issue, if you come across any other problem feel free to open a new issue. Thanks for your report!


missing axiom from Braided Monoidal Category definition?

https://github.com/agda/agda-categories/blob/2eb7858ec634a2fe44767528014b4ac21f5ef061/Categories/Category/Monoidal/Braided.agda#L37-L53
I believe the definition of a braided category should also include the small axiom:
    triangle : [ X ⊗₀ unit ⇒ X ]⟨
                 B                            ⇒⟨ unit ⊗₀ X ⟩
                 unitorˡ.from
               ≈ unitorʳ.from
               ⟩

according to https://ncatlab.org/nlab/show/braided+monoidal+category, it is provable. c.f. #16.

Huh, I might try proving it in the library then if nobody else did that already.

that will be fantastic. in case of difficulties, please ask for help here.

Indeed, that would be great.  I looked at doing it myself, and it was a bit more complex than I had the time right then.  The link in issue #16 should be quite useful.  The resulting commutative diagram is big, but not too bad to turn into an equational proof.  There are quite a few combinators in Morphism.Reasoning that ought to help a lot, especially for gluing together parts of diagrams.

I suppose that the proof heavily relies on the involved morphisms being isomorphic. you might want to check https://github.com/agda/agda-categories/blob/master/Categories/Category/Monoidal/Properties.agda for an example of lifting and projecting morphisms to/from isomorphisms.

I checked multiple sources, answers and papers but I couldn't actually find a clear step-by-step proof. I tried to prove it myself and I did find that it is equivalent to a number of other identities but I couldn't prove any of them... The Stack Exchange answer has some mistakes in it and I can't make really follow it. Do you have any idea where I can find a better source?

Could you nevertheless document the sources, answers & papers that you found?  I tried to look for other sources, and couldn't find anything obvious.

Things that don't help (but may save others time in not going down these dead ends):

Lean formalization doesn't deal with this
searching for "braided monoidal category" coherence ("agda" OR "Coq" OR "Isabelle" OR "Lean" OR "Idris") on Google.

I can't remember what I found at the time but I just found a proof of an equivalent identity in https://ncatlab.org/nlab/show/monoidal+category that now allows me to prove the original statement (on paper, that is). Formalising it is going to take a while but at least I now know the steps.

I got stumped by the seemingly simple identity id {A ⊗₀ B} ≈ id {A} ⊗₁ id {B}
Do any of you have any idea how to prove this?
I can't quite find anything that does it.

With @MirceaS's hint I was able to write down my own pen-and-paper proof, which I'll document here in the hope that it'll be helpful.  (There may be easier proofs, but one has to start somewhere.)
The key insight I gained by looking at the nLab article @MirceaS cites is that 1⊗_ is equivalent to the identity functor. That means it's enough to prove
1⊗triangle : [ unit ⊗₀ (X ⊗₀ unit) ⇒ unit ⊗₀ X ]⟨
                 id ⊗₁ B                            ⇒⟨ unit ⊗₀ X ⟩
                 id ⊗₁ unitorˡ.from
               ≈ id ⊗₁ unitorʳ.from
               ⟩

The actual triangle law can then be recovered by eliminating all the unit-multiplications via naturality of unitorˡ.
In the remainder, I'll use simplified notation: l, r, denote the left and right unitors, a the associator, 1 the unit, and I'll use juxtaposition for ⊗. I'll use primes to denote inverses, i.e. a' is associator.to. I'll write X for id{X} and I'll use = instead of ≈.
The proof goes like this:

We'll need two derived laws for monoidal categories:

  kelly  :
      (XY)1 ⇒⟨ a ⟩ X(Y1) ⇒⟨ Xr ⟩ XY
    = (XY)1 ⇒⟨ r ⟩               XY
  kelly'  :
      X(Y1) ⇒⟨ a' ⟩ (XY)1 ⇒⟨ r ⟩ XY
    = X(Y1) ⇒⟨ Xr ⟩              XY

The first one is one of Kelly's laws (coherence₂ from Categories.Category.Monoidal.Properties.Kelly's); the second one is easy to prove from the first (using the fact that a is an iso).

  There might be a shorter proof in the picture from this thread on twitter.

Isn't that picture about Kelly's laws? Maybe we should tweet back a link to the mechanized version in  https://github.com/agda/agda-categories/blob/master/Categories/Category/Monoidal/Properties.agda 😄

Haha, I spent the last 2 days trying to formalise coherence₁ before realising that it was already done in Properties.agda. Oh well. Reading through Properties.agda, I think I now got the gist of how to work with lifting in the iso category.
Isn't the fact that 1⊗_ is equivalent to the identity functor precisely what's being used in our proof of Kelly's coherence, using triangle-prism? I believe we can reuse quite a bit of that code in this proof to get rid of the 1⊗ part.
I'll have a go at formalising this today and I think I should have a PR ready by the end of today (UK time).

Oh, by the way, during my work I also discovered some interesting facts such as the following identities:
  (X1)1 ⇒⟨ r  ⟩ X1
= (X1)1 ⇒⟨ r1 ⟩ X1

  1(1X) ⇒⟨  l ⟩ 1X
= 1(1X) ⇒⟨ 1l ⟩ 1X

Are these known/formalised anywhere in the code?

yeah, you know, I spent two days on building infrastructure to make Kelly's coherence properties look sufficiently trivial.

Oh, by the way, during my work I also discovered some interesting facts such as the following identities:
  (X1)1 ⇒⟨ r  ⟩ X1
= (X1)1 ⇒⟨ r1 ⟩ X1

  1(1X) ⇒⟨  l ⟩ 1X
= 1(1X) ⇒⟨ 1l ⟩ 1X

Are these known/formalised anywhere in the code?

not that i am aware of. this library doesn't have lots of properties so if you don't find anything after a quick search, the chances are it is not there.

Perhaps they belong under Morphism somewhere?

unlikely. it's a property of the right unitor. should be around Monoidal.Properties if there is one.

Wouldn't that mean that a fair bit of Morphism ought to be in Category.Properties then?

Wouldn't that mean that a fair bit of Morphism ought to be in Category.Properties then?

I feel like a very sensible approach would be to create a module inside Monoidal called, say Properties, and have each of Braided, Symmetric etc. extend it as needed with properties related to that type of category (the same way we do it with HomReasoning and my MonoidalReasoning).

Wouldn't that mean that a fair bit of Morphism ought to be in Category.Properties then?

which bit you have in mind?

All of Morphism.Reasoning. Probably more.  Morphism.Cartesian should stay where it is. Parts of the rest.
This is a discussion similar to that in issue #32 about where to put things.

Isn't the fact that 1⊗_ is equivalent to the identity functor precisely what's being used in our proof of Kelly's coherence, using triangle-prism? I believe we can reuse quite a bit of that code in this proof to get rid of the 1⊗ part.

Yes indeed!
It also suggests that triangle-prism could be strengthened slightly so it doesn't require the morphisms on the top and bottom face of the prism to be isos, only those in the "pillars". (In fact, probably only one of them needs to be an iso.)

All of Morphism.Reasoning. Probably more. Morphism.Cartesian should stay where it is. Parts of the rest.
This is a discussion similar to that in issue #32 about where to put things.

hmm... i agree with it to some degree. not sure which way is the best.

Isn't the fact that 1⊗_ is equivalent to the identity functor precisely what's being used in our proof of Kelly's coherence, using triangle-prism? I believe we can reuse quite a bit of that code in this proof to get rid of the 1⊗ part.

Yes indeed!
It also suggests that triangle-prism could be strengthened slightly so it doesn't require the morphisms on the top and bottom face of the prism to be isos, only those in the "pillars". (In fact, probably only one of them needs to be an iso.)

Indeed, only the pillar where the arrow heads meet has to be an iso.
I put up a PR with my work so far which proves the initial coherence minus the -x1 bit. I don't know when I will get around to strengthening the prism or using the current one to finish the proof but if anybody has the time to look into this, that would be nice! :)

I guess this can be closed now as it was addressed in #67 and #77 ?

BTW:

Indeed, only the pillar where the arrow heads meet has to be an iso.

I implemented this in Categories.Morphism.Reasoning.Iso now (as push-eq) and then extended it so it can be used conveniently when one has a natural isomorphism (see the last part of the proof in #77 for an example).


Evaluation MC Questions

I have a question regarding evaluating LLMs on mc questions using loglikelihood of tokens. From existing implementations like MMLU etc, the code snippet would look like this:
# Create the model
model = AutoModelForCausalLM.from_pretrained(...)
tokenizer = AutoTokenizer.from_pretrained(...)

# Tokenize the inputs
inputs = tokenizer(...)

# Get the logits
with torch.no_grad():
    outputs = model(input_ids=inputs["input_ids"].to(model.device), attention_mask=inputs["attention_mask"].to(model.device))
logits = outputs.logits[:, -1, :]

# Compute probabilities
probs = (
    torch.nn.functional.softmax(
        torch.tensor(
            [
                logits[tokenizer(" A").input_ids[-1]], # or sometimes tokenizing "A" instead of " A" etc
                logits[tokenizer(" B").input_ids[-1]],
                logits[tokenizer(" C").input_ids[-1]],
                logits[tokenizer(" D").input_ids[-1]],
            ]
        ).float(),
        dim=0,
    )
    .detach()
    .cpu()
    .numpy()
)
prediction = {0: "A", 1: "B", 2: "C", 3: "D"}[np.argmax(probs)]

However, the result from doing this is usually very different from lm evaluation harness. For example, running evaluations on llama would result in 2 different numbers. I looked into the code snippet within this repo but cannot be sure what is the difference. I understand that this repo has more wrappers around, but under the hood, is the idea the same as the code snippet? I am a bit confused since I get vastly different results from lm eval harness by coding in this way.

Something to flag is that occasionally model predictions can flip due to things as minor as running with a different batch size, if logits on two answer choices were very very close or the same floating point value. But there should not be large divergences


Support per-node or per-daemonset cost adjustments

Description
What problem are you trying to solve?
Karpenter currently calculates pricing for a set of instances based solely on the cloud provider pricing for the instance.  However, sometimes there are external costs to consider that are on a per-node basis -- for example, monitoring or security agents that have a per-node-hour fee that doesn't scale with instance size.
This results in situations where Karpenter might provision 16 "large" instances @ $0.04/hr each ($0.64/hr total cost) rather than choosing (or consolidating to) 1 "8xlarge" instance @ $0.65/hr, which on its face is cheaper but if we factor in the $0.03/node-hr for our monitoring solution, the math changes drastically ($1.12/hr for the "large" instances, $0.68/hr for the "8xlarge" instance).
Supporting a config option for a per-node-hour adjustment at the NodePool level, or perhaps creating a standard annotation that can be added to DaemonSets, would solve many of these cases.  The NodePool approach may be more accurate since the user can account for e.g. node taints where the DaemonSet wouldn't run.  I think the remaining cases (for example, where provisioning a node for even 30 seconds results in a node-month fee, i.e. SaaS providers with no hourly option) is probably outside of Karpenter's scope.
How important is this feature to you?
"Nice-to-have".  It could result in some significant cost savings over time, but it is manageable for now by simply using larger minimum instance sizes in our NodePools, which of course is a tradeoff between wasted compute resources vs the external costs.

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

I am currently in a similar situation for much the same underlying reasons (additional third-party costs correlating with the number of nodes running).
I would like to propose a slightly more nuanced configuration though, because in a given NodePool, not all available InstanceTypes will cost the same, because machine architecture has an impact on which DaemonSets we actually need to run on a given node (free vs. paid for similar functionality, etc.)
Also, not all of our cost inputs scale in the same dimension, some of the costs are per-core costs, and others are per-host costs.
If we could configure an add-on cost in a map by instance-type that the karpenter cost calculator adds on to the raw host costs returned from AWS, then we can build out our own mapping of how these costs affect different hardware.
If these costs are in the same unit as the machine cost lookups (hourly? monthly?) then the implementation could be as simple as an addition in the inner cost calculation loop from this per-node-pool map of add-on costs. Which seems like a good balance between complexity-to-implement and functional richness of the added capability.


complete webserver interface(s)

should have standard REST and WebSocket interface. improve UI design.
the REST interface will allow OpenNARS to be used as a component in websites

i like both camel routes as netty endpoints equally as well with 1xio and bbcursive core io as a handoff to NARS pre-biased toward zero-copy IO that camel will never achieve.

is this relevant
https://github.com/undertow-io/undertow-io-site/blob/53e8c5bdb8c99d1d953c7d81d7d61ed0ad2295aa/documentation/faq.asciidoc#why-isnt-undertow-based-on-mina-netty-rnio-grizzly-or-insert-network-framework
On Thu, May 28, 2015 at 6:59 PM, Jim Northrup<EMAIL_ADDRESS>wrote:

i like both camel routes as netty endpoints equally as well with 1xio and
bbcursive core io as a handoff to NARS pre-biased toward zero-copy IO that
camel will never achieve.
—
Reply to this email directly or view it on GitHub
https://github.com/opennars/opennars/issues/19#issuecomment-106622341.

doesn't appear relevant to zero-copy - though i haven't dug in.
undertow < camel i think in terms of sticking a piece of glue code in and getting a network service out.
the web service benchmarks all show pretty close performance behind netty for most toolkit flavors.  "high performance" and "lightwieght" are implicit to anything that stands up. here again i dont see a distinction.
i hope that by projecting structs onto slabs the object count can be closer to a steady-state for a given thread and to make lazy access available for memoized references.
1xio would hand nars a bytebuffer and set that as a basis for projecting structs on network buffers andemit getters and setters for interfaces

it will be an awesome but temporary solution until FIPA replaces HTTP
https://code.google.com/p/hypergraphdb/source/browse/trunk/p2p/src/java/org/hypergraphdb/peer/Performative.java#72
On Thu, May 28, 2015 at 7:35 PM, Jim Northrup<EMAIL_ADDRESS>wrote:

doesn't appear relevant to zero-copy - though i haven't dug in.
undertow < camel i think in terms of sticking a piece of glue code in and
getting a network service out.
the web service benchmarks all show pretty close performance behind netty
for most toolkit flavors. "high performance" and "lightwieght" are implicit
to anything that stands up. here again i dont see a distinction.
i hope that by projecting structs onto slabs the object count can be
closer to a steady-state for a given thread and to make lazy access
available for memoized references.
1xio would hand nars a bytebuffer and set that as a basis for projecting
structs on network buffers andemit getters and setters for interfaces
—
Reply to this email directly or view it on GitHub
https://github.com/opennars/opennars/issues/19#issuecomment-106629047.

1988: http://en.wikipedia.org/wiki/Buran_programme#Missions
2000: http://en.wikipedia.org/wiki/Agent_Communications_Language
2015: HTTP REST

2013: http://en.wikipedia.org/wiki/QUIC

post-1.7
someone should restate issues relating to orthogonally away from middleware fabrics and more descriptive project elements for action potential, signalling,  actors, and usecases


The "Line" field in the result file .json generated after execution, does not match with the line of  .feature file

Environment (please complete the following information):

Node.js version: [14.16]
NPM version: [6.14.11]
webdriver.io version: [7.5.3]
@wdio/cucumber-framework version: [7.5.3]
wdio-cucumberjs-json-reporter version: [2.0.4"]

Config of webdriver.io and the reporter
reporters: [
'spec',
    [ 'cucumberjs-json', {
        jsonFolder: '.tmp/new/',
        language: 'en',
    },
    ],

    ['allure', {
        outputDir: './tests/reports/allure-results',
        disableWebdriverStepsReporting: true,
        disableWebdriverScreenshotsReporting: false,
    }],
],
cucumberOpts: {
    backtrace: false,
    requireModule: ['@babel/register'],
    failAmbiguousDefinitions: true,
    failFast: false,
    ignoreUndefinedDefinitions: false,
    names: [],
    snippets: true,
    source: true,
    profile: [],
    require: [
        './tests/steps/*.js',
    ],
    scenarioLevelReporter: false,
    order: 'defined',
    snippetSyntax: undefined,
    strict: true,
    // <string> (expression) only execute the features or scenarios with
    // tags matching the expression, see
    // https://docs.cucumber.io/tag-expressions/
    tagExpression: 'not @Pending',
    // <boolean> add cucumber tags to feature or scenario name
    tagsInTitle: false,
    timeout: 720000,
},

Describe the bug
The "Line" field in the result file .json generated after execution, does not match with the line of  .feature file.
To Reproduce

Create a .feature file with one or more scenarios, outline scenarios, or some scenarios with te same background
Run the created .feature
Compare the ""Line" field of the steps in the results file generated with the line of the steps in .feature file

Feature Example:
Feature: New Feature
As a User
In order to use the App
I want to be able to login into App
Scenario: Correct credentials
    Given I am in the login screen of the App
    When I login with correct credentials
    Then I see the onboarding

Expected behavior
In the results file generated:
"Given" step should to have "Line" : 7
"When" step should to have "Line" : 8
"Then" step should to have "Line" : 9
Snapshots

Additional context
The snapshot is a simple Scenario, I tried with multiple scenarios in one feature, with outline scenarios, with Background keyword etc etc.

@Gomezjh
I've been checking this, but it seems to be coming from @wdio/cucumber-framework, not from this module
See also this
Can you please check there?
Closing this for now


Corrupt/glitched graphics

Issue checklist
Note: these are not necessarily requirements

[x] I have checked existing OPL issues for duplicates and found none @ http://psx-scene.com/forums/official-open-ps2-loader-forum/ & https://github.com/ifcaro/Open-PS2-Loader/issues

Open PS2 loader version
OPL 0.9.3 (VMC+GSM+PS2RD), OPL r1193 and a few DBs.
Open PS2 loader mode (USB, SMB, HDD)
USB, SMB and SATA-HDD
Open PS2 type (Normal, Childproof, VMC, GSM, PS2RD)
See above.
Issue description - Describe the problem you are having
Not sure if these are two different problems, but I'll report them as one.
When playing Phantasy Star generation:1 the party graphics will sometime glitch or get corrupted after a battle.
USB: Haven't been able to get this issue with USB.
SMB: After every battle the party graphics glitches.

The textures are from the battle screen.

SATA-HDD:
UDMA 0-4: If the game doesn't glitch after the first few battles it can take hours. But it does happen eventually.
MDMA 0-2/MODE 1: The party graphics gets corrupt within a few battles every time.

Expected result
The game should load the proper graphics for the party.
Actual result
Depending on how it is loaded and with what options the graphics is corrupted/glitched.
How to reproduce the issue
Depending on how it is loaded and with what options it either happens within a few battles or after extended play.
If not within a few battles, fastest to just reload the game.

@zappepappe, thank you for your report, i suggest that you post this report in this
thread of psx-place where we put the game bugs:
http://www.psx-place.com/threads/open-ps2-loader-game-bug-reports.19401/
Best regards.

Ok, figured it would be just as good here as I already had an account and development is done here.
But I will make an account and post it there.
Is the report good as it is or should I complement it?

I think is good enough, if they need something more they ask you, also you can
take an example in the last page of the thread about other reports, thank you.
Best regards.

I suppose this still happens on the newest build(s)?

From I remember there was no specific patch for this game like with Shadow Man: 2econd Coming,
so currently only lowering transfer should help.

SMB still reliably glitches. HDD is a bit more tedious to test, but I'll report back if anything have changed.


Kong Cloud

Just confirming a few things:

This sentence seems incomplete. "Kong Cloud comes with high performance [...], SLA, Backups...". Is it "high-performance hosting"?

Should the form heading be all-caps? I think it should be split into header-paragraph:

Before

After

"On-Prem Deployment/Cloud Deployment" options seem redundant if it's a cloud service. Remove?

https://github.com/Mashape/marketing/issues/4

This task is in hiatus.


What do these errors indicate?

I have noticed those as well when running airbash on Linux and not Android (there those messages are suppressed). I still need to test whether this also appears in POSIX mode on Linux.
To answer your question, they indicate that the current airodump process (which tries to capture a handshake after a deauth packet was sent via aireplay-ng) was killed to proceed to the next client.

Hum ... if this is just related to the killed messages, those are not errors. They should be suppressed, but in any way this should not affect functionality.
I will test this tonight on my Kali VM and see if handshakes are actually captured.
Could you a) check the database afterwards and b) try executing airbash using sh instead of bash?

I could reproduce the issue on Kali and added a fix that will suppress those messages. They were not errors, but caused by the fact that kill works asynchronously and reports back once executed successfully.


Load DataFrame from IDataReader

Not quite as straightforward as a LoadSql() method but I think this route offers a great amount of flexibility for importing data from many sources without introducing a dependency nightmare.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jmbeamer sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@jmbeamer: Thank you for opening this PR. Do you think you could add some unit tests too?

@pgovind: Sure — I'll add some this weekend


cli error when query and subscription has same operation name

Which packages are impacted by your issue?
No response
Describe the bug
When the operations documents contains queries and subscriptions with same name I got this error:
 ✖ Not all operations have an unique name:
Your Example Website or App
https://stackblitz.com/edit/github-ikasjw?file=document.graphql
Steps to Reproduce the Bug or Issue
Simply run the repro project above
Expected behavior
maybe add some config to allow user to prefix query or subscription?
Screenshots or Videos
No response
Platform

OS: [e.g. macOS, Windows, Linux]
NodeJS: [e.g. 18.5.0]
graphql version: [e.g. 16.3.0]
@graphql-codegen/* version(s): [e.g. 2.6.2]

Codegen Config File
import { CodegenConfig } from "@graphql-codegen/cli";
const config: CodegenConfig = {
schema: "schema.graphql",
documents: "document.graphql",
generates: {
"types.ts": { plugins: ["typescript", "typescript-operations"] },
},
};
export default config;
Additional context
No response

We don't plan to support this.


Need to Upgrade from Python 3.7

While working on #378, I found this error: (see run https://github.com/RTXteam/RTX-KG2/actions/runs/9669257959/job/26675319107)
NEEDRESTART-KEXP: 6.5.0-1022-azure
NEEDRESTART-KSTA: 1
++ curl -s -k https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py
++ apt-get download python3-distutils
Get:1 file:/etc/apt/apt-mirrors.txt Mirrorlist [142 B]
Get:2 http://azure.archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-distutils all 3.10.8-1~22.04 [139 kB]
Fetched 139 kB in 0s (1126 kB/s)
++ '[' -f python3-distutils_3.6.9-1~18.04_all.deb ']'
++ '[' -f python3-distutils_3.8.10-0ubuntu1~20.04_all.deb ']'
++ '[' -f python3-distutils_3.10.6-1~22.04_all.deb ']'
++ '[' -f python3-distutils_3.10.8-1~22.04_all.deb ']'
++ python3_distutils_filename=python3-distutils_3.10.8-1~22.04_all.deb
++ mv python3-distutils_3.10.8-1~22.04_all.deb /tmp
++ sudo dpkg-deb -x /tmp/python3-distutils_3.10.8-1~22.04_all.deb /
++ sudo -H python3.7 /tmp/get-pip.py
++ grep -v 'WARNING: Running pip as the '\''root'\'' user'
ERROR: This script does not work on Python 3.7 The minimum supported Python version is 3.8. Please use https://bootstrap.pypa.io/pip/3.7/get-pip.py instead.

It looks like we need to update to at least Python 3.8. Should we upgrade more than that, @saramsey? (The latest version of Python is 3.12).

I was able to fix this by downloading https://bootstrap.pypa.io/pip/3.7/get-pip.py instead. However, this seems like a temporary fix.

This is also causing issues with Dependabot:
https://github.com/RTXteam/RTX-KG2/security/dependabot/19
https://github.com/RTXteam/RTX-KG2/security/dependabot/9
(See commits f2fb19b and 80266d9)

Also, see: https://github.com/RTXteam/RTX/issues/2290

I think we really should look into upgrading the build system so that it can be run on a modern version of cpython. Marking high priority.


Fix readme

Update according to the current working status, especially regarding XMPP.

Describe the default behavior of options when not passed.

Describe the default behavior of options when not passed.

Copy the full help (output of <bot> -h) for both bots into the README.

Copy the full help (output of <bot> -h) for both bots into the README.

Check XMPP group chat status

Check XMPP group chat status

Describe better the different behaviours of transbot (see transbot.py) : implicit translation on keyword, explicit translation on pattern, target/random language


Hosting in a .Net Core 2.1 Windows Service on Windows Server 2016

I've followed the instructions here to the letter and my .Net Core 2.1 Windows Service works perfectly when I create and start the service on my Windows 10 Pro machine.
The problem arises when I deploy on to our Windows Server 2016 Datacenter OS. It creates the service as expected, but when I go to start the service I immediately get the following error:
The service did not respond to the start or control request in a timely fashion.
To emphasize, the service instantly fails to start with the above error.
If I try to run the exe itself I get the following error:
Error:
An assembly specified in the application dependencies manifest (TestService.deps.json) was not found:
package: 'Microsoft.AspNetCore.Hosting.WindowsServices', version: '2.1.1'
path: 'lib/netstandard2.0/Microsoft.AspNetCore.Hosting.WindowsServices.dll'
I suspect the two error are interlinked somehow.
I'm fairly convinced that the code is not being hit in Program.Main as I've added logging in the first line of code to be ran but nothing happens. I've tried adding the ServicesPipeTimeout registry entry to extend the time given for the service to start but I still get an instant fail. I've ensured that the server has the required versions of .Net Core installed and they are:
Microsoft. NET Core 2.1.1 - Windows Server Hosting - v2.1.13007.0
Microsoft .NET Core Runtime - 2.1.1 (x64) - <IP_ADDRESS>606
I've also ensured the IIS is enabled on the system.
I've been struggling with this for 2 days and cannot find the answers to resolve these issues anywhere.
Can this document be updated with instructions on how to deploy to the server to ensure all the pre-requisites are covered so that these issues don't occur?
I'd really like an answer to this, thanks in advance :)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b0bd8cb8-7e28-8c4d-a550-3aea8f3ab4dd
Version Independent ID: 4b60d24e-ef05-15e5-10b4-483094081bf3
Content: Host ASP.NET Core in a Windows Service
Content Source: aspnetcore/host-and-deploy/windows-service.md
Product: aspnet-core
GitHub Login: @guardrex
Microsoft Alias: tdykstra

Hello @mmaguirehub ... Do you have a package reference for Microsoft.AspNetCore.Hosting.WindowsServices in the project file?

Hi @guardrex, yes the package reference is there, here is the csproj:
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>
    <RuntimeIdentifier>win7-x64</RuntimeIdentifier>
  </PropertyGroup>

  <ItemGroup>
    <Folder Include="wwwroot\" />
  </ItemGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="Microsoft.AspNetCore.Hosting.WindowsServices" Version="2.1.1" />
  </ItemGroup>

</Project>

As you can see the version is set correctly at 2.1.1.

You're publishing the project and creating the service from the publish folder? bin\release\netcoreapp2.1\publish
btw- You won't need the Folder Include, as using the Microsoft.NET.Sdk.Web SDK has a target to move those assets on publish.

Your SDK is 2.1.300? dotnet --version

Thanks @guardrex, I wasn't creating the service from the Publish folder.  I've just tried that and it now works which is a massive relief.  I'll now remove the Folder Include.
The SDK is 2.1.301.

Ok, so the problem is in Step 4 of the topic. I'm going to make an enhancement there to make sure that readers know. Although Step 3 calls for publishing, Step 4 doesn't say to create the service from the publish folder.
I'll take care of it. Sorry u had so much trouble. We don't find out what's wrong with topics until someone (you!) runs into a problem.

@guardrex How do I make it so that it only those dependencies required are published, it currently looks like everything is output and its currently 90Mb.  I'm sure I don't need all those files

The level of trimming is rather modest I believe ... I'm just basing that on some remarks that @DamianEdwards has made about dependency trimming in general. The sample app has the same result. The publish folder comes in at 83.7 MB. I'm afraid there's nothing that can be done that I'm aware of to further trim.
I'd like to run a quick test tho. I have the prior set of specific packages for the sample app here. Let me toss those into the sample, publish it, and see if it shrinks the app at all over using the App package.
Be back in a few minutes with the answer ...

Ah ... turns out that I'm working on a topic on an old branch that has the prior version of the sample. That version calls for the packages explicitly ...
<PackageReference Include="Microsoft.AspNetCore" Version="2.1.0" />
<PackageReference Include="Microsoft.AspNetCore.CookiePolicy" Version="2.1.0" />
<PackageReference Include="Microsoft.AspNetCore.Hosting.WindowsServices" Version="2.1.0" />
<PackageReference Include="Microsoft.AspNetCore.HttpsPolicy" Version="2.1.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.1.0" />
<PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="2.1.0" />
<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="2.1.0" />

... and the result is 83.7 MB. So yeah ... unfortunately, that's producing the size I see.
You could ask as a general question on the aspnet/Home issues. One of the engineers there might comment on the state of package trimming and possibly even what they plan in the future.
https://github.com/aspnet/Home/issues

Thanks @gaurdrex for your help on this.
I'll do as you say and ask that general question on aspnet/home issues.

FYI - not sure if this is related, I see this error also even though I am referencing results of the publish folder.
I get this service timeout error only when installing in a path that uses a space in the path name (such as "C:\Program Files").  I can solve my issue by installing to a path without spaces.
I don't see the error when running interactively with the --console argument.
I got the idea to try a path without spaces by referencing an issue Peter Kottas saw - https://github.com/PeterKottas/DotNetCore.WindowsService/issues/13

Thanks - I finally tracked my problem down and it was an error in my  appsettings.json - I had grabbed the wrong appsettings.json for that test.   It had an unescaped '' in the db connection string, which made the JSON parse fail.   This was further complicated because it happens before the service startup handshake completes, which apparently doesn't log the error to the Windows Event Viewer.
So now it works as a service in paths with spaces when I use the correct appsettings.json

@bikeoid Cool. Thanks for letting me know the outcome.


In-place upgrade attempted on incompatible server type

Vultr docs specify that in-place upgrades are not supported when changing server types. However if you change an instance type using the Python interface from a vc2 type plan to a vhp type plan, Pulumi attempts to modify the server type in-place, resulting in an error from Vultr:
"{\"error\":\"Unable to upgrade plan: This plan is not an upgrade.\",\"status\":500}"
To reproduce: create a Pulumi configuration with an plan ID of vc2-1c-1gb, deploy, change plan ID to vhp-1c-1gb-amd, attempt redeploy.
I have only tested the specific scenario described above, and only using the Pulumi Python interface.

Hey @SubstructureDek,
can you try a deleteBeforeReplace
https://www.pulumi.com/docs/concepts/options/deletebeforereplace/


Functions run with compute are ignored by flutter test coverage

Steps to Reproduce

create flutter project
add a compute function
test compute function with unit test
run flutter test --coverage

Logs
Analyzing schoolsen-mobile...
No issues found! (ran in 3.7s)

[✓] Flutter (Channel beta, v1.7.8+hotfix.2, on Linux, locale en_US.UTF-8)
    • Flutter version 1.7.8+hotfix.2 at /home/gregor/development/flutter
    • Framework revision 2e540931f7 (5 hours ago), 2019-07-02 09:31:07 -0700
    • Engine revision b1cb0d9e9b
    • Dart version 2.4.0

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /home/gregor/Android/Sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-Q, build-tools 28.0.3
    • Java binary at: /snap/android-studio/76/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)
    • All Android licenses accepted.

[✓] Android Studio (version 3.4)
    • Android Studio at /snap/android-studio/76/android-studio
    • Flutter plugin version 37.0.1
    • Dart plugin version 183.6270
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1343-b16-5323222)

[!] Connected device
    ! No devices available

! Doctor found issues in 1 category.

@The-Redhat can you give an example of such a function?

Here an example:
static RequestState _parseJson(String response) {
    // parse Something
  }

  @override
  Future<RequestState> parseData(String response) =>
      compute(_parseJson, response);

@The-Redhat you need to use runAsync() whenever you perform real async work within a Flutter test.
https://api.flutter.dev/flutter/flutter_test/WidgetTester/runAsync.html
void main() {
  testWidgets('compute', (WidgetTester tester) async {
    dynamic data = await tester.runAsync(() => parseData('[1]'));
    print(data);
  });
}

Future<dynamic> parseData(String str) => compute(parseJson, str);

dynamic parseJson(String str) {
  return json.decode(str);
}

Ok thanks. I test this function in a unit test. Is runAsync there also available?

If I run this in a unit test, it works for me out of the box:
void main() {
  test('compute', () async {
    dynamic data = await parseData('[1]');
    print(data);
  });
}

Future<dynamic> parseData(String str) => compute(parseJson, str);

dynamic parseJson(String str) {
  return json.decode(str);
}

Yes sure I said it works. My problem is that the test coverage ignores _parseJson function and marks it as not tested.

lol, sorry, I lost track of the original reported bug 😆

I think it is safe to close the issue.
If you disagree please comment and I will reopen it.

@BondarenkoStas can you comment on why you re-opened this issue?

@csells I thought https://github.com/flutter/flutter/issues/35484#issuecomment-517933104 may require an attention, but it was on October and it seems that the issue is obsolete. Feel free to close it.

Unfortunately not

closing it as the issue is solved


Does not install breeze with api when routes directory does not exist

Laravel Version
11.0.6
PHP Version
8.3
Database Driver & Version
No response
Description
I like to put my routes in a directory other than the default one. If you delete the routes directory and try to install breeze with the api, the installation is interrupted with an error.
I expect that when installing breeze, if the routes directory does not exist - it will simply be re-created and all the necessary files will fit into it. Then, in my case, I would further distribute them myself according to my architecture.
Steps To Reproduce

Install laravel 11
Delete /routes directory
composer require laravel/breeze --dev
php artisan breeze:install with api option

Exception:
copy(project-root/routes/api.php): Failed to open stream: No such file or directory
at vendor\laravel\framework\src\Illuminate\Foundation\Console\ApiInstallCommand.php:53

I like to put my routes in a directory other than the default one.

This is not something we support out of the box sorry. If you want, you can attempt a PR in a non breaking way.

I made 2 PRs with fixes for this issue but they were closed. What could be the reason?


Fix android open link verify (#1956 #1397)

Proposed changes
TL;DR
I fixed a whole lot of stuff that was wrong with the autoVerify. It should now work as expected.

[!NOTE]
I cherry picked the commit from #1998 locally as I seemed to stumble on that same issue in other places as I tested

Long version

I started investigating this #1956 issue. It was pretty straightforward to reproduce and fix.

The origin is due to the fact that the reading of the app name to tap on the right cell of the Chooser is done through apkMeta.name
This value which contains the value defined in <application android:label="name"> in the Manifest. Which obviously is not mandatory, not guaranteed to be there, and even better, not what the chooser shows as value
The fix was simply to check the nullability of the value and throw if it's nullable (because it's a String! from Java-world)
However, while doing this, I noticed the code related to autoVerify and I opened another can of worms...

autoVerify is supposed to do 2 things. Select the right app from the Chooser and validate some of the onboarding screens from Chrome. Both had issues in the way they were implemented

Chrome onboarding

Depending on the version of Android and Chrome that you have, there are different variations of up to 4 onboarding screens
I just fixed the function to handle all of those

Selection of the wanted app

Before API 31 (Android 12) you always get a Chooser if your app has declared an IntentFilter with ACTION_VIEW on the domain

I fixed the code that selects the right app from the Chooser
I kept the selection of "Just once" over "Always" even though I've hesitated to switch it 🤔
This is still not perfect as we ultimately would need to get the result of something like
intent.resolveActivity(packageManager) in order to get the labels associated with the right Activities

Starting from API 31, there's no more Chooser but there's the whole "Open by default" settings menu

I added handling for all of that automatically when autoVerify is selected
I did not reverse the setting after the link is opened. I think it makes more sense like that I think. Maybe we should align "Just once" in the Chooser?

Testing
This is interesting to test. I'm not sure how we can automate that (or if we should). It would make a lot of integration tests. How were the various drivers tested?
What I did

First I had a very simple test flow

appId: dev.mobile.maestro.app
---
- openLink:
    link: 'https://maestro.mobile.dev'
    autoVerify: true
    browser: false
- openLink:
    link: 'https://maestro.mobile.dev'
    autoVerify: true
    browser: true

I created a new module with a very barebones app dev.mobile.maestro.app that I used to test the null name

    <application>
        <activity
            android:name=".MainActivity"
            android:exported="true"
            >
            <intent-filter>
                <category android:name="android.intent.category.LAUNCHER"/>
                <action android:name="android.intent.action.MAIN"/>
            </intent-filter>
            <intent-filter>
                <action android:name="android.intent.action.VIEW" />
                <category android:name="android.intent.category.DEFAULT" />
                <category android:name="android.intent.category.BROWSABLE" />
                <data android:scheme="https" />
                <data android:host="maestro.mobile.dev" />
            </intent-filter>
        </activity>
    </application>

class MainActivity : Activity() {
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(TextView(this).apply { text = "${applicationInfo.name}" })
    }
}

I created a whole bunch of brand new emulators (API 24, 25, 26, 28, 32, 33, 34, 35)
And then I ran the flow over and over on all the emulators until everything ran as expected
In order to get the different Chrome screens, you have to Force stop and Clear data for it and they are a bit random

Chrome

Accept
Sign in
Sync
Notifications

Chooser

New
App
Browser

Open by default

Before
After

Issues fixed
Fixes #1956 (Definitely fixed)
Fixes #1397 (Fixed in spirit but not in letter. I think it's just better to use autoVerify for that purpose and not add another option)

Whoah, this is awesome. Thanks!

I created a new module with a very barebones app dev.mobile.maestro.app that I used to test the null name

Could you share this code? We could add this to our e2e testing suite:

add app binary link to manifest
add tests to workspaces/<your_app_name>

Alternatively, the relevant intent filter stuff could be added to our demo_app. Wdyt?

 Currently we run only on API 28.

This topic is much more complex that I though. Thank you very much for sharing your experience and contributing.
If you think that the documentation for openLink could be expanded, we would greatly appreciate that as well.

Why was this reverted? 🤔 Looks like a super useful bunch of changes and I was looking forward to finally get rid of the custom steps we were doing in our tests. Were there breaking issues?

Hey folks, just an FYI on this PR. This created a regression in openlink command with auto verify flag on cloud. While I think this can be reworked and fixed. I'll do a root cause on what exactly caused issue and we can take in rest of the changes and fix rest. I'll do a follow up here this week


Error when running scan_package pipeline from scancode.io AppImage

Reported by @AyanSinhaMahapatra

I also could not run a package scan after uploading from local machine:

ScanCode output not available.

Traceback:
  File "/tmp/.mount_scancolQDTPj/usr/share/scancodeio/scanpipe/pipelines/__init__.py", line 122, in execute
    step(self)
  File "/tmp/.mount_scancolQDTPj/usr/share/scancodeio/scanpipe/pipelines/scan_package.py", line 113, in run_scancode
    raise FileNotFoundError("ScanCode output not available.")

I also encounter this issue when running the AppImage on my Ubuntu 22.04 instance on WSL2, but not on my laptop that runs Ubuntu 18.04.

Is any update regarding this issue ? I could not run scan_package pipeline as well


Set logging level

Fixes #4

Thanks for the PR, i was more thinking about using a more common logger like logrus. I could totally merge it but what's your thought about it ?

Sounds good! Would this be done through a logging interface?

You can now use logrus, i'm not sure about your question but you can set log level and format with logrus, let me know if you feel better with this interface

@ArthurHlt logrus works for us yes! Thanks!


Suggestion: having a dedicated key for Shoulder Surfing's 3rd person view

Which only toggles between default 1st person view and Shoulder Surfing's 3rd person view. Obviously this key can be unbinded if someone doesn't like it.
Also an option in the config file for whether Shoulder Surfing should be on F5's camera cycle or not.

A keybind option will be implemented in the next version. But i dont really see a reason why you would need a config option to disable shoulder surfing in the f5 cycle.

I only said that because I thought having options might be nice :)
But the first paragraph is what I think is gonna be useful.

I agree that its nice to have options but in this paticular case the cost/use factor is pretty high. It would need significant code changes and possibly rewrites to create a toggle switch for the shoulder surfing perspective. Its just not worth the effort

I understand, you're right. Maybe for the time-being a seperate button for switching between the default first person and Shoulder Surfing will do the job.
Also is there a reason for when I change my shaders or texturepacks inside the game I get forced to Shoulder Surfing's third person every time? Is there a way to make it use the last view that the player was using?

This issue will be fixed for 1.14.4-1.21

It's fixed, thanks.


Implement no plus plus eslint rule

Increases code consistency. Removes eslint rule from config.
This one may require manual UI testing of affected components, due to possible refactor side effects. I have manually checked Lookup up/down key, App launch tile truncate, datepicker calendar, low priority warning works, progress indicator manually and they are working correctly.

Much clearer:


⚠ Bofrost.com has degraded performance

In dffe090, Bofrost.com (https://www.bofrost.com) experienced degraded performance:

HTTP code: 200
Response time: 1050 ms

Resolved: Bofrost.com performance has improved in 3bd2646.


[WIP] pkgs: pulseaudio -> pressureaudio; nixos: override when running a daemon

Motivation for this change
An idea that struck me when writing https://github.com/NixOS/nixpkgs/pull/35292#issuecomment-367564855 (see https://github.com/NixOS/nixpkgs/pull/35355#issuecomment-367864633 for in-depth explanation of the motivation behind every part of that comment).
With this we link against pressureaudio by default to provide libpulse API over pure ALSA to apps that refuse to properly work over ALSA themselves (e.g. Firefox).
NixOS then overrides this with pulseaudioLib via LD_LIBRARY_PATH when user enables the daemon service (pressureaudio and pulseaudioLib are link-time-compatible).
This way users that don't run PulseAudio daemon and don't want to link against pulseaudioLib for religious and/or security reasons don't need to make any overrides, while PulseAudio daemon users stay happy.
This also significantly reduces closure size when not running PulseAudio daemon since pressureaudio (apulse) is tiny in comparison to pulseaudioLib and requires only alsaLib and glib.
It is a mass rebuild, but none of my PRs to staging got merged this month, so.
Things not done

[X] Not tested with hardware.pulseaudio.enable set. I refuse to set it for ethical reasons. But the change seems trivial enough that it should work.

Things done

[X] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of some pkgs that depend on this change using nix-shell -p nox --run "nox-review wip" Seems to work.
[X] Tested execution of many binary files of the changed packages. Seem to work.
[X] Fits CONTRIBUTING.md.

@GrahamcOfBorg Hooray!
I wonder what gets rebuilt on darwin. Is pulseaudio even supposed to work on darwin?

Ah. It's all those packages that do config.pulseaudio or true instead of config.pulseaudio or stdenv.isLinux.

I've set config.pulseaudio = true in my config, all that's left is to rebuild everything and see what will happen when the whole system uses fake pulseaudio.

System-wide LD_LIBRARY_PATH turned out to be a fragile approach, IMHO, and we'd like to get rid of it for OpenGL.  I wouldn't go this way.

System-wide LD_LIBRARY_PATH turned out to be a fragile approach, IMHO, and we'd like to get rid of it for OpenGL.

And replace it with what? Whatever you replace it with, this can use the same mechanism.
I think most of config.pulseaudio patches from this branch should be squashed and merged into master with some cleanups (will do later).
As to the general idea, it works well, but the whole thing together doesn't work as well as I want because many apps start using apulse instead of pure ALSA, which is tolerable, but is not what I want. I'm working on a better thing ATM.

Probably replace it with glvnd, as that provides indirection to load the right libGL anyway.  (We may still need some custom hacks, e.g. to find libGL on non-nixos, etc.)

I can see the reasoning behind glvnd - you can have multiple screens and LD_LIBRARY_PATH is not per-screen.
Note, however, that we will still need an out-of-band way to provide vendor libGLs to glvnd (or else setting it up will cause only-for-your-system mass-rebuild, so no determinism added here) and it will still do the same thing internally (I mean, all it does is just dynamically links those functions).
What is the reasoning behind your argument against this? You can't run two PulseAudio daemons at the same time.
Also not that GL and this PR are not the two users of the mechanism. "sane" module in NixOS does this for the same reason.

I think the problem with LD_LIBRARY_PATH is that some programs randomly decide to change this variable in a weird way but expect that they will still be able to find all the system libraries including libGL. Special-case side-channel variables are much less likely to be manipulated.

That was discussed at length in #31263. I'm not aware of any counterarguments to my arguments there.
In short: most things have to be fixed anyway. Things that can't be fixed can be hacked around with simple ld hacks like always appending some static paths to the end of LD_LIBRARY_PATH in ld itself or adding them to /etc/ld-nix.so.cache.

I think Steam alone disproves the numerical bound you give.
I mean, if no one would want to either write or use garbage software, we wouldn't have the current discussion.
Also. there are setuid applications.

Well, okay, point taken.
However, in any case, that discussion is orthogonal to this discussion. Any solution applicable there can be transferred here.
As long as there's no solution there I would prefer to solve problems one at a time.

Status report.
As I said in https://github.com/NixOS/nixpkgs/pull/35292#issuecomment-368217334

Something very related I made: https://github.com/oxij/libcardiacarrest
It describes the problems with PulseAudio in some detail (including direct pointers at the crazy code in the lib) and provides what seems to be a working solution for all of this shit.
Will switch #35374 to using that after I come back to my senses from making the thing.

I switched this to libcardiacarrest and now rebuilding everything the nth time for libcardiacarrest v4+. On my laptop I'm running on v2 without any problems after applying patches from #36377, so it is very usable approach.
On v4 even official freedesktop PA tools gracefully fail as they should, so it's time to get excited. Will publish after I confirm Xfce and KDE work.

Status: Done. See #36881 for the version with libcardiacarrest instead of libpressureaudio. Keeping this open till that gets merged (also to remind myself to PR the rest of the changes here eventually).

Can this be closed in favor of #36881?

I think most changes here are useful irrespective of #36881, but I'm a bit burned out on the issue to rebase them ATM.

Closing, due to the same reasons as expressed in https://github.com/NixOS/nixpkgs/pull/36881#issuecomment-455919528.
This is a controversial change, and needs to go through an RFC.
Keeping this PR open doesn't really help, it has bitrotten and doesn't apply cleanly anyhow.


Cannot override PaginationItem component slots with a button component due to ValidateDOMNesting error.

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Steps to reproduce 🕹
Link to live example:
Codesandbox
Steps:

Load sandbox
See error in console

Current behavior 😯
The slots inside PaginationItem provide a method for adding your own custom component for previous/next buttons etc. Currently you can do this, but if your custom component is a button, you get a validateDOMNesting error as the slot is for the child of a button component.
Expected behavior 🤔
I'd have expected that when adding your own custom component for the previous/next slots, it would replace the entire button component instead of just being a child of button.
Context 🔦
I get that this functionality may have been created to replace the icons for the navigational buttons. However, it makes much more sense to allow the user to add in their own custom button components for these slots.
Your environment 🌎

npx @mui/envinfo
  System:
    OS: macOS 12.4
  Binaries:
    Node: 14.19.3 - ~/.nvm/versions/node/v14.19.3/bin/node
    Yarn: 1.22.19 - ~/.nvm/versions/node/v14.19.3/bin/yarn
    npm: 6.14.17 - ~/.nvm/versions/node/v14.19.3/bin/npm
  Browsers:
    Chrome: 114.0.5735.106
    Edge: 114.0.1823.37
    Firefox: 109.0.1
    Safari: 15.5
  npmPackages:
    @emotion/react: ^11.9.3 => 11.10.6
    @emotion/styled: ^11.9.3 => 11.10.6
    @mui/base:  5.0.0-alpha.119
    @mui/core-downloads-tracker:  5.11.11
    @mui/icons-material: ^5.8.4 => 5.11.11
    @mui/lab:  5.0.0-alpha.134
    @mui/material: ^5.9.2 => 5.11.11
    @mui/private-theming:  5.11.11
    @mui/styled-engine:  5.11.11
    @mui/styles: ^5.9.2 => 5.11.11
    @mui/system:  5.11.11
    @mui/types:  7.2.3
    @mui/utils:  5.11.11
    @types/react: ^18.0.28 => 18.0.28
    react: ^18.2.0 => 18.2.0
    react-dom: ^18.2.0 => 18.2.0
    typescript: ^4.3.4 => 4.9.5

any update on this? I am currently running into the same issue

The slots inside PaginationItem provide a method for adding your own custom component for previous/next buttons etc. Currently you can do this, but if your custom component is a button, you get a validateDOMNesting error as the slot is for the child of a button component.

From the code, those slots are meant for icons. If you want to replace the button you can use component prop:
<PaginationItem component={YourCustomButton}>

How would that look in the context of rendering a Pagination component like this if I wanted my own buttons to go where the previous / next buttons would normally be?
<Pagination   count={10}   renderItem={(item) => (     <PaginationItem       slots={{ previous: ArrowBackIcon, next: ArrowForwardIcon }}       {...item}     />   )} />
As far as I can tell, there is no way to directly alter the buttons that contains the previous and next icon slot. The only example on the mui website with a custom button is using usePagination rather than the Pagination component.

How would that look in the context of rendering a Pagination component like this if I wanted my own buttons to go where the previous / next buttons would normally be?

Got it. Yeah, it's impossible to use the component prop at this moment. The workaround is to create your own Pagination with usePagination but that's quite tedious.
I mark this as an improvement for v6 or v7.


improved responsiveness

Improved navbar responsiveness
Added JavaScript for responsive navigation with a hamburger menu
Corrected footer styling

Thanks for the PR. It has been merged.


内存缓存问题

把项目中的AlamofireImage框架换成Kingfisher，然后在有UImageView的控制器总多次切换发现，内存很高，而Alamofire得很低，怀疑Kingfisher在内存缓存处理有问题。

If you find a issue, you could go deeper to find what happens and then submit it with detail to point out where and how the problem is.
Opening an issue like this has nothing to help, but a waste of time.


TextMeshPro could not be found

I'm getting CS2001 errors for about 45 resources such as the following:
\Origami\Packages\com.unity.textmeshpro\Scripts\Runtime\TMP_UpdateRegistery.cs' could not be found. Project: Unity.TextMeshPro
I'm new to C# and Visual Studio, so I apologize if this is an obvious solution, but when trying to follow the HoloLens academy videos and using the source files, I get these errors. Is TextMeshPro something I should download on my own, or are they supposed to be included in these source files?
Thanks!

Huh, we don't use TextMeshPro in any of the Academy courses.
Which version of Unity are you using?

Linking with https://github.com/MicrosoftDocs/mixed-reality/issues/557, which came to the same conclusion as the comment above. Thanks for the solution!

Sorry, still very new to the Visual Studio/C#/Unity world. Thanks for the solution! It's working now!


Truvari refine doesn't complete

Looks like I have to create a new issue (I don't have permission to re-open).
Reopening this issue, but now with Truvari refine (Truvari bench now succeeds on this input in ~15 minutes, thanks!). I've filtered out all remotely large variants and all inversions as follows:
bcftools view \
    -i 'TYPE=="SNP" || (ILEN < 1000 && ILEN > -1000)'\
    pav.all.vcf.gz |
    grep -v "INV" > pav.most.vcf
bgzip -f pav.most.vcf
tabix -p vcf pav.most.vcf.gz

I ran Truvari WFA refine on the bench results, limiting the GIAB-TR regions to candidate.refine.bed. I've included my log file here: pav.log.
On this attempt, I defined the default /tmp directory to be located on an external hard drive. It crashed after 2.5 hours with a MemoryError, and used 475GB of /x/tmp memory.
On a previous attempt, it hung for a few days after filling the /tmp directory with 185GB of data and then the current directory with another 600GB of data.
All of this data is located in files named tmp********, and they appear to be FASTA files auto-generated by samtools or something. How much space is this expected to take, and how long should the analysis run for?
Thanks in advance, sorry for opening so many issues. I'd be happy to help provide any more info to get this working.
Originally posted by @TimD1 in https://github.com/ACEnglish/truvari/issues/182#issuecomment-1877555333

Those temporary files are created by pysam.bcftools.consensus. A recent ticket about that same submodule forced me to reimplement the haplotype consensus creation step. The new procedure doesn't use temporary files (and is faster, which I wasn't expecting). Please install directly from develop and re-try.

Thanks, I'll try this now and let you know how it goes.

Looks like it's been fixed in v4.2.0. Thanks!


Add FileAppenderFactory validations and tests

I have added validations and corresponding tests to FileAppenderFactory so that the error messages are understandable.

Looks good to me :+1:


disable specific days && time slots available

Hi every one,
How you can disable specific days with react big calendar ? like disable all sundays and saturdays; or any other day.
Antoher question: How to show customers only the time slots available of a professional ( used react big react) ??
Thank's for your help

Or, how can I use a min and max to hide some days and time from calendar?

Hello @nwabdou85 were you able to resolve this?

@nwabdou85 @uvaldez I'm also interested in this one, did any of you guys find any solution?

not really sure what the question is here...what does "disabling" mean in this context

@jquense specify business hours is at lease what I mean 😄
https://fullcalendar.io/docs/display/businessHours/

you can set min/max to limit times on the week/day views

@jquense all right, great. Thanks 👍

you can also more minutely control waht is selectable by returning false form the onSelecting callback (I think thats the name)

Hy guys, sorry to be late, yeah I fixe it but by using directly fullcalendar not react-big-calendar

New pull request for business hours https://github.com/intljusticemission/react-big-calendar/pull/309, you can test it, I added some doc in the API page.

Looking for the solution on disabling past dates in the big calendar, any suggestion guys....?

Looking for solution on disabling ranges - ideally data should be passed in as if they are event data. Anyone?

past dates in the big

I've managed to do this using the timeSlotWrapper component:
<ReactBigCalendar components={{ timeSlotWrapper: TimeSlotWrapper }} />
TimeSlotWrapper:
function (props: { children: React.ReactNode, resource: null /* grid */ | undefined /* gutter */, value: Date }) {
  if (!isBanned(value)) {
    return props.children;
  }

  const child = React.Children.only(props.children);
  return React.cloneElement(child, { className: child.props.className + ' rbc-off-range-bg' });
}


All submissions time out right now, at least in Python

Seeing the same for Java and JavaScript submissions

Worked fine for a couple minutes, now times out again. :(

The background server's memory usage blew up for some reason. I've restarted the server with more memory and am looking into the underlying issue. Thanks.


Add Homebrew installation instructions

I currently have a PR open to create a Homebrew formula for pack. Don't merge this until that PR goes through and pack is available via brew install.

Closing this because we now provide a homebrew tap and the upstream PR was closed.