How to export and import segmentation?

"adapters" has no segmentation, I've been looking at how it works but it's too complicated for me :S

I'm almost done with exporting segmentations and will create a PR soon. The import is done at OHIF here, although probably I will move it to adapters soon

I'm almost done with exporting segmentations and will create a PR soon. The import is done at OHIF here, although probably I will move it to adapters soon

hi,i want to ask you that exporting segmentations is done? :)

@sedghi  How did the contourSets data come from

@sedghi i know the data structure, I want to know  How data is obtained from other software。 because i saw these data in OHIF.

I was looking at how imports done by "ohif" work. And I couldn't get it to work or something is missing in the code. If anyone knows how it works, can you create a new example?

I have already managed to present the segmentation

Here is the code just to show how it works
//
const { viewport } = libCornerstone.getEnabledElement(element);

//
await libCornerstone.imageLoader.loadAndCacheImages(imageIds);

//
const volume = await libCornerstone.volumeLoader.createAndCacheVolume(volumeId, {
    imageIds: imageIds
});

//
volume.load();

//
await viewport.setVolumes([{ volumeId }]);

//
viewport.render();

//
const result = libCornerstone.adapters.segmentation.generateToolState(viewport.imageIds, arrayBuffer);

//
const derivedVolume = await libCornerstone.volumeLoader.createAndCacheDerivedVolume(volume.volumeId, {
    volumeId: segmentationId
});
const derivedVolumeScalarData = derivedVolume.getScalarData();
derivedVolumeScalarData.set(new Uint8Array(result.labelmapBufferArray[0]));

//
libCornerstoneTools.segmentation.addSegmentation(segmentationId);
//
await libCornerstoneTools.segmentation.addSegmentationRepresentation(toolGroupId, segmentationId);

@jlopes90 how can i get segmentations in arrayBuffer format? is it SEG file?

@dima9911
Export the file is dicom with segmentation.
How to search for arraybuffer in segmentation, it is stored in the cache
var segmentations = cornerstoneTools.segmentation.state.getSegmentations();
var volumeSegmentation = cornerstone.cache.getVolume(segmentations[0].segmentationId);
var arrayBuffer = volumeSegmentation.getScalarData();

I'm almost done with exporting segmentations and will create a PR soon. The import is done at OHIF here, although probably I will move it to adapters soon

can you show me a demo how to load dicom segment?

@jlopes90 libCornerstone.adapters do not exist.
my version: "@cornerstonejs/core": "^1.65.3"

@jlopes90
i import @cornerstonejs/adapters and find function generalToolState.
but when use this, catch some error in adapters.es.js at 1660 row. generalImageModule from metaData is undefined.

but for example imagePlaneModule works well

@dima9911 https://github.com/cornerstonejs/cornerstone3D/issues/847
PR https://github.com/cornerstonejs/cornerstone3D/pull/1165

@jlopes90 can i used this with clear cornerstone?
i want to import SEG dcm file (as array buffer) for my volume viewport

@dima9911
There is a way you can do it like this:
import * as libCornerstone from '@cornerstonejs/core';
import * as libCornerstoneDicomImageLoader from '@cornerstonejs/dicom-image-loader';

function metaDataProvider(type, imageId) {
    if (Array.isArray(imageId)) {
        return;
    }

    //
    const imageUri = libCornerstone.utilities.imageIdToURI(imageId);

    //
    const dataSet = libCornerstoneDicomImageLoader.wadouri.dataSetCacheManager.get(imageUri);
    //
    if (!dataSet) {
        return;
    }

    // ------------------------------------------------------------------------

    // TODO
    if (type === 'generalImageModule') {
        return {
            sopInstanceUid: dataSet.string('x00080018')
        };
    }
}

libCornerstone.metaData.addProvider(metaDataProvider, 1);

@jlopes90 @sedghi can i used it with clear cornerstone without OHIF?
i want to import SEG dcm file (as array buffer) for my volume viewport

see my demo https://github.com/cornerstonejs/cornerstone3D/issues/1170#issuecomment-2021841681

PR #1177
I just made a new example Segmentation Volume
Here is a demo example (deploy):
https://deploy-preview-1177--cornerstone-3d-docs.netlify.app/live-examples/segmentationVolume

I just made a new Segmentation Volume example, see here #1177

thank you!!

@sedghi @jlopes90 thank you very much for this example its safe me.
the last question for this: how i can clear brush (all strategies for example circularBrush) state?
brush set tool disabled, derivedVolume.destroy, removeSegmentation, removeSegmentationRepresentation and then volumeViewport.render does nothing.
segments do not clear.
case: before import new Seg file, clear previous

@dima9911 updated pr https://github.com/cornerstonejs/cornerstone3D/pull/1177/commits/97f43780854cde279367d089ec5017c2541b0cf7

I want to provide a way to import and render segmentation using Labelmap.
const segmentationVolume = cornerstone.cache.getVolume(segmentationId)
    const scalarData = segmentationVolume.scalarData

    for (let i = 0; i < arrays.length; i++) {
      const array = arrays[i]
      const value = i + 1
      for (let j = 0; j < array.length; j++) {
        const index = array[j]
        scalarData[index] = value
      }
    }

Where, arrays is the index corresponding to non-zero coordinates of a segmentation file.
The correspondence between coordinates and index is
index = tIndex + zIndex + yIndex + xIndex
tIndex = img.nx * img.ny * img.nz * t
zIndex = img.nx * img.ny * z
yIndex = img.nx * y
xIndex = x

Read each coordinate value of the nifti file and store the index corresponding to the coordinate that is not 0 in arrays to render.

@su-yunhe
I 've tried your solution and it works pretty well. Thanks.

@jlopes90 hi
you have an error at your example

csToolsSegmentation.activeSegmentation.getActiveSegmentationRepresentation is not a function
any idea for fix it?

@sedghi hello. your examples for export segmentation do not work: stack, volume - i have error

can u fix it?

@sedghi i catch this error. what is this problem?
seg file successfully imported. but cant exported

@sedghi
pls see bug report with import seg
https://www.cornerstonejs.org/live-examples/segmentationvolume example after click "load seg"

u can see many segments with differect colors at dicoms, but i get one segment with name "Segment 1"


Changed pipenv install to pipenv sync

Attempt to fix failing build on Azure Pipelines due to timeouts on Pipfile lock.
pipenv sync does not re-lock the Pipfile.
Make the pipeline script identical to the Presidio Python base Dockerfile

Solved by #335


Check files before commit

Command:
git elegant check

Base:

git diff --check allows to check unstaged files
git diff --cached --checkallows to check staged files

Logic:

[ ] by default, unstaged and staged changes has to be tested (-a or --all option)
[ ]  -u or --unstaged option runs the check for unstaged changes
[ ] -s or --staged option runs the check for staged changes
[ ] make sure autocompletion works for --unstaged and --staged and --all options.

@extsoft Is there any sense in -a and --all flags?
Do you mean by saying:

Logic:
by default, unstaged and staged changes has to be tested (-a or --all option)

that all other arguments passed including -a and --all flags have to check both staged and unstaged files?

@ivan-podorozhnyi if

no arguments - the command behaves like --all argument
--all argument  - the command checks staged and unstaged changes
...

Is it better now?

@extsoft clear, thanks!


Incompatibility with Sodium mod

Sodium mod breaks textures for most machines, pipes, and cables.
Tested with only the following mods installed
Industrial Revolution - indrev-1.12.0-BETA.jar (also with 1.11.0 Alpha and 1.12.0 Alpha)
Fabric API - fabric-api-0.42.1+1.17.jar
REI - RoughlyEnoughItems-6.1.329.jar
Sodium - sodium-fabric-mc1.17.1-0.3.2+build.7.jar
Fabric Language Kotlin - fabric-language-kotlin-1.6.5+kotlin.1.5.31.jar
Architectury API - architectury-2.6.35.jar
Cloth Config API - cloth-config-5.1.40-fabric.jar
Also tested without Sodium, and there were no broken textures.

It's sodium fault, use Indium + sodium to fix that.

I do, and all that stuffs is just invisible for me. If I place in world.. crash D:

Those blocks are unfinished and missing a lot of things, you shouldn't use them yet

ah okay :]


NEXT: Descriptions and Cleanup

Magma Armor in the code says it changes to Battle Armor not Shell Armor

Didn't mean to add the Gen Next work :v

just the README but oh well

You could just rebase this to make the changes what you had intended.

That, combined with using unhelpful commit names such as "UPDATES" and "Oops" makes for a more annoying proposal to review.
Rebasing isn't difficult, and if you need help with rebasing feel free to ask me or the Dev room. Knowing how to rebase is really useful and it makes for a much easier way of communicating what you actually want to change.

@Zarel done and ready for you to review the descriptions

Did you just ignore the part where panpawn told you to rebase?

I will have to learn how to rebase o3o

git rebase -i HEAD~6
will give you an interactive rebase of the last 6 commits, at which point you can edit the text to rearrange, merge (squash), and rename those commits.
Tell me if you need more help.

Ok will do when I get computer access, thanks


For backwards compatibility make sure to always update react state when state changes

Huh?
For optimization the React::State module does not update the underlying react state hash unless somebody is looking at the particular state variable.
After all if nobody is looking than why update it?  (Very Zen!)
But to keep backwards compatibility, and perhaps in case for some reason you need to actually look at the react state hash, we will update it.   We can do this and avoid the re-render because we also have a "secret" state variable that only changes when observable state changes.  The default needs_update? method checks this value. By updating the underlying the state WITHOUT updating this we can avoid unnecessary re-renders, but keep the underlying react state consistent.

This issue was moved to reactrb/reactrb#78


Implement readBytesAsStream() and writeBytesFromStream()  in tcp Client

Purpose

Implemented readBytesAsStream() and writeBytesFromStream() in tcp:Client

@MohamedSabthar it seems, my previous suggestions are not addressed properly? Please recheck.

@MohamedSabthar it seems, my previous suggestions are not addressed properly? Please recheck.

@MohamedSabthar it seems, my previous suggestions are not addressed properly? Please recheck.

Updated now.

@MohamedSabthar it seems, my previous suggestions are not addressed properly? Please recheck.

Updated now.

Closing this PR as we decided not to include this for swan lake release.


im having issues making it run

when I select the notes and run the plugin nothing happens.

Firstly try

You should install ENUNU models (ENUNU_夏目悠李_RMDN_v0.0.3.zip this case) by D&D into UTAU editor so that the models are installed into username\appdata\roaming\UTAU\voice.

If the problem does not solve

UTAU should not be installed in C:\Program Files (x86) but should be installed in  C:\UTAU or D:UTAU to avoid bugs.
I reccommend you to uninstall UTAU ind re-install it into C:\UTAU.


compile for arm64

Hi, guys. can anyone explain how to compile pistache for arm64.
I saw the instructions buy it wasn't very clear. tks

Hey @annnhj. To compile from source it is the same as on any other supported architecture. There are currently pre-compiled binaries for arm64 on the unstable PPA so there is no need to waste time trying to do it yourself unless you need to modify the source.


TensorFlow 1.8 doesn't work well with PrettyTensor

STOP!
Please don't waste my time!
Most of the problems people are having are already described in the installation instructions.
You should first make a serious attempt to solve your problem.
If you ask a question that has already been answered elsewhere, or if you do not
give enough details about your problem, then your issue may be closed immediately.
Python 3
These tutorials were developed in Python 3.5 (and higher) and may give strange errors in Python 2.7
Missing Files
You need to download the whole repository, either using git clone or as a zip-file. See the installation instructions.
Questions about TensorFlow
General questions about TensorFlow should either be asked on StackOverflow or the official TensorFlow repository.
Modifications
Questions about modifications or how to use these tutorials on your own data-set should also be asked on StackOverflow.
Thousands of people are using these tutorials. It is impossible for me to give individual support for your project.
Suggestions for Changes
The tutorials cannot change too much because it would make the YouTube videos too different from the source-code.
Requests for New Tutorials
These tutorials were made by a single person on his own time. It took a very long time to
research and produce the tutorials. If a topic is not covered then the best thing is to make
a new tutorial by yourself. All you need is a decent microphone, a screen-grabbing tool, and a
video editor. I used the free version of DaVinci Resolve.
Other Issues?
Please carefully read the installation instructions and only open an issue if you are still having problems.
I got the following error message from prettytensor
AttributeError: module 'tensorflow.python.ops.variable_scope' has no attribute '_VARSCOPE_KEY'
The major cause of this issue is because prettytensor stopped development for a while and tensorflow 1.8 no longer support _VARSCOPE_KEY (Github commit).
From my own perspective there are two possible actions:

Replace PrettyTensor with Keras
To me it is a better solution because it is compatible to future versions of tensorflow. But I guess author has no intention to record the video again
Put tensorflow<1.8 in requirements.txt
From a tutorial perspective it works but it is not compatible to future versions of tensorflow

Also, starting from tensorflow 1.8, the line tf.keras.__version__ no longer works

Thanks for reporting this problem. I'm travelling at the moment so it'll be a while before I can look into this. I'll have to think about what might be the best solution that won't take me too long time to fix.

@Hvass-Labs Thanks for your effort
Really appreciated

I think a similar error is occurring with tutorial 9


Dashed Legend label as per dashed dataset border

Feature Proposal
I am trying to render the legend label as dashed w.r.t the dashed border line, as says in screenshot (Enterprise Rate Goal & Site Rate Goal). How can we achieve this? I don't see a straightforward way to do this. Any help would be appreciated!

Possible Implementation
No response

You can write a custom generatelabels function.
In here for the pointstyle you can pass a canvas element. For the first item you can draw a green rect, for the other two you can draw a custom dotted line yourself

@Joy-300799 I have a working example, please check line 58 of javascript file https://codepen.io/luaneaquino/pen/wvLWmWV


Converter in base class doesn't apply on subclasses

I am using model mapper to map my entities into serialized objects.
I have a case scenario where I need to convert a source method into a targeted property.
I have been able to make this work if I define a converter for my subclass, but since I have lots of extended classes, need to find a generic way of doing it in the base class and extending this conversion to it's children.
public interface AuthorizedEntity {
   default boolean getHasPermission(User user) {
   }
}

public class Location implements AuthorizedEntity {
}

public abstract class AuthorizedEntityDTO {
   private boolean hasPermission;
}

public class LocationDTO extends AuthorizedEntityDTO{
}

private void configureAuthorizedEntityMapping(ModelMapper mm){
        Converter securityConverter = (Converter<AuthorizedEntity , AuthorizedEntityDTO >) context -> {
            AuthorizedEntity authorizedEntity = context.getSource();
            AuthorizedEntityDTO authorizedEntityDTO = context.getDestination();

            boolean hasPermission = authorizedEntity .getHasPermission(UserContext.getLoggedUser());
            authorizedEntityDTO.setHasPermission(hasPermission);

            return authorizedEntityDTO;
        };

        mm.createTypeMap(AuthorizedEntity.class, AuthorizedEntityDTO.class, "permission")
                .include(Location.class, LocationDTO.class)
                .setPostConverter(securityConverter);

        mm.createTypeMap(AuthorizedEntity.class, AuthorizedEntityDTO.class)
                .include(Location.class, LocationDTO.class);
    }

private void configureLocationMapping(ModelMapper mm) {
        mm.typeMap(Location.class, LocationDTO.class)
                .addMapping(Location::getCoordinates, LocationDTO::setGPSCoordinates);
}

On my example I have a generic interface with some logic about permissions.
I define two types of mappings in the base, one that includes fetching the permission using the converter and setting it to the DTO, the other that doesn't perform that logic.
I include a sub class in the type mapping.
I want to have the chance to tell the mapper to include the permission calculation or not depending on the type of mapping I invoke.
When I try to use the "permission" type mapping, the conversion from the securityConverter is not invoked in the children.
If I defined the converter for the subclass, it works, but having a converter for each child is too much repeated code.

Hi @tatosmv,
It's expected behavior to only include the mappings without providers/converters as it's not type safe.
Please let me know if you have any idea or new api we can introduce on handling for your scenario.
Thanks!


Install of openshift-origin-client "Cannot remove item" error

On fresh scoop install and then install of openshift-origin-client, running into:
Remove-Item : Cannot remove item <path....\scoop\apps\openshift-origin-client\3.7.2\_tmp\oc.exe: Access to path is denied
Error is in install.ps1:367 char:21; command executed is Remove-Item -r -force "$dir\_tmp" -ea stop
Additional information:
CategoryInfo : InvalidArgument: (oc.exe:FileInfo) [Remove-Item], ArgumentException FullQualifiedErrorId: RemoveFileSystemItemArgumentError,Microsoft.PowerShell.Commands.RemoveItemCommand
Any thoughts on what is causing this error? Was working a few days ago; there are no changes in that application's manifest since I had validated this was working. Manually running Remove-Item on that _tmp directory via PowerShell yields removal of that directory.
Checking the permissions on that directory, the user I'm running as appears to have 'full control'.

This issue belongs in the Main bucket. Unfortunately, we cannot move this issue on this end, so it is being closed.
If this is still an issue, please create a new issue in the Main bucket via https://github.com/ScoopInstaller/Main/issues/new?body=Moved+from+https%3A%2F%2Fgithub.com%2Flukesampson%2Fscoop%2Fissues%2F2149


How can dynamic assign other worker for pending job.

I have a critial problem due to some issue there are 1200 jobs in a queue that queue assign an worker1 so how can I dynamic assign my free worker on the same queue to finish all pending jobs. is there any good solution for live server?

This does not sound like it is related to resque-scheduler, but you could just start up a bunch of workers for the same queue (e.g. bundle exec rake resque:work QUEUE=worker1).  A queue can have multiple workers working on it.


clojure-lsp: migrate to GraalVM

Motivation for this change
clojure-lsp now supports GraalVM builds that makes the binary faster to start and also seems to reduce memory usage.
Also bump version, to ensure we are using the latest fixes for GraalVM builds.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build clojure-lsp

Converting it in draft since the tests doesn't pass.
@GrahamcOfBorg build clojure-lsp

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-darwin 1

1 package failed to build and are new build failure:

clojure-lsp: log https://termbin.com/chog

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-linux 1

1 package failed to build and are new build failure:

clojure-lsp: log https://termbin.com/kxb1

Ok, the issue with the tests is that we are not using the same jar clojure-lsp use to build the native-image, I'll change the clojure-lsp release to provide that jar too

@GrahamcOfBorg build clojure-lsp

@GrahamcOfBorg build clojure-lsp

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-darwin 1

1 package failed to build and are new build failure:

clojure-lsp: log https://termbin.com/rqm4

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-linux 1

1 package failed to build and are new build failure:

clojure-lsp: log https://termbin.com/0ayw

Seems we will probably need to bump GraalVM to version 21.0.0 first.

Commit f6490f2 is building clojure-lsp completely from source. It will not work since clojure-lsp needs GraalVM 21.0.0 to compile, but since it is not working right now I decided to push this refactoring.

Should be ready to merge now.
@SuperSandro2000 @ericdallo, can you please review?

@GrahamcOfBorg build clojure-lsp

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-darwin 1

1 package failed to build and already failed to build on hydra master:

clojure-lsp: plain log  |  pretty log

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

clojure-lsp:

buildPhase should probably contain runHook preBuild and runHook postBuild.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:33:3:
   |
33 |   buildPhase = with lib; ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
checkPhase should probably contain runHook preCheck and runHook postCheck.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:44:3:
   |
44 |   checkPhase = ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:38:3:
   |
38 |   installPhase = ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
Unused argument: fetchurl.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:1:29:
  |
1 | { lib, stdenv, callPackage, fetchurl, fetchFromGitHub, leiningen, openjdk11
  |                             ^

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-linux 1

1 package built:

clojure-lsp

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

clojure-lsp:

buildPhase should probably contain runHook preBuild and runHook postBuild.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:33:3:
   |
33 |   buildPhase = with lib; ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
checkPhase should probably contain runHook preCheck and runHook postCheck.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:44:3:
   |
44 |   checkPhase = ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:38:3:
   |
38 |   installPhase = ''
   |   ^

See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/missing-phase-hooks.md
Unused argument: fetchurl.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:1:29:
  |
1 | { lib, stdenv, callPackage, fetchurl, fetchFromGitHub, leiningen, openjdk11
  |                             ^

Fixed the issue with the tests, now it is working.
@ericdallo Can you review it please?

@GrahamcOfBorg build clojure-lsp
@GrahamcOfBorg test clojure-lsp

@SuperSandro2000 Can we have this PR merged?

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 111864 run on x86_64-linux 1

1 package built:

clojure-lsp

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

clojure-lsp:

Unused argument: fetchurl.
Near pkgs/development/tools/misc/clojure-lsp/default.nix:1:29:
  |
1 | { lib, stdenv, callPackage, fetchurl, fetchFromGitHub, leiningen, openjdk11
  |                             ^

One last rebase since @r-ryantm created another bump PR.
Could we approve and merge this? @SuperSandro2000 @ericdallo ?


Add some samples source code..

Hi,
you expose some source code of simple usage of your library but not complete samples with source code..
Can add some working simple sample with source code in examples folder even a simple "hello world" render nothing would be nice!

hey that's a good idea, the programs in the example folder are rather outdated and not that simple. Since I am currently reworking some parts of vpp this might take a while, but will definitely do it.
In the end such a triangle "hello world" example would look much like the default vulkan example but with much less uninteresting code like setting the structure sTypes, handling the buffers memory requirements and allocation or caring much about the rather complex device and swapchain creation.


How to connect a client that uses the web interface

I am trying to use headscale with the home assistant tailscale integration, that runs tailscale with the web command. I am in a position to modify/fork the integration, but I cannot get shell access to the tailscale container. How should I approach this?

Is tailscale running on the Home assistant container or how does that work?

It runs inside a docker container hosted by the home assistant OS. It runs using the host_network mode with the NET_ADMIN capability enabled and the /dev/net/tun device mounted.
The repo can be found here.

Not sure what is being asked here

Closing this as we dont have enough info to do anything about it.


resize_images: Alignment of upsampled image seems incorrect with method=NEAREST_NEIGHBOR and align_corners=True

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): CentOS Linux release 7.2.1511 (Core)
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): ('v1.1.0-0-g1ec6ed5', '1.1.0')
Bazel version (if compiling from source): 0.4.5- (@non-git)
CUDA/cuDNN version: 7.5/5.1
GPU model and memory: M40, 12GB
Exact command to reproduce:

Describe the problem
My understanding of the behaviour of tf.image.resize_images with align_corners=True is that it would resize
[A B C D] => [A x x B x x C x x D]

if the input size was 4 and the output size was 3*(4-1)+1=10, where the x values would be determined by the method of interpolation.
(I will provide 1D examples, although I am actually working with images.)
I see the expected behaviour with method=tf.image.ResizeMethod.BILINEAR:
[0 3 6 9] => [0 1 2 3 4 5 6 7 8 9]

however, with method=tf.image.ResizeMethod.NEAREST_NEIGHBOR the result is not what I expect:
[0 3 6 9] => [0 0 0 3 3 3 6 6 6 9]    # actual result
[0 3 6 9] => [0 0 3 3 3 6 6 6 9 9]    # desired result

Note that the nearest neighbor of the third 0 in the "actual result" is the element with value 3.
Source code / logs
I have attached a simple example that demonstrates this effect: nearest.py.zip

#6720
Seems to be fixed already.

Oh thanks! It is fixed in 1.2.0


MM-18571: Include current user in suggestions

Summary
Includes the current user in autocomplete suffixed with '(you)'
Ticket Link
Fixes https://github.com/mattermost/mattermost-server/issues/12313
Related Pull Requests

Has mobile changes https://github.com/mattermost/mattermost-mobile/pull/3761

Creating a new SpinWick test server using Mattermost Cloud.

Mattermost test server created! :tada:
Access here: https://mattermost-webapp-pr-4605.test.mattermost.cloud

Account Type
Username
Password

Admin
sysadmin
Sys@dmin123

User
user-1
User-1@123

/update-branch

Test server destroyed

New commit detected. SpinWick will upgrade if the updated docker image is available.

Test server creation failed. See the logs for more information.


Use FILE_ALLOWED_EXTENSIONS to validate form upload documents

Related to hypha-issues/2

We are already having these validations on file fields(on both Multi and Single FileField). These are checking the forms before saving. https://github.com/HyphaApp/hypha/blob/main/hypha/apply/stream_forms/fields.py#L51
@frjo Where else do you think we need it?

The "Forms" are not using the uploads field from stream forms but a custom solution. So somewhere here maybe.
https://github.com/HyphaApp/hypha/blob/main/hypha/public/forms/models.py#L73-L93

Rebased to get in some fixes so test run as they should.

Codecov Report

Merging #2498 (d4a9f8f) into main (183aab8) will not change coverage.
The diff coverage is 50.00%.

@@           Coverage Diff           @@
##             main    #2498   +/-   ##
=======================================
  Coverage   81.19%   81.19%
=======================================
  Files         636      636
  Lines       23157    23157
=======================================
  Hits        18803    18803
  Misses       4354     4354

Impacted Files
Coverage Δ

hypha/public/forms/models.py
68.75% <50.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 183aab8...d4a9f8f. Read the comment docs.

@frjo I have tried to add validation in backend with custom clean method in form builder. Let me if it works good enough.

Can this be merged?


Allow demos to be opened fullscreen in new tab

Fixes #294

dstockwell approved these changes just now

oh, so that's what that button does.

Pending comments on #294


Extension method for collection validation

#753 had a discussion about the options for validating a collection of items.  There were two options discussed.  Given the following class and validator
public class Person
{
	public string FirstName { get; set; }
	public string LastName { get; set; }
}

public class PersonValidator : AbstractValidator<Person>
{
	public PersonValidator()
	{
		RuleFor(p => p.FirstName).NotNull().NotEmpty().Length(1,10);
		RuleFor(p => p.LastName).NotNull().NotEmpty().Length(1,15);
	}
}

Option 1 - Create an extension method that simply iterates and validates each item
public static class ValidatorExtensions
{
	public static ValidationResult Validate<T>(this IValidator<T> validator, IEnumerable<T> list)
	{
		var allResults = list.Select(item => validator.Validate(item)).ToList();
		var errors = allResults.SelectMany(r => r.Errors);
		return new ValidationResult(errors);
	}
}

The problem with this approach is that you lose the context of which items in the collection are the invalid ones.  In the ValidationFailure results, the PropertyName will always be 'FirstName' with no context of which item was invalid.   In contrast, when you use FluentValidation with ASP.NET MVC Core, if the input to an action method is an IEnumerable, then the model binding/validation will report validation errors that include property names like '[13].FirstName', indicating that the item at index 13 in the collection was invalid.  This is helpful when an API consumer is sending a list of data and diagnosing a particular invalid item from a collection.  My goal is to achieve similar context/feedback but with manual validation (outside of MVC model binding).
Option 2 - Create a validator for the collection
public class PeopleValidator : AbstractValidator<IEnumerable<Person>>
{

	public PeopleValidator()
	{
		RuleForEach(p => p).SetValidator(new PersonValidator());
	}
}

This is better, as the ValidationFailure results do contain context.  In my example, the PropertyName will contain a value like  p[0].FirstName.  There are two problems with this:

I need to create a collection validator for each type I need to validate in this way.  This isn't big, but feels like it should be unneccessary
The property name comes out with the variable name used in the lambda expression in the validator.  For example, p[0].FirstName  This is pretty minor, but I like the MVC error messages that would just have [0].FirstName] since the collection is really the root object.

This lead me to try a third option.
Option 3 - Extension method with custom PropertyChain
public static class ValidatorExtensions
{
	public static ValidationResult Validate<T>(this IValidator<T> validator, IEnumerable<T> list)
	{
		var i = 0;
		var allResults = list.Select(item =>
		{
			var context = new ValidationContext(item);
			context.PropertyChain.Add($"[{i++}]");

			return validator.Validate(context);
		}).ToList();

		var errors = allResults.SelectMany(r => r.Errors);
		return new ValidationResult(errors);
	}
}

This option seems to work without needing to define a custom validator per type, and also seems to yield the PropertyName property set in a way that emulates the out-of-the-box behavior with ASP.NET MVC.
So here are my two questions:

Is this a valid/proper way of accomplishing what I want?  I wasn't sure if the PropertyChain usage was appropriate or what it is intended for.
Have you considered adding an extension method or some other support for collection validation into the core package?  It seems to me this would be a common need and many people would find it useful to avoid figuring some of these things out.

Thanks for any feedback you can provide!

Hi, yes, this is a valid use of the PropertyChain. It's exposed like this so you can make any amendments to the path that you'd like.
I wouldn't say that this is particularly common. FluentValidation is primarily designed to work in a viewmodel/inputmodel scenario, where a single class represents the screen. Elements on the screen are then represented as properties (or collections of properties). The collection validation approach goes against this model, which is why I'm not very keen on it.
I'll have a further think about it, but as the method is pretty simple I'd be inclined to say leave it as an extension method that you can use in your project if needed, rather than including it in the core.
Thanks for the suggestion!


AIR SDK home issue

I add (I think a valid) AIR SDK path to local.properties file
sdk.dir=/Volumes/Work/develop/as3/_SDK_/osx/air27flex4160
But when I refresh the gradpe project I got this:
Error:Cannot find AIR SDK home, set a valid AIR SDK home from your local.properties file under project root

When you open path with finder does it work?
With PC i used double back slash for path instead, could you try on Mac
sdk.dir=\\Volumes\\Work\\develop\\as3\\_SDK_\\osx\\air27flex4160
Finnaly could you check air-sdk-description.xml file exists from your SDK (this is the file i'm looking for to validate SDK)

The
sdk.dir=\\Volumes\\Work\\develop\\as3\\_SDK_\\osx\\air27flex4160
is not help, but there was no air-sdk-description.xml in the sdk folder, just flex-sdk-description.xml. Renamed to air-sdk-description.xml and works like a charm!
thanks


Updated UI, and changed milestone for a feature.

updated Ui.png to actual current UI.
reflected new milestone for second part for student find feature.

Coverage remained the same at 65.474% when pulling f04c6e0eae85515222c4efd958d8301f1f0649c2 on m-aslam-mj2:staging10 into a92a9fb5e47ef01ce99108276c1bb5d3fe661d80 on CS2113-AY1819S1-T16-1:master.


Add pact to gemspec dependencies

A rake task was added in 501d7ad which includes "pact/tasks", but it was not added as a dependency.
Also bumps the version to 2.1.1, as the current release breaks our Jenkins pipeline for non-pact tested apps.
Trello card: https://trello.com/c/rnfUFP5o

This should also have a climate_control dependency.

This should also have a climate_control dependency.

Would that not be covered by the add_development_dependency?

Nah, that only adds that dependency for people working on this gem, it won't install climate control in apps that use this gem.
You can relatively simply refactor the code to not need climate control though.


Feature Add hdfs benchmark tools.

Proposed changes
Issue Number: close #xxx
Add hdfs benchmark tools.
Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Add local dynamic filter support in IcebergPageSourceProvder

Uses the dynamic filter in IcebergPageSourceProvider to filter results

@knwg145 could you please push the code changes for the comments you've marked as resolved ?


Fixes Issue #257: Changed call to Get-WmiObject to be Get-CimInstance to be compatibile…

… with Windows Nano.

This change is

Hi @NickMRamirez, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

Still curious whether this change fully fixes this resource on Nano, but even if not this change should be made anyways.

Reviewed 1 of 1 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

@mbreakey3 Watch out. This is a tiny change but will probably trigger a merge with your code.

Review status: :shipit: all files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from Reviewable

Even though this has been merged...I'm still learning the ropes about how to test Powershell modules. I have a Nano VM. Is there a way to replace the default module with this updated one? For testing.

You can clone whichever branch you want on your local machine, is that what you mean? You can then copy it into your programs file (where powershell modules are downloaded by default) and delete any other copies. It should then run as if that version were the one installed.

However, I don't believe there's a way to change the inbox DSC modules. You'll have to run any testing through your cloned module.

@NickMRamirez This module does not replace the default in-box module. All the resources here have different names (with the 'x' prefix) than the in-box resources, so you can use this module just as you would use any custom DSC resource module. Only PSDscResources overwrites the default in-box resources. This particular resource is not yet included in PSDscResources though.
If you have a previous version of xPSDesiredStateConfiguration installed, you will need to use Remove-Module to uninstall it.
To install the module, copy it into the C:\Windows\System32\WindowsPowerShell\v1.0\Modules folder or the C:\Program Files\WindowsPowerShell\Modules folder. Make sure one if these paths is defined in your $env:PSModulePath variable. Then just run your configuration/tests. This should import the module automatically from the correct folder. You can explicitly call Import-Module xPSDesiredStateConfiguration or Import-Module with the specific path to the module folder.

Thanks. I was able to test this change on Nano server by doing the following:

Copy the C:\Program Files\WindowsPowerShell\Modules\xPSDesiredStateConfiguration to another directory
Call Uninstall-Module on xPSDesiredStateConfiguration
Make changes to the module and then copy it back to the C:\Program Files... directory

Then, the error about using Get-WmiObject was gone.
I get a new error on Nano, but that's not related and I'll do some research on it separately:

PowerShell DSC resource MSFT_xWindowsFeature  failed to execute Test-TargetResource functionality with error message: Installing roles and features using PowerShell Desired State Configuration is supported only on Server SKU's. It is not supported on Client SKU.

@NickMRamirez Yeah. That's more what I expected. WindowsFeature is supposedly not compatible with Nano server because there are some types missing/the types don't have all the full properties on Nano. If you can figure out how to get it to work on Nano, I would love to hear it, but as it is now DSC is not supporting WindowsFeature on Nano. Sorry 😢


The problem of choose a thread?

`public final class DefaultEventExecutorChooserFactory implements EventExecutorChooserFactory {
public static final DefaultEventExecutorChooserFactory INSTANCE = new DefaultEventExecutorChooserFactory();

private DefaultEventExecutorChooserFactory() { }

@SuppressWarnings("unchecked")
@Override
public EventExecutorChooser newChooser(EventExecutor[] executors) {
    if (isPowerOfTwo(executors.length)) {
        return new PowerOfTwoEventExecutorChooser(executors);
    } else {
        return new GenericEventExecutorChooser(executors);
    }
}

private static boolean isPowerOfTwo(int val) {
    return (val & -val) == val;
}

private static final class PowerOfTwoEventExecutorChooser implements EventExecutorChooser {
    private final AtomicInteger idx = new AtomicInteger();
    private final EventExecutor[] executors;

    PowerOfTwoEventExecutorChooser(EventExecutor[] executors) {
        this.executors = executors;
    }

    @Override
    public EventExecutor next() {
        return executors[idx.getAndIncrement() & executors.length - 1];
    }
}

private static final class GenericEventExecutorChooser implements EventExecutorChooser {
    private final AtomicInteger idx = new AtomicInteger();
    private final EventExecutor[] executors;

    GenericEventExecutorChooser(EventExecutor[] executors) {
        this.executors = executors;
    }

    @Override
    public EventExecutor next() {
        return executors[Math.abs(idx.getAndIncrement() % executors.length)];
    }
}

}`
Choose a thread to execute,this arithmetic will be cause a thread handle many request,and some threads handle less request.Is there a more efficient arithmetic to choose a thread?

@zyb2013 Thats the whole point of making it pluggable. By default it will just use round-robin but you can implement your own that may take into account how many tasks are queued etc.


Localization shared resources cs file should be able to live outside of root

I used to have a website with .Net Core 2.0 that had the cs SharedResources.cs and associated resx files all under a custom folder "Myresources", and it worked well. Now it seems that the cs file needs to be moved out of the "MyResources" folder and put in the root, see this PR https://github.com/dotnet/AspNetCore.Docs/pull/19905 .
To me this causes two issues:

Documentation should be updated and it is yet not clear what should be done (I lost a day on that)
Link to the issues below and coordinate with the sdk team (?)
Imposing the user to put the shared resource file at the root of the folder is a loss of flexibility and not so tidy in my opinion; it was better before

I still do not understand what is going on, eventhough I made it work following the PR above.
Related issues for coordination
https://github.com/dotnet/aspnetcore/issues/12412
https://github.com/dotnet/aspnetcore/issues/17960
https://github.com/dotnet/docs/issues/16964

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1833676-ff1b-c587-cc9c-81afffa8ef68
Version Independent ID: ad989395-3f49-4d55-521e-cf6fe56281ee
Content: Globalization and localization in ASP.NET Core
Content Source: aspnetcore/fundamentals/localization.md
Product: aspnet-core
Technology: aspnetcore-fundamentals
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

@hishamco please review

For example one could add  to the doc how to circumvent the breaking change, by adding this to the csproj
	<PropertyGroup>
		<EmbeddedResourceUseDependentUponConvention>false</EmbeddedResourceUseDependentUponConvention>
	</PropertyGroup>

In this way the cs and resx resource file can live in "MyResources".

https://docs.microsoft.com/en-us/dotnet/core/compatibility/3.0#resource-manifest-file-name-change

Now it seems that the cs file needs to be moved out of the "MyResources" folder and put in the root

You mean the SharedResource.cs? It's just a marker, please share a minimal repo if you have any issue

@hishamco , yes I mean the SharedResources.cs file. I know it is a marker but you cannot put it in the resources folder, "MyResources" in my exampe case, and this is a loss with respect to previous versions. This is why you made the PR linked above. The only solution is to modify the csproj file. If you take the sample project and move the file to the Resources folder then you would see the problem. My point was to improve the docs explainng this change since 3.0 and I provided above the links where this is discussed.

I will check this one if I got time in netcore5.0

I know it is a marker but you cannot put it in the resources folder, "MyResources" in my example case, and this is a loss with respect to previous versions

Are you sure it's working in the previous versions?

BTW the SharedResources should be in the default namespace - even if you move it into another folder -

@hishamco, yes it is wokring in 2.0 . And yes I am using the root namespace regardless of in which folder I put it....

/cc @pranavkm @ryanbrandenburg

@hishamco I no longer work in the localization area. @mkArtakMSFT should be able to direct you to the folks currently working on localization.

You are one of the very old localization friends that I knew ;)
Thanks

See Serve files from multiple locations


Consider deprecating expect { }.not_to raise_error(SpecificErrorClass)

An expression like expect { }.not_to raise_error is fine, as it expresses intent (particularly when paired with a corresponding example that demonstrates when a particular error is raised), but expect { }.not_to raise_error(SpecificErrorClass) is dangerous.  It's prone to producing false positives.  Consider:
expect {
  expect(some_helper(foo)).to eq(:bar)
}.not_to raise_error(SomeError)

With this in place, it's impossible for the inner expectation to ever cause the example to fail.  When the inner expectation fails, it raises a RSpec::Expectations::ExpectationNotMetError, but this is not a SomeError, so the outer expectation causes it to pass.
Thus, I think we should deprecate a specific error class with the negative raise_error matching.  In 3.0 it'd be nice to disallow it entirely.

For anyone who still wants a replacement, there's a pretty simple way in RSpec 3.1+ (3.1 is the first release to have define_negated_matcher) to get it back, with an expectation that is much more explicit about what it's doing (and thus, IMO, better than what we had before):
RSpec::Matchers.define_negated_matcher :an_error_other_than, :an_instance_of

RSpec.describe "Using `an_error_other_than` as an argument to `raise_error`" do
  example "passing" do
    expect {
      raise ArgumentError
    }.to raise_error(an_error_other_than(SyntaxError))
  end

  example "failing" do
    expect {
      raise SyntaxError
    }.to raise_error(an_error_other_than(SyntaxError))
  end
end

Running this produces:
Using `an_error_other_than` as an argument to `raise_error`
  failing (FAILED - 1)
  passing

Failures:

  1) Using `an_error_other_than` as an argument to `raise_error` failing
     Failure/Error: expect {
       expected an error other than SyntaxError, got #<SyntaxError: SyntaxError> with backtrace:
         # ./spec/raise_error_spec.rb:12:in `block (3 levels) in <top (required)>'
         # ./spec/raise_error_spec.rb:11:in `block (2 levels) in <top (required)>'
     # ./spec/raise_error_spec.rb:11:in `block (2 levels) in <top (required)>'

Finished in 0.00248 seconds (files took 0.11564 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/raise_error_spec.rb:10 # Using `an_error_other_than` as an argument to `raise_error` failing

Note that this isn't exactly the same: expect { }.not_to raise_error(SyntaxError) would pass if no error was raised or if any other type of error was raised.  However, these two cases are entirely different, and IMO, if you want the former case, then expect { }.not_to raise_error is much better, and for the latter case, this is much more explicit about what it is actually doing.


⚠ gorzdrav.org has degraded performance

In 61bb57b, gorzdrav.org (https://gorzdrav.org) experienced degraded performance:

HTTP code: 200
Response time: 1937 ms

Resolved: gorzdrav.org performance has improved in 69ed6f9.


Add main menu to topbar and footer

Coverage remained the same at 100.0% when pulling 67000fe890ff2b57dcfc3b681ab2ece9a6e3d7c1 on main_menu_locations into 5d4ed07be739f19eac4300d3837cc1f0f50a43d3 on master.


Add check for required system libraries.

[x] configuration must be downloaded from   yaml file.

[x] system library description section looks for the configuration yaml file.

[x] system libraries are checked for existence

https://github.com/smart-cloud/RocketJoe/pull/14


Provide a mechanism for cross-worker resource management

So in a whole bunch of places in colossus there are settings to limit the sizes of buffers or other resources like request processing concurrency.  These settings are always per-connection or per-request-handler, and in practice that is usually sufficient.  But it can create issues if for example the workload for a service varies highly across connections.  With a per-connection limit you end up either being too restrictive to a single connection or not restrictive enough across a whole service.  I think it shouldn't be too hard to allow these settings to be global, so if we set a global concurrency limit of 100 it can be enforced across workers or across an entire service.
This should be pretty easy to do with our trusty friend AtomicLong, which has proven to work extremely well for this kind of stuff and introduces very little overhead.  Still, I think we can do one step better and allow the user to choose how the limit is enforced.
I'm envisioning a new Limit ADT that looks like:
trait ActiveLimit {
  def atLimit: Boolean
  def value: Long
  def limit: Limit
}

sealed trait Limit {
  def apply()(implicit system: IOSystem): ActiveLimit
}
case class Local(limit: Long) extends Limit
case class Shared(name: String, limit: Long) extends Limit

Every configuration setting would then be a Limit and the user can choose whether to use a Local limit, which would have the current behavior, or a Shared limit which is name-based and managed by the IOSystem.  Since shared limits are name-based, it provides a lot of flexibility to users to determine if they want one limit per service, or maybe one per worker or possibly different limits for the same service based on some other parameter.

Is this still a thing?

Closing as not sure on the use case.


Linear problem when using eval(feature) with a date property

In viewportfeatures-perf.html an expression like this is used:
strokeColor: ramp(linear($tpep_pickup_datetime,time('2016-05-01T00:08:39Z'),time('2016-05-06T10:29:11Z')),[#CF1C90,#F8CA00,#9BBD4D,#07A5AB,#B100EB])
The animation works, but trying later on to apply a blendTo or a style reset on a feature, raises an error like this:
colorspaces.js:12 Uncaught TypeError: Cannot read property 'r' of undefined
    at interpolateRGBAinCieLAB (colorspaces.js:12)
    at mix (utils.js:207)
    at RampGeneric._calcEval (RampGeneric.js:66)
    at RampGeneric.eval (RampGeneric.js:48)
    at LightweightFeature.get (lightweightFeature.js:66)
    at _renderer_viz_utils_properties__WEBPACK_IMPORTED_MODULE_0__.default.forEach (lightweightFeature.js:128)
    at Array.forEach (<anonymous>)
    at LightweightFeature.reset (lightweightFeature.js:127)
    at eval (eval at layer.on (viewportfeatures-perf.html:46), <anonymous>:1:16)
    at layer.on (viewportfeatures-perf.html:81)

The underlying cause is a hidden type error inside linear --> eval(feature), as displayed in this snippet:

Linear misses a conversion before evaluating the previous operation, so the value to be mapped is wrong.

Fix merged to master 🚀


Traffic Graph Dashboard Widget fails to draw graph and flashes units displayed back and forth

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[ X] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[ X] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
As pictured below many times when I load the dashboard it will display as below and just flash back and forth with no graph

Tip: to validate your setup was working with the previous version, use opnsense-revert (https://docs.opnsense.org/manual/opnsense_tools.html#opnsense-revert)
To Reproduce
Open dashboard.. It seems to stay this way for hours or days, then sometimes corrects. Will correct after a reboot, for a while then come back sometimes.
Expected behavior
Show graph data.
Describe alternatives you considered
Remove graph as the flashing is annoying and it isn't working much of the time.
Screenshots

Relevant log files
Not sure what logs would be applicable.
Additional context
Using Chrome on Windows as my browser, I issue also happens on Edge if I open it while the issue is happening. So I think this is a widget / data issue not a client / browser issue.
Environment
Software version used and hardware type if relevant, e.g.:
OPNsense 24.7.3-amd64

Having exactly the same issue, since 24.7.2.. Updated to 24.7.3 and it still remains.

I cannot reproduce this, any errors in Chrome developer console (F12 -> Console)?

Windows could be a factor here especially size this seems consistent across browsers on the same system. Maybe something about ChartJS that's already solved in a later version?

Case in point: https://forum.opnsense.org/index.php?topic=42640.0

@fichtner I'm on Linux, so Windows can be ruled out.
Also, I tried with FF and Brave.

Ok, someone else start this research from scratch. ;)

Here's the console log output of Firefox.
However, when I navigate to https://<IP_ADDRESS>/api/diagnostics/traffic/stream/1, I can see new streams coming in and the content refreshing
Even tho the same error appears for the cpu_usage stream, the graph is shown correctly.

Here's the console log output of Firefox. However, when I navigate to https://<IP_ADDRESS>/api/diagnostics/traffic/stream/1, I can see new streams coming in and the content refreshing

This is "normal" on Firefox and is unrelated to the problem, you won't see this on Chromium and it does not affect functionality on our end. If you switch from the dashboard to the license page, you'll see the same.

In a fresh Windows 11 23H2 install machine, I have the same issue:
https://i.imgur.com/24zu2qJ.mp4
No matter which brower I use, I get same results (tested already on Chrome, Edge, Firewall, DuckDuckGo).

In case it helps to troubleshoot it, on the PC where the Traffic widget is not working, it doesn't work also the Wireguard widget (in case you want to compare it).
It's showing as "down" the tunnels, but they are UP and you can see the traffic.
https://i.imgur.com/Rhebmcr.png

On my mobile or another PC, the same widgets are working perfectly.

In a fresh Windows 11 23H2 install machine, I have the same issue:

Zero or more interfaces selected? Errors in browser console as asked earlier?
The wireguard tunnel status is likely unrelated as it only considers a tunnel to be UP if the latest handshake was within 3 minutes, but then again, i'm stabbing in the dark here as I don't see any actual data.

One or all interfaces, same result. Here is what Chrome console is showing:
Failed to load one or more widgets: Array(1) (anonymous) | @ | opnsense_widget_mana…2da39492b655ab9:149
Refused to load the image 'data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" fill="none" stroke="%23666" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" viewBox="0 0 20 20"><path d="m10 3 2 2H8l2-2v14l-2-2h4l-2 2"/></svg>' because it violates the following Content Security Policy directive: "img-src 'self'".
Let me know if you need me to do any kind of test that could help you, or if you need more info. Happy to help!

I also just had a look into brave console, and this is what's showing up. Strangely this error does not occur in FF.

From firefox:

One or all interfaces, same result. Here is what Chrome console is showing:
Failed to load one or more widgets: Array(1) (anonymous) | @ | opnsense_widget_mana…2da39492b655ab9:149

Can you expand the caret to the left of the Array and share a screenshot?

In Chrome:

@Hipoglos Does this also happen if you restore the default dashboard?

Sorry, just make things more complicated... I checked the console in a W11 Laptop (where all widgets are working fine)...and the same error it's also there.

@Hipoglos What version are you on? Running any manual patches? Can you run a health audit in the firmware page?

@Hipoglos can you patch the zenarmor widget as described here? https://forum.opnsense.org/index.php?topic=42560.msg210817#msg210817

Ran the command. The error in the console now is gone and I can see Zenarmor on the dropdown list of the widget (before was not there)...but the graph is still not working.
I cleaned up cache just in case, and tried all browsers. Same issue.

what would help at this point is a video of the behaviour. This may have something to do with screen size or widget arrangement... since it's somehow persistent for a single installation.

I will record my screen in a moment and if it's ok for you, I will send the video privately via the forum, to avoid expose personal information here.

what would help at this point is a video of the behaviour. This may have something to do with screen size or widget arrangement... since it's somehow persistent for a single installation.

I had the issue STOP for a little while today and I had to wait for it to occur again but this might also point to screen resolution.
I have the issue right now on my windows desktop with chrome. My monitor is an ultrawide 2560x1080, however I normally keep the browser windows non full screen so if the widget looks at screen resolution instead of viewport / browser window size that might be a factor.
I have tried resizing the widget and removing and adding it with the edit action but the issue persists.
I at this moment I also have the interface open on my iPhone and it is displaying correctly in safari. So screen size MIGHT be a factor.

Guys, I just tried again, and now in the computer that was not working, is working. I didn't touch anything or apply any patch. I didn't install anything on the PC or change anything. Both (traffic graph and wireguard) are working as expected. Going to reboot and try again. Not understand what solved the issue.

Sorry, I can't reproduce the error anymore.

@Hipoglos Perhaps an outdated browser version?
Let's keep this open for now if anyone is able to reproduce it consistently.

As per my original report the issue is also intermittent for me.
My browsers should be up to date.
Issue was present in booth chrome and Edge on windows across two separate PCs. Not present when viewing on iPhone however.

@bagobones I  have a UW monitor, but it doesn't work on mobile FF either.

Same here. Found out that it works on two of my machines and not on 2 others
The Working machines (MacBook and Clean Windows 10 PC) have in common that they show me the Firewall Card as Pie Chart
The non working machines show me the firewall live log (old fashionned with IP addresses and destination etc)
Maybe thats something to check on your side as well.

Just for added confusion: I also observe this issue.
OS: Linux
Browsers: Chrome (latest), Firefox (latest)
Ona whim, I tried removing all widgets but the traffic graph, but there was no improvement. The units, flickering as they are, are also quite incorrect (toggling between fractions of a byte, and several Kb, when in reality quite a bit of traffic should be flowing.

I could reproduce this once for a couple of minutes but there was no indication of a JS error anywhere. The issue disappeared after trying to reproduce it more.
What we want to do here is to attempt updating Chart JS to version 4, but apparently it requires updating a no longer supported streaming plugin. It doesn't seem impossible, but we need to see how practical it is to go forward.
There have been a lot of fixes in Chart JS in the past 2 years. Chances are good that it has addressed this oddity already.
Cheers,
Franco

Did this issue suddenly disappear? It's unusually quiet since September.

@fichtner Surprisingly, today I realized that it just started working

Did this issue suddenly disappear? It's unusually quiet since September.

If you look up at the changes to the ticket, it has been pushed all the way to 25.1 milestone

@fichtner Surprisingly, today I realized that it just started working

I haven't run it for a while but it has always worked "intermittently" for me. I don't think it has been fixed but I have put it back on my dashboard to keep an eye on it again.

If you look up at the changes to the ticket, it has been pushed all the way to 25.1 milestone

Ok but that's what I said and usually people follow up constantly when issues like these persist... the lack of any feedback coupled with the fact that this was impossible to reproduce reliably hints to an outside factor (browser related).

@Hipoglos hey no worries, appreciate the feedback :)

OK the graph is displaying now without the flicker or failure..
I might have to open a NEW ticket as it appears there are some behavior changes?

the Pencil  / Edit icon does not consistently display / is covered by the expand full screen icon?
It doesn't remember its filters for WAN, LAN etc sort of making it useless as a dashboard item for me.

But the original issue appears to be resolved at least with a days worth of testing.

I'm just adding another data point. After 1mo on the road I came back home and powered my home router back up, updated it, and got the same behavior. My hotel router which was also updated did not exhibit this behavior on my laptop.
It's flickering and while the traffic graph shows fine under reporting, it does not show fine on the dashboard. My hotel router which was also updated has not exhibited this behavior on my laptop over the last month.

Update: It is now working again on the front page with nothing done other than letting it sit for a couple days.

My graph is still flickering after updated to 24.7.11. I'm using Firefox 133.0.3 on macOS.

https://github.com/user-attachments/assets/c41d92c5-6ed8-4606-bade-89de4b40d619

I've been having the same issue since the graphs were updated, still present on the latest update.  I don't see any errors related to it in Firefox's inspect tool though.  Same exact issue as recorded in the video above though.  Tried clearing the browser's cache and cookies with no success.

Having the exact same issue as what's shown in the screen recording above since it was updated, tried clearing my cache and cookies several times over the past few months with no luck.


added superscript plugin for adding  tags

Created new plugin to provide the ability for  tags in content.

I've added super/subscripts to core since your PR on develop branch.


Fix some arjdbc postgres tests to support AR 7.1

Hi @headius & @enebo
Hopefully we will have postgres arjdbc tests green.
In the next coming days I will look at Rails tests

@JesseChavez Looks like only one test now complaining about Time.zone leaking.  That is great though.  Local tests are nearly perfect now.


'unsupported/Eigen/CXX11/Tensor' file not found on iOS/Xcode

when I ran this project( https://github.com/yjmade/ios_camera_object_detection )on ios. I met this warning.
Stack Overflow didn't give any help, case #4680 is for Raspberry Pi. It didn't help me.
Please go to Stack Overflow for help and support:
https://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug, a feature request, or a significant problem with documentation (for small docs fixes please send a PR instead).
The form below must be filled out.
It shouldn't be a TensorBoard issue. Those go here.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
#18174
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
macOS10.13.4 Xcode9.2 iOS11.2
TensorFlow installed from (source or binary):
binary
TensorFlow version (use command below):
v1.7
Python version:
python3.6
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:
GPU model and memory:
Exact command to reproduce:

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
“
== cat /etc/issue ===============================================
Darwin Davids-New-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
Mac OS X 10.13.4
== are we in docker =============================================
No
== compiler =====================================================
Apple LLVM version 9.1.0 (clang-9<IP_ADDRESS>)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
== uname -a =====================================================
Darwin Davids-New-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
== check pips ===================================================
numpy (1.13.1)
numpydoc (0.7.0)
protobuf (3.5.0)
tensorflow (1.4.0)
tensorflow-tensorboard (0.4.0rc3)
== check for virtualenv =========================================
False
== tensorflow import ============================================
tf.VERSION = 1.4.0
tf.GIT_VERSION = v1.4.0-rc1-11-g130a514
tf.COMPILER_VERSION = v1.4.0-rc1-11-g130a514
Sanity check: array([1], dtype=int32)
/Users/David/anaconda3/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset
== nvidia-smi ===================================================
/Users/David/tensorflow-1.7.0/tools/tf_env_collect.sh: line 105: nvidia-smi: command not found
== cuda libs  ===================================================
== cat /etc/issue ===============================================
Darwin Davids-New-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
Mac OS X 10.13.4
== are we in docker =============================================
No
== compiler =====================================================
Apple LLVM version 9.1.0 (clang-9<IP_ADDRESS>)
Target: x86_64-apple-darwin17.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
== uname -a =====================================================
Darwin Davids-New-MacBook-Pro.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64
== check pips ===================================================
numpy (1.13.1)
numpydoc (0.7.0)
protobuf (3.5.0)
tensorflow (1.4.0)
tensorflow-tensorboard (0.4.0rc3)
== check for virtualenv =========================================
False
== tensorflow import ============================================
tf.VERSION = 1.4.0
tf.GIT_VERSION = v1.4.0-rc1-11-g130a514
tf.COMPILER_VERSION = v1.4.0-rc1-11-g130a514
Sanity check: array([1], dtype=int32)
/Users/David/anaconda3/lib/python3.6/importlib/_bootstrap.py:205: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6
return f(*args, **kwds)
== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset
== nvidia-smi ===================================================
/Users/David/tensorflow-1.7.0/tools/tf_env_collect.sh: line 105: nvidia-smi: command not found
== cuda libs  ===================================================
”
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

Nagging Assignee @rohan100jain: It has been 15 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @rohan100jain: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

What exactly did you do with https://github.com/yjmade/ios_camera_object_detection ? And why is it a TensorFlow issue?

Nagging Assignee @allenlavoie: It has been 16 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @allenlavoie: It has been 32 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.


Add XP and levels to avatars

Based on #15 , we can calculate XP and thus levels for each viewer in the chat.
A viewer will gain XP by just being on the chat (and in the overlay) and more XP each time he chats, interacts with the bot, and more in the future.

[x] Calculate XP based on different actions
[x] Store level in config
[x] Display level just next to the name

Done in https://github.com/MrEliptik/godot_stream_overlays/pull/24


Add Japanese translation for tutorial/index.md

This PR translates the tutorial/index.md to Japanese.
Please review tutorial/index.md.
~Issue #1571 generates a new directory to support Japanese.
I will fix dependency after #1571 is merged.~

Hi, @tiangolo
This PR has been approved by two reviewers (github actions to label approved PR didn't work, though).
Would you like to require something more for this to be merged?

Awesome! Thank you @tokusumi :bow: :coffee:
And thanks @ryuckel and @SwftAlpc for the reviews! :clap: :nerd_face:


Add Node Info to SQLQueryStatus

Resolves #452

Description
Use the get_node_info method to populate node info into SQLQueryStatus. Tested locally and validated node_info is correctly populated in logs.
Checklist

[ ] I have run this code in development and it appears to resolve the stated issue
[ ] This PR includes tests, or tests are not required/relevant for this PR
[ ] I have updated the CHANGELOG.md and added information about my change to the "dbt-databricks next" section.

Running on the integration infra now.  Thanks for the contribution.


Add worker label support

Using a worker-label.json settings file like following
[
  {
    "host" : "localhost",
    "labels" : {
      "group": "groupA",
      "location": "Beijing"
    }
  },
  {
    "host" : "host2",
    "labels" : {
      "group": "groupB",
      "location": "Shanghai"
    }
  }
]

With this PR, we can use distributedLoad shell like this.
bin/alluxio fs distributedLoad --hosts LABEL:group=groupA  /testdir

So, job master can dispatch the task to the job worker which contains the label "group=groupA".

Codecov Report

Merging #13314 (9743e82) into master (d4890f9) will decrease coverage by 43.72%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master   #13314       +/-   ##
=============================================
- Coverage     43.72%    0.00%   -43.73%
=============================================
  Files          1366       20     -1346
  Lines         79073     6083    -72990
  Branches       9594     1543     -8051
=============================================
- Hits          34575        0    -34575
+ Misses        41581     6083    -35498
+ Partials       2917        0     -2917

Impacted Files
Coverage Δ
Complexity Δ

...r/src/main/java/alluxio/master/table/Database.java
0.00% <0.00%> (-14.18%)
0.00% <0.00%> (-18.00%)

.../src/main/java/alluxio/master/table/Partition.java
0.00% <0.00%> (-5.81%)
0.00% <0.00%> (-7.00%)

...rc/main/java/alluxio/worker/grpc/WriteRequest.java
0.00% <0.00%> (-5.55%)
0.00% <0.00%> (-5.00%)

.../src/main/java/alluxio/master/journal/Journal.java
0.00% <0.00%> (-4.18%)
0.00% <0.00%> (ø%)

...ter/src/main/java/alluxio/master/table/Domain.java
0.00% <0.00%> (ø)
0.00% <0.00%> (ø%)

.../alluxio/worker/block/io/LocalFileBlockWriter.java

...ain/java/alluxio/cli/UfsVersionValidationTask.java

.../main/java/alluxio/master/file/UfsSyncChecker.java

...alluxio/underfs/oss/OSSUnderFileSystemFactory.java

...va/alluxio/worker/block/BlockMasterClientPool.java

... and 1330 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c21ec2c...9743e82. Read the comment docs.

@apc999 Can you please take a look at this PR, thank you.

@maobaolong  before we proceed with detailed review, I have a high level question:

is it possible to leverage https://docs.alluxio.io/os/user/stable/en/operation/Tiered-Locality.html for your purpose, without introducing new concept?
since this is a new feature to add, moving forward, we want a design doc similar to the doc attached here: https://github.com/Alluxio/alluxio/issues/13260 . I will tag corresponding owner to review the design doc. in this case it is @ggezer

@apc999 After a quick overview the document of Tirered-Locality, and a discussion with the original feature requester, I guess maybe Tiered-Locality cannot satisfy the request.
I have a simple design doc for this feature, please take a look.

@maobaolong
bin/alluxio fs distributedLoad --hosts LOCALITY_ID:rackA  /testdir looks confusing to me
How about supporting the following:
// already supported
bin/alluxio fs distributedLoad --hosts host1,host2  /testdir
bin/alluxio fs distributedLoad --host-file /host/file  /testdir
// to support
bin/alluxio fs distributedLoad --locality id1,id2  /testdir
bin/alluxio fs distributedLoad --locality-file /locality/file  /testdir

@apc999 Thank you for the suggestion, I push a new commit to address your suggestion, PTAL.

alluxio-bot, merge this please


Webpage not generated by cloned template

I have cloned this the template using instructions at https://perseids-publications.github.io/treebank-template/instructions/getting-started/
No webpage appears at https://rgorman.github.io/ugrad-research-trees/

I cloned your repository and it built correctly. See https://zfletch.github.io/ugrad-research-trees/. It's possible there was a minor GitHub outage. Have you tried running the action again?


Provide placeholders for cases

Depending on #155.
C.f. #113

Then I suppose the other PR should be a merge instead of squash? I will leave it to you.

Squash is fine. I can rebase this.


Fix 'unexpected EOF' in make_jar.sh

Fix this:
$ make -C pentadactyl xpi
Packaging chrome...
/.../dactyl/pentadactyl/../common/make_jar.sh: line 70: unexpected EOF while looking for matching `''
make: *** [chrome/] Error 2

What shell are you seeing a problem in? The current version works fine in ash, dash, bash, and POSIXLY_CORRECT bash.

This is common error on Mac OS X (all versions) with Apple's default bash and sh shells which is at version 3.2.53. Note that Apple will most likely never update to newer version of BASH because newer BASH uses GPL 3.0.
If, however you update your /bin/bash and /bin/sh to latest GNU BASH (sh is just renamed bash) version 4.3.33 you will not have this problem.
The best thing is to install GNU BASH to /usr/local/bin (this is the default), make a backup copy of Apple's default /bin/bash to /bin/bash3 and /bin/sh to /bin/sh3 and then copy /usr/local/bin/bash to /bin/bash and /usr/local/bin/bash to /usr/sh.

I'm not willing to replace the heredoc with a multi-line quoted string like this, especially just to support an ancient version of bash. I've made a different change instead. If it doesn't work, please file a bug.

just FYI, the following shells have problem with it:
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin14)
zsh 5.0.7 (x86_64-apple-darwin13.4.0)


Improve Peek

Right now peek just looks at the most recent submission,
but it would be great if you can give it an id and it can fetch any submission.

Implemented in the new API.


Add the filter categories for all Letter L and MayaData documentations

Research on each documentation in the Letter L and MayaData section to filter them into Company Owned Documentation and Community Owned Documentation
After getting your answer, add the category like this in the README.md file at the front of the documentation wrapped in backticks ``
e.g.

Amazon S3 Docs - Amazon Simple Storage Service (Amazon S3) is storage for the internet. You can use Amazon S3 to store and retrieve any amount of data at any time, from anywhere on the web. - Company Owned Documentation

i would love to work on this issue please

@Navlos you can work on this.

I will like to work on this issue if am assign to

@Captain-Tee01 you can work on this

Solved #134


Using byChunk inside a std.parallelism's parallel loop causes crash when compiled in 32 bits

ancien9 reported this on 2014-01-03T07:23:48Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=11862
CC List

ancien9
dlang-bugzilla (@CyberShadow)
kozzi11 (@Kozzi11)
nick (@ntrel)

Description
The following code produce some weird behavior when compiled in 32bits on my Windows 7 64b machine. Removing the parallel or compiling in 64b removes the problems.

import std.md5;
import std.stdio;
import std.file;
import std.conv;
import std.getopt;
import std.string;
import std.process;
import std.parallelism;
import std.exception;

struct Entry
{
    string name;
    ubyte[16] md5;
}

void main(string[] args)
{
    string folder1 = args[1];
    string folder2 = args[2];

    Entry[] entries1;
    foreach (name; parallel(dirEntries(folder1, SpanMode.breadth), 1)) // not working
    {
        if(name.isFile())
        {
            entries1 ~= Entry(name ,mdFile(name));
        }
    }

    writeln(folder1," has ",entries1.length, " entries");
    Entry[] entries2;
    foreach (string name; dirEntries(folder2, SpanMode.breadth)) //working fine
    {
        if(name.isFile())
        {
            entries2 ~= Entry(name ,mdFile(name));
        }
    }
    writeln(folder2," has ", entries2.length, " entries");
}
/// Digests a file and prints the result.
ubyte[16] mdFile(string filename)
{
    MD5_CTX context;
    ubyte[16] digest;
    context.start();
    File f = File(filename,"r");

    foreach (buffer; f.byChunk(4096))
        context.update(buffer);

    context.finish(digest);
    try{
        f.close();
    }
    catch(ErrnoException e)
    {
        writeln(e.errno);
    }
    writefln("MD5 (%s) = %s", filename, digestToString(digest));
    return digest;
}

When executed, the program starts opening files and calculating their MD5. After a short amount of files have been scanned, depending on the WorkUnitSize (between 2 and 10 if set to 1, if default to 100, around 30-40 are processed), the program begins malfunctioning
- Seemingly random files throw an ErrnoException (error n° 0) when closed (not interrupting the rest of the program)
- The program sometime just freeze (need to close it manually)
- The program generates "Object.Error: Access violation"
- The program crashes (program.exe has stopped working).

The culprit seems to be the function byChunk used to calculate the MD5.
Using std.stream like suggested in this question's answer ( http://stackoverflow.com/questions/20888596/opening-files-in-a-parallel-loop-crash-the-program ) makes the program work in 32 and 64bits.

While this issue can be fixed by compiling in 64bit, it's still a major problem.

PS : It's my first time reporting a bug, so if some information is lacking or I've submitted this wrong, please let me know and I'll try to fix it.

kozzi11 (@Kozzi11) commented on 2014-01-03T13:11:11Z
Problem seems to be in T[] rawRead(T)(T[] buffer)

 version(Win32)
        {
            immutable fd = ._fileno(_p.handle);
            //immutable mode = ._setmode(fd, _O_BINARY); when comment this line
            //scope(exit) ._setmode(fd, mode); and this line it seems to works much better
            version(DIGITAL_MARS_STDIO)
            {
                // @@@BUG@@@ 4243
                immutable info = __fhnd_info[fd];
                __fhnd_info[fd] &= ~FHND_TEXT;
                scope(exit) __fhnd_info[fd] = info;
            }
        }

nick (@ntrel) commented on 2023-01-14T21:36:08Z
Note that writing to `entries1` here causes a race condition:
```
    Entry[] entries1;
    foreach (name; parallel(dirEntries(folder1, SpanMode.breadth), 1)) // not working
    {
        if(name.isFile())
        {
            entries1 ~= Entry(name ,mdFile(name));
```
See also issue 15129.