Generate numbers image with CLI

Checklist

[X] I've given this PR a concise, self-descriptive, and meaningful title
[X] I've linked relevant issues in the PR body
[X] I've applied the relevant labels to this PR
[ ] I've added relevant tests for my contribution
[X] I've updated the documentation and/or added correct docstrings
[ ] I've assigned a reviewer
[ ] I've consulted ADS's internal developer documentation to ensure my contribution is in line with any relevant design decisions

Description

This PR implements a change in the CLI axondeepseg_morphometrics
This change generates two output files with the following suffixes:

_numbers_seg.png: Only numbers
_colored_seg.png: Axon+myelin seg with numbers on it.

In the FSLeyes plugin, the numbers image will be saved in the same repository as the output Excel file

Linked issues
Resolves #512

Coverage increased (+0.1%) to 82.922% when pulling af9b0470424f2af0f091e385ef3011ead56fef71 on sia/512/numbers_CLI into e4ffb28afe118566ec2dc5bba7e2bb859d55a0c6 on master.

Ignore my second suggestion, we would be requiring overlaid numbers on the EM image instead of just the numbers, especially for Alan's project. Is that possible?

Ignore my second suggestion, we would be requiring overlaid numbers on the EM image instead of just the numbers, especially for Alan's project. Is that possible?

This may be tricky because of the color of the numbers vs. the color of the image background, depending on the modality, the background can be more black or more white. We could implement it for Alan's project if he needs it but I don't think that generating an original image with overlay numbers on it should be a permanent feature. Also, converting the original image from grayscale to color to overlay the numbers will generate a lot of data. That could be partially alleviated by saving the new files as jpeg as they are for QC only.
I'm in favor or your suggestion 2 with "numbers_seg.png" as a permanent feature (as we did last week for Alan's project). As you mentioned: "users would less likely overlay the generated numbers.png on the segmented axonmyelin mask and they would expect us to do that". @vs74, can you share with Stoyan the script you used to generate that in the CLI last week?

This may be tricky because of the color of the numbers vs. the color of the image background, depending on the modality, the background can be more black or more white. We could implement it for Alan's project if he needs it but I don't think that generating an original image with overlay numbers on it should be a permanent feature. Also, converting the original image from grayscale to color to overlay the numbers will generate a lot of data. That could be partially alleviated by saving the new files as jpeg as they are for QC only.

I agree with you @mariehbourget. Perhaps we have to modify our scripts if Alan asks us to overlay EM images on top of numbers.

I'm in favor or your suggestion 2 with "numbers_seg.png" as a permanent feature (as we did last week for Alan's project). As you mentioned: "users would less likely overlay the generated numbers.png on the segmented axonmyelin mask and they would expect us to do that". @vs74, can you share with Stoyan the script you used to generate that in the CLI last week?

Here is the modified launch_morphometrics_computation.py  script which me and @mariehbourget created for generating _numbers_seg.png numbers overlaid on axonmyelin masks. This script also takes care of my suggestion 1, which is renaming numbers.png by adding _numbers_seg.png  as a suffix to the image name:

# coding: utf-8

from pathlib import Path
import argparse
from argparse import RawTextHelpFormatter
from matplotlib import image
import sys

# Scientific modules imports
import numpy as np
import pandas as pd

# AxonDeepSeg imports
from AxonDeepSeg.morphometrics.compute_morphometrics import (
                                                                get_axon_morphometrics,
                                                                save_axon_morphometrics,
                                                                draw_axon_diameter,
                                                                save_map_of_axon_diameters,
                                                                get_aggregate_morphometrics,
                                                                write_aggregate_morphometrics
                                                            )
import AxonDeepSeg.ads_utils as ads
from config import axon_suffix, myelin_suffix
from AxonDeepSeg.ads_utils import convert_path

from PIL import Image, ImageOps
from AxonDeepSeg.postprocessing import generate_axon_numbers_image
from config import axonmyelin_suffix

def launch_morphometrics_computation(path_img, path_prediction):
    """
    This function is equivalent to the morphometrics_extraction notebook of AxonDeepSeg.
    It automatically performs all steps (computations, savings, displays,...) of the
    morphometrics extraction of a given sample.
    :param path_img: path of the input image (microscopy sample)
    :param path_prediction: path of the segmented image (output of AxonDeepSeg)
    :return: none.
    """

    # If string, convert to Path objects
    path_img = convert_path(path_img)
    path_prediction = convert_path(path_prediction)

    try:
        # Read image
        img = ads.imread(path_img)

        # Read prediction
        pred = ads.imread(path_prediction)
    except (IOError, OSError) as e:
        print(("launch_morphometrics_computation: " + str(e)))
        raise
    else:

        # Get axon and myelin masks
        pred_axon = pred > 200
        pred_myelin = np.logical_and(pred >= 50, pred <= 200)

        # Get folder path
        path_folder = path_img.parent

        # Compute and save axon morphometrics
        stats_array = get_axon_morphometrics(pred_axon, path_folder)
        save_axon_morphometrics(path_folder, stats_array)

        # Generate and save displays of axon morphometrics
        fig = draw_axon_diameter(img, path_prediction, pred_axon, pred_myelin)
        save_map_of_axon_diameters(path_folder, fig)

        # Compute and save aggregate morphometrics
        aggregate_metrics = get_aggregate_morphometrics(
            pred_axon, pred_myelin, path_folder
        )
        write_aggregate_morphometrics(path_folder, aggregate_metrics)

def main(argv=None):
    ap = argparse.ArgumentParser(formatter_class=RawTextHelpFormatter)

    # Setting the arguments of the saving the morphometrics in excel file
    ap.add_argument('-s', '--sizepixel', required=False, help='Pixel size of the image(s) to compute morphometrics, in micrometers. \n' +
                                                              'If no pixel size is specified, a pixel_size_in_micrometer.txt \n' +
                                                              'file needs to be added to the image folder path. The pixel size \n' +
                                                              'in that file will be used for the morphometrics computation.',
                                                              default=None)

    ap.add_argument('-i', '--imgpath', required=True, nargs='+', help='Path to the image.')

    ap.add_argument('-f', '--filename', required=False,  help='Name of the excel file in which the morphometrics will be stored',
                                                              default="axon_morphometrics")

    # Processing the arguments
    args = vars(ap.parse_args(argv))
    path_target_list = [Path(p) for p in args["imgpath"]]
    filename = str(args["filename"])

    # Tuple of valid file extensions
    validExtensions = (
                        ".jpeg",
                        ".jpg",
                        ".tif",
                        ".tiff",
                        ".png"
                        )

    for current_path_target in path_target_list:
        if current_path_target.suffix.lower() in validExtensions:
            # load the axon mask
            if (Path(str(current_path_target.with_suffix("")) + str(axon_suffix))).exists():
                pred_axon = image.imread(str(current_path_target.with_suffix("")) + str(axon_suffix))
            else:
                print("ERROR: Segmented axon mask is not present in the image folder. ",
                              "Please check that the axon mask is located in the image folder. ",
                              "If it is not present, perform segmentation of the image first using ADS."
                      )
                sys.exit(3)

            # load myelin mask
            if (Path(str(current_path_target.with_suffix("")) + str(myelin_suffix))).exists():
                pred_myelin = image.imread(str(current_path_target.with_suffix("")) + str(myelin_suffix))
            else:
                print("ERROR: Segmented myelin mask is not present in the image folder. ",
                              "Please check that the myelin mask is located in the image folder. ",
                              "If it is not present, perform segmentation of the image first using ADS."
                      )
                sys.exit(3)

            if args["sizepixel"] is not None:
                psm = float(args["sizepixel"])
            else:  # Handle cases if no resolution is provided on the CLI

                # Check if a pixel size file exists, if so read it.
                if (current_path_target.parent / 'pixel_size_in_micrometer.txt').exists():

                    resolution_file = open(current_path_target.parent / 'pixel_size_in_micrometer.txt', 'r')

                    psm = float(resolution_file.read())
                else:

                    print("ERROR: No pixel size is provided, and there is no pixel_size_in_micrometer.txt file in image folder. ",
                                  "Please provide a pixel size (using argument -s), or add a pixel_size_in_micrometer.txt file ",
                                  "containing the pixel size value."
                          )
                    sys.exit(3)

            x = np.array([], dtype=[
                                        ('x0', 'f4'),
                                        ('y0', 'f4'),
                                        ('gratio', 'f4'),
                                        ('axon_area', 'f4'),
                                        ('axon_perimeter', 'f4'),
                                        ('myelin_area', 'f4'),
                                        ('axon_diam', 'f4'),
                                        ('myelin_thickness', 'f4'),
                                        ('axonmyelin_area', 'f4'),
                                        ('axonmyelin_perimeter', 'f4'),
                                        ('solidity', 'f4'),
                                        ('eccentricity', 'f4'),
                                        ('orientation', 'f4')
                                    ]
                         )

            # Compute statistics
            stats_array = get_axon_morphometrics(im_axon=pred_axon, im_myelin=pred_myelin, pixel_size=psm)

            for stats in stats_array:

                x = np.append(x, np.array(
                        [(
                            stats['x0'],
                            stats['y0'],
                            stats['gratio'],
                            stats['axon_area'],
                            stats['axon_perimeter'],
                            stats['myelin_area'],
                            stats['axon_diam'],
                            stats['myelin_thickness'],
                            stats['axonmyelin_area'],
                            stats['axonmyelin_perimeter'],
                            stats['solidity'],
                            stats['eccentricity'],
                            stats['orientation']
                        )],
                        dtype=x.dtype)
                    )

            # Generate numbers layers and png file with overlay

            mean_diameter_in_pixel = np.average(x['axon_diam']) / psm
            axon_indexes = np.arange(x.size)
            numbers = generate_axon_numbers_image(axon_indexes, x['x0'], x['y0'],
                                                                  tuple(reversed(pred_axon.shape)),
                                                                  mean_diameter_in_pixel)
            seg = Image.open(str(current_path_target.with_suffix("")) + str(axonmyelin_suffix))
            overlay = Image.fromarray(numbers)
            numbers_file = ImageOps.colorize(seg, black="black", white="blue", mid="red", blackpoint=0, whitepoint=255, midpoint=127)
            numbers_file.paste(overlay, (0, 0), overlay)
            numbers_file.save(current_path_target.parent / (str(current_path_target.with_suffix("")) + "_numbers_seg.png"),"PNG")

            # save the current contents in the file
            if not (filename.lower().endswith((".xlsx", ".csv"))):  # If the user didn't add the extension, add it here
                filename = filename + '.xlsx'
            try:
                # Export to excel
                if filename.endswith('.xlsx'):
                    pd.DataFrame(x).to_excel(current_path_target.parent / filename)
                # Export to csv
                else:
                    pd.DataFrame(x).to_csv(current_path_target.parent / filename)

                print(f"Moprhometrics file: {filename} has been saved in the {str(current_path_target.parent.absolute())} directory")
            except IOError:
                print("Cannot save morphometrics data in file '%s'." % filename)

        else:
            print("The path(s) specified is/are not image(s). Please update the input path(s) and try again.")
            break

    sys.exit(0)
# Calling the script
if __name__ == '__main__':
    main()

Let me know if you have any questions in regards to the script.

All right, I should be able to update this PR before Monday.

Thanks for the script, I pretty much copied the logic. You can check this PR again now.

In b4245e9, I've addressed the merge conflicts after merging #399.

The FSLeyes output and the naming of the files have been addressed.
However, this PR is still not ready for review.
TODO:

Write tests
Add docstring
Find out why current tests are not passing.

This PR is ready for review

@Stoyan-I-A sorry my comment was misleading, i suggest to not use capital letters as suffix

Functionality-wise, both the FLSeyes and CLI are working as expected, that is, the indices and indices_seg are saved after computing the morphometrics in the directory where morphometrics.csv file is saved. Thanks @Stoyan-I-A  for this contribution. This PR is good to go after review comments on this PR have been resolved.
One question:  In one of our meetings, we briefly discussed about having indices image saved or not, but I vaguely remember what did we discuss about it. In particular, did we decide to go ahead with saving indices image as well by default?

In one of our meetings, we briefly discussed about having indices image saved or not, but I vaguely remember what did we discuss about it. In particular, did we decide to go ahead with saving indices image as well by default?

As far as I remember, we were still not sure if want to keep both images or just the colored one, so I decided to keep both for the moment.

I need to add two new images to the __test_files__. I can provide them right now or I can wait for #533 to be merged and add them myself.

I believe #533 is only related to models and https://github.com/axondeepseg/data-testing would be where __test_ files__ where these extra 2 images need to added. @hermancollin correct me if I'm wrong : the test files link in the ADS repo hasn't been changed right in the PR #533?

I believe #533 is only related to models and https://github.com/axondeepseg/data-testing would be where __test_ files__ are present, where these extra 2 images needs to added. @hermancollin correct me if I'm wrong : the test files link in the ADS repo hasn't been changed in the PR #533?

That is correct, this PR is for models only. @Stoyan-I-A if you want to add files to the testing folder, you can directly commit to the master branch here: https://github.com/axondeepseg/data-testing (or I guess the "correct" way would be to start a branch, then do a PR).

Also, we need to open another PR to update the link of __test_files__ in ADS and link it to upload test files on  https://github.com/axondeepseg/data-testing

Also, we need to open another PR to update the link of __test_files__ in ADS and link it to uploaded test files on https://github.com/axondeepseg/data-testing

@Stoyan-I-A I've created PR #534 for the same. If you wish to upload the 2 additional images, you can create a PR in https://github.com/axondeepseg/data-testing repo or if you want me to upload them, I can do the same.

@jcohenadad I requested a review because GitHub doesn't let me merge since you have made a change request a few weeks ago. Those change have been made however I can't mark them as such.


Fix typo ('datbase' instead of 'database')

When creating a new PR, please do the following and delete this template text:

Reference the issue number if there is one:
Fixes #Issue_Number

The "Fixes #nnn" syntax in the PR description causes
GitHub to automatically close the issue when this PR is merged.

Suggest reviewers if you know who should review the PR, using the GitHub Reviewers UI.

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

small

Yes.

(but important)

Not really. But thanks anyway, Pull Request Bot.

Maybe it was all about datbase?

@ExplodingCabbage  Thanks!


Indent v. 1.0

Dare I say v. 1.0 of the indent code is done?
I created a livecode-indent.tmpreferences file, and a livecode-indent regex rationale file.  The second exists to go into detail as to the thinking in the regex's, because there will soon be a time when someone wants to understand what all that means, and why it was written that way.
A couple of issues:

ST is case-sensitive - "command" and "COMMAND" are not the same thing.  This version assumes, lowercase
There are some minor issues with the ST auto-indenter.  I don't know if they're bugs in the indenter, or in my regex, but I think they're in the indenter, because if I add or remove tabs from the lines that the indenter treats weirdly, then re-indent, their tab level does not change.

@macMikey I'm testing this out on my machine. I'm going to use it for a couple of days to see if I find any issues.

@macMikey When I type out a switch statement the indentation looks like this:
switch the short name of the target
case "Reply"

  break
case "Edit"

  break
end switch

The indentation I use (and which LiveCode uses) is this:
switch the short name of the target
    case "Reply"

      break
    case "Edit"

      break
  end switch

Is reducing the indent of case "" intentional?

It is intentional.
You cannot tell ST to back-indent (or fore-indent, for that matter) more than one tab (AFAIK).  So, structures like switch/case, which will leave you in two levels, can't get back to where they started.  The only solution is to not indent after the switch, so that the end switch is at the same level as the switch.

@macMikey This has now been merged in.


DataCapture, restrict the user from accepting value after decimal point in integer textfield widget.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Datacapture support decimal and integer type.
For decimal and integer input field, soft keyboard is shown with decimal point.
As decimal input textfield already allows user to provide value after decimal point, same should not be happened when type is integer(integer input edit text view).
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Would you like to work on the issue?
Please state if this issue should be assigned to you or who you think could help to solve this issue.

if this issue is not resolved, please assign it to me.

@samiali12 thanks for showing interest, let me connect you with @jingtang10

Marking it as fixed, if user enters decimal point then it starts showing error.


spec: Allow specifying service schema names & encodings

Public-Facing Changes
spec: Allow specifying service schema names & encodings
Description
Allows for customization of service schema names and schema encodings.
Partially fixes https://github.com/foxglove/studio/issues/6970
Partially resolves FG-5280

Nice, LGTM

Awesome! Thank you for adding the descriptions. This is all clear to me now.


Publish 05/21/2021, 10:30 AM

Docs Build status updates of commit 07d5380:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/Get-ActiveSyncDevice.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


@overload the guidance decorator in order to improve static type-checking

Improve annotations to the guidance decorator via typing.overload.

Allow static type checkers to infer the return types of guidance decorated functions (GrammarFunction, Rawfunction or the Union of the two for stateless True, False and Callable, respectively)
Allow static type checkers to properly infer the arguments (and their types) of guidance-decorated functions. If the function being decorated has typed arguments, these types will now be preserved and enforced on the decorated function, greatly improving type safety.

Example:
Before

Note: despite claiming knowledge of the types of x and y, mypy treats these args as having type Any and will not complain if bad types are passed
After

lm is correctly dropped from the inferred signature, argument types are properly enforced
Note that I had to add a few minor changes elsewhere in the codebase because mypy got slightly more aggressive after its type inference was improved by the main body of this PR.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 83.78378% with 6 lines in your changes missing coverage. Please review.

Project coverage is 58.51%. Comparing base (c0a0bfb) to head (c45d1e4).

Files with missing lines
Patch %
Lines

guidance/_guidance.py
79.31%
6 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

:exclamation:  There is a different number of reports uploaded between BASE (c0a0bfb) and HEAD (c45d1e4). Click for more details.
HEAD has 27 uploads less than BASE

Flag
BASE (c0a0bfb)
HEAD (c45d1e4)

28
1

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #1014       +/-   ##
===========================================
- Coverage   70.14%   58.51%   -11.63%
===========================================
  Files          62       62
  Lines        4441     4467       +26
===========================================
- Hits         3115     2614      -501
- Misses       1326     1853      +527

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Fix broken Puppeteer upgrade to version 13.0.0

The Puppeteer development dependency is now pinned to version 12.0.1 because version 13.0.0 fails for (to us) unknown reasons since nothing really stands out in the 13.0.0 changelogs. It's strange because everything about the new release works, except for one integration test ("must change colors") on one platform (Windows) with the following traceback:
Failures:
1) Interaction in js-colors.pdf must change colors
  Message:
    ProtocolError: Protocol error (Runtime.callFunctionOn): Debugger: expected Debugger.Object, got Proxy _returnError@chrome://remote/content/cdp/domains/content/runtime/ExecutionContext.jsm:185:22
    callFunctionOn@chrome://remote/content/cdp/domains/content/runtime/ExecutionContext.jsm:260:23

  Stack:
    error properties: Object({ originalMessage: 'Debugger: expected Debugger.Object, got Proxy' })
    ProtocolError: Protocol error (Runtime.callFunctionOn): Debugger: expected Debugger.Object, got Proxy _returnError@chrome://remote/content/cdp/domains/content/runtime/ExecutionContext.jsm:185:22
    callFunctionOn@chrome://remote/content/cdp/domains/content/runtime/ExecutionContext.jsm:260:23
        at C:\pdfjs\botio-files-pdfjs\private\6f70ec10008fe96\node_modules\puppeteer\lib\cjs\puppeteer\common\Connection.js:226:24
        at new Promise (<anonymous>)
        at CDPSession.send (C:\pdfjs\botio-files-pdfjs\private\6f70ec10008fe96\node_modules\puppeteer\lib\cjs\puppeteer\common\Connection.js:222:16)
        at ExecutionContext._evaluateInternal (C:\pdfjs\botio-files-pdfjs\private\6f70ec10008fe96\node_modules\puppeteer\lib\cjs\puppeteer\common\ExecutionContext.js:204:50)
        at ExecutionContext.evaluateHandle (C:\pdfjs\botio-files-pdfjs\private\6f70ec10008fe96\node_modules\puppeteer\lib\cjs\puppeteer\common\ExecutionContext.js:155:21)
        at WaitTask.rerun (C:\pdfjs\botio-files-pdfjs\private\6f70ec10008fe96\node_modules\puppeteer\lib\cjs\puppeteer\common\DOMWorld.js:547:41)
        at runMicrotasks (<anonymous>)
        at processTicksAndRejections (internal/process/task_queues.js:95:5)

Maybe someone with Windows can try to reproduce this and find out what is special about the "must change colors" test that triggers this. Once we know what causes it, we can either fix it, or work around it and file an upstream bug report.

Now that the integration tests on Linux work again, I found that the same test permafails on Linux as well with Puppeteer 13, but fortunately with a much more helpful stack trace:
1) Interaction in js-colors.pdf must change colors
  Message:
    Error: Evaluation failed: DOMException: Failed to execute 'querySelector' on 'Document': '[object HTMLDocument]' is not a valid selector.
        at eval (eval at waitForPredicatePageFunction (:3:23), <anonymous>:4:45)
        at eval (eval at waitForPredicatePageFunction (:3:23), <anonymous>:5:28)
        at onRaf (__puppeteer_evaluation_script__:50:35)
        at pollRaf (__puppeteer_evaluation_script__:43:15)
        at waitForPredicatePageFunction (__puppeteer_evaluation_script__:8:22)
  Stack:
    Error: Evaluation failed: DOMException: Failed to execute 'querySelector' on 'Document': '[object HTMLDocument]' is not a valid selector.
        at eval (eval at waitForPredicatePageFunction (:3:23), <anonymous>:4:45)
        at eval (eval at waitForPredicatePageFunction (:3:23), <anonymous>:5:28)
        at onRaf (__puppeteer_evaluation_script__:50:35)
        at pollRaf (__puppeteer_evaluation_script__:43:15)
        at waitForPredicatePageFunction (__puppeteer_evaluation_script__:8:22)
        at ExecutionContext._evaluateInternal (/home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
        at runMicrotasks (<anonymous>)
        at processTicksAndRejections (node:internal/process/task_queues:96:5)
        at async WaitTask.rerun (/home/timvandermeij/Documenten/Ontwikkeling/pdf.js/Code/node_modules/puppeteer/lib/cjs/puppeteer/common/DOMWorld.js:547:27)

Given this information I'm still on the fence if this is actually a Puppeteer 13 bug, or just a mistake in our test that just happens to surface in Puppeteer 13...

If I remove https://github.com/mozilla/pdf.js/blob/master/test/integration/scripting_spec.js#L699-L707 the test passes under Puppeteer 13. I see this way added in https://github.com/mozilla/pdf.js/commit/17363bbc6fa46f04bf308a23443b931c85856ff2 to fix another issue. @calixteman Does this ring a bell for you on how we can fix this issue?

Lo and behold: _ref in the code fragment above actually refers to the document object now in Puppeteer 13. It looks like the arguments are shifted because the _current argument (the next one) now refers to what _ref used to refer to. At the very least we now know something changed in Puppeteer and looking at the documentation this seems like a bug.

I'll try to reduce this for an upstream bug since the documentation doesn't indicate this behavior and the changelogs also don't mention any changes here, so I strongly suspect an upstream regression. Fortunately I do think we can fix it on our side too; I'll try that out later today.

I have created a minimal reproducer and filed https://github.com/puppeteer/puppeteer/issues/7836 upstream since this is in fact a regression in Puppeteer 13.

Despite us fixing this on our side, it looks like this also got fixed upsteam now in https://github.com/puppeteer/puppeteer/pull/7845, so hopefully in a next Puppeteer release this won't be a problem anymore for new code.


Specifying specific database connection

Not sure if there is a way that i am missing but it would be beneficial to be able to specify a specific database connection so that we can utilise for example our readonly mysql servers for hydration of elasticsearch results.

Hey @spiritinlife, there is refineModels method in the elastic-scout-driver-plus package that you can use to set different connections, e.g.:
$models = Book::searchQuery($query)
   ->refineModels(function (EloquentBuilder $query) {
       $query->setConnection('my_connection');
   })
   ->execute()
   ->models();

Would that resolve your issue?

Hey @babenkoivan,
That could work but laravel query builder doesn't allow to change the connection easily. I am digging into it to see if there are any viable workarounds and revert.

Hey @spiritinlife, sorry for the late reply. I think my initial code example was misleading because setConnection belongs to the model and not to the query builder. I need to think of a better approach and implement it in the package.
Meanwhile, you can try to switch the connection as follows:
$models = Book::searchQuery($query)
   ->refineModels(function (EloquentBuilder $builder) {
       $conn = DB::connection('my_connection');
       $query = $builder->getQuery();
       $query->connection = $conn;
       $query->grammar = $conn->query()->getGrammar();
       $query->processor = $conn->query()->getProcessor();
   })
   ->execute()
   ->models();

I didn't try it, so not sure if this would work.

Hey @babenkoivan ,
Thank you for providing the snipper. Indeed that works in my tests although it doesn't work for relationships when you do the eager loading through the search query. I am glad you reached the same conclusion that a bigger change is required as i was also trying to find the time to assist on this.
From the consumer of the package point of view imho we should provide an override on the model for the database connection related to hydrating the elastic results and if not given it should default to the default model's connection.
I was able to have a proof of concept by overriding the LazyModelFactory and setting the connection like so in mapModels. This solves the issue with the relationships in my tests.

Indeed it's not the latest. Unfortunately I am not able to upgrade to latest because our elastic deployment is at 7 and cannot be easily upgraded to 8 due to plugin dependencies.
I wanted to ask you on that also if there is any reason that the latest versions of the package don't support elastic 7.

Can you please tell me what versions of packages you have (driver and driver plus)?
If I remember correctly, there were some breaking changes in ES 8 and I always focus on maintaining the latest versions due to time/capacity limitations.

babenkoivan/elastic-adapter: v2.4.0
babenkoivan/elastic-client: v1.2.0
babenkoivan/elastic-scout-driver: v2.0.0
babenkoivan/elastic-scout-driver-plus: v3.5.1

Hey @spiritinlife, I spent some time drafting a solution but concluded that such a feature must be implemented on the Laravel Scout side. If I add something like this in the Searchable trait in the elastic-scout-driver-plus:
trait Searchable
{
    // you can override this method in your model to define a connection that will be used when mapping search results to models
    public function scoutConnection(): ?string
    {
         return $this-connection;
    }
}

then it works perfectly fine with Model::searchQuery() but not with Model::search(), because the latter uses the standard functionality of the Scout package. I always try to keep elastic-scout-driver as close as possible to the original Scout package and avoid changing its behavior.
If I add something like this to the query builder, which limits the usage to the elastic-scout-driver-plus package:
$result = Model::searchQuery()->scoutConnection('readonly');

then it solves the issue, but it's cumbersome cause normally, you would want to define it every time you do a search request.
This all feels a bit off because this feature should live in the Scout package and be supported by Laravel.

Now, let's talk about the options you currently have:

Read-only models. Let's say you have a model User, now you want to use readonly connection when searching users. You can extend User as follows:

class ReadonlyUser extends User
{
    protected $table = 'users';
    // alternatively, you can override getConnectionName() method if you need to define some logic
    protected $connection = 'readonly';
}

then when you want to make sure readonly connection is used for queries, you can search using the new model:
$result = ReadonlyUser::searchQuery();

Use refineModels as suggested here https://github.com/babenkoivan/elastic-scout-driver/issues/64#issuecomment-2355581162. It is cumbersome, but if you extract it to a function it will be readable and reusable.

Fork the repository and apply the changes you need. It would be really difficult to extend/override existing functionality without a fork because many methods are private.

Create a feature request in https://github.com/laravel/scout.

I hope this helps and at least one of the solutions above works for you 🙂

Hey 👋
Thank you for taking time to look into this.
Indeed that sounds like something it should be done in scout first.


Test 1.8 is WARN on docker.service, but auditing is enabled.

# Docker Bench for Security v1.3.4
[PASS] 1.5  - Ensure auditing is configured for the Docker daemon
[PASS] 1.6  - Ensure auditing is configured for Docker files and directories - /var/lib/docker
[PASS] 1.7  - Ensure auditing is configured for Docker files and directories - /etc/docker
[WARN] 1.8  - Ensure auditing is configured for Docker files and directories - docker.service
[WARN] 1.9  - Ensure auditing is configured for Docker files and directories - docker.socket

The files are in the right place, auditing is enabled.
# ls -la /lib/systemd/system/docker.service
-rw-r--r-- 1 root root 1185 Mar  3 16:51 /lib/systemd/system/docker.service

# sudo auditctl -l
-w /usr/bin/docker -p wa
-w /var/lib/docker -p wa
-w /etc/docker -p wa
-w /lib/systemd/system/docker.service -p wa
-w /lib/systemd/system/docker.socket -p wa
-w /etc/default/docker -p wa
-w /etc/docker/daemon.json -p wa
-w /usr/bin/docker-containerd -p wa
-w /usr/bin/docker-runc -p wa

I can't see why this test is failing. I'm passing through /lib/systemd/system (this is Arch linux, so following the Ubuntu receipe)
# sudo docker run --rm --net host --pid host --userns host --cap-add audit_control \
    -e DOCKER_CONTENT_TRUST=$DOCKER_CONTENT_TRUST \
    -v /etc:/etc:ro \
    -v /lib/systemd/system:/lib/systemd/system:ro \
    -v /usr/bin/containerd:/usr/bin/containerd:ro \
    -v /usr/bin/runc:/usr/bin/runc:ro \
    -v /usr/lib/systemd:/usr/lib/systemd:ro \
    -v /var/lib:/var/lib:ro \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    --label docker_bench_security \
    docker/docker-bench-security

# uname -a
Linux ripper 5.11.2-arch1-1 #1 SMP PREEMPT Fri, 26 Feb 2021 18:26:41 +0000 x86_64 GNU/Linux

Running directly on the host without docker I get a pass:
# source helper_lib.sh
# source 1_host_configuration.sh
# source output_lib.sh
# check_1_2_6
tee: '': No such file or directory
[PASS] 1.2.6  - Ensure auditing is configured for Docker files and directories - docker.service (Scored)

So I'm obviously doing something wrong, but what?

Hi @paulhargreaves, it seems you're using the Docker container, can you reproduce this error with the script as well?

I haven't ran the full script, but when I just source in that one test (per above) it passes, so it's definitely something about it running in the container, I just don't know where to start :-/

No need, I can help you with that. It's the classic https://github.com/docker/docker-bench-security/issues/405 from 2019.

Ah, sigh, ok thanks. I'll close and stop running it as a container.


when opening the webui, the undersampling pre-search step is invalid

Please make sure that this is a bug.
System information

OS Platform: linux
Python version:3.9
HyperGBM version: 0.3 or <IP_ADDRESS>
hboard: 0.1.1

Describe the current behavior
if make_experiment() set down_sample_search=True and webui=True, hypernets will report the following warning, which results in invalid search steps.
10-26 11:32:31 W hypernets.e.init.py 1463 - Exception: Unseen Step class SpaceSearchWithDownSampleStep
Traceback (most recent call last):
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hypernets/experiment/_experiment.py", line 86, in run
callback.experiment_start(self)
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hboard/callbacks.py", line 125, in experiment_start
super(WebVisExperimentCallback, self).experiment_start(exp)
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hypernets/experiment/_callback.py", line 533, in experiment_start
d = ExperimentExtractor(exp).extract()
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hypernets/experiment/_extractor.py", line 670, in extract
steps_meta = [self.extract_step(i, step) for i, step in enumerate(exp.steps)]
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hypernets/experiment/_extractor.py", line 670, in
steps_meta = [self.extract_step(i, step) for i, step in enumerate(exp.steps)]
File "/home/jiapj/miniconda3/envs/gbm_venv_3.9/lib/python3.9/site-packages/hypernets/experiment/_extractor.py", line 633, in extract_step
raise Exception(f"Unseen Step class {step_type} ")

bug已定位
文件路径：hypernets\experiment_extractor.py
在extractors中添加'SpaceSearchWithDownSampleStep': SpaceSearchStepExtractor。
如下所示：
extractors = {
StepType.DataAdapting: DataAdaptionStepExtractor,
StepType.DataCleaning: DataCleanStepExtractor,
StepType.FeatureGeneration: FeatureGenerationStepExtractor,
StepType.DriftDetection: DriftStepExtractor,
StepType.FeatureSelection: FeatureSelectionStepExtractor,
StepType.CollinearityDetection: MultiLinearityStepExtractor,
StepType.PseudoLabeling: PseudoStepExtractor,
StepType.DaskPseudoLabelStep: PseudoStepExtractor,
StepType.PermutationImportanceSelection: PermutationImportanceStepExtractor,
StepType.SpaceSearch: SpaceSearchStepExtractor,
'SpaceSearchWithDownSampleStep': SpaceSearchStepExtractor,
StepType.FinalTrain: FinalTrainStepExtractor,
StepType.Ensemble: EnsembleStepExtractor,
StepType.DaskEnsembleStep: EnsembleStepExtractor
}


Dockerfile COPY section needs update

data dir is now underneath ./morpheus
Relates to #62

rerun tests

@gpucibot test

ok to test


addons: Response object is too long.

Describe the bug
I can't deploy the blueprint when adding the Nginx addon (not sure it is related to Nginx...)
Expected Behavior
I expect the eks blueprint to be able to split the cloudformation template otherwise its usage is very limited
Current Behavior
Response object is too long. is thrown and stack is getting deleted.
Reproduction Steps
#!/usr/bin/env node
import "source-map-support/register";
import * as cdk from "aws-cdk-lib";
import * as blueprints from "@aws-quickstart/eks-blueprints";
import { ArgoCDAddOnProps } from "@aws-quickstart/eks-blueprints";
import { KubernetesVersion, NodegroupAmiType } from "aws-cdk-lib/aws-eks";
import { TeamDatasciencePreprod, TeamDatascienceProd } from "../lib/teams";
import { InstanceType } from "aws-cdk-lib/aws-ec2";

//Secrets required in Secrets Manager : "argocd/repo/ssh_key" "argocd/admin_password"

const app = new cdk.App();

// ---- CONSTANTS ----
const clusterName = "clustername";
export const account = "000000000";
const region = "eu-west-3";
const domainAWSID = "Route53ZoneID"
const subdomain = 'my.domain.com'

const boostrapRepoUrl =
  "git@github.com:team/repo.git";

// ---- KARPENTER ----
const karpenterAddonProps = {
  provisionerSpecs: {
    "kubernetes.io/arch": ["arm64"],
  },
  subnetTags: {
    Name: `${clusterName}/${clusterName}-vpc/Priv1`,
  },
  securityGroupTags: {
    [`kubernetes.io/cluster/${clusterName}`]: "owned",
  },
};

// ---- ArgoCD ----
const argoCDAddonProps: ArgoCDAddOnProps = {
  bootstrapRepo: {
    repoUrl: boostrapRepoUrl,
    credentialsSecretName: "argocd-repo-ssh_key", //Not working :(
    credentialsType: "SSH",
    path: "main",
  },
  adminPasswordSecretName: "argocd/admin_password",
};

// ---- Default Instance ----
const clusterProvider = new blueprints.MngClusterProvider({
  version: KubernetesVersion.V1_25,
  desiredSize: 1,
  instanceTypes: [new InstanceType("t4g.medium")],
  diskSize: 50,
  amiType: NodegroupAmiType.BOTTLEROCKET_ARM_64
});

// ---- Teams ----
const teams: Array<blueprints.Team> = [
  new TeamDatasciencePreprod(app),
  new TeamDatascienceProd(app),
];

// ---- Full EKS cluster ----
const addOns: Array<blueprints.ClusterAddOn> = [
  new blueprints.addons.SecretsStoreAddOn(),
  new blueprints.addons.AwsLoadBalancerControllerAddOn(),
  new blueprints.addons.NginxAddOn({externalDnsHostname:subdomain, certificateResourceName: "kscert"}),
  new blueprints.addons.ArgoCDAddOn(argoCDAddonProps),
  new blueprints.addons.MetricsServerAddOn(),
  new blueprints.addons.VpcCniAddOn(),
  new blueprints.addons.KarpenterAddOn(karpenterAddonProps),
  new blueprints.addons.ExternalDnsAddOn({
    hostedZoneResources:["kszone"],
  })
];

blueprints.EksBlueprint.builder()
  .account(account)
  .region(region)
  .resourceProvider("kszone", new blueprints.ImportHostedZoneProvider(domainAWSID, subdomain))
  .resourceProvider("kscert", new blueprints.CreateCertificateProvider('wildcert', `*.${subdomain}`, "kszone"))
  .addOns(...addOns)
  .clusterProvider(clusterProvider)
  .teams(...teams)
  .build(app, clusterName);

lib/teams.ts
import { ApplicationTeam } from "@aws-quickstart/eks-blueprints";
import { App } from "aws-cdk-lib";
import { account } from "../bin/eks";
import { ArnPrincipal } from "aws-cdk-lib/aws-iam";

export class TeamDatasciencePreprod extends ApplicationTeam {
  constructor(app: App) {
    super({
      name: "datascience-preprod",
      users: [new ArnPrincipal(`arn:aws:iam::${account}:user/test`)],
    });
  }
}

export class TeamDatascienceProd extends ApplicationTeam {
  constructor(app: App) {
    super({
      name: "datascience-prod",
      users: [new ArnPrincipal(`arn:aws:iam::${account}:user/test`)],
    });
  }
}

Possible Solution
Split in multiple Stack ? No documentation about how to do that with the blueprint
Additional Information/Context
No response
CDK CLI Version
2.76.0 (build 78c411b)
EKS Blueprints Version
No response
Node.js Version
18.14.0
Environment details (OS name and version, etc.)
Mac OS
Other information
No response

@cyril94440 We have nginx pattern (running atm) with Route53 and ACM integration working in our patterns here.
Do you mind providing the full stack of the error?
The full blueprint that you have is referencing teams that we don't have locally to reproduce.

@shapirov103  I've added the teams definition.
From your pattern, does buildAsync help split the cloudformation template in multiple files?
The other issue I am encountering with Nginx is the following:
Error: values don\'t meet the specifications of the schema(s) in the following chart(s):\nnginx-ingress:\n- controller.service.httpPort.enable: Invalid type. Expected: boolean, given: string\n- controller.service.httpsPort.targetPort: Invalid type. Expected: integer, given: string\n\n'
And it looks like I am not the only one: https://stackoverflow.com/questions/75778139/aws-cdk-eks-blueprint-fails-with-error-invalid-type/76175704#76175704
I'll take a look at the pattern you provided and try to get as close as possible from its definition.
I really appreciate your help.
Thanks!

@cyril94440 thank you for bringing it up to our attention. Initial look: appears to be a regression issue from the chart upgrade. This will be investigated and solved shortly.

@shapirov103 ok, any way to choose a stable previous version?
Also what about the "Response object is too long." how to overcome the cloudformation template size issue with eks blueprint?
Thanks

I am reproducing the nginx issue now. I cannot pinpoint exactly the release, but I see that in 1.6.0 and 1.5.3 there was a version change in the nginx helm chart. You can track this through https://github.com/aws-quickstart/cdk-eks-blueprints/releases (generally there is a manifest with addon versions for each release).
With respect to the response object too long: I never saw this issue before, regardless of how big the pattern was. Are you sending large chunks of data with the blueprint with your teams?

Ok, I will try with a previous blueprint version.
I am not sure to understand your question but as you can see the team definition is very simple and I am not doing anything else.
Here are some informations that might help you:
Here is the output of the cdk deploy:

The lambda function cluster-awscdkawseksClust-ProviderframeworkisCompl-ID is emitting an error:
2023-05-05T15:55:42.955Z	df0c9041-4e05-4fb9-8efb-43065d40d9e1	ERROR	Invoke Error 	{
    "errorType": "Error",
    "errorMessage": "{\"RequestType\":\"Create\",\"ServiceToken\":\"arn:aws:lambda:eu-west-1:009032792877:function:udiniv2-awscdkawseksClust-ProviderframeworkonEvent-Clq544FUaJ94\",\"ResponseURL\":\"https://cloudformation-custom-resource-response-euwest1.s3-eu-west-1.amazonaws.com/arn%3Aaws%3Acloudformation%3Aeu-west-1%3A009032792877%3Astack/udiniv2/cf4d3bd0-eb5b-11ed-b86f-0290e7d204db%7Cudiniv2F9EAC977%7Cf5858173-784d-4f65-8ca5-366e4e890b2b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230505T155129Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=AKIAU7SEXKRM6LZ53QKY%2F20230505%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=67a14852beb531707cbc723ee06b9f858e11a0f5f31b6895af9708d0068f7cff\",\"StackId\":\"arn:aws:cloudformation:eu-west-1:009032792877:stack/udiniv2/cf4d3bd0-eb5b-11ed-b86f-0290e7d204db\",\"RequestId\":\"f5858173-784d-4f65-8ca5-366e4e890b2b\",\"LogicalResourceId\":\"udiniv2F9EAC977\",\"ResourceType\":\"Custom::AWSCDK-EKS-Cluster\",\"ResourceProperties\":{\"ServiceToken\":\"arn:aws:lambda:eu-west-1:009032792877:function:udiniv2-awscdkawseksClust-ProviderframeworkonEvent-Clq544FUaJ94\",\"Config\":{\"encryptionConfig\":[{\"provider\":{\"keyArn\":\"arn:aws:kms:eu-west-1:009032792877:key/97c8a0ac-e464-4066-ae10-86f7738306c8\"},\"resources\":[\"secrets\"]}],\"resourcesVpcConfig\":{\"endpointPrivateAccess\":\"true\",\"securityGroupIds\":[\"sg-0d5d38a16d8efc7d7\"],\"endpointPublicAccess\":\"true\",\"subnetIds\":[\"subnet-05057a31990d956b2\",\"subnet-0a788b272fb5e0124\",\"subnet-033049c8dc53e52d9\",\"subnet-0ec6b39361635c6df\",\"subnet-053e6c0477130ee28\",\"subnet-08720f7c9be1919ed\"]},\"roleArn\":\"arn:aws:iam::009032792877:role/udiniv2-udiniv2Role23F9DEAA-TTWVXGVIEC1B\",\"name\":\"udiniv2\",\"version\":\"1.25\"},\"AssumeRoleArn\":\"arn:aws:iam::009032792877:role/udiniv2-udiniv2CreationRole3D2A06E4-EHFG5BBS1ORS\",\"AttributesRevision\":\"2\"},\"PhysicalResourceId\":\"udiniv2\"}",
    "stack": [
        "Error: {\"RequestType\":\"Create\",\"ServiceToken\":\"arn:aws:lambda:eu-west-1:009032792877:function:udiniv2-awscdkawseksClust-ProviderframeworkonEvent-Clq544FUaJ94\",\"ResponseURL\":\"https://cloudformation-custom-resource-response-euwest1.s3-eu-west-1.amazonaws.com/arn%3Aaws%3Acloudformation%3Aeu-west-1%3A009032792877%3Astack/udiniv2/cf4d3bd0-eb5b-11ed-b86f-0290e7d204db%7Cudiniv2F9EAC977%7Cf5858173-784d-4f65-8ca5-366e4e890b2b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20230505T155129Z&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credential=AKIAU7SEXKRM6LZ53QKY%2F20230505%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=67a14852beb531707cbc723ee06b9f858e11a0f5f31b6895af9708d0068f7cff\",\"StackId\":\"arn:aws:cloudformation:eu-west-1:009032792877:stack/udiniv2/cf4d3bd0-eb5b-11ed-b86f-0290e7d204db\",\"RequestId\":\"f5858173-784d-4f65-8ca5-366e4e890b2b\",\"LogicalResourceId\":\"udiniv2F9EAC977\",\"ResourceType\":\"Custom::AWSCDK-EKS-Cluster\",\"ResourceProperties\":{\"ServiceToken\":\"arn:aws:lambda:eu-west-1:009032792877:function:udiniv2-awscdkawseksClust-ProviderframeworkonEvent-Clq544FUaJ94\",\"Config\":{\"encryptionConfig\":[{\"provider\":{\"keyArn\":\"arn:aws:kms:eu-west-1:009032792877:key/97c8a0ac-e464-4066-ae10-86f7738306c8\"},\"resources\":[\"secrets\"]}],\"resourcesVpcConfig\":{\"endpointPrivateAccess\":\"true\",\"securityGroupIds\":[\"sg-0d5d38a16d8efc7d7\"],\"endpointPublicAccess\":\"true\",\"subnetIds\":[\"subnet-05057a31990d956b2\",\"subnet-0a788b272fb5e0124\",\"subnet-033049c8dc53e52d9\",\"subnet-0ec6b39361635c6df\",\"subnet-053e6c0477130ee28\",\"subnet-08720f7c9be1919ed\"]},\"roleArn\":\"arn:aws:iam::009032792877:role/udiniv2-udiniv2Role23F9DEAA-TTWVXGVIEC1B\",\"name\":\"udiniv2\",\"version\":\"1.25\"},\"AssumeRoleArn\":\"arn:aws:iam::009032792877:role/udiniv2-udiniv2CreationRole3D2A06E4-EHFG5BBS1ORS\",\"AttributesRevision\":\"2\"},\"PhysicalResourceId\":\"udiniv2\"}",
        "    at isComplete (/var/task/framework.js:55:15)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
        "    at async Runtime.handler (/var/task/cfn-response.js:52:13)"
    ]
}

The CFN Object Too Long maybe a side effect of the error on the NGINX side. This will be out shortly with 1.7.3 (#677 ), I suggest you retest it then.
I saw similar issues against CDK, e.g. this one.

Please try 1.7.3 which is out. I anticipate it to fix the issues.

Hi @shapirov103  It is working perfectly now! Thank you for your help.


Remove unnecessary word in sentence.

Summary
removed unnecessary "we" in sentence.

@hmhealey This documentation is and was really helpful during onboarding.  I made some notes when I first started that are very much nits, but figure I'll submit them now :)


[Bug]: Jest-Runner 28.0.0 no longer exports runTest()

Version
28.0.0
Steps to reproduce
Try to import runTest from jest-runner.
Expected behavior
The function should be imported.
Actual behavior
I get the following error:
Uncaught Error: Package subpath './build/runTest' is not defined by "exports" in /home/jmarcou/dev/jest-electron/node_modules/jest-runner/package.json

Additional context
With Jest 27.0.0 I was able to import the runTest function with import runTest from 'jest-runner/build/runTest';.
But since Jest 28.0.0 the runTest.d.ts file is missing from the jest-runner package, and I cannot import the runTest function.
Environment
System:
    OS: Linux 5.10 Debian GNU/Linux 10 (buster) 10 (buster)
    CPU: (8) x64 Intel(R) Core(TM) i7-4712HQ CPU @ 2.30GHz
  Binaries:
    Node: 18.6.0 - ~/.nvm/versions/node/v18.6.0/bin/node
    npm: 8.14.0 - ~/.nvm/versions/node/v18.6.0/bin/npm
  npmPackages:
    jest: ^28.1.0 => 28.1.3

What's the use case for using runTest directly instead of the exported class?

Sorry for the delay 😅
The runTest function is used by this package: https://www.npmjs.com/package/jest-electron
You can see the import here: https://github.com/hustcc/jest-electron/blob/b5887476eb639549c9317dd26b839d961ac58573/src/electron/renderer/uitl.ts#L7
I'm not sure how I would do the same by using the exported class

Can't it use https://www.npmjs.com/package/create-jest-runner?


Not handling failed tests

I have the following TAP output
1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Summarized correctly # TODO Not written yet

And using 1.2.1.
This is the output I get:
##teamcity[testStarted name='Input file opened' captureStandardOutput='true']
##teamcity[testFinished name='Input file opened' duration='1']
##teamcity[testStarted name='First line of the input valid' captureStandardOutput='true']
##teamcity[testStarted name='Read the rest of the file' captureStandardOutput='true']
##teamcity[testFinished name='Read the rest of the file' duration='1']
##teamcity[testStarted name='Summarized correctly # TODO Not written yet' captureStandardOutput='true']

Which obviously does not highlight that any of the tests have failed. And that TAP input is totally valid, as it is from the TAP site itself.

Hi @Poetro, thanks for reporting this! Although I’ve stopped using TeamCity and no longer actively develop this project, I’d happily review and merge a PR that resolves this issue.
If you’d rather use actively-developed projects, you might want to explore tap-mocha-reporter together with mocha-teamcity-reporter.


WIP: Introduce kind provider

The bare minimum are in place for LoadProvider with some basic tests (that should pass).
Opted for structs of coreKinds, composableKinds and customKinds to basically allow splitting schemas into multiple files for a provider more easily, i.e. coreKinds: Foo and coreKinds: Bar  vs coreKinds: [ Foo, Bar ]. Maybe just a detail/nice to have for defining the test providers?
Notes:

Had to upgrade thema to get latest changes regarding lineage.#LatestVersion

cc @AgnesToulet

@AgnesToulet

In which provider should live the schemas shared by the plugins defined in: https://github.com/grafana/grafana/tree/main/packages/grafana-schema/src/common? If the answer is grafana, this doesn't fit with the file structure expected in this PR.

Great question, not something I've considered or heard any discussions of. For reference, I did ask some related question to these types. I'll share the link in DM with you. Given https://github.com/grafana/grafonnet I imagine they already include generated jsonnet of these common types somehow so question is if they need to be in the registry?

What should happen if the providers have more metadata than the ones defined here? I think they should still be published in the kind/provider registry, I don't know if it would have an impact on this PR.
provider.go

Yes, that's the idea. That's why I've considered to move the provider metadata into a struct to allow some kind of "namespacing" of fields and to allow provider to have more metadata included without colliding with the existing provider fields/properties. In general this line
https://github.com/grafana/kindsys/blob/e69c9d94cbfc8c256c944bddcaee8bc57f848a4a/provider.cue#L31 makes the Provider open for extension and allowing any other field/property (I think) :smile: Guess the provider could as well have some specific structure that allows additional metadata in a more structured manner, but open for discussion.

Another question regarding importing grafana-schema/common, see below, which is supported in core Grafana.
import (
	"github.com/grafana/grafana/packages/grafana-schema/src/common"
)

Don't have any idea how we're going to support this for providers. @sdboyer any existing ideas around this?

Don't have any idea how we're going to support this for providers.

Yeah, this is one of the nastiest bits we have to deal with. We have to cheat the otherwise normal Thema rules of not allowing references to external things.
This is particularly painful for registries and Thema itself, which are generally responsible for the compatibility guarantees of the system. But we can start the answer to it in kindsys, because it's OK for it to know about some Grafana patterns (whereas Thema does not and will not). So, kindsys can provide an allowlist on certain import paths. It's then on us to create other checks (e.g. in CI for grafana/grafana) that ensure those common schemas only change in acceptable ways. We've started outlining what those have to be here.

Summary of today's mob session with @AgnesToulet and @andresmgot:
Tried to pick up where we left off, i.e. copying grafanaplugin schema from grafana repo to kindsys for the sake of validating some of our ideas in a provider-test involving the grafanaplugin schema. Running a bit in circles at first, confused with certain errors etc.
Eventually @AgnesToulet explained the current state of the kind registry and that grafana kinds are moved/transferred to the kind registry in a slightly different shape (certain fields/properties are explicitly set) compared to how its defined within core Grafana. As an example, https://github.com/grafana/grafana/blob/main/public/app/plugins/panel/annolist/panelcfg.cue, are  being converted to a kind registry schema using https://github.com/grafana/grafana/blob/main/.github/workflows/scripts/kinds/verify-kinds.go and the result is https://github.com/grafana/kind-registry/blob/main/grafana/next/composable/annotationslist/panelcfg.cue. To quote Agnes, the kind registry still needs to define (have) providers for each kind
Above is very much in line with Proposal 2 in the kind providers design doc where it was suggested to use the pfs package as an intermediate that take plugin schemas as input and outputs a kind provider. This seems like the way forward.
Given above and this PR there's a few identified things left to fix:

Cleanup/remove grafanaplugin test
Fix imports of common, e.g. failed to load instance: import failed: cannot find package "github.com/grafana/grafana/packages/grafana-schema/src/common"
Possibly, auto-generate a cue.mod/module.cue file when its missing

In regards to grafana/pfs package we want to modify that to make it able to output a kind provider schema (fs.FS filesystem) which can be either provided as input to kindsys.LoadProvider or either written to disk.

Don't have any idea how we're going to support this for providers.

Yeah, this is one of the nastiest bits we have to deal with. We have to cheat the otherwise normal Thema rules of not allowing references to external things.
This is particularly painful for registries and Thema itself, which are generally responsible for the compatibility guarantees of the system. But we can start the answer to it in kindsys, because it's OK for it to know about some Grafana patterns (whereas Thema does not and will not). So, kindsys can provide an allowlist on certain import paths. It's then on us to create other checks (e.g. in CI for grafana/grafana) that ensure those common schemas only change in acceptable ways. We've started outlining what those have to be here.

Fix imports of common, e.g. failed to load instance: import failed: cannot find package "github.com/grafana/grafana/packages/grafana-schema/src/common"

I guess the main question for me is how making github.com/grafana/grafana/packages/grafana-schema/src/common cue files available within kindsys since it seems like the actual files must be available for imports to succeed or can we just "allow" the package for now as Sam suggests and that will "just work"? The only possible way I see right now is to reference https://github.com/grafana/grafana/blob/39a18ca6ba8d46c5e10572d8ab44affb0785f516/embed.go#L10 to get a hold of the common schemas.
@AgnesToulet do you have any other idea?


Follow-up #1213: Add support for DRUPALVM_ANSIBLE_TAGS env variable

I refactored the provisioner declaration so that it's more DRY. Doesnt matter much now but maybe if we add SKIP_TAGS or some more config variables in the future.

LGTM!


[Feature Request] @beartype + @functools.lru_cache integration

I'm eating a lot of honey but sometimes I get stung. How can I get honey without any bees???
# trying_stuff_module.py

from functools import lru_cache

@lru_cache(maxsize=3)
def plus_one(n: int) -> int:
    return n +1

# trying_stuff_main.py

import beartype.claw

beartype.claw.beartype_packages(["trying_stuff_module"])

from trying_stuff_module import plus_one

print(plus_one(2))

$ python trying_stuff_main.py
/projects/open_source/beartype/beartype/_decor/decorcore.py:414: BeartypeClawDecorWarning: Object trying_stuff_module.plus_one() :
Traceback (most recent call last):
      File "/projects/open_source/beartype/beartype/_decor/decorcore.py", line 367, in _beartype_object_nonfatal
        return _beartype_object_fatal(obj, conf, **kwargs)
      File "/projects/open_source/beartype/beartype/_decor/decorcore.py", line 229, in _beartype_object_fatal
        return beartype_object_monkeypatch(  # type: ignore[return-value]
      File "/projects/open_source/beartype/beartype/_decor/_decormore.py", line 382, in beartype_object_monkeypatch
        obj.__class__.__call__ = _beartype_descriptor_method_bound(  # type: ignore[assignment,method-assign]
      File "/projects/open_source/beartype/beartype/_decor/_decormore.py", line 411, in _beartype_descriptor_method_bound
        assert isinstance(descriptor, MethodBoundInstanceOrClassType), (
    AssertionError: <method-wrapper '__call__' of functools._lru_cache_wrapper object at 0x7f42fb82dc70> not builtin bound method descriptor.

  warn(warning_message, warning_category)
3

I'm eating a lot of honey but sometimes I get stung.

:smile: -> :laughing: -> :rofl:

AssertionError: <method-wrapper '__call__' of functools._lru_cache_wrapper object at 0x7f42fb82dc70> not builtin bound method descriptor.

:disappointed: -> :cry: -> :sob:
Thankfully, this should be mostly trivial. It looks like any arbitrary object – including low-level C-based private functools._lru_cache_wrapper objects – can be a @functools.wrap wrapper. That's unexpected, but ultimately fine... and probably a good thing. This means we'll just need to generalize our assumptions about what constitutes a valid @beartypeable (i.e., an object that can be decorated by @beartype).
Let's do this. Team Bear-Bear: Assemble! :muscle: :man_beard:

So. It begins. (╯°□°)╯︵ ┻━┻

So. It ends. ( ͡° ͜ʖ ͡°)


Enhanced depends logic not working when task is omitted

Checklist

[x] Double-checked my configuration.
[x] Tested using the latest version.
[x] Used the Emissary executor.

Summary
What happened?
When a task is Skipped/Omitted because a dependent task is skipped you can't use enhanced depends on that task.
  O
 /  \
A    B <- B is skipped
|    |
C    D <- D gets omitted
|   /
E <- if C.Succeeded && (D.Succeeded || D.Skipped)*

*doesn't work

What you expected to happen?
If a task is omitted/skipped due a dependent predecessor task being skipped, you should be able to use depends.Skipped on that task in another task.
What version are you running? 3.3.1
Diagnostics
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: broken-depends
  namespace: argo
spec:
  serviceAccountName: argo-workflow-sa
  entrypoint: diamond
  arguments:
    parameters:
      - name: input_var
        default: "true"
  templates:
    - name: diamond
      inputs:
        parameters:
          - name: input_var
      dag:
        tasks:
          - name: A
            template: pass
          - name: B
            template: pass
            when: "{{ inputs.parameters.input_var }} == false"
          - name: C
            template: pass
            depends: B.Succeeded
          - name: D
            template: pass
            depends: A.Succeeded
          - name: E
            template: pass
            depends: D.Succeeded && C
    - name: pass
      container:
        image: alpine:3.7
        command:
          - sh
          - -c
          - exit 0
    - name: fail
      container:
        image: alpine:3.7
        command:
          - sh
          - -c
          - exit 1

# Logs from the workflow controller:
time="2022-05-06T05:20:29.162Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.176Z" level=info msg="Updated phase  -> Running" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.198Z" level=info msg="DAG node broken-depends-pn58z initialized Running" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="All of node broken-depends-pn58z.B dependencies [] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="Skipped node broken-depends-pn58z-2614862280 initialized Skipped (message: when 'true == false' evaluated false)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.199Z" level=info msg="Skipped node broken-depends-pn58z-2631639899 initialized Omitted (message: omitted: depends condition not met)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.200Z" level=info msg="All of node broken-depends-pn58z.A dependencies [] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.200Z" level=info msg="Pod node broken-depends-pn58z-2665195137 initialized Pending" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.228Z" level=info msg="Created pod: broken-depends-pn58z.A (broken-depends-pn58z-2665195137)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.229Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.229Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:29.267Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=359615338 workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.234Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.235Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=broken-depends-pn58z-2665195137 old.message= old.phase=Pending old.progress=0/1
time="2022-05-06T05:20:39.236Z" level=info msg="All of node broken-depends-pn58z.D dependencies [A] completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.241Z" level=info msg="Pod node broken-depends-pn58z-2715527994 initialized Pending" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.262Z" level=info msg="Created pod: broken-depends-pn58z.D (broken-depends-pn58z-2715527994)" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.263Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.263Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.278Z" level=info msg="Workflow update successful" namespace=argo phase=Running resourceVersion=359615414 workflow=broken-depends-pn58z
time="2022-05-06T05:20:39.283Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/broken-depends-pn58z-2665195137/labelPodCompleted
time="2022-05-06T05:20:49.267Z" level=info msg="Processing workflow" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.269Z" level=info msg="node changed" new.message= new.phase=Succeeded new.progress=0/1 nodeID=broken-depends-pn58z-2715527994 old.message= old.phase=Pending old.progress=0/1
time="2022-05-06T05:20:49.270Z" level=info msg="Outbound nodes of broken-depends-pn58z set to [broken-depends-pn58z-2631639899 broken-depends-pn58z-2715527994]" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="node broken-depends-pn58z phase Running -> Succeeded" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="node broken-depends-pn58z finished: 2022-05-06 05:20:49.270813223 +0000 UTC" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Checking daemoned children of broken-depends-pn58z" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg=reconcileAgentPod namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Updated phase Running -> Succeeded" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Marking workflow completed" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.270Z" level=info msg="Doesn't match with archive label selector. Skipping Archive" namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.272Z" level=info msg="Checking daemoned children of " namespace=argo workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.277Z" level=info msg="cleaning up pod" action=deletePod key=argo/broken-depends-pn58z-1340600742-agent/deletePod
time="2022-05-06T05:20:49.288Z" level=info msg="Workflow update successful" namespace=argo phase=Succeeded resourceVersion=359615487 workflow=broken-depends-pn58z
time="2022-05-06T05:20:49.300Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo/broken-depends-pn58z-2715527994/labelPodCompleted

Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

This is fixed in https://github.com/argoproj/argo-workflows/pull/8672. Thanks for reporting!

@terrytangyuan I'm using Omitted with dag, and sensor bumps into this error https://github.com/argoproj/argo-workflows/blob/master/workflow/common/ancestry.go#L102

@terrytangyuan I'm using Omitted with dag, and sensor bumps into this error https://github.com/argoproj/argo-workflows/blob/master/workflow/common/ancestry.go#L102

Thank you! I am fixing this in https://github.com/argoproj/argo-workflows/pull/8776.

in version 3.3.8 this issue still exists.
Its impossible to launch/create a workflow with depends on .Omitted step.
You get an error: Internal Server Error: templates.main.tasks.E task result 'Omitted' for task 'D' is invalid
Example Workflow:
apiVersion: argoproj.io/v1alpha1
metadata:
  name: broken-depends
  generateName: broken-depends-
  namespace: argo
spec:
  serviceAccountName: workflow
  entrypoint: main
  templates:
    - name: main
      inputs:
        parameters:
          - name: input_var
            default: 'true'
      dag:
        tasks:
          - name: A
            template: pass
          - name: B
            template: pass
            when: '{{ inputs.parameters.input_var }} == false'
          - name: C
            template: pass
            depends: A.Succeeded
          - name: D
            template: pass
            depends: B.Succeeded
          - name: E
            template: pass
            depends: (D.Succeeded || D.Omitted) && C
    - name: pass
      container:
        image: 'alpine:3.7'
        command:
          - sh
          - '-c'
          - exit 0
    - name: fail
      container:
        image: 'alpine:3.7'
        command:
          - sh
          - '-c'
          - exit 1


xmlpeek does not allow DTD

We are trying to use the external entity in XML. When we run msbuild, it calls the xmlpeek task, which seems does not allow DTD by default according to the error from xmlpeek task attached below.
error MSB3733: Input file "..." cannot be opened. For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.
Is there anyway to change the property of DtdProcessing for the xmlpeek task.

Team Triage:
This seems reasonable. The security settings should be OK to turn off since you should only open project files from a trusted source anyway. This would need to be turned off be default but could be opt-in. Marking as up-for-grabs.

Any progress on this? This behavior makes impossible to read values from Apple plist files. Thus, if the file starts with:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
...

it can be read by XmlPeek task.

I'd like to second this issue. Using latest Visual Studio For Mac, which in theory uses the latest release of MSBuild, fails on the XmlPeek task for iOS .plist files. However XmlPoke works just fine.


⚠ API (search) has degraded performance

In d69364d, API (search) (https://api.dev.package-search.services.jetbrains.com/search-packages) experienced degraded performance:

HTTP code: 200
Response time: 6733 ms

Resolved: API (search) performance has improved in 7009073 after 15 minutes.


Increment version for translation releases

Increment package version after release of com.azure azure-ai-translation-document

API change check
API changes are not detected in this pull request.


Fix transceiver initialization for janus streaming

Description
peerConnection.OnTrack was never being invoked if using the defaults,
recvonly must be explicitly set.
Reference issue
Fixes #120

Description
peerConnection.OnTrack was never being invoked if using the defaults, recvonly must be explicitly set.

My assumption here is that if unspecified, the default is sendrecv which will never actually start receiving in this instance.

For reference I got the idea from here:
https://github.com/pion/example-webrtc-applications/blob/707e13f8ed2ed35c8f0d4bbd744e68ab34c4235f/sfu-ws/main.go#L245

Your on 🔥 thanks @tmatth !


Ticket issues

Fixes #1216 #1215
Ticket configuration page:

Order complete page:

Sample order complete email:
epcon_1  | Hello!
epcon_1  |
epcon_1  | Thanks for buying tickets for EuroPython 2020.
epcon_1  |
epcon_1  | This is the summary of order O/20.0008:
epcon_1  |
epcon_1  |
epcon_1  | Ticket number: 8 ��� TESP ��� EuroPython 2020 - Early Bird Standard Personal 1/4 ��� 210.00
epcon_1  |
epcon_1  | Ticket number: 9\ ��� TESP ��� EuroPython 2020 - Early Bird Standard Personal 2/4 ��� 210.00
epcon_1  |
epcon_1  | Ticket number: 10 ��� TESP ��� EuroPython 2020 - Early Bird Standard Personal 3/4 ��� 210.00
epcon_1  |
epcon_1  | Ticket number: 11 ��� TESP ��� EuroPython 2020 - Early Bird Standard Personal 4/4 ��� 210.00
epcon_1  |
epcon_1  |
epcon_1  |
epcon_1  | Your tickets were created and are available on your profile page here:
epcon_1  |
epcon_1  | https://example.com/user-panel/
epcon_1  |
epcon_1  | Your invoice is also available on the same url.
epcon_1  |
epcon_1  | If you run into any issues, please contact us at<EMAIL_ADDRESS>epcon_1  |
epcon_1  | Kind regards,
epcon_1  | --
epcon_1  | EuroPython 2020 Team
epcon_1  | https://ep2020.europython.eu

Just need to update to the last dev/ep2020


917 improve code coverage of ide secir

Changes and Information
Please briefly list the changes (main added features, changed items, or corrected bugs) made:

Changed one value in the test for parameters.h so that the uncovered lines are now covered.
Deleted the first check of check_constraints in modelbecause this case already leads to an error in the construction of a model.

If need be, add additional information and what the reviewer should look out for in particular:

Merge Request - Guideline Checklist
Please check our git workflow. Use the draft feature if the Pull Request is not yet ready to review.
Checks by code author

[x] Every addressed issue is linked (use the "Closes #ISSUE" keyword below)
[x] New code adheres to coding guidelines
[x] No large data files have been added (files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
[x] Tests are added for new functionality and a local test run was successful
[x] Appropriate documentation for new functionality has been added (Doxygen in the code and Markdown files if necessary)
[x] Proper attention to licenses, especially no new third-party software with conflicting license has been added
[ ] (For ABM development) Checked benchmark results and ran and posted a local test above from before and after development to ensure performance is monitored.

Checks by code reviewer(s)

[ ] Corresponding issue(s) is/are linked and addressed
[ ] Code is clean of development artifacts (no deactivated or commented code lines, no debugging printouts, etc.)
[ ] Appropriate unit tests have been added, CI passes, code coverage and performance is acceptable (did not decrease)
[ ] No large data files added in the whole history of commits(files should in sum not exceed 100 KB, avoid PDFs, Word docs, etc.)
[ ] On merge, add 2-5 lines with the changes (main added features, changed items, or corrected bugs) to the merge-commit-message. This can be taken from the briefly-list-the-changes above (best case) or the separate commit messages (worst case).

Closes #917

The problem with the uncovered lines 227-230 in parameters.h was that there is a check for negative probabilities (210-214), so that the value -1 already leads to a returned true there.

Merged with #959 .


Update settings from dict format to variable format

It seems that in django-templated-email, you're using straight variables inside the settings file, instead of a nested dictionary like in djoser, and the variable names should reflect that in the documentation for that

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master     #7   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      1
  Lines          56     56
=====================================
  Hits           56     56

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bcadd81...8cbcbd9. Read the comment docs.

I think we should go the other way around here – change the way we load settings.


Improper event bus handling

In Module, you are registering modules on enable and on disable. This means you are forced to check isToggled at the top of each listener

I have no idea what that means, but with what have I to replace it?

Ohhhh


Download failed: Invalid Oracle response.

Hi,
Update php from 7.3.5 -> 7.3.6
I enter the correct login / password (rechecked the username and password in the browser, that's right) from the account on the oracle site and give this error:

Thanks for the heads up and the log/output capture. Fixed at 9598dec6165f39d8036cb4e6a631b9b17a1cde3e. Always a cat/mouse game with Oracle auth changes.


Get authentication errors after many requests are executed

Hi Team,
I am fetching email message from office 365 account following sample code given in the npm documentation.
I see below error after executing around thousand requests to get email message against office265.
-ERR Logon failure: unknown user name or bad password.
The error goes off automatically after sometime. Any idea what is going wrong ?

The issue was actually with Office mail server. I don't see this error after I started using MSN server instead of office server.


Lerp FractionalOffset and friends from their origin

(I'm going to close this PR for now just to get it off our review queue.)


First added event not being rendered

Hi, I'm using the ui-calendar together with angular-dragdrop to add new events on certain days. The thing is, after adding a new event, it does not get drawn. The events data structure is being updated, because if the calendar is moved one month forward and then back, the event is then rendered. Further events after the first one appear correctly.
Any ideas will be greatly appreciated!
Artur

hey so can you create a simple reproduction of this issue? I know it's a bit late ot the game but I think this one slipped through the cracks.


Select action doesn't respect disabled flag

Given this:
<select name="my-select">
  <option value="0" disabled="disabled">Disabled option</option>
  <option value="1">Enabled option</option>
</select>

When I do this:
select('Disabled option', :from => 'my-select')

Everything is fine, but I expect an exception to be thrown.

We can't throw an exception currently (breaking behavior change) - but a warning does seem reasonable (like happens when attempting to set read only fields).  See PR #1532

At least it should not be selected. Thank you!


Feature/ted 938

Codecov Report
Base: 99.11% // Head: 98.99% // Decreases project coverage by -0.12% :warning:

Coverage data is based on head (e59f4ee) compared to base (e73ea9b).
Patch coverage: 97.09% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #354      +/-   ##
==========================================
- Coverage   99.11%   98.99%   -0.13%
==========================================
  Files         119      123       +4
  Lines        6000     6175     +175
==========================================
+ Hits         5947     6113     +166
- Misses         53       62       +9

Impacted Files
Coverage Δ

..._metadata_processor/services/notice_eligibility.py
97.77% <ø> (-0.05%)
:arrow_down:

...metadata_processor/services/metadata_normalizer.py
98.58% <75.00%> (-1.42%)
:arrow_down:

...ter_data_registry/services/entity_deduplication.py
96.40% <93.05%> (-2.54%)
:arrow_down:

ted_sws/core/adapters/cmd_runner.py
100.00% <100.00%> (ø)

ted_sws/core/adapters/config_resolver.py
75.47% <100.00%> (+0.47%)
:arrow_up:

ted_sws/core/adapters/xml_preprocessor.py
100.00% <100.00%> (ø)

ted_sws/core/model/metadata.py
100.00% <100.00%> (ø)

ted_sws/core/model/supra_notice.py
100.00% <100.00%> (ø)

ted_sws/core/model/transform.py
100.00% <100.00%> (ø)

ted_sws/data_manager/adapters/triple_store.py
97.08% <100.00%> (+0.02%)
:arrow_up:

... and 20 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Redux-form compatibility with RichText editor (draft-js) control

Hello:
I am using a rich text editor control (draft-js)  on a redux form. The issue I am having is that the object  returned to redux-forms is a ContentState  type by the OnChange function which causes an iodash deprecation error. Returning this ContentState object value is the only way to get Redux-forms to work for me using this control but getting warning errors.
Does redux-forms support data value types other than strings, numbers, and arrays? It appears the Rich TextEditor (draft-js) is a ContentState object thus causing parsing errors within redux-forms.
This is the warning that appears everytime the OnChange or a key is pressed within the RichTextEditor field:
immutable.js:4655 iterable.length has been deprecated, use iterable.size or iterable.count(). This warning will become a silent error in a future version. Error
at ContentState.get (webpack:///C:/Users/bbuscemi/Documents/AE2S/Origin-Development/ae2s/AE2S/~/draft-js/~/immutable/dist/immutable.js?:4650:21)
at baseIsTypedArray (webpack:///C:/Users/bbuscemi/Documents/AE2S/Origin-Development/ae2s/AE2S/~/lodash/_baseIsTypedArray.js?:57:19)
at arrayLikeKeys (webpack:///C:/Users/bbuscemi/Documents/AE2S/Origin-Development/ae2s/AE2S/~/lodash/_arrayLikeKeys.js?:26:47)
at keys (webpack:///C:/Users/bbuscemi/Documents/AE2S/Origin-Development/ae2s/AE2S/~/lodash/keys.js?:34:32)

I saw a previous post on this issue that someone recommended react-rte which is an enhancement to the draft-js component. That enhancement returns a string format instead of an object. I used that and it works great with redux-forms. I do like redux-forms and appreciate Erik's work very much. Thanks.


target:

Versions

nuxt:
node:

Reproduction

Additional Details

Steps to reproduce
What is Expected?
What is actually happening?

This report does not contain information. Thus, I will close the issue. Please create a new one with proper content if the issue/bug/... still exists


Problem with collate function

Hi! Thanks for this wonderful application, it helped me a lot.
However, I ran into an issue with the collate function I can't figure out to solve:
picardmetrics collate filename /path/filename/
picardmetrics version 0.2.4 2016-07-06
2018-09-17 10:47:47	START	filename
2018-09-17 10:47:47	Collating 96 alignment_summary_metrics files
2018-09-17 10:47:53	Collating 96 quality_distribution_metrics files
2018-09-17 10:47:57	Collating 96 rnaseq_metrics files (summary)
2018-09-17 10:48:01	Collating 96 rnaseq_metrics files (coverage)
2018-09-17 10:48:05	Collating 96 gc_bias_metrics files
2018-09-17 10:48:07	Collating 96 gc_bias_histogram files
2018-09-17 10:48:12	Collating 96 duplicate_metrics files
2018-09-17 10:48:16	Collating 96 insert_size_metrics files
2018-09-17 10:48:21	Collating 96 insert_size_metrics files (histogram)
2018-09-17 10:48:25	Collating 96 base_distribution_by_cycle files
2018-09-17 10:48:32	Collating 96 library_complexity files
2018-09-17 10:48:36	Collating 96 library_complexity files (histogram)
2018-09-17 10:48:40	Collating 96 mapq_stats files
2018-09-17 10:48:41	Joining all files into 'filename-all-metrics.tsv'
Error: all(dat_align_metrics$SAMPLE == dat_duplicate_metrics$SAMPLE) is not TRUE
Execution halted
2018-09-17 10:48:42	DONE	filename
I don't understand where the problem lies as there are 96 files in each category. When I count the lines in the intermediate files, it give 97 for duplicate_metrics and 289 for alignment_metrics.
Thanks
Alex

Could I ask you to inspect the contents of the two files?

alignment_summary_metrics
duplicate_metrics

I wonder if one of the files is missing a result.

Hi! I inspected the files and they seem valid. Each sample has one line in duplicate_metrics and 3 lines in alignment_metrics. No empty lines, no duplicates... I even realigned my samples in case something went wrong there, but still get the same error.
Anything else I could check?

If you could share the output files, I might be able to fix the code to work with your files.

8Fat-alignment-metrics.xlsx
8Fat-duplicate-metrics..xlsx

I was confused because you added .xlsx to the file names. These files are not Microsoft Excel spreadsheets, they're just plain text files.
You cannot open them with Excel:

You can read the contents anyway:
$ head 8Fat-alignment-metrics.xlsx | cut -f1-3 | column -t
SAMPLE                                                                   CATEGORY        TOTAL_READS
scratch60/ercc/8Fat/picardmetrics/8FTreg10_S57Aligned.sortedByCoord.out  FIRST_OF_PAIR   333
scratch60/ercc/8Fat/picardmetrics/8FTreg10_S57Aligned.sortedByCoord.out  SECOND_OF_PAIR  333
scratch60/ercc/8Fat/picardmetrics/8FTreg10_S57Aligned.sortedByCoord.out  PAIR            666
scratch60/ercc/8Fat/picardmetrics/8FTreg11_S82Aligned.sortedByCoord.out  FIRST_OF_PAIR   964252
scratch60/ercc/8Fat/picardmetrics/8FTreg11_S82Aligned.sortedByCoord.out  SECOND_OF_PAIR  964252
scratch60/ercc/8Fat/picardmetrics/8FTreg11_S82Aligned.sortedByCoord.out  PAIR            1928504
scratch60/ercc/8Fat/picardmetrics/8FTreg12_S84Aligned.sortedByCoord.out  FIRST_OF_PAIR   723032
scratch60/ercc/8Fat/picardmetrics/8FTreg12_S84Aligned.sortedByCoord.out  SECOND_OF_PAIR  723032
scratch60/ercc/8Fat/picardmetrics/8FTreg12_S84Aligned.sortedByCoord.out  PAIR            1446064

I tried running the R code in the picardmetrics script:
https://github.com/slowkow/picardmetrics/blob/94cb6512be20083925ce9f046133efea2b03ff44/picardmetrics#L929-L932
It looks like your files are ok:
setwd("~/Downloads/8Fat")
read_tsv <- function(filename, ...) {
  if (!file.exists(filename)) {
    warning("File does not exist: ", filename)
    return(NULL)
  }
  dat <- read.delim(filename, stringsAsFactors = FALSE, ...)
  return(dat)
}

dat_align_metrics <- read_tsv("8Fat-alignment-metrics.tsv")
dat_duplicate_metrics <- read_tsv("8Fat-duplicate-metrics.tsv")
idx = !dat_align_metrics$CATEGORY %in% c("FIRST_OF_PAIR", "SECOND_OF_PAIR")
dat_align_metrics = dat_align_metrics[idx, ]
all(dat_align_metrics$SAMPLE == dat_duplicate_metrics$SAMPLE)
#> [1] TRUE

Created on 2018-10-11 by the reprex package (v0.2.0).
Here are my suggestions:

Ensure you are using the latest version of the picardmetrics script.

Copy the R code in the picardmetrics script and paste it into a new file.  You might find the problem that way.

Don't bother collating all of the files into 1 file. Just work with the files you have.

Good luck!

Ok, I'll try! Thanks!


Make init job annotations optional

Some kubernetes executors like ArgoCD will not deploy this chart correctly with the given annotations. I've found that these annotations need to be removed and replaced with:
argocd.argoproj.io/hook: Sync
argocd.argoproj.io/hook-delete-policy: HookFailed

This PR makes it possible to customize the annotations

This was addressed in #250


Add Picnic CSS

I created Picnic CSS with many similar, pure css components. For instance, there are Modals, animated responsive Navs, Tabs and Tooltips

I think some of these things like the tooltips would make great additions. Do you want to make a PR for those?


Allow Template script to use scripts in turn_on and turn_off

Description:
Allow scripts to be placed in the turn_on: and turn_off: parts of template switch
Example entry for configuration.yaml (if applicable):
switch:
  platform: template
  switches:
    delaylight:
      friendly_name: 'Test script'
      value_template: '{{ states.light.buitedl.state }}'
      turn_on:
        - delay: 00:00:01
        - service: light.turn_on
          entity_id: light.buitedl
      turn_off:
        service: light.turn_off
        entity_id: light.buitedl

Checklist:
If the code does not interact with devices:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[N/A] Tests have been added to verify that the new code works.
N/A reuse script

Seems straight-forward enough to me.

@rmkraus Almost too straight forward...
It might be better to create the Script objects for turn_on and turn_off during the initialization of the TemplateSwitch rather than creating from config on each toggle.
Not necessarily for speed but at least to get config validation on startup and not only on toggle.
I can add it to this PR if the idea is ok?

👍 Looks good 🐬
And yes, it is that easy.
@kellerza caching the script object is a good idea.
(not perse for config validation as that should go via PLATFORM_SCHEMA)

I'm going to merge this one now, we can add script object caching in a new PR so this makes it into the next release.


Error: Running FINN in Docker

Hi, as I already installed Vivado HLS 2018.3 in Ubuntu 18.04, when I try to bash ./run_docker.sh in terminal. There are some errors about Vivado_Path, and I don't know how to set the Path in ubuntu, Could you help me with that?
Errors :
Please set the VIVADO_PATH that contains the path to your Vivado installation directory.
FINN functionality depending on Vivado or Vivado HLS will not be available.
Please set the PYNQ_IP env.var. to enable PYNQ deployment tests.
Please set the VITIS_PATH that contains the path to your Vitis installation directory.
FINN functionality depending on Vitis will not be available.

Obtaining file:///workspace/finn
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/workspace/finn/setup.py'"'"'; file='"'"'/workspace/finn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vcxelnw6
cwd: /workspace/finn/
Complete output (29 lines):
Traceback (most recent call last):
File "", line 1, in
File "/workspace/finn/setup.py", line 50, in
setup(use_pyscaffold=True)
File "/opt/conda/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/opt/conda/lib/python3.8/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/opt/conda/lib/python3.8/site-packages/setuptools/dist.py", line 423, in init
_Distribution.init(self, {
File "/opt/conda/lib/python3.8/distutils/dist.py", line 292, in init
self.finalize_options()
File "/opt/conda/lib/python3.8/site-packages/setuptools/dist.py", line 695, in finalize_options
ep(self)
File "/opt/conda/lib/python3.8/site-packages/setuptools/dist.py", line 702, in _finalize_setup_keywords
ep.load()(self, ep.name, value)
File "/workspace/finn/.eggs/PyScaffold-3.2.3-py3.8.egg/pyscaffold/integration.py", line 105, in pyscaffold_keyword
version_keyword(dist, keyword, setuptools_scm_config(value))
File "/workspace/finn/.eggs/PyScaffold-3.2.3-py3.8.egg/pyscaffold/contrib/setuptools_scm/integration.py", line 17, in version_keyword
dist.metadata.version = get_version(**value)
File "/workspace/finn/.eggs/PyScaffold-3.2.3-py3.8.egg/pyscaffold/contrib/setuptools_scm/init.py", line 150, in get_version
parsed_version = _do_parse(config)
File "/workspace/finn/.eggs/PyScaffold-3.2.3-py3.8.egg/pyscaffold/contrib/setuptools_scm/init.py", line 105, in _do_parse
raise LookupError(
LookupError: setuptools-scm was unable to detect version for '/workspace/finn'.
Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
bash: /settings64.sh: No such file or directory
Thanks,

Moved to existing discussion: https://github.com/Xilinx/finn/discussions/387


cake fails to build on travis-ci with latest mono (6.8)

What You Are Seeing?
Cake fails to compile and run build script.  The error message is.
Error: The assembly name is invalid.
The problem appears to be related to Mono 6.8 - a temporary workaround that has fixed the issue for me was to add the line mono: 6.6.0 into the below travis.yml.
What is Expected?
Cake compiles and runs build script.
What version of Cake are you using?
The below log was generated using Cake 0.32.1, but as part of diagnosing the issue, I updated to 0.36.0 and it didn't make a difference.
I also created a hello world Cake script
Are you running on a 32 or 64 bit system?
What environment are you running on?  Windows? Linux? Mac?
travis, linux, csharp environment as below.
Are you running on a CI Server?  If so, which one?
travis
yml:
language: csharp
dotnet: 2.1.504
sudo: required
services:
  - docker
before_install:
  - export PATH=~/bin:$PATH
script:
  - chmod +x build.sh
  - ./build.sh

Cake script
var target = Argument("target", "Default");

Task("Default")
  .Does(() =>
{
  Information("Hello World!");
});

RunTarget(target);

How Did You Get This To Happen? (Steps to Reproduce)
A hello world cake script in conjunction with the above travis.yml should be enough to repro this issue.
Output Log

```
9.77s$ ./build.sh -t Build
Downloading NuGet...
  https://api.nuget.org/v3/index.json
Restoring NuGet package Cake.0.32.1.
  GET https://api.nuget.org/v3-flatcontainer/cake/0.32.1/cake.0.32.1.nupkg
  OK https://api.nuget.org/v3-flatcontainer/cake/0.32.1/cake.0.32.1.nupkg 18ms
Installing Cake 0.32.1.
Adding package 'Cake.0.32.1' to folder '/home/travis/build/linn/mylinn/tools'
Added package 'Cake.0.32.1' to folder '/home/travis/build/linn/mylinn/tools'
Analyzing build script...
Processing build script...
Installing addins...
Compiling build script...
Error: The assembly name is invalid.
The command "./build.sh -t Build" failed and exited with 1 during .
```

A version with this fixed is now up on our pre-release NuGet feed
https://dev.azure.com/cake-build/cake/_packaging?_a=package&feed=cake%40Local&package=Cake&protocolType=NuGet&version=0.37.0-alpha0026&view=overview


Implement index deletion comand

Give the user the posibility to have the index removed. via the exact command: "@ agent delete [reponame] index" .

done


Logout does not support SafariViewController

Checklist

[X] The issue can be reproduced in the auth0_flutter sample app (or N/A).
[X] I have looked into the Readme, Examples, and FAQ and have not found a suitable solution or answer.
[X] I have looked into the API documentation and have not found a suitable solution or answer.
[X] I have searched the issues and have not found a suitable solution or answer.
[X] I have searched the Auth0 Community forums and have not found a suitable solution or answer.
[X] I agree to the terms within the Auth0 Code of Conduct.

Description
Reading the docs it state that we can use the SafariViewController for authentication on login by providing
safariViewController: const SafariViewController(), to the login params.
However when doing the Logout, we can't specify that we want to use the safariViewController, as such, the credentials aren't cleared correctly.
When trying to log in again, it instant logs in as it is using the SafariViewController again and that has not been cleared.
I am not sure if I am missing something or that this is an actual bug/improvement.
Reproduction
call .login(safariViewController: const SafariViewController()) and enter your credentials.
call logout, this will not show the SafariViewController
call .login(safariViewController: const SafariViewController()) again, this now instant logs in without asking for your credentials as it uses the SafariViewControllers cookies/cache (assumption)
Additional context
No response
auth0_flutter version
^1.2,1
Flutter version
3.10.0
Platform
iOS
Platform version(s)
17

Hi @MatthiasDh, thanks for raising this.
This is intentional; according to SFSafariViewController's docs, it must be used "to visibly present information to users":

This is the case for login, but not for logout. You can delete the stored credentials though (using the Credentials Manager) and use 'prompt': 'login' to force the login page.
final credentials = await auth0.webAuthentication().login(
    useEphemeralSession: true,
    parameters: {
      'prompt': 'login'
    }); // Ignore the cookie (if present) and show the login page

Since the SFSafariViewController's cookies are scoped to your app, this should not pose an issue.

This makes sense, thanks for the reply and help.
Can confirm that the prompt login worked!


Automatic increase of Docker memory through specification in image file?

Investigate the possibility.

@landreman,
On linux, docker containers can consume the full memory. On mac, if you are using docker desktop, the default max mem limit is 2GB, which can be changed as you did. I think this is mac specific, so adding a line in documentation cautioning the users is the only thing we can do.