Render markdown for enum member documentation. Fixes #1081.

Fixes #1081 .
Changes proposed in this pull request:

Render markdown for enum member documentation

Before:

After:

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Cleanup README

Remove trailing spaces

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Documentation and wording stuff!

I don't want to drop 5000 lines of documentation changes in mid February so I'll mark this as ready for review now and divide other documentation changes based on directories I think


What Should We Do With Terraform Cloud Integrations?

In the process of scrubbing "Terraform" and "Hashicorp" out of the code base at least for areas that end users will see for trademark reasons, it has come to my attention that the terraform code has a lot of terraform cloud integration baked into it.
For my part, I just conservatively left it there, but I think we'll have to make a conscious decision about this.
Given that my workplace doesn't use Terraform Cloud, its easy for me to say "lets rip it out and it will make the code simpler", but I'm guessing a decent number of Terraform users are using Terraform Cloud and the set of people that would want to use OpenTF AND Terraform Cloud might not be empty (I have no idea how large it is).
However, if we keep Terraform Cloud into the code, Hashicorp won't maintain it as Terraform diverges from the OpenTF repo and we would have to shoulder the constant reconciliation of that divergence ourselves.
Alternatively, something we could try to do is remove Terraform Cloud integration from the repo and optionally support it as a separate plugin if we need (I'm naively guessing that is possible, though it would require significant architectural work and either way I think it is safe to guess that Hashicorp would not join the party in maintaining Terraform Cloud as a separate plugin either even if they would potentially have a commercial interest in it).

IMO this project should separate itself from the legacy hashi maintained products/projects. I suggested all TFC integrations be removed. Why would opentf support direct integrations with hashi but not the likes of spacelift or env0? Maybe a plugin framework if hashi/spacelift/env0 want to build something to do that, but it being core to opentf I think is wrong.

I bet people relying on the TF Cloud will stick with HashiCorp Terraform rather than OpenTF because of compatibility and support.
Moreover, keeping Terraform Cloud integration will primarily benefit HashiCorp and allow it to monetize the community work.

Fair point, we should strive to make the project vendor-neutral with support for plugin integration that vendors can take advantage of to integrate services.

If we rip the cloud part out then it will fragment the community. There are a lot of Terraform Cloud users, and we should want them to be able to use the open source version. Maintaining that functionality will make it easier for people to migrate to the fork over time. If we leave it in then people won't have to immediately "pick a side" (which would also mean migrate to a different TACOS for people on Terraform Cloud), and since status quo tends to win out I think it would really harm this project.
I also think that the "cloud" protocol itself shouldn't be specific to Terraform Cloud. If the protocol has benefits it should be opened up so other TACOS can also use that same cloud block.  Although Hashicorp could in theory introduce regular backwards compatibility breaks, the reality is that would create a negative experience for their own users as they get locked into smaller version windows. So they have an incentive to have a stable protocol as well, which should make it easier for this project to keep working with terraform cloud.

The current Terraform Cloud backend is used by at least some of the vendors offering similar services. We at Spacelift took a different path, but I know some of the other vendors chose to implement it in their offerings. So that's one thing. Another thing is that dropping the support for it would suddenly make OpenTF incompatible with legacy Terraform, so I think the best course of action will be to treat it as a "cloud backend" - a brand-agnostic API that anyone can implement if they so wish.

In Terraform there are two similar concepts called TFC (Terraform Cloud) and TFE (Terraform Enterprise) where both implement the same API but TFC is the official SaaS cloud whereas TFE is the on-premise.
The TFC/TFE API implements the .well-known URI concept, so which means the client can discovery which services the cloud implements.
This is the services exposed by the TFC: https://app.terraform.io/.well-known/terraform.json
For integrating with TFE, the user can provide an alternative base URL, for example:
terraform login my.cloud.example.org

and if the domain implements the TFE API, Terraform just works.
The Terraform codebase makes use of the go-tfe library which abstracts the cloud type, so in principle, it could work with any third-party clouds.
Of course, TFC/TFE are Hashicorp products and probably there are important legal matters in this case. Still, maybe an interesting alternative would be if OpenTF also documents a community cloud API (which initially can be compatible with the MPL go-tfe SDK). Then we can keep all the cloud-related code in the fork but disable/remove the use of the TFC domain (Hashicorp default API domain). Have a look here.
Then all the Terraform Cloud customers can switch to opentf fork and the only change is that they will need to configure the Hashicorp domain (as people using TFE do).

TL;DR is that TFC users should still be able to use OpenTF as a drop-in replacement.
FYI I was handling that in this PR. This is a name change TBH, the remote backend remained unchanged, and the cloud backend uses vendor-agnostic language with no change in the functionality.
Removing TFC as the default for terraform login could make sense, though, because it gives privilege to a particular vendor.


Jkt/separate messaging

Closing in favour of https://github.com/duckduckgo/content-scope-scripts/pull/818


Add some progress printing to the test262 tests

Previously, these would run for several minutes without printing
any progress. Now, at least print the number of tests executed.

I like this patch, but time to time we ran into the log limit on Travis CI. Fortunately it looks like this change does not reach the limit, because the bots are green. However it would be great to add a quiet option to easily turn off unnecessary logs when it is needed.

The commit is not properly signed, which makes the Travis CI fail. As long as the PR is red (i.e., some checks fail), it may get less attention from reviewers because they know that it will still certainly change. So, it's better to fix CI issues as early as possible.

@akosthekiss: Oops, my bad! I didn't realize the CI builds were failing, thanks for the heads up :).
@LaszloLango: If the logs become too long, you can also adjust the space between logs. Right now it is 100, which should print about 115 lines. If that became too many, you can make it 200, which would only print 58 lines. Otherwise I am not sure the best way to add such a quiet option to the bash script (do you already have such a system in place for other scripts?).


Detect out-of-range array access

Bicep should be able to block the following:
var arrayWithTwoItems = [
  1
  2
]

var badAccess = arrayWithTwoItems[2]

This was implemented in #9724:


Running ASP.NET in IISExpress from command line

I am having trouble running my ASP.NET5 from command line under IISExpress.
Having browsed questions in here and on StackOverflow I thought I had made progress, but alas I haven't.
My current command line setup (thanks to this answer on SO) looks like so

iisexpress.exe /config:"[project_dir].vs\config\applicationhost.config" /site:"WebUI" /apppool:"Clr4IntegratedAppPool"

When I run this, IISExpress starts up, it even finds the correct port it should run under.  The main problem is every request returns a 502.3.
Looking at the contents of IISExpress\TraceLogFiles I see this error

ModuleName: httpPlatformHandler
Notification: EXECUTE_REQUEST_HANDLER
HttpStatus: 502
HttpReason: Bad Gateway
HttpSubStatus: 3
ErrorCode: The server is currently disabled. (0x8007053d)

What is causing this? Why can't I get this to run!?

From what I understood, no Application pool is used by ASP.NET 5 / ASP.NET Core 1.0. They have their own process. However, I haven't played particularly with IISExpress with this scenario.
What IIS (not sure about IIS Express) does at the moment is do a reverse proxy on the process being launched. Nothing is "hosted" in IIS. It just forwards the request. Even then it needs the HTTP Platform Handler plugin.
What are you trying to do? Are you hosting it through IIS Express? Is the "F5" experience not working in Visual Studio?

Hi Maxim, thanks for your response.
I thought that too, but when using "F5" in VS2015 everything just works.  When running in this way (under IISExpress still) I used Task Manager to grab the command line used to run the app which is how I ended up with the one I originally posted.
What I'm trying to do it get automated integration tests working.  We had them under the previous ASP.NET and im porting our code to ASP.NET5.  These integration tests run the website and all other services via command line then run using the C# Fluid API with Selenium.

If you are running integration tests, you should run them as close as possible to how they would be run in production. That means behind IIS itself. IISExpress was made for the good old development workflow.
If you are just testing stuff like routing, view generation, api calls... I would use a self-host.
If you are testing certificates, Windows Auth and other IIS dependent scenario, then I would go with a proper IIS.

Here's a test using our infrastructure for launching IIS Express: https://github.com/aspnet/IISIntegration/blob/dev/test/Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests/HelloWorldTest.cs

I am still interested in a solution to this.  I am trying to run multiple asp.net projects locally through IIS Express using command line. See related issue #1012.

Any progress on this? I need to run IISExpress at the command line because we're using JWT Bearer Authentication that doesn't work with the self hosted approach. We're using F# (at Jet.com), and the Visual Studio tooling for F# under with ASP.NET Core is a mess. Best bet is VS Code + CLI, but IISExpress is an issue... thoughts?

JWT Bearer should work fine for self-host. Please open a bug in Security to investigate that.

@Tratcher thanks for adjusting my focus - problem solved getting JWT Authentication working in the self-host, it was my bad. For me this obviates my need to run IISExpress from the command line. Though I still think it should be doable, and I assume from your response that it isn't yet. I'll keep my eye out for a solution...

It's possible, we do it in our functional tests, it's just not pretty. See https://github.com/aspnet/Hosting/blob/dev/src/Microsoft.AspNetCore.Server.Testing/Deployers/IISExpressDeployer.cs#L53

You're right. That's not pretty - but you've done the leg work, so I may just steal that. Thanks!

@Tratcher that link was broken. Here is the working one:
https://github.com/aspnet/Hosting/blob/dev/src/Microsoft.AspNetCore.Server.IntegrationTesting/Deployers/IISExpressDeployer.cs

Amehdi, Thanks for sharing. How do I use the IISExpressDeployer? I have many different applications in my solution and I don't want to open all my applications when I am doing the development testing. So I am looking for a way to start the IIS EXPRESS from the command line.
I tried "C:\Program Files\IIS Express\iisexpress" /config:C:\VS_TFS\InvestorPortal\Main\Source.vs\config\applicationhost.config /site:IP.IdentityServer but the application didn't work when I launch it from the browser.

Closing because no further action is planned for this issue.


Improve client compatibility

/release
Summary
Currently eth-duties only supports Teku and Lighthouse due to the fact the requests were not implemented following the official beacon api spec. This PR will add compatibility for:

Nimbus
Lodestar
Prysm

Note on Lodestar
I needed to add a specific change how requests are sent because Lodestar does not support URI percent-encoding currently. This was reported as bug in the Lodestar github repository. If it will be added/fixed, I will adapt eth-duties as well (back to usage of percent-encoding for all requests).
Update on Lodestar specifics
It turns out that Lodestar is the only client implementing percent-encoding correctly or using the right library for handling requests. The , is a reserved delimiting character which therefore should not be encoded. Fur further details please follow the discussion in the aforementioned issue in the Lodestar repository.
Closes #32

@nflaig Thanks for reviewing. I'm currently waiting for @LuisNaldo7 to finish the review on #40 . If this is done and I rebased the changes into this branch I would ask you to approve this PR which will then trigger a new release :raised_hands: .

@TobiWo sounds good, will give it a final review after rebase

@nflaig Sorry for the delay. This PR is now ready for review.

@nflaig Unfortunately a review is only valid when you have write access. Therefore, if you like, I would add you as collaborator so you are able to do valid reviews in the future. Don't know if you have time for that?

@TobiWo Thanks for the offer but I won't be able to find the time right now to familiarize myself with the code base to do proper reviews. That being said, feel free to just tag me on any PRs related to Lodestar or any other CL as well if you feel like my feedback would be useful.


XPath queries may start on ancestor-or-self::

Global XPath queries may not start with /

Investigate other possible axes: https://www.w3schools.com/xml/xpath_axes.asp


make slider work even when refreshed

Coverage decreased (-0.7%) to 77.155% when pulling b8f2aebad94cad78ddc45a55ed595bab230d7aae on aramin:bugfix/refresh-slider-caption-toggle into 4f078e0094e7421813e36c03c96ea09efebde1f9 on WeltN24:master.

Coverage increased (+4.9%) to 82.759% when pulling 0f88789467753c688240a6bdb9cc62a3100bcd1d on aramin:bugfix/refresh-slider-caption-toggle into 4f078e0094e7421813e36c03c96ea09efebde1f9 on WeltN24:master.


Website redirects to adware

The URL http://www.permabots.com redirects to a scam site that auto downloads an adware.
Please remove from project or update.

Good point. Legacy web permabots.com reference removed from README
thx


Change use of Mycroft JSON config file to a skill-only YAML configuration file

Using Config file with self.config in skills will end with the 19.08 release. Change so that it uses another file if users don't want to use home. The file will be ~/.mycroft/email-skill/email.yml. I will use yaml.
Before 19.08:

[ ] Define self.config  in the skill.

In the Longer Run:

[ ] Change code to use YAML file, over home settings, if existent.

[ ] Keep backwards compatibility with Config file

I might want to have the skill say if the user is using the old config file something like: You are using the old version of configuring the skill on the device. Please go to [insert url here] to learn more.

[ ] Update docs

[ ] Change the config file to the yaml file
[ ] Create a "moving" page which user will land on when directed by the skill (see above)

[ ] Promote change.

Found that I could get around this with the Mycroft Config. just define self.config  My self. However, I should just switch it to yaml because its is nicer.
I'm going to keep the bug label on this issue until I define self.config in the skill.

So what exactly do I have to do to get this skill to work? I've been fighting with this for the last couple days, so any help would be appreciated.

right now, the local configuration isn't working due to some changes made in Mycroft core. This is on my todo list.

Hi @JonahEmbry The skill has been updated to fix this problem, and has been submitted to the market place for review. Once it is accepted, you should use the new docs to configure the skill. Alternatively, just using the home.mycroft.ai settings has been verified to work.


ci(components): viewport size to improve ui tests performance

NICE!! there are 2 ong's that are missing: calendar-event: Safari popup: FF

not sure why they are missing. perhaps I just hit their exact width. up until now the viewports were different for each browser


Remove CODEOWNERS and mirror-rolling-to-master workflow.

They are both outdated and both no longer serving their intended purpose.

https://github.com/Mergifyio backport jazzy humble


Switch LaunchAgent to StartInterval

Hey Erik -
We inherited a custom LaunchAgent for Nudge, and are discussing how we'd want to modify it - whether we want to switch back to the default one, or further customize it to use StartInterval instead of StartCalendarInterval. We're already randomizing Nudge with the randomDelay key, but StartInterval would launch Nudge based on when the computer boots, not specifically on the hour and half-hour.
I couldn't find any discussion here or on Slack about the choice to use StartCalendarInterval instead of StartInterval. Do you think StartInterval would be an improvement? If so, I can submit a pull request that launches Nudge every 30 minutes, and update the wiki to describe the new behavior.
Thanks!
Mike

Not a bad idea. That might lead to more randomization though I can imagine some people wouldn't want that.
I would point to this on macadmin slack and ask people to chime in.

I have similar issues,  startinveral works after reboot.
Testing now startcalenderinterval

It's been tricky to build a consensus, but here's the discussion thus far on the MacAdmins Slack.
I think StartInterval makes more sense as a default LaunchAgent. As mentioned in the thread, if users prefer the current behavior, they can always ship a custom LaunchAgent. I think James Kane's explanation is the easiest for new users to grasp:

Nudge will check if you’re up-to-date every 30 minutes after the initial pop up.

Confusion around the current :00 and :30 behavior comes up occasionally in the channel, so hopefully this would reduce some of the support questions that arise.
@sjha967 If you need help, please join the #nudge or #nudge-jamf channels on the MacAdmins Slack.

I'll add my vote for StartInterval.  I've never seen a LaunchAgent or Daemon fire exactly on the top or bottom of the hour.  There's already enough entropy introduced by the user's startup time.  And Nudge can add additional randomness with the random delay setting.  That should be enough to scatter the notifications across the fleet at different times.

After reading the conversation on Slack, I'm thinking for now to keep it as is. I already merged something for the next release of Nudge that will update the LaunchAgent and I don't think there is consensus.
I think having consistency as the default is a good thing and if someone doesn't like that consistency they can just use randomDelay or ship their own launch agent.
Changing the LaunchAgent now will lead to wider confusion as the behavior is somewhat of a breaking change. Since more admins are less technical, I'd rather not do this - at least not for now.

Sounds good. Thanks for considering this! We'll resume the discussion around modifying our own LaunchAgent, then.


Prototype loading privacy declarations directly from source code

The separate system declarations are a potential burden for users. A good middle-ground between code analysis and what we have now is to co-locate the declarations and the code.
The two implementation methods I can think of for the POC are as follows:

A very python-specific implementation where we ingest the python code, extract the docstrings and then extract the system declarations from there

A major issue here is that this is not generalizable to other languages

We go for a more general approach, and treat each source code file as a txt file. We then use regex to look for matching cases and attempt to load it into a system declaration

Because we would still expect it to be yaml-like, this would only work in languages with multi-line comments

Option 1: Declaration inside of the docstring
def some_func(some_parameter: str) -> None:
    """
    Do something important with user data.

    system:
      - fides_key: demo_analytics_system
        name: Demo Analytics System
        description: A system used for analyzing customer behaviour.
        system_type: Service
        privacy_declarations:
          - name: Analyze customer behaviour for improvements.
            data_categories:
              - user.provided.identifiable.contact
              - user.derived.identifiable.device.cookie_id
            data_use: improve.system
            data_subjects:
              - customer
            data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
            dataset_references:
              - demo_users_dataset
    """"

    user_data = get_user_data(some_parameter)
    advertise_to(user_data)

Option 2: Declaration as a multi-line comment:
"""
system:
  - fides_key: demo_analytics_system
    name: Demo Analytics System
    description: A system used for analyzing customer behaviour.
    system_type: Service
    privacy_declarations:
      - name: Analyze customer behaviour for improvements.
        data_categories:
          - user.provided.identifiable.contact
          - user.derived.identifiable.device.cookie_id
        data_use: improve.system
        data_subjects:
          - customer
        data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
        dataset_references:
          - demo_users_dataset
"""

def some_func(some_parameter: str) -> None:
    """
    Do something important with user data.
    """

    user_data = get_user_data(some_parameter)
    advertise_to(user_data)

An additional caveat here is that it would be extremely difficult if not impossible for a plugin to help with these annotations, as they're embedded in other source code.
Additional questions to think about:

Do we have the user define a system in a system.yaml file, and then attribute all of the nearby code declarations to that?
Do they need to define a system-per-declaration? that seems weird, so this ^ option seems better
How should this be handled during evaluations? Should it be done at apply/evaluate time, or should there be a separate command that generates a full system.yaml file from the source code declarations?

I think a generalized option is the most valuable path forward. Fidesctl isn't meant to be a Python-specific tool, and so it should support all languages. @ThomasLaPiana your second suggestion is a good first step, and we can decide if it's worth working through an implementation of the feature using that format, but the drawbacks you mentioned are significant.
What if we instead borrowed from other static analysis tools? Users could create the same detailed markup using a single-line comment syntax that we define. Your example above could look like:
# fidesctl-system-declaration start
#
# fides_key: demo_analytics_system
# name: Demo Analytics System
# description: A system used for analyzing customer behavior.
# system_type: Service
# privacy_declaration_1_name: Analyze customer behavior for improvements.
# privacy_declaration_1_data_categories: [user.provided.identifiable.contact, user.derived.identifiable.device.cookie_id]
# privacy_declaration_1_data_use: improve.system
# privacy_declaration_1_data_subjects: [customer]
# privacy_declaration_1_data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
# privacy_declaration_1_dataset_references: [demo_users_dataset]
#
# fidesctl-system-declaration end

def some_func(some_parameter: str) -> None:
    """
    Do something important with user data.
    """

    user_data = get_user_data(some_parameter)
    advertise_to(user_data)

This is just a first pass. It's fairly verbose, and tbh I'm not sure either solution really improves the user experience vs. authoring YAML files.
Another idea could be to enforce file-scoped markup. This has the drawback of prescribing how users structure their projects' files, but improves things by allowing annotations to live closer to the code that they annotate. I'm thinking in the context of dataset annotations describing a Postgres DB. The annotations could live in a schema file, like you would see in a Rails app, or in a DB setup SQL script like you might see elsewhere. For example, in a Rails schema.rb file, you could have:
# fidesctl-dataset todos_db "ToDo Database" "Stores ToDo items created by users"
ActiveRecord::Schema.define(version: 2015_05_19_181601) do

  # fidesctl-dataset-collection todos "ToDo items"
  create_table "todos", force: :cascade do |t|
	# fidesctl-dataset-field title "ToDo item's title"
    t.string "title"
	# fidesctl-dataset-field is_completed "ToDo item completion state"
    t.boolean "is_completed", default: false
	# fidesctl-dataset-field created_at "ToDo item creation timestamp" [system.operations]
    t.datetime "created_at", null: false
    t.datetime "updated_at", null: false
    t.string "session_user_id"
  end

end

This is an incomplete example, but it's already verbose in its own right. In general, a first pass could be to create a few single-line comment "functions" that take "arguments" at specific indices. In a more robust implementation, we could infer a lot of the argument values from the code's structure. This has the huge benefit of being "compilable" (the comments could be used to create YAML files like we expect users to author today). It enables us to simultaneously support inline annotations and manifest file annotations, which I think is critical and would be tricky to do otherwise.

Do we have the user define a system in a system.yaml file, and then attribute all of the nearby code declarations to that?
Do they need to define a system-per-declaration? that seems weird, so this ^ option seems better

First thought here is that it makes the most sense for the system to already exist outside of each code declaration. What about giving the option of referencing an existing system within the declaration?
privacy_declarations:
  - name: Analyze customer behaviour for improvements.
    data_categories:
      - user.provided.identifiable.contact
      - user.derived.identifiable.device.cookie_id
    data_use: improve.system
    data_subjects:
      - customer
    data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
    dataset_references:
      - demo_users_dataset
    *system_reference: system1*

For simplicity it would be nice to keep the same exact model, so it could just be a field outside of the declarations also:
*system_reference: system1*
privacy_declarations:
  - name: Analyze customer behaviour for improvements.
    data_categories:
      - user.provided.identifiable.contact
      - user.derived.identifiable.device.cookie_id
    data_use: improve.system
    data_subjects:
      - customer
    data_qualifier: aggregated.anonymized.unlinked_pseudonymized.pseudonymized.identified
    dataset_references:
      - demo_users_dataset

The project having a base system.yaml file which declarations are added to seems okay to me also. Not sure if there will be complexities around finding this file though(will a config be needed to locate system.yaml?).

This is just a first pass. It's fairly verbose, and tbh I'm not sure either solution really improves the user experience vs. authoring YAML files.

This is along the lines of what I am thinking. Based on the feedback we've gotten, user's want to do as little as possible but still be able to track their data usages. Even learning the fides model can be a turn off and asking them to create these blocks in their code is just shifting the pain. @PSalant726 mentioned single line comments and that makes the most sense to me as a way to make this feel better. A function could just have these three lines and we could create a declaration out of it:
""" or #
@fides.data_categories: [ "user.provided.identifiable.contact", "user.derived.identifiable.device.cookie_id"]
@fides.data_use: "improve.system"
@fides.data_subjects: ["customer"]

Maybe this defeats the purpose of this issue since we are looking for a middle ground, but I am on the fence about how useful the yaml definitions in code will be.

The first decision we make should answer the question "how do we want to address the tester feedback?" We can either:

Take the smallest possible step forward and double-down on YAML declarations, simply move the location of that YAML from dedicated manifest files to inline comments in source code, and continue to iterate on that design over time until we reach a reasonable compromise
Treat this like a fundamental redesign of the developer interface with fidesctl, design a system from scratch that abstracts away the most cumbersome parts of annotating your systems and datasets, and implement/maintain robust tooling to support both first-time and veteran users that want to integrate fidesctl into large and small projects

Both approaches have their merits and their drawbacks. Personally, I'm in favor of the latter, but I fully acknowledge that it represents a tremendous effort.
FWIW - I never felt like this feedback was worth addressing in the first place. IMO, implementing fidesctl is like implementing anything else; you have to learn a new system, make a large one-time investment to get it implemented, and then make small changes over time to maintain that implementation. Having dedicated YAML files feels like the cleanest compromise between allowing fidesctl to maintain a strong hold on the project it annotates/monitors, while staying out of the way and not cluttering the project's code. If testers are complaining about writing YAML, then maybe they aren't thinking deeply about the ramifications of a version of fidesctl without writing YAML. It might be enough for us to consider how we can cause fidesctl evaluate to fail when a new system has been added, or a dataset modified, but changes haven't been made to any manifests.

I want to hone back in on one of the potential problems this is solving. While it isn't going to "solve" the problem of them writing the declarations, having the declaration co-located with the code itself has enormous benefits and by itself does reduce friction.
If I went to a random project that was using fidesctl, and I opened their system manifest file I could see all of the declarations, but I would have no idea where any of that functionality actually lives within the codebase. Getting the declarations as close to the code as we can is valuable.
By having it live with the code itself, I can easily see exactly what is going on in that function from a privacy perspective. It is also much easier to remember to update a declaration if it is right next to the function it describes, as opposed to having to go to a system manifest file, figure out which declaration describes that function, and then update it.
I think the solution here is to support both approaches. People should be able to choose whether they want to write the yaml files or if they want to embed the definitions in code. I am completely ok with having the code definitions look different and not follow the exact same syntax as the yaml. I think the following format suggested by @earmenda & @PSalant726 makes perfect sense:
Multi-Line Comment
"""
@start_privacy_declaration
@parent_system: system1*
@name: Analyze Customer Behaviour
@data_categories: [user.provided.identifiable.contact, user.derived.identifiable.decide]
@<other fields>
@end_privacy_declaration
"""
def some_func():
    pass

Multiple Single-Line Comments
#@start_privacy_declaration
#@parent_system: system1*
#@name: Analyze Customer Behaviour
#@data_categories: [user.provided.identifiable.contact, user.derived.identifiable.decide]
#@<other fields>
#@end_privacy_declaration
def some_func():
    pass

YAML in code comments feels pretty wrong, so the simplified format above makes more sense
We can then also support generating yaml files from these definitions. (once they are loaded into the right model, we already have the code to create yaml from that)
@PSalant726 I push back on the idea that this feedback isn't worth addressing. We can't completely remove the overhead of fidesctl for our users, but there are still plenty of places where we can make a better user experience. This definitely falls under that umbrella

I see the value in colocated annotations, but it's still predicated on the assumption that humans should be writing them vs. simply reviewing and maintaining them. Maybe it's just my opinion that they shouldn't be handled that way? Nevertheless, I still believe that the real value that we can provide as the core team maintaining fidesctl is to better enable programmatic creation and maintenance of annotations through enhancements to the CLI. We can improve the discoverability and functionality of the generate-dataset and annotate-dataset commands, add new commands to handle guided creation of system & policy annotations + updates to existing dataset annotations, and introduce static analysis tooling that can call attention to outdated annotations.
There's no reason why the colocation features can't/shouldn't exist, but to me it feels like a red herring.

@PSalant726 I agree there are other places we can also do work to ease users into using fidesctl, and we can/will open issues for those pain points. But to refocus on the question at hand, we do want the ability to put the declarations in the code, so this POC is still valid and worth doing. This is a low-risk first iteration to get a feel for how this looks/feels, and we can go from there. It's a prototype, and if it turns out that it doesn't feel right or we want to go in a different direction altogether, we can do that. But we need to have something to play around with first to make that decision.
That all being said, are we in agreement on how those annotations should look? It seems like we're all on the same page there and now its down to implementing it

Should this work include an implementation of the multi-line comment syntax? Given that we know single-line comments are supported by all languages, I think it would be simplest to start with the single-line syntax only, and determine if support for multi-line comments is worth the additional complexity later on.
Otherwise, I'm in agreement.

Let's do the smallest possible iteration and stick with single-line comments in Python as our POC

Using Python as an example- is there typically really a very clear place in the code where a system declaration would come from? Remember the system declaration specifies:

what the system is (e.g. some kind of web app, or database)
what the data uses are (e.g. provide the product, perform analytics)
what data categories are needed for those data uses (e.g. emails, device IDs)

For this project, for example, is there a single source file where it'd be clear where those comments would live? Would it be...

https://github.com/ethyca/fides/blob/main/fidesctl/src/fidesapi/main.py
https://github.com/ethyca/fides/blob/main/fidesctl/src/fidesctl/cli/cli.py
others?

Alternatively, we might find it a lot more obvious to switch gears a bit and do inline dataset declarations instead of systems? Feels like that might be this file: https://github.com/ethyca/fides/blob/main/fidesctl/src/fidesapi/sql_models.py

One other potential option where the system declaration might feel "natural" is the package setup for pip/etc.: https://github.com/ethyca/fides/blob/main/fidesctl/setup.py?
In other words, what I'm saying is that I'm currently not optimistic that "inline" system declarations will feel any better than their YAML equivalent. This is because I believe the general "work" of a system declaration is modeling the reason why the system exists, which is something that source code fundamentally doesn't do. I can't think of many examples where you'd be writing source code that says "this is an ecommerce app, so we need to collect someone's address so we can deliver their order to them". Since we can't infer this, there's no way to make that magical.
However, I do think that we could (in theory) reasonably infer what data is being used by the source code by analyzing the source. That's why it might feel more natural to do a dataset declaration inline (which makes the system declaration easier to write, as you can just reference the dataset there).
Regardless, I do think we should just "try something" and see how it feels; treat it as experimental, try a couple options out in practice, then rework it for a final implementation that we want to live with. Whether we do inline system declarations or inline dataset declarations we'll still need to decide some kind of syntax for how to parse those.

The reason for trying to do privacy declarations (note the system declaration would still happen elsewhere, and the privacy declarations in the source code would be attributed to a system defined elsewhere) is my thinking that you could write a privacy declaration for each function that is making calls out to external systems (data leaving or coming into the system).
Dataset declarations within the source code is also a cool idea, and very feasible since most projects will have these explicit models written out

When trying to get to the heart of the issue of maintainability vs. "getting started", we came up with some of the following ideas:

Implementing a "generate system" and "generate policy" CLI command
More robust documentation around maintaining fidesctl, best practices, identify data uses
Adding a "pointer" in source code that describes which privacy declaration describes it
Inline dataset declarations

Most specifically relevant to item (2) above, opened issues for additional documentation:

#269
#268
#270

Re 3, I can see the utility of that kind of "backpointer" comment reference...
Imagine a system declaration like:
system:
  - fides_key: user_mgmt_service
    name: User Management Service
    privacy_declarations:
      - name: Account Login
         ...
      - name: Data Sharing
         ...
      - name: Support Chat
         ...

Imagine that this service implements a couple different data uses: the ability to login, the option to share data with some 3rd party integration, and the option to initiate a user support chat, etc. Those are all different types of data uses that would have involve different data categories.
It might be helpful to be able to "tag" in the source code the parts of the code that implement each of those declarations, e.g.
# user_login.py

def login(email, password):
  '''Logs in the user by validating their credentials. See @fides "Account Login" for privacy policy'''

In theory, the tagging can make it easier to be reminded to update the login declaration when changes are made to that file.

@NevilleS for the sake of simplicity, wouldn't it actually be easier to put that in the system declaration itself? Then we could instantly parse it out of the system file and check the path for any changes.
If we go the route of having to parse source code for comments, that's a lot more effort for functionally the same thing, since I don't think the subtle nuance of having them in the source vs. the declaration makes a huge difference in functionality.
If we have the source code pointer in the system declaration file, it is also very easy to see which declarations are missing source code pointers

I think the ideal outcome is being able to report (ideally during a scheduled automated CI/CD step) when new data handling code arrives that isn't covered. If that can be automated (sort of like how the Python coverage command handles unit test coverage) then that's the best case.
An 80% as good alternative is to require a standard comment or decorator in the source code so that when developers copy/paste everything from one module into others, it at least becomes apparent that the comment or decorator is being copy/pasted and re-training can happen (and the changes reviewed) when that is discovered.

@edthedev With the feature I proposed, the coverage report would show how many of your system declarations had associated code files. The new system declaration would look like this
system:
  - fides_key: fidesctl_system
    name: Fidesctl System
    code_paths:
      - src/some_code_file.py
      - src/another_code_file.py

This method has the benefit of being language agnostic, and we can then throw errors for when the code_paths section is empty. We could also move it down into the declarations section

closing this as the discussion is stale and we've determined some other ways forward here
starting with the #341 and the general infra scanning project, will revisit as needed!


Discussion: Keep the People app?

It used to be trivial to reinstall the People app for those few time is proved necessary (someone wanted to use Mail, Calendar, People instead of Outlook).
However, Microsoft is turning the People app into a background app, accessed only through the Mail, Calendar app and through the icon on the taskbar, and has removed the People app from the store.
I can't figure out a way to reinstall it anymore.
Maybe it shouldn't be removed anymore? Or make its removal a toggle? I know the GUI script has a whitelist, so maybe just a toggle for the SysPrep script? And maybe print a warning for apps that are impossible to reinstall?

I suggest you simply look up an external way to reinstall it. Most people do not have use for this app, so it's incredibly rare for someone to keep it. It's a debloater for a reason, and I personally consider People to be bloat. Anyways, close this issue, please. Modify the script if you want People to remain.


BUG can't add blocks to page

When you install the elemental module and add the page extension to a page, you can't add any blocks until you save the page. This is because until you save the page, the ElementalArea doesn't exist.
In v3 you're able to add the block, but then its not attached back to the page even though its saved in the database.
In v4 the loading indicator displays in the block area and gets stuck, preventing you from adding any blocks.
cc @sachajudd

Have you tried the latest 4.0.x branch? We patched this yesterday: #496 . It hasn't been merged into the 4.x branch yet.
As long as you dev/build there should always be an area on your pages - and you have to dev/build after adding the extension.


The times that are booked needs to be greyed out in CupertinoDatePicker

Hi,
I have used cupertinoDatePicker mode time inside  alertDialog.I want The time should be a list as shown. With the times that are booked needs to be greyed out.like as given below please given any
reference.

Hi @mahes-waran
This platform is not meant for assistance on personal code.
Please see https://flutter.dev/community for resources and asking questions like this,
you may also get some help if you post it on Stack Overflow.
Closing, as this isn't an issue with Flutter itself,
if you disagree please write in the comments and I will reopen it
Thank you

Hi VladyslavBondarenko,
thanks to  you replied for me.I need Time picker plugin or widget support platform android and ios. that picker display times for available times and booked times(should not clickable and grey color).I have  already searching but   i can not so please given any idea to customize time picker.

Related to https://github.com/flutter/flutter/issues/23717, https://github.com/flutter/flutter/issues/56987

Hi Vladyslav Bondarenko,
Thank you very much replied for me.
On Fri, May 15, 2020 at 12:26 PM Vladyslav Bondarenko <
<EMAIL_ADDRESS>wrote:

Related to #23717 https://github.com/flutter/flutter/issues/23717,
#56987 https://github.com/flutter/flutter/issues/56987
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/57208#issuecomment-629065112,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJFV5SEDX3QUMI5K36TUDHLRRTRRBANCNFSM4NAVDOWQ
.

I believe this is already possible with minimumDate and maximumDate, in CupertinoDatePickerMode.time mode. The documentation is a bit misleading.

Hi LongCatlsLoong,
Thanks for you replied me
On Tue, May 19, 2020, 2:49 AM LongCatIsLooong<EMAIL_ADDRESS>wrote:

I believe this is already possible with minimumDate and maximumDate, in
CupertinoDatePickerMode.time mode. The documentation is a bit misleading.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/flutter/flutter/issues/57208#issuecomment-630440844,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJFV5SBDOP2MLDG45NDBG7TRSGQ6FANCNFSM4NAVDOWQ
.


Roll engine 4893b0760df0..c6baaaf75a76 (13 commits)

https://github.com/flutter/engine/compare/4893b0760df0...c6baaaf75a76
git log 4893b0760df0..c6baaaf75a76 --date=short --no-merges --format='%%ad %%ae %%s'
2018-08-05<EMAIL_ADDRESS>Replace acquire+release thread annotation with excludes (flutter/engine#5944)
2018-08-05<EMAIL_ADDRESS>Roll src/third_party/skia 59aabbcf3b0d..2e77f54f46e8 (1 commits) (flutter/engine#5943)
2018-08-03<EMAIL_ADDRESS>Roll src/third_party/skia 71fe8baccc01..59aabbcf3b0d (1 commits) (flutter/engine#5942)
2018-08-03<EMAIL_ADDRESS>Roll src/third_party/skia 4c3b63e289c5..71fe8baccc01 (3 commits) (flutter/engine#5940)
2018-08-03<EMAIL_ADDRESS>Call drawPaint instead of drawPath if there's clip (flutter/engine#5937)
2018-08-03<EMAIL_ADDRESS>Roll src/third_party/skia cdfa16d086b8..4c3b63e289c5 (9 commits) (flutter/engine#5936)
2018-08-03<EMAIL_ADDRESS>Roll src/third_party/skia 1400d38e0863..cdfa16d086b8 (10 commits) (flutter/engine#5934)
2018-08-03<EMAIL_ADDRESS>Flush all embedded Android views on hot restart. (flutter/engine#5929)
2018-08-03<EMAIL_ADDRESS>Roll src/third_party/skia 578ef2847b72..1400d38e0863 (7 commits) (flutter/engine#5933)
2018-08-02<EMAIL_ADDRESS>Don't drop MotionEvents with unknown tool type. (flutter/engine#5931)
2018-08-02<EMAIL_ADDRESS>Roll src/third_party/skia 64cc576b1fa7..578ef2847b72 (20 commits) (flutter/engine#5930)
2018-08-02<EMAIL_ADDRESS>Roll Dart to b04def964c428ada007cca7ef6b4936001db965d (flutter/engine#5928)
2018-08-02<EMAIL_ADDRESS>Add an explicit -[FlutterViewController init] implementation (flutter/engine#5924)
The AutoRoll server is located here: https://flutter-engine-flutter-roll.skia.org
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff, who should
be CC'd on the roll, and stop the roller if necessary.

@rmistry @liyuqian : New roll has been created by https://flutter-engine-flutter-roll.skia.org

Commit queue failed; closing this roll.


Access to File init

@brightsider
Hi.
https://github.com/1amageek/Pring/blob/master/Pring/File.swift#L121
Files can now have additional data.

Thanks for your PR!!
I am looking forward to next PR

@1amageek make init public, please

You are mistaking your usage.
Front and Back should be defined separately.

I have not disclosed the initializer to prevent such usage.


Debug drawing sphere

Fixes #313

SonarQube analysis reported 2 issues
Note: The following issues were found on lines that were not modified in the pull request. Because these issues can't be reported as line comments, they are summarized here:

 AncientFungus:LGEPR: Cppcheck cannot find all the include files (use --check-config for details)
 ScriptManager.cpp#L529: The function 'GetScripts' is never used.


macOS API Documentation

Hi! I am really interested in the development of alternative windowing systems in macOS but have always hit walls in my own projects since there are no publicly documented APIs.
Is there some place where one can find out more about the APIs used to create chunkwm

Most of the documents I've been reading are legacy references. A lof of the stuff written do not apply anymore and so you have to be able to filter and look for the important things that are still available today.
The publicly documented API's that I use in chunkwm is more or less the following:
NSWorkspace (SharedWorkspace ??)
Carbon
CoreGraphics
ApplicationServices
Probably other stuff,
See here https://developer.apple.com/documentation/applicationservices
I don't remember where I found my other stuff, but I have saved some pdfs.
In short, you need curiosity and persistence to keep trying and experimenting.
There is not a whole lot of docs out there in this area.


simplify request input convertation

Continuing #216
I've found out that tests for Node.js Url and WHATWG's URL are present.
But I propose to simplify input convertation regarding to template literals specification

The string conversion semantics applied to the Expression value are like String.prototype.concat rather than the + operator.

Current coverage is 100% (diff: 100%)

Merging #217 into v2 will not change coverage

@@            v2   #217   diff @@
=================================
  Files        6      6
  Lines      397    395     -2
  Methods     53     53
  Messages     0      0
  Branches   115    115
=================================
- Hits       397    395     -2
  Misses       0      0
  Partials     0      0

Powered by Codecov. Last update 29e9f5e...fff04c6

The template literal change is fine.
The removal of if statement is not, since input can technically be null, in which case it should be stringified.
Please just convert the + '' to template literals, and leave the if statement.

Closed by 7970491.


fix(regsitry): User should be able to pull the image from internal registry

fixes #894

there's a typo in the commit title: should be "registry". This won't impact us in any way, right?

[Current coverage][cc-pull] is 85.71% (diff: 100%)

Merging [#912][cc-pull] into [master][cc-base-branch] will decrease coverage by <.01%

@@             master       #912   diff @@
==========================================
  Files            30         30
  Lines          3012       3011     -1
  Methods           0          0
  Messages          0          0
  Branches        502        502
==========================================
- Hits           2582       2581     -1
  Misses          287        287
  Partials        143        143

Powered by Codecov. Last update [ecfcea6...3dfa0e4][cc-compare]
[cc-base-branch]: https://codecov.io/gh/deis/controller/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/deis/controller/compare/ecfcea6129e5ddced353aaa369cfd656f733d9bd...3dfa0e4008def819f8600f70744a3709f35f0320?src=pr
[cc-pull]: https://codecov.io/gh/deis/controller/pull/912?src=pr

Could you amend the second commit title? Maybe:
tests(build): fix for internal registry access

all the tests are passing https://ci.deis.io/job/workflow-test-pr/4108/console

Doesn't look like Jenkins agrees with that ^ ;)


WHIP endpoint fails on CORS preflight requests

I would like to develop an app allowing for ingesting the data from the user camera into my go2rtc server. I find the software amazing and extremely useful and I have managed to set it up for uploading my camera feed from OBS using WHIP.
However, the WHIP endpoint does not seem to work in the browser, which is IMO the primary purpose of using it. In particular, the POST request that sends the SDP offer is never sent, because the server does not properly respond to the preflight OPTIONS request, yielding the following error in the console:

Access to fetch at 'http://localhost:1984/api/webrtc?dst=camera1' from origin 'http://localhost:8000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

For testing, I tried the Allow CORS extension, but it just produces another error:

Access to fetch at 'http://localhost:1984/api/webrtc?dst=camera1' from origin 'https://web.whip.eyevinn.technology' has been blocked by CORS policy: Request header field authorization is not allowed by Access-Control-Allow-Headers in preflight response.

What I tried is two example WHIP client implementations, available in a release here as well as another one available also as an online demo.
According to the linked WHIP/WISH spec (last paragraph of sesction 4.2):

The WHIP endpoints MUST support OPTIONS requests for Cross-Origin
Resource Sharing (CORS) as defined in [FETCH].  The "200 OK" response
to any OPTIONS request SHOULD include an "Accept-Post" header with a
media type value of "application/sdp" as per [W3C.REC-ldp-20150226].

My config
Pretty simple, no matter what I set in the cors option, the error stays the same:
api:
  listen: ":1984"
  cors: "*"

streams:
  camera1:

webrtc:
  listen: ":8555"
  candidates:
    - stun:8555

Minimal working example
I guess if the demo is able to work properly, then it will be fine. I would like to help, but have zero knowledge of Go unfortunately.

Check docs about API module
https://github.com/AlexxIT/go2rtc#module-api

I've read it several times, did not find anything CORS related apart from the setting that I already have. Could you please be more specific?

origin: "*"        # default "", allow CORS requests (only * supported)

As I said and can be seen in the included config, I already have this setting. Both the asterisk and default option do not work as they do not change the fact that the backend doesnt handle CORS preflight requests, i.e. HTTP requests of type OPTIONS

The WHIP standard requires dispatching a POST request, while your CORS solution only works for GET requests. Other types require preflight handling with additional headers, implementing the functionality that I quoted in my original comment.
Perhaps this package might be of help?
https://github.com/rs/cors

You don't have this option in your config.

Oh damn. You're right sorry for the confusion. It seems to work now.


Terms Filter with empty list compiles to nothing

ElasticSearch v2
Problem
I have this method, that when I pass in an empty string, the resulting json sent to Elasticsearch has no filter.
C# Method:
public Func<FilterDescriptor<ElasticSearchEmployee>, FilterContainer> GenerateFilter(string x)
{
    return f => f.And(
f2 => f2.Name(Filters.UnitView).Terms(a => a.UnitScopes, !x.SplitByComma().Any() ? new List<string> { string.Empty } : x.SplitByComma()))
}

public static IEnumerable<string> SplitByComma(this string value)
{
    return value.Split(new[] { "," }, StringSplitOptions.RemoveEmptyEntries);
}

Parameters expected are: "1,3,4". Any combination of integer values deliminated by a string
ElasticSearch Query:
"query": {
    "filtered": {
      "filter": {
        "and": {
          "filters": [
            { }
          ]
        }
      }
    }
  }

I would have expected to see this as the ElasticSearch Query:
"query": {
    "filtered": {
      "filter": {
        "and": {
          "filters": [
            { terms: { "UnitScopes": [ "" ]  }  }
          ]
        }
      }
    }
  }

Justification
UnitScopes is a scope of control field, it contains id's of the things a user is allowed to view. So if no scopes are passed in, then I would expect the result not to return anything. Now ElasticSearch does this properly and respects the empty terms giving the result I would expect. Nest on the other hand tries to be too clever by removing the field completely.
Conclusion
I can see why Nest does this - seems like an optimisation thing. But it is rather unexpected (principle of least astonishment).

This is a feature called conditionless queries kicking in see also:
http://nest.azurewebsites.net/nest/writing-queries.html
You can get the behaviour you want by specifying .Verbatim() on the terms query.
Is this using NEST 1.x or NEST 2.0 ?

closing this due to inactivity and appearing to be intended behaviour.


error " raise RuntimeError('Subprocess error!') "

Hi, Atfer I import features extracted by sosnet, runing run.py, it shows errors"    raise RuntimeError('Subprocess error!')".Thanks

That's very cryptic. Can you post the entire error here?

Traceback (most recent call last):
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 272, in call
return self.fn(*self.args, **self.kwargs)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel_backends.py", line 608, in call
return self.func(*args, **kwargs)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 256, in call
for func, args, kwargs in self.items]
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 256, in
for func, args, kwargs in self.items]
File "compute_match.py", line 56, in compute_matches
kps2)
File "/home/pcl/image-matching-benchmark/methods/feature_matching/nn.py", line 95, in match
matches = bf.knnMatch(desc1, desc2, k=max(2, num_nn + 1))
cv2.error: OpenCV(3.4.2) /io/opencv/modules/flann/src/miniflann.cpp:315: error: (-210:Unsupported format or combination of formats) in function 'buildIndex'

type=1021

"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "compute_match.py", line 128, in
main(cfg)
File "compute_match.py", line 97, in main
for pair in tqdm(pairs))
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 1017, in call
self.retrieve()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 909, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 562, in wrap_future_result
return future.result(timeout=timeout)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self.exception
cv2.error: OpenCV(3.4.2) /io/opencv/modules/flann/src/miniflann.cpp:315: error: (-210:Unsupported format or combination of formats) in function 'buildIndex'

type=1021

interactive_res: CompletedProcess(args=['bash', './jobs/28e35d3e74643f776f1f66146ac626ed109e431d9c9f62eaa82fc19cf01bb7ae'], returncode=1)
Traceback (most recent call last):
File "run.py", line 266, in
main(cfg)
File "run.py", line 204, in main
job_dict)
File "run.py", line 48, in create_eval_jobs
job = create_and_queue_jobs(cmd_list, cfg, dep_str)
File "/home/pcl/image-matching-benchmark/utils/queue_helper.py", line 316, in create_and_queue_jobs
cpu=cpu)
File "/home/pcl/image-matching-benchmark/utils/queue_helper.py", line 164, in queue_job
raise RuntimeError('Subprocess error!')
RuntimeError: Subprocess error!

Seems like the same error as #19?

I only changed the config_common of the JSON file, my changes are as following:
"config_common": {
"json_label": "sosNet-upright-2k-degensac-inlier-th-0.1",
"keypoint": "sosNet-def",
"descriptor": "sosNet-upright",
"num_keypoints": 2048,
}

Can you try disabling FLANN? (Two lines in the JSON).

It doesn't work.
`Traceback (most recent call last):
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 418, in _process_worker
r = call_item()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/externals/loky/process_executor.py", line 272, in call
return self.fn(*self.args, **self.kwargs)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 608, in call
return self.func(*args, **kwargs)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 256, in call
for func, args, kwargs in self.items]
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 256, in
for func, args, kwargs in self.items]
File "compute_match.py", line 56, in compute_matches
kps2)
File "/home/pcl/image-matching-benchmark/methods/feature_matching/nn.py", line 95, in match
matches = bf.knnMatch(desc1, desc2, k=max(2, num_nn + 1))
cv2.error: OpenCV(3.4.2) /io/opencv/modules/core/src/batch_distance.cpp:238: error: (-215:Assertion failed) type == src2.type() && src1.cols == src2.cols && (type == 5 || type == 0) in function 'batchDistance'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "compute_match.py", line 128, in
main(cfg)
File "compute_match.py", line 97, in main
for pair in tqdm(pairs))
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 1017, in call
self.retrieve()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/parallel.py", line 909, in retrieve
self._output.extend(job.get(timeout=self.timeout))
File "/home/pcl/anaconda3/envs/match/lib/python3.6/site-packages/joblib/_parallel_backends.py", line 562, in wrap_future_result
return future.result(timeout=timeout)
File "/home/pcl/anaconda3/envs/match/lib/python3.6/concurrent/futures/_base.py", line 432, in result
return self.__get_result()
File "/home/pcl/anaconda3/envs/match/lib/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
cv2.error: OpenCV(3.4.2) /io/opencv/modules/core/src/batch_distance.cpp:238: error: (-215:Assertion failed) type == src2.type() && src1.cols == src2.cols && (type == 5 || type == 0) in function 'batchDistance'

interactive_res: CompletedProcess(args=['bash', './jobs/55492b573ad9cecf0fe814c6499a7f7e3b6e68969ba4594147dc4ecec6d8c8e0'], returncode=1)
Traceback (most recent call last):
File "run.py", line 266, in
main(cfg)
File "run.py", line 204, in main
job_dict)
File "run.py", line 48, in create_eval_jobs
job = create_and_queue_jobs(cmd_list, cfg, dep_str)
File "/home/pcl/image-matching-benchmark/utils/queue_helper.py", line 316, in create_and_queue_jobs
cpu=cpu)
File "/home/pcl/image-matching-benchmark/utils/queue_helper.py", line 164, in queue_job
raise RuntimeError('Subprocess error!')
RuntimeError: Subprocess error!
`

Seems like there's an issue with how the descriptors are being passed? Have you checked that they are correct?


feat: move requests from execution payload to beacon block body

Changes
Containers

Remove deposit, withdrawal and consolidation requests from execution payload
Define new container ExecutionRequests to encapsulate the three requests
Add ExecutionRequests to beacon block body
Update Light client containers to use Deneb execution payload

APIs

Update definition of engine_newPayloadV4 and engine_getPayloadV4 to include execution requests
Remove  engine_getPayloadBodiesByRangeV2 and engine_getPayloadBodiesByHashV2

Light Client

Update fork upgrade flow to discard the deposit/withdrawal/consolidation requests from payload

Spec

Update spec test version to v1.5.0-alpha.6
Disable electra slashing related tests and to be enabled in #7071

Waiting for next spec test release and ethereum/execution-apis#587 to be finalized
This PR does not contain update to the builder flow. Will update it in the next PR.

:tada: This PR is included in v1.23.0 :tada: