External subscribers to multicast feed generated in Openstack fails if no internal VM is subscribed first

Commit d02fb560aa35c4931d40000e8ca4c0e4105299b7 (1)(2) to Neutron disabled 'mcast_flood' on localnet ports, in order to avoid multicast packet duplication.  This introduces a regression because outbound multicast traffic from VMs is no longer sent to the provider network unless it is part of a multicast group that OVN has snooped.  This makes it impossible for something external to OpenStack to subscribe to a multicast group running on a VM (in OpenStack) unless another VM (in OpenStack) subscribes to it first (because the multicast traffic never leaves the OVS).
In my opinion https://review.opendev.org/c/openstack/neutron/+/797418 has to be reverted and a change implemented in ovn itself.
Regarding the change in OVN, ovn_igmp_group_aggregate_ports() has to be changed to only add the localnet port if mcast_flood is false, eg:
diff -ur orig/ovn-20.03.2/northd/ovn-northd.c ovn-20.03.2/northd/ovn-northd.c
--- orig/ovn-20.03.2/northd/ovn-northd.c        2022-03-25 03:33:41.000000000 +0000
+++ ovn-20.03.2/northd/ovn-northd.c     2022-03-25 13:07:43.879867403 +0000
@@ -3819,7 +3819,8 @@
         free(entry);
     }

-    if (igmp_group->datapath->localnet_port) {
+    if (igmp_group->datapath->localnet_port &&
+       !igmp_group->datapath->localnet_port->mcast_info.flood) {
         ovn_multicast_add_ports(mcast_groups, igmp_group->datapath,
                                 &igmp_group->mcgroup,
                                 &igmp_group->datapath->localnet_port, 1);

1 https://review.opendev.org/c/openstack/neutron/+/797418
2 https://bugs.launchpad.net/neutron/+bug/1933207

Thanks for reporting the issue.
@dceara  Do you have any comments for the suggestions in northd here ?

@dparv I think you're right.  Could you, please, post a patch to the mailing list with your change?
If possible, it would be nice if you could also add a test for this, please see AT_SETUP([IGMP snoop/querier/relay]) in https://github.com/ovn-org/ovn/blob/main/tests/ovn.at for an example.  Alternatively, I can also try to help with writing the test, just let me know.

By the way, I am not sure if that is the right approach. Ultimately what needs to happen is when an internal VM (can also be the same sending the traffic) subscribes to a the feed generated within the cloud this flow appears:
cookie=0x54cb0442, duration=30.708s, table=27, n_packets=9215, n_bytes=11440562, idle_age=0, priority=90,ip,metadata=0x26,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,nw_dst=<IP_ADDRESS> actions=set_field:0x8006->reg15,resubmit(,32)
Then the receiver machine outside the cloud starts to get that traffic, but when the internal VM leaves that IGMP group, the external receiver immediately stops.
I see in main branch that ovn_igmp_group_aggregate_ports is a bit re-worked and it now does:
    if (igmp_group->datapath->n_localnet_ports) {

instead of
    if (igmp_group->datapath->localnet_port) {

so maybe this needs to be a loop over the n_localnet_ports and check each for not being mcast_info.flood then add them one by one, like so:
diff --git a/northd/northd.c b/northd/northd.c
index 2fb0a93c2..5cb24f314 100644
--- a/northd/northd.c
+++ b/northd/northd.c
@@ -4545,10 +4545,15 @@ ovn_igmp_group_aggregate_ports(struct ovn_igmp_group *igmp_group,
     }

     if (igmp_group->datapath->n_localnet_ports) {
-        ovn_multicast_add_ports(mcast_groups, igmp_group->datapath,
-                                &igmp_group->mcgroup,
-                                igmp_group->datapath->localnet_ports,
-                                igmp_group->datapath->n_localnet_ports);
+        for (size_t i = 0; i < igmp_group->datapath->n_localnet_ports; i++) {
+            if (!igmp_group->datapath->localnet_ports[i]->mcast_info.flood) {
+                ovn_multicast_add_ports(mcast_groups,
+                                        igmp_group->datapath,
+                                        &igmp_group->mcgroup,
+                                        &igmp_group->datapath->localnet_ports[i],
+                                        1);
+            }
+        }
     }
 }

By the way, I am not sure if that is the right approach. Ultimately what needs to happen is when an internal VM (can also be the same sending the traffic) subscribes to a the feed generated within the cloud this flow appears:
cookie=0x54cb0442, duration=30.708s, table=27, n_packets=9215, n_bytes=11440562, idle_age=0, priority=90,ip,metadata=0x26,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00,nw_dst=<IP_ADDRESS> actions=set_field:0x8006->reg15,resubmit(,32)
Then the receiver machine outside the cloud starts to get that traffic, but when the internal VM leaves that IGMP group, the external receiver immediately stops.

You raise a good point here.
I'm guessing the external receiver also registered for the same multicast group, right?
If so, why didn't we add the localnet port in the SB.IGMP_Group corresponding to that multicast group?  It seems to me that this is the real issue.  Along side with the fact that ovn_igmp_group_aggregate_ports() will blindly merge SB.IGMP_Group records from multiple chassis for the same multicast group, without checking for port duplicates.
Or am I missing something else?

The external receiver is subscribed to this group, yes.
I am really no ovn expert here.. but how would ovn know this IGMP group exists - meaning if no VMs are subscribed to it, then how would it add the localport of the VM generating the traffic?
Easiest workaround I have found far is to just subscribe the VM that generates the traffic to the same IGMP group it multicasts it to, but that's 'in-workload' solution, not something that can happen on the cloud networking.

@dparv Would it be possible to attach or share the NB and SB databases?  I'll try to look into it but it makes it easier if we can test on the exact same network topology.  It would also really help to know the sender VM id or logical_switch_port.
Thanks!

@dceara
I have attached a zip with 2 scenarios:

subscribed - 1 internal VM streaming ffmpeg on port 5002 IGMP group <IP_ADDRESS> and subscribed to the same group; external receiver gets the feed

not-subscribed - 1 internal VM streaming ffmpeg on port 5002 IGMP group <IP_ADDRESS> and not subscribed to the same group; external receiver doesn't get the feed

In both cases the external receiver is s subscribed to the <IP_ADDRESS> IGMP group.
dbs.zip

The generating traffic VM inside the cloud is: <IP_ADDRESS>/27, name elmo-test-multicast2
The outside VM is <IP_ADDRESS>/27
Logical switch is f8e4ee08-9635-4f1b-9787-304ff9176a57
Logical switch port is: 51b8d609-23ab-4524-8c25-17e78d6200eb

@dceara Can you please take a look at the submitted patch ?

@aserdean @numansiddique I apologize, I missed this before, I'll review it these days.


add a cleanup task to BLC

@rgardler,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


Add error handling utilities for the compiler

Add error handling utilities for the compiler
Summary: These will be shared by both the arm64 and x64 compilers

/cc @Goldabj


🔨 Add default metrics to Engine

📝 Description

Metric values in Engine is currently None. When Engine is used with default values, the validation step is not run since there is no metric. Due to this there is no test results. This PR adds AUROC and F1Score as the default metrics to both image and pixel level metrics.
🛠 Fixes #1747

✨ Changes
Select what type of change your PR is:

[ ] 🐞 Bug fix (non-breaking change which fixes an issue)
[x] 🔨 Refactor (non-breaking change which refactors the code base)
[ ] 🚀 New feature (non-breaking change which adds functionality)
[ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] 📚 Documentation update
[ ] 🔒 Security update

✅ Checklist
Before you submit your pull request, please make sure you have completed the following steps:

[ ] 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
[ ] 📚 I have made the necessary updates to the documentation (if applicable).
[ ] 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

I see that some tests are failing as they directly access pixel_metric_names


New Project - Resource model ssh key path button etc?

Maybe I'm confused, but all ansible dialogues except for when you create resource model has an "open" button which lets you select your ssh keys from path.
Also resource model dialogue does not support privilege escalation password stored in key storage either which forces me to enter this password here, but not in the other dialogues.

From what I understand - you can only use ssh key storage for everything EXCEPT when importing the nodes? I still need to have my private ssh-key file stored on the server, besides rundeck key storage?

Hi @pentiumoverdrive ,
Unfortunately that's a limitation in rundeck, you can not access the key storage in the resource model since there is no execution context associated with it. The reason for this is that typical resource models like xml files does not need accessing target hosts to gather resources, so the rundeck API for the resource model does not provide a way to read the key storage.
Thanks,

Ok, thanks, Now I understand the limitation :)

Hi @yassineazzouz,
if resource model is designed to be a simple file, why ansible plugin can not use inventory file for this purposes? Because right now, key storage feature is useless for this plugin.

@jarosite because that is not exposed to the plugin API.


Create Networking Policy to allow traffic within tag-0lly namespaces due to the introduction of policies by the K8s team

Currently the networking plugin the k8s teams are using isn’t configured to enforce network policy rules. But they will in the future and when they do, current network policies maybe rendered somewhat ineffective.
:warning: Therefore, if your application is currently deploying network policies these policies may suddenly cause your application to break if these policies don’t have the desired effect.
Our network policies and namespaces are  below, so we need to identify any traffic that is necessary but not explicit in the network policies and apply them to the appropriate namespaces:
NetworkPolicy | Namespace | Type

allow-cluster-ingress | otel-gateway | Ingress

allow-delivery-cluster-ingress | tag-o11y-loki | Ingress

allow-delivery-cluster-ingress | otel-delivery | Ingress

allow-loki-access | tag-o11y-loki | Ingress

https://github.com/liatrio/tag-o11y-manifests/pull/100


Support SelectStatement

SelectStatements are currently not supported
ref: https://www.autoitscript.com/autoit3/docs/keywords/Select.htm

Added with 23595f88c33567f02db90dbe5047631b56ac5811


azure-ai-ml failing on python 36 windows

The core of the problem is that...

We have a transitive dependency on pywin32 from azure-identity in the dev_requirements for package azure-ai-ml
During initial venv setup, we install eng/test_tools.txt and dev_requirements.txt
Right before invoking pytest, we install the target package from whatever packaging method we're supposed to be testing.

is where the error pops up:

Installing collected packages: pyjwt, pywin32, websocket-client, pyrsistent, importlib-resources, tqdm, pydash, pathspec, jsonschema, docker, azure-storage-file-share, azure-storage-blob, azure-identity, applicationinsights, azure-ai-ml
  Attempting uninstall: pyjwt
    Found existing installation: PyJWT 2.4.0
    Uninstalling PyJWT-2.4.0:
      Successfully uninstalled PyJWT-2.4.0
  Attempting uninstall: pywin32
    Found existing installation: pywin32 304
    Uninstalling pywin32-304:
      Successfully uninstalled pywin32-304
ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\\repo\\sdk-for-python\\sdk\\ml\\azure-ai-ml\\.tox\\whl\\lib\\site-packages\\~ywin32_system32\\pythoncom36.dll'

This even happens when the correct, identical version of pywin32 is installed _already.
(snippet from pip freeze)
...
pytoml==0.1.21
pywin32==304
PyYAML==5.3.1
...

I cannot for the life of me understand why it's uninstalling, but that's where the error is coming from.
If it would just recognize the presence of the correct version, we would be ok. Need to resolve this error.

Resolved as commit in this pr


selectAllChildren in the revealed checkboxes variant of onsCheckboxes doesn't work when multiple instances are present

What happened?
When more than one checkbox contains an "'otherType": 'checkboxes', selectAllChildren only works for the first group of checkboxes.
What was the expected outcome?
Multiple instances of revealed checkboxes should work independently with selectAllChildren.
How is this problem impacting your service?
Unable to complete build of DAC prototype
What list of steps can be followed to reproduce the bug or issue?

Add at least 2 checkboxes with revealed otherType: checkboxes both with selectAllChildren: true.
Select the second parent checkbox
Children are not selected

Example
https://user-images.githubusercontent.com/43346934/161251687-e3096340-08be-4521-b1cc-de94d0c7543b.mp4
What version of the ONS Design System are you using?
47.0.0
What devices are you seeing the problem on?

[X] Desktop
[X] Mobile
[X] Tablet

Device details
No response
What operating systems are you seeing the problem on?

[ ] Windows
[X] macOS
[ ] Android
[ ] iOS
[ ] Other

Operating system details
10.14.6
What browsers are you seeing the problem on?

[ ] Firefox
[X] Chrome
[ ] Safari
[ ] Microsoft Edge
[ ] Microsoft Internet Explorer
[ ] Other

Browser details
Version 99.0.4844.83 (Official Build) (x86_64)

@yatesn I need this one in current Sprint backlog please. It needs to be fixed for DAC test proto.

My initial thought on this one was that each group of checkboxes need to be initialised inside "checkboxes.dom.js" (atm it seems like they are all treated as a single group).
I added the following to grab all checkbox elements and then group them by fieldset; and then to initialise the checkboxes of each fieldset:
  const checkboxesByFieldset = [...document.querySelectorAll('.ons-js-checkbox')]
    .map(checkbox => [
      checkbox,
      checkbox.closest('.ons-fieldset'),
    ]);

  const fieldsets = new Set(checkboxesByFieldset.map(entry => entry[1]));
  for (let fieldset of fieldsets) {
    const checkboxes = checkboxesByFieldset
      .filter(entry => entry[1] === fieldset)
      .map(entry => entry[0]);

@kruncher Whats the current status of this? Do you have this work on a branch?

@boxadesign there is no branch, I was experimenting locally to diagnose/fix the issue but my changes didn't have much impact to the problem. I pasted in a comment above the most significant part which I think might contribute to a solution.


[Snyk] Upgrade @types/chai from 4.3.6 to 4.3.8

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade @types/chai from 4.3.6 to 4.3.8.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 2 versions ahead of your current version.
The recommended version was released 23 days ago, on 2023-10-12.

Release notes

Package name: @types/chai

4.3.8 - 2023-10-12

4.3.7 - 2023-10-10

4.3.6 - 2023-09-04

    from @types/chai GitHub release notes

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (11f3ca7) 74.74% compared to head (f76c020) 74.74%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #42   +/-   ##
=======================================
  Coverage   74.74%   74.74%
=======================================
  Files           8        8
  Lines        1964     1964
  Branches      205      204    -1
=======================================
  Hits         1468     1468
  Misses        489      489
  Partials        7        7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add styles to summary page

[x] Apply markup and styles so that the summary page reflects the latest design:
[ ] Calculate and render age

@gtvj The addition of the age of the prisoner is included in PR #17

The prisoner information presenter currently provides a default text("Not entered yet") when no value is present. Do we want this or do we want to remove it & just leave the space blank?
If we leave it blank the presenters can be made simpler & i also think it looks better on the output. At a glance having text in the space may make it look like its been filled in.

@gtvj Age has been added in PR #21

For the record.. We've agreed not to have placeholder text on the summary page when properties aren't present in the escort record.
(cc @gtvj @cesidio)


Add support for mirgecom from forks.

This tiny change set adds the ability to install MIRGE-Com from a fork.

Sorry :)  I know forks aren't popular.


add diagnostics support

diagnostics is a ROS diagnostics package.
We publish CPU usage,actuator temperature etc...
So, it is important to support this message type.

diagnostics is a ROS diagnostics package.
We publish CPU usage,actuator temperature etc...
So, it is important to support this message type.

Hi @hakuturu583, since there are many different message types that users might want to emit to CloudWatch, we made a decision to support a single, relatively simple type into which you would need to convert your messages - rosgraph_msgs/Log for the CloudWatch Logs node, and ros_monitoring_msgs/MetricList for the CloudWatch Metrics node.
Please let us know if you're having any issues converting your message type into those.
Furthermore, we have a health metric collector node which supports seamless metric emitting for some of the metrics you described, and we'd greatly appreciate any contributions or suggestions there, too: https://github.com/aws-robotics/health-metrics-collector-ros1

Closing this. Feel free to reopen if you need to pursue it further.


Refactor how js translations are done.

This is an idea on how to re-factor js translations to be more modular and have the ability to allow packages to add custom strings.
Currently this is how strings in js are translated: https://github.com/concrete5/concrete5/blob/develop/web/concrete/controllers/frontend/assets_localization.php
We manually build js and call json_encode hundreds of times and then print out the result.
I would like to propose a new system that would reduce the number of json_encode calls and allow packages to add custom translations.
The benefits of this is that packages will not have to register custom routes or put translations inline in js, this will reduce the number of requests per page load. As a package developer when I have strings in js files I register a new route and put my translations there.
For this there would be a new service provider for js translations; for existing translations we would probably move them into a new file in /concrete/config/, like js_translations.php.
Packages would register translations like so:
\Core::make('derp')->registerTranslations($handle, $array);
in their on_start() where $handle is a unique string to the package, and $array is a key/value array of strings.
The service provider will json_encode $array and construct an object with the $handle and when requested it will print out all register translations.
I would suggest that we use the route /ccm/system/translations.js for the path
Also we should make it respect the caching settings set in the dashboard (send caching headers basically)
What are others thoughts on this idea?

I'd like to add that if we include a generic TranslationRepositoryInterface and add a TranslationRepositoryManager to manage them. Any translation software would then be able to extract these values from the registered singleton directly after loading the packages:
<?php
namespace Concrete\Core\Translation;

interface TranslationRepositoryInterface {

    /**
     * Get the list of translation strings.
     *
     * @return string[]
     */
    public function getTranslationStrings();

}

<?php
namespace Concrete\Core\Translation;

class TranslationRepositoryManager
{

    /** @type TranslationRepositoryInterface[] */
    protected $translationRepositories = array();

    /**
     * Add a translation repository to the list
     *
     * @param string $handle
     * @param \Concrete\Core\Translation\TranslationRepositoryInterface $repository
     */
    public function addRepository($handle, TranslationRepositoryInterface $repository)
    {
        $this->translationRepositories[$handle] = $repository;
    }

    /**
     * Retrieve a repository by handle
     *
     * @param string $handle
     * @param mixed|null $default
     * @return mixed
     */
    public function getRepository($handle, $default = null)
    {
        return array_get($this->translationRepositories, $handle, $default);
    }

    /**
     * Remove a repository
     *
     * @param string $handle
     */
    public function removeRepository($handle)
    {
        unset($this->translationRepositories[$handle]);
    }

    /**
     * Get the internal array of repositories
     *
     * @return \Concrete\Core\Translation\TranslationRepositoryInterface[]
     */
    public function getRepositories()
    {
        return $this->translationRepositories;
    }

}

Then, any translation string generation software would be able to just do:
<?php
class Foo
{

    /**
     * @type \Concrete\Core\Translation\TranslationRepositoryManager
     */
    protected $translationRepositoryManager;

    /**
     * Make a Foo.
     * @param \Concrete\Core\Translation\TranslationRepositoryManager $translationRepositoryManager
     */
    public function __construct(\Concrete\Core\Translation\TranslationRepositoryManager $translationRepositoryManager)
    {
        $this->translationRepositoryManager = $translationRepositoryManager;
    }

    /**
     * Fetch the translations from the repositories
     * @return string[];
     */
    public function fetchTranslations()
    {
        $translation_strings = array();
        $repositories = $this->translationRepositoryManager->getRepositories();

        foreach ($repositories as $repository) {
            $translation_strings = array_merge($translation_strings, $repository->getTranslationStrings());
        }

        return $translation_strings;
    }

}

and
<?php

$foo = new Foo(\Core::make('Concrete\Core\Translation\TranslationRepositoryManager'));
$translations = $foo->fetchTranslations();

thoughts?

@mlocati I'm interested in your thoughts on how this would impact translating.

I started implementing this, and was wondering about something,
Certain "files" are lazy loaded, for example redactor's translations, is there any reason not to include these up front and not lazy load them? The fact that we will minify and cache the generated js I feel like it actually would help.
For now I see no reason they shouldn't be.
Any feedback?

It's unclear so I'll ask, can/do the strings also include context? I know that today concrete5 has the tc() function which allows a string to be provided for certain contexts and I also think I recall comments being passed around as part of the translation strings too to help provide a frame of reference for certain things.
So for example if you needed an abbreviated version vs a long version due to screen real estate you might leverage context. Also, say for instance I want to translate the word "Good", in one scenario this might mean "Correct" as in "The part is good" (think a table value indicating if a part made was good or bad) and in another it might be used to describe how your day is going (Think a mood selection dropdown). A bit contrived but it's one concern I have with the array merge on the repository strings (providing the context / frame of reference couldn't be included in the translation string).

@joe-meyer In my latest mock up I threw together for mnkras, TranslationRepositoryInterface implementations must handle context.

Hi guys, sorry for this late feedback, I was quite busy.
I see the following problems (or at least they seems to me problems, I didn't digger in Korvin's code):

Browser caching
If two translations gets loaded from the same URL, browsers may use a cached version from the worng language. It's a potential problem that we're having now too, but a Cache-Control: no-cache  response header is preventing this. BTW it's not a great approach.
It'd be better to have different URLs for different languages.
For instance, now we have:
http://www.example.com/index.php/ccm/assets/localization/jquery/ui/js
What about these alternatives?

http://www.example.com/index.php/ccm/assets/localization/jquery/ui/js?it_IT
http://www.example.com/index.php/ccm/assets/localization/it_IT/jquery/ui/js

Currently, the translation assets contains javascript code in addition to simple javascript objects.
See for instance these lines of code. Will the new system handle this?

We sometimes don't need to send the response header, just the response body: see the current $setResponseHeaders variable.
This is useful when we enable the CSS and JavaScript Cache: the c5 asset system needs to retrieve the output of the localized assets in order to merge them with the other assets (see here and here).

@mlocati

Good point, It might be worthwhile to make the route actually determine what language is being outputted. We can handle all of that stuff in the controller implementation, I don't think we've really discussed that much yet.
We can have any kind of legacy or extra functionality outputted in the view file we use.
Hopefully the new controller won't be outputting like this, we should have a view file like /concrete/views/translation/javascript/translation.js.php, so if we need to cache the output, we can just load the view.

@aembler I'd be interested in your thoughts on this


BZ1070087: [ALL LANG] Help not localized. i18n for Splash Screen selector.

See https://bugzilla.redhat.com/show_bug.cgi?id=1070087#c5
(cherry picked from commit 170e0d713bf54b2c2a49278f567eabc85d2cdec0)

Let's keep this open until we have confirmation to merge into 6.3.x.

Not be included in 6.3.x at this moment. Closing.


I followed the provided tutorial of RASA, but now  I'm struck here while training nlu.md file. Please help me.

File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/aman17/.local/lib/python3.7/site-packages/rasa_nlu/train.py", line 173, in
num_threads=cmdline_args.num_threads)
File "/home/aman17/.local/lib/python3.7/site-packages/rasa_nlu/train.py", line 142, in train
training_data = load_data(data, nlu_config.language)
File "/home/aman17/.local/lib/python3.7/site-packages/rasa_nlu/training_data/loading.py", line 51, in load_data
data_sets = [_load(f, language) for f in files]
File "/home/aman17/.local/lib/python3.7/site-packages/rasa_nlu/training_data/loading.py", line 51, in
data_sets = [_load(f, language) for f in files]
File "/home/aman17/.local/lib/python3.7/site-packages/rasa_nlu/training_data/loading.py", line 108, in _load
raise ValueError("Unknown data format for file {}".format(filename))
ValueError: Unknown data format for file nlu.md

Thanks for the question, our Github issue board is for bugs and feature requests only, though 😕
Please ask this question in the forum✨
If you were describing a bug, please create a report and fill out the template.


hold the specific http request

hi all,
I'm having a problem of hold the specific http request, which is for retrieving the additional user information when token has received from the server.
////app.component, ngOnInit()
this.oauthService.loadDiscoveryDocumentAndTryLogin().then(_ => {
if (!this.oauthService.hasValidIdToken() || !this.oauthService.hasValidAccessToken()) {
this.oauthService.initImplicitFlow('test');
} else {
// This is where i'm going retrieve additional user info. user id, claims....etc
}
});
but the problem is while its processing, my default routing component also get fired, therefore, its trying to hit a another http request without having a userid.
any idea of hold the app.component request until its received the required information.
Thanks,

Related to #352 I think.

Thank you @jeroenheijmans , i will look into that.

@jeroenheijmans , i have fixed it on another way. have a service function to retrieve userid and it can be accessed by other module using Observable.zip method.


Use official Alpine and upgrade to latest v3.6

Keeping rootfs in sync with Alpine releases took more effort than desired.
Also the build infrastructure and the options provided by official alpine image go beyond the available resources of individuals on this project.
The changes presented in this branch:

Migrate to official alpine image and lock to latest v3.6 release
Update apk-install to fix exit code issue and avoid cache leftovers
Remove rootfs builder and binary from repository.

This repo was an inspiration to the official alpine image project!


i18n: Add Simplified Chinese language support

Thank you so much for your help with translation and pull request.


Block timestamp not updating properly

Supersim's nodes are not properly updating the block timestamp, making this issue to ensure we look into it.

@nitaliano @hamdiallam when anvil is in automine mode, if you simulate an executeMessage tx with id.timestamp set to the most recent block using debug_traceCall, then the call will fail here since debug_traceCall will use the most recent block's timestamp for block.timestamp, so you end up with id.timestamp == block.timestamp. If you send this call using eth_sendRawTransaction, it will succeed since the block.timestamp gets set to the next block's timestamp.

isn't id.timestamp externally supplied by the called, not set by the chain? id.timestamp should be the timestamp of the log event in the past. Maybe something i'm missing

Hi @hamdiallam @nitaliano doesn't look like this is resolved yet! If it is resolved, can someone update the issue? And if not, what's needed to get there/how can I help?

Closing this issue since we set up interval mining at 2s


dvlan: dhcpsnoop fix

When a DVLAN is initialized, it creates a new eth0.XXX interface. This interface didn't have DHCP snooping enabled on it, causing uCentral to not have the connecting clients IP address info

merged, Thanks !


Fix getting started typescript command

The previous command failed to complete successfully and in the error log seemed to be trying to check-out react-native-template-react-native-template-typescript. So I think some kind of refactor in the template param of the cli caused that and tried npx react-native init MyApp --template typescript which worked properly.

If this command fails, it means that you have react-native-cli installed globally. Please remove it and it will work.
cc @charpeni this can be closed

Dupe of https://github.com/facebook/react-native-website/pull/1390


Add missing gf180mcu_fd_bd_sram to .gitmodules

Fixes #13.
git submodule update --init --recursive
fatal: No url found for submodule path 'libraries/gf180mcu_fd_bd_sram/latest' in .gitmodules

Signed-off-by: Tim 'mithro' Ansell<EMAIL_ADDRESS>Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@vijayank88 - This should fix your problem.


DMS Prerequisites: What IP to whitelist in the firewall on the source side?

Under prerequisites it says "When using a firewall appliance in front of your source database(s), you may need to add firewall rules to allow the Azure Database Migration Service to access the source database(s) for migration." I didn't which IP needs to be whitelisted in the firewall, so I kept searching for this piece of information BEFORE I attempted to create the DMS.
It would be helpful to add that the IP that needs to be whitelisted can be found by checking the NIC resource once the DMS was created. Alternatively, there could be a link to the FAQ: https://learn.microsoft.com/en-us/azure/dms/faq#how-do-i-find-the-ip-address-for-azure-database-migration-service-so-that-i-can-create-an-allowlist-for-the-firewall-rules-used-to-access-my-source-database-for-migration-

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 261bad06-6551-260a-cbae-9f17f7e84246
Version Independent ID: 3e542277-4062-97dd-5039-b460d1e7f140
Content: Tutorial: Migrate PostgreSQL to Azure Database for PostgreSQL online via the Azure portal - Azure Database Migration Service
Content Source: articles/dms/tutorial-postgresql-azure-postgresql-online-portal.md
Service: dms
GitHub Login: @apduvuri
Microsoft Alias: adityaduvuri

@maximilianpalm
Thanks for your feedback! We will investigate and update as appropriate.

@maximilianpalm
Thank you for bringing this to our attention.
I've delegated this to content author, who will review it and offer their insightful opinions.

@maximilianpalm
That is a good suggestion.
We will consider adjusting the content to do so.

Thank you for your continued support and contributions. Please note that we are no longer managing Git Issues. We encourage you to use the new feedback option available in the MS Learn article for any future feedback or inquiries. Your input is valuable to us, and we appreciate your understanding and cooperation.
#please-close


Improvements for plotting API

This PR makes two changes:

Adds a output_notebook function at the top-level to load the holoviews notebook extension
Adds a .holoplot() API to datasources to mirror .plot() in case someone gets confused with the holoplot API.

This PR should be on hold until holoplot is rereleased (this weekend or early next week).

HoloPlot 0.1.0 has been released now.


Updating the examples files

Proposed changes
Some of the example files are outdated. Some of them are not working anymore.
Types of changes
What types of changes does your code introduce?
Put an x in the boxes that apply

[ x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist
Put an x in the boxes that apply.

[ ] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works

I just run the whole inf-imb notebook with both scipy 1.7.3 and scipy 1.8.1 and everything went smooth

They work for me too, I have scipy 1.6.1 and numpy 1.22.3

@AldoGl , yes sure. Go ahead!

@AldoGl , yes sure. Go ahead!

Done it, merging after the tests pass


Running script on pi

I getting the following message:
Package chromedriver is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'chromedriver' has no installation candidate

Yes. I ever tried it on Pi, same issue. However, I didn't deploy it on Pi finally.
Maybe you can try this: https://www.raspberrypi.org/forums/viewtopic.php?t=206439
Another choice is to run docker on Pi :)

I modified the script and it's work, need to wait if my domain is renewable.


Inconsistent SQL pronunciation

Currently, when writing SQL alone or as a component of other words (for example "SQLite"), there is some inconsistency in the way that "SQL" is assumed to be pronounced

Sometimes, it is proceeded by an "a", indicating a "sequel" pronunciation ("a SQL database")
Other times it is proceeded by an "an", indicating an "S-Q-L" pronunciation ("an SQL database")

As far as which is correct, there's no real consensus.  Apparently "SQL" was originally intended to be SEQUEL, but was changed due to copyright issues.  This ISO doc seems to use the S-Q-L pronunciation: https://www.iso.org/standard/53681.html
As far as SQLite specifically, the author also seems to have intended the S-Q-L pronunciation: http://sqlite.1065341.n5.nabble.com/SQLite-Pronunciation-tp88186p88194.html
In general, no one seems to really mind either pronunciation, so rather than this being an issue with correctness, it's just one of consistency  For "SQLite", I suggest we follow the author's original intention since we don't have a clear reason not to.  For standalone "SQL", let's just pick one and use it consistently.

Changed to 'a'.


Add gateway container ports to match up the LoadBalancer

This seems to make the aws implementation of the load balancer
more stable.
Related-Issue: https://github.com/submariner-io/submariner/issues/1410
Signed-off-by: Miguel Angel Ajo<EMAIL_ADDRESS>

@mangelajo does this change solve the issue described in https://github.com/submariner-io/submariner/issues/1410? It sounds that not completely, so can you explain what/how this improve things?

@mangelajo does this change solve the issue described in submariner-io/submariner#1410? It sounds that not completely, so can you explain what/how this improve things?

That's correct, I will explain better in the PR/Commit message. What I've seen is less error states in the LoadBalancer object when the ports are named the same. It could be accidental but, it will harm nobody to name the ports ':-)
It doesn't fix it.
I'm adding a workaround to avoid lengthy waits for the natt discovery protocol with special handling for the load balancer cases.


Keep source IP using TPROXY

I need the attacked server to see the original source IP. Since I am in the middle, I have control over the route paths of both sides. Traffic from both sides will flow through my router.
A single IP address taking all the TLS connections is going to raise eyebrows.
TPROXY seems to be where this could be possible as it keeps the original destination address and doesn’t rely on NAT.
Is this currently possible with SSLSplit?
I run it with this and I can see on the outgoing packets that the source IP is of the router I run the sslsplit on.
sslsplit -P -Z -t certs/ -l connections.log -S log/ ssl <IP_ADDRESS> 8443 tproxy

What I understand is we can read the source and destination IP address on the packets intercepted via TPROXY.
Currently (with masquerading)
I'm not sure how exactly masquerading works on SSLSplit, so below is my vague vision of it.
Outgoing traffic (traffic originating from SSLSplit):
Source IP address is translated to the external address, source port is either kept or translated to whatever port is available on the server SSLSplit runs on.
Incoming traffic (traffic coming into SSLSplit from the attacked server):
Traffic for a client is distinguished by the destination port on the incoming traffic. After we figure out which client the incoming packet is supposed to go to, destination IP address is translated to the said client's address, destination port is kept if it wasn't changed on outgoing traffic, or translated to the original if it was.
Would be (without masquerading)
Outgoing traffic:
Since SSLSplit opens a new connection, packets will leave with the server's external address as the source IP. SSLSplit needs to read the source IP of the intercepted packets and rewrite the source IP when going out of SSLSplit. Source port is kept.
Incoming traffic:
Traffic for a client is distinguished by the destination address on the incoming traffic. Destination address and port is kept.

Sslsplit establishes a new connection with the server. There is no address translation in sslsplit. And I have no idea how sslsplit can use the client address to connect to the server (sslsplit and the client run on separate machines).

Sslsplit establishes a new connection with the server. There is no address translation in sslsplit.

Understood, that's misconception from my part. Although there's a certain mechanism to distinguish traffic intended to multiple clients, similar to what I pictured above, correct? I don't really know how handling clients over transparent proxy works so I had assumed the proxy program would do sort of what the MASQUERADE target does.

And I have no idea how sslsplit can use the client address to connect to the server (sslsplit and the client run on separate machines).

It's technically originating a packet from SSLSplit with a fake IP. It doesn't belong in the network MitM proxy is in. So, once the packet with fake IP is delivered to the targeted server, it's going to reply back to that said IP address, which will flow through our device being the MitM. We can easily intercept that and feed it to SSLSplit. Then SSLSplit would do what I explained above to distinguish the packet to a certain client, do the SSL/TLS inspection and send the packet back to that client.
Reading the source and destination address of the intercepted packet from SSLSplit though, that, I do not know. Coding is not my strong suit.

You apparently have a very special network configuration. Underneath sslsplit is libevent. I don't think libevent can spoof IP addresses. So, in the end, the answer is that sslsplit does not support IP spoofing.

I wouldn't call it special but rather a specific attack vector. I'm currently writing a research piece about the importance of layer 2 security on hosting providers. I want to point out the catastrophy their carelessness of properly isolating virtual servers would bring upon them.
This gives me the ability to do ARP spoofing attack on the targeted webserver which allows me to get a valid Let's Encrypt certificate for their domain and run SSLSplit with it to sniff client's sensitive information without them seeing any insecure connection warnings.
This process is already completely transparent to the client as the only thing that would give this away is the different certificate but since it's valid, no one would look at it twice.
Webserver side could very easily notice the attack with the current state of SSLSplit. Suddenly, all the traffic they serve over TLS becomes this single IP address. This is why I wanted to hand the traffic to the webserver with the original source as the source IP so the attack would become much harder to detect.
Squid's TPROXY page seems to imply that this is possible via the spoof_client_ip config directive. I basically want the same thing on SSLSplit.

Here's an answer related to bind() to remote addresses:
https://stackoverflow.com/a/26507847

I also have the exact same network setup and am looking for a way to implement this!  Did you ever find anything that would allow you to keep client source IP, if you control the upstream router that any returning traffic would be passing through?  I really want to use sslsplit for this for the packet mirror feature, but in my architecture there is a second router upstream of me that is responsible for all NAT, so I don't want to introduce a second layer of NAT, especially with IPv6!

Unfortunately I didn't. I had to move on to other things at hand and have been handling them since. Life amiright?


Prototype using a self signed certificate fails as 'certificate verify failed'

Upon trying to leverage an internal data-source in a prototype it continues to fail reporting;
root@xxxxxxx:/usr/local/share/ca-certificates# cat /opt/minemeld/log/* | grep -i handshake | head -n 5
2017-08-14T09:40:39 (25678)basepoller._poll ERROR: Exception in polling loop for 20170814-BL01-Out-YYYYYYYY_public_known_bad: [Errno bad handshake] [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
SSLError: [Errno bad handshake] [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
2017-08-14T09:40:41 (25678)basepoller._poll ERROR: Exception in polling loop for 20170814-BL01-Out-YYYYYYYY_public_known_bad: [Errno bad handshake] [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
SSLError: [Errno bad handshake] [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
2017-08-14T10:40:39 (25678)basepoller._poll ERROR: Exception in polling loop for 20170814-BL01-Out-YYYYYYYY_public_known_bad: [Errno bad handshake] [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]
root@xxxxxxx:/usr/local/share/ca-certificates#

The Root + CA certs are installed correctly and
'openssl s_client -showcerts -connect xxxxx.yyyy.local:443'
Reports fine, as does a 'curl -v'
Certificates are installed in ' /usr/local/share/ca-certificates/' and update-ca-certificates was run.
Looking around online there is reference to Python using a separate location '/etc/ssl/certs/ ' however the   update-ca-certificates process has correctly populated this with symlinks.
Does anyone have any idea as to how to get Python to trusted a self signed cert? (about to start checking strace!)

strace seems to reference this?
/opt/minemeld/engine/current/local/lib/python2.7/site-packages/certifi/cacert.pem

Hi @simonsigre,
if you place your cert in /opt/minemeld/local/certs/site and then:
/opt/minemeld/engine/current/bin/mm-cacert-merge --config /opt/minemeld/local/certs/cacert-merge-config.yml --dst /opt/minemeld/local/certs/bundle.crt /opt/minemeld/local/certs/site/

It should create a new bundle merging your cert with certifi certs and use it.

@jtschichold is it possible this folder was not created as part of the Ansible deployment process? I dont seem to be able to find any cert bundles  except for;

Could it be using this?
/opt/minemeld/engine/current/lib/python2.7/site-packages/requests/cacert.pem

@simonsigre yes, I have to update the Ansible playbook to create that directory. You can do it easily by:
sudo -u minemeld mkdir -p /opt/minemeld/local/certs/site/
sudo -u minemeld touch /opt/minemeld/local/certs/cacert-merge-config.yml
[copy you CA file in /opt/minemeld/local/certs/site]
sudo -u /opt/minemeld/engine/current/bin/mm-cacert-merge --config /opt/minemeld/local/certs/cacert-merge-config.yml --dst /opt/minemeld/local/certs/bundle.crt /opt/minemeld/local/certs/site/

I assume this line
sudo -u /opt/minemeld/engine/current/bin/mm-cacert-merge --config /opt/minemeld/local/certs/cacert-merge-config.yml --dst /opt/minemeld/local/certs/bundle.crt /opt/minemeld/local/certs/site/
``

Should start out

sudo -u minemeld
``

The following was observed
root@xxxxxxxxxx:~# sudo -u minemeld /opt/minemeld/engine/current/bin/mm-cacert-merge --config /opt/minemeld/local/certs/cacert-merge-config.yml --dst /opt/minemeld/local/certs/bundle.crt /opt/minemeld/local/certs/site/
2017-08-17T11:56:17 (31400)cacert_merge.main INFO: config: {'cafile': ['/opt/minemeld/local/certs/site/'], 'dst': '/opt/minemeld/local/certs/bundle.crt', 'config': '/opt/minemeld/local/certs/cacert-merge-config.yml', 'no_merge_certifi': False}
WARNING: old python version (< 2.7.9) - certificate verification not performed
WARNING: old python version (< 2.7.9) - certificate verification not performed
WARNING: old python version (< 2.7.9) - certificate verification not performed

And I can now see those certificates added into '/opt/minemeld/local/certs/bundle.crt' ... however.. I still receive the same error message.
Should '/opt/minemeld/local/certs/cacert-merge-config.yml' contain any configuration?
How does MM know to check the newly created '/opt/minemeld/local/certs/bundle.crt' ?

Hi @simonsigre,
right, you should add
MM_CA_BUNDLE="/opt/minemeld/local/certs/bundle.crt"

to the environment of minemeld-engine in /opt/minemeld/supervisor/config/conf.d/minemeld-engine.conf, in the environment section. And then restart minemeld service.

@jtschichold thank you this worked. Will all of the above be included as part of an upcoming release?

Hi @simonsigre,
yes, we will fix this. If you already have PR for minemeld-ansible we will be happy to accept it.
Thanks for troubleshooting this !


Serialized SpeckleProperties.Data using ISerializationCallbackReceiver and Speckle.Core.Api.Operations

Resolves #27

I'm getting some very slow performance when switching between editor and play mode that I wasn't getting before. I'm going to investigate before this PR is ready to review.

With my previous solution, I am getting slightly faster times. Still ~1s for serilisation, but ~3-6s deserilisation.
It seems that perhaps JsonUtility is faster than Operations for lots of small objects.
@teocomi Maybe 10s is acceptable, I don't see any obvious ways to improve Operations performance. I want to dig a little deeper just to confirm my measurements.

This is quite interesting! During previous tests, I found that the root cause of issue https://github.com/specklesystems/speckle-unity/issues/17 is the deserialization step.
I'm pending a deep dive into Core to see what's going on, @didimitrie and I assume it might have to do with handling dynamic stuff.
I'm happy you merge this for now and then tackle the slowness issue later on 👍


Multiple images at once

Hi.
I'm working on a research process. I need to process a large amount of images. Like, a lot. Is there any form within Nefi2 to feed multiple inputs at once?
yechp

Have you tried python3 nefi2.py --help?
usage: nefi2.py [-h] [-p PIPELINE] [-d DIR] [-f FILE] [-o OUT]

optional arguments:
  -h, --help            show this help message and exit
  -p PIPELINE, --pipeline PIPELINE
                        Specify a saved pipeline xml file.
  -d DIR, --dir DIR     Specify a directory with images for batch processing.
  -f FILE, --file FILE  Specify an image file to process.
  -o OUT, --out OUT     Specify output directory.

PIPELINE is a json file that contains a list of algorithms and settings for processing:
cat nefi2/default_pipelines/a_junius.json
  [
    ["Watershed DE Adaptive",{"Foreground Iteration":0, "Background Iteration":0,"Threshold Block Size":11,"Threshold Constant":1, "type":"Segmentation", "store_image":false}],
    ["Median Blur",{"channel1":true,"channel2":true,"channel3":true,"kernelsize":2, "type":"Preprocessing", "store_image":false}],
    ["Guo Hall Thinning",{"store_image":false,"type":"Thinning"}],
    ["Guo Hall Graph Detection",{"store_image":false,"type":"Graph Detection"}],
    ["Keep only LCC",{"store_image":false,"type":"Graph Filtering"}],
    ["Simple Cycle",{"store_image":false,"type":"Graph Filtering"}]
  ]

you can find a set of default pipelines under nefi2/default_pipelines dir.

Ok... I'm sorry but I just started with python. I come from R and NetLogo. Probably this question is going a very ignorant one but... where do I run this code?

You need a Linux, where you can open console, navigate to nefi directory and run nefi2.py.
For example:
git clone https://github.com/05dirnbe/nefi.git
cd nefi/
python3 nefi2.py -p nefi2/default_pipelines/a_junius.json -d dir_with_images -o processed_images

I cannot test if current nefi2 version is working on Windows unfortunately.


Add a philosophy document to exercises directories?

In https://github.com/exercism/x-common/pull/375#issuecomment-247817577 @petertseng says:

these things should be documented so that we can avoid duplication of past efforts.

How about if we create a file in the exercise subdirectory that documents our discussions and efforts?
We wouldn't need to add this to all exercises, nor all at once. But as we have discussions it would be incredibly useful to be able to have the results of these discussions on hand.

I also suggest that it could be put in the canonical-data.json (if it should be mainly targeted at track maintaineres), or the description.md (if it should be targeted at students)

to reuse existing files and not introduce a new one

I'd rather keep this out of the description.md since this is about why we ended up making the choices we did, or which trade-offs we considered and then rejected.
This also feels different from canonical-data.json as well, as that feels like it is the end result of the discussion, and most people who want to use it to generate a new implementation should be able to follow it without worrying about the details, unless they're interested in them.

I really like this idea.  I've seen the same on long-lived systems and been grateful for the insight on why certain design choices were made.  Knowing why can really make work more efficient: it increases my confidence that I'm making "good" changes and has saved me from repeating history.
The most useful ones I've seen have a simple formats, something like:

Overall context (where idea came from, original problem being solved).

Key Descision X:

Problem/Aspect
Decision (expected benefits + known trade-offs)
Other options considered (terse, but some hint as to why deemed unsuitable)

The latest iteration of such documents I was involved in for a system were deliberately and specifically version controlled so that text could be safely deleted without fully losing that history.
Is this the kind of thing you're talking about @kytrinyx?

Is this the kind of thing you're talking about?

Yes, that's exactly it.


Updated README and BufferList doc

BufferList and README documentation changes. Split from a large PR to make these changes easier to get into master.

Codecov Report

Merging #244 into master will decrease coverage by 0.26%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #244      +/-   ##
==========================================
- Coverage   74.92%   74.65%   -0.27%
==========================================
  Files          31       31
  Lines        4171     4171
==========================================
- Hits         3125     3114      -11
- Misses       1046     1057      +11

Flag
Coverage Δ

#CHTTPParser
57.46% <ø> (ø)
:arrow_up:

#KituraNet
74.65% <100%> (-0.27%)
:arrow_down:

Impacted Files
Coverage Δ

Sources/KituraNet/BufferList.swift
89.58% <100%> (ø)
:arrow_up:

...ces/KituraNet/Server/ServerLifecycleListener.swift
79.16% <0%> (-4.17%)
:arrow_down:

Sources/KituraNet/IncomingSocketHandler.swift
78.53% <0%> (-3.66%)
:arrow_down:

Sources/KituraNet/HTTP/HTTPServer.swift
79.88% <0%> (-1.12%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b4221e...b4925bd. Read the comment docs.

@KyeMaloy97 the change to automatically generate the jazzy docs when a specific label is added to a PR is not complete yet, so when you make the requested changes above you'll need to generate all the jazzy docs and attach those to this change set if you want them to be visible from kitura.io.


Android(< 3.0) SSLEngine Bug 12955 workaround?

Moved form pull #153:
@2fours:
There seem to be problems on older android platforms because of this bug in SSLEngine : https://code.google.com/p/android/issues/detail?id=12955#c12. It appears that the only way this will work on these platforms is to abandon SSLEngine and use SSLSocket. Any chance of that happening?

@Davidiusdadi
Do you mean that your code is working fine but there is a problem with what i merged?
If there is a general problem i would like to know the exact exception or build error message and the android version you are working on.
The solution suggested in #141 might solve the problem.

This Android version is not active any more!
Closing.
Greetings marci4


Should provide integration with OAUTH2_ACCESS_TOKEN_URL env variable

Add some tests to keep coverage.

👍

Codecov Report

Merging #79 into master will increase coverage by 0.1%.
The diff coverage is 80%.

@@             Coverage Diff             @@
##             master      #79     +/-   ##
===========================================
+ Coverage     71.12%   71.23%   +0.1%
- Complexity      225      227      +2
===========================================
  Files            46       46
  Lines           807      817     +10
  Branches         39       40      +1
===========================================
+ Hits            574      582      +8
- Misses          211      213      +2
  Partials         22       22

Impacted Files
Coverage Δ
Complexity Δ

src/main/java/org/zalando/stups/tokens/Tokens.java
69.23% <80%> (+35.89%)
3 <2> (+2)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d0ba26...74b6885. Read the comment docs.

👍


Cannot enable  plugin

I'm relatively new to Godot, so I don't know if this could be a problem on my part or not. I followed the instructions, but these errors came up in the output when I tried to enable the plugin.  I went to the script it listed, and I saw the error
"Identifier "MetSys" not declared in the current scope."  on lines 106-108 and 111.
I am using the most up to date version of Godot.
Errors:
res://addons/MetroidvaniaSystem/MetSysPlugin.gd:106 - Parse Error: Identifier "MetSys" not declared in the current scope.
res://addons/MetroidvaniaSystem/MetSysPlugin.gd:107 - Parse Error: Identifier "MetSys" not declared in the current scope.
res://addons/MetroidvaniaSystem/MetSysPlugin.gd:108 - Parse Error: Identifier "MetSys" not declared in the current scope.
res://addons/MetroidvaniaSystem/MetSysPlugin.gd:111 - Parse Error: Identifier "MetSys" not declared in the current scope.
modules/gdscript/gdscript.cpp:2726 - Failed to load script "res://addons/MetroidvaniaSystem/MetSysPlugin.gd" with error "Parse error". (User)
core/config/project_settings.cpp:341 - Property not found: editor_plugins/enabled

Sounds like this issue again #50
See if anything in that topic helps.

That PR seems to have fixed it for me.


Epg not working

Tvtv.us epg not working no information

Previously the guides were available in two version,
uncompressed : guidename.epg.xml
compressed : guidename.epg.xml.gz
Some of the uncompressed guides were starting to hit GitHub's file size limit, so it was decided to only publish the compressed version of all guides.

Thanks man just changed it and it worked thank you


Feature request: widgets for document image understanding and document image VQA

I'm currently adding LayoutLMv2 and LayoutXLM to HuggingFace Transformers. These models, built by Microsoft, have impressive capabilities for understanding document images (scanned documents, such as PDFs). LayoutLM, and its successor, LayoutLMv2, are extensions of BERT that incorporate layout and visual information, besides just text. LayoutXLM is a multilingual version of LayoutLMv2.
It would be really cool to have inference widgets for the following tasks:

document image understanding
document image visual question answering
document image classification

Document image understanding
Document image understanding (also called form understanding) means understanding all pieces of information of a document image. Example datasets here are FUNSD, CORD, SROIE and Kleister-NDA.
The input is a document image:

The output should be the same image, but with colored bounding boxes, indicating for example what part of the image are questions (blue), which are answers (green), which are headers (orange), etc.

LayoutLMv2 solves this as a NER problem, using LayoutLMv2ForTokenClassification. First, an OCR engine is run on the image to get a list of words + corresponding coordinates. These are then tokenized, and together with the image sent through the LayoutLMv2 model. The model then labels each token using its classification head.
Document visual question answering
Document visual question answering means, given an image + question, generate (or extract) an answer. For example, for the PDF document above, a question could be "what's the date at which this document was sent?", and the answer is "January 11, 1999".
Example datasets here are DocVQA - on which LayoutLMv2 obtains SOTA performance, who might have guessed.
LayoutLMv2 solves this as a extractive question answering problem similar to SQuAD. I've defined a LayoutLMv2ForQuestionAnswering, which predicts the start_positions and end_positions.
Document image classification
Document image classification is fairly simple: given a document image, classify it (e.g. invoice/form/letter/email/etc.). Example datasets here are [RVL-CDIP](https://www.cs.cmu.edu/~aharley/rvl-cdip/#:~:text=The RVL-CDIP (Ryerson Vision,images%2C and 40%2C000 test images.). For this, I have defined a LayoutLMv2ForSequenceClassification, which just places a linear layer on top of the model in order to classify documents.
Remarks
I don't think we can leverage the existing 'token-classification', 'question-answering' and 'image-classification' pipelines, as the inputs are quite different (document images instead of text). To ease the development of new pipelines, I have implemented a new LayoutLMv2Processor, which takes care of all the preprocessing required for LayoutLMv2. It combines a LayoutLMv2FeatureExtractor (for the image modality) and LayoutLMv2Tokenizer (for the text modality). I would also argue that if we have other models in the future, they all implement a processor that takes care of all the preprocessing (and possibly postprocessing). Processors are ideal for multi-modal models (they have been defined previously for CLIP and Wav2Vec2).

DOcument QA is now supported in transformers and widget, so closing

This is still failing for Donut:

This is because donut does not conform to the expected output specification of the document QA pipeline in transformers. All other models output both the answer and a score, while Donut only outputs the answer.

Thanks for looking into that @osanseviero. Donut is a generative model, so it indeed does not output a score. All newer models (including Nougat - which is also a Donut model) are generative, so would be great to make the widget more general.

Let's open a new issue as this is more of a bug and don't want to clutter the huggingface.js with the whole request


'Copy link to this section' feature not working for certain sections.

URL(s) with the issue:

https://www.tensorflow.org/install/source_windows
https://www.tensorflow.org/install/source

Browser used:
Tried on Microsoft Edge and Mozilla Firefox up-to-date versions.
Description of issue (what needs changing):
The 'Copy link to this section' feature is not working for the View sample configuration session sections for the Linux and Windows build-from-source instruction docs.
Clear description:
The issue-causing sections are located under these sections:

https://www.tensorflow.org/install/source#sample_session (Linux)
https://www.tensorflow.org/install/source_windows#configure_the_build (Windows)

Submit a pull request?
Are you planning to also submit a pull request to fix the issue?:  Yes, I can fix the markdown to correct the issue.

@lamberta From where should this issue be fixed? It doesn't look like a MarkDown issue.
Thank you in advance.

Possibly related to internal bug: b/115916696
Will wait and see

The 'Copy link to this section' feature is not working for the View sample configuration session sections for the Linux and Windows build-from-source instruction docs.

@nikochiko,
Looks like this issue is fixed. Links for View sample configuration session section works for both Windows and Linux.

Windows : https://www.tensorflow.org/install/source_windows#expandable-1
Linux : https://www.tensorflow.org/install/source#expandable-1

Please feel free to close the issue if resolved. Thanks!


Update python and protobuf requirements

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Update versions per latest upstream change: https://github.com/kubeflow/pipelines/pull/10305/ to unblock https://github.com/conda-forge/kfp-pipeline-spec-feedstock/pull/13

@conda-forge-admin, please rerender

@manuelwalserqc can we get this merged?


⚠ Processors has degraded performance

In 8df7ae3, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 134 ms

Resolved: Processors performance has improved in d6a264e.


Don't show anything by default on the shopping screen

Steps to reproduce

Select Travel from the navigation drawer
Select Hotel Booking

It automatically searches for bags & show results.

Expected behaviour

Don't search for anything by default
Show this animation on the screen in the center: https://www.lottiefiles.com/373-streetby-test-loading
Below this, write a text, Want to shop something? Select the search icon on top

Would like to work on this.


Add a section to the README for icon generation

Uses the ember-cli-image-transformer addon to convert a single image
into multiple needed icon sizes.
@san650 as we discussed in Slack here are some basic docs for using ember-cli-image-transformer to build icons for web apps. I wasn't too sure what should be included here so please let me know if I missed the mark.

@jrjohnson as you can see, one of the original ideas of the addon was to tackle the image generation as one of the features, but I'm glad there's a different addon that do that work!
I like the documentation you added, I left some comments. Let me know your thoughts!

Thanks for the quick feedback @san650. Those all looks like great additions. I will try and get them done in the next 24 hours.

I ended up having some time tonight, so these changes are made now.

Thanks for contributing!


Add overseas passport logic for St Martin

We have been told the procedure is the same as for the Dutch side of the same island, St Maarten. So it should be safe to merge without FC.
This will stop 500'ing for users in St Maarten.
We're still trying to find out the procedure in Saint-Barthélemy.

Other than my small comments, this looks good to me. :+1:


Feature: RSS controller and feed

We already have postings and a PostingController, but we also need an RSS feed that displays the last X Postings.
We could probably introduce a separate controller for the RSS feed and extract the "load postings for the start page" logic from PostingController to always have the same postings in the feed that are on the frontpage.

@rrrene I can put some time on this issue during this week

:+1: That would be great!


"High Resolution Timestamps for Events" update.

R: @PaulKinlan
Importing from a previous reviewed draft document.

LGTM


[Bug] Critical security vulnerability

Critical security vuln discovered and patched in #83

This is hilarious but I think I'm still going to give Libra a shot.

This is hilarious but I think I'm still going to give Libra a shot.

You probably own Ripple too.

Please keep at least the GitHub issues an objective place. Every software has security vulnerabilities.

This is hilarious but I think I'm still going to give Libra a shot.

You probably own Ripple too.

I should have been more clear - I'm going to try building something with Move; I'm not interested in owning anything with built-in KYC enforcement.

This is hilarious but I think I'm still going to give Libra a shot.
You probably own Ripple too.
I should have been more clear - I'm going to try building something with Move; I'm not interested in owning anything with built-in KYC enforcement.

@crawfordleeds My understanding enforced KYC is property of the calibra wallet, not the protocol. Correct me if I'm wrong.

This is hilarious but I think I'm still going to give Libra a shot.
You probably own Ripple too.
I should have been more clear - I'm going to try building something with Move; I'm not interested in owning anything with built-in KYC enforcement.

@crawfordleeds My understanding enforced KYC is property of the calibra wallet, not the protocol. Correct me if I'm wrong.

@alexindigo I don't know for sure, there's a good chance you are correct and I am wrong. That was something I heard second hand, and that seemed to vibe with the terminology of the whitepaper. There is a mention something along the lines of "people should be entitled to the fruits of their legal labor". Maybe it's imprecise language (I would be surprised) but that seems to mandate KYC if Libra is going to ensure legality. For example, to know which country's laws to apply, they would need to know some information about the customer, at the very least location.
I'll have to dig some more. If it's just the wallet, and other wallets are going to be available in the future, that would be a nice relief.

I mean, it’s only enforceable on wallet level, for protocol itself it’s just public keys.
The question is how much they’d be controlling other wallets on the network, but I don’t see that any KYC data transfer baked into the protocol. So if it’d be done, only through audit. And most likely will be enforced through local laws rather than network itself.

Closing as discussed on the pull request and subsequent bug.


Set up stress test

Ref #224

I will develop this in the fractal-demos repo, and then eventually bring back here what is relevant.


Routing: alternatives: some alternatives can give results even if travel time > max travel time

It's a trRouting bug. See https://github.com/chairemobilite/trRouting/issues/246


Error / Warning ole

Thanks for this tool - I just tried it out.
I noticed that when I ran it, I got this printed to the console even though the export was successful:

WARN: Ignore errors like 'Qt: Could not initialize OLE (error 80010106)'
Qt: Could not initialize OLE (error 80010106)

Sorry I see thats a known issue

The tool, sometimes shows an error "Qt: Could not initialize OLE (error 80010106)" - this can be ignored.


Dependency CI Update

This PR moves dependency checking to Dependency CI as VersionEye is being sunset. It also updates the Gradle wrapper to 4.5.1, and makes sure that all dependencies up to date.

Codecov Report

Merging #8 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #8   +/-   ##
=========================================
  Coverage     80.55%   80.55%
  Complexity      199      199
=========================================
  Files            13       13
  Lines           833      833
  Branches         54       54
=========================================
  Hits            671      671
  Misses          155      155
  Partials          7        7

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd55a4c...5169bb4. Read the comment docs.


Refactor utils and integrate json-to-ast

Currently there are qx.tool.compiler.utils.* and qx.tool.cli.Utils, plus the json-to-ast module which is a git dependency; AFAICR the compiler and cli code use each others utils, because a lot (most? all?) of it is more generic than just one specific tool.
The json-to-ast module is a branch which has been an outstanding PR for nearly 2 years.
I propose that we refactor the utils into a generic qx.tool.utils namespace, and pull the json-to-ast code in there too, removing the extra dependency on a git branch.
This probably should not be done until https://github.com/qooxdoo/qooxdoo-compiler/pull/380 is merged

From gitter:
The compiler directory structure is quite confusing. WE
We have "lib", "resource", "tool", "utils". Wouldn't it be better to mirror the framework folder organization -> source/class/qx/tool, source/resource/qx/tool/(cli|compiler)/(serve|templates|utils) ?
That would also prepare a later integration, if that is still on the horizon.

#380 is merged, but please wait until #394 is implemented (currently in branch https://github.com/qooxdoo/qooxdoo-compiler/tree/cboulanger-new-config-system).

Done with #416


Don't block reuploads of a file based solely on the hash

What's the problem this feature will solve?
Because pip resolves wheel platforms by filename, occasionally you want to upload the same wheel under different platforms. For example, a wheel that will only work on Windows (regardless of 32/64-bitness) has to be provided in both ...-win32.whl and ...-win_amd64.whl variants.
Warehouse currently uses the contents of the file (apparently?) to check whether a file has already been uploaded. This means you can't upload the same wheel twice under different names.
Describe the solution you'd like
(I'd really like better platform resolution in pip, but that doesn't solve the problem of people who haven't/can't upgrade, so...)
I'd like it if you were allowed to upload the same file multiple times under different filenames. Totally happy for it to resolve to the same bytes on the server, provided pip thinks they are distinct.
Additional context
See https://pypi.org/project/ptvsd/4.1.1a1/#files for an example package where we have done this.

I think if you’re changing the wheel file name, you should also be changing the metadata inside of the wheel which would change the hash. Is that not what you’re doing? If not is there a reason?
Sent from my iPhone

On Jun 1, 2018, at 2:58 PM, Steve Dower<EMAIL_ADDRESS>wrote:
What's the problem this feature will solve?
Because pip resolves wheel platforms by filename, occasionally you want to upload the same wheel under different platforms. For example, a wheel that will only work on Windows (regardless of 32/64-bitness) has to be provided in both ...-win32.whl and ...-win_amd64.whl variants.
Warehouse currently uses the contents of the file (apparently?) to check whether a file has already been uploaded. This means you can't upload the same wheel twice under different names.
Describe the solution you'd like
(I'd really like better platform resolution in pip, but that doesn't solve the problem of people who haven't/can't upgrade, so...)
I'd like it if you were allowed to upload the same file multiple times under different filenames. Totally happy for it to resolve to the same bytes on the server, provided pip thinks they are distinct.
Additional context
See https://pypi.org/project/ptvsd/4.1.1a1/#files for an example package where we have done this.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

That's our workaround, but since we're only building a single wheel (with multiple ABI-tagged extension modules in it) it has to be done manually.

But the platform tags that you’re changing are also part of the wheel metadata. So if you’re changing the file name you should also be updating those files anyways. It the WHEEL file iirc
Sent from my iPhone

On Jun 1, 2018, at 4:10 PM, Steve Dower<EMAIL_ADDRESS>wrote:
That's our workaround, but since we're only building a single wheel (with multiple ABI-tagged extension modules in it) it has to be done manually.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Except the platform really is "multiple platforms" (specifically, it's "either win32 or win_amd64"). No installation tools are able to resolve a dual-purpose tag like this, so we use two files to help them out, but the installed wheel is identical and has no platform.

@zooba In case it's not clear -- what Donald's pointing out is that the platform tag is listed twice: once in the wheel filename, and once in a special metadata file inside the wheel, and the two copies of this metadata are supposed to match. Some tools only look at the filename, so you can get away with having a mismatch -- in particular I think pip currently looks only at the filename. But in the future this might change.
Also, I think you should be able to use a single file with the tag win32.win_amd64, where . is a special operator in wheel tags that means or? See https://www.python.org/dev/peps/pep-0425/#compressed-tag-sets

It was clear, but you're right that a compressed tag will work for my current case. Funny how I knew about that for the Python version, but it hadn't clicked that it works for all the parts.
So that deals with my current problem, but I still think Warehouse shouldn't be blocking reuploads of a file based solely on the hash. If there's a good reason then I'm happy to hear it and leave alone, but it seems like an arbitrary and potentially confusing restriction.

The reason is basically to allow content addressable files-- I don't think we're currently using that today, but it leaves it open to do so in the future. Allowing file uploads for the same hash blocks that off completely-- so we could do it, but we'd have to decide we're never going to want that if we do.

FTR it appears your wheels are slightly invalid right now, you have a wheel named ptvsd-4.1.1a1-py3-none-win32.whl, but the internal metadata says Tag: py3-none-any, so that's a mismatch between the two values which should be the same. Potentially pip should start at least warning in this case, if not erroring because the metadata is effectively broken here.

How does it block it off? Wouldn't you just be allowing using the hash as the filename? Or are you thinking more of a reverse search where you can map back to the original filename (in which case, why not map back to multiple?)

it appears your wheels are slightly invalid ...

Watch for the next update ;)

Sorry, I missed this last message on this, and yes, content address files would mean you'd be able to locate a file knowing nothing but it's hash. Like I said, I don't know we'd ever actually do that or not, but the current restriction enables that to happen in the future if we ever desired it.
The other reason is that because the internals of the wheel file have to change if you change the tags, that it's likely an error if you're trying to upload a file with the same hash as an existing file-- it likely means you forgot to adjust your tags inside the wheel file too.
Generally I don't feel strongly one way or another about this, but the fact I can't think of a case where you'd actually want to upload the same file under two different filenames that isn't actually a case of one (or both) of the uploaded wheels having incorrect metadata associated with it I'm leaning towards leaving the current restriction in place. Is there a use case you can think of when you'd want to upload two files like this where the wheel files don't have erroneous metadata inside them?

@zooba hey, ping in case you missed Donald's question in the last comment and have time to respond.

Closing this issue as I don't think we have any plans to support this at this time.


Error with southpark.cc.com

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.12.31. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[ ] I've verified and I assure that I'm running youtube-dl 2017.12.31

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best', '-o', '\\%(title)s\\%(title)s%(format_id)s.%(ext)s', '--geo-bypass-country', 'US', 'http://southpark.cc.com/full-episodes/s01e01-cartman-gets-an-anal-probe']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.12.28
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg git-2017-12-29-0c78b6a, ffprobe git-2017-12-29-0c78b6a
[debug] Proxy map: {}
[debug] Using fake IP <IP_ADDRESS> (US) as X-Forwarded-For.
[southpark.cc.com] s01e01-cartman-gets-an-anal-probe: Downloading webpage
Traceback (most recent call last):
  File "__main__.py", line 19, in <module>
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\__init__.py", line 465, in main
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\__init__.py", line 455, in _real_main
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\YoutubeDL.py", line 1988, in download
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\YoutubeDL.py", line 784, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\extractor\common.py", line 438, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\extractor\mtv.py", line 286, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\extractor\mtv.py", line 212, in _get_videos_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpheifp6ll\build\youtube_dl\extractor\mtv.py", line 38, in _id_from_uri
AttributeError: 'NoneType' object has no attribute 'split'
...
<end of log>

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.

--geo-bypass-country', 'US

Don't.


Clarify path of the OAuth callback URL

Clarifies what URL should be used when creating the GitHub OAuth app.
Ref. https://github.com/benbjohnson/wtf/discussions/34

That's much better than what I wrote. Thanks! :)


Trackobot deck analysis

I created an algorithm to find the most common decks based on tracked games:
https://ganwell.github.io/card-complete.html
It would be interesting to try this with a large data-set.

How can get access to anonymized database of matches? I need to get real match history data. More than from my local matches(I played only 100 of matches and 10 of decks). But I'm planning to carry on analysis of thousands of matches with hundrends of decks to understand the best winning decks, most effective combos, etc. Can anyone help me?

Track-o-Bot data is only available to the individual user.


Missing method pages for Qiskit API docs before Qiskit 0.44

Until Qiskit 0.44 via https://github.com/Qiskit/qiskit/pull/10455, in Qiskit docs, methods lived on dedicated HTML pages.
~For some reason, when generating the historical API docs for Qiskit, the script failed to generate those method pages and it only generated class pages, module pages, and function pages. So, our redirects from qiskit.org to ibm.com are working properly but resulting in 404s because the method pages are not there.~
This would have been caught by https://github.com/Qiskit/documentation/issues/495.
As explained in https://github.com/Qiskit/documentation/pull/543, we intentionally inlined methods/attributes into their class page via the file mergeClassMembers.ts. That in general worked great, but there are a couple of instances where it fails, like https://docs.quantum.ibm.com/api/qiskit/0.33/qiskit.result.Result mentioned in that PR description. We need to figure out why those failed and get them to work consistently.
The other issue was that I did not realize this inlining was happening when setting up redirects from qiskit.org to ibm.com, so we were trying to redirect method pages to non-existent method pages. I have a PR up for review to fix our redirect logic to instead go to the parent class's page.
To close this issue, we need to:

Properly inline all methods. That can be verified by running our link checker over these API docs and making sure that all internal links work.
Land the redirects change.

Every version of the API docs has been regenerated. This should be fixed. Thanks @arnaucasau 🚀


Export of build module into plugin (+ MPS 2020.2)

Currently it is not possible to import the build script of ecmascript4mps into another external build script because it is not included in the plugin. This pull request allows to export it in the plugin.
Migration to 2020.2 is also included.

If you think this pull request is complete I'll merge it.

By the way can you update also the Travis file so the CI will run against MPS 2020.2, should be super easy just change the MPS path.

I updated the travis file, and I think the pull request is complete (after testing the generated plugin it seems to be working as expected). Thanks !


publish latest to npm?

You can always publish as beta if you're concerned about the latest release not being ready...

@jonschlinkert Would love to, but I don't have permissions on NPM yet.
@michael or @ingalls — mind adding me to npmjs.org for github-api when you get a chance? Many thanks!

understood! thanks for the hard work on this, I'm totally fine waiting :)

Done.

@michael Awesome! Many thanks!
@jonschlinkert 0.10.6 now published!

great! thanks!


Design issue: Discrepancy with font color token selection in poster

Support issue:
See images, 2 different font colors and client is asking if this is an error.

This is an error, poster was incorrect. Now fixed.


Add a few examples to the README

A small contribution from my own working code towards the effort in #21 - hopefully this fits and gets us started with including working examples.

💯 @lornajane
Great work - keep them coming :)


currentWeek doesn't restart on Monday?

I installed this module yesterday (Sunday) and today (Monday) see this:

As you can see, there is already a reading on Sunday this week, which is actually yesterday's (which is in last week) reading.
I notice currentYear.consumed.02_currentWeek shows 4.893kWh as expected but there is no reading for currentYear.consumed.02_previousWeek. Shouldn't that be 3.705kWh in this scenario?
FYI, objects under currentYear.consumed.previousWeek (not shown in the screenshot) were all zero.

solved starting from 0.4.8-alpha.5

solved starting from 0.4.8-alpha.5


tbb: 2018_U3 -> 2018_U4

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools.
This update was made based on information from https://repology.org/metapackage/tbb/versions.
These checks were done:

built on NixOS

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 2018_U4 with grep in /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4

directory tree listing: https://gist.github.com/0b2f954042266f19cb4e819607e57c03

du listing: https://gist.github.com/f115812285d438e3dc88112762777976

meta.description for tbb is: '"Intel Thread Building Blocks C++ Library"'.
cc @peti @thoughtpolice @dizfer for testing.
Experimental: this build is cached with Cachix. To use the cache follow these experimental instructions:
One time setup in nixpkgs Git checkout:
cachix use r-ryantm
git remote add r-ryantm https://github.com/r-ryantm/nixpkgs.git

Test this build:
git fetch r-ryantm && git checkout 1475ec2a7275457eb1536edcfbdbbb6612af932b && nix-shell --pure -I nixpkgs=. -p tbb

Success on x86_64-linux (full log)
Attempted: tbb
Partial log (click to expand)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4
shrinking /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4/lib/libtbb.so.2
shrinking /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4/lib/libtbbmalloc_proxy.so.2
shrinking /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4/lib/libtbbmalloc.so.2
strip is /nix/store/21ymadblbmsbb2bk4q7gl4kjasp8zmgd-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4/lib
patching script interpreter paths in /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4
checking for references to /build in /nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4...
/nix/store/bi7ml1lf51x1pvf8c9q9sg1vgyr93i10-tbb-2018_U4

Success on aarch64-linux (full log)
Attempted: tbb
Partial log (click to expand)
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4
shrinking /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4/lib/libtbbmalloc_proxy.so.2
shrinking /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4/lib/libtbbmalloc.so.2
shrinking /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4/lib/libtbb.so.2
strip is /nix/store/qg2agrqkf240s656d207zqhipl0bc2id-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4/lib
patching script interpreter paths in /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4
checking for references to /build in /nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4...
/nix/store/9wq79znbwaiyc8crjw8ig1x5q898amvj-tbb-2018_U4

Success on x86_64-darwin (full log)
Attempted: tbb
Partial log (click to expand)
clang++ -c -MMD -g -O2 -DUSE_PTHREAD -DDO_ITT_NOTIFY -stdlib=libc++ -m64 -mrtm    -fno-rtti -fno-exceptions -D__TBBMALLOC_BUILD=1  -Wno-non-virtual-dtor -Wno-dangling-else -fPIC  -I../../src -I../../src/rml/include -I../../include -I../../src/tbbmalloc -I../../src/tbbmalloc ../../src/tbbmalloc/large_objects.cpp
clang -fPIC -o libtbbmalloc.dylib backend.o large_objects.o backref.o  tbbmalloc.o  itt_notify_malloc.o frontend.o  -ldl -lpthread -dynamiclib -install_name @rpath/libtbbmalloc.dylib -stdlib=libc++ -m64  -Wl,-exported_symbols_list,tbbmalloc.def
clang++ -fPIC -o libtbbmalloc_proxy.dylib proxy.o tbb_function_replacement.o  -ldl -lpthread libtbbmalloc.dylib -dynamiclib -install_name @rpath/libtbbmalloc_proxy.dylib -stdlib=libc++ -m64
make[1]: Leaving directory '/private/tmp/nix-build-tbb-2018_U4.drv-0/source/build/macos_intel64_clang_cc5.0.2_os10.12.2_release'
installing
post-installation fixup
strip is /nix/store/yyak5sjv68n9vdgnkd2cb5djk1l9sqj5-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/07yybbbndl7v73cda63a19k1c41zsds6-tbb-2018_U4/lib
patching script interpreter paths in /nix/store/07yybbbndl7v73cda63a19k1c41zsds6-tbb-2018_U4
/nix/store/07yybbbndl7v73cda63a19k1c41zsds6-tbb-2018_U4


How to import data from Chrome extension to smartphone app?

Is it possible to use multiple devices for Google Authenticator? Currently i am using it mainly on Chrome, but sometimes i need it on my smartphone (iOS). How can i copy the keys and use them on my smartphone?

please see here: https://github.com/Sneezry/authenticator/issues/69

Thank you... Didn't noticed it 👍


Huge increase in learning time introduced in version 0.81

I recently updated my version of xgboost from 0.80 to 1.0.2 and I noticed a huge increase in learning time.
I investigated the problem and found that this increase is caused by the Span class introduced in 2c50278 and first released in version 0.81. Compared to the structure that was used before, this class does some checks (SPAN_CHECK instructions in span.h). If you remove these checks, you will notice a huge decrease in learning time (> x2 !). In fact, the learning time gets back to what it was before version 0.81.
These checks should only be present in Debug mode.

@ldesreumaux, thank you for reporting the issue.
Could you, please, provide more details (e.g. parameters and dimensions) to reproduce the issue on my side?

Here is a python script to reproduce the issue. With this script, there is a x2 improvement on my laptop when I remove the checks in version 1.0.2.
import numpy as np
import xgboost as xgb

import time

def generate_data():
    X = np.random.normal(5, 2, [n_obs, n_features],)
    y = np.random.gamma(2, 4, n_obs)
    return X, y

seed = 42
num_boost_round = 1000
early_stopping_rounds = 10
learning_rate = 0.05
tree_method = "hist"

n_obs = 5*10**6
n_features = 50

params = {
    "learning_rate": learning_rate,
    "tree_method": tree_method,
    "subsample": 0.9,
    "seed": seed,
    "silent": 1,
    "verbosity": 0
}

np.random.seed(seed)
X_train, y_train = generate_data()
X_test, y_test = generate_data()
dtrain = xgb.DMatrix(X_train, label=y_train)
dtest = xgb.DMatrix(X_test, label=y_test)

start_time = time.time()

model = xgb.train(
    params=params,
    dtrain=dtrain,
    num_boost_round=num_boost_round,
    early_stopping_rounds=early_stopping_rounds,
    evals=[(dtest, 'test')],
    verbose_eval=False
)

print("Time:", time.time() - start_time)
print("Trees:", model.best_iteration, "iterations")

A bit surprised your bottle neck is in objective.  Is the slow down still in 1.0.2?

Which compiler are you using?

I am using Visual Studio 2015 on my Windows laptop.
I also had the same difference between version 0.80 and 1.0.2 with xgboost installed via pip in a Linux environment.

Thanks for sharing the information.

@ldesreumaux Could you give latest master branch a try?  See if it makes thing a little better?

@trivialfis Thanks for the fix. With the latest master branch, the training times are in the same order of magnitude as before version 0.81, and there does not seem to be a very significant difference when I completely remove the span checks.


b2t not kicking in

val id : 'a -> 'a
let id x = x

let test5 : Type = id true

File.fst(6,22-6,26): (Error 54) bool is not a subtype of the expected type Type

The problem is that the implicit for id is not solved at the time when we're trying to coerce:
(Bug2015.fst(6,25-6,29)) Trying to coerce true from type (Prims.bool) to type ((*?u1*) _)
(Bug2015.fst(6,19-6,29)) Trying to coerce Bug2015.id true from type ((*?u1*) _) to type (Type)
Bug2015.fst(6,25-6,29): (Error 54) Prims.bool is not a subtype of the expected type Type

Annotating the implicit as either bool or Type fixes it (placing the coercion either outside or inside the id).

So this works when the x argument to id is marked with an equality. The thing is that then we get a constraint of shape ?1 = bool, so we can solve it and make progress. If not, we just get bool <: ?1.
But... from bool <: ?1 it must necessarily the case that ?1 is bool. The type bool is not a subtype of anything else as it is unrefined. I'm gonna try patching the unifier to solve these problems without deferring, which should fix this.
Also... this reminds me that on_domain could have a similar behaviour: on_domain bool f (for f : bool -> ...) could be indeed equal to f without bringing any unsoundness (or so it seems to me).

bool <: ?1, ?1 could be a refinement of bool with a formula that is valid in the current environment?

Indeed, can't say I thought of that just now :)
But... we might be OK solving the uvar to just bool there too, since those types would then be equal in that context. Inference may be affected, but it's unclear (to me) if this would be a net gain or loss.
I just wrote a quick and dirty patch which accepts this example and passes micro-benchmarks, still looking though.

I think I understand what you meant now Aseem :). The problem is something such as:
let l : list (b:bool{b==true}) : [true]

With my patch, the implicit for Cons and Nil get solved to bool, and then this fails to typecheck. I forgot that subtyping problems carry an element with them, so a problem such as bool <: (b:bool{b==true}) actually does make sense and can be solved if one can prove that the element satifies the refinement (as in this case for true).
Doing my change for problems without an element also rejects the original example (I think all subtyping problems carry an element? not sure).