k8s service names do not conform to DNS standards

Because Mantl now uses a forked version of k8s (available at https://github.com/MustWin/kubernetes), our k8s services are automatically exposed in Consul as services. For example, if you create a service named "example", you'll have the following service in Consul: "cluster-local/svc/kube-system/example". However, these /s break DNS resolution:
[root@lb-control-01 ~]# curl 'cluster-local/svc/kube-system/example.service.consul'
curl: (6) Could not resolve host: cluster-local; No address associated with hostname

Upstream issue at https://github.com/MustWin/kubernetes/issues/7

Ansible version (ansible --version): 1.9.4
Python version (python --version): 2.7.11
Git commit hash or branch: master
Cloud Environment: all
Terraform version (terraform version): 0.7.0

The latest version of kube2consul does generate valid dns names. We will need to incorporate updated k8s packaging for 1.3 to fix this but are waiting on a couple of things

an up-to-date 1.3.x kubernetes release: https://github.com/MustWin/kubernetes/issues/6
1.3 mantl rpm repo (and maybe #1736)

@ryane Was this fixed in #1793?

Not yet. The developers of kube2consul are moving the code into another repository and hopefully building a new image for us to integrate. ping @mikejihbe @johnweldon


Cannot find module 'hyperscroll'

Cannot install minbay at Win7 computer.
It fails when I try npm run build:
C:\mv\minbay>npm run build

><EMAIL_ADDRESS>build C:\mv\minbay
> node scripts/style.js && mkdir -p build && browserify index.js | indexhtmlify
> build/index.html

Error: Cannot find module 'hyperscroll' from 'C:\mv\minbay\modules'
    at C:\mv\minbay\node_modules\browserify\node_modules\browser-resolve\node_mo
dules\resolve\lib\async.js:46:17
    at process (C:\mv\minbay\node_modules\browserify\node_modules\browser-resolv
e\node_modules\resolve\lib\async.js:173:43)
    at ondir (C:\mv\minbay\node_modules\browserify\node_modules\browser-resolve\
node_modules\resolve\lib\async.js:188:17)
    at load (C:\mv\minbay\node_modules\browserify\node_modules\browser-resolve\n
ode_modules\resolve\lib\async.js:69:43)
    at onex (C:\mv\minbay\node_modules\browserify\node_modules\browser-resolve\n
ode_modules\resolve\lib\async.js:92:31)
    at C:\mv\minbay\node_modules\browserify\node_modules\browser-resolve\node_mo
dules\resolve\lib\async.js:22:47
    at FSReqWrap.oncomplete (fs.js:82:15)

Note that npm install ran successfully. My Node version is v4.2.6.

I'm trying to figure out why minbay is using hyperscroll at all. I thought it depended on https://github.com/dominictarr/hyperloadmore
I'm investigating. In the meantime, if you want this to work type npm install hyperscroll to install the dependency and any other missing dependencies of dependencies.

it turns out hyperscroll is used in the patchnav-less module that I forked. We're attempting to resolve depending on this via issue #4

@severak Ok, closing.


[Q] doc.validate() validates each field or only the modified ones?

It validates all the fields

Shouldn't it validate only the modified fields? It will make it a lot faster.
I have a collection with quite a complex + big document schema and it takes a lot of time to save it.

I can think about such solution. It shouldn't be hard to do and in deed it will speed up validation a little bit. However I'm just concerned about data being modified in a database without using Astronomy. Let's say somebody changed first name of a user:
// NOTE: Collection update
Users.update(userId, {$set: {firstName: 123}});

Now when I fetch this document and try to validate it, it will not throw any errors.
// NOTE. Class fetch
const user = User.findOne(userId);
user.validate();

True, in that case we can have a flag {all: true} as an argument to validate. Or the opposite for backwards compatibility, {modified: true}.

Feature implemented and will appear in the next release


Add input property validation for all message create() methods

Add input validation for all message create() methods by reusing the JSON schema already being used during message processing.
Example create method needing update:
https://github.com/TBD54566975/dwn-sdk-js/blob/main/src/interfaces/collections/messages/collections-write.ts#L26

Hi, @thehenrytsai I am just starting to contribute to open source projects. and I would like to work on this issue.


opencv_createsamples: command not found

Hi, the contributor and others. I am having the same problem as described previously by others.
opencv_createsamples: command not found
My OpenCV ver is 4.5 and python is 3.10
would be really nice to see any updates on this by the contributor. Thanks, we appreciate your contribution

First I've faced these issues Now its solved by using these method...
You have to create a new conda environment
conda create -n opencv_legacy
Activate that environment
conda activate opencv_legacy
install opencv3 by following command
conda install -c menpo opencv3
Note: Before running python code you should activate that environment Otherwise error will come...


[Roadmap] Gather Ideas for Q1 Sprint 2 Prioritisation

This is the issues to gather ideas for prioritization of the following sprint.
Context
You can find here a detailed FIGMA FILE that explains the Sprint Process
You can find here the Spring Planning Project Board. You can filter per milestone if you want to see other sprints. The view should normally default to the current sprint.
Who should use this issue?
This issue should be used by the Sprint Lead of the following Sprint (not the current one) to start gathering some ideas for future prioritization.
This issue should be used by anyone who wants to submit an idea for immediate consideration.
How to use this issue?
This is to gather ideas for things to be immediately built. This is not a backlog issue to dump every bug, request, etc.
Please refer to the current Strategic Focus Issues to get a sense of the current Roadmap Priorities. You can find them at the top of the Planning Board.
For FY22Q1S1 (year 22, 1st quarter, 1st sprint) and FY22Q1S2 these are:
API
Automation, Flexibility and Programmability of expenses and contributions
Community Governance
Community Growth
These areas will be further refined and reviewed every 12 weeks during the retreat week. Please refer to the Sprint Calendar to see where we are and when the next retreat week is.
Template
Here's a template you can use to add a comment below with an idea.

Who is your user?

What are they trying to achieve?

When , I want to , so I can
How are they currently doing this?

Triage Template (core team only)
This template is for members of the team to triage for prioritisation. For more guidance see https://www.loom.com/share/369ab467fbc64dec848085d38ff57ca0:
P1 high frequency, high impact
P2 low frequency, high impact
P3 high frequency, low impact
P4 low frequency, low impact
Examples of high impact - a problem affects users during an essential process (expenses and payments > onboarding and registration > contributing) with no workaround.
High frequency - >10% of users affected (measured as a proportion of total potential users for this case).

#4024 Host admin can define more manual payment methods
OCF really needs this to create a check option, and I really think it will be used by other hosts in diverse ways as well

#5129 Automate Benevity transactions
OCF is seeing growth through this payment channel and doing it manually is time consuming
MVP would be a script as described in the issue, but we could also use it as a use case for API integrations if it seems like a good option.

#3593 Host can require minimum number of admins
Another one OCF spends a lot of time enforcing manually. This has already been fully specc'd and designed. I think this is a good example of a simple buy powerful governance related feature.

#4824 Activate Goals for Projects
Users continue to create awkward workarounds using tiers with goals in projects, etc, but if goals are already working for Collectives is it really hard to just enable them in Projects?

#4806 Add consolidated balance in Budget (including projects and events)

#3903 Semi-automation of allocation payments for GitHub Sponsors
Doing this manually is getting out of hand... maybe this issue should be closed in favor of a full automation, but if that's going to take a long time, this semi-manual version would still be worthwhile.

#4056 Remove the Host Plan selection settings
I consider this still outstanding from the pricing model project I don't know how many sprints ago, so it would be great to close it off.

https://github.com/opencollective/opencollective/issues/4356 Main nav bar updates

Displaying all our marketing pages. Easier to navigate and find stuff
New pattern for notifications (Both platform and user's)

https://github.com/opencollective/opencollective/issues/3438 and https://github.com/opencollective/opencollective/issues/4346 Tiers update
Page to edit tiers
Tier card updates (Accessibility, background, consistency)

RSS FEED https://github.com/opencollective/opencollective/issues/3504

Update the Transaction and Expense Lists
Following up https://github.com/opencollective/opencollective/issues/4691, we could use the already existing designs to update the transaction and expense lists;
Expense List: https://www.figma.com/file/ZQBMWhnGGtRWeIZknFW1eP/[OC-Design]-Production-Ready-✅?node-id=15912%3A246112.
Transaction List: https://www.figma.com/file/ZQBMWhnGGtRWeIZknFW1eP/[OC-Design]-Production-Ready-✅?node-id=15912%3A246144

https://github.com/opencollective/opencollective/issues/5172 Add Support to Remove Attendees from Events

Don't think we have an existing design for this but this has been brought up multiple times so maybe we could plan this for the future.

#4798 Virtual cards for Projects
It would be really helpful for certain Project expenses to be paid using a Virtual Card. If it is simple, this would help me with a manual process I and Alina take care of on a monthly basis.

https://github.com/opencollective/opencollective/issues/2682 Customizeable Thank You Email
As requested by a user: https://opencollective.slack.com/archives/C6JTTA4SK/p1644007590497949

#3593 Host can require minimum number of admins
I think if we're going to build more governance features, this should be the next one up. Both OCF and OSC really need this, and it would help a lot for compliance and best practice. It's a governance thing that's actually a great fit for a tech solution (while many others are a messy mix of tech and human stuff). It can also prevent situations like we're having now with one OCF Collective where they are having an internal dispute and one admin kicked out all the others to take over the project. Hosts should not be dragged into this kind of thing when the tech could help prevent it.

#4694 Host admin can freeze Collective activity
This would be very useful for compliance stuff. Right now it's basically impossible to manually keep track of a special circumstance that means expenses for a certain host shouldn't be paid, as their expenses are just one of hundreds.

#3330 Host admin can unhost Collective
Another compliance one... This is just a disaster waiting to happen if there's some kind of serious breach of policy and a host can't kick a Collective out. We're about to have an ethics policy and we need basic tools like this to implement stuff like that in practice.

#5179 do not require payout method selection for cross host expenses
We'll need to solve this for Season of docs 2022. (And any other lingering issues for cross host expenses, but I'm not sure what's still needed)

https://github.com/opencollective/opencollective/issues/5179 do not require payout method selection for cross host expenses

And any other lingering issues for cross host expenses, but I'm not sure what's still needed)

@alanna Everything is tracked there https://github.com/opencollective/opencollective/issues/5064 If you have any problem with the feature, send your feedback this way.

For next sprint, we’ll normally not be changing our strategic focus. So, if you want to see a project prioritized, it really helps make it fit into one of those 4 topics. And mentioning it. https://github.com/opencollective/opencollective/issues?q=is%3Aissue+is%3Aopen+label%3A"strategic+focus"
If you think we should change our strategic focus without waiting for next discussion round (April 18th - April 22th), then start an internal discussion in the team channel before we start the prioritization process (before the end of the month).
Have in mind that we can also prioritize smaller issues through a "papercuts" process. For next sprint, we'll pick one or two "papercuts" topics out of the following one: https://github.com/opencollective/opencollective/labels/papercuts. This sprint we're currently focusing on Projects and Events.

I wasn't involved in setting the strategic focus so I'm just going to share
the issues I think are important. I don't have capacity to pre-filter them.
That needs to be owned and facilitated by whoever owns that part of the
process (who is that, by the way?). To me the strategic priorities seem
very broad. I could argue either for or against most of these issues on
that basis. And some things are prior commitments, like expenses between
hosts for season of docs, regardless of current strategy.
On Sun, 6 Feb 2022, 02:27 François Hodierne, @.***>
wrote:

For next sprint, we’ll normally not be changing our strategic focus. So,
if you want to see a project prioritized, it really helps make it fit into
one of those 4 topics. And mentioning it.
https://github.com/opencollective/opencollective/issues?q=is%3Aissue+is%3Aopen+label%3A"strategic+focus"
If you think we should change our strategic focus without waiting for next
discussion round (April 18th - April 22th), then start an internal
discussion in the team channel before we start the prioritization process
(before the end of the month).
Have in mind that we can also prioritize smaller issues through a
"papercuts" process. For next sprint, we'll pick one or two "papercuts"
topics out of the following one:
https://github.com/opencollective/opencollective/labels/papercuts. This
sprint we're currently focusing on Projects and Events.
—
Reply to this email directly, view it on GitHub
https://github.com/opencollective/opencollective/issues/5159#issuecomment-1030624856,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABJ6P7QAT62KYB7353ZVOCLUZUQUFANCNFSM5M2XYH6Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

I wasn't involved in setting the strategic focus so I'm just going to share the issues I think are important. I don't have capacity to pre-filter them.

@alanna please try to share issues here that are inside the strategic focus. Otherwise we end up with a backlog of issues here and this is not the intent of this process. I understand you were not present in the strategic focus but they are an application of the long term vision I shared in the loom.
We also have several Strategic Focus issues that not only explain the global context but they explain each focus individually. I understand you were not present in that discussion, but we had it during the retreat and I believe engineering did a fairly good job at communicating it.
Adding totally unrelated issues here will not make it easier for anyone, those issues won't get prioritized in this sprint and it only makes it harder to prioritize and frustrating for you.
If there are any questions regarding the focus, I can do another loom with @znarf to clarify.

That needs to be owned and facilitated by whoever owns that part of the process (who is that, by the way?).

As I shared in Slack and multiple calls and as it is stated in the figma file linked above, each sprint has a sprint leader. When I shared this issue in slack I was very clear that I was the next sprint leader while @znarf is this one.

This format also doesn't seem to take into consideration the weight that should be given to the priorities of our hosts, so it would be good to understand how that fits in.

This is a good point, but in this particular case it is clear that OCF and OSC priorities are taken into consideration.
The github script is ready to be implemented this batch and the expense governance is an issue OCF made a priority.
The idea is that the strategic focus can be debated and updated every 12 weeks during retreat week.

if we shut down gathering people's ideas prematurely, we're going to miss out on feedback from outside those who can directly participate in the sprint process.

I would be a bit careful with wording here, words carry weight. People are not and will never be shut down. There are multiple channels for everyone to raise an issue, we have calls every day or every week with most of the team. This is literally the sprint process, it's async and online. This is the first one and priorities were discussed during the live retreats but again I think we are communicating this quite clearly. The next one will happen in April.

And some things are prior commitments, like expenses between hosts for season of docs, regardless of current strategy.

The sprint is designed to leave space for papercuts and other issues like these. This is also specified in the figma file above or to @znarf´s comment in this thread.

All the issues I suggested are either related to the strategic focus points, or are things we've already committed to do and need to finish (like removing the pricing plans and the cross-host expenses). That is why I suggested them. I stand by my suggestions and want them considered for the sprint. If they don't make the cut, so be it, but they should be considered. I don't think it's constructive to try to exclude suggestions. It's useful data even if ultimately other issues are prioritised.

All the issues I suggested are either related to the strategic focus points, or are things we've already committed to do and need to finish.

Ok great. If this is the case, then what's the issue with the process? I am missing the source of the conflict withe the sprint design.

For reference here are the Papercut issues that are prioritized and the team has been working on: https://github.com/opencollective/opencollective/issues/5084

I'm a bit lost about this process to be honest, will post an issue I think is important here, I think it fits in the Strategic Focus too:

#2682 Customizable Thank You Email
As requested by a user: https://opencollective.slack.com/archives/C6JTTA4SK/p1644007590497949

@kaylarep it is unlikely we are able to undertake this one, since it's quite a big change. Do you have a second priority you can share with us?

For API Strategic Focus: Programmatically authenticate users for metagov integration https://github.com/opencollective/opencollective/issues/5140#issuecomment-1031733333

Avatar design update
I think is a quick win and it would improve the current onboarding experience

### Event papercuts

as a event organiser I would like to remove an event attendee

as an event attendee I would like to cancel my attendance at an event

#4521 Add 'add to calendar' buttons for event sign ups.
As an event attendee, I would like to make it more likely that I'll remember to show up for an event I've signed up for.
And: as an event organiser, I'd like to make it more likely that the attendees at my event will show up, so I can better plan to work with the participants.
Currently, the user does get a .ics attachment as part of the confirmation email, but I suspect that if we add an "add to calendar" button just after the registration flow, people will use it and we'll see a decrease in no-shows.

Reviewing email contact details
In some cases we receive support requests that would be more appropriate for the host or Collective to answer. This results in unnecessary load on our support teams. It would be good to review when we notify users of a situation that includes a support@ email address and replace it with the contact details of the host/Collective.

Expense Across Hosts
Follow up: https://github.com/opencollective/opencollective/issues/5064

https://github.com/opencollective/opencollective/issues/5179
https://github.com/opencollective/opencollective/issues/5068
https://github.com/opencollective/opencollective/issues/5260

oAuth
https://github.com/opencollective/opencollective/issues/1660
We can't make it happen in one sprint, but we can work on the foundation and kick start design.
Strategic Focus: https://github.com/opencollective/opencollective/issues/5140

Recurring Expenses
#5144

We'll have to iterate on https://github.com/opencollective/opencollective/issues/5146, the project is not completed yet. More info to come about what's left to do.

Update sender can select which Project/Event/Tiers
Currently we send parent collective updates to child collectives. But it is mentioned that the best case solution is to let the user select which users they want to send the updates; https://github.com/opencollective/opencollective/issues/4673#issue-995325149

Update the page to find a host https://github.com/opencollective/opencollective/issues/4240
I deleted the stuff I proposed about tier cards and left this on instead. Hearing from folks in the last calls I had, I realized that it's super important, and we are not doing a very good job at that.

Virtual Card Receipt Upload Reminder and Card Freeze https://github.com/opencollective/opencollective/issues/5269
Right now this process is manual, and that is not ideal/scaleable

Automated event reminders for signed-up participants #5237
As an event attendee, I would like to make it more likely that I'll remember to show up for an event I've signed up for, and to do any pre-work the organizer has assigned ahead of time.
And: as an event organiser, I'd like to make it more likely that the attendees at my event will show up, so I can better plan to work with the participants. In order to inform the participants of pre-work/homework they should do to make the session more effective, or remind them to attend, I have to manually create an event update that gets emailed to them, and it does not have a .ics calendar invitation attachment. If I am busy then that task may not happen in time.
Currently, the user does get a .ics attachment as part of the confirmation email, but does not get any automated reminders from OpenCollective about the event. I got requests for these kinds of automated reminders from participants who accidentally did not show up to events I ran recently.

Increase flexibility for cancelling event registrations #5283
As a person who has registered for an event, or as an admin/project owner for an event, I can cancel my/the user's existing registration for an event, to reflect that the user is not planning to attend or has been removed from the participant group.
For recent workshops, I had to contact Ben or Nathan to ask them to remove participant registrations (either because the participants emailed to say they couldn't make it, or because I was removing a participant who had not shown up for multiple previous events they'd registered to attend).
Making this more self-serve, for users and for event organizers/admins, would have helped reduce this work and make it faster, and would have made it easier and faster to recruit more participants to take up those newly-freed slots.

Event signup waitlist #5284
As a prospective participant in an event that is currently full, I want to be able to sign up for a waitlist, so that in case a slot opens up, I can get registered and attend. I don't want to have to manually go check the event frequently to see whether a slot has opened up.
As an event organizer for an event that is currently full, I want to have prospective participants on a waitlist and have them automatically notified in case there is space for them, so that if a user cancels, I'll have greater utilization and less waste at the event.

New Homepage

Finish design
Test with users
Push to implementation

Track Balance on Virtual Cards https://github.com/opencollective/opencollective/issues/4556
if not this sprint, roll over to the next. It's difficult for collective admins to track their virtual card balances - they either need to do it manually, or ask a host admin to look- neither option is great.

Closing this issue as we have the board set up over at https://github.com/orgs/opencollective/projects/5/views/19 and the discussion recorded at https://drive.google.com/file/d/1o9ExAZxo-xlYfP4ZeC-Hlr-snEcNnHLw/view?usp=sharing


mesh_get_index returns a value of -1 when a particle lies on mesh minimum boundary

The mesh_get_index returns a value of -1 when a particle is at the lower boundary of a mesh in any of space or time. This is problematic for having an initial condition or boundary source. Often, the effect is unnoticed as Python will treat the -1 index to reference the last element of the array without error or warning.
A workaround has been to modify the mesh in the input file with some small perturbation.

That is true. The current solution is that users need to move the boundary/point source into the domain by manually specifying it in the mcdc.source.


[Java] Development Java JNI Wrappers for Substrait (*.h , *.cc)

Describe the enhancement requested
Review last PoC https://github.com/apache/arrow/issues/34223
Define if this new functionality will be created on:

Extend current Java component, or
Create new Java component to handle Java Substrait consumption

Component(s)
Documentation, Java, Packaging

This enhancement are ready for review at: https://github.com/apache/arrow/pull/34227


Quick and Practical example of Hexagonal Architecture in Java

Pull request: Hexagonal Architecture in Java

Pull request for https://github.com/eugenp/tutorials/pull/10871/commits/2b7cbdc9ef1892abb55f1c0bf34689184c88397e


Add Yuval207

Related issue

What changes does this PR do?
Add Yuval207
Screenshots (If any)
Questions

Is there something that I need to fix as my code is not being merged in the master branch ?

Is there something that I need to fix as my code is not being merged in the master branch ?

No just wait for another maintainer to approve the pull request and than the changes will get merged.


Disable Local Auth in Event Hub namespaces.

If  the client is using Event Hub cryptographic keys, or Entra ID/ Managed ID-- can we please disable local auth in the Bicep template here https://github.com/Azure-Samples/explore-iot-operations/blob/125ada5db88f6359e6a31d1e88c27869e8b30104/tutorials/mq-realtime-fabric-dashboard/deployEdgeAndCloudResources.bicep#L363-L363
See : https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.EventHub.DisableLocalAuth/
Tracking issue for:

[ ] https://github.com/Azure-Samples/explore-iot-operations/security/code-scanning/1

This doesn't work with Fabric Real-Time Intelligence yet which needs SAS token based auth.


use <> consistently for includes

"" will look in local directory first. <> uses only the includes passed via -I, which makes it more universal

closed with #189  and #186


Add calculate rotated bounds plus extends transform by child styles.

Could you add some notes on what this accomplishes?

Hello, sure. Its fixing problem if you have draggable element (like text) with rotate css transform with bounds to his parent.

Thanks for the contribution, but I consider this out of scope for Draggable - this is a lot of code to maintain for something that is truly niche. If you want to rotate your text, simply wrap the text in a <span> so the rotation doesn't mess up the transform.
If that doesn't fix your problem, feel free to build on top of <DraggableCore> with your custom transform.


SDL 0167 - App Services - Media Service

Proposal: SDL 0167 - App Services - Media Service

SmartDeviceLink has always been limited to a 1 - 1 relationship between module and smartphone apps. However, to create truly compelling experiences this should be expanded. This proposal will aim to introduce a paradigm that allows apps to advertise a "service" they offer that the module and other apps can leverage. This allows users to augment their vehicle's IVI system with off-board solutions. This proposal also details how SDL apps can communicate with each other through the module.

https://github.com/smartdevicelink/sdl_evolution/issues/557
Steering Committee Decision:

The Steering Committee voted to accept the Media Service type section of the larger App Services proposal, with the revisions included in this comment.

The accepted revisions were made on 2019-01-24.

Closed via #149


Cannot find module '@angular/platform-browser/animations'

I have followed the official documentation ro install material design for angular2, but when I import BrowserAnimationsModule:
import {BrowserAnimationsModule} from '@angular/platform-browser/animations';
I get this error:
Cannot find module '@angular/platform-browser/animations'
this is my package.json file:
"dependencies": {
    "@angular/animations": "^4.2.5",
    "@angular/common": "^2.4.0",
    "@angular/compiler": "^2.4.0",
    "@angular/core": "^2.4.0",
    "@angular/forms": "^2.4.0",
    "@angular/http": "^2.4.0",
    "@angular/material": "^2.0.0-beta.7",
    "@angular/platform-browser": "^2.4.0",
    "@angular/platform-browser-dynamic": "^2.4.0",
    "@angular/router": "^3.4.0",
    "core-js": "^2.4.1",
    "hammerjs": "^2.0.8",
    "rxjs": "^5.1.0",
    "zone.js": "^0.7.6"
  },

You need to update your `@angular/platform-browser

this is not related to a bug as I have answered your question in depth on StackOverflow over here. @DevVersion issue to be closed.


Feature request:  add animations to shapes

Request: add animation effects to existing PowerPoint shapes.

Would also love to have this for my use case!
Any idea @scanny if this is still something you're looking to implement in the near future? Or is it somewhere below in the backlog?
Many thanks for the awesome package btw, great work!


Ability to propagate custom options to kubelet

Main motivation for us is to utilize
https://kubernetes.io/docs/concepts/cluster-administration/kubelet-garbage-collection/
Anything from https://kubernetes.io/docs/admin/kubelet/
can be potentially used as a value

Codecov Report

Merging #789 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #789      +/-   ##
==========================================
+ Coverage   34.71%   34.73%   +0.01%
==========================================
  Files          57       57
  Lines        3678     3679       +1
==========================================
+ Hits         1277     1278       +1
  Misses       2256     2256
  Partials      145      145

Impacted Files
Coverage Δ

core/controlplane/config/config.go
59.83% <100%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2b77725...e3a5e62. Read the comment docs.

@ytsarev Thank you very much for your contribution 👍
Not saying this shouldn't be merged but just curious - what would be pros of exposing this via cluster.yaml?
Like DOCKER_OPTS passed to dockerd, we've been recommending users to edit and version-control files generated by kube-aws so that they can make all the kinds of changes possible via not only cluster.yaml but cloud-configs and stack templates.
Do you find exposing it via cluster.yaml better for some reasons?

@mumoshu actually we intentionally never modify cloud formation templates. We version control them , but modification go strictly through kube-aws render stack. This way we avoid any ad-hoc modifcations and possible incompatibility with a tool. I believe such approach brings reliability I would like to stick with it.

@ytsarev Thanks, I understand the motivation 👍

Hey @redbaron, WDYT on this?
I believe we can't prioritize these kinds of improvements over other scalability, availability, reliability things we're focusing on - creating a cloudformation.yaml isn't our focus :wink:
However, I also believe that we can accept contributions of these kinds as long as it doesn't badly affect our main focus. I understand @ytsarev's motivation on this and it doesn't seem to affect our focus badly. It is useful and orthogonal to our focus.
So, I'm wanting this to be merged. WDYT?

@ytsarev , there is an issue about git workflow where it is possible to keep your customizations and continue receiving updates from kube-aws: https://github.com/kubernetes-incubator/kube-aws/issues/462
@mumoshu , PR looks allright, cluster.yaml already grew beyond point of no return :)

@redbaron

cluster.yaml already grew beyond point of no return :)

True :rofl:

Looks good now, thanks for your contribution!


feat(.travis.yml,deploy.sh): push docker image from all branches

images pushed from branches will be pushed with the tag test-$BRANCH
Thoughts @mboersma? It looks like this could work with Travis and it'd sure be nice to have all PRs built and deployed. Maybe a good idea to push to a different org entirely (deispulls for example)?
If this or some future variant of it are ok I'd like to make the change across the other repos that push Docker images as well.

As long as @sgoings is okay with this, it's alright with me. I'll LGTM it, but I'd like the second LGTM to come from @sgoings, since he's the one who drew the short straw for keeping Travis configs consistent.

thanks @technosophos. @sgoings can you give feedback here? this isn't part of v2.0-alpha1 so no rush


Importing large CSV file results in erroneous NULL values

I recently imported a CSV file of twitter data I collected that contains over 120,000 rows. When I inspected this data in CartoDB's table view I find many NULL values for rows towards the bottom of the table. I do not have this problem when viewing the CSV in Open Office or QGIS. Here is a link to the CSV file, and Here is where the imported table is in my account.
Using Chrome, Mac OS X 10.9.4

Not reproducible anymore, neither the CSV or the CartoDB table are live anymore. Chris feel free to reopen if you have some clues that we could use. We have been working in some improvements for CSV lately.


Compilation error: ContextBootstrapInitializer.java:[24,56] cannot find symbol

I get this error when trying to follow the instructions in my forthcoming JHipster Native blog post. I swear it worked last week!
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/mraible/Downloads/jhipster-native/target/generated-runtime-sources/spring-aot/
  src/main/java/org/springframework/cache/annotation/ContextBootstrapInitializer.java:[24,56] cannot find symbol
  symbol:   method setConfigurers(java.lang.Object)
  location: variable bean of type org.springframework.cache.jcache.config.ProxyJCacheConfiguration
[INFO] 1 error

I'm not sure how to fix this since it's a generated class.
If I revert to Spring Native 0.11.2, it happens too.

It seems like this code isn't working for some reason.

I'm not sure if it's related, but I also get this error when I first create the app.
sh: rimraf: command not found
ERROR! Command failed with exit code 127: npm run clean-www

Steps to reproduce:
git clone https://github.com/oktadev/auth0-full-stack-java-example.git jhipster-native
cd jhipster-native
rm -rf *
jhipster-native --with-entities

Steps to reproduce:
git clone https://github.com/oktadev/auth0-full-stack-java-example.git jhipster-native
cd jhipster-native
rm -rf *
jhipster-native --with-entities

Tested using -b spring-native at clone command and the error didn't happened.
jhipster-native should be used with --skip-jhipster-dependencies since it's not released yet (already done at native branch).

If I do the following, my apps still has caching:
npm i -g "mshima/generator-jhipster-native#config"
git clone https://github.com/oktadev/auth0-full-stack-java-example.git jhipster-native
cd jhipster-native
rm -rf *
jhipster-native --with-entities --skip-jhipster-dependencies

If I cat the .yo-rc.json file for cache, it shows ehcache is still enabled.
$ cat .yo-rc.json | grep cache
    "cacheProvider": "ehcache",

Hmmmm, I thought it would update .yo-rc.json to disable caching. For my blog post, should I change it from:
rm -rf *
jhipster-native --with-entities

To:
rm -rf *

Edit .yo-rc.json to disable caching and disable Hibernate's cache (since it's not supported by Spring Native yet).
"cacheProvider": "no",
"enableHibernateCache": false,

Then, re-create your app:
jhipster-native --with-entities


Text completion with packing?

Why we cant use packing with text completion training? Also can we use text completion training inside of the trl for teaching new information? Instead of finetuning model with instruction response pairs we finetune on raw corpus text.

Are you talking about masking the instructions? It would require a complicated attention mask to make sure we out the instructions of each example. In principle it is not quite clear yet if masking the instructions really helps and might depend a lot on the diversity of the instructions for example. And yes, you can use the SFTTrainer to fine-tune on raw text.

Are you talking about masking the instructions? It would require a complicated attention mask to make sure we out the instructions of each example. In principle it is not quite clear yet if masking the instructions really helps and might depend a lot on the diversity of the instructions for example. And yes, you can use the SFTTrainer to fine-tune on raw text.

I was considering both aspects. Indeed, I've learned that raw text fine-tuning is supported in TRL. I have a related question: suppose I aim to train the model on English quotes, like the example below. According to the TRL SFTTrainer documentation, dataset preprocessing, including packing, is automatically handled by SFTTrainer. Does this also mean that beginning-of-sentence (BOS), end-of-sentence (EOS), and padding tokens are automatically managed if we simply provide the necessary fields to the trainer? I attempted training with this logic, and the loss consistently decreased across multiple epochs; the model's loss was down to 0.6. Does this indicate successful learning? Also, if I disable packing, will the tokenization and handling of special tokens still occur?
Here's how I loaded the dataset and set up the trainer:
Load the dataset
from datasets import load_dataset
dataset = load_dataset("Abirate/english_quotes", split="train")
Set up the trainer
from trl import SFTTrainer
from transformers import TrainingArguments
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
dataset_text_field="quote",
max_seq_length=max_seq_length,
packing=True,  # This option packs short sequences together to save time.
args=TrainingArguments(
per_device_train_batch_size=3,
per_device_eval_batch_size=1,
gradient_accumulation_steps=1,
warmup_ratio=0.05,
max_grad_norm=1.0,
num_train_epochs=3,
learning_rate=2e-5,
fp16=not torch.cuda.is_bf16_supported(),
bf16=torch.cuda.is_bf16_supported(),
logging_steps=1,
optim="paged_adamw_8bit",
weight_decay=0.01,
lr_scheduler_type="cosine",
seed=3407,
output_dir="outputs",
),
)


Add climate and clusters to Industrial buildings

First:
None of the buildign have artic climate. If the have a snow graphic, and seems to fit for north of the polar circle, please consider using artic as well, or town growth will be very much screwed up in  artic.
Second:
cluster show not be used much, and only for identical style. Like when there is a certain row of houses, then cluster makes it more likely, that a building with the same rowhouse (or a level up or down). Industrial houses are clustered next to industrial houses already automatically. So please remove the cluster parameter unless these are really identical and must be next to each other.
Some examples for clusterign would the the res block houses.
I think the current layout with a single file per images is not very friendly to edits or to get an overview. I will see if I can merge the files into larger image to see all corresponding images at once.
Also did you check the look of the cities by creating a and grow a city (with and without timeline).


Update HomeController.cs

added new action

aaa

It was a mistake during a RedHat workshop. Sorry for inconvenience.


add special node-labeller annotations

What this PR does / why we need it:
add special node-labeller annotations
this annotations causes that node-labeller will skip reconciling the
node which has this annotation
Which issue(s) this PR fixes:
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1956792
Special notes for your reviewer:
Release note:
add node-labeller.kubevirt.io/skip-node annotation

Signed-off-by: Karel Šimon<EMAIL_ADDRESS>

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1956792

Did there something specific happen, or was this just a thought that it would be great to have this?

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1956792

Did there something specific happen, or was this just a thought that it would be great to have this?

In the past, there was an issue, that libvirt incorretly reported one cpu as supported, but it wasn't. So customers had to replace label with that cpu model. And currently it is not possible, because labeller will remove all user defined labels with labller syntax.

@rmohr there is already functionality obsoleteCPUModels, where you can specify models which should not be on the nodes. Issue is it is cluster-wide functionality and we want something to be only per node.

/retest

@davidvossel  can you please review?

/retest

/retest

/retest

@davidvossel, @rmohr can you please review?

/lgtm
/approve

There was an inm cloud outage around that time.

@rmohr can we please backport it to 4.8?

/cherrypick release-0.41


ci: Add the pull request validation workflow

:tada: This PR is included in version 2.0.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


error: illegal IP address string passed to inet_pton

I get this error when bots are trying to access my django site. Any idea how illegal IP address should be dealt with?
Internal Server Error: /
Traceback (most recent call last):
File "/var/lib/openshift/5409d566e0b8cd8c7a0002b6/python/virtenv/lib/python2.7/site-packages/django/core/handlers/base.py", line 87, in get_response
response = middleware_method(request)
File "/var/lib/openshift/5409d566e0b8cd8c7a0002b6/python/virtenv/lib/python2.7/site-packages/easy_timezones/middleware.py", line 42, in process_request
tz = db.time_zone_by_addr(ip)
File "/var/lib/openshift/5409d566e0b8cd8c7a0002b6/python/virtenv/lib/python2.7/site-packages/pygeoip/init.py", line 583, in time_zone_by_addr
ipnum = util.ip2long(addr)
File "/var/lib/openshift/5409d566e0b8cd8c7a0002b6/python/virtenv/lib/python2.7/site-packages/pygeoip/util.py", line 39, in ip2long
return int(binascii.hexlify(socket.inet_pton(socket.AF_INET6, ip)), 16)
error: illegal IP address string passed to inet_pton

@vinitkumar's patch works for me.  I was encountering this in the https://github.com/torchbox/wagtail CMS admin panel when previewing a page.
In case anyone is following along at home...
Wagtail's "preview this page" functionality issues a dummy request to compute the previewed content.  This presumably passes a fake requestor IP address into the dummy request.  The django-easy-timezones library has a piece of middleware installed which attempts to infer a time zone based on geolocating the requestor IP.  So, when Wagtail issues the dummy request, the bad IP address causes django-easy-timezone to pass a malformed IP to inet_aton, which blows up.
@vinitkumar's additional well-formedness check will defensively prevent not only IPV6 addresses from causing an issue, but any malformed IP address.

Looking more closely at easy_timezones.middleware.EazyTimezoneMiddleware.process_request, is it reasonable for it to change:
            if ip != '<IP_ADDRESS>':
                # if not local, fetch the timezone from pygeoip
                tz = db.time_zone_by_addr(ip)

to also check for a missing IP:
            if ip and ip != '<IP_ADDRESS>':
                # if not local, fetch the timezone from pygeoip
                tz = db.time_zone_by_addr(ip)

?

There seems to be some prior Django discussion that discourages relying on REMOTE_ADDR: https://www.djangoproject.com/weblog/2009/jul/28/security/#secondary-issue so perhaps the if ip and ip != '<IP_ADDRESS>' is the way to go?

I like that @jasonm.
PR's graciously accepted - if you're using the IPV6 database, please update the documentation to reflect that as well.
Thanks!

The supplied code wasn't totally valid, but I've added a different solution and now it should work. I've also started adding tests.

Nice! will you push it to pip as well? Will you add IPv6 support with GEOIP file for IPv6?

@lucas03 - IPv6 support based on the GeoIPv6 file is available in version 0.6.0, published to pip.
Both of the GeoIP files are now included by default for convenience.


added info about sensitivity labels

https://github.com/MicrosoftDocs/microsoft-365-docs/issues/1765

Dear @JohanFreelancer9,
kindly do the copy editor review for this PR.
Dear @mypil, for your information
Thank you

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

@MaratMussabekov  - please merge the commit on the copy/edit review done by JohanFreelancer9 or make the necessary changes on the PR following the suggestions made. Thank you.

@KCCross - PR has been copy edited and is ready for final review. This is based on issue #1765.
Thank you.
cc: @kenwith

@KCCross - PR is ready for final review.

Hi Carol, can you please confirm this before I accept it? I realize it's old.

Hello @cabailey , thank you for your comment,
we can introduce sensitivity labels that apply encryption as the recommended way to encrypt email - I am not sure @DeanGross was talking about this, for me it looks more like it was asked whether labels could be applied to messages that already encrypted.
Dear @DeanGross, could you please have a look?
Thank you


Error when sending transactions

What is the bug?
When sending a specific amount the tx submission fails with bad-tnx-sapling-unknown-anchor
Additional context
A note is marked as unspent when it is. Rescanning fixes the issue
Solution
"Rescan from height"

Mentioned internally here on 2/18 and here on 2/10

Let's discuss all the analytics we'd like to have when this occurs in our internal wallet and then add whatever helpful information we can. For instance: a running list of reorgs that occurred and how far we rewound the chain to address them. Perhaps also the anchor height of a failed submission, along with a timestamp so we can cross-check the expected height at that time.

That's a great idea.
I don't think that we have access to the anchor height chosen but we can estimate it from the expiry height. Not the best solution though :/

@defuse I can recreate the conditions based on our suspicions. That means, using librustzcash crates 0.4 crates with sdk 0.9.3, creating a no change transaction, send it to lwd and reorg the transaction. The problem is that we will never get to see the error in reality because darksidewalletd can't validate consensus rules since it holds a synthetic chain.
Any Ideas?

Here's the complete test on the experiment branch. it obviously fails
https://github.com/zcash/ZcashLightClientKit/pull/new/experiment/bad-anchor-research

Ah, good point @pacu! To properly test this, lightwalletd will need to compute the correct merkle tree roots (anchors) and check that all the transactions that get submitted to it reference a valid anchor. It would also be nice to have lightwalletd check the zero-knowledge proof.
We should definitely avoid going down the path of re-implementing zcash consensus rules in darksidewalletd, because that will be a lot of work and it will inevitably behave differently from zcashd making the tests inaccurate. It's probably worth adding just these checks since they are what will catch the most severe kinds of scanning errors.
I'll open a ticket in the lightwalletd repository for discussing and tracking this.


VS says it is error (not just warning)

C VS19
PCONSOLE_SCREEN_BUFFER_INFO myConInf;
GetConsoleScreenBufferInfo(hConsole, myConInf);
~~~~~~~

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: aae3ddaa-0a4d-3cdf-9d99-0b5c4f79ac16
Version Independent ID: 3bdbd4be-d153-6717-351f-d1f7cebc9848
Content: Compiler Warning (level 1 and level 4) C4700
Content Source: docs/error-messages/compiler-warnings/compiler-warning-level-1-and-level-4-c4700.md
Product: visual-cpp
Technology: cpp-diagnostics
GitHub Login: @corob-msft
Microsoft Alias: corob

ok, my bad... I forgot '&' to get the address... it makes sense


[RFC] A faster version of Gamma sampling on GPU.

Description
Sampling from Gamma distribution requires rejection sampling, which is applied in the implementation of mxnet.ndarray.random.gamma().  However, two main drawbacks exist in the current implementation ( sampler.h )

Random numbers used in the rejection sampling ( N(0,1) and U(0,1) ) are generated inside the kernel using CUDA device api. Also, although every batch of threads has its own RNG, samples are actually generated in serial inside each batch of threads.

Rejection sampling is achieved by using an infinite while loop inside the kernel, which may potentially affect the performance on GPU.

To solve the problems above, I write a new version of  Gamma sampling on GPU innovated by  this blog post
Implementation details
My implementation differs from the current version in the following aspects:

Instead of generating samples in the kernel, we generate them in advance using host api, which allows us to  fill a buffer with random samples directly.

Redundant samples are generated to replace the while loop. Suppose we are going to generate a Gamma tensor of size (N,),  N x (M + 1)  zero-one gaussian samples and N x (M + 1) zero-one uniform samples will be generated before entering the kernel, where M is a predefined const. For each entity, we generate M proposed Gamma r.v. and then select the first accepted one as the output.  The one extra sample is required when \alpha is less than one.

In case all M proposed samples get rejected in some entities (which would be marked as -1), we simply resample the random buffer again and perform another round of rejection sampling, but only at the entities that fail the last round.

Here's part of the implementation :
(https://gist.github.com/xidulu/cd9da21f2ecbccd9b784cadd67844e23)
In my experiment, I set M to be 1 ( i.e. no redundant samples are generated.) as the adopted policy(Marsaglia and Tsang's method) has a rather high acceptance rate of around 98%.
The profiling result is listed below:

Size
native numpy
ndarray on GPU
my implementation

10e2
<0.1ms
3~5ms
0.5~0.7ms

10e4
0.76ms
7.6~7.8ms
0.72~0.76ms

10e6
70ms
12~13ms
3.1ms

10e8
7200ms
1600~1700ms
150~160ms

The new version is currently under development on  numpy branch. It also designed to support broadcastable parameters.

Correctness has been verified with the following script:
import numpy as _np
import mxnet as mx
from mxnet import np, npx
from mxnet.base import MXNetError
from mxnet.gluon import HybridBlock
from mxnet.base import MXNetError
from mxnet.test_utils import same, assert_almost_equal, rand_shape_nd, rand_ndarray
from mxnet.test_utils import check_numeric_gradient, use_np
from mxnet.test_utils import verify_generator, gen_buckets_probs_with_ppf, retry
import scipy.stats as ss
import random

samples = 1000000
trials = 8
num_buckets = 5
ctx = npx.gpu(0)

for alpha, beta in [(2.0, 3.0), (0.5, 1.0)]:
    buckets, probs = gen_buckets_probs_with_ppf(lambda x: ss.gamma.ppf(x, a=alpha, loc=0, scale=beta), num_buckets)
    buckets = np.array(buckets).tolist()
    generator_mx = lambda x: np.random.gamma(alpha, beta, size=samples, ctx=ctx).asnumpy()
    verify_generator(generator=generator_mx, buckets=buckets, probs=probs,
                        nsamples=samples, nrepeat=trials)
    generator_mx_same_seed =\
        lambda x: _np.concatenate(
            [np.random.gamma(alpha, beta, size=(x // 10), ctx=ctx).asnumpy()
                for _ in range(10)])
    verify_generator(generator=generator_mx_same_seed, buckets=buckets, probs=probs,
                        nsamples=samples, nrepeat=trials)

UPDATE:
ndarray on GPU is actually 50 times faster than I profiled, as I did not turn on the release option during my testing, which prevented the CUDA compiler from further optimization.
In short, my implementation is not fast. However, features like broadcastable parameters and global random seed mechanism still remain useful.
I will be looking into the performance issue in the following weeks. Hope I could find a solution.

cc @apache/mxnet-committers I think we can gradually refactor current implementation (ndarray api) by adopting this new approach.
@xidulu could you please fix the url links in your post.

cc @apache/mxnet-committers I think we can gradually refactor current implementation (ndarray api) by adopting this new approach.
@xidulu could you please fix the url links in your post.

Links fixed.

Hi @xidulu. I did not look at the differences in the implementation of host-side vs device-side API for RNG in MXNet, but if they are comparable in terms of performance, a possible better approach would be something like this:

launch only as many blocks and threads as necessary to fill the GPU, each having their own RNG
use following pseudocode

while(my_sample_id < N_samples) {
  float rng = generate_next_rng();
  bool accepted = ... // compute whether this rng value is accepted
  if (accepted) {
    // write the result
    my_sample_id = next_sample();
  }
}

There are 2 ways of implementing next_sample here - either by atomicInc on some global counter or just by adding the total number of threads (so every thread processes the same number of samples). The atomic approach is potentially faster (as with the static assignment you could end up hitting a corner case where 1 thread would still do a lot more work than the other threads), but is nondeterministic, so I think static assignment is preferable here.

Hi @ptrendx , thanks for your reply, according to my discussion with @yzhliu , device-side API is much slower than host-side API.
Also, could you please talk a little bit about the advantage of your approach compared with mine? thx :)

@ptrendx If I understand correctly, "static assignment" is what current mxnet is doing, which is "ndarray on GPU" in @xidulu 's table.

@yzhliu No. What MXNet currently does is a scheme where, yes, each thread gets assigned statically some number of elements, but it has a while loop for each of them. The scheme I proposed has a single while loop that processes all elements assigned to a given thread. There is a big difference between these approaches, due to SIMT architecture of the GPU. Basically you can treat some number of threads (called warp, 32 threads on NVIDIA's GPU) as lanes in SIMD vector instruction on the CPU. This means that if 1 thread needs to perform some computation, all threads in the warp need to perform the same instruction (and possibly discard the result).
So in the current MXNet's implementation for each output element every group of 32 threads is always doing the number of loop iterations equal to the slowest thread (because no thread in warp can exit the while loop while at least 1 thread is still not finished).
In the proposed implementation there is only 1 while loop and the only difference between threads lies inside the if (accepted) part, which is cheap compared to generating a random number. In this implementation every warp does the number of loop iterations equal to sum of the steps for the slowest thread (which is hopefully pretty uniform across threads, especially as we are talking RNG and not some crafted input, and definitely much better than the previous "for each element take the slowest and sum that").
@xidulu What is the RNG used for host-side and device-side API? cuRAND ones should not really differ much in perf between device-side and host-side.
There are a few advantages:

you don't need to store and load the RNG numbers you made (and in the fully optimized case making random numbers should actually be pretty bandwidth-limited operation)
you don't need additional storage (besides the RNG generator state which you need anyway)
you compute only as many RNG numbers as you really need

@ptrendx
The device-side api I mentioned is the RandGenerator class. (the one used in ndarray.random()), it generates random number with curand_uniform():
https://github.com/apache/incubator-mxnet/blob/master/include/mxnet/random_generator.h#L111
Host api can be seen here (the one I used)
https://github.com/apache/incubator-mxnet/blob/master/3rdparty/mshadow/mshadow/random.h#L370
Random numbers are generated with curandGenerateUniform()
In terms of random number generation, RandGenerator (which is basically a wrapper over the CUDA device api, IMO) may be comparable to mshadow/random.
However, is it possible that the overhead of managing random states in RandGenerator affects its performance ?

@ptrendx Thanks now I got what you mean.  I'm open to what you proposed. while I think one of the major problems with the device api is the maintenance of the random generator (and it's states).

@ptrendx @yzhliu
I will create a PR for np.random.gamma implemented using the method I proposed before the end of the week, as I need to proceed to implement more distribution samplers, in which the gamma sampler serves as a necessity.
Refactoring  nd.random may be left for further discussion.
(P.S. Personally speaking, I think performance is not the only problem in nd.random, the poor support for broadcasting parameters and shapes may bring trouble to heavy users of the random sampling module. )

This feature has been accepted. Thanks @xidulu for the contribution!


Please update the document accordingly (For Networking and Group Policy).

According the support engineer, "the VM is encrypted using single pass ADE, for single pass ADE, the VM is not need to have connection with internet or even with Azure public IP. the public doc is not up to date yet I think. The VM will use the internal wireserver IP <IP_ADDRESS> to upload the keyvault directly to host instead of the keyvault endpoint IP address. And the IP <IP_ADDRESS> can only be block by VM Guest level firewall or UDR".

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b7e35766-c75a-9294-70b3-dcd49fe17c5c
Version Independent ID: 65f236ae-1cbf-f986-42ee-327a25a6df56
Content: Azure Disk Encryption with Azure AD (previous release) - Azure Windows Virtual Machines
Content Source: articles/virtual-machines/windows/disk-encryption-overview-aad.md
Service: security
GitHub Login: @msmbaldwin
Microsoft Alias: mbaldwin

Closing. #please-close


Invoke-DbaDatabaseUpgrade Update

Updated output

hum -hey @Staggerlee011 - can you sync dev before submitting PR? This one has conflicts.

Nm, I'll merge and fix <3


Make avoiding circular allocation flow an option in the TOML

What
When investigating the behavior of the allocation solver, it was noticed that users use their own return flow if their return flow edge is upstream of their inflow edge. In general this is undesired behavior, since this flow does not originate from a source. Therefore in allocation.jl there is the function avoid_using_own_returnflow!, which removes the return flow edge from the allocation graph. Currently this always happens, but it can be desirable in certain cases to leave these edges in for allocation.
Why
I don't know the hydrological reasons for leaving these edges in, but I believe @gijsber suggested to make it an option.
How
Several steps must be undertaken to include this option:

[ ] add the option of deleting these problematic edges to the struct allocation (with default value true) in config.jl
[ ] In allocation.jl, make sure avoid_using_own_returnflow! is only called when this option is true
[ ] Update the documentation to include this option (docs.toml)

@SouthEndMusic : is currently this provided in a log-message?

This issue is made obsolete by https://github.com/Deltares/Ribasim/issues/78.

This issue is made obsolete by https://github.com/Deltares/Ribasim/issues/78.


fix(hono-base): custom not found with middleware like compress

Fixed Custom NotFound not working with middleware that creates a new Response object after next() such as Compress Middleware. And added proper tests.
This was fixed by #2080, but it may not be merged to the main branch well. And there is no test to handle it, so we can't notice it has the bug.
Author should do the followings, if applicable

[x] Add tests
[x] Run tests
[x] yarn denoify to generate files for Deno

Hey @usualoma !
Could you see this?

@yusukebe LGTM!


test: add more store limit tests

What problem does this PR solve?
There is a lack of tests for store limit.
What is changed and how it works?
This PR adds more tests for store limit.
Check List
Tests

Unit test

/run-unit-test

Codecov Report

Merging #1673 into master will increase coverage by 0.23%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1673      +/-   ##
==========================================
+ Coverage    76.3%   76.53%   +0.23%
==========================================
  Files         157      157
  Lines       15347    15352       +5
==========================================
+ Hits        11710    11750      +40
+ Misses       2644     2589      -55
- Partials      993     1013      +20

Impacted Files
Coverage Δ

server/schedule/operator_controller.go
89.07% <100%> (+4.78%)
:arrow_up:

server/cluster.go
84.27% <100%> (+0.31%)
:arrow_up:

server/kv/etcd_kv.go
72.72% <0%> (-9.1%)
:arrow_down:

server/schedulers/shuffle_hot_region.go
60.52% <0%> (-6.58%)
:arrow_down:

server/tso/tso.go
74.52% <0%> (-4.72%)
:arrow_down:

server/leader.go
80.24% <0%> (-2.06%)
:arrow_down:

client/client.go
67.97% <0%> (-0.21%)
:arrow_down:

server/grpc_service.go
59.3% <0%> (+0.86%)
:arrow_up:

server/handler.go
49.86% <0%> (+2.14%)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e0d1bf...e2b5c26. Read the comment docs.

/merge


Throw error when country not referenced

It helps to debug which country code is invalid when data are imported into the lib.

Pull Request Test Coverage Report for Build 142

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 95.495%

Totals

Change from base Build 140:
0.0%

Covered Lines:
79

Relevant Lines:
81

💛  - Coveralls

LGTM. Thanks! @AlexisAnzieu


On embedded dashboards, show filter bar without "add filter" button OR control the fields that "add filter" allows.

Is your feature request related to a problem? Please describe.
I have a user that only has access to certain fields, but they are able to see the restricted fields when using the "add filter" button on embedded dashboards.
Of course, I can disable the filter bar, but then when the user interacts with the dashboard (clicking a keyword, time range, etc.), they are unable to see the filters that they have applied.
Describe the solution you'd like
There are three possible solutions that would resolve this issue for me (in order of preference):
1.) Allow option to show filter bar without the "add filter" button for embedded dashboards - this would be the best solution for me right now.
2.) Allow some way to control of the fields that the user can see when adding a filter in dashboard configuration - or specifically do not show fields that the user does not have read access to.
3.) Give option to disable filtering on dashboards.  This would block the user from adding filters when interacting with the visualizations.
Additional context
I am trying to set up embedded dashboards for read-only non-technical users that only have access to certain fields in certain indices.  I am using opensearch internal user management for permissions.  Regarding the second solution above, my expectation would be that the user shouldn't be able to see restricted fields when adding a filter, but they can and it seems that they will get 0 results without explanation (if anyone has any tips regarding the internal user management permissions that may remedy this, please let me know).

I traced through the call stack to try to understand how OpenSearch dashboards resolved from an index pattern to a list of fields to try to understand where it may be appropriate to apply a fix for this issue. indexPattern.fields is ultimately what gets displayed in the filter dropdown (see https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/public/ui/filter_bar/filter_editor/index.tsx#L275) so at some point in the resolution of fields the fields excluded by FLS need to be removed.
The trace starts in index_pattern.ts here: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/fba380b149e491d62b3e5eec708f53d8dd83fed1/src/plugins/data/common/index_patterns/fields/field_list.ts#L57
The fields come from the saved object index pattern or from a refresh here: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts#L394-L416
Refresh details can be found here: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts#L266
This calls getFieldsForWildcard: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts#L252
Which calls the _fields_for_wildcard API: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/server/index_patterns/routes.ts
Which under the hood uses an IndexPatternFetcher: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/server/index_patterns/fetcher/index_patterns_fetcher.ts#L66
And this calls the field capabilities API under the hood: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/rest/action/RestFieldCapabilitiesAction.java
When I create an index pattern as a user with FLS, it correctly does not include the fields that FLS excludes. When I create a dashboard from the index pattern and try to add a filter, I do not see the excluded fields in the filter dropdown. I believe that this issue arises when a dashboard is created from an index pattern that was created by a user with greater permissions than the user with FLS restrictions. Since the resolution from index pattern -> fields is done by getting the saved object via the saved object client, the fields are stored as part of the saved object.
Setting the isFieldRefreshRequired variable to always be true fixes the issue here: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/common/index_patterns/index_patterns/index_patterns.ts#L413

Reached out to some Dashboards core maintainers because this change will require modifying the code base over there. I would like to be able to minimally determine whether the Security Dashboards is installed in order to force the refresh variable to true. Ideally, we would only check for the active user having DLS/FLS active, but even determining whether the Security Dashboards is installed is challenging since there is not an obvious method of checking the installed plugins.

@ananzh this was the issue we spoke about just so it adds to your GitHub notifications. :)


[ktx] only remove dlls in static builds

Describe the pull request

What does your PR fix?
Fixes #23879

Which triplets are supported/not supported? Have you updated the CI baseline?
unchanged

Does your PR follow the maintainer guide?

Yes

Thanks!


Release as a universal wheel

By releasing as a Python wheel as well as a source distribution, you can speed up end user’s installs. After merging this command, to release you just need to run python setup.py clean sdist bdist_wheel upload.

It's been disabled due to #50

I don't see the 'build process' mentioned in #50 ?

I think @bbangert does this manually.
Btw, @bbangert why don't you configure deployment provider in TravisCI. It is possible to set up flow, so that package will be uploaded to PYPI once you push a new git tag.


Feat: polyfill CSS placeContent support on native

Summary
This PR adds the polyfill for placeContent
Test plan
EDIT:  We only support single values that are common to both properties (alignContent, justifyContent)

Thanks. We should only support single values that map the same to both long form properties, except for first/last baseline which is only for alignItems IIRC. This is generally how stylex and React Native work for other longforms

I've updated the PR. Now only single values are supported.

Thanks! I made a few changes and merged this into main


Add np.trim_zeros numpy function

Addresses: #70

np.trim_zeros typically takes a string argument : https://github.com/numpy/numpy/blob/v1.18.4/numpy/lib/function_base.py#L1573 , but jax doesn't support strings as arguments, so I'm assuming trim='fb'

Thanks for the contribution!
Jax can accept string arguments, so long as the arguments are marked as static whenever the function is jit-compiled (see, for example jax.numpy.einsum). I'd suggest implementing the trim argument if you're willing!

@jakevdp Thanks for the info! Would love another review. (Apologies for the delay, have been traveling)

There is an error in our new test that checks that the signatures of the numpy and jax.numpy versions of the function match. In your implementation, you call the first argument a, where numpy calls the first argument filt. To pass the test, you can use filt for the first argument.

Ah, didn't realize that the name of the parameter also mattered, thanks for that! Would love a final review.

We merged https://github.com/google/jax/pull/4027 which added this function.


Don't leave AttributeBuilder in an inconsistent state on exceptions

When one of the LazyValues raises an exception, don't leave its name in __pending stack of the AttributeBuilder, preventing evaluation of any other LazyValues.
The use case for this is: several declarations on a factory have to be in sync with each other, but can supply default values if needed. For example:
class UserFactory(factory.Factory):
    @factory.lazy_attribute
    def username(self):
        return self.email[:self.email.index('@')]

    @factory.lazy_attribute
    def email(self):
        try:
            return self.username +<EMAIL_ADDRESS>        except factory.containers.CyclicDefinitionError:
            return<EMAIL_ADDRESS>
This allows any of the following:
UserFactory(username='jane')
<EMAIL_ADDRESS>UserFactory()

Hmm, the idea is quite interesting!
However, I'd like to find a cleaner way to handle it than to rely on cyclic definition; I'll try to think of a cleaner API (maybe with some helper).
I'll keep this PR open for now, I hope to get some more time to think about this next week :)

Regardless of CyclicDefinitionError, any exception raised by a declaration will make AttributeBuilder unusable. This won't normally present problems as AttributeBuilder is recreated each time the factory is built. However, leaving it in an inconsistent state is not something that should happen at all, in my opinion.

Is there any decision on this?

Well, the change improves reliability, so let's go ;)
However, for the record, I do not advise relying on CyclicDefinitionError ;)


[BUG] High Security Vulnerability found (com.github.jnr:jnr-posix)

Agent Environment
Latest Datadog Agent container image version 7.43.1
Describe what happened:
Scanned docker image with Snyk and found a high vulnerability:
https://security.snyk.io/vuln/SNYK-JAVA-COMGITHUBJNR-1570422
Describe what you expected:
Snyk scan would not report any issues
Steps to reproduce the issue:
Scan Datadog agent container
Additional environment details (Operating System, Cloud provider, etc):

Thank you for having opened this issue. This issue tracker is primarily used to track bugs in the Agent codebase to completion. For issues directly related to your use of the agent, we have a dedicated team who can investigate your reports directly. Please contact Datadog support and send them a flare demonstrating the issue.
Thanks!

This is still not fixed in DD 7.45.x :(.

This is still not fixed in DD 7.52.x :(.

Hello @rdsedmundo  thanks for reaching out. This issue has been reported multiple times before and we cannot update the library because of our support of java 1.7. You will find more explanation about this and why we think this CVE does not impact us in https://github.com/DataDog/java-dogstatsd-client/pull/155#issuecomment-918278319


Weird error in HTTP/Chunked sources

Hello,
I'm working on some code to read xrootd files via HTTP(s) and have hit an error I don't get. With the following test script
#!/usr/bin/env python

import uproot
f = uproot.open("https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root")

with the following patch applied (to relax the SSL settings)
diff --git a/uproot/source/http.py b/uproot/source/http.py
index 4a1564a..bcc71a1 100644
--- a/uproot/source/http.py
+++ b/uproot/source/http.py
@@ -30,6 +30,7 @@

 import os.path
 import re
+import ssl
 try:
     from urllib.request import urlopen, Request
 except ImportError:
@@ -46,6 +47,13 @@ class HTTPSource(uproot.source.chunked.ChunkedSource):
     def __init__(self, path, *args, **kwds):
         super(HTTPSource, self).__init__(path, *args, **kwds)
         self._size = None
+        if path.startswith("http://"):
+            self._sslcontext = None
+        else:
+            # Temporarily disable server cert verfication while I work out how
+            # to always find the grid CA bundle
+            self._sslcontext = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
+            self._sslcontext.verify_mode = ssl.CERT_NONE

     defaults = {"chunkbytes": 16*1024, "limitbytes": 16*1024**2}

@@ -59,7 +67,7 @@ class HTTPSource(uproot.source.chunked.ChunkedSource):

     def _read(self, chunkindex):
         request = Request(self.path, headers={"Range": "bytes={0}-{1}".format(chunkindex * self._chunkbytes, (chun
-        handle = urlopen(request)
+        handle = urlopen(request, context=self._sslcontext)
         data = handle.read()
         if self._size is None:
             m = self._contentrange.match(handle.headers.get("content-range", ""))

I get the error
Traceback (most recent call last):
  File "./test.py", line 4, in <module>
    f = uproot.open("https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root")
  File "/home/meloam/projects/uproot/uproot/rootio.py", line 80, in open
    return http(path, httpsource)
  File "/home/meloam/projects/uproot/uproot/rootio.py", line 101, in http
    return ROOTDirectory.read(openfcn(path), **options)
  File "/home/meloam/projects/uproot/uproot/rootio.py", line 164, in read
    streamerkey = TKey.read(source, Cursor(fSeekInfo), streamercontext, None)
  File "/home/meloam/projects/uproot/uproot/rootio.py", line 842, in read
    out = cls._readinto(out, source, cursor, context, parent)
  File "/home/meloam/projects/uproot/uproot/rootio.py", line 864, in _readinto
    self._fNbytes, self._fVersion, self._fObjlen, self._fDatime, self._fKeylen, self._fCycle, self._fSeekKey, self._fSeekPdir = cursor.fields(source, self._format_small)
  File "/home/meloam/projects/uproot/uproot/source/cursor.py", line 70, in fields
    return format.unpack(source.data(start, stop))
  File "/home/meloam/projects/uproot/uproot/source/chunked.py", line 102, in data
    raise IndexError("indexes {0}:{1} are beyond the end of data source {2}".format(gstart + len(chunk), stop, repr(self.path)))
IndexError: indexes 29630001:29630027 are beyond the end of data source 'https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root'

Though the file contains 29635611 bytes. I can verify that's what seen by HTTP as well:
$ curl -L -k -vvv https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root -o out.txt
$ ls -l out.txt
-rw-r--r--. 1 meloam cms 29635611 Dec 11 21:08 out.txt

I'm not exactly sure of what's going on -- the same file opens fine from the local filesystem -- is it possible there's a silly bug in the chunked source I'm missing? I'm gonna leave this as a placeholder while I try to solve it

Ahh, I see now -- This is a bug in xrootd - it returns the following for context range context-range: bytes 0-16384/16385, which makes uproot think that the whole file is 16k bytes. Is there a reason that you didn't parse content-length, which should also provide the length of the file?

It is utterly possible that I parsed the wrong header, but I'm surprised that you're the first to notice if it's taking the file size to be 16 kB. I'm sure we've opened much bigger files.
I can't remember what we'd lose by not knowing the total size at all.

Actually, content-length is supposed to be the length of the returned range, so it doesn't help. Xrootd has the wrong behavior here, the number after the slash in content-range is supposed to be the total length of the document:
https://tools.ietf.org/html/rfc7233#page-12

I have a workaround, but it's somewhat annoying to use with urllib(2) .. how much do you oppose adding a dep to the requests library? In particular, it handles HTTP keepalive, which is a pretty huge win for making multiple requests to the same TLS-encapsulated connection

but... shouldn't it be:
import uproot
f = uproot.open("root://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root")

i.e. explicitly use the xrootd protocol?

but... shouldn't it be:
import uproot
f = uproot.open("root://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root")

i.e. explicitly use the xrootd protocol?

Xrootd (with the correct server-side configuration) supports HTTP(s) access, obviating the need to link the xrootd C++ library, and letting regular HTTP(s) clients transfer files to/from xrootd endpoints
https://twiki.cern.ch/twiki/bin/view/Main/XRootDoverHTTP
You can see where I explicitly curl the file in my previous post:
$ curl -L -k -vvv https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root -o out.txt
$ ls -l out.txt
-rw-r--r--. 1 meloam cms 29635611 Dec 11 21:08 out.txt

ah!
$> go get -u go-hep.org/x/hep/xrootd/cmd/xrd-ls
$> xrd-ls https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root
xrd-ls: could not list "https://xrootd.accre.vanderbilt.edu:1094/store/data/Run2017E/SingleMuon/NANOAOD/31Mar2018-v1/710000/5400FB33-0A47-E811-AE48-0023AEEEB703.root" content: could not create client: xrootd: could not authorize:
[xrootd: could not authorize using gsi: provider was not found]

(well, the GSI thing is xrootd/xrootd#757, but other than that, it's indeed working...)

well, the bigger point is that you don't have to get any special XRootD clients at all - the server supports bog-standard HTTP, so you can access it like any other HTTP host. There is one small extension that's used to enable 3rd party transfers between two different XRootD endpoints, but in particular for uproot, it supports what you want -- reading byte ranges out of files, and the ability to upload files back

I would not be opposed to adding improvements in this area that I know little about if you provide it as a pull request. I also wouldn't mind adding the requests library as a strict dependency if it's pure Python or easy to install with pip. If it's the sort of thing that's only easy to install with something like conda, then I'd prefer to make it an optional dependency like pyxrootd and backports.lzma are now.
If it's possible to drop pyxrootd entirely, that would be a win. This package has no easy install method (I resort to CMake) and has been a considerable source of bugs. Also, I'm not using it any differently than HTTP: both the XRootD and HTTP backends use the same chunk-loading routines. (I don't use XRootD as a replacement for fread, as that would be too many small requests.) So if the solution is to translate "root://" URLs into their "http(s)://" equivalents, that would be great on many levels!
I'm planning a new semi-major release soon, and that would be a good time to change the dependencies.

@PerilousApricot I'd like to release 3.3.0 today to require upstream changes from awkward and uproot-methods. Is there something simple that I can add now that will improve this situation? Should I try to determine if I can ignore the output from "content-length"? I think it's only used to make that error message. Without it, users would just get more confusing error messages in the case of an actual error, which is better than getting wrong error messages when things aren't in error.

TBH, trying to eyeball it, I'm not quite following how the size is being
passed between source <-> chunked <-> HTTP and if that would fix it.

In either case, even fixing the length mismatch doesn't make us able to
read from xrootd hosts -- because we use non-standard CAs, urllib will
cowardly fail the TLS handshake.
I realistically won't have time to polish up this until Monday, so I'd say
to just leave it as-is and I'll try to get in on the next release.
Cheers
Andrew

as an aside, don't you want to call super here?
https://github.com/scikit-hep/uproot/blob/master/uproot/source/chunked.py#L40

I put in exactly one change that will propagate to 3.3.0: uproot no longer trusts the "content-range" length if it is not longer than the fragment. That should solve your original issue, though we've talked about a lot more than that here. I've checked and this size is not very important at all: if the file size is known and there's an attempt to read beyond the file size, then uproot raises an exception. If the file size is not known, there's no immediate exception, but reading is very likely to fail soon thereafter.

That's helpful, but to be clear, urllib wont work with any existing CMS
xrootd server because the SSL certs are signed by custom CAs. (Eg don't
announce it as a new feature!)
It's dark in this basement.

I won't announce it. I also haven't applied your patch, which I guess is about certificates. I'm willing to consider depending on the requests library, but not for this release.
About the call to super, I haven't been following a style that requires it in each __init__. I can see that being a good discipline to impose on class inheritance, but actually, I've never heard of it as a rule. (I call super when I explicitly need it.)

No, I don't think that patch was ready, it was just to show a reproducer.
Not passing super up the class hierarchy is what confused me trying to
eyeball the code, but you do you :)
It's dark in this basement.

There are still open issues in this discussion, but I'm confused about where they are. Therefore, I'll close this (because the original issue was fixed) and you can reopen when you're ready to propose something concrete. Thanks!


Configurable auto scroll

Hi
Thanks for the library. Is it possible to turn off the auto scroll feature when the drop down is opened? I dont want the list to auto scroll to the selected Item.
Thanks

I think you can achieve that by manually defining the selected item and also manually updating the hint to show the selected item. In that case, you need to always pass null to value parameter "We don't specify any selected item". Refer to example (4. DropdownButton2 as Multiselect Dropdown with Checkboxes) it has the same idea of manually updating the selected item but you'll use value of null and use hint instead of selectedItemBuilder. Hope that helps.


It's possible to add timezone?

I want to include timezone, it's possible to?

The current Cron component output a simple string as value and is not related to any timezone and will work for all cases.
Working with a timezone means you have to set up an env variable for that on Linux, for example.
See: https://linuxhint.com/set-timezone-crontab/
I prefer to keep this component simple and only output a string since the selection of the timezone is on your side, a simple Select with the different timezones you want to support. No need to make this component more complex for that.
I hope you will succeed to do the timezone on your end 🙂


.Net: CJK support for text splitter

Motivation and Context

The text splitting algorithm has hardcoded values for punctuation based on ASCII punctuation (,.; etc.) Chinese and Japanese use both full-width Unicode punctuation values https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block) or ideographic punctuation.
If you give the text splitter Japanese text it will incorrectly split in the middle of sentences
The current algorithm defaults to an approximation token counter, which is partly accurate for Indo-European/Latin Alphabet languages but totally out for CJK. This tests uses the BPE token length for cl100k_base (GPT3/4)

Description
This change adds some basic full-width characters and ideographic punctuation to the builtin list.
It also adds a test which checks that it splits a sentence correctly. The first part of the test string (田中の猫はかわいいですね.) is 16 tokens for cl100k_base.
Contribution Checklist

[ ] The code builds clean without any errors or warnings
[ ] The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
[ ] All unit tests pass, and I have added new tests where possible
[ ] I didn't break anyone :smile:

The build is broken because this uses the new Microsoft.ML.Tokenizers library which must have a breaking change because one of the sample files (unrelated to this PR) stops compiling. I'll take a look but it would have broken on the next release anyway

I made some changes to this PR to add the comma and fullstop variations to the string inside the list rather than individual items. This reduces the number of calls to TokenCounter. Implemented snapshot testing for some short stories in different languages to verify that those changes didn't impact how text was split and to ensure future regressions are captured.

@tonybaloney could you take a look at the conflicts, thanks!

Sure, also the ML library API is going to change and there will be a sync version so the sample code in this PR will need updating.
https://github.com/dotnet/machinelearning/issues/7077


Bottle shouldn't use X_FORWARDED_HOST without checking

https://github.com/bottlepy/bottle/blob/332215b2b1b3de5a321ba9f3497777fc93662893/bottle.py#L1446
Other than being a security issue (since that header can be set by anyone), it doesn't work when you have two servers forwarding. Just like X-Forwarded-For, if there are multiple successive proxies, X-Forwarded-Host would be a ,  separated list of hostnames. This will break things like redirect('/path'), which would be turned into something like this:

And then:

An easy fix would be to just add .split(', ')[0], if you want me to create a pull request I can do so.

X-Forwarded-Host is usually single-valued. There is no standard, but the intention is to preserve the original Host header value, as sent by the client, before forwarding the request to a back-end server. Multiple values make no sense here. Intermediate proxies that override or append to this header are broken, in my view. The same is true for any of the X-Forwarded-* headers. They should represent the clients intent, not some internal path tracing through your infrastructure.
And sure, you can forge this header. You can also forge a Host header or any other header. It is the job of the front-end proxy/lb/gateway to ensure that only requests with a valid Host header are accepted, and that the X-Forwarded-* headers are all set to the correct values. If you do not do that, but still trust these headers in your back-end (directly or indirectly), then sure, you might have a problem there. This is true for any form of user input.

I'd gladly accept pull requests for Forwarded support, though.

OK then, thanks for the explanation. For the record, that's the default behavior of Apache.

Yes, unfortunately, but they have a warning in their docs. Note that Apache also explicitly preserves and appends to user-provided values for that header, even on the first hop. Silently accepting the first value in bottle would actually increase the security risk. A broken and invalid hostname is better than a valid but malicious one, after all.
You see why the Apache default behavior is broken? There is no way to find the actual trustworthy original value of the Hostheader behind an Apache proxy, because the first value ofX-Forwarded-Host` can be forged. The header is rendered useless.


build.prod.rollup.aot throws an error (on a fresh git clone)

I'm submitting a ...  (check one with "x")
[x ] bug report => search github for a similar issue or PR before submitting

I cannot get build.prod.rollup.aot to work.
git clone https://github.com/mgechev/angular-seed
cd angular-seed
npm install
npm run build.prod.rollup.aot

It throws the following error
{ Error: Unexpected token
    at error (/home/alarive/angular-seed/node_modules/rollup/src/utils/error.js:2:14)
    at Module.error$1 [as error] (/home/alarive/angular-seed/node_modules/rollup/src/Module.js:303:2)
    at tryParse (/home/alarive/angular-seed/node_modules/rollup/src/Module.js:27:10)
    at new Module (/home/alarive/angular-seed/node_modules/rollup/src/Module.js:53:15)
    at /home/alarive/angular-seed/node_modules/rollup/src/Bundle.js:301:20
  code: 'PARSE_ERROR',
  pos: 92,
  loc:
   { file: '/home/alarive/angular-seed/node_modules/rxjs/util/isArrayLike.js',
     line: 2,
     column: 78 },
  frame: '1: "use strict";\n2: var isArrayLike_1 = function (x) { return x && typeof x.length === \'number\'; });\n

npm version : 3.10.10
node version : v6.10.0

Same here. Mac OS 10.11.6, npm 3.10.9, node 6.9.2.

Reverting RxJS to version 5.1.1 or earlier removed the error and the build succeeded.

Reverted back temporary.

I can confirm it is working well with this version of RxJS.

It's an issue in the commonjs plugin for rollup https://github.com/rollup/rollup-plugin-commonjs/issues/168.