rake-db addTo

The new syntax for addTo on rake-db doesn't seem to be working properly. I have not yet tried other commands, but I tried every variation of addTo[Table], addTo [Table], etc., and it fills inchangeTable(tableName). I mean literally the string tableName, not the name of the table ie changeTable('user'), so it seems like it's not getting the arg correctly from the command line. If I'm doing it wrong... then the wording is very confusing and needs to be made more clear. Either way I would use [] as args in the text on the command line and not *, which seems to be much more normal cli convention to me.

Ok I just tested create[Table] and that seems to work fine.
Another question I have related to this is, if I have varchar in the cli (like for my id) how can I set the default amount, or is there no way to do that on the cli and I just have to add it manually? It did not take varchar(36).

For reference, docs are here: generate migration,
One of the names it should parse is add${any string}To${table name}, I should add examples to this list, I mean here addColumnsToMyTable - it should generate a migration with
db.changeTable('myTable', (t) => ({
  //  ...add columns
}))

Either way I would use [] as args in the text on the command line and not *

Could you explain because I'm not following which [] and which * do you mean?

if I have varchar in the cli (like for my id)

The doc says to take varchar(36) into quotes, are you providing it in the quotes and it doesn't work?
btw, postgres docs suggest against varchar(n)

If you look at the text the cli gives you, it is using stars and you should use brackets to indicate an arg.
add[Column]To[Table] like that.

New migration file arguments:
- (required) first argument is migration name
  * create*               template for create table
  * change*               template for change table
  * add*To*               template for add columns
  * remove*From*          template for remove columns
  * drop*                 template for drop table

I see, okay sure, let it be add[any string]To[table name], I'll update and re-check that it works as expected.

I'd probably just call it column instead of any string as that's what you're actually doing, right?

It's any string because, for example, addFewColumnsToMyTable: this will generate a file with the following:
change(async () => {
  await db.changeTable('myTable', (t) => ({
    // nothing here
  }))
})

Because if you want to add a few columns, it would be a too-long migration name. So I thought that it be better to leave migration names without specific column names.

...Wouldn't you just use change[Table] then?

It is already there:

change${table name} for changing a table

The name of the migration could be any string at all, it's not required to follow the template. This template is a minor bonus feature so the cli tries to guess by name what's expected and tries to help a little bit with the content. But helping only a little bit, you have to edit and write migration by hand anyway.
In the distant future migrations could be generated from model files, so this cli will be less important.

...What? I was saying if you are using add[]To[] you're probably just adding one column, otherwise just use change[Table]. Yet you mentioned add[FewColumns]To[Table]. I was saying when would you ever add multiple columns via that command? Which is why I was suggesting it just be add[Column]To[Table] because there is no reason you'd ever want to use it for more than one column.

For instance, "addDetailsToProduct", "addSupplierInfoToProduct", "addLegalInfoToUser", etc.
"addSupplierInfoToProduct" is a more descriptive name than just "changeProduct", so this "add something to table" form may be preferred.

That's fair I guess.

I improved docs to have add[something]To[table] and to have examples.
Also removed the feature of command line arguments being parsed into columns, because it's not really useful but it added confusion.


[Feature Request] launch same request several times with many args

Details
This is a feature request.
Sometimes you want to use a list of values to trigger the same request multiple times. This is a tedious task now because you need to change the value manually and trigger the same request again.
Ideally, I want to provide a list, maybe extracted from a previous request and launch one request several times, one per each value on that list.
Regard

I would love some human interaction. Even just for telling me that this is not going to be implemented

Sorry @danielo515! Some issue slip through the cracks. You may want to take a look at this: https://github.com/getinsomnia/insomnia/issues/774#issuecomment-489363559

Thanks for the answer @gschier
Does that mean that I have to write a plugin per each set of requests I want to send? Or that I should install a plugin that allows me to do that?
Thanks!

The latter – Install/create a plugin that allows you to do that

I'm astounded this feature does exist yet in Insomnia. Might start making the long switch back to PostMan.


Migrate games and !wip

Refer to #170 and #169

An additional note so I don't forget: /game minecraft-skin did not work for me when I put in a user. Not sure why, will circle back around since it might've been an issue with my dev DB.

An additional note so I don't forget: /game minecraft-skin did not work for me when I put in a user. Not sure why, will circle back around since it might've been an issue with my dev DB.

Fixed that in the latest commit

Channel-based functionality of /game playing also tested and good. Only /game Destiny left to test


Change share buttons

Use this and this as inspiration.

Other resources: https://colorlib.com/wp/free-html5-social-sharing-buttons-templates/


Add the coverage directory to the gitignore

Probably don't need to include these test artifacts

👍

Thanks!


Palliative workaround for #900

This uses https://docs.rs/fix-hidden-lifetime-bug to replace the compiler's heuristic with lifetimes with a simpler and more straight-forward one, which does dodge the regression from https://github.com/rust-lang/rust/issues/98890.
How to use
Add the following to your workspace[^1]'s Cargo.toml file:
[patch.crates-io.async-graphql]
git = "https://github.com/danielhenrymantilla/async-graphql.git"
branch = "workaround-for-1-62-bug"
# or, instead of branch, you may pin-point its `HEAD` commit, if you are scared of a sneaky branch update
# rev = "6dae5c864c009de9dda03fc0b7796ced02e895bd"

[^1]: the directory where the Cargo.lock file is generated.

Thanks a lot, it's strange that I didn't have this problem. 😂

@danielhenrymantilla looks like you may need a cargo fmt --all. I ran into this issue as well and ended up here so I hope you get your PR merged soon!

I confirmed that with this PR applied to async-graphql, I cannot reproduce the issue in my comment https://github.com/async-graphql/async-graphql/issues/900#issuecomment-1175787840

this is not worked for my team after adding this patch., so, we reverted to rust 1.61.

@leefordjudes interesting, could you share what errors you got with this patch?

No longer needed as of the new toolchain release


Getting started with Node.js Paypal checkout integration

Article checklist
NOTE: (Please ensure that you have only one open issue + linked pull request at a time. This will ensure that we complete the article in a timely manner from inception to publishing.)
Preliminary Checks - Formatting and Structure

[x] Does your article follow any of the suggested structure formats? - see example formats
[x] Is your article properly formatted in Markdown? - see Github Markdown guide
[x] Have you used the correct folder and file structure? - see Contribution Guidelines
[x] Is your article over 750 words? - Tool: Wordcounter.net
[x] Have you used our preferred heading structure? - H3 (###) and up
[x] Does your article provide enough value and detail about your topic? - Articles should be clear, accurate, and fully explained.
[x] Can your article be understood by beginners? - Assume the audience is smart but has no prior exposure to the common terminology in your article.
[x] Have you included a hero image that is 600x400 pixels and have the copyright to use it? - Tools: Pixlr Image Editor for resizing and Unsplash for Creative Commons images
[x] In your PR - add a comment that states "This fixes #(enter your Idea Content Form #)" to link and close your Idea Content (issue) when the PR is merged.

Grammar & plagiarism checks

[x] Have you spellchecked and grammar checked your article? - Tools: Write&Improve - ProWritingAid -  Grammarly
[x] Please place your article through a 3rd party plagiarism checker? We suggest using Quetext, this tool is free and has a daily limit. We typically accept articles with 10% or less.
[x] Have you checked your article for readability? - Tool: Hemmingway
[x] Have you added sources for quotes and images that aren't yours?

Technical checks

[x] Are your code snippets properly formatted for syntax highlighting - see Syntax guide

Contribution guidelines
For first-time contributors and for more details, see Contributing Guidelines
This fixes #1962

@paulodhiambo I have made the requested changes.

@okeloviolet amazing article. @hectorkambow you can give it a final review.

plagiarism check done

👍  nicely done @okeloviolet
I made some minor edits for flow - please review 👍
cc @paulodhiambo


Update the button component

Fixes https://github.com/MLH-Fellowship/react-native-website/issues/6
Summary

Update the title and prop description from docs.
Add default value to props.

An attempt to solve the problem of specifying the default value of a prop in a component.
I added the default values to the props while destructuring their actual values.
This would not only help the docs to get the default prop value(when automated) but also reduce hardcoded styles/values in the code.

Update platforms in props.

Use comma separated string with platform decorator.

Remove unused image from comments.

It seems outdated and no longer in use, so I removed it from the comments.

@ayushjainrksh I started a review but have to jump off and continue later. Posted few comments, more to follow

Okay, no problem.


Option to only include filename instead of filepath in $print emit

Current emit:
[The/Industrial/Revolution/and/its/consequences/have/been/a/disaster/for/the/human/race.tsx:21]: Hello world
Proposed emit:
[race.tsx:21]: Hello world
The point is to make resulting print statements shorter. I find myself labelling .lua files with the latter method sometimes, and pretty much all the time in C++/Rust.

This is the wrong library, you may be thinking of https://github.com/roblox-aurora/rbxts-transform-debug!


Add more info to produce

Very rough around the edges, but better than nothing for now. Feel free to edit of course.

Thanks @SystemFw! I did some minor cleanup.


[INLONG-10159][Sort] Kafka connector support audit ID

Prepare a Pull Request

[INLONG-10144][Sort] Redis connectors support audit ID

Fixes https://github.com/apache/inlong/issues/10159

parent issue: https://github.com/apache/inlong/issues/8803

Motivation
Kafka connector not support audit function. So this pr will make it support audit function.
Modifications
Modify Kafka connector source.when source/sink receive data, it will send audit information at same time.

Add licenses please


chore(main): release 0.1.1

:robot: I have created a release beep boop
0.1.1 (2023-04-26)
Bug Fixes

Also tag the latest docker-image (17e0deb)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/philips-software/SPDXMerge/releases/tag/v0.1.1 :sunflower:


Add new hybrid responsive table pattern to TCCP app

Adds a new hybrid responsive table pattern that stacks some columns on smaller screens, specified with a new preserve_column_on_mobile property in our base table template. I also added a right_align property to let us right align table cells.
See https://github.local/Design-Development/Design-Thinking-and-User-Research/issues/232
Additions

New hybrid responsive table pattern.
Optional preserve_column_on_mobile and right_align table column properties.
New stylesheet and front-end "app" for TCCP.

How to test this PR

./frontend.sh
Visit http://localhost:8000/consumer-tools/credit-cards/find-cards/cards/
Shrink your viewport.
The table columns should stack with exception of card name and apr.

Screenshots

desktop view
partially-stacked mobile view

Notes and todos

The hybrid table pattern will evolve and eventually get added to the DS. We'll then remove the CSS from this repo and import the new DS code.

Browser testing
Visually tested in the following supported browsers:

[x] Firefox
[x] Chrome
[x] Safari
[x] Edge 18 (the last Edge prior to it switching to Chromium)
[x] Internet Explorer 11 and 8 (via emulation in 11's dev tools)
[x] Safari on iOS
[x] Chrome on Android

Accessibility

[x] Keyboard friendly (navigable with tab, space, enter, arrow keys, etc.)
[x] Screen reader friendly
[x] Does not introduce new errors or warnings in WAVE

Other

[x] Is useable without CSS
[x] Is useable without JS
[x] Does not introduce new lint warnings
[x] Flexible from small to large screens

Thanks @chosak! I added a couple test cases.


TorchElastic standalone test silently fails

Bug
The test case
https://github.com/Lightning-AI/lightning/blob/a008801e25a6e93214b410b68ca1edf2e3ddae14/tests/tests_pytorch/run_standalone_tasks.sh#L27-L32
silently fails in our CI.
Output
Running plugins/environments/torch_elastic_deadlock.py
WARNING:__main__:
*****************************************
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
*****************************************
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Initializing distributed: GLOBAL_RANK: 0, MEMBER: 1/2
Initializing distributed: GLOBAL_RANK: 1, MEMBER: 2/2
----------------------------------------------------------------------------------------------------
distributed_backend=nccl
All distributed processes registered. Starting with 2 processes
----------------------------------------------------------------------------------------------------

LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0,1]
LOCAL_RANK: 1 - CUDA_VISIBLE_DEVICES: [0,1]

  | Name  | Type   | Params
---------------------------------
0 | layer | Linear | 66
---------------------------------
66        Trainable params
0         Non-trainable params
66        Total params
0.000     Total estimated model params size (MB)
/__w/2/s/src/pytorch_lightning/trainer/connectors/data_connector.py:224: PossibleUserWarning: The dataloader, train_dataloader, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 64 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  rank_zero_warn(
/__w/2/s/src/pytorch_lightning/trainer/trainer.py:1555: PossibleUserWarning: The number of training batches (5) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch.
  rank_zero_warn(
/__w/2/s/src/pytorch_lightning/trainer/connectors/data_connector.py:224: PossibleUserWarning: The dataloader, val_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the `num_workers` argument` (try 64 which is the number of cpus on this machine) in the `DataLoader` init to improve performance.
  rank_zero_warn(
[W reducer.cpp:1251] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[W reducer.cpp:1251] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())

ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 1 (pid: 54410) of binary: /usr/bin/python
Traceback (most recent call last):
  File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/run.py", line 765, in <module>
    main()
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 345, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/run.py", line 761, in main
    run(args)
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/run.py", line 752, in run
    elastic_launch(
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/usr/local/lib/python3.9/dist-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

Observed on latest commit on master.
Expected Behavior
The test should pass. Additionally, should the test fail like it does now, the error should be reported in the CI. We currently grep for SUCCEEDED but it doesn't make the CI fail.
cc @tchaton @rohitgr7 @borda @awaelchli @carmocca

I think this shall be a type script or set -e in the CI workflow
https://github.com/Lightning-AI/lightning/blob/a557952fabafa188578c31596c81c64b3f5e65f6/.azure/gpu-tests-pytorch.yml#L146

@Borda The script sets it already. It's an issue with the test not propagating the failure correctly as only one rank fails. https://github.com/Lightning-AI/lightning/blob/04e1e925daaf18acd3ec1b816575308f76f5f813/tests/tests_pytorch/run_standalone_tasks.sh#L15

Yes, I saw it in the script, but not sure what we can do then, as this si quite critical for visibility...

There's nothing we can do other than to fix the test and grep for "error" in its text output as I did to resolve https://github.com/Lightning-AI/lightning/issues/12474

Closing this since I don't see anything to address via this issue. The improvement in #15341 by Carlos is reasonable, IMO.

Note that even though we don't have a reliable solution to surface these distributed issues, we could still fix this test in particular so that it passes again.


Proposal of a convention about npm modules distribution.

There was an discussion about generating ambient declarations for external CommonJS modules: http://typescript.codeplex.com/workitem/2080
But even if typescript generates the definitions as we what we want, we'll still need to add references manually, which is not doing better job than JS libs with NuGet DefinitelyTyped packages.
I propose that when we publish a typescript package via npm, publish the .d.ts file(s) generated with the current style along with it.
So that we can manage to write a tool (or add a feature to something like WebEssentials) to watch related node_modules directory, resolve the definition references and create wrapped definition file(s) under scripts/typings folder.
Thoughts?

We're interested in a cleaner solution than ts-pkg-installer, which we wrote to address TypeScript modularity in the NPM/TSD ecosystem.
https://github.com/RedSeal-co/ts-pkg-installer
We also use dts-bundle, which complements ts-pkg-installer in that it generates a declaration file that can  be installed with ts-pkg-installer.

@vladima  has  proposal in #2338 for managing typings as part of resolving node_modules. the idea is that the package.json would have a "types" as a property, and the typescript compiler will look up that file and use it if it exists. would that work?

@mhfrantz I am also designing a package manager for TypeScript. Hoping it could be compatible with NPM and other package managers like bower. And I would like it to support some simple pre-compile features for supporting libraries with or without module system. https://github.com/vilic/tying

@mhfrantz I am also designing a package manager for TypeScript. Hoping it could be compatible with NPM and other package managers like bower. And I would like it to support some simple pre-compile features for supporting libraries with or without module system. https://github.com/vilic/tying

@mhegazy, having reference paths work with node_modules/package.json would be ideal.  The most brittle part of the reference path feature is the necessity of specifying the exact location of files.  Resolving reference paths within node_modules makes the most sense from the NodeJS standpoint.
It is important to incorporate DefinitelyTyped declarations as well, as all type information is not embedded within the NodeJS module itself.
It is also important to de-dupe the declarations.

Thanks @mhfrantz  for the feedback, a few comments:

The most brittle part of the reference path feature is the necessity of specifying the exact location of files. Resolving reference paths within node_modules makes the most sense from the NodeJS standpoint.

The proposal in  #2338 looks in node_modules and reads package.json

It is important to incorporate DefinitelyTyped declarations as well, as all type information is not embedded within the NodeJS module itself.

Any proposals on how to do that, and how would that be diffrent from something like what tsd is doing today.

It is also important to de-dupe the declarations.

What are the rules? any proposals on how this should be done?

@mhegazy If you look at what ts-pkg-installer is trying to do, it builds a TSD config file so that if multiple modules depend on the same declaration, they will all resolve to the same file.  This is awkward, since it involves building config files, and then running the TSD utility on the result.
If it were incorporated into TSC, then perhaps reference paths would contain simple names, like a module name.  It could look in node_modules first, and then in typings.
For example, suppose you see this:
///<reference path="mocha"/>

This is a reference to the module "mocha", which (today) has no TypeScript declaration of its own.  Thus, it would resolve to typings/mocha/mocha.d.ts.

so what is the proposal, add <root>/typings as an implicit include path? though I am not a fan of include paths in general.

@mhegazy It is not as simple as adding <root>/typings as an include path, since the logic for determining the exact *.d.ts file name varies.  Given a reference to "@scope/foo":

If we find a module in node_modules called "@scope/foo", and its package.json contains a "types" section, then use that *.d.ts file.
If we find a module in typings called "@scope/foo", then use foo.d.ts in that directory.

Thus the proposal is to support both NPM packages written in TypeScript with embedded type declarations as well as NPM packages whose type declarations come via TSD.  This can be accomplished with a single, coherent reference path syntax that is agnostic of the distinction between these two cases.

@mhfrantz Just re-read this thread, and noticed that I seem to have run into a similar issue as what you describe (de-dupeing declarations, and supporting TSD case).
I created #2839 with a proposal for a solution, would it fit to your use-case too?


Bug fix/update foxx services after restore (#6774)

Update foxx services after restore operations.
Ported from devel.
Jenkins:
http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/2219/

The only red test is unrelated and under investigation.


Development: Fix flaky modeling assessment component tests

Warning
Before merging:

This PR needs to be merged in Apollon: https://github.com/ls1intum/Apollon/pull/296
The Apollon package needs to be updated to @ls1intum/Apollon 2.<newversion> and all references to @palsch/apollon need to be removed.

Checklist
General

[x] I tested all changes and their related features with all corresponding user types on a test server.
[X] This is a small issue that I tested locally and was confirmed by another developer on a test server.
[X] I chose a title conforming to the naming conventions for pull requests.

Client

[X] I followed the coding and design guidelines.
[X] I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
[X] I documented the TypeScript code using JSDoc style.

Motivation and Context

Some modeling assessment tests are currently flaky:

Bamboo: https://bamboo.ase.in.tum.de/build/result/viewBuildResults.action?buildKey=ARTEMIS-TESTS5145-TSTEST&buildNumber=4
Github: Failing more consistently: https://github.com/ls1intum/Artemis/actions/runs/5547057107/job/15030418353?pr=6892

Description

Used Angular change detection with whenStable to wait for the editor to be properly initialized. This still was not enough, as the React render cycle still does not line up with the one from Angular. Therefore, I introduced a new promise to wait for the next React render cycle in Apollon (PR https://github.com/ls1intum/Apollon/pull/296), and I'm waiting for that one now. This also allowed me to remove many test and production code delays.
For testing (locally, on a test server, running the client tests), I changed
Steps for Testing

Generally test modeling exercise assessment again and try to be creative:

Assess a normal modeling exercise or one in an exam
Make sure that the correct elements are highlighted in the assessment editor at all times
Make sure that feedbacks on all kinds of elements still work and can be saved, also on Safari if possible (see also https://github.com/ls1intum/Apollon/pull/295)

Code Review
Ignore all replacements @ls1intum/apollon -> @palsch/apollon. They are only for testing and will be removed.
Review Progress

Manual Tests

[x] Test 1
[x] Test 2

Code Review

[x] Code Review 1
[x] Code Review 2

Test Coverage

Unchanged.

I have to work on this some more because the bug fixed in #6893 appeared again :(

Currently blocked only by https://github.com/ls1intum/Apollon/pull/296


Public Trackers that Don't Work

Isohunt2 - Doesn't load or seem to "exist" anymore
The Proxy Bay - Don't know if this comes and goes, but it's not loading for me right now
MagnetDL - Seems to be a CDN Host down error 522, not sure if this is temporary or how long it's been going on for. But i know this is a reputable site
Pirateiro - Doesn't appear to be loading
TorrentKing - Redirects to "amomama.com" and is unrelated now
Torrent9 - Seems to redirect to maddatings.com or astrowow.com now
Torrenty.org -
WorldWide Torrents - Doesn't load or seem to "exist" anymore

Pirateiro and Torrenty.org are okay. Will wait a bit for MagnetDL. The rest of them are removed.


Add action link to the GitHub comment of action-plan-preview

What would you like to be added:
Why is this needed:

/assign @Hosshii

Thank you.
I'm not sure where to add it, should I add it in the omitted comments section?

@nghialv

@Hosshii How about adding it as a new badge of the comment?
Like this.
[![ACTIONS](https://img.shields.io/static/v1?label=PipeCD&message=Action_Log&style=flat)](link-to-the-action)

This is very good. I'll do this. Thank you!


[RFC] Backpressure in Search Path

Introduction
BackPressure in Search Path aims to enhance the overall resiliency of OpenSearch. The current protection mechanisms on OS nodes such as ThreadPoolQueueSize and CircuitBreakers are not fully efficient to protect the cluster against traffic surge, partial failures, slow node or a single rogue (resource-guzzling) query. Search BackPressure aims to introduce constructs to have fair rejections, minimal wastage of useful work done, search request cost estimation and ability to stabilise the cluster when under duress.
Problem Statement
In OpenSearch, for Search requests there are few gating mechanisms to prevent a node from meltdown when under a duress situation. These are essentially queue rejections and circuit breakers. However, these gating mechanism are all static in limits, take local decisions, and are often too late to act upon due to configuration issues.
Problem with existing constructs

ThreadPool queue size: OpenSearch users today can provide a node level settings to fix the number of tasks that can be queued up for search in the Search ThreadPool queue. With this we see 2 major problems:

Queue size doesn’t define actual work to be done by the tasks it contains. A query can have a very high cardinality and might consume a lot of system resources. While other times query could be related to smaller indices, touching very few shards and relatively less data. Query type also plays a huge role.
Tuning the queue size based on workload adds a major management overhead. It is also hard to keep it consistent with the change in traffic. Since every customer has different query patterns, coming up with defaults which fits for all is not possible.

Circuit breakers: Although we have circuit breakers which track real time memory resource utilisation of requests, they act at a blanket level. It provides mechanism to cut off requests if the defined thresholds are breached. However this lacks fairness, as decision is limited to local node performance, and they lack view on the hot shards or request types as well. Also, Circuit breakers lack prioritizations, as they are unable to expedite requests which are more likely to be completed soon. For example, requests in the fetch phase have more chances of finishing early compared to query phase. Arriving at the right thresholds is again a management overhead for users.

Other problems which OS users can run into

Often times, even a Single Rogue query can adversely impact data nodes and can create cascading affect of node failures. There is a need to proactively identify, block or terminate such queries, ensuring other requests could still continue, preventing any further duress on the data nodes.
During the distributed execution of search requests, a single slow path can impact the overall service quality and availability of the cluster. Coordinator fans out the request to multiple data nodes and if there’s a degraded node in the path, coordinator keeps on waiting and even attempts retries, while even accepting new requests in the meantime. This creates a build up on the coordinator, just because the rate at which Coordinator accepts request is higher compared to the rate at which it is able to complete the request. There is a need to identify such degraded paths, to proactively short circuit or fail fast requests and prevent cascading node failures. These issues sometimes doesn’t get detected with the ThreadPool queue size comparison in ARS just because the the cost of each task in the queue can significantly vary across different nodes and can end up overwhelming a node which is already under duress.

Proposed Solution
Summary
We propose to introduce BackPressure framework in the search path to address the above concerns and improve the overall resiliency of the OpenSearch cluster. For this we’ll introduce resource tracking framework for end to end memory/cpu tracking of Search request, covering different phases of execution across all shard requests. Adaptive replica selection will consider these resource metrics from different nodes, while ranking the nodes to reduce requests on target nodes already running into resource contention issues. Framework will also use the same resource utilisation metrics to take decisions whether to reject or cancel an in-flight request, with added fairness. Task level prioritizations based on current state of request can be achieved, such as preferring requests already in fetch phase over those yet to be picked for query phase. This will help stabilising the node under duress by completing requests faster which has some useful work already done. In addition to this, framework would have a Search request cost-estimation model to support a pro-active form of backpressure, which takes decision on incoming requests/tasks, if a node will be able to serve the query of certain cost or not. We have divided this proposal into 3 milestones as follows:
Milestone 1
Goals

Aims to achieve visibility into resource utilisation of Search requests (Both at Coordinator and Data node)
Fan-out the search request to nodes having higher chances of completing the request

1.1 Resource Tracking Framework
Build a resource tracking framework for Search requests at both Rest level (Client requests) and Transport level (Shard search requests), which tracks resource consumption on OpenSearch nodes, for various search operations at different level of granularity:

Individual Search Request (Rest) - On the Coordinator Node across phases (such as query and fetch phase) for end to end resource tracking from coordinator perspective.

Shard Search Requests (Transport) - On the Data Node per phase, for discrete search task tracking.

Shard level Aggregated View - Total consumption of resources mapped to every shard for searches on the node.

Node level Aggregated View - Total consumption of resources for all search request on the node.

Characteristics:

Resources to track for each of the above proposed metrics:

Memory Consumption (JVM)
CPU Consumption (CPU Time)

Frequent checkpointing and footprint update during the progress of a search phase : The resource tracking should continuously be done as the search request progresses, and need not necessarily wait for a particular phase to complete. This is important as a single phase execution can consume significant amount of resources itself. So, tracking resources within a phase itself as it progresses becomes important for these metrics to represent the accurate state on the node.
Data Node feedback to build Coordinator state : Have a capability for data node to piggyback on the response and send its current node utilisation state due to search requests to coordinator.

1.2 Adaptive replica Selection - Factor in node resource utilisation
Currently, for Adaptive replica selection the tasks in search thread-pool queue are factored in while coming up with the ranking for the target shards. Due to the fact that cost of each search request can vary significantly the queue count doesn’t accurately tells what more resources the node have to accept and complete more search requests. To be more accurate on this we can factor in the resource utilisation for search requests on these nodes, and take better routing decisions on the coordinator.

This will help avoiding the shards under stress giving them time to recover
Increase the chances of request landing on the node which can complete it
Queueing and Fast-Fail decision can be taken on the Coordinator node itself if required.

Milestone 2
Goals

Improved fairness in Search request rejections
Reducing chances of node getting overwhelmed due to Search request load
Ability to stabilise overloaded nodes by identifying and cancelling resource guzzling queries.
Achieving resiliency with reduced dependency on Circuit breaker and Threadpool queue configurations as the accuracy of rejections due to these depends on user input.

2.1 Server Side rejection of in-coming search requests
Currently, Search rejections are solely based on the number of tasks in queue for Search ThreadPool. That doesn’t provide fairness in rejections as multiple smaller queries can exhaust this limit but are not resource intensive and the node can take in much more requests and vice versa. Essentially count is not the reflection of actual work.
Hence based on metrics in point 1.1 above, we want to build a frame which can perform more informed rejections based on point in time resource utilisation. The new model will take the admission decision for search requests on the node. These admission decisions or rejection limits can have different levels to it:

Level 1: At this point system has detected overload due to search requests and it’ll prioritise which requests to accept. Example: It’ll accept fetch requests over Query requests as for Fetch phase we have already done some work to reach at this point whereas Query is going to be more resource intensive and have least wastage of work if rejected. Similar logic can be applied for Force search requests as well.
Level 2: At this point we’ll start rejecting all search requests beyond capacity to prevent any impact on the availability of node.

This can be further evolved to support Shard level priority model, where user can set priority on an index or every request, so that framework can consume them for taking admission/rejection decisions.
If user has configured partial results to be true, then upon these rejections and Coordinator’s inability to retry the request on another shard on a different node might result in user’s getting partial response.
The above will provide the required isolation of accounting and fairness in the rejections which is currently not there. This is still a reactive back-pressure mechanism as it only focusses on the current consumption and does not estimate the future work which is to be done for these search requests.
2.2 Server side Cancellation of in-flight search requests based on resource consumption
This is the 3rd level which kicks in after we’re cancelling all search request coming to a node. Here, we take decision to
cancel on-going requests, If the resource limits for that shard/node have started breaching the assigned limits (point 2.1), and there is no recovery seen for a certain time threshold. The BackPressure model should support identification of queries which are most resource guzzling with minimal wasteful work. These can then be cancelled for recovering a node under load and continue doing useful work.
Milestone 3
Goals

Have Search request cost estimation capability
Have improved capability to take request admission decision based on the request cost and available resources (Think Token buckets). (Refer Section 3.2 C3 paper )

3.1 Query resource consumption estimation
Improve the framework added as part of (point 2) to also estimate the cost of new search queries, based on their potential resource utilisation. It can be achieved by looking at the past query patterns, with similar query constructs and the actual data on shard. This will help in building a pro-active back-pressure model for search requests, where estimates will be compared against the available resources during the admission decision for granular control.

I think in general it makes a lot of sense. But here is the counter example (seen in the wild): the query phase take a very long time, following the fetch phase. At that moment, the client has already lost any hope to get the response (may be even gone), but the server is still working on it, the live queries are going to be rejected because the fetch phase for this "dead" query is still ongoing. Would it make sense to take into account the overall age of the query (when kicking off the fetch phase) to weight its completion relevance to the client?

Perhaps we can check if the client connection is still active, and if not, propagate it to the data nodes to cancel that query.

Perhaps we can check if the client connection is still active, and if not, propagate it to the data nodes to cancel that query.
@asafm I think something along these lines, for example interpreting client disconnects as tasks cancellation and propagating that across the nodes (since coordinator is aware of client state and tasks).

This is a good proposal 👍. Resource consumption is great, but it's a lot of work, so I would in parallel consider simpler approaches.

hard limit on number of items in the search queue (exists today) cancels existing queued items (if queue is full, flush it, cancel existing requests, cool down)
general resource consumption (e.g. cancel all existing operations and cool down if overall CPU is too high over a window)
evaluating queries and measuring their cost (heavy queries or queries known to have high execution cost get rejected without being executed)
time in queue over a window (new? overall search duration is increasing)

Done!  https://github.com/opensearch-project/OpenSearch/tree/feature/task-resource-tracking

w? ove

Thanks @dblock for going through the proposal. We have the PRs related to task resource tracking in progress and that should complete soon. Targeting march for that.
Soon we'll fast follow that up with initial low effort rejection strategies which you and others mentioned which we'll include as part of Milestone 2.
Milestone 3 is a lot of work and we'll have to evaluate the need and criticality after we have initial improvements merged in.

I am curious how to get cpu time and memory consumption for each query? It's a very difficult job

Adding Meta Issue and the child issues opened previously to this RFC in order to track deliverables:
https://github.com/opensearch-project/OpenSearch/issues/1042

@tushar-kharbanda72 @getsaurabh02 since you have META issue to track the progress of this initiative. Could you close the RFC? and what is the target release for this project.

Is this still on track for 2.2?

This (milestone 2) will come in 2.3 - we are merging in the changes for resource tracking framework in 2.2 (milestone 1)

Thanks @rramachand21!

@rramachand21 is there an issue cut for milestone 2? I'd like to sync it up with the related documentation issue.

@rramachand21 please provide a feature document for this so we can get it documented if it's still on track for 2.3.  The docs issue is https://github.com/opensearch-project/documentation-website/issues/795
Currently I'm blocked.

@tushar-kharbanda72 is this still on track for 2.4 release? code freeze on 11/3
Is there anything pending? otherwise, feel free to close it.

@rramachand21 Is there anything pending on this? If not, can we close this issue since 2.4 is already in production and update with latest version that you are tracking?

@rramachand21 is this on track for 2.6.0 release? The code freeze for this is today (Feb 21, 2023)

@ketanv3 could you please update the latest on this?

Hi @tushar-kharbanda72 , This issue will be marked for next-release v2.8.0 on (Apr 17) as that is the code-freeze date for v2.7.0. Please let me know if otherwise.

Tagging it to next release: v2.8.0


uncaught syntax error. requested module doesn't provide an export named 'format'

import  { format } from 'sql-formatter' ;
console.log(format('select * from tbl'))
after npm i tried to access the format method. But it throwing "uncaught syntax error: the requested module ../node_modules/sql-formatter/lib/sqlFormatter.js" does not provide an export named 'format'

Unable to reproduce.


A couple of fixes for nodepool resources and cloudwatch logging

fix issue with nodepool names containing a - character.
Properly pass cloudwatch logging arn from root stack into the network stack

Codecov Report

Merging #1504 into master will decrease coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #1504      +/-   ##
==========================================
- Coverage   25.47%   25.46%   -0.02%
==========================================
  Files          97       97
  Lines        5001     5003       +2
==========================================
  Hits         1274     1274
- Misses       3580     3582       +2
  Partials      147      147

Impacted Files
Coverage Δ

pkg/api/worker_node_pool.go
12.06% <0%> (-0.22%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc29af4...47ca520. Read the comment docs.

@davidmccormick what problem did you have here? Asking since all our node pool names have hyphens in them and not spotted a problem with that so far but perhaps missed something.

@mumoshu @c-knowles are you happy to merge this?

@davidmccormick probably just wondering what the problem was? Please see my comment.

The issue I was having is that our nodepool names contain hyphens - so am getting the following error when I try and build a cluster using the master branch: -
generating assets for control-plane, etcd, network, nodepool-a, nodepool-b, nodepool-c
Error: failed to validate network: invalid cloudformation stack template https://s3.amazonaws.com/034324643013-k8tool-us-west-2/cf/davem-lab-secure/kube-aws/clusters/davem-lab-secure/exported/stacks/network/stack.json: ValidationError: Template format error: Resource name nodepool-aIAMManagedPolicyWorker is non alphanumeric.
	status code: 400, request id: 14a5f3f8-fa39-11e8-80b2-7101c712114a

The change makes sure that the nodepool names get sanitized in much the same way we do for controllers and etcd stack names etc.

@c-knowles @mumoshu I'm not sure why Chris isn't seeing this but as I see it santizing the name can't hurt and this also fixes an issue with cloudwatch logging which I'm guessing you have disabled.  I'd like to get these PRs merged so we are not running a forked codebase.

@davidmccormick we have CloudWatch logging turned on and I verified logs are coming through. I think this might only be a problem on latest large master refactor then, we are using v0.12.0 patched with the two PRs for stack existence checks with over 100 stacks in one account so a few commits behind.
As you say it seems like we'd want to sanitise anyway.

/lgtm
/approve


Problems with cache and custom image processor

I have implemented a custom ImageProcessor for applying a grayscale filter when downloading images with Kingfisher, and I'm using it on a tableview's cells (one image per cell).
The problem that I've found is that, while I was developing the filter, Kingfisher cached the images without applying the processor. In an attempt to surround this (at least while developing), I added a  bar button item for clearing the image cache whenever I want.
But the cache is not been cleared. Kingfisher always fetches previously cached images without the filter applied, and it doesn't even call the "process" method of my custom processor (it's just being ignored).
For clearing the cache, I've tried both:
KingfisherManager.shared.cache.clearDiskCache()
KingfisherManager.shared.cache.clearMemoryCache()
The only way I've found for clearing the cache successfully is to add the .forceRefresh option when calling to kf_setImage, but of course I don't want Kingfisher to ignore the cache every time I scroll the tableview (in fact, it would destroy the cache system).
¿Am I doing something wrong? ¿Could it be an issue on ImageProcessor or ImageCache?

May I know how did you set the image with processor?
If you are using the image view extension, Kingfisher will send the processed image to cache by default, so it is intended that the processor method not being called again when retrieving data from cache, since the data should already be precessed before being stored.

@AlbertoGuerreroMartin Hmmm, interesting. You mean the process method didn't get called at all?
Could you post your implementation of grayscaleProcessor here? Or even better, preparing a sample project which could reproduce it. I could have a detail look at it then.

ok @onevcat , nevermind, I had an old version of Kingfisher. After updating it, everything is working fine 😅  sorry and thanks!


[5.2] Sessions Aren't Persisting

Using a fresh installation of Laravel 5.2, I've tried setting a session like so:
Session::put('foo', 'bar');

I visit the page in my browser to set the session, then remove that line. However, when I try to output the session like so:
 dd(Session::get('foo'));

...it returns null. I get the expected result of bar when using the same code and scenario in Laravel 5.1.

That's because dd kills the app before the session is saved...

The put method doesn't actually persist the data, if you like, it rather, just queues it to be persisted at the end.

Also, make sure you include these routes inside the group with the web middleare applied.


Update management.properties

The cas prefix is not used for mgmt properties in 5.3. It will cause errors like, "Invalid property 'mgmt[serverName]'

Thank you! Could you fix the line-endings please as well?

I would, but I can't see them in any editor I use. I tried Vim, nano and even the web editor. They didn't seem to cause any problems.
Also, I don't know where the problem is, but this branch pulled in two different versions of spring_web and Tomcat kept complaining about using absolute ordering. This caused Tomcat not to start cas-management. I didn't know where to make the change, so I just removed one of the spring_web jar files and it worked. I actually had to do the same when I added in mongodb service repo dependencies to pom.xml. It brought in two and one of the versions didn't work with the older version of spring_boot.


Is there any way to use JSON like this ?

{
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"}
}
Is there anyway to parse JSON like this ? In the example you have given in the GitHub ReadMe, You were saying that "user" must be needed to get each dictionary from the JSON. But in my case i don't have any key its just the plane and simple dictionary. I was just curious about this, cause i could not find any workaround this, I just wanted to know if anyone from this Argo community know how to parse this JSON. Please help me out here.

Interesting, is that valid json? I though objects needed key/value pairs. Is this just an array of objects?

What's your desired end result? In other words, what's your desired model object? An array names, or are all of these part of the same model?

Nope, that's not valid JSON.

Would NSJSONSerialization still parse it with fragments enabled?

This is the URL -> http://thetopfives.net/thetopfivesposts

Or should i loop through all the dictionaries and apply Argo way to parse those values .?

I'm guessing your meant this then:
[
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"},
{"name":"alvin"}
]

Yes i was, Sorry about the parenthesis. Yes it was a bracket.

So what you guys think about this. Is this a valid JSON URL ? Or Argo still needs key to process this JSON ?

you should just be able to say let users: Decoded<[User]> = decode(json) where json is the AnyObject returned from NSJSONSerialization

//
//  DataFromServer.swift
//  The Top Fives
//
//  Created by Alvin Varghese on 8/25/15.
//  Copyright (c) 2015 The Top Fives. All rights reserved.
//
import UIKit
import Foundation
import Argo
import Runes
struct TheTopFivesData {
let title : String
let redirectUrl : String
let discussionUrl : String
let serialNumber : Int
let color : String
let newsSourceName : String
let newsSourceId : Int

}
extension TheTopFivesData: Decodable {
static func create(title: String)(redirectUrl: String)(discussionUrl: String)(serialNumber: Int)(color: String)(newsSourceName: String)(newsSourceId : Int) -> TheTopFivesData {

    return TheTopFivesData(title: title, redirectUrl: redirectUrl, discussionUrl: discussionUrl, serialNumber: serialNumber,color : color, newsSourceName: newsSourceName, newsSourceId: newsSourceId)
}

static func decode(j: JSON) -> Decoded<TheTopFivesData> {
    return TheTopFivesData.create
        <^> j <| "title"
        <*> j <| "redirectUrl"
        <*> j <| "discussionUrl"
        <*> j <| "serialNumber"
        <*> j <| "color"
        <*> j <| "newsSourceName"
        <*> j <| "newsSourceId"
}

}
class DataFromServer: NSObject, NSURLConnectionDataDelegate, NSURLConnectionDelegate {
//MARK: Local Variables

var mutableData : NSMutableData!

//MARK: Request for Data

func requestData()
{
    NSURLConnection(request: NSURLRequest(URL: NSURL(string: "http://thetopfives.net/thetopfivesposts")!), delegate: self, startImmediately: true)
}

//MARK: NSURLConnection Delegate

func connection(connection: NSURLConnection, didReceiveResponse response: NSURLResponse)
{
    self.mutableData = NSMutableData()
}

func connection(connection: NSURLConnection, didReceiveData data: NSData)
{
    self.mutableData.appendData(data)
}

func connectionDidFinishLoading(connection: NSURLConnection)
{
    // Wherever you receive JSON data:

    if let json: AnyObject? = NSJSONSerialization.JSONObjectWithData(self.mutableData, options: NSJSONReadingOptions(0), error: nil) {

        if let j: AnyObject = json {

            let allValues : Decoded<[TheTopFivesData]> = decode(j)

            println("COUNT : \(allValues.value?.count)") // prints COUNT : nil
        }
        else
        {
            println("NO DATA")
        }

    }
    else
    {
        println("NO JSON")
    }

}

func connection(connection: NSURLConnection, didFailWithError error: NSError)
{
    utilitiesObjet.alertThis("Oh ho !!", message: "\(error.localizedDescription)")
}

}

I solved it, there was some optional values. Added optional value ? notation to the variables. Now its working fine. Thanks to all who spent their valuable time with me to make me re think on this.


Document how to add and work with images

Document how to:

Add images via the Netlify CMS image and markdown widgets
Customize images from Netlify CMS
Add incidental images directly in the code (not via Netlify CMS)

These most likely can be done by looking at existing code and content, but it will be better—especially for new and/or beginner contributors—to have a documentation on hand.

Should based on this gist, if anyone want to help on this issue
https://gist.github.com/ekafyi/1a86987c111a96976309c12f047d43a9

Should based on this gist, if anyone want to help on this issue
https://gist.github.com/ekafyi/1a86987c111a96976309c12f047d43a9

Bad news: That gist is outdated.
Good news: It's even more simple now since Cloudinary has been integrated to Netlify CMS. This issue is mostly to remind myself to update that gist. 😁

Copying and pasting this from here:

We should also add some documentations on how to gain credentials to our content dashboards (e.g. Netlify CMS and Cloudinary), so that no further confusion arises like in #696.
For the time being, as mentioned here by @zainfathoni, the only way to gain these credentials is to ask either of us. And do take note that for security reasons, we only provide these credentials to those who, really, really, really need it.

In summary, we should also document how to properly gain credentials to the dashboards we use.


Fixed .mbedignore logic to ignore the walk() root

Match walk() root against the ignore list.
For example .mbedignore in tools/ contains '*' and naturally should match all files, folders including tools/ itself. Without this fix, tools/ is added to the include path

LGTM 👍


Simplify template and schema caching

What is the context of this PR?
Remove flask-caching, because we are only caching in memory we don't need it to be serializing the objects and can save some CPU time. By removing flask-caching templates will instead use the default
in-memory LRUCache, for schemas we just use a dict.
For now with the schemas I've had to deepcopy them as they get modified after being fetched.  In future we could track down where this is happening to avoid the copy.
How to review
Run a couple of surveys, check it still works.

Various unit tests were failing when run together, but not in isolation.


Incorrect Ruby compatibility in gemspec

Since version 3.15.0, webmock is incompatible with Ruby 2.2 or older because it started using the &. syntax here https://github.com/bblimke/webmock/blob/v3.15.0/lib/webmock/request_signature.rb#L38.
It's still reporting Ruby >= 2.0 in the gemspec so bundle install install this version incorrectly when it should pick 3.14.0.
It does work fine for Ruby >= 2.0 with version 3.14.0.

@arielj sorry for that.
I have released 3.15.1 and 3.16.1 with required ruby 2.3,
and I have yanked 3.15.0 and 3.16.0

thanks!

@bblimke Hey! Thanks for fixing this. Unfortunately yanking gems in this case quite possibly caused more breakage in application delivery than those few people still using Ruby < 2.3. Yanking gems should only really be done if there are urgent security concerns in a release, and not just because it is in some way broken. I appreciate all of your efforts in open source maintenance, and I know from personal experience how hard it is, but I feel an obligation to spread the word about the dangers of withdrawing packages.

Can a released gem be updated with the correct ruby requirement? I feel like that should be safer than yanking (I don't know if a version can be un-yanked either)

Can a released gem be updated with the correct ruby requirement? I feel like that should be safer than yanking (I don't know if a version can be un-yanked either)

No. Modifying a release is just as bad or worse than yanking it. I don't believe a yanked gem can be re-published.
The most appropriate solution here would have been to publish a new fixed version as was done, but without yanking the prior broken versions; by doing so, those who already encountered breakage can fix it by upgrading and you don't break anyone for whom the previous releases were working.

I'm sorry for the mess created 🤦 Yanking was a bad idea.
@benlangfeld thank you for the useful feedback.

The most appropriate solution here would have been to publish a new fixed version as was done, but without yanking the prior broken versions; by doing so, those who already encountered breakage can fix it by upgrading and you don't break anyone for whom the previous releases were working.

I agree, but the fixed versions should have been fixed, by making 3.15 and 3.16 ruby < 2.3 compatible,
and not by increasing the ruby version requirement in the gemspec, as I have done.
I believe that if I haven't yanked the 3.15.0 and 3.16.0, then Ruby < 2.3 would still try installing 3.15.0 and 3.16.0,
since 3.15.1 and 3.16.1 require Ruby >= 2.3.
I'm till considering creating 3.15.2 and 3.16.2, Ruby >= 2.0 compatible if it is useful to anyone.

The fix is easy enough, but it would be useful to surface this communication more clearly.
Some ideas:

https://github.com/bblimke/webmock/tags (+ CHANGELOG too maybe) - add note to yanked versions
Create and pin a new issue with a summary

@JBallin how can I add notes to the tags page?

I'm till considering creating 3.15.2 and 3.16.2, Ruby >= 2.0 compatible if it is useful to anyone.

My use case was running tests for the next_rails gem that we use at work to upgrade old apps so we need the old ruby support for the gem's tests, but it's enough for us to pin the version to 3.14.0 so no need at least for us to have 3.15+ with support for ruby 2.

I don't have direct experience with tags, but you should be able to annotate them and it appears you could alter existing tags, but this may not be worth the effort. Pinning an issue is likely easier.


Scipy-1.7.0 build fails in AIX

pip install scipy fails with the below error in AIX. The earlier version 1.6.3 had no problem in AIX.
setup.py:492: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-1jt97u3e'), proceeding with generating Cython sources and expanding templates
  ' '.join(sys.argv[1:])))
Running from SciPy source directory.
Traceback (most recent call last):
  File "/opt/freeware/lib64/python3.7/configparser.py", line 788, in get
    value = d[option]
  File "/opt/freeware/lib64/python3.7/collections/__init__.py", line 916, in __getitem__
    return self.__missing__(key)            # support subclasses that define __missing__
  File "/opt/freeware/lib64/python3.7/collections/__init__.py", line 908, in __missing__
    raise KeyError(key)
KeyError: 'defines'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/freeware/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 280, in <module>
    main()
  File "/opt/freeware/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 263, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/opt/freeware/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 133, in prepare_metadata_for_build_wheel
    return hook(metadata_directory, config_settings)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 166, in prepare_metadata_for_build_wheel
    self.run_setup()
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 259, in run_setup
    self).run_setup(setup_script=setup_script)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 150, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 629, in <module>
    setup_package()
  File "setup.py", line 625, in setup_package
    setup(**metadata)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/core.py", line 137, in setup
    config = configuration()
  File "setup.py", line 536, in configuration
    config.add_subpackage('scipy')
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 1036, in add_subpackage
    caller_level = 2)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 1005, in get_subpackage
    caller_level = caller_level + 1)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 942, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy/setup.py", line 12, in configuration
    config.add_subpackage('interpolate')
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 1036, in add_subpackage
    caller_level = 2)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 1005, in get_subpackage
    caller_level = caller_level + 1)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib64/python3.7/site-packages/numpy/distutils/misc_util.py", line 942, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy/interpolate/setup.py", line 62, in configuration
    config=['compiler.blas=none']
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib/python3.7/site-packages/pythran/dist.py", line 145, in __init__
    cfg_ext = cfg.make_extension(python=True, **kwargs)
  File "/tmp/pip-build-env-qym5xzvr/overlay/lib/python3.7/site-packages/pythran/config.py", line 139, in make_extension
    cfg.get('compiler', 'defines').split()],
  File "/opt/freeware/lib64/python3.7/configparser.py", line 791, in get
    raise NoOptionError(option, section)
configparser.NoOptionError: No option 'defines' in section: 'compiler'
----------------------------------------

Any idea/hint about this error ?

Can it be related to Pythran @serge-sans-paille??

It certainly is! Looks like we need to ship a configuration file for aix, which we don't.
@ayappanec  what's the output of sys.platform on AIX?

It is "aix6" or "aix7" based on the AIX OS level where python is built. We can safely assume it contains "aix".

Can you try export SCIPY_USE_PYTHRAN=0 and see if that fixes things @ayappanec? We haven't tested the AIX build after the recent build system changes for Pythran, so it failing is not completely surprising.

@rgommers It worked , exporting SCIPY_USE_PYTHRAN=0

Great, I think that's fine for now on AIX. For the 1.8.0 release I expect we have to do some real work on AIX, to make both Pythran and Meson work. So might as well take those two things together and not worry about it too much right now.

It certainly is! Looks like we need to ship a configuration file for aix, which we don't.
@ayappanec what's the output of sys.platform on AIX?

Any update on having a configuration file for AIX in pythran ?

@ayappanec can you try that branch? https://github.com/serge-sans-paille/pythran/pull/1824

@ayappanec can you try that branch? serge-sans-paille/pythran#1824

Tried it . Now the script build fails with the below error.
C compiler: g++ -maix64 -fPIC -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -fPIC -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -fPIC
compile options: '-DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DPYTHRAN_BLAS_NONE -I/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran -I/opt/freeware/lib64/python3.7/site-packages/numpy/core/include -I/opt/freeware/lib64/python3.7/site-packages/numpy/core/include -Ibuild/src.aix-7.1-3.7/numpy/distutils/include -I/opt/freeware/include/python3.7m -c'
extra options: '-std=c++11'
g++: scipy/interpolate/_rbfinterp_pythran.cpp
In file included from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/types/xsimd_traits.hpp:17,
from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/xsimd.hpp:16,
from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/pythonic/include/numpy/exp.hpp:8,
from scipy/interpolate/_rbfinterp_pythran.cpp:26:
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/types/xsimd_types_include.hpp:83:7: warning: #warning "No SIMD instructions set detected, please enable SIMD instructions or activate the fallback mode. (e.g. for x86 -march=native or for ARM -mfpu=neon)" [-Wcpp]
#warning "No SIMD instructions set detected, please enable SIMD instructions or activate the fallback mode. (e.g. for x86 -march=native or for ARM -mfpu=neon)"
^~~~~~~
In file included from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_math.hpp:23,
from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/xsimd.hpp:17,
from /opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/pythonic/include/numpy/exp.hpp:8,
from scipy/interpolate/_rbfinterp_pythran.cpp:26:
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp: In function 'void xsimd::sincos(float, float&, float&)':
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp:362:11: error: '::sincosf' has not been declared
::sincosf(val, &s, &c);
^~~~~~~
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp:362:11: note: suggested alternative: 'sincos'
::sincosf(val, &s, &c);
^~~~~~~
sincos
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp: In function 'void xsimd::sincos(double, double&, double&)':
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp:374:11: error: '::sincos' has not been declared
::sincos(val, &s, &c);
^~~~~~
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp:374:11: note: suggested alternative:
/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran/xsimd/math/xsimd_scalar.hpp:369:17: note:   'xsimd::sincos'
inline void sincos(double val, double&s, double& c)
^~~~~~
/opt/freeware/lib64/python3.7/site-packages/numpy/distutils/system_info.py:933: UserWarning: Specified path /usr/local/include/python3.7m is invalid.
return self.get_paths(self.section, key)
########### EXT COMPILER OPTIMIZATION ###########
Platform      :
Architecture: unsupported
Compiler    : gcc
CPU baseline  :
Requested   : optimization disabled
Enabled     : none
Flags       : none
Extra checks: none
Requested   : optimization disabled
CPU dispatch  :
Enabled     : none
Generated   : none
CCompilerOpt.cache_flush[804] : write cache to path -> /tmp/pip-req-build-_jmugyn8/build/temp.aix-7.1-3.7/ccompiler_opt_cache_ext.py
########### CLIB COMPILER OPTIMIZATION ###########
Platform      :
Architecture: unsupported
Compiler    : gcc
CPU baseline  :
Requested   : optimization disabled
Enabled     : none
Flags       : none
Extra checks: none
Requested   : optimization disabled
CPU dispatch  :
Enabled     : none
Generated   : none
CCompilerOpt.cache_flush[804] : write cache to path -> /tmp/pip-req-build-jmugyn8/build/temp.aix-7.1-3.7/ccompiler_opt_cache_clib.py
error: Command "g++ -maix64 -fPIC -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -fPIC -I/usr/include -I/opt/freeware/include -I/opt/freeware/include/ncurses -DAIX_GENUINE_CPLUSCPLUS -D_LINUX_SOURCE_COMPAT -Wl,-brtl -O2 -fPIC -DENABLE_PYTHON_MODULE -D__PYTHRAN_=3 -DPYTHRAN_BLAS_NONE -I/opt/freeware/lib/python3.7/site-packages/pythran-0.9.12.dev0-py3.7.egg/pythran -I/opt/freeware/lib64/python3.7/site-packages/numpy/core/include -I/opt/freeware/lib64/python3.7/site-packages/numpy/core/include -Ibuild/src.aix-7.1-3.7/numpy/distutils/include -I/opt/freeware/include/python3.7m -c scipy/interpolate/_rbfinterp_pythran.cpp -o build/temp.aix-7.1-3.7/scipy/interpolate/_rbfinterp_pythran.o -MMD -MF build/temp.aix-7.1-3.7/scipy/interpolate/_rbfinterp_pythran.o.d -std=c++11" failed with exit status 1
ERROR: Failed building wheel for scipy

Thanks for the feedback. Can you give it a try with https://github.com/serge-sans-paille/pythran/pull/1829 ?

@ayappanec did you have the opportunity to test the above PR?

Yes, now i am able to pip install scipy 1.7.0 with the above pythran changes.

Thanks! I'll merge the PR and it will be in next Pythran release, cc @rgommers

Awesome, thanks @serge-sans-paille and @ayappanec! Let's close this issue then.


chore: roll back to xcode10 for xcode universal build issue

Summary

Universion-SDK build fails with Xcode11+
Rollback to Xcode10 which works ok

Coverage remained the same at 98.138% when pulling b559d555f62a861e362a97af99fde5a8b5e61e33 on jae/release-build-error into 2093f81f78f5ec10f94352a14b54f130ab62a004 on master.


RpcId not working.

Godot Engine V3.0.stable.official.mono
Ubuntu 64 bit
Actual:
RpcId(int peerId, string method, [args]);
Fails silently.
Expected:
RpcId(int peerId, string method, [args]);
should make a remote call to the specified peer.
Steps to reproduce:
1.Unzip included Example file.
2. Edit Server project in godot editor.
3. Edit Client project in godot editor. (Make sure debug ports are different between godot instances.)
4. Run server in editor.
5. Run Client in server.
6. You should see Rpc calls succed and RpcId calls fail.
Example.tar.gz

To be clear: This is in C#, I just tested rpc_id() in GDScript and it works fine.

I also hit this issue in an earlier build, but it appears to be fixed in master

Is it fixed in 3.0.2 too, or is the fix missing there?

Yeah it works there.  Sorry for the delay!

Fixed as per above comments.


Flutter to packages roller fails from failure to click play button in video_player_avfoundation XCUI test

The XCUI test for plugin video_player_avfoundation has started failing and is blocking the Flutter to packages roller:
/Volumes/Work/s/w/ir/x/w/packages/packages/video_player/video_player_avfoundation/example/ios/RunnerUITests/VideoPlayerUITests.m:32: error: -[VideoPlayerUITests testPlayVideo] : Failed to Error Domain=com.apple.dt.xctest.ui-testing.error Code=10202 "Failed to scroll to visible (by AX action) StaticText, {{48.3, 35.1}, {33.3, 33.3}}, label: 'Play', error: Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37" UserInfo={NSLocalizedDescription=Failed to scroll to visible (by AX action) StaticText, {{48.3, 35.1}, {33.3, 33.3}}, label: 'Play', error: Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37, NSUnderlyingError=0x6000019eab80 {Error Domain=XCTDaemonErrorDomain Code=0 "Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37" UserInfo={NSLocalizedDescription=Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37}}}: Failed to scroll to visible (by AX action) StaticText, {{48.3, 35.1}, {33.3, 33.3}}, label: 'Play', error: Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37. (Underlying Error: Error kAXErrorCannotComplete performing AXAction kAXScrollToVisibleAction on element AX element pid: 28392, elementOrHash.elementID:<PHONE_NUMBER>.37)
    t =     6.56s Tear Down
Test Case '-[VideoPlayerUITests testPlayVideo]' failed (6.776 seconds).
Test Suite 'VideoPlayerUITests' failed at 2023-02-27 07:33:58.192.
	 Executed 1 test, with 1 failure (0 unexpected) in 6.776 (6.777) seconds
Test Suite 'RunnerUITests.xctest' failed at 2023-02-27 07:33:58.192.
	 Executed 1 test, with 1 failure (0 unexpected) in 6.776 (6.778) seconds
Test Suite 'All tests' failed at 2023-02-27 07:33:58.192.
	 Executed 1 test, with 1 failure (0 unexpected) in 6.776 (6.779) seconds

The failing test: https://github.com/flutter/packages/blob/main/packages/video_player/video_player_avfoundation/example/ios/RunnerUITests/VideoPlayerUITests.m#L22
Failing line:

It seems to be fail when clicking the play button in the example found in main: https://github.com/flutter/packages/blob/main/packages/video_player/video_player_avfoundation/example/lib/main.dart#L243
Potentially related to the semanticLabel?
After running the test on different flutter/flutter commits, it seems that the culprit is https://github.com/flutter/flutter/pull/113461
cc @chunhtai @goderbauer

revert https://github.com/flutter/flutter/pull/121562

lower to P2 since the revert is landed, and it is now blocking https://github.com/flutter/flutter/issues/111931 instead

note to myself
./script/tool_runner.sh native-test --ios --ios-destination "platform=iOS Simulator,name=iPhone 13,OS=latest" --packages=video_player


SSL verification is Running - [BRIEF DESCRIPTION]

App Id: d1dmgnwdmwopvw
Region: us-east-1
Step: SSL verification
Status: Running

Note: Do not include information that is sensitive in nature such as your domain name, company etc.

Issue/question
SSl verification pending, message :
"
We validated that the CNAME record exists in your DNS file. We will now verify ownership of the domain. If this takes longer than 8 hours, please create an issue in our GitHub issue tracker.
"
Error message
No error, just pending
Note: Be sure to check the message for sensitive information.
Additional information
Im using Cloudflare

I looked into your domain, the verification record seems fine. Can you goto the domain manage page and click the re-try button?

duplicate of https://github.com/aws-amplify/amplify-console/issues/57


land cover classification notebook -- switches, data downloads, metadata

Some changes to

allow using either the RGB or the larger data set
download data automatically if needed (to tempdir)
set the degree of optimization (or 0 to speed things up)
better inform, including about data set
fix the accuracy function (?)
tweak parameters to improve convergence and reduce wait time

@rafaqz @Alexander-Barth , please take a look if you get a chance

I just reran notebook (on this config) with up to date dependencies.
Any objection to merging?


Connection Issue on Oracle Apex installed in NAS

Hi!
I have created an oracle Apex container in my NAS.
My NAS IP is <IP_ADDRESS> and NAS uses the 8080 port to access it, so I can access to APEX.
Any idea to access it ?
Thanks!

Solved! My QNAP NAS creates a 8080 redirection with another port! :) 👍


Expand ci build & verify across all feeds with workflow matrix

This looks to work towards  #77
As lint/format commits are somewhat noisy,  it may make sense to apply build/verify to the entire codebase (not just the feeds) in a single PR instead of gradually. Also there may be a preference for having the lint commit before instead of after the commit in which the feature is enabled in CI, as such each commit passes.
Using workflow matrices allows the parallelisation across feed packages whilst reducing the amount of repetition in the source files, or reliance on Make Recursion.

This is superseed by #92


Stop Web3signer from importing EXITED validators

Now that Shapella has come and gone, we are more likely to see keys in the EXITED state.  As per protocol, these keys can never be re-used. We are expecting to generate more keys into existing clusters, taking the total up from 10K to, for example, 20K. Half of these potentially could have been exited and importing those no-longer-in-use keys will take time on start-up.
Is there a way for web3signer to understand 'state' ? In our infrastructure, we are reluctant to either delete keys from the vault/secrets manager or, in fact, add further data into that secure space. Perhaps the signer can grab state from a flat-file or some other mechanism on start-up? There's no real need for this to be super dynamic because our impression is the main time cost will be on start-up.
We have the validator state and can make that available alongside the public key.

One thing to note: Right now, this doesn't feel like a priority action/requirement but just want to put this on all our radars.

@beetrootkid Let's have our team's connect on this

@non-fungible-nelson  - yes, let's talk before my exit. Shall I set up a meeting with the team? (It might be hard to include you because of your timezone)

Hi @non-fungible-nelson  - after a conversation between the teams, we've decided to close this issue for now. The only point this really creates load is during start-up. There's a lot of complexity involved to address it and the benefits delivered don't warrant addressing it.

Thank you! Let's wait for validator indices reuse


Week 1 Review Ticket - Brian

SCORE = ~/3, REVIEWED BY Alex Do FOR Brian Zhang
Individual GitHub pages (3 points)

GitHub page for Test Prep study, requiring 5.3, 5.4 notes and actions

[ ] One Check 5.3
[ ] One Check 5.4

GitHub page that documents key learnings and code snippets using GitHub and replit.

[ ] InfoDB lists
[ ] One Check for New Data, minimum of 4 items

InfoDB loops

[ ] One Check For Loop running and sensible output
[ ] One Check While Loop running and sensible output
[ ] One Check Recursion running and sensible output

Fibonacci - (For Fibonacci(n) print complete list 0 - n and not just the nth term)

[ ] Four checks:

Algorithm and code with comments
Prints result
Prints Complete List
Work on complex sequence with multiple words, spaces, special characters

[ ] One check: comments and documentation throughout

Score = 2.8/3, Reviewed by Alex Do
Individual GitHub pages (3 points)

GitHub page for Test Prep study, requiring 5.3, 5.4 notes and actions

[x] 1- One Check 5.3
[x] 1- One Check 5.4

GitHub page that documents key learnings and code snippets using GitHub and replit.

[x] InfoDB lists
[x] 1 - One Check for New Data, minimum of 4 items

InfoDB loops

[x] 1- One Check For Loop running and sensible output
[x] 1- One Check While Loop running and sensible output
[x] 1- One Check Recursion running and sensible output

Fibonacci - (For Fibonacci(n) print complete list 0 - n and not just the nth term)

[x] 4- Four checks: Algorithm and code with comments, Prints result, Prints Complete List, Work on complex sequence with multiple words, spaces, special characters
[x] 1- One check: comments and documentation throughout


Problem with PDOStatement and consuming objects

Hi, this is more of a feature request than a issue.
Could i prevent kint from iterating PDOStatement?
The problem is that it consumes the resultset iterating over it and the data is not available past the dump, took me a lot to realize that the only problem in my page was the dump to verify the connection.
Here's my code for reference:
    $query = "select * from lollo";
    $db = Database::getInstance(); //returns PDO
    $result = $db->query($query);
    d($result); //dump
    $result->setFetchMode(PDO::FETCH_CLASS,'stdClass');
    while($row = $result->fetch()){ //does not even enter this
        d($row);
    }

Your only option here (Until 2.x is done) is to alter Kint source code - specifically you can delete parsers/custom/objectiterable.php to disable iterator debugging, or alter it to filter out PDOStatements.
diff --git a/parsers/custom/objectiterateable.php b/parsers/custom/objectiterateable.php
index bbc0de0..50010c2 100644
--- a/parsers/custom/objectiterateable.php
+++ b/parsers/custom/objectiterateable.php
@@ -8,6 +8,7 @@ class Kint_Parsers_objectIterateable extends kintParser
 			|| !is_object( $variable )
 			|| !$variable instanceof Traversable
 			|| stripos( get_class( $variable ), 'zend' ) !== false // zf2 PDO wrapper does not play nice
+			|| $variable instanceof PDOStatement
 		) return false;

Fixed in 2.x-dev. Please test and give feedback, thanks!

Working like a charm, thanks!


Separate input and output

Put the output in a different excel.
Have a default path and name, but allow the user to override it.
Put timestamp as part of the default output file name.

fixed
will now create output in the file path given as the third argument or will use the default (creating it at the user folder)


[SPARK-10060] [ML] [DOC] spark.ml DecisionTree user guide

New user guide section ml-decision-tree.md, including code examples.
I have run all examples, including the Java ones.
CC: @manishamde @yanboliang @mengxr

Merged build triggered.

Merged build started.

Added WIP tag since I realized I still need to document some new features such as class probabilities.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41036/
Test FAILed.

Merged build triggered.

Merged build started.

Test build #41042 has started for   PR 8244 at commit 6b2606a.

Merged build triggered.

Merged build started.

Test build #41045 has started for   PR 8244 at commit 9dec3d6.

Test build #41042 has finished for   PR 8244 at commit 6b2606a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
The Pipelines API for Decision Trees offers a bit more functionality than the original API.  In particular, for classification, users can get the predicted probability of each class (a.k.a. class conditional probabilities).

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41042/
Test PASSed.

Test build #41045 has finished for   PR 8244 at commit 9dec3d6.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
The Pipelines API for Decision Trees offers a bit more functionality than the original API.  In particular, for classification, users can get the predicted probability of each class (a.k.a. class conditional probabilities).

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41045/
Test PASSed.

Merged build finished. Test PASSed.

Btw, what do you think of the table of input/output columns?  Too much?

I think they are nice and clear:)

OK updated!

Merged build triggered.

Merged build started.

Test build #41168 has started for   PR 8244 at commit 56a831e.

Test build #41168 has finished for   PR 8244 at commit 56a831e.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
The Pipelines API for Decision Trees offers a bit more functionality than the original API.  In particular, for classification, users can get the predicted probability of each class (a.k.a. class conditional probabilities).

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/41168/
Test PASSed.

Note: I'll need to update the examples after [https://github.com/apache/spark/pull/8290] gets merged.

LGTM. Merged into master and branch-1.5. We can update the examples in a separate PR. Thanks!


Migration is not  being applied to db

Project has  single migration - Initial.  Applied to db and single record in  __EFMigrationsHistory.
Next I add new migration and try to apply it to db.  update-database say done but I have exception like
SqlException: Invalid object name 'Files'.
h2>Applying existing migrations for ApplicationDbContext may resolve this issue

There are migrations for ApplicationDbContext that have not been applied to the database

20170122215933_files

and no record added to __EFMigrationsHistory.
also then I try remove-migration I got message that migration Initial is applied to db and cannot be undone (then is should be second migration named "file")
In package management console it looks like  this
PM> add-migration  file
To undo this action, use Remove-Migration.
PM> update-database
Done.
PM> remove-migration
The migration '20170122010641_Initial' has already been applied to the database. Unapply it and try again. If the migration has been applied to other databases, consider reverting its changes using a new migration.
Update dotnet ef database  update  from console did apply migration

What version of the Microsoft.EntityFrameworkCore.Tools package were you using? What is the output of the command with -Verbose?

EF Team Triage: Closing this issue as the requested additional details have not been provided and we have been unable to reproduce it.
BTW this is a canned response and may have info or details that do not directly apply to this particular issue. While we'd like to spend the time to uniquely address every incoming issue, we get a lot traffic on the EF projects and that is not practical. To ensure we maximize the time we have to work on fixing bugs, implementing new features, etc. we use canned responses for common triage decisions.

1.1.0-preview4-final
PM> add-migration comm2 -Verbose
Build started...
Build succeeded.
Setting app base path c:\Users\dima\Source\Repos\CET\src\CET\bin\Debug\netcoreapp1.0
Finding DbContext classes...
Using context 'ApplicationDbContext'.
Writing migration to 'c:\Users\dima\Source\Repos\CET\src\CET\Migrations\20170131193152_comm2.cs'.
Writing model snapshot to 'c:\Users\dima\Source\Repos\CET\src\CET\Migrations\ApplicationDbContextModelSnapshot.cs'.
To undo this action, use Remove-Migration.
PM> update-database -Verbose
Build started...
Build succeeded.
Setting app base path c:\Users\dima\Source\Repos\CET\src\CET\bin\Debug\netcoreapp1.0
Finding DbContext classes...
Using context 'ApplicationDbContext'.
Done.
Build started...
Build succeeded.
Setting app base path c:\Users\dima\Source\Repos\CET\src\CET\bin\Debug\netcoreapp1.0
Finding DbContext classes...
Using context 'ApplicationDbContext'.
Done.
and nothing happened.  Just added new field to one of classes
Was
`    public class StudyHistory
{
public int Id { get; set; }
    public Person Student { get; set; }

    public Person Teacher { get; set; }

    public Level Level { get; set; }

    public Term Term { get; set; }
}`

become
`    public class StudyHistory
{
public int Id { get; set; }
    public Person Student { get; set; }

    public Person Teacher { get; set; }

    public Level Level { get; set; }

    public Term Term { get; set; }

    public string Comment { get; set; }
}`

add-migration create empty migration.
ef dotnet migrations add
creates normal migration with new field
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn(
name: "Comment",
table: "StudyHistories",
nullable: true);
}

@dvdobrovolskiy as your intention to reopen the issue to consider the additional information?

It is an issue for me.

@dvdobrovolskiy can you provide the project so that we can debug the issue?

Sure
https://1drv.ms/u/s!AvdtTGXIWO4bicV9o8lUgutbBX57aw
just added new field to StudyHistory and
PM> add-migration comment2
gives me empty migration
dotnet ef migrations add comment3
from cmd executed straight after generates normal migration with new field

After upgrading your project to the RTM version of the tooling, I'm unable to reproduce this issue.


Fix some typos and terminology

Just typos really, and not referring to a thread group as a 'set' but rather a group.

I pushed a v0.55 update with other addition, but your fixes should be in it.


Flux-related Changes

These changes relate to issues that were encountered during the development of the JSON schema exporter or in support of Flux.  They represent the element definitions that were used to generate the schemas and ES6 classes that Flux is currently using.
In summary:

Updated "example" config (inconsequential to Flux/SHR)
Changed Entry.EntryType cardinality from 1..* to 1..1

JSON was extremely verbose w/ whole type hierarchy encoded, and the type hierarchy can be easily inferred from the most specific type.

Fixed RouteIntoBody/ExposureRoute mixup in IntravenousDrugUse and NicotineUse
Fixed incorrect paths in Ratio

Numerator and Denominator paths were missing the Quantity subpath in their constraints

Trimmed lots of trailing whitespace (side-effect of editing files)

Since these changes reflect the current state of flux, it's best to merge them into master.  If other changes are necessary, they can be made in following pull-requests.

looks good to me


firefly3 python module

Hi, i've firefly3 running in dockers. I've tried your script but it can't find the firefly3 python module.
from firefly3 import Firefly3Client, Firefly3Transactions
where are you running this script from or where can I download this module? It's not in the requirements.txt
Ok problem is not the module but syntax error on firefly3.py line 162, there is an empty IF
Thank you in advance

Ok thank you.


fix 69:when simulate 7 in ie11 console, will not be detected as ie

https://github.com/3rd-Eden/useragent/issues/69

Thanks for your contribution


MultiGPU Traning

I have 4 gpus and I tired to use your code for traning and I'm getting the following errors
Caused by op 'BackupVariables/char_embeddings/BackupVariables/cond/read/Switch_BackupVariables/char_embeddings_0', defined at:   File "worker.py", line 31, in <module>     model = cm.CorefModel(config)   File "coref-ee/coref_model.py", line 92, in __init__     for var in self.trainable_variables]   File "coref-ee/coref_model.py", line 92, in <listcomp>     for var in self.trainable_variables]
would you please update point me on how exactly to run the code using multi gpus as the current instruction doesn't reflect that
Thanks

Multi GPU training doesn't work currently, because of the ExponentialMovingAverage. If you want you can remove all references to it at the cost of slightly reduced performance and training stability.


Training README datasets table: text column and id column

To pseudo-label the three open-source datasets I had to re-order the table data in the Text Column and ID Column.
| Dataset                                                                                | Languages | Domain                                | Speaking Style | License   | Text Column  | ID Column    |
|-----------------------------------------------------------------------------------------------|-----------|---------------------------------------|----------------|-----------|--------------|--------------|
| [Multilingual LibriSpeech](https://huggingface.co/datasets/facebook/multilingual_librispeech) | 6         | Audiobooks                            | Narrated       | CC-BY-4.0 | `"text"`     | `"id"`       |
| [Common Voice 13](https://huggingface.co/datasets/mozilla-foundation/common_voice_13_0)       | 108       | Wikipedia text & crowd-sourced speech | Narrated       | CC0-1.0   | `"sentence"` | `"path"`     |
| [VoxPopuli](https://huggingface.co/datasets/facebook/voxpopuli)                               | 15        | European Parliament recordings        | Spontaneous    | CC0       | `"raw_text"` | `"audio_id"` |

For VoxPopuli I found "raw_text" to contain empty strings and think that is why the psuedo-labelling script to fail after hours of compute with a ValueError("one or more references are empty strings").

Copied from #98
When pseudo-labelling the Voxpopuli dataset the "raw_text" (needed for option  --text_column_name) may be an empty string for some examples - see HF dataset model card here for an empty "raw_text" example.
Question: how do I check which text name ("raw_text" or "normalized_text") was used when creating the pseudo-labelled datasets on HF, such as https://huggingface.co/datasets/distil-whisper/voxpopuli ?

Hey @guynich - the provided transcriptions in the original VoxPopuli dataset are only used for computing the WER in the pseudo-labelling and distillation scripts. Since the WER is computed on normalised transcriptions, you can safely use the "normalized_text" column in the dataset, which is what was done for the Distil-Whisper datasets.

Thank you for the helpful comment and for the fix #102.  Closing.


Basic game components

By the way it would be a good idea to pull main into your branch and move game_components into the server folder.

didnt mean to close and reopen my bad. Made the changes suggested and added some basic other things.


there is any possibility to use our server images i want to access our server image so please help where i need to set configurations in the react js

:hand: We are a small team with limited resources. Your question is much more
likely to be answered if it is
a good question

Description
Questions can often be answered by our documentation. Unable to find an answer
in our docs? We'll try to help. In the meantime, if you answer your own
question, please respond with the answer here so that others may benefit as
well. Better yet, open a PR to expand our docs ^_^

@shubhamvishwakarma19, what have you tried? Where are you getting stuck? Have you read through our readme and documentation? ( https://docs.ohif.org )

I have clone that repo https://github.com/OHIF/Viewers/tree/master/platform/viewer
i want to integrate the our dicom images . where i need to set configuration i had change in the some files default.js and other files getting some issue so please guide me where i can put my all dicom image  url which i want to show.

Are you able to see images from the default test server? If so, you have a few options:

Use /local
Mimic the DICOMWeb endpoints
Use the Standalone JSON query parameter
Help add support for something more robust

I'm on my phone so I can't provide any links, but you should be able to find similar issues with each of these solutions detailed.

Are you able to see images from the default test server? If so, you have a few options:
* Use `/local`

* Mimic the DICOMWeb endpoints

* Use the Standalone JSON query parameter

* Help add support for something more robust

I'm on my phone so I can't provide any links, but you should be able to find similar issues with each of these solutions detailed.

Hi danny,
I can get localy files using http://localhost:3000/local, but is it possible
when run http://localhost:3000 to load directly local dicomm images without using url above and
Drag and Drop DICOM files.
I would like to load local diccom file directly as list in application, i guess that need to modify this a part of code, but not so skilled to fit this
dicomWeb: [
{
name: 'DCM4CHEE',
wadoUriRoot: 'https://server.dcmjs.org/dcm4chee-arc/aets/DCM4CHEE/wado',
qidoRoot: 'https://server.dcmjs.org/dcm4chee-arc/aets/DCM4CHEE/rs',
wadoRoot: 'https://server.dcmjs.org/dcm4chee-arc/aets/DCM4CHEE/rs',
qidoSupportsIncludeField: true,
imageRendering: 'wadors',
thumbnailRendering: 'wadors',
enableStudyLazyLoad: true,
Can y help me, what exactly need to modify and point on my local dicom folder.
Thansk in advance

@nenadfemic, I have responded to your other issues. Please do not ask new questions on old/stale issues.


NFT ids

Currently unique NFT ids are generated from the index of the vote in getTxsForVotes. That obviously doesn't work when minting to an already existing collection. We need to think about a unique id for collection NFTs of type u32.
E.g. ${refId}${1-n for rarity index}{voteIndex}
or query nfts.collection to get the number of items and start from numItems + 1

Ideas?

querying the nfts.collection is not an option as there might be referenda in the queue that assign certain IDs.
Technically also there could be multiple nft sendouts for the same ref. It's important that we consider that too.

New collections i will build, the ui is already ready in a way. still i think when we as poc want to start we should mint to a default collection (and maybe encourage others). I imagine having 10s of rewards collections with only 3 different rarities will clutter all NFT marketplaces UIs like crazy and make it even harder for people to follow / organize. I liked how it was done before.


Reduce temp vars

One-time variables in a return statement reduce readability and can take up system resources. This PR attempts to remove some temporary variables from the API
In some cases, the reduction of temporary variables has led to the reduction of nesting, which makes the code a little easier to follow. When reducing nesting, I have favored returning early.
I have attempted to make the commits atomic so that if a type of change is undesired, it can be excised from this PR.

Can one of the admins verify this patch?

ok to test

Build finished.

Build finished.