Add HiPhase workflow for phasing, stats, and haplotagging.

Some design notes:

HiPhase generates phased VCFs, phasing statistics, and haplotagged BAMs in a single step, so this workflow replaces multiple tasks.
HiPhase phases small variants and structural variants simultaneously (with TRGT planned in the future), so I made the vcf input an array, vcfs.  The plan is to pack the input VCFs and unpack the output VCFs in the same order.
Removed some of the inputs relative to WhatsHap:

regions - HiPhase has built-in parallelization
reference_chromosome_lengths - unneeded

Added some new inputs:

id is the sample_id or cohort_id to be used in the filename.  Since there are multiple input VCFs, I didn't want to base the output filename on input file basenames.
refname will be used in output filenames.
sample_ids is an Array of sample names to phase.
haplotag is a Boolean indicating whether we want haplotagged BAMs as output.

Output stats are a little different from WhatsHap:

hiphase_stats is the equivalent of whatshap_stats_tsv
hiphase_blocks is the equivalent of whatshap_stats_blocklist, except with a tsv extension.
whatshap_stats_gtf has no equivalent in HiPhase
hiphase_haplotags is a list of per-read haplotag assignments that could be used in other tasks

The haplotagged BAMs are merged and indexed at the end of the task.  The outputs are captured by glob, to handle haplotag = false.


Update SingleR celltyping

Closes #441
I have tested the celltyping workflow using label.main for SingleR (including "model training"), and found 2 quick things that needed to be fixed! Everything is working smoothly now, all tested.

First, this wasn't a bug, but it seemed like the right move. We previously made a parameter called label_name for the singler reference label, but this was before we added cellassign. So, I just changed this workflow parameter to be named singler_label_name for clarity. I left all R scripts alone.
Second, this was a bug! The QC report delta median plot x-axis did not the right levels in an ontology-free scenario, so I added levels and now the plot looks as expected.

Report: qc_report.html.zip

I fixed the crossbar-not-showing-IQR situation by changing it to a boxplot without whiskers/outliers! I did this because crossbar docs were surprisingly cryptic about what they are even showing by default, but I feel very confident in what a boxplot displays! Here's a quick screenshot of that part of the rendered report:


Problem trying to use with webpack

Hi,
I'm trying to use webpack to create a bundle that includes eslint4b to run in a WebWorker.
I have the following webpack.config.js:
{
    entry: path.resolve(__dirname, 'lib/rules/function-eslint/index.js'),
    output: {
        filename: 'nrlint-plugin-rules-function-eslint.js',
        path: path.resolve(__dirname, 'resources'),
        library: {
            name: ['nrlint','rules'],
            type: 'assign-properties'
        }
    },
    resolve: {
        fallback: {
            "assert": path.resolve(__dirname, 'lib/rules/function-eslint/empty.js'),
            "util":   false,
            "path":   false
        }
    },
    mode: 'production',
    target: ['webworker']
}

When I run webpack, I get multiple errors of the sort:
Module not found: Error: Can't resolve '@eslint/eslintrc/conf/environments' in '/Users/nol/code/node-red/nrlint/node_modules/eslint4b/dist'

This happens for:

@eslint/eslintrc/conf/environments'
@eslint/eslintrc/lib/shared/config-ops
@eslint/eslintrc/lib/shared/config-validator

I have found I need to have the full eslint installed as a dependency of my own module for the webpack to work. Is that expected?

Since ESLint v7.8.0, some features have been extracted to @eslint/eslintrc, so I think we need to register it in dependencies in package.json.
In scripts/build.js, package id extraction logic does not deal with scoped modules (modules starting with @). This may be why  @eslint/eslintrc is not automatically registered in the dependencies.

Maybe a PR is needed?


Extend primitive conversions and implement TryFrom trait for integer values

Description
There have been a number of requests to convert between integer values. This is especially important in some of the recently introduced standards and changes made in the storage optimizations.
This PR extends the primitive conversions to extend values and also implements the TryFrom trait for u64, u32, u16, and u8.
Note: The TryFrom implementations are not included in the prelude and must be imported. The primitive conversions do not need to be imported as they are included in the core.
Checklist

[ ] I have linked to any relevant issues.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have updated the documentation where relevant (API docs, the reference, and the Sway book).
[x] I have added tests that prove my fix is effective or that my feature works.
[x] I have added (or requested a maintainer to add) the necessary Breaking* or New Feature labels where relevant.
[x] I have done my best to ensure that my PR adheres to the Fuel Labs Code Review Standards.
[x] I have requested a review from the relevant team or maintainers.

Why do we have try_as_uX methods and some TryFrom impls? Can we not only use the latter?

These exist prior to the PR in the primitive_conversion.sw file. They will be replaced with TryInto once the trait exists.


Re-export zbus for convenience

A bit of  janitorial and quality of life:
Removes export of no longer existing atspi-client
Re-exports zbus for convenience.

Especially if you know (almost) all users will need it anyway its a very simple quality of life thing.
Just add atspi and get zbus for free (if you have the feature set).
Added advantage comes when experimenting and you want to try a different or patched zbus, with this in, you only need to change atspi's dependency and not worry about doing this in both places.


fix: avoid writing statistics for binary columns to fix JSON error

Description
Avoid writing statistics for binary columns to fix JSON error thrown by Arrow
Related Issue(s)

closes #1493

Unfortunately you can't do this with stats conversion, the error occurs when making the ReaderBuilder, not when applying the values.

We have a new failure for decimal, but I'm a little stumped on it for now.

I think for decimal, we might be able to start using the arbitrary_precision feature of serde_json. Not sure how nicely it plays with decimals yet, but it seems to be the suggested route.

@rtyler - i fear it may have been a bit too early to merge this., since the rust test are failing for this. since 0.13 was just released, we should look into releasing a fix asap. bot sure if @wjones127 already knows whats going on? Otherwise I might look into it once available again.

@roeap Yes, I'm looking at this now. I must have been moving too quickly through some PRs. I now see that the test was failing but I'm not sure why I merged this :facepalm:  I must have mistaken this pull request for another one which I had open at the time.
I am going to land a revert into main directly, and cut a 0.13.1 with that revert just to avoid causing anybody problems


feat: Enable Multi-Instance GPU Support for Mariner AKS

What type of PR is this?

What this PR does / why we need it:
This PR adds support for MIG mode for Mariner AKS. During testing, two problems are captured:

nvidia-smi takes longer than the timeout value set in the code to return success on A100 SKU. Based on current investigation, the issue seems to be caused by the NVIDIA GPU driver and thus, enabling persistence mode as a workaround which will accelerate nvidia-smi operations later on.

mig-partition.service takes longer to complete on Mariner than Ubuntu. Increase the timeout value both in the service file and systemctl restart call to make sure that this operation can always be successful on Mariner

Which issue(s) this PR fixes:

Fixes #
Requirements:

[ ] uses conventional commit messages

[ ] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall First build on lihl/mariner-mig-support at 60.585%

Totals

Change from base Build<PHONE_NUMBER>:
60.6%

Covered Lines:
2570

Relevant Lines:
4242

💛  - Coveralls


Layout breaks on iOS/Safari

Some things are not rendered correclty on iOS/Safari. Have this issue here to not forget about it since I'm not developing on a mac

Add Point button doesn't layout correctly
Select element is not styled nicely
Can't select routes on the map with current event listener setup
Elevation widget doesn't work on small screens (it is too big and needs to be layout differently)
Map Style Switch is not visibile on small screens

This was fixed in #74


Connections leaking and actively sending commands on 0.12.0

Despite the multiple improvements to prevent connection leakage, I have observed a sharp spike in leaked connections moving to 0.12.0 from 0.11.2 (the migration is required by our move to Spark 2.x).
Here is what happened after the upgrade.

The steps happen when we run a scheduled Spark job that pulls 15+ collections from Mongo. The drops are cluster restarts for library upgrades.
Zooming out, here what the situation looked like under 0.11.2 before the upgrade to 0.12.0.

Since 0.12.0 is the only release compatible with Spark 2.x, there is nothing to downgrade to.

Interestingly, calling MongodbClientFactory.closeAll() after the job is done only removes 50% of the leaked connections.


Add option to pass size numOccurrences parameter to FromIntegerTypeConverter.getAppropriate(Volatile)Type

This is useful for creating multi-set types with different counts, e.g. for downsampled constant pixels.

See https://github.com/saalfeldlab/paintera/commit/bf9353b59dfd322a1eaabcdcc368e2ccea41359e for an example use case

@hanslovsky Would it make sense to deprecate or remove these methods getAppropriateType() and getAppropriateVolatileType() and just use the constructor directly? I think they used to have low-level initialization code, but a while ago it was moved into the constructors of LabelMultisetType.

That would make sense but for convenience we should then probably add a constructor LabelMultisetType( final int numOccurrences ) that would call LabelMultisetType( LabelMultisetEntry ) and initialize the entry to Label.INVALID. What do you think?

For LabelMultisetType I'm afraid that a constructor with a single int may give the wrong idea that it's the number of entries (and not the number of occurrences in a single entry). How about adding another constructor to the entry itself such as LabelMultisetEntry(final int numOccurrences)?

Then maybe a static method in the LabelMultisetType could be useful, something like this:
LabelMultisetType.singleEntryWithOccurences

or similar.

351cdd3 shows what it could look like. Naming is not necessarily final. What do you think about that?

Thanks @igorpisarev I will go ahead and merge.


Azure AD Integration

autoconfigure for AAD integration jwt token filter

@yaweiw,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Codecov Report

Merging #97 into master will increase coverage by 0.59%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #97      +/-   ##
==========================================
+ Coverage   52.06%   52.65%   +0.59%
==========================================
  Files          35       35
  Lines         557      564       +7
  Branches       47       50       +3
==========================================
+ Hits          290      297       +7
+ Misses        244      243       -1
- Partials       23       24       +1

Impacted Files
Coverage Δ

.../mediaservices/MediaServicesAutoConfiguration.java
70.96% <0%> (+8.46%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 86aa124...a56832d. Read the comment docs.


--dry-run creates folders

Hi,
Found some odd behavior regarding folders with Google drive:
rclone version: v1.27
system: Ubuntu 14.04.3 LTS x86_64
Issue 1: --dry-run created a folder
The command rclone --dry-run sync TestFolder/ gdrive:TestFolder
created an empty TestFolder on my Google Drive. Here is the Drive Activity log:
You created an item in
rclone • 4:49 PM
Google Drive Folder
My Drive
Google Drive Folder
TestFolder

But since this was a --dry-run, I expected only a report that TestFolder would be created.
Issue 2: empty folders not deleted after sync
After properly creating TestFolder and its hierarchy on Drive with
rclone sync TestFolder/ gdrive:TestFolder

my Drive Activity log shows creation of files and the subfolder test folder 2 (as expected):
You created an item in
rclone • 5:02 PM
Google Drive Folder
TestFolder
Text
test.txt

You created an item in
rclone • 5:02 PM
Google Drive Folder
test folder 2
Text
test2.txt

You created an item in
rclone • 5:02 PM
Google Drive Folder
TestFolder
Google Drive Folder
test folder 2

But then I delete the contents of TestFolder on my local system:
cd TestFolder; rm -rf *

and resync with
rclone sync TestFolder/ gdrive:TestFolder

I belive this should leave an empty TestFolder on my Drive, but the Activity log shows only:
You permanently deleted 2 items
rclone • 5:11 PM
Text
test2.txt
Text
test.txt

and leaves behind the folder test folder 2 even though it was deleted locally.
If if do: rclone -v ls gdrive:TestFolder I get:
2016/02/06 17:26:39 Google drive root 'TestFolder': Modify window is 1ms
2016/02/06 17:26:39 Google drive root 'TestFolder': Reading test folder 2/
2016/02/06 17:26:39 Google drive root 'TestFolder': Finished reading test folder 2/
2016/02/06 17:26:39 Google drive root 'TestFolder': Finished reading

so rclone is seeing the left-behind empty folder test folder 2; this is verified also with lsd:
rclone lsd gdrive:TestFolder
-1 2016-02-06 22:02:16        -1 test folder 2

But I expected rclone to delete the empty folder test folder 2
I encountered this while syncing a number of local folders after some local changes.
Let me know if I am missing something...
But thanks for a great tool; better than other options!

Issue 1 is a bug - thanks for pointing that out.
Issue 2 is a known problem with rclone. Rclone doesn't really understand directories - see #100
I may do a stop gap measure to get rclone to delete empty directories which would fix most of the problems.

Thanks for the quick follow up!
Sorry I missed the earlier discussion on Issue 2, I'm a new user. A 'delete empty directories' step at the end of sync would be most welcome.

This also happens with swift so is likely a missing check in operations.go.

The fix for creating directories on --dry-run will be in the next release - thanks for reporting it.
Nick

@ncw Unfortunately, this is reproduced on 1.44. When running move with --dry-run, empty folders are created in Google Drive.

@orange-buffalo can you please make a new issue on github about this with a sequence of commands to reproduce please - thanks :-)

@ncw, sure thing, created #2676. Thanks for your fast response!


Added support for tag options and field to ignore when Marshalling a struct

You can now use the following syntax in a tag struct :
type myStruct struct {
// Ignored during Marshalling
Field string `bencode:"-"`
// Ignored during Marshalling if empty otherwise named  "name"
Field2 string `bencode:"name,omitempty`
// Ignored during Marshalling if empty otherwise named with default field.Name in this case "Field3"
Field3 string `bencode:",omitempty`
}

In this version, the old tag style is no longer supported.

Nice, except why break backwards compatibility by making the old un-decorated tag style?

Updated it. The old tag is now supported again.
You should note though that there is a slight difference with your version. Before if you had the following :
type foo struct {
A string `json:"a"`
}

The bencode marshaller wouldn't marshall the A field. In this new version it does.
It doesn't have to be this way, I just thought it made more sense, why not allow bencode marshaller to parse something that has been tagged for another encoder ? If you don't want the bencode marshaller to parse it, you can use the bencode:"-" tag.
What's your take on that ?

I am  willing to try it the way you wrote it. (I probably would have written it the other way.)


[MM-34765][MM-34824][MM-34995][MM-34998] - Minor UX fixes

Summary
This PR addresses several minor UX fixes to borders and hard coded values.
Ticket Link
https://mattermost.atlassian.net/browse/MM-34765
https://mattermost.atlassian.net/browse/MM-34824
https://mattermost.atlassian.net/browse/MM-34995
https://mattermost.atlassian.net/browse/MM-34998
Added new API endpoints POST /api/v4/foo, GET api/v4/foo, and GET api/v4/foo/:foo_id.

Added a new config setting ServiceSettings.FooBar. Added a new column Foo to the Users table.

Release version is listed as 'Queued' not sure if this can make the next cloud release? Maybe @amyblais can advice.

@matthewbirtch fixed

sorry @hmhealey I wasn't sure about the other places..


feat: Implement Request Throttling for User Accounts

Is your feature request related to a problem? Please describe.
Currently, there isn't any way to set limits on the number of requests that can be made to the large language models (LLMs) within a given time period for each user account. This could lead to abuse and unfair usage of the LLMs.
Describe the solution you'd like
I would like Open WebUI to have the ability to set limits on the number of requests that can be made to the LLMs within a given time period for each user account. This feature would help prevent abuse and ensure fair usage of the LLMs. The feature should allow admins to set custom throttle rates for each user account, depending on their needs. When a user reaches their throttle limit, a message should be displayed indicating that they have exceeded their request limit and to try again later, possibly indicating the time in which they can try again.
Describe alternatives you've considered
I have considered manually monitoring and limiting user requests, but this would not provide a scalable or automated solution and would require manual intervention.
Additional context
I propose implementing request throttling for user accounts to prevent abuse and ensure fair usage of the LLMs. Admins should be able to set the values of X, Y, Z, and T through the WebUI for request throttling per user. Additionally, it might be helpful to provide default values or recommended values based on best practices or common usage patterns.
Here are some suggested rate limit rules for the request throttling feature:

The rate limit is set to X requests per minute and Y requests per hour, operating simultaneously. Whichever limit is reached first triggers the rate limit: if you make more than X requests in any single minute, the per-minute limit applies; if you spread out the requests but exceed Y within an hour without breaching the per-minute cap, then the hourly limit comes into effect.
Alternatively, a sliding window rate limit could be set, where the limit is reset after a certain period of time. For example, you could set a limit of Z requests per sliding window of T minutes. This means that if a user makes Z requests within a T-minute period, they will be rate limited until T minutes have passed since their first request.

These are just a few examples, and the actual rate limit rules will depend on your specific use case and the resources available. The key is to find a balance between preventing abuse and ensuring fair usage, while still allowing users to make sufficient requests to accomplish their tasks.

For some free sites, rate limiting can prevent abuse.

This has also been an issue that has troubled me for a while; how wonderful it would be to add rate limiting for users. The current user management features are still too limited.

@tjbck Do you have any plans for future development?

The rate limit should also support a token limit and a USD limit. Limiting by requests per time is not viable, as a single long request could be a few dollars, while thousands of small requests might not be a single dollar.

hey team, isn't this already supported via litellm - https://docs.litellm.ai/docs/proxy/users
Curious what we're missing to be useful here? @NightMachinery @silentoplayz @OAIBOT @silentoplayz

Removed comment to maintain focus on relevant discussions to Open WebUI.

@tjbck curious why it's unrelated? Since open webui supports litellm proxy which is built to handle this scenario

@krrishdholakia We have decided that for our future releases (0.2.0), it's best to allow our users to host LiteLLM externally to our web UI, to fully benefit from everything LiteLLM offers. This way, just like in this case, they can leverage the LiteLLM admin UI. However, this feature request is specifically for Open WebUI. This ensures that if users prefer to only host Open WebUI, they can still have this functionality.


Make override_switch usable with coroutines

I've encountered a situation when I needed a wrapper for coroutines. I had to use it in async tests. That's why I create a PR to add the functionality to main repo.
A wrapper for coroutines has to be different than for normal functions.

I don't believe this solution worked anyway - it does not address the fact that the decorators need to be explicitly changed to support async. (ie, you can't just do a .get() in the async function call).


Add AppPlatform new API schema for new pricing tier

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Reviewed swaggger change: https://github.com/Azure/azure-rest-api-specs-pr/pull/4120
Changelog
Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?

[ ] new service onboarding
[x] new API version
[ ] update existing version for new feature
[ ] update existing version to fix swagger quality issue in s360
[ ] Other, please clarify

When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific langauge SDKs and CLIs that must be refreshed after the swagger is published.

[ ] SDK of .NET (need service team to ensure code readiness)
[ ] SDK of Python
[ ] SDK of Java
[ ] SDK of Js
[ ] SDK of Go
[ ] PowerShell
[ ] CLI
[ ] Terraform
[ ] No refresh required for updates in this PR

Contribution checklist:

[ ] I commit to follow the Breaking Change Policy of "no breaking changes"
[x] I have reviewed the documentation for the workflow.
[x] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

Applicability: :warning:
If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs
Adding new properties
All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

[ ] Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.

Adding a new service
Adding new API(s)
Adding a new API version
-[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits.

[ ] Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in a stable version
[ ] Removing properties in a stable version
[ ] Removing API version(s) in a stable version
[ ] Updating API in a stable or public preview version with Breaking Change Validation errors
[ ] Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
Please follow the link to find more details on PR review process.

/azp run

/azp run

Hi @yuwzho, are these breaking changes expected:

@msyyc Yes, they are. We already got the breaking change approval and ARM signed off in the private repo. You can check the linked PR in description.

@yuwzho , please ping me when you get chance coz I have some questions on the associated PR of private repo.

Closed this as it's replaced by new PR.


feat(SPV-1038): decouple sync job from SyncTx

SyncTransaction model is no longer "used" (I left the model for now because that would delete this table from database)
The column tx_status has new values (which basically replaces SyncTransaction model purpose)
The main changes were in strategies (outgoing, internal/external incoming) and for SYNC task.

Pull Request Checklist

[ ] 📖 I created my PR using provided  : CODE_STANDARDS
[ ] 📖 I have read the short Code of Conduct: CODE_OF_CONDUCT
[ ] 🏠 I tested my changes locally.
[ ] ✅ I have provided tests for my changes.
[ ] 📝 I have used conventional commits.
[ ] 📗 I have updated any related documentation.
[ ] 💾 PR was issued based on the Github or Jira issue.

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 22.32143% with 174 lines in your changes missing coverage. Please review.

Project coverage is 46.19%. Comparing base (dd9c1d1) to head (6bb40ef).

Files with missing lines
Patch %
Lines

engine/tx_sync_task.go
0.00%
71 Missing :warning:

engine/record_tx_strategy_outgoing_tx.go
29.26%
22 Missing and 7 partials :warning:

engine/chainstate/transaction.go
44.82%
11 Missing and 5 partials :warning:

engine/action_transaction.go
13.33%
12 Missing and 1 partial :warning:

engine/model_transactions.go
8.33%
11 Missing :warning:

engine/tx_broadcast.go
73.91%
4 Missing and 2 partials :warning:

engine/record_tx_strategy_internal_incoming_tx.go
0.00%
5 Missing :warning:

models/errors.go
0.00%
5 Missing :warning:

actions/transactions/broadcast_callback.go
0.00%
4 Missing :warning:

engine/record_tx_strategy_external_incoming_tx.go
0.00%
4 Missing :warning:

... and 7 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #701      +/-   ##
==========================================
- Coverage   46.44%   46.19%   -0.26%
==========================================
  Files         297      296       -1
  Lines       13042    12926     -116
==========================================
- Hits         6058     5971      -87
+ Misses       6390     6372      -18
+ Partials      594      583      -11

Flag
Coverage Δ

unittests
46.19% <22.32%> (-0.26%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines
Coverage Δ

engine/chainstate/chainstate.go
54.28% <100.00%> (+1.65%)
:arrow_up:

engine/cron_job_declarations.go
53.12% <100.00%> (ø)

engine/model_bump.go
56.84% <100.00%> (+0.45%)
:arrow_up:

engine/model_sync_transactions.go
41.17% <ø> (-54.07%)
:arrow_down:

engine/tx_service.go
67.81% <ø> (+5.71%)
:arrow_up:

engine/db_model_transactions.go
59.80% <0.00%> (-0.78%)
:arrow_down:

engine/examples/client/custom_cron/custom_cron.go
0.00% <0.00%> (ø)

engine/paymail_service_provider.go
27.64% <0.00%> (+0.86%)
:arrow_up:

mappings/transaction.go
0.00% <0.00%> (ø)

engine/cron_job_definitions.go
0.00% <0.00%> (ø)

... and 12 more

... and 6 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd9c1d1...6bb40ef. Read the comment docs.

Looks good to me, all paths that I know seems covered.


Improve text-terminal output for check/shaping/collides

It is currently not very user friendly because it prints raw SVG markup. What can we do to make it better?

(issue originally found at #3554)

If you want to get fancy, check what hb-view does to render graphical output in the terminal.


Error in captions

So, for the interactive figures, an error pops up in the captions. Namely, the caption points you to the condition report for more info, but the interactive figures aren't in the condition report. Take a look at the red box in the image below.

What we need is to remove this pointer to the condition report for interactive figures, meaning that the function nms4r::md_caption needs to be modified.

Caption issue fixed. See red box at the bottom of the image below.


XXX: Don't merge

@tisnik Your image is available in the registry: docker pull registry.devshift.net/MISSING_IMAGE_NAME:SNAPSHOT-PR-14


SCRD-3497 Use install os provision playbook

Replace call to cobbler deploy with a call to the
install-ui-os-provision playbook, which is also used by the install
wizard.
Support defaulting and passing extra-vars in both install and update
scenarios.

Note that the first two commits in this PR actually belong to PR #201 (stupid github)


At least 1 Modal

As a user
I want to click some element on the page
To have a well styled modal pop-up with some extra content on it.

There are some new error messages for no search term entered, and for no matches found. These could both be replaced with modals is wanted. I've put a comment next to each of the JS sections that handles them with the word 'modal' in them for control + f ease of finding


[Bug] Cannot create chatmessages in macros with dnd5e helpers enabled

Version 3.0.3. I was trying to help someone out with macro-ing some lingering injuries reminders, and suddenly ran into some weird errors from ActionManagement.js at the _preCreateChatMessage hook. Just so you don't have to dig through the code of the macro, it rolls from a table, gets a result and creates an ActiveEffect with the result as a flag, then outputs the result as well in a ChatMessage. The latter fails with DnD5e helpers enabled, succeeds when disabled.
I've attached the macro here.
shouldhavebeenajson.txt
i.
Error here, the syntax is the table roll result in text.

oof, that's no good. Sorry for the issues. I'll try to get a fix by this weekend.

This may have been implicitly fixed by the work done for #212. If this issue persists in 3.0.4, please reopen this issue.


Update and rename leo.localplayer.dev.json to *.leo.localplayer.dev.json

i'd like to use more addresses from this subdomain. i want to use for my email alias in simple login and nginx proxy manager and also to get for  tls/ssl certificate for my my cloud vm services.

Requirements

[x] You have completed your website.
[x] The website is reachable.
[x] The CNAME record doesn't contain https:// or /.
[x] There is sufficient information at the owner field.
[x] There is no NS Records (Enforced as of Sepetember 4th, 2024)

Description

i'd like to use more addresses from this subdomain. i want to use for my email alias in simple login and nginx proxy manager and also to get for  tls/ssl certificate for my my cloud vm services.
Link to Website

lenadlm.github.io

tls/ssl certificate for my my cloud vm services like adguard and cockpit


⚠ Avatar API Endpoint has degraded performance

In 3711820, Avatar API Endpoint (https://avatar.roblox.com/v1/avatar-rules) experienced degraded performance:

HTTP code: 200
Response time: 3058 ms

Resolved: Avatar API Endpoint performance has improved in a338c03 after 19 minutes.


fix: use comprehend to detect dominant language before firing bang event

Fixes https://github.com/rickklaasboer/wego-overseer/issues/379

:tada: This PR is included in version 3.5.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update-PSResource fails with required nuget version

Prerequisites

[X] Write a descriptive title.
[X] Make sure you are able to repro it on the latest released version
[X] Search the existing issues.

Steps to reproduce
If you try and Update-PSResource using a nuget version syntax it fails, Install-PSResource works. This workflow worked in beta22 so something has changed to break it.
I found this while running CI for my module: CI run
PS C:\Users\Thomas> Install-PSResource -Name NetworkingDsc -Version '[8.0.0]' -PassThru

Name          Version Prerelease Repository Description
----          ------- ---------- ---------- -----------
NetworkingDsc 8.0.0              PSGallery  DSC resources for configuring settings related to networking.

PS C:\Users\Thomas> Update-PSResource -Name NetworkingDsc -Version '[8.1.0]' -PassThru

Name          Version Prerelease Repository Description
----          ------- ---------- ---------- -----------
NetworkingDsc 8.1.0              PSGallery  DSC resources for configuring settings related to networking.

PS C:\Users\Thomas> Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]' -PassThru
Update-PSResource: Package 'NetworkingDsc' with version '[8.2.0-preview0001]' could not be found
WARNING: Installed package 'NetworkingDsc':'[8.2.0-preview0001]' was not found in repositories and cannot be updated.

PS C:\Users\Thomas> Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]' -PassThru -Prerelease
Update-PSResource: Package 'NetworkingDsc' with version '[8.2.0-preview0001]' could not be found
WARNING: Installed package 'NetworkingDsc':'[8.2.0-preview0001]' was not found in repositories and cannot be updated.

Expected behavior
# works

Actual behavior
Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]' -PassThru -Prerelease
Update-PSResource: Package 'NetworkingDsc' with version '[8.2.0-preview0001]' could not be found
WARNING: Installed package 'NetworkingDsc':'[8.2.0-preview0001]' was not found in repositories and cannot be updated.

Error details
Exception             :
    Type           : Microsoft.PowerShell.PSResourceGet.UtilClasses.PackageNotFoundException
    Message        : Package 'NetworkingDsc' with version '[8.2.0-preview0001]' could not be found
    InnerException :
        Type    : Microsoft.PowerShell.PSResourceGet.UtilClasses.V2ResourceNotFoundException
        Message : Package does not exist on the server
        HResult : -2146233088
    HResult        : -2146233088
TargetObject          : Microsoft.PowerShell.PSResourceGet.Cmdlets.FindHelper
CategoryInfo          : InvalidResult: (Microsoft.PowerShel….Cmdlets.FindHelper:FindHelper) [Update-PSResource],
PackageNotFoundException
FullyQualifiedErrorId : FindNameConvertToPSResourceFailure,Microsoft.PowerShell.PSResourceGet.Cmdlets.UpdatePSResource
InvocationInfo        :
    MyCommand        : Update-PSResource
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 4
    Line             : Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]' -PassThru -Prerelease
    Statement        : Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]' -PassThru -Prerelease
    PositionMessage  : At line:1 char:1
                       + Update-PSResource -Name NetworkingDsc -Version '[8.2.0-preview0001]'  …
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : Update-PSResource
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo :

Environment data
beta23

Visuals
No response

@SydneyhSmith feel free to compare to my tests for anypackage/psresourceget to see if there are any gaps.

Just working on the bug fix for this we did have a test to check for this scenario.  Running the test locally it fails before the bug fix and is resolved after the bug fix.  I don't know how this test didn't fail in CI, but from what I'm looking at it both should always be a test that's run and should definitely have failed.

In that linked test I don't see where it is specifying the required version using the square bracket syntax. Am I missing something?

@ThomasNieto to backtrack a bit, I was looking at tests to see if we had anything to catch this bug and I saw the test I linked to above.  It's different than your original issue but fails because of the same underlying problem (the -Prerelease param not getting passed through).  So that's the context around that comment.  Ideally this test should have caught that regression but for some reason didn't (ie it's failing locally as it should and passing in CI even though it shouldn't).
The reason why the test linked to is similar to your issue is because they both treat the version for the package to be updated to as a version range, which follows a different code path than a "specific version" (ie "9.0.0" instead of "[9.0.0]").
I opened a dummy PR here, to try to figure out why CI is giving a false positive, in the hopes of properly fixing that test and having it act as a (properly working) test for the linked PR I opened, but I still can't quite figure out what's going on there.  So I'll just add a test to the PR with the bracket syntax.


Is it possible to proportionately shrink windows down like in Expose mode?

I think I've accidentally triggered it a couple years ago (no window management at that time) where one of my windows stayed "shrunken" after exiting Expose.  It got me thinking it would be pretty nifty for apps with bloated UI or when using as a reference.  That's what I thought the Picture in Picture mode would do at first.

In theory, yes. See #148 for related information on how this could be achieved.

You need to look at it from three angles:

Visuals: Definitely possible, see #148. I've played around enough with these functions from with the scripting addition to know that this is an easy task.
yabai: Definitely possible. You'd need to refactor yabai to use the windows transformed size and position instead of their actual size and position.
Resized applications: Impossible to do consistently. There's no way to know how applications respond to mouse events. Any application that manually maps coordinates from events to anything it renders will behave incorrectly.

Implemented a basic picture-in-picture mode on master that uses visual scaling instead of actually resizing the window: yabai -m window --toggle pip
I found myself using it in combination with other commands, making the following bind:
# toggle sticky(+float), topmost, border and picture-in-picture
alt - p : yabai -m window --toggle sticky;\
          yabai -m window --toggle topmost;\
          yabai -m window --toggle border;\
          yabai -m window --toggle pip

@koekeishiya @dominiklohmann Sorry for mentioning you in an old ticket and the very basic question: how is PIP supposed to look/work?
yabai -m window --toggle sticky;
yabai -m window --toggle topmost;
yabai -m window --toggle pip

Nothing In the error logs, in the debug ones simply:
EVENT_HANDLER_DAEMON_MESSAGE: window --toggle sticky
EVENT_HANDLER_DAEMON_MESSAGE: window --toggle topmost
EVENT_HANDLER_DAEMON_MESSAGE: window --toggle pip

The functionality I get is the same as a regular sticky + topmost. When the now floating window is resized, its content gets scaled regularly. Am I missing anything?

Before:

After:

Gotcha, thanks for confirming! 👍Is the PIP window manually resizable?
But yeah, here the window goes in float+sticky+topmost but it doesn't get scaled. I even restarted my MacBook and setup the accessibility permissions again, just in case. Is there any other way to troubleshoot this? Logs look fine here!

I don't think resizing a window works properly after the transformation has been applied. You can move it around just fine though.
This functionality relies on the scripting-addition to work, so if this does not work for any window on your machine (e.g Safari works just fine for me), you could try to reinstall the scripting addition and see if that helps.
I've been meaning to add proper exit codes for issues with the scripting addition (#181), but haven't gotten there yet.

@koekeishiya I finally had some time to play with this: I can confirm reinstalling the scripting addition gave me the PIP mode, thanks!


247 drawer vars

connects to #247

added drawer vars to harmonium-settings zipped file

jk this has already been done


different results even if seed is fixed

Depending on the environment(32bit or 64bit / SSE2, AVX or AVX2) in which tomotopy are installed, different results will be produced with the same seed.
It is possibly related to #60.

There are three possible causes.

Random Number Engine
https://github.com/bab2min/tomotopy/blob/400f06006f928db39ce2832830271ad0c87981b3/src/TopicModel/TopicModel.hpp#L13-L28

Prefix Sum
https://github.com/bab2min/tomotopy/blob/400f06006f928db39ce2832830271ad0c87981b3/src/Utils/sample.hpp#L123-L140

Eigen's redux function(sum())
The summation order can be varied by SIMD options.

(1) and (2) will can be fixed soon, but (3) seems not easy to fix.

@bab2min could you tell the timeline by which it will be finished?
even setting -
np.random.seed(123)
random.seed(123)
on same avx2 machine giving different results on a different run.

Hi, @mayankchatteron1
Since version 0.8.2, you can obtain the same result in the same machine when the seed is fixed with only workers=1.
If the result is different even though the seed is fixed on the same machine, please check that you set workers=1 first.
If you get different results even if workers=1, this is maybe another bug, so please report your source code.
Thank you!

@bab2min Thanks it working now. Thank you again for the quick resolution.
But curious to know why different workers giving different results, is it due to some kind of model parallelism?

@mayankchatteron1 Yes.
In a multithreaded environment, it is impossible to completely control the execution order by software.
In order to control it, each inference step needs to be synchronized, which reduces parallelization efficiency and makes it virtually no difference from running on a single thread.
To avoid confusion, I'll update package to print a warning message when seed is fixed and workers > 1.


General style cleanup (header, data catalog, applications)

Header

Right align "Request access" button
Enhance vertical padding
Minor enhancements to make the overall size and spacing more closely match other Microsoft websites

Data catalog

Right-align the last column in a row with the right side of the page
Move the tag filter to be closer to the content it filters
Visually distinguish tags from buttons
Minor spacing enhancements

Applications

Remove margin from headings so the main content aligns with the main content on the "Data catalog" page

Dataset Detail

Use headings instead of text for metadata
Remove padding on bottom of sticky header, which was causing a gap

Closing in favor of #224 which resolve commit history issues.


main: simplify the method signature

Please no.  My design is not broken, as I explained in https://github.com/adsharma/py2many/pull/256#discussion_r637569301 .  The C entry point is the common interface.  We will need to use proc cmain and #[start] and others when we implement more features, mark my words.  We will be implementing languages other than C++ which do require argc in their main signature.

A better way to achieve this is for each language to declare whether they need argc or argv, so the generic rewriter can use that info to generate the appropriate generic signature.  I could put up a PR to demo that if you're not 100% committed to your PR.

A better way to achieve this is for each language to declare whether they need argc or argv, so the generic rewriter can use that info to generate the appropriate generic signature. I could put up a PR to demo that if you're not 100% committed to your PR.

Yeah - this seems preferable. The implementation could be one PythonMainRewriter per language.
I'll put this on hold until I understand your idea.

This can be closed now ?

ping @adsharma

Covered by  other PRs


Importing Large Map Crashes Unreal Engine

CARLA version:0.9.12 built from source
Platform/OS: Ubuntu 18.04
After importing Large map into Carla, When I try to Simulate, it crashes with a "Caught signal 11 Segmentation fault" error. I'm able to import a small map and everything seems to work fine, but I'm facing this issue when I try with a larger tiled map (20km x 9 km).
Attaching the crash screenshot below, Would be great if anyone can help me fix it. Thanks!

Same issue for me, I would be interested in finding a fix. My map is 10km by 10km. On the first launch of carla (after booting the computer) it works fine and I can even drive in the map, but a lot of times it has the same segmentation error.


Fixed lack of Specular property in DirectionalLight3D

I've re-added the specular property for the DirectionalLight3D to fix #80737.

Thank you for opening this PR!
The documentation needs to be updated to reflect the new default values, see the test with a cross next to it

Thank you for opening this PR!
The documentation needs to be updated to reflect the new default values, see the test with a cross next to it

I think I've fixed it, but I'm not entirely sure whether or not the documentation is correct


Stuck on loading after changing 'Members list' values prior to upload

The Champions app is stuck on loading within MS Teams if we customise the Member list values following the customisation guidance. The 'Region' and 'Country' values aren't applicable to us, so we were wanting to amending these prior to uploading the app onto SharePoint.
If we upload the standard package it works fine, but if we amend config file to change the Members list values it gets stuck on loading. If we refresh the page when it's stuck, it returns the below error:

All the lists appear fine within the SharePoint site, and adding the user into the members list gets the rest of the app to load but continues to show the above error for everyone that tries to access the app.
Please could someone advise what we're doing wrong?
APIs are all approved etc.

Hi @DabHub2,
Thanks for reaching out to us.
Currently the only customization allowed with standard installation is as mentioned here in the wiki.
Any other changes would require changes in the code as well. Changing the config list value alone would not work. This means any changes you want to do, will have to be done with a custom version.
Please let me know if you need more information.
Thanks.

That's worked, thank you!
Apologies, I had only got as far as the section at the top where it mentions amending the .json.


can't save the model by using keras example

Environment:

Python 3.6.9
Spark version 2.4.3
open jdk11
TensorFlow version 1.14.0

I tried to execute sample program of stand alone by keras at (https://github.com/yahoo/TensorFlowOnSpark/blob/master/examples/mnist/keras/README.md)
I tried to execute "Run MNIST MLP using InputMode.SPARK" on README.md
but I could not save the model.
Do you know why?
${SPARK_HOME}/bin/spark-submit \

--master ${MASTER}
--conf spark.cores.max=${TOTAL_CORES}
--conf spark.task.cpus=${CORES_PER_WORKER}
--conf spark.executorEnv.JAVA_HOME="$JAVA_HOME"
${TFoS_HOME}/examples/mnist/keras/mnist_mlp_estimator.py
--cluster_size ${SPARK_WORKER_INSTANCES}
--input_mode spark
--images ${TFoS_HOME}/mnist/csv/train/images
--labels ${TFoS_HOME}/mnist/csv/train/labels
--epochs 5
--model_dir ${TFoS_HOME}/mnist_model
--tensorboard
19/08/07 22:36:59 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/07 22:36:59 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/07 22:37:00 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/shina/anaconda3/envs/py36/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
WARNING: Logging before flag parsing goes to stderr.
W0807 22:37:01.925125<PHONE_NUMBER>05376 deprecation_wrapper.py:119] From /opt/TensorFlowOnSpark/examples/mnist/keras/mnist_mlp_estimator.py:11: The name tf.train.SessionRunHook is deprecated. Please use tf.estimator.SessionRunHook instead.

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/07 22:37:02 INFO SparkContext: Running Spark version 2.4.3
19/08/07 22:37:02 INFO SparkContext: Submitted application: mnist_mlp
19/08/07 22:37:02 INFO SecurityManager: Changing view acls to: root
19/08/07 22:37:02 INFO SecurityManager: Changing modify acls to: root
19/08/07 22:37:02 INFO SecurityManager: Changing view acls groups to:
19/08/07 22:37:02 INFO SecurityManager: Changing modify acls groups to:
19/08/07 22:37:02 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
19/08/07 22:37:02 INFO Utils: Successfully started service 'sparkDriver' on port 37115.
19/08/07 22:37:02 INFO SparkEnv: Registering MapOutputTracker
19/08/07 22:37:02 INFO SparkEnv: Registering BlockManagerMaster
19/08/07 22:37:02 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
19/08/07 22:37:02 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
19/08/07 22:37:02 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-75e07f01-bcc4-4558-ab6a-05d107110af8
19/08/07 22:37:02 INFO MemoryStore: MemoryStore started with capacity 434.4 MB
19/08/07 22:37:02 INFO SparkEnv: Registering OutputCommitCoordinator
19/08/07 22:37:02 INFO Utils: Successfully started service 'SparkUI' on port 4040.
19/08/07 22:37:02 INFO SparkUI: Bound SparkUI to <IP_ADDRESS>, and started at http://<IP_ADDRESS>:4040
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark://shina:7077...
19/08/07 22:37:03 INFO TransportClientFactory: Successfully created connection to shina/<IP_ADDRESS>:7077 after 39 ms (0 ms spent in bootstraps)
19/08/07 22:37:03 INFO StandaloneSchedulerBackend: Connected to Spark cluster with app ID app-20190807223703-0006
19/08/07 22:37:03 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 41019.
19/08/07 22:37:03 INFO NettyBlockTransferService: Server created on <IP_ADDRESS>:41019
19/08/07 22:37:03 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20190807223703-0006/0 on worker-20190807211209-<IP_ADDRESS>-46649 (<IP_ADDRESS>:46649) with 1 core(s)
19/08/07 22:37:03 INFO StandaloneSchedulerBackend: Granted executor ID app-20190807223703-0006/0 on hostPort <IP_ADDRESS>:46649 with 1 core(s), 1024.0 MB RAM
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20190807223703-0006/1 on worker-20190807211206-<IP_ADDRESS>-37383 (<IP_ADDRESS>:37383) with 1 core(s)
19/08/07 22:37:03 INFO StandaloneSchedulerBackend: Granted executor ID app-20190807223703-0006/1 on hostPort <IP_ADDRESS>:37383 with 1 core(s), 1024.0 MB RAM
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor added: app-20190807223703-0006/2 on worker-20190807211203-<IP_ADDRESS>-38143 (<IP_ADDRESS>:38143) with 1 core(s)
19/08/07 22:37:03 INFO StandaloneSchedulerBackend: Granted executor ID app-20190807223703-0006/2 on hostPort <IP_ADDRESS>:38143 with 1 core(s), 1024.0 MB RAM
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20190807223703-0006/0 is now RUNNING
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20190807223703-0006/1 is now RUNNING
19/08/07 22:37:03 INFO StandaloneAppClient$ClientEndpoint: Executor updated: app-20190807223703-0006/2 is now RUNNING
19/08/07 22:37:03 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, <IP_ADDRESS>, 41019, None)
19/08/07 22:37:03 INFO BlockManagerMasterEndpoint: Registering block manager <IP_ADDRESS>:41019 with 434.4 MB RAM, BlockManagerId(driver, <IP_ADDRESS>, 41019, None)
19/08/07 22:37:03 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, <IP_ADDRESS>, 41019, None)
19/08/07 22:37:03 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, <IP_ADDRESS>, 41019, None)
19/08/07 22:37:04 INFO StandaloneSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
args: Namespace(batch_size=100, cluster_size=3, epochs=5, images='/opt/TensorFlowOnSpark/mnist/csv/train/images', input_mode='spark', labels='/opt/TensorFlowOnSpark/mnist/csv/train/labels', model_dir='/opt/TensorFlowOnSpark/mnist_model', num_ps=1, output='predictions', steps=2000, tensorboard=True)
19/08/07 22:37:07 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 118.5 KB, free 434.3 MB)
19/08/07 22:37:07 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 22.9 KB, free 434.3 MB)
19/08/07 22:37:07 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:41019 (size: 22.9 KB, free: 434.4 MB)
19/08/07 22:37:07 INFO SparkContext: Created broadcast 0 from textFile at NativeMethodAccessorImpl.java:0
19/08/07 22:37:07 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 118.5 KB, free 434.1 MB)
19/08/07 22:37:07 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 22.9 KB, free 434.1 MB)
19/08/07 22:37:07 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on <IP_ADDRESS>:41019 (size: 22.9 KB, free: 434.4 MB)
19/08/07 22:37:07 INFO SparkContext: Created broadcast 1 from textFile at NativeMethodAccessorImpl.java:0
19/08/07 22:37:08 INFO FileInputFormat: Total input paths to process : 10
19/08/07 22:37:08 INFO FileInputFormat: Total input paths to process : 10
E0807 22:37:09.127454<PHONE_NUMBER>41472 TFCluster.py:323] Exception in TF background thread
19/08/07 22:37:09 INFO SparkUI: Stopped Spark web UI at http://<IP_ADDRESS>:4040
19/08/07 22:37:09 INFO StandaloneSchedulerBackend: Shutting down all executors
19/08/07 22:37:09 INFO CoarseGrainedSchedulerBackend$DriverEndpoint: Asking each executor to shut down
19/08/07 22:37:10 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/08/07 22:37:10 INFO MemoryStore: MemoryStore cleared
19/08/07 22:37:10 INFO BlockManager: BlockManager stopped
19/08/07 22:37:10 INFO BlockManagerMaster: BlockManagerMaster stopped
19/08/07 22:37:10 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/08/07 22:37:10 INFO SparkContext: Successfully stopped SparkContext
19/08/07 22:37:10 INFO ShutdownHookManager: Shutdown hook called
19/08/07 22:37:10 INFO ShutdownHookManager: Deleting directory /tmp/spark-ab9e4057-b5f5-4341-8388-a0ef1a36b78b/pyspark-114cda21-28ee-40b8-92a7-dc060d9fe0fb
19/08/07 22:37:10 INFO ShutdownHookManager: Deleting directory /tmp/spark-ab9e4057-b5f5-4341-8388-a0ef1a36b78b
19/08/07 22:37:10 INFO ShutdownHookManager: Deleting directory /tmp/spark-cff099b2-da85-4444-817a-9abdd209625c
I think the following error is issue.
E0807 22:37:09.127454<PHONE_NUMBER>41472 TFCluster.py:323] Exception in TF background thread

Can you re-run your example while monitoring the Spark UI?  (looks like it'll be available at http://<IP_ADDRESS>:4040 per your logs above).  While the job is running, view the "Executors" tab and look for any failed tasks... also view the "stderr" logs of each of the executors to look for any errors/exceptions.

I run example with spark UI(http://<IP_ADDRESS>:4040), but nothing is wirrien in "Executor" tab...
so I viewed  http://localhost:8080, the below log is output.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/08 10:26:49 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 3894@shina
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for TERM
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for HUP
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for INT
19/08/08 10:26:49 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/08 10:26:49 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/08 10:26:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/08 10:26:52 INFO SecurityManager: Changing view acls to: root
19/08/08 10:26:52 INFO SecurityManager: Changing modify acls to: root
19/08/08 10:26:52 INFO SecurityManager: Changing view acls groups to:
19/08/08 10:26:52 INFO SecurityManager: Changing modify acls groups to:
19/08/08 10:26:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
19/08/08 10:26:54 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:46081 after 156 ms (0 ms spent in bootstraps)
19/08/08 10:26:54 INFO SecurityManager: Changing view acls to: root
19/08/08 10:26:54 INFO SecurityManager: Changing modify acls to: root
19/08/08 10:26:54 INFO SecurityManager: Changing view acls groups to:
19/08/08 10:26:54 INFO SecurityManager: Changing modify acls groups to:
19/08/08 10:26:54 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(root); groups with view permissions: Set(); users  with modify permissions: Set(root); groups with modify permissions: Set()
19/08/08 10:26:54 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:46081 after 17 ms (0 ms spent in bootstraps)
19/08/08 10:26:54 INFO DiskBlockManager: Created local directory at /tmp/spark-50f1a4b6-494a-403b-8f48-8d740b1736b0/executor-48de468e-760c-42e5-9ccb-b9bae1ba4c96/blockmgr-3efd72b3-dc94-4e4b-9862-f974a7b5eba5
19/08/08 10:26:54 INFO MemoryStore: MemoryStore started with capacity 434.4 MB
19/08/08 10:26:55 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@<IP_ADDRESS>:46081
19/08/08 10:26:55 INFO WorkerWatcher: Connecting to worker spark://Worker@<IP_ADDRESS>:43011
19/08/08 10:26:55 INFO CoarseGrainedExecutorBackend: Successfully registered with driver
19/08/08 10:26:55 INFO Executor: Starting executor ID 2 on host <IP_ADDRESS>
19/08/08 10:26:55 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:43011 after 87 ms (0 ms spent in bootstraps)
19/08/08 10:26:55 INFO WorkerWatcher: Successfully connected to spark://Worker@<IP_ADDRESS>:43011
19/08/08 10:26:55 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
ker@<IP_ADDRESS>:43011

After the job has completed/failed, browse to http://localhost:8080.  You should see "Completed Applications".  Click on the most recent one.  Then, you should see "Removed Executors".  Click through each of the "stderr" links  (on the right side of the page) and look for any errors.  You will need to view each of the executors, since the error may be logged on just one.

I clicked "stderr" link, then it was logged.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/08 10:26:49 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 3894@shina
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for TERM
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for HUP
19/08/08 10:26:49 INFO SignalUtils: Registered signal handler for INT
19/08/08 10:26:49 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/08 10:26:49 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/08 10:26:52 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/08 10:26:52 INFO SecurityManager: Changing view acls to: root
19/08/08 10:26:52 INFO SecurityManager: Changing modify acls to: root
19/08/08 10:26:52 INFO SecurityManager: Changing view acls groups to:
19/08/08 10:26:52 INFO SecurityManager: Changing modify acls groups to:
19/08/08 10:26:52 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
19/08/08 10:26:54 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:46081 after 156 ms (0 ms spent in bootstraps)
19/08/08 10:26:54 INFO SecurityManager: Changing view acls to: root
19/08/08 10:26:54 INFO SecurityManager: Changing modify acls to: root
19/08/08 10:26:54 INFO SecurityManager: Changing view acls groups to:
19/08/08 10:26:54 INFO SecurityManager: Changing modify acls groups to:
19/08/08 10:26:54 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set()
19/08/08 10:26:54 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:46081 after 17 ms (0 ms spent in bootstraps)
19/08/08 10:26:54 INFO DiskBlockManager: Created local directory at /tmp/spark-50f1a4b6-494a-403b-8f48-8d740b1736b0/executor-48de468e-760c-42e5-9ccb-b9bae1ba4c96/blockmgr-3efd72b3-dc94-4e4b-9862-f974a7b5eba5
19/08/08 10:26:54 INFO MemoryStore: MemoryStore started with capacity 434.4 MB
19/08/08 10:26:55 INFO CoarseGrainedExecutorBackend: Connecting to driver: spark://CoarseGrainedScheduler@<IP_ADDRESS>:46081
19/08/08 10:26:55 INFO WorkerWatcher: Connecting to worker spark://Worker@<IP_ADDRESS>:43011
19/08/08 10:26:55 INFO CoarseGrainedExecutorBackend: Successfully registered with driver
19/08/08 10:26:55 INFO Executor: Starting executor ID 2 on host <IP_ADDRESS>
19/08/08 10:26:55 INFO TransportClientFactory: Successfully created connection to /<IP_ADDRESS>:43011 after 87 ms (0 ms spent in bootstraps)
19/08/08 10:26:55 INFO WorkerWatcher: Successfully connected to spark://Worker@<IP_ADDRESS>:43011
19/08/08 10:26:55 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
ker@<IP_ADDRESS>:43011
I think "19/08/08 10:26:55 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM" is error, but I don't know why.

How many executors are you using?  i.e. $SPARK_WORKER_INSTANCES
(The error that you highlighted is just a standard error msg when spark is shutting down the executors due to some other error).

Never mind, I see cluster_size=3 in your logs above.  In that case, have you been able to look at the stderr logs for each of the 3 executors?

I use 3 worker, SPARK_WORKER_INSTANCES=3
・executor ID 2
cores : 1
Memory : 1024
state : killed
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/09 08:44:29 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 2724@shina
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for TERM
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for HUP
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for INT
19/08/09 08:44:30 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/09 08:44:30 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/09 08:44:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/09 08:44:31 INFO SecurityManager: Changing view acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing view acls groups to:
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls groups to:
19/08/09 08:44:31 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()
19/08/09 08:44:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
abled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()
・executor ID 1
cores : 1
Memory : 1024
state : killed
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/09 08:44:29 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 2714@shina
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for TERM
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for HUP
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for INT
19/08/09 08:44:30 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/09 08:44:30 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/09 08:44:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/09 08:44:31 INFO SecurityManager: Changing view acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing view acls groups to:
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls groups to:
19/08/09 08:44:31 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()
19/08/09 08:44:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
abled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()
・executor ID 0
cores : 1
Memory : 1024
state : killed
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
19/08/09 08:44:29 INFO CoarseGrainedExecutorBackend: Started daemon with process name: 2723@shina
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for TERM
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for HUP
19/08/09 08:44:29 INFO SignalUtils: Registered signal handler for INT
19/08/09 08:44:29 WARN Utils: Your hostname, shina resolves to a loopback address: <IP_ADDRESS>; using <IP_ADDRESS> instead (on interface enp0s25)
19/08/09 08:44:29 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/opt/spark-2.4.3-bin-hadoop2.7/jars/spark-unsafe_2.11-2.4.3.jar) to method java.nio.Bits.unaligned()
WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
19/08/09 08:44:31 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/08/09 08:44:31 INFO SecurityManager: Changing view acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls to: shina
19/08/09 08:44:31 INFO SecurityManager: Changing view acls groups to:
19/08/09 08:44:31 INFO SecurityManager: Changing modify acls groups to:
19/08/09 08:44:31 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()
19/08/09 08:44:32 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL TERM
abled; ui acls disabled; users  with view permissions: Set(shina); groups with view permissions: Set(); users  with modify permissions: Set(shina); groups with modify permissions: Set()

Closing due to inactivity.  Feel free to re-open if still an issue.


Failed to map CK executable into memory

I have an issue generating a lip file for Fallout 4, using a command like this:
FaceFXWrapper.exe Fallout4 USEnglish ./FonixData.cdf ./voice.wav ./voice_r.wav ./voice.lip "It was no big deal."
I get this error:

Failed to map CK executable into memory

Generating for Skyrim, however, works well, like so:
FaceFXWrapper.exe Skyrim USEnglish ./FonixData.cdf ./voice.wav ./voice_r.wav ./voice.lip "It was no big deal."
(Only changed "Fallout4" to "Skyrim")

I also have this issue. i do not have Skyrim installed so I can't test with that. If it's related, I wanted to use FaceFXWrapper because the 32 bit CK fails to open at all.


WIP - DO NOT merge Dynamic sync

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

I do have concerns with how this will work with a cluster - is this going to introduce conflicts to the ACL?

@jimmidyson what do you mean by 'conflicts to the ACL'  @sosiouxme discussed our issues with doing dynamic ACLs and for this to work we believe each member will have to keep its own copy of the ACL unless @ewolinetz determines we can do othewise.  Please review.  added ACL expiration as well.

Yeah if we use separate acl for each node that's OK. I was worried about concurrent updates to the same acl doc.

@jimmidyson Updated to remove interface to fabric8 openshift client.  @ewolinetz added readme with info about running from eclipse.  PTAL

:+1:


Keep static bzip2 for Python build

When/if bzip2 removes the static library it will be necessary to update the python recipes as they statically link to libbz2 to avoid bus errors when the bzip2 is updated by conda
Originally posted by @jjhelmus in https://github.com/conda-forge/conda-forge.github.io/issues/1095#issuecomment-656188267

@jjhelmus I guess this means that we should add an output here to also build libbz2.a and use that in Python? I guess otherwise we will run into the bus error ahain.

Yes, Python really needs the static library. A bzip2-static or libbz2-static output would be great.

On Ubuntu, lib-dynload/_bz2.*.so is linked to the shared library. Can we do the same? Or am I missing something here?

We can try dynamically linking the _bz2 module.
My recollection is that this caused issues if the past when conda updates the bzip2 packages because Python was using libbz2.so for unpacking a .tar.bz2 file which is unlinked during the transaction.  This may not be the case anymore, it would be worthwhile to test this.

Ah, this must be before conda started using libarchive

Yes, this was before the move to conda-package-handling (cph) which uses libarchive, although even cph falls back the bz2 module (via tarfile) when libarchive is not available.

_libarchive_static_for_cph-feedstock needs a static version of libbz2 as well as a number of other libraries.

None of the pkgs mentioned here needs a static library anymore.


GitHub fails to commit

(From @eanders-MS:)
A GitHub project can get into a state where it will fail to commit changes. I think this occurs when, in TypeScript, a file is added that was previously deleted.
When GitHub fails to commit changes, the user sees this message:

In the console, this is the error message:
Error: Nothing to commit!

(But there are local changes. The GitHub view shows the diffs.)
After getting this message, cloud sync fails. Reopening the tab clears the cloud save error, but GitHub is still stuck unable to commit.

I can't repro this


chore: fix unexpected cfg warning

Fixes unexpected cfg warnings, which cause the ci fail.
https://doc.rust-lang.org/nightly/cargo/reference/build-scripts.html#rustc-check-cfg

Started with https://github.com/rust-lang/cargo/pull/13571

A build file slows down compilation measurably. Can it be solved without?

I think another workaround is to allow this lint.

Changed to allow it.

Note that it's possible to add a ci check that still runs the lint, with your custom --cfgs allowed. See #6538.

I have known it, but I do not think that it is a good idea to use --cfg for configs which are always intended to be given.

What do you mean by that?

@dtolnay has a neat approach to avoid the build.rs build time penalty: https://github.com/tokio-rs/tokio/pull/6542

That's clever, but I think currently my preferred solution is to just allow the lint normally, and optionally add a CI job checking it.

Updated to allow unexpected_cfgs as default and check it with a new ci job.

Heads up, with the release of rust-lang/cargo#13913 (in nightly-2024-05-19), there is no longer any need for the kind of workarounds employed in this PR. Cargo has now gain the ability to declare --check-cfg args directly inside the [lints] table with [lints.rust.unexpected_cfgs.check-cfg][^1]:
Cargo.toml:
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(foo)'] }

Note that the diagnostic output of the lint has been updated to suggest the [lints] approach first. You can use it to guide you through the --check-cfg arguments that may need to be added.

[^1]: take effect on Rust 1.80 (current nightly), is ignored on Rust 1.79 (current beta), and produce an unused warning below


Add podspecs for CocoaPods

This patch adds two podspec files for the two parts of the library.
One podspec with two subspecs didn't work, because they are not
supposed to create different modules:
https://github.com/CocoaPods/CocoaPods/issues/3979
To fix CocoaPods' lint warnings, a git tag should be created in the
repository first, and the podspecs should be updated.
Also, the project file is updated to include the SimulatorStatusMagic
library as a subproject instead of the sources directly. This makes it
easier to import the framework in CocoaPods.
Closes https://github.com/zmeyc/UITestUtils/issues/2

Thank you for the PR, I'll test and merge it in in nearest few days.


.cloudfront.net domain activation - returns 404

App Id: d18a38ve9o5yls
Region: us-east-2
Step: Configure DNS settings
Status: Domain activation
Issue/question
d38qr63xn9qqf1.cloudfront.net is returning a 404. I've tried to set things up twice and both times the cloudfront URL just 404s.  I'm trying to connect AWS amplify to my GoDaddy domain.  I've added 2 CNAMEs in Godaddy:
CNAME mysite.com pointing to d38qr63xn9qqf1.cloudfront.net
as well as my SSL configuration which seems to be successful.
Am I missing something here?  The cloudfront URLs shouldn't just 404 right?

You have an app for d18a38ve9o5yls the custom domain is a separate Cloudfront distribution against d38qr63xn9qqf1.
Hitting d38qr63xn9qqf1.cloudfront.net will not work because Amplify removes that as a valid CNAME on the CF distribution (Amplify manages all the Cloudfront distributions in our service accounts - and not your AWS account)
In other words (in your specific case) - the CF distribution for the custom domain only has 2 cnames:

mysite.com
www.mysite.com

I.E. it does not contain the default of d38qr63xn9qqf1.cloudfront.net so that will not work - but your custom domains that you setup mysite.com and www.mysite.com would.
Closing for now - please re-open if you have additional questions or concerns.


Everycompat is broken since last updates

It's not possible to play with everycompat in 1.18.2 ever since the mod updated a few days ago

The issue unfortunately lies with everycompat, as the integration is on their side. I might be able to make a bandaid fix, but all it would do is prevent the crash. The resulting board blocks still wouldn't work.


How do I install it ?

Copy files to .config

Yep, you can just copy all folders to your ~/.config directory
Sorry about the delayed response. I took a break a long time ago.


[WFLY-14865] XMLXaDataSourceRuntimeHandler related if/else statement …

…with same condition
Fixes https://issues.redhat.com/browse/WFLY-14865

@tadamski Hello Tomasz, could you please review and approve this PR? Tests passed, fails seem unrelated.

@boris-unckel LGTM


RHEL for Edge default Ref hint

When no reference is specified in the UI rhel/8/$ARCH/edge will be used, it might be nice to hint at that value in the UI (input box?) so that it is clear to the user what happens if nothing is filled in there.

Interesting. I thought about this too. On the one hand, that would be helpful, on the other, we create a tight coupling between the UI and the backend this way, so updating the backend might mean they get out of sync. An argument for why it might be ok not to have a hint would be that anyone who cares should fill it in, and the default is just so we don't have to fail.
We have a similar situation with other defaults like the image size, which I don't think we expose, and which differs between distros/image types.
A more involved solution would be to have an API to query these sorts of things.

@teg I agree that it might be useful to get default values for the image customizations that can differ by type. Currently we call /compose/types to get a list of enabled image types. Perhaps we could expose more data for each of the types such as image size defaults and ostree defaults. I think it would also be nice to have the api pass strings for the image type label and the actual image type (i.e. amazon vs .raw).

@gicmo Is there a default for fedora iot as well?

@teg I agree that it might be useful to get default values for the image customizations that can differ by type. Currently we call /compose/types to get a list of enabled image types. Perhaps we could expose more data for each of the types such as image size defaults and ostree defaults. I think it would also be nice to have the api pass strings for the image type label and the actual image type (i.e. amazon vs .raw).

This would make sense to me. Essentially all the info we pass in ImageOptions to Manifest might have defaults and might make sense to query.


Returns Xcode.Project from generate()

Motivation
I'd like to test generated Xcode Project on XcodeprojTests.
Currently, Xcodeproj.generate returns only its path.
So we can't investigate contents of the project.
Description
I refactored Xcodeproj related code.
Xcodeproj.generate() returns Xcode.Project object.
Thanks to this change, we can write tests easily.

@ikesyo After discussion with you, I introduce buildXcodeprojPath to build a path to Xcode project wrapper directory.
How about this?
Could you check this in your spare time 🙏

Could you review anyone?

@hartbit Sorry for my late response.
I've fixed documentation of generate method.
Could you review this again?

@hartbit Thanks! your point is good.
I fixed on this commit.
https://github.com/apple/swift-package-manager/pull/1517/commits/667b9922a596929d68c7bbd206921babc5eb96ed

@swift-ci please smoke test

@aciidb0mb3r This LGTM.

Thanks!!


Shader code compilation issue on android browsers only

three 0.164.1:
@pmndrs/vanilla 1.19.1:
node 20.16.0:
npm (or yarn) 10.8.1:

Problem description:
THIS OCCURS ONLY ON ANDROID BROWSERS
Note: This issue can be reproduced on : https://vis-prime.github.io/explore-vanilla-drei/?scene=Cloud
The shader code for clouds class (drei-vanilla/src/core/Cloud.ts) is causing an error due to a conflict in names for shader attributes in the CloudMaterialMaker function
Below is the complete error message
_three.module.js:20131 THREE.WebGLProgram: Shader Error 0 - VALIDATE_STATUS false
Material Name:
Material Type: MeshLambertMaterial
Program Info Log: Name conflicts between an uniform and an attribute: opacity_
Suggested solution:
Just rename the repeated attribute and all its used instances and it'll be fixed

Confirmed

Renaming the attribute removes the error but the cloud mesh is fully invisible, so that's not the way.
Happens on the react version too https://drei.pmnd.rs/?path=/story/staging-cloud--cloud-st


Sensor Library - DCT500

Test Script Author: Alex Zeng @AlexZengXi
Current Hardware Owner: Alex Zeng @AlexZengXi
Final Review: Urmil Modi @urmilmodi

Author reviews library contents
Jayden Lefebvre writes Arduino test script using library
Current owner tests library implementation on hardware, documents results

Comment on this PR if you have any questions

The code for the DCT500 runs without errors. But we don't have the equipment to test if it will output the expected values yet.


Font size fix

Describe the problem
Hello,
I am really excited for svelte kit release 🎉, I was going through the website and the font size on some sections were really small even parts of the documentation, I think it would be a good idea to increase the font size to improve readability it maybe only me or my personal preference but I think there is room for improvement bellow are some examples
Example 1:

Example2:

Describe the proposed solution
It would be great if fonts had  size consistency inside the landing page of svelte kit and increasing fonts in some places.
Alternatives considered
Importance
nice to have
Additional Information
No response

Which one could be the approach to resolving this issue? Should we define a base font size for all the pages base on REM value?
Should we define sizes for paragraphs, titles, subtitles, and more?

I personally think that the later off the two is the way to go, defining different sizes for paragraphs, titles and other elements would help better differentiate visually.
Moreover, instead of REM can't we use ch for font-size? [Reference]("ch
Character unit. The CSS ch unit is defined as the width of the character 0 (zero, or U+0030) of the font.
While the ch unit works as an exact measurement for monospaced / fixed width fonts like Courier, it can be unpredictable with proportional fonts like Arial."
https://www.freecodecamp.org/news/css-unit-guide/#:~:text=ch,fonts like Arial.)

defining different sizes for paragraphs, titles and other elements would help better differentiate visually.

Strongly agree.

Agree, there's nothing that needs changing here — the font-size for body text on the docs is 16px which is very standard (and larger than the 14px on GitHub — typefaces aren't directly comparable, but Overpass at 16px is perceptually larger than the system font at 14px), and if you really want to read the wall of text then you can zoom in or copy-and-paste.


Liqoctl Add Cluster Labels

Description
This pr adds a simple logic to add some cluster labels to the liqoctl installed clusters.
It adds the liqo.io/provider label with the provider name as value, and (on managed clusters, i.e. AKS, EKS, GKE) it adds the topology.kubernetes.io/zone and the topology.kubernetes.io/region basing on the cluster zone/region retrieved from the cloud provider APIs. In that way, the virtual node is now compliant with the topology keys and with the topology aware hints labeling system.

/test

/test

/test

/merge


"Recombinant" or "Recombination node"

I'm going through the codebase and the manuscript, and I see we use the word "recombinant" in lots of places (e.g. class names, variable names). I suggest that most people will think of a recombinant as a sample node that has recombination in its ancestry, e.g. "I've been infected by a recombinant (virus)". So I wonder if we want to change the terminology in the code to refer to the nodes in which recombination events actually took place as "recombination nodes" instead? E.g. instead of the Recombinant class, we should have a RecombinationNode class.
It's a bit of a large change, but should be reasonably easy to make with auto search-and-replace I think. Thoughts?

I have a (very) slight preference for recombinant nodes, because these nodes represent recombinant sequences (sampled or reconstructed) or products of recombination events.

I'm still uneasy about this, TBH. E.g. in this plot:

I think of the black node as a "recombination node", not a "recombinant". I think of the immediate children of the black node as recombinants. I also suspect that lots of researchers would think of all the purple and blue nodes as "recombinant viruses"?
Could we see what other people on the project think, perhaps?

We're splitting hairs here - the recombination node represents the original recombinant sequence. The children of a recombinant aren't necessarily recombinants, they may be later descendants of that sequence. We only insert recombination nodes as a convenience to make it easy to find what we think are unique recombinants - we may have multiple sequences from the same day which have the same copying path and we therefore "path compress" these together into a single node, representing the unique recombinant origin.

There is the distinction of the "causal" sequences for a recombinant/recombination node. These are the sequences that required a path of > 1 parent, resulting in the creation of the recombination node, which represents the original recombinant sequence.

Also, talking about "recombination nodes" is confusing because it ties to that specific ARG. We want to be able to refer to the original recombinant sequence that was the result of recombination between two strains within a host. Using "node" to refer to this sequence is misleading.

I agree that the "recombination node" is tied to a specific ARG, and a recombinant is a sequence that results directly from a recombination. But I often want to refer to the specific (essentially, inferred) node in the specific ARG when e.g. describing a plot like the one above. I think labelling the black dot a "recombination node" here is less confusing, as it is deliberately tied to this particular ARG.


Should the source distribution contain the generated HTML documentation?

At the moment, it doesn't. We would need to adapt the MANIFEST.in file to do it.
To generate the HTML documentation you need to install the dev requirements. On the other hand, the documentation can always be viewed online.
Opinions?

When going off-line, Markdown sources are still available. So, I would say no.

What about having a command for serving the documentation (offline)?

make docs
cd site
$BROWSER http://localhost:8000/ &
python -m http.server

But the point is that making the docs requires installing a lot of additional stuff, which could be a PITA when you don't have a dedicated virtualenv for Watson, which you probably don't, if you just want to use Watson.

Okay for including the docs, but who's gonna go in the installed sources to read it? :)
Anyway, that would be :+1: for me if it doesn't add much space to the package.

Unfortunately, including the HTML docs blows up the sdist from 8 Mb to 18 Mb. About 95% of the size are the images in doc/img, which then get included again under site/img. And 90% of the size of the images is the animated gif of the cat. :)

BTW, the wheel is only about 20 kB :) which should make downloading and installing Watson much faster for most users.

Let's not add the doc then. It is fine online, and we could have a command to open <doc url>.

Should this issue be closed? It looks like a decision was made :)


Automated Package Upgrade

:tada: This PR is included in version 25.5.55 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 25.5.55 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Cop idea: IdenticalSiblingSetup

RSpec.describe 'Foo' do
  context 'when bar' do
    let(:qux) { 111 }
    let(:unique) { 'no one like me' }
    before { setup(qux, unique) }
    it { is_expected.to eq 222 }
  end

  context 'when bar' do
    let(:qux) { 111 }
    let(:unique) { 'I am different!!!' }
    before { setup(qux, unique) }
    it { is_expected.to eq 333 }
  end
end

In the case above let(:qux) and before can be pulled up to the outer example group.

Perhaps this should have some minimum number of common setup?
What if the default value of :qux is already set at the top-level to 112, but in those two cases we want to have 111
I like the idea of the cop, but it should be scoped better. I suggest:

Flag only if all siblings have the same value
Or if there is more then MinDuplicatedElements between two siblings
In any case, be mindful if a value is already present on an outer scope