python310: 3.10.9 -> 3.10.10; python311: 3.11.1 -> 3.11.2

Description of changes
https://docs.python.org/release/3.11.2/whatsnew/changelog.html
https://docs.python.org/release/3.10.10/whatsnew/changelog.htm
Based on master for testing purpose, will rebase against staging once testing is complete.l

Things done

Built on platform(s)

[x] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Welp, building it on master went nowhere, due to libuv test flakiness, that was fixed in #211896, which is currently on staging-next.

x86_64-linux

[x] python310.tests
[x] python311.tests

aarch64-linux

[x] python310.tests
[x] python311.tests

aarch64-darwin

[ ] python310.tests
[ ] python311.tests

x86_64-darwin

[ ] python310.tests
[ ] python311.tests

x86_64-darwin failing in darwin.apple_sdk_11_0.IOKit
IOKit-x86_64-darwin> source root is .
IOKit-x86_64-darwin> setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file ./IOUSBFamily-IOUSBFamily-630.4.5/USBProberV2/version.plist
IOKit-x86_64-darwin> patching sources
IOKit-x86_64-darwin> configuring
IOKit-x86_64-darwin> no configure script, doing nothing
IOKit-x86_64-darwin> building
IOKit-x86_64-darwin> no Makefile or custom buildPhase, doing nothing
IOKit-x86_64-darwin> installing
IOKit-x86_64-darwin> /nix/store/0lk5sy3ik47jz6wxyf1lbbmgkhk621g6-IOKit-/Library/Frameworks/IOKit.framework/Headers /private/tmp/nix-build-IOKit-.drv-0
IOKit-x86_64-darwin> /private/tmp/nix-build-IOKit-.drv-0
IOKit-x86_64-darwin> cp: cannot stat 'IOUSBFamily-630.4.5/IOUSBFamily/Headers/IOUSBLib.h': No such file or directory

@mweinelt this has broken eval
error: Function called without required argument "sha256" at /home/qyliss/src/nixpkgs/pkgs/development/interpreters/python/cpython/2.7/default.nix:27

https://github.com/NixOS/nixpkgs/pull/216189


Feature Request: Support for Cloud Front using Discovery

As far as i can see, there is no support for Cloud Front using Discovery. This means that we have to manually add labels.
Are there any limitations that are preventing Cloud Front metrics from being used with Discovery?

https://docs.aws.amazon.com/resourcegroupstagging/latest/APIReference/Welcome.html - It is in tagging API supported so should not be so difficult. Will try to do it next maintenance hour.

@thomaspeitz im playing with implementation https://github.com/ivx/yet-another-cloudwatch-exporter/compare/master...mentos1386:feat-add-cloud-front. What do you mean with "next maintenance hour"?

Will try to look over your implementation on friday.


Remove use of deprecated webkitGetUserMedia

navigator.webkitGetUserMedia has been deprecated, and Electron's GitHub issues has good information on the replacement API and links to usage examples & code here.

@codysherman It now looks like this is only used in the MultiStreamsMixer.js file in @/utils/background/helpers, which also doesn't look like it's used anywhere in the project any more either. We can probably remove that file, unless there was a reason we were keeping it around, and/or close this ticket.

Based on @codysherman's investigation, MultiStreamsMixer seems to be used for combining multiple video streams into 1 stream -- we'll want to do this at some point if we choose to implement video chat, but that said, we'll probably also want to use the NPM package to bring it in instead of the hard-coded file.


Got Error: Error Domain=com.apple.AuthenticationServices.AuthorizationError Code=1000 "(null)"

Got Error: Error Domain=com.apple.AuthenticationServices.AuthorizationError Code=1000 "(null)" when try to login with Apple ID. I have added Sign In with Apple capability in my project already.
I have checked on simulator and on real device and got same issue on both and also tested the production build but got same issue.

Got Error: Error Domain=com.apple.AuthenticationServices.AuthorizationError Code=1000 "(null)" when try to login with Apple ID. I have added Sign In with Apple capability in my project already.

@pankajatocdlab, solved a problem?

Got Error: Error Domain=com.apple.AuthenticationServices.AuthorizationError Code=1000 "(null)" when try to login with Apple ID. I have added Sign In with Apple capability in my project already.

Can any one help to fix this issue.

@pankajatocdlab and @vevnkataSukumar, could you guys please check your XCode's Capabilities section for the project in which you are integrating this lib and let me know if Sign in with Apple has enabled or not?
Sign in with Apple capability should be enabled.

Closing this issue for now. Feel free to open if it is still coming with 2FA enabled accounts.


Validation passes when array type is invalid

Describe the bug
When creating a custom type and using with the array rule, it appears that the validations applied by the type are not applied always by the rule. See the example below, and note that the last test is failing when it shouldn't:
To Reproduce
require 'dry/validation'
require 'dry/types'

module Types
  include Dry.Types(default: :nominal)

  StringOrHash = Types::Strict::String.enum('foo') | Types::Strict::Hash.schema(bar: Types::Strict::String.enum('baz'))
end

class TestContract < Dry::Validation::Contract
  params do
    required(:test).array(Types::StringOrHash)
  end
end

These tests all pass as expected:
RSpec.describe Types::StringOrHash do
  it 'works with a valid hash input' do
    expect { described_class[{ bar: 'baz' }] }.not_to raise_error
  end

  it 'works with a valid string input' do
    expect { described_class['foo'] }.not_to raise_error
  end

  it 'fails with an invalid string input' do
    expect { described_class['bar'] }.to raise_error
  end

  it 'fails with a hash with invalid values' do
    expect { described_class[{ bar: 'quz' }] }.to raise_error
  end

  it 'fails with a hash with invalid keys' do
    expect { described_class[{ quz: 'baz' }] }.to raise_error
  end
end

The first test passes, the second fails:
RSpec.describe TestContract do
  subject { described_class.new.call(params) }

  context 'when passing a valid array' do
    let(:params) { { test: ['foo', { bar: 'baz' }] } }

    it { is_expected.to be_success }
  end

  # THIS TEST FAILS
  context 'when passing an invalid array' do
    let(:params) { { test: ['foo', { bar: 'qux' }] } }

    it { is_expected.to be_failure }
  end
end

Expected behavior
It is expected that the validation fails when the input value is not of the described value.
My environment

Affects my production application: YES
Ruby version: 2.7.5
OS: MacOS Monterrey 12.0.1 (Apple chip)

Thanks for reporting this. Looks like the right-side hash type is not expanded fully to a schema rule:
dry-validation> TestContract.schema
=> #<Dry::Schema::Params keys=["test"] rules={:test=>"key?(:test) AND key[test](array? AND each(str? AND included_in?([\"foo\"]) OR hash?))"}>


Styled-Components 4.x support

Big fan of react-reveal.
I'm trying to use it with the latest version of styled-components.  Previously styled components exposed an innerRef prop that allowed react-reveal to gel nicely but with refForwarding in the latest versions of React, that styled-components prop has been deprecated & removed.   This means I can't 'reveal' a styled component without a wrapper div being placed, I believe due to this line, but I'm not sure.
I've made a code pen to exemplify this issue. https://codesandbox.io/s/zrz35jnymp / https://zrz35jnymp.codesandbox.io/
I think it'd probably be a semver major change to update this default prop behavior, so I wouldn't be opposed to adding some sort of flag in the mean time to specify I don't want wrapper div even if the refProp is ref.  Thoughts?

I'm facing a similar issue. I'm trying to force re-mount a styled-component. I mean, it does remount, but the animation is only occurring on the first mount.
// THIS WORKS
<Slide right>
  <img key={keyThatWillChangeWithSrc} src={props.src}/>
</Slide>

// THIS DOESN'T WORK
<Slide right>
  <Styled_IMG key={keyThatWillChangeWithSrc} src={props.src}/>
</Slide>

Is react-reveal still under development?


(q)renderdoc/CMakeLists.txt: remove flag unsupported by GCC

-Wno-unknown-warning is not supported by GCC, only by Clang so we
remove it because newer GCC versions like 9.2.0 will complain, older
versions would just ignore it.

On closer inspection, -Wunknown-warning is not supported but the negative form -Wno-unknown-warning is supported and only has the effect of suppressing the "unknown-warning" unsuported option complaint during compilation. Closing.


update contract to objectively fail in eosio::onerror notification handler

A deferred transaction that soft_fails will be billed to the sending contract. This could potentially be abused. But a hard_fail is billed to the same entities that would normally pay for the deferred transaction if it were to successfully execute.
Given the above facts, it seems reasonable for contracts that do not wish to handle deferred transaction errors to by default objectively fail in the eosio::onerror notification handler: for example by calling eosio_assert( false, "eosio::onerror handler not supported by contract" );.
This is particularly useful for the eosio.msig contract.
If the CPU limit of the eosio.msig account is limited to 0, the real cause of an error encountered during execution of the proposed transaction is not captured by plugins like mongo_db_plugin and state_history_plugin.
Any objective failure of the proposed transaction while executing will trigger a notification of the eosio::onerror action to be sent to the eosio.msig account. However, because the eosio.msig account would not have enough CPU to run the notification handler, it would raise a subjective CPU usage exceeded exception which would cause the deferred transaction to be silently dropped and then blacklisted by the producer attempting to execute the deferred transaction. This process would repeat with all of the remaining active block producers until eventually the deferred transaction expires (with no meaningful error message recorded in the corresponding transaction trace).
Raising the CPU limit of the eosio.msig account would resolve this problem, but without any further changes it exposes the eosio.msig account's CPU resources to being wasted by abusers intentionally executing proposed transactions that are guaranteed to objectively fail. However, if in addition to raising the CPU limit for the account, the eosio.msig contract was also modified to objectively fail on the eosio::onerror notification handler, the failing proposed transaction could retire with a hard_fail status (which could allow capable block explorers to record the original error message of the deferred transaction) rather than with an expired status.

By using the dispatchers generated by the code generator of eosio.cdt v1.6.x (see #265 and #266), all contracts will automatically reject notifications of the eosio::onerror action. So this issue is resolved as of v1.7.x.


Add end-to-end testing for charts using kind/chart-testing

What changes were proposed in this pull request?
This PR adds the ability to run chart-testing against the charts in this repository as discussed in #485. The tool will install the modified chart(s) in an ephemeral kubernetes environment using kind. CircleCI was chosen as the build tool (initially tried with Semaphore). This is just the first step in adopting E2E for helm.
To further this effort, the following should happen:

This repository will need to be setup in CircleCI
Add a templates/tests directory to each chart to perform some sort of check that the components of the chart are operating as expected. chart-testing runs a helm test after the chart installs automatically but will no-op if no tests exist. Currently, all that will be tested now is that the charts install successfully with their default values
Should probably add a ci/test-values.yaml for each chart if we want to change default values to test in CI
Should definitely add a build step to package/sync the charts to the Confluent helm repo

How was this patch tested?
With a CI run, however no charts have been modified yet. You can see the runs here. I have seen issues with multiple charts being tested on a single branch, but we can try doing a version bump to all charts to see if they pass and what remediations might be needed before this is merged so it does not become a blocker.

It looks like @hobbsh hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot

It looks like @hobbsh hasn't signed our Contributor License Agreement, yet.

The purpose of a CLA is to ensure that the guardian of a project's outputs has the necessary ownership or grants of rights over all contributions to allow them to distribute under the chosen licence.
Wikipedia

You can read and sign our full Contributor License Agreement here.
Once you've signed reply with [clabot:check] to prove it.
Appreciation of efforts,
clabot

[clabot:check]

[clabot:check]

@confluentinc It looks like @hobbsh just signed our Contributor License Agreement. :+1:
Always at your service,
clabot

@confluentinc It looks like @hobbsh just signed our Contributor License Agreement. :+1:
Always at your service,
clabot

That's pretty awesome @hobbsh !!
I will take a look at this hopefully this week (Wednesday or Thursday)

That's pretty awesome @hobbsh !!
I will take a look at this hopefully this week (Wednesday or Thursday)

@hobbsh Could you kindly redirect this PR to https://github.com/confluent-helm-charts/cp-helm-charts?

@hobbsh I moved it to https://github.com/confluent-helm-charts/cp-helm-charts/pull/7. Let's continue there. I will try to migrate it to Github actions


v22.1.4 update composer.json

[Current coverage][cc-pull] is 100%

Merging [#565][cc-pull] into [master][cc-base-branch] will not change coverage

@@           master   #565   diff @@
====================================
  Files          28     28
  Lines         439    439
  Methods       111    111
  Messages        0      0
  Branches        0      0
====================================
  Hits          439    439
  Misses          0      0
  Partials        0      0

Powered by Codecov. Last updated by [a4a0a74...c1d54be][cc-compare]
[cc-base-branch]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/compare/a4a0a744f993d403490b39af9402b9fac9bacf40...c1d54bea57ccf84330496102dae79751490e1fc2?src=pr
[cc-pull]: https://codecov.io/gh/eugene-matvejev/battleship-game-api/pull/565?src=pr


[GH-32] Improve the naming and comments for the CLI.

https://github.com/input-output-hk/cardano-node/issues/32
Fixing the names of the CLI arguments and trying to make thing more clear. Small technical debt resolution.

bors r+


Support for Angular 5+

This is a feature request
Additional Notes:
Currently this project does not support Angular 5.0. A build warning appear:
warning<EMAIL_ADDRESS>has incorrect peer dependency "<EMAIL_ADDRESS>Is there any pre-release that does?
What do we need to change to implement the support?

You need to upgrade to 1.2.4 where the peer dependency is fixed 😄  https://github.com/mattlewis92/angular-resizable-element/blob/master/CHANGELOG.md#124-2017-10-21

Thank you!


Callback chapter 13

Why we have the arguments 'oldvalue' , 'initiator' in the method if we don't use it? and how does it know that the first argument will be passed is the target and the second is the value, I think my question is related to the python programming language features.
thanks in advance.

This is not specifically Python, it is a feature of SQLAlchemy. Here is the documentation for this feature: http://docs.sqlalchemy.org/en/latest/orm/events.html#sqlalchemy.orm.events.AttributeEvents.set


Fix non-ascii letters sent to Gateway

Attempts to replace accented chars with ascii counterparts, attempts to apply translit for any foreign chars α or β etc.

I don't think we need this one now that the the other PR is merged.


Action Parameter for "Broadcast Intent" not sent?

I'm trying to send a "Broadcast Intent" when finding a specific beacon, using the Action parameter to identify the beacon. However, while the intent is received by the App "Automagic" without problems there seem to be no parameters attached to the intent, so I cannot distinguish the beacons.
You should also specify the complete name of the intent somewhere. Since I'm not an Android programmer it took me a while to guess the correct name com.somebits.beacon.locator.action.NOTIFY_BEACON_ENTERS_REGION ;-)

for sending intent you need full name, the app does not support any parameters for that intent.
the idea was to trigger some app what support intent receivers.
NOTIFY_BEACON_ENTERS_REGION should not be used as it is an internal action to notify an app logic on beacon entering region, so you do need it. if you want send some intent + parameters, you may use Tasker action , it is very flexible

Unfortunately the Tasker action only seems to work when Tasker is installed.
I only get the message "Tasker application is not installed" when I try to use it :-(
What would be the "full name" of the intent?
Automagic support receiving a "General Broadcast", an example given is "PowerAMP Track" with the action "com.maxmpz.audioplayer.TRACK_CHANGED". Is that something like what beacanloc would send with the Tasker action?

for tasker task you need tasker app installed, that is correct.
the full name is like a package name, so if PowerAmp can change the track only receiving com.maxmpz.audioplayer.TRACK_CHANGED it is enough to use "Broadcast Intent" action, if you need to send additional params, like a track number, only an option is to use the tasker

Automagic can extract additional information from the intent extras. The PowerAMP example lists something called a "bundle" to extract track information. That looks a little bit like something from the TaskerAction.java code. I will try to install Tasker to allow the Tasker intent to be send, maybe I can catch it with automagic ;-)


   fs2020 label not found. KEY_ELEVATOR_TRIM_DISABLED_TOGGLE context PLANE device SaitekX56Throttle

Error message returned from generator:

  fs2020 label not found. KEY_ELEVATOR_TRIM_DISABLED_TOGGLE context PLANE device SaitekX56Throttle

Full text of inputprofile_xxx ([XML textfile]https://github.com/ankurkotwal/MetaRefCard/files/5493999/inputprofile_0903700928.txt))
for X56 Saitek Rhino is attached for further analysis.
CMDR Trium

Thanks for reporting the issue. MRC still doesn't have all the strings from the game but I'm adding them as they pop up. The change will go live shortly (bundling a few updates)

New release is now live with this fix. Thanks again for reporting the bug. I hope you're finding MetaRefCard useful.


[BEAM-7630] add ITs for writing and reading bytes from pubsub

Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---

---
---

Java

Python

---

---
---

Pre-Commit Tests Status (on master branch)

---
Java
Python
Go
Website

Non-portable

Portable
---

---
---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

Run Python PostCommit

Run Python PostCommit

Run Python PostCommit

R: @tvalentyn

Run Python PostCommit

Run Python PostCommit

run python postcommit

The only test failure on this PR is (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... FAIL, which is a known flake unrelated to this change.
@pabloem Could you help merging this PR please?

The only test failure on this PR is (apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT) ... FAIL, which is a known flake unrelated to this change.
@pabloem Could you help merging this PR please?

Okay fair enough. Thanks Valentyn!


TypeError: argument of type 'NoneType' is not iterable

From https://github.com/cquery-project/cquery/wiki/Build#windows
F:\cquery>python waf configure --clang-prefix "C:\Program Files\LLVM\"
Setting top to                           : F:\cquery
Setting out to                           : F:\cquery\build
Checking for 'msvc' (C++ compiler)       : F:\VisualStudio2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64\CL.exe
Checking for header stdio.h              : yes
Checking for clang prefix                : clang prefix not found: "C:\Program Files\LLVM""

F:\cquery>clang
clang.exe: error: no input files

F:\cquery>python waf configure --clang-prefix "C:\Program Files\LLVM"
Setting top to                           : F:\cquery
Setting out to                           : F:\cquery\build
Checking for 'msvc' (C++ compiler)       : F:\VisualStudio2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64\CL.exe
Checking for header stdio.h              : yes
Checking for clang prefix                : C:\Program Files\LLVM
Checking for library clang               : yes
Traceback (most recent call last):
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 118, in waf_entry_point
    run_commands()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 178, in run_commands
    ctx=run_command(cmd_name)
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 169, in run_command
    ctx.execute()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Configure.py", line 85, in execute
    super(ConfigurationContext,self).execute()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 84, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 125, in recurse
    user_function(self)
  File "F:\cquery\wscript", line 254, in configure
    stderr=subprocess.STDOUT).decode()).strip()
  File "F:\Python\lib\subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "F:\Python\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "F:\Python\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "F:\Python\lib\subprocess.py", line 971, in _execute_child
    args = list2cmdline(args)
  File "F:\Python\lib\subprocess.py", line 461, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'NoneType' is not iterable

F:\cquery>python --version
Python 3.6.3 :: Anaconda, Inc.

F:\cquery>clang --version
clang version 5.0.1 (tags/RELEASE_501/final)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin

F:\cquery>python waf configure
Setting top to                           : F:\cquery
Setting out to                           : F:\cquery\build
Checking for 'msvc' (C++ compiler)       : F:\VisualStudio2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64\CL.exe
Checking for header stdio.h              : yes
Checking for clang
Downloading tarball
   destination: build/LLVM-5.0.1-win64.exe
   source:      http://releases.llvm.org/5.0.1/LLVM-5.0.1-win64.exe
Extracting build/LLVM-5.0.1-win64.exe
Traceback (most recent call last):
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 118, in waf_entry_point
    run_commands()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 178, in run_commands
    ctx=run_command(cmd_name)
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 169, in run_command
    ctx.execute()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Configure.py", line 85, in execute
    super(ConfigurationContext,self).execute()
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 84, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "F:\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 125, in recurse
    user_function(self)
  File "F:\cquery\wscript", line 287, in configure
    download_and_extract(CLANG_DIRECTORY, CLANG_TARBALL_URL, CLANG_TARBALL_EXT)
  File "F:\cquery\wscript", line 118, in download_and_extract
    subprocess.call(['7z', 'x', '-o{0}'.format(destdir), '-xr!$PLUGINSDIR', dest])
  File "F:\Python\lib\subprocess.py", line 267, in call
    with Popen(*popenargs, **kwargs) as p:
  File "F:\Python\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "F:\Python\lib\subprocess.py", line 997, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified

@DaanDeMeyer

--clang-prefix breaks with the same error with me as well. I do not get an error when using downloaded clang however. Do you have 7zip installed?

I have, but it is not on my system path. I just added it to the path, but still throwing error:

task in 'testprog' failed with exit status 1181 (run with -v to display more information).txt

Adding 7z to path is only to fix waf with downloaded llvm. I don't know yet why --clang-prefix doesn't work. The txt file seems to only show waf invocations with --clang-prefix. Can you try one without --clang-prefix?

Without the --clang-prefix it worked correclty:
f:\cquery>python waf configure
Setting top to                           : f:\cquery
Setting out to                           : f:\cquery\build
Checking for 'msvc' (C++ compiler)       : F:\VisualStudio2017\VC\Tools\MSVC\14.12.25827\bin\HostX64\x64\CL.exe
Checking for header stdio.h              : yes
Checking for clang
Found tarball at build/LLVM-5.0.1-win64.exe
Extracting build/LLVM-5.0.1-win64.exe

7-Zip 17.00 beta (x64) : Copyright (c) 1999-2017 Igor Pavlov : 2017-04-29

Scanning the drive for archives:
1 file, 94256909 bytes (90 MiB)

Extracting archive: build\LLVM-5.0.1-win64.exe
--
Path = build\LLVM-5.0.1-win64.exe
Type = Nsis
Physical Size = 94256909
Method = LZMA:25
Solid = +
Headers Size = 205253
Embedded Stub Size = 114176
SubType = NSIS-3

Everything is Ok

Files: 292
Size:       874601444
Compressed: 94256909
Checking for library libclang            : yes
copy_or_symlink src=f:\cquery\build\release\lib\LLVM-5.0.1-win64\bin\libclang.dll, dst=f:\cquery\build\release\bin\libclang.dll
Clang includes                           : ['f:\\cquery\\build\\release\\lib\\LLVM-5.0.1-win64\\include']
Clang library dir                        : ['f:\\cquery\\build\\release\\lib\\LLVM-5.0.1-win64\\lib']
'configure' finished successfully (32.419s)

f:\cquery>

Alright, that should allow you to further try out cquery (although the rest of the Windows experience isn't exactly on point yet either, I'm trying to fix it though).

pls update wiki/Build
On Mar 7, 2018 11:18, "DaanDeMeyer"<EMAIL_ADDRESS>wrote:

Alright, that should allow you to further try out cquery (although the
rest of the Windows experience isn't exactly on point yet either, I'm
trying to fix it though).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cquery-project/cquery/issues/499#issuecomment-371252310,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAZaQuZ2DzLehqKUjaB3JAnpLxDkJ-rFks5tcDJugaJpZM4SfqDG
.

I've added a note saying --clang-prefix doesn't work on Windows. Wiki already mentions that 7zip should be installed and in the PATH.

@DaanDeMeyer I know you mentioned adding a note to the wiki, but I don't see any such note (and as a result have found myself here with the same issue)

If you would like to use your own installed clang compiler, instead of running python wad configure you should run:
python waf configure --clang-prefix "C:\Program Files\LLVM\"
python waf build

I could be wrong, but I think what is happening is that the backslash paths aren't being parsed properly, leading to the extra double quote at the end of the path:
Checking for clang prefix                : clang prefix not found: "C:\Program Files\LLVM""

If you switch the direction of the slashes (--clang-prefix "C:/Program Files/LLVM/"), the configure command reports being able to find the clang prefix, but fails with a traceback like the one above:
Checking for clang prefix                : C:\Program Files\LLVM
Checking for library clang               : yes
Traceback (most recent call last):
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 118, in waf_entry_point
    run_commands()
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 178, in run_commands
    ctx=run_command(cmd_name)
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Scripting.py", line 169, in run_command
    ctx.execute()
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Configure.py", line 85, in execute
    super(ConfigurationContext,self).execute()
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 84, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "D:\Desktop\cquery\waf3-2.0.2-b8fa647d13364cbe0c1c8ec06042b54d\waflib\Context.py", line 125, in recurse
    user_function(self)
  File "D:\Desktop\cquery\wscript", line 254, in configure
    stderr=subprocess.STDOUT).decode()).strip()
  File "C:\Users\akbro\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 336, in check_output
    **kwargs).stdout
  File "C:\Users\akbro\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "C:\Users\akbro\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "C:\Users\akbro\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 971, in _execute_child
    args = list2cmdline(args)
  File "C:\Users\akbro\AppData\Local\Programs\Python\Python36-32\lib\subprocess.py", line 461, in list2cmdline
    needquote = (" " in arg) or ("\t" in arg) or not arg
TypeError: argument of type 'NoneType' is not iterable

If python waf configure --clang-prefix "C:\Program Files\LLVM\" does not work.. it should be removed from the wiki

Can you try building with CMake? Instructions are in the wiki.


Updated javadoc of Seq.apply(Integer)

Codecov Report

Merging #2596 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #2596      +/-   ##
============================================
+ Coverage     92.84%   92.85%   +0.01%
  Complexity     5347     5347
============================================
  Files            89       89
  Lines         12740    12740
  Branches       1608     1608
============================================
+ Hits          11828    11830       +2
+ Misses          723      721       -2
  Partials        189      189

Impacted Files
Coverage Δ
Complexity Δ

src/main/java/io/vavr/collection/Seq.java
96.15% <ø> (ø)
46.00 <0.00> (ø)

src/main/java/io/vavr/concurrent/FutureImpl.java
75.18% <0.00%> (+1.50%)
27.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 26c54c3...e81d54b. Read the comment docs.


Rename config.sample.php to config.php

BUG :/

The file name is not a bug, you should manually rename config.sample.php to config.php upon installation.


Discussion: Sponsoring the REST API development + applying for grants to fund ArchiveBox development

I would like to sponsor (pay a developer cash monies in USD) for the development of the API for ArchiveBox. I need this for my digital preservation and ML/NLP work and I suspect a lot of people will end up doing a lot of cool things with it. I would like to talk to @pirate  and get his blessing and solicit feedback from the community on how we could work together to accomplish this as well as what a reasonable rate and time line would be. @pirate  is this something you have the bandwidth and inclination to do? if not could you recommend anyone as tribute? :P
Ideally I see the community coming to consensus on what an initial API would look like in the form of an Issue, that would serve as the scope of work for the commission, a $ amount would be set against that issue and once complete and the PR is accepted the money would be paid to the developer from money in escrow for this task.
Thank you all for your time with this and I look forward to hearing from you.

Hey @djkemmet, this is definitely something I'm interested in and I'm excited that you're willing to contribute towards seeing it happen.
I'm getting ready to write some grant applications for ArchiveBox and I'm hoping to fund part-time development for the next few years. The API is definitely one of the main things I want to build in the next year, and I want to work on ArchiveBox as my main job if I can pull off finding a big sponsor.
The grants I'm aiming for are $150k+ so if your budget for this task is <$2k I wouldn't bother sponsoring this if you cant wait a few months, as I'll likely get around to it anyway. If your needs are more urgent than that or your budget is higher then lets discuss further via chat/call: https://calendly.com/nicksweeting/choose-a-time/
I agree either way we should finalize an API spec and framework choice, the work so far toward that is here: https://github.com/ArchiveBox/ArchiveBox/issues/496#issuecomment-1097310582
Either way you can comment your thoughts on that spec in that issue, and we can continue the discussion about timeline/sponsorship/and your specific needs here.

WOO! So excited to see that grant money might be a possibility! Btw, grant writing is becoming trivially easy with GPT.

heyy @pirate ,
yeah my budget for this was $1000 USD If there's anyone in your network who might be suited for this work and assuming 1000$ is a resonable amount of money to get the API off the ground I would definitely be happy to sponsor someone based on your recommendation. that said, as far as grants go, have you looked at futo? https://futo.org/fellows/

Hello guys!
I can step up to work on this.
@pirate let me know if there's anything to consider while doing grant work for the project and what things to have in mind (like what you mentioned of grants already being discussed with other parties, and guidelines for the code to be more likely to be merged).

Hey @pe1uca thanks for the offer! I'd definitely accept help with grant applications, want to chat on a call sometime? https://calendly.com/nicksweeting/choose-a-time?month=2023-10
A quick update for everyone here, I am currently in the process of setting up a contract with the United Nations to help with their internet archiving needs! That contract will focus directly on ArchiveBox development (keeping it all open source), as well as enhancing it for their specific needs. It's a slow moving bureaucratic process so it may not start for a few more months until we get all the approvals, but it's a major step forward in securing real money for part/full time ArchiveBox development going forward.

Just a call with @pe1uca and we discussed a good approach to get some API functionality before my event sourcing refactor is complete:

create a new app under archivebox/api (separate from core and CLI)
create an endpoint for each archivebox CLI subcommand (e.g. add, remove, list, schedule) and allow it to take config/env variables and CLI parameters as the POST request parameters
call the CLI code directly using the requests parameters, and return the stdout/stderr as the response

Notes:

for long-running requests (e.g. an add that takes 20min), we can return a 200 and use this hack to run the job after the response is returned. later on we will use a proper job queue system, but because I am working on this in my refactor we don't want to depend on this just yet otherwise it will block the entire API development process. if the job is cancelled or times out early, it can be picked up automatically and resumed by an archivebox schedule ONLY_NEW=False job in the background, acting as a janky but cheap/easy background job system until  my refactor is finished
some off-the-shelf django library or snippet should be used to allow users to create and manage API tokens from the UI/CLI to be used with the new API endpoints. I think allowing the user to create named API tokens (that inherit the same permissions as that user) is enough for now, no need to do OAuth2 or JWT

We're also discussing this further on the ArchiveBox chat server here: https://zulip.archivebox.io/#narrow/stream/158-development/topic/REST.20API.20development/near/1166232
@pe1uca is going to take a look at doing this, @djkemmet if you're still willing to sponsor development it would be great to discuss it further between the three of us :)

Will this cover, say, returning a JSON payload of document content via a search? That would be the key thing for me.

Yes, certainly @rcarmo

The new REST API is rapidly evolving on dev! Big thanks to @Brandl for the first PR, keep an eye out for new releases!
https://github.com/ArchiveBox/ArchiveBox/releases/v0.8.0-rc

I'm going to close this issue for now and move these discussions to:

REST API bugs and feature requests here: https://github.com/ArchiveBox/ArchiveBox/issues/496
discuss ArchiveBox grants and funding here: https://zulip.archivebox.io/#narrow/stream/178-funding/topic/grants.20to.20apply.20to


feat: Add o'clock project in the EXTRA PROJECTS folder

done #10

Thanks, @anonimak for your contribution.


[DOCFIX] Update Chinese version of docs/cn/Developer-Tips.md

Update Chinese version of docs/cn/Developer-Tips.md corresponding to modified part of docs/en/Developer-Tips.md

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17081/
Test PASSed.


Docker containers for Hygieia and UI

Ability to run separate Docker containers for the API and the UI, similar to production layouts.
The API is using a tomcat container and the UI is using an nginx container
The docker images are part of the maven compile using spotify's docker plugin
cd api
mvn clean package docker:build

cd ../UI
mvn clean package docker:build

This will create 2 containers for you, hygieia_api and hygieia_ui
You can now create a swarm, connect a docker or managed mongo, to run it all within a standard docker-machine
docker run -h mongo -d --name mongo mongo:latest
docker run -h hygieia_api --link mongo:mongo -P -d --name hygieia_api hygieia_api
docker run -d -P -h hygieia_ui --name hygieia_ui --link hygieia_api:api hygieia_ui

hygieia_api expects a network link / alias called mongo
hygieia_ui expects a network link / alias called hygieia_api

closed by pull #129


Update README.md fixing grammar

Fixes wrong grammar in the readme file.
References
nope
Submitter checklist

[ ] Recommended: Join WireMock Slack to get any help in #help-contributing or a project-specific channel like #wiremock-java
[x] The PR request is well described and justified, including the body and the references
[x] The PR title represents the desired changelog entry
[x] The repository's code style is followed (see the contributing guide)
[x] Test coverage that demonstrates that the change works as expected
[x] For new features, there's necessary documentation in this pull request or in a subsequent PR to wiremock.org

The action "CI / lint-test (pull_request)" fails with

✖ wiremock => (version: "0.2.0", path: "charts/wiremock") > Chart version not ok. Needs a version bump!

I the version bump be including in this PR? What are the steps for this?


Authorization Fails Yet Gridview Displays With No Data

Hi,
I'm authenticating users using AD. The first time user put their credentials and if they're in AD they get into the Serenity App. The users don't have any permissions( in my case), an empty Grid gets displayed with a message "Authorization has been denied for this request".
I would want to display a message similar to "Authorization has been denied" instead of an empty Grid. The Grid should display data only when the user is given permission through User Mgmt. Can anyone help me with this?

Achieved this by checking for permissions in DashboardPage index method.
List<string> permissionList = new List<string>(new string[] { "Administration:Security", "Administration:General", "Administration:Translation" });             var permissions = new Dictionary<string, bool>(StringComparer.OrdinalIgnoreCase);             foreach (var permissionKey in permissionList)             {                 if (Authorization.HasPermission(permissionKey))                     permissions[permissionKey] = true;                 else                     permissions[permissionKey] = false;             }             if (permissions["Administration:Security"]==false && permissions["Administration:General"] == false) {                 return View(MVC.Views.Errors.AccessDenied);             }             else             {                 return View(Name of the View you want to display);             }


Unify + Improve dialects documentation

At the moment each dialect's documentation page is maintained separately. By that often the content seems to be reduced to a minnimum. For example, adding hints on how to find BucketFS user and port #311 need to be added to all pages. Even worse this information must also be reviewd and maintained at all pages in parallel.
This could be simplified by a template system. Therefore we could generate the dialects documents  from templates and add shortcodes like [bucket_fs_upload] that the template renderer would then render to the markdown file.
We could ensure that the generated version is recent by a maven plugin that breaks the build in CI.
This approach would also facilitate to add badges for the adapter capabilities (#363)
Even more we could generate the tables with the supported data type conversions (example) from a more convenient syntax like yaml. Wh did ever sort such a list in alphabetic order by hand, will like this feature ;-)

If things repeat, it might be a good idea to extract them into a central document, such that they can't diverge.

we already have that. Now in the docs portal. But the dialects documentations are at the moment as a full tutorial for setting up the dialect. I think this is quite good. As such it should be complete and also contain all the relevant hints.

We discussed this ticket and decided not to implement a unified generation for the dialects documentation , because they have too many differences to unify them.


apicommunity release 2.3 don’t work

I noticed two problems:

the sign-up token is always invalid:

2023-09-04T11:18:45.621837+02:00 server envkey-api[487878]: {"ts":"2023-09-04T09:18:45.621Z","proc":"main","msg":"api error: email token invalid","status":401,"stack":"Error: email token invalid\n    at Object.handler (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/api_handlers/email_verifications.ts:190:13)\n    at process.skipLogging (node:internal/process/task_queues:96:5)\n    at res (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:866:9)\n    at Object.t.tryHandleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:500:13)\n    at Object.t.handleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:227:7)","elapsed":"5ms"}

the invitation don’t work either:

2023-09-04T10:46:38.969682+02:00 jimbo envkey-api[484712]: {"ts":"2023-09-04T08:46:38.969Z","proc":"main","msg":"api error: not found","status":404,"stack":"Error: not found\n    at Object.t.authenticate (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/auth.ts:123:17)\n    at process.auth (node:internal/process/task_queues:96:5)\n    at Object.t.handleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:147:9)","elapsed":"6ms"}

I used the 2.1.4 release to be able to sign-up, then upgraded the server and invite a coworker, who couldn’t join the organization.
I retried the whole process (sign-up, then invite) with 2.1.4 release and it works well.
I did my first tests last week and I don’t remember if I tested one of the 2.2.x releases too.

Closing for now as we haven't been able to reproduce this one. Are you still having this issue?

I’ll test 2.3.6 as soon as possible.

I upgraded to 2.3.6 and it still fails.
I got a better error log, though:
janv. 10 15:28:44 jimbo envkey-api[52236]: {"ts":"2024-01-10T14:28:44.888Z","proc":"main","msg":"Transaction error:","alert":true,"err":{"message":"You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AS put ON DUPLICATE KEY UPDATE fullKey = put.fullKey, body = put.body, data =...' at line 1", "code":"ER_PARSE_ERROR","errno":1064,"sqlState":"42000","sqlMessage":"You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AS put ON DUPLICATE KEY UPDATE fullKey = put.fullKey, body = put.body, data =...' at line 1","name":"Error","stack":"Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AS put ON DUPLICATE KEY UPDATE fullKey = put.fullKey, body = put.body, data =...' at line 1
    at c.query (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/node_modules/mysql2/promise.js:92:22)
    at Object.execQuery (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db_fns.ts:69:30)
    at Object.executeTransactionStatements (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db_fns.ts:778:12)
    at Object.executeTransactionStatements (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db.ts:189:15)
    at execTransaction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:618:15)
    at Object.t.tryHandleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:662:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.t.handleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:233:7)"},"stack":"Error: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'AS put ON DUPLICATE KEY UPDATE fullKey = put.fullKey, body = put.body, data =...' at line 1
    at c.query (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/node_modules/mysql2/promise.js:92:22)
    at Object.execQuery (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db_fns.ts:69:30)
    at Object.executeTransactionStatements (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db_fns.ts:778:12)
    at Object.executeTransactionStatements (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/db.ts:189:15)
    at execTransaction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:618:15)
    at Object.t.tryHandleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:662:11)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at Object.t.handleAction (/tmp/webpack:/Users/daneschneider/projects/envkey/v2/public/app/api/shared/src/handler.ts:233:7)"}

My database is MariaDB on Debian 12
mariadb --version
mariadb  Ver 15.1 Distrib 10.11.4-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper

@ldidry It looks like you may not be using a compatible DB. EnvKey requires MySQL 8.

Ok, thx.

I confirm that using MySQL 8 makes it work. Thank you 🙂
I guess that you’re unlikely to support PostgreSQL in the future?

We've been happy enough with MySQL for our own purposes so it's not something we're likely to prioritize, but I'd be willing to offer support if a contributor wants to take it on. There's a thread on it here: https://github.com/envkey/envkey/discussions/101

Thx for the link 🙂


Update fence_pre_attrs.js for mermaid language

update fense render for support mermaid.js in md
Type of Change

Build Scripts:  fence_pre_attrs.js
Something else: support mermaid language in code with  environment-local as default style

What issue does this relate to?

mermaid language code is not supported yet

What should this PR do?

add a mermaid interpreter for markdown-it in fence_pre_attrs.js (simply by add class)
add mermaid default style as environment-local for a wrap container instead of original code default dark style

provide just like:

mermaid code:

flowchart TD
    A[Christmas] -->|Get money| B(Go shopping)
    B --> C{Let me think}
    C -->|One| D[Laptop]
    C -->|Two| E[iPhone]
    C -->|Three| F[fa:fa-car Car]

must add mermaid as js or module
personally recomanded:  https://www.jsdelivr.com/package/npm/mermaid
TODO:

make a full markdown-it render for combine existing features like label,other environment style ...

What are the acceptance criteria?

Need add mermaid, the simpliest way : https://www.jsdelivr.com/package/npm/mermaid
Need to run npm test on local before this pr is approved.

👋 Please could you explain why this is not possible using the existing functionality in do-markdownit?

This change as it stands is not going to be merged, I do not wish to add a branch specifically for mermaid in here that bypasses all the existing functionality of the fence_pre_attrs plugin.

It seems it is very possible to use do-markdownit as it stands to have a code block generated with the mermaid class on it:
const md = require('markdown-it')({ langPrefix: '' })
    .use(require('.'), {
        fence_label: false,
        fence_environment: {
            allowedEnvironments: [ 'local' ],
        },
        fence_pre_attrs: false,
        prismjs: false,
    });

console.log(md.render('```mermaid\n[environment local]\ngraph TD\n    A --> B\n```'));

graph TD
    A --> B

You can enable fence_pre_attrs too if you'd prefer the classes on the pre instead:

graph TD
    A --> B

However, if you enable prismjs for syntax highlighting on code blocks, that will interfere with things as it will use the language set on the code block to drive its dedicated language class:

graph TD
    A --> B

Though if you enable prismjs but disable fence_pre_attrs you will get the original mermaid class still:

graph TD
    A --> B

All that to say, I would appreciate a much more thorough description of exactly what situation you're running into that doesn't work currently that you're trying to solve for, and then we can look at how we can build a generic solution to that rather than hastily hard-coding a branch for mermaid.

I agree hard-coding or put it in fence_pre_attrs.js is not a graceful way, just thought a PR has much sincerity than a ISSUE. I'm not a PM : )
In my case, I'm building both a live editor(with editor and previewer, just like digitalocean website has) and a markdown auto-render form-element(contents may generate by code to bulid various contents) used in form.
By adjusting config(lang-prefix:'' and disable fence_pre_attrs prismjs) for markdown-it or do-markdownit, it can render markdown like you provide case 1.

graph TD
    A --> B

In fact, mermaid can't out-boxes html elements(,) in mermaid code which would led to Syntax error in mermaid. So case 2,3,4 in latest comment wouldn't work actually or something I don't know to config?
Adjusting config is also a global config for all markdown elements and feels like it's a hard-coding outside rather than inside. If I want to keep styles like  fence_pre_attrs prismjs, I don't want to have a two markdown render separately and handle mermaid code and other codes in markdown by myself. A default mermaid support is necessary for do-markdownit i thought.
Maybe provide a origin attr for user, or even disable-* attr for not using some of global configured in  specific code

An issue probably would've been better here to actually discuss the problem + desired fix, rather than just opening a haphazard PR.
To clarify your ask here, you want to keep fence_pre_attrs enabled and also prismjs, so that most of your code blocks work as would normally be the case in do-markdownit, but for Mermaid codeblocks specifically, you want to have Prism not run on them, and for the mermaid class to stay on the code element and not be moved by fence_pre_attrs?
So, what you're asking for, assuming that's the case, is a way to exclude specific languages from the prismjs plugin, and a way to exclude specific classes from fence_pre_attrs?

Because not all author always keep passion or have time in open-source module like u, i worry it will be a long time for next fix so I provide my simple way fix it temporarily until next fix.
YES, I mean a way to exclude specific languages from the prismjs plugin, and a way to exclude specific classes from fence_pre_attrs.
After that may need test if [label ] works as I supposed it should be. [label ] doesn't hinder code areas

I leave it with you then to add the logic to handle those exclusions -- the exclusion for prismjs should be simple enough, but the exclusion for fence_pre_attrs might be a bit more complex as it's handling all attributes, not just classes.
Do you want to leave this PR open and work on those changes in this, or would it be better off starting afresh once you've got the changes ready in a new PR?

I close this PR first


Build fails - LICENSE_HEADER is missing

Building on Windows 10 Pro with Java 15 (Zulu-15 including JavaFX):
gradlew clean build
...
> Task :licenseGradle FAILED

FAILURE: Build failed with an exception.

* What went wrong:
A problem was found with the configuration of task ':licenseGradle' (type 'LicenseCheck').
> File 'C:\dev\opensource\monitor\gradle\LICENSE_HEADER' specified for property 'header' does not exist.

Should be fixed with commit 8a0f4828bdf9a083fcdee30384e07760c877b348


Add Fitbit to ADOPTERS.md

Happy to join adopters family :-)

Codecov Report

Merging #399 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #399   +/-   ##
========================================
  Coverage    94.34%   94.34%
========================================
  Files          224      224
  Lines         3288     3288
  Branches       255      255
========================================
  Hits          3102     3102
  Misses         186      186

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb9d6c8...d1e7a30. Read the comment docs.

thanks @apakulov !

thanks @apakulov, this has been merged in 895f281b7cb58559f52fdc1d1eca91f00705bc63.


Basic/type_traits: use __is_identifier for some feature checks

When building on Alpine with Musl __has_feature(is_trivially_constructible) does not work, even though __is_trivially_constructible is defined and __has_trivial_constructor is deprecated. Same for __is_trivially_destructible and __has_trivial_destructor respectively.

@swift-ci smoke test

@swift-ci smoke test


Docker image not using the /data volume for actions_log and packages

Description
Hi, and thanks for gitea. I especially love the new gitea actions!
I've noticed I can't look at old gitea actions logs. The HTTP request fails with the following HTTP body (status 500):
storage open "w-flo/[..]/27/39.log": open /app/gitea/data/actions_log/w-flo/[..]/27/39.log: no such file or directory

And indeed that is expected, because my docker container only has a persistent /data/ volume like the gitea docs recommend, everything else will be dropped along with the container itself when I upgrade gitea.
I could probably set a PATH for [storage.actions_log], or preferably [storage], in app.ini, but I'd say the docker image should be configured to store actions logs in /data by default, along with everything else that should be stored permanently?
I have noticed previously that avatars go missing on a regular basis, so maybe avatars are not stored in /data either. Looking at the possible app.ini keys, maybe packages and attachments are affected as well? Edit: I have checked the container file system, and indeed the /app/gitea/data contains packages, avatars, attachments etc. (but I haven't used these features a lot so I hadn't noticed in the past years). So it's not only about actions_logs.
Repositories and issue trackers are not lost on docker upgrade though, as I have manually set [database] PATH and [repository] ROOT in app.ini. These should probably default to something inside /data as well if that is not the case already.
If the issue is that I should have changed something in app.ini to /data, I think it would be great to mention that in the "Installation with Docker" documentation.
Gitea Version
1.19.3
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Screenshots
No response
Git Version
No response
Operating System
No response
How are you running Gitea?
Official docker image through docker-compose, with some modifications to app.ini (not through environment variables, directly in the file on the volume)
Database
SQLite

Updated the title because I think avatars and `attachments  should work
https://github.com/go-gitea/gitea/blob/5e5740af69cb0f44ff0c1576a2387388d75bb4c2/docker/root/etc/templates/app.ini#L39-L50

Oh, I haven't seen that file. I need to look at how that template file is used. I don't use these settings in my own app.ini.
My installation is a couple of years old, migrated from gogs very early on. The /data volume actually contains some old avatars from 2018 in /data/gitea/data/avatars/, with file names "1", "2" etc. (probably user IDs), but the current avatars appear to be in /app/gitea/data and have different file names that look like a hash.
Are we supposed to add things to our app.ini files manually when the template file changes?
Ideally, I'd set only the things that I really want/need to change in app.ini (or through environment variables), and everything else would use default values appropriate for the docker environment, for example from that template file.

This issue has now basically turned into a feature request (to handle app.ini changes in a better way in the docker container), so the label & issue template are incorrect.
I'm closing the issue, let me know if I should open a new feature request issue about the "automatic docker app.ini upgrading".

automatic docker app.ini upgrading

I don't think so, it is always dangerous to touch an in-use config file.

What's the root problem of this issue?
If packages are stored in /app, the packages would be lost when upgrading the docker instance?

What's the root problem of this issue?
If packages are stored in /app, the packages would be lost when upgrading the docker instance?

In the document, it's recommended to mount /data for the Gitea container, and the default app.ini for docker has APP_DATA_PATH=/data.
But some old (maybe very old) app.ini doesn't have APP_DATA_PATH, so it fallback to APP_WORK_PATH/data(/app/gitea/data).

But in (new) docker, the WORK_PATH should also be "/data", maybe some old docker image do not have so?

But in (new) docker, the WORK_PATH should also be "/data", maybe some old docker image do not have so?

No, the WORK_PATH is /app/gitea
The logs of a fresh instance started by gitea/gitea:1.20.0-rc2:

That's really a mess and pretty dangerous ...... 😭

automatic docker app.ini upgrading

I don't think so, it is always dangerous to touch an in-use config file.

I agree it would be dangerous to change the file if it was user-modified. However, if the only one who creates / changes that file is the docker container entrypoint script, it should be safe, right? The only issue might be gitea itself changing the app.ini sometimes, I'm not sure if that is possible? (Avatar settings in the admin view come to mind, I've never used these checkboxes, do they change the ini?)
So it would have to be "opt-in", so that existing users would not be surprised and have their app.ini files destroyed on upgrade. Also, it should say "This file was generated automatically, don't change it, set config in your docker environment variables instead!" at top of the file. And it shouldn't be stored in /data when the user has opted into that way of managing the config, maybe simply in in /app, so that the user wouldn't see that file when browsing the docker volume. The app.ini file would become an "implementation detail" of gitea in the container, nothing the user needs to look at or change.
So the user basically adds something like this to the docker environment variables (docker-compose.yml in this case):
    environment:
      - USER_UID=1000
      - USER_GID=1000
      - GITEA_REWRITE_APP_INI=true
      - GITEA__security__SECRET_KEY=abcdefgh
      - GITEA__database__DB_TYPE=postgres
      - [....]

The docker entrypoint script would  check if "GITEA_REWRITE_APP_INI" is set to "true", and in that case it would create a new /app/gitea/app.ini based on the current template file from the new docker image and use that when starting gitea. I think that would be a pretty nice way to set up the config when using docker.
For setups that don't set "GITEA_REWRITE_APP_INI", everything stays the way it is and their app.ini files will eventually become out of date (which is quite honestly a problem caused by the gitea user, I should have checked changes to app.ini in the years since I have started using gitea!).
Another important setting that I was missing in my app.ini because it was created > 5 years ago is LOG_SQL  = false, so my logs were quite huge with lots of SQL statements.


Minor edits

Reference Issues/PRs
What does this implement/fix? Explain your changes.

README.md TOC removed
SUPPORTED_MODELS.md TOC removed
Badges bug fixed

Any other comments?

LGTM
@sepandhaghighi

I know that now, we can still traverse to the section with # at the end of the URL, but still, for indexing purposes, we do need the TOC, don't we? to find out what we have in this file.
@sepandhaghighi

I know that now, we can still traverse to the section with # at the end of the URL, but still, for indexing purposes, we do need the TOC, don't we? to find out what we have in this file. @sepandhaghighi

Recently, GitHub implemented its own automatic table of contents, making our manual TOC unnecessary.

Impressive, I thought maybe legacy users(like me :) not you) may not find this out.
I myself didn't see the compressed menu icon.
@sepandhaghighi

another question, as you know we've defined the TOC part as the way we wanted it to be, like referring to the Authors.md file(although it doesn't have a specific part in the README itself) and GitHub doesn't completely make it just like ours, it just extracts some important HTML tags.

Another question, as you know we've defined the TOC part as the way we wanted it to be, like referring to the Authors.md file(although it doesn't have a specific part in the README itself) and GitHub doesn't completely make it just like ours, it just extracts some important HTML tags.
What's your idea here? Updating the readme file to have the exact same TOC(with GitHub's generated one I mean) @sepandhaghighi

It's not my taste.
Increasing the manual portion of a document can lead to a higher number of bugs.


Button

[ ] Investigate focus state revision to differentiate between keyboard focus and mouse focus.

[ ] Investigate focus state revision to differentiate between keyboard focus and mouse focus.

Revising button interactions

[ ] Focus state
[ ] Active state
[ ] Save state / loader

Revising button interactions

[ ] Focus state
[ ] Active state
[ ] Save state / loader

Proposed Button Updates
Below is a screen recording of Stripe's admin in Safari and is the ideal way I'd like us to handle our button interactions. It looks like the focus rings only show when tabbing through the UI or on certain elements like input fields, instead of every time you click on something. However, when viewing the Stripe admin in Chrome, these focus rings show on click for every element. If this is a technical or browser limitation that is fine, but I think I'd prefer whatever we implement to be the same experience across all browsers.

Screen recording here: https://www.dropbox.com/s/fnpealli1au9jme/Stripe-interactions.mov?dl=0

With that being said I also like the focus styling they use. Our current implementation feels too heavy-handed and could be more subtle. I took a stab at redefining these states in Figma.

Figma file here: https://www.figma.com/file/SCZt9jMd7OcTMrNsOFifDp/Sage-Components-WIP?node-id=2828%3A0

Proposed Button Updates
Below is a screen recording of Stripe's admin in Safari and is the ideal way I'd like us to handle our button interactions. It looks like the focus rings only show when tabbing through the UI or on certain elements like input fields, instead of every time you click on something. However, when viewing the Stripe admin in Chrome, these focus rings show on click for every element. If this is a technical or browser limitation that is fine, but I think I'd prefer whatever we implement to be the same experience across all browsers.

Screen recording here: https://www.dropbox.com/s/fnpealli1au9jme/Stripe-interactions.mov?dl=0

With that being said I also like the focus styling they use. Our current implementation feels too heavy-handed and could be more subtle. I took a stab at redefining these states in Figma.

Figma file here: https://www.figma.com/file/SCZt9jMd7OcTMrNsOFifDp/Sage-Components-WIP?node-id=2828%3A0


Twitter profile username not detected and marked as invalid

@Nishantjain10  Yeah that happens because of these reasons, we have listed them on footer of the webpage

@Nishantjain10 retweets won't count, i just saw your Twitter, saw that you haven't tweeted recently
After this username error you did tweet your GitHub wrapped, so check it it'll work now, but will generate result only on that single tweet

Thank you, aadarsh; I had overlooked the footer section! Also, it is now operational and generating responses from a single tweet! This project is fantastic.


Adding DETA to 🤗 Transformers

Hi DETA authors,
As this work is very nice and it builds upon DETR and Deformable DETR, both of which are available in 🤗 Transformers, it was relatively straightforward to implement DETA as well (as the only difference is a tweak in the loss function + postprocessing).
Here's a notebook that illustrates inference with DETA models: https://colab.research.google.com/drive/1epI4ejrD0dbrSR9vRRhEPE7duoALqIk9?usp=sharing.
Now I'd also like to make a fine-tuning tutorial for people, illustrating how to fine-tune DETA on a custom dataset. For that I'm using my original DETR fine-tuning tutorial, and tweaking it for DETA. However here I got a question; I'm fine-tuning on the "balloon" dataset which only consists of 1 class (balloon). However during inference, I'm getting an error stating that that "topk is out of range". This is because of this line which seems to select the top 10,000 scores, however when you're fine-tuning on a single class, then the number of queries * number of classes = 300 * 1 = 300. Hence this is smaller than 10,000 => so was wondering what the recommendation here is when fine-tuning on a dataset with only a single class (or more generally, for any custom dataset).
Also, I'm currently hosting the DETA checkpoints on my personal username on HuggingFace:

DETA-resnet-50
DETA-swin-large

It would be cool if you could create an organization on the 🤗 Hub and host the checkpoints there (or under your own personal username if you prefer so). This way, you can also write model cards (READMEs) for those repositories etc. It seems there's already an org for the UT-data-bootcamp, but not sure we should host the checkpoints there.
Let me know what you think!
Open-sourcely yours,
Niels
ML Engineer @ HF

Hi Niels,
Thanks for this effort, I'm delighted to hear about it! Also glad to hear that it was relatively straightforward to implement.

Hence this is smaller than 10,000 => so was wondering what the recommendation here is when fine-tuning on a dataset with only a single class (or more generally, for any custom dataset).

This preNMS topk is important because it reduces the number of predictions fed into NMS. If the number of predictions is too high, NMS becomes too slow. In your case of 300 predictions, NMS runs fast enough and thus this topk step is not needed and can be removed. In general for custom dataset, it's fine to run something like score.topk(min(10000,len(score))).

It would be cool if you could create an organization on the 🤗 Hub and host the checkpoints there.

Cool! I've uploaded our models to three model repos at https://huggingface.co/jozhang97/deta-resnet-50 and https://huggingface.co/jozhang97/deta-swin-l  and https://huggingface.co/jozhang97/deta-swin-l-o365 Is this what you mean?
Let me know if there is anything else I can do to help.

Cool, I tried it by fine-tuning DETA-resnet-50 with the exact same training hyperparameters as my DETR tutorial (300 steps with a learning rate of 1e-4 for the Transformer and 1e-5 for the backbone, weight decay of 1e-4 and gradient clipping of 0.1), this is giving me the following result on one of the validation images:

So this might still need some tweaking (the results after fine-tuning DETR looked a lot better, see the bottom of this notebook). So this might have to do with postprocessing settings, or training settings.
However it seems basic training works, so I'll first integrate the model in the library, and you can then perhaps go over my fine-tuning tutorial and see if some things can be improved, if you're up for that of course.

Ahh yes, I reckon its due to the NMS postprocessing. NMS is not ideal in crowded scenes (see SoftNMS paper).
Hopefully it can be quickly fixed by tweaking the NMS box threshold at this line. If not, perhaps NMS variants, like SoftNMS, would do better. I'd be happy to take a stab at this.

Hi @NielsRogge , thanks for making the tutorial! A quick note on the demo results: DETA used sigmoid activation for classification (see here), while the DETR visualization code above seems using softmax. Please make sure this is changed accordingly.
Best,
Xingyi

Hi @xingyizhou,
Yes I'm aware of that (the author of Conditional DETR said it to me when porting Conditional DETR to 🤗 Transformers 😄 ). The post_process_object_detection method in the notebook above is using sigmoid as can be seen here. I've removed the softmax part from the notebook since it wasn't used anyway.

Hi @xingyizhou @jozhang97,
I'd like to merge DETA into HuggingFace Transformers, just wondering at which organization we can host the checkpoints. I see in the thread above that you uploaded the original checkpoints to your personal account, but was wondering where you'd like the HF-compatible DETA checkpoints to be hosted. Typically, they are hosted as part of an organization (like University of Texas).
Kindly let me know!
Kind regards,
Niels

Hi @NielsRogge
Lets use my personal one. I noticed you sent a PR today, is there anything I need to do for that?

Hi,
We've created a demo for DETA: https://huggingface.co/spaces/hysts/DETA. But the model often has low confidences even for seemingly easy examples, the results don't look that impressive despite the 63 AP on COCO (YOLO models for instance would recognize all objects in the demo images above), could you take a look?


The code for GRAD algorithm

Thanks for your great work in ICLR 2024, I am quite interested in your work. As I want to compare the performance of RL and LLM, Could you please also share the code for the GRAD algorithm?

Please refer to https://github.com/zerongxi/graph-sdc


Rights for screenshots

Hi André,
I am the author of Functional Programming in C# and I'd like to request permission to include a few screenshots of rxmarbles.com in the chapter on RX, to explain some of the operators.
Credit will be given by including the link to the site and your name in a footnote accompanying the first marble diagram that appears. I hope you're ok with this; let me know either way.
Kind regards,
EB

Sure, I'm ok with it.

Excellent, Thank you!


It would be nice, if the Wiki page could be updated accordingly. Currently, it reads to me like, that no additional steps are needed.


Supergalactic system integrated with boost::units

coordinate/supergalactic.hpp updated to work with unit integrated geometric coordinate system.
closes #49

Codecov Report

:exclamation: No coverage uploaded for pull request base (gsoc19@940a00d). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             gsoc19      #50   +/-   ##
=========================================
  Coverage          ?   56.88%
=========================================
  Files             ?        7
  Lines             ?      225
  Branches          ?       97
=========================================
  Hits              ?      128
  Misses            ?        0
  Partials          ?       97

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 940a00d...9df6052. Read the comment docs.

Codecov Report

:exclamation: No coverage uploaded for pull request base (gsoc19@38f408f). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             gsoc19      #50   +/-   ##
=========================================
  Coverage          ?   56.88%
=========================================
  Files             ?        7
  Lines             ?      225
  Branches          ?       97
=========================================
  Hits              ?      128
  Misses            ?        0
  Partials          ?       97

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38f408f...8d76168. Read the comment docs.


Fix indent issue in go tutorial.

Fix an indent issue in go example code.
before:

after:

@kvii Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@kvii Thank you for signing the Contributor License Agreement!


KeypadInputManager's SendKey callback is never called.

Problem
We have two implementations of emberAfKeypadInputClusterSendKeyCallback in the tree:
bool emberAfKeypadInputClusterSendKeyCallback(chip::app::Command * commandObj, unsigned char)

in src/app/clusters/keypad-input-server/keypad-input-server.cpp and
emberAfKeypadInputClusterSendKeyCallback(EmberAfKeypadInputCecKeyCode keyCode)

in examples/tv-app/linux/include/keypad-input/KeypadInputManager.cpp.
The codegen, if it had that command enabled (which it does not) would call the two-arg version, which just sends a DefaultResponse and does nothing.  It definitely does not call into KeypadInputManager in any way, which is what I would expect it to do.
Proposed Solution

Enable the SendKey command in the tv-app zap configuration.
Make the SDK-provided cluster callback call the app-provided callback.
Add a CI test to make sure this all works so we don't break it again.

@lazarkov

This issue is being resolved under PR

Looks resolved?

Yes, this looks fixed now.


Config does not need system

Previous version of example does not work.
Resulted in:
{:error, "The requested resource /2010-04-01/Messages.json was not found", 404}

Thank you! :star: :star: :star:


[POC] fix: improve asset page ui and accounts handling

Description

Disable the opportunities if no accounts can be used for this asset
Fix load more buttons spacing on transaction history
Replace button by a simple text if no transactions inside the history
Kick borders from card headers if there are no bodies (it was weird looking having a border going outside of the container)

Pull Request Type

[ ] :bug: Bug fix (Non-breaking Change: Fixes an issue)
[ ] :hammer_and_wrench: Chore (Non-breaking Change: Doc updates, pkg upgrades, typos, etc..)
[ ] :nail_care: New Feature (Breaking/Non-breaking Change)

Issue (if applicable)

closes #7028
Risk

High Risk PRs Require 2 approvals

What protocols, transaction types or contract interactions might be affected by this PR?

Testing

Go to any asset page (I used BTC and AVAX)
Select a wallet supporting the asset
Look at the opportunities, account rows, history list and compare them with the screenshots I sent in the description/read the description
Do the same but with an account not supporting BTC/AVAX and compare them with the screenshots I sent
Overall, if you feel like the UI is not good yet, please feel free to send any feedbacks!

Engineering
n/a

Operations
n/a

Screenshots (if applicable)
Enable opportunities

Disabled opportunities

Load more button before

Load more button after

Load more button  with transactions (and fixed the spacing)

Before:

No accounts before

Not accounts after

@shapeshift/product I would like to have your validation on these fixes, particularly on:

The gray filter logic on the opportunities if there are no accounts supporting this asset
the You have no accounts for this asset, so staking opportunities are currently unavailable. sentence of the hint
The history "no transaction" logic

the You have no accounts for this asset, so staking opportunities are currently unavailable. sentence of the hint

Perfect

History tx logic is a great fix.


home-manager doesn't install (19.03)

$ nix-channel --list
home-manager https://github.com/rycee/home-manager/archive/release-19.03.tar.gz

$ nix-shell '<home-manager>' -A install

Creating initial Home Manager generation...

these derivations will be built:
  /nix/store/fqzj42h4rij34hfaqi4gf5xj5sppbsgr-home-manager.drv
  /nix/store/cp13s0sd0zrn9qya40m62avc5zw60fq2-home-manager-path.drv
  /nix/store/p0wvrirzwif780spm7inki5f86as6sar-activation-script.drv
  /nix/store/franayimpmjmifqv9fb83bzw5gbbqkby-home-manager-generation.drv
building '/nix/store/fqzj42h4rij34hfaqi4gf5xj5sppbsgr-home-manager.drv'...
install: creating directory '/nix/store/gz7lb9grv8j8qi65h20c4n9zmd3fv0in-home-manager'
install: creating directory '/nix/store/gz7lb9grv8j8qi65h20c4n9zmd3fv0in-home-manager/bin'
'/nix/store/6amj4gzq7vk8b2fsqi93km7saga832iv-home-manager' -> '/nix/store/gz7lb9grv8j8qi65h20c4n9zmd3fv0in-home-manager/bin/home-manager'
building '/nix/store/cp13s0sd0zrn9qya40m62avc5zw60fq2-home-manager-path.drv'...
collision between `/nix/store/ghzg4kg0sjif58smj2lfm2bdvjwim85y-gcc-wrapper-7.4.0/bin/ld.gold' and `/nix/store/0y7jmqnj48ikjh37n3dl9kqw9hnn68nq-binutils-2.31.1/bin/ld.gold'
builder for '/nix/store/cp13s0sd0zrn9qya40m62avc5zw60fq2-home-manager-path.drv' failed with exit code 25
cannot build derivation '/nix/store/franayimpmjmifqv9fb83bzw5gbbqkby-home-manager-generation.drv': 1 dependencies couldn't be built
error: build of '/nix/store/franayimpmjmifqv9fb83bzw5gbbqkby-home-manager-generation.drv' failed

Hmm, seems that something you are installing are causing a collision between gcc-wrapper and binutils. Are you having an existing configuration? That is, do you have a ~/.config/nixpkgs/home.nix file?

Yes. I was trying to migrate home-manager at first from 18.09 to 19.03.

I thought collisions where only warnings. I read that somewhere.
Anyway the problem was caused because I had binutils-unwrapped in my home.nix for some reason.
Thanks.


CI: build images

Build a set of images from the cross of cross of KUBE_RUNTIME={docker,cri-containerd} and KUBE_NETWORK={weave,bridge}, build both BIOS and EFI images in all cases.
WIP while I play with the CI setup and understand the various interactions of workspaces, docker image cache, artifact publishing from multiple jobs etc.
The previous used of /workspace/images is switched to /workspace/packages to free up images for the VM images.

Hrm, 1 successful check (which is the dco check). Either Circle is having a hiccup or I've somehow managed to disable it...

Hrm, I buggered the syntax, so it errorred out but didn't say anything. Nifty.
The webpage says:
We weren't able to start this workflow.
The following jobs are unreachable: images
images: requires job "build-image-docker-bridge" but "build-image-docker-bridge" is not part of this workflow.
images: requires job "build-image-docker-weave" but "build-image-docker-weave" is not part of this workflow.
images: requires job "build-image-cri-containerd-weave" but "build-image-cri-containerd-weave" is not part of this workflow.
images: requires job "build-image-cri-containerd-bridge" but "build-image-cri-containerd-bridge" is not part of this workflow.
For more examples see the Workflows documentation.

That's something to watch for!

Previously the CI was taking 3-3½ minutes. So far in my iterations here it looks to be taking more like 7-10 minutes with parallelism of 4. We are building 4 packages followed by 4 images, so more parallelism seems unlikely to help much.

OK, I think this is about ready. I'll keep #9 open for now in case review requires retesting.

Some times observed during testing.
Package builds (in parallel up to 4):

Package
NOP (unchanged, pulled from hub)
Built

kubelet
02:52
11:18

kubernetes-docker-image-cache-common
00:51
01:30

kubernetes-docker-image-cache-control-plane
01:27
02:06

cri-containerd
00:31
02:10

I didn't do lots of measurements and there's some variance (mostly due to the network traffic I expect).
Image building, up to 4 in parallel, each depends on the relevant package builds:

↓ Runtime / Network →
Weave
Bridge

Docker
02:56
02:44

CRI
02:04
03:26

Again some variance.
Push to hub takes ~2:00 if it pushes both kubelet and cri-containerd and ~1:30 if it pushes neither (it still does a bunch of setup and imports the images even if there turns out to be nothing to push).
If all 4 packages need building then the elapsed time was around 24:30, without anything needing to be built was ~8:10, I expect this will vary depending on the scheduling of the parallel jobs and which packages have actually changed.
Without the parallelism I'd expect the full nop run to take ~20 mins and a full build more like ~30 mins so I think splitting into jobs like this is worthwhile.


Fixed generating Metal shaders that make use of fragment barycentric coords

The title says all.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Is there any forward progress on this PR?

If DecorationNoPerspective is allowed this line https://github.com/KhronosGroup/SPIRV-Cross/blob/main/spirv_msl.cpp#L11981 should not be here.
Not exactly sure where center_perspective would come from.

Are you seeing fragment shaders with barycentric inputs that have one of the interpolation decorations? What app/game?
My app, and that's enough for me: https://www.youtube.com/watch?v=_omNIB8q4vM

I'm all ears... How do you suggest this could be fixed?

@cdavis5e @rcaridade145 @HansKristian-Work
It's been more that one year since this issue was brought up, and no fix, and no work or discussion for my PR. This is frustrating, this makes this utility unusable albeit for generating GLSL and very outdated and limited MSL or HLSL, despite specifying latest generator profiles set for both (MSL and HLSL). All what I got was that my code was called "wrong", but it indeed works and fixes a major issue, at least with my case, and I have example renderings to show. In the end I discovered that my code was not exactly "wrong", but it was a hack to an actually extremely broken MSL generator.
Allow me to elaborate on that in more detail:
Let take this very simple HLSL code:
struct PSInput
{
    linear float3 baryCoord : SV_BARYCENTRICS0;
    noperspective float3 baryCoord1 : SV_BARYCENTRICS1;
};

float4 PSMain(PSInput input) : SV_TARGET
{
	float3 baryCoord = input.baryCoord;
	float3 baryCoord1 = input.baryCoord1;

    [[branch]] if (baryCoord1.x > 0.1)
    {
        return float4(baryCoord1.x, baryCoord1.y, 1.0f - baryCoord1.x - baryCoord1.y, 1.0f);
    }

    return float4(baryCoord.x, baryCoord.y, 1.0f - baryCoord.x - baryCoord.y, 1.0f);
}

While it's useless shader and doesn't make any sense, but it serves as test case for MSL and HLSL generator in SPV-Cross.
Now perspective and non-perspective correct (linear and noperspective respectively in the code above) barycentrics can co-exist in the same fragment shader in GLSL, HLSL and MSL. Now when this code is compiled with DXC with this command:
dxc .\test.hlsl -T ps_6_1 -E PSMain -O3 -spirv -fspv-target-env="universal1.5"
Then this spirv disassembly is generated for reference:
; SPIR-V
; Version: 1.5
; Generator: Google spiregg; 0
; Bound: 36
; Schema: 0
               OpCapability Shader
               OpCapability FragmentBarycentricKHR
               OpExtension "SPV_KHR_fragment_shader_barycentric"
               OpMemoryModel Logical GLSL450
               OpEntryPoint Fragment %PSMain "PSMain" %2 %3 %out_var_SV_TARGET
               OpExecutionMode %PSMain OriginUpperLeft
               OpSource HLSL 610
               OpName %out_var_SV_TARGET "out.var.SV_TARGET"
               OpName %PSMain "PSMain"
               OpDecorate %2 BuiltIn BaryCoordKHR
               OpDecorate %3 BuiltIn BaryCoordNoPerspKHR
               OpDecorate %out_var_SV_TARGET Location 0
      %float = OpTypeFloat 32
    %float_1 = OpConstant %float 1
%float_0_100000001 = OpConstant %float 0.100000001
    %v3float = OpTypeVector %float 3
%_ptr_Input_v3float = OpTypePointer Input %v3float
    %v4float = OpTypeVector %float 4
%_ptr_Output_v4float = OpTypePointer Output %v4float
       %void = OpTypeVoid
         %13 = OpTypeFunction %void
       %bool = OpTypeBool
          %2 = OpVariable %_ptr_Input_v3float Input
          %3 = OpVariable %_ptr_Input_v3float Input
%out_var_SV_TARGET = OpVariable %_ptr_Output_v4float Output
       %uint = OpTypeInt 32 0
     %uint_0 = OpConstant %uint 0
     %PSMain = OpFunction %void None %13
         %17 = OpLabel
         %18 = OpLoad %v3float %2
         %19 = OpCompositeExtract %float %18 0
         %20 = OpCompositeExtract %float %18 1
         %21 = OpLoad %v3float %3
         %22 = OpCompositeExtract %float %21 0
         %23 = OpCompositeExtract %float %21 1
               OpSelectionMerge %24 None
               OpSwitch %uint_0 %25
         %25 = OpLabel
         %26 = OpFOrdGreaterThan %bool %22 %float_0_100000001
               OpSelectionMerge %27 DontFlatten
               OpBranchConditional %26 %28 %27
         %28 = OpLabel
         %29 = OpFSub %float %float_1 %22
         %30 = OpFSub %float %29 %23
         %31 = OpCompositeConstruct %v4float %22 %23 %30 %float_1
               OpBranch %24
         %27 = OpLabel
         %32 = OpFSub %float %float_1 %19
         %33 = OpFSub %float %32 %20
         %34 = OpCompositeConstruct %v4float %19 %20 %33 %float_1
               OpBranch %24
         %24 = OpLabel
         %35 = OpPhi %v4float %31 %28 %34 %27
               OpStore %out_var_SV_TARGET %35
               OpReturn
               OpFunctionEnd

Now SPIRV-Cross will generate this GLSL correctly:
#version 450
#extension GL_EXT_fragment_shader_barycentric : require
#if defined(GL_EXT_control_flow_attributes)
#extension GL_EXT_control_flow_attributes : require
#define SPIRV_CROSS_FLATTEN [[flatten]]
#define SPIRV_CROSS_BRANCH [[dont_flatten]]
#define SPIRV_CROSS_UNROLL [[unroll]]
#define SPIRV_CROSS_LOOP [[dont_unroll]]
#else
#define SPIRV_CROSS_FLATTEN
#define SPIRV_CROSS_BRANCH
#define SPIRV_CROSS_UNROLL
#define SPIRV_CROSS_LOOP
#endif

layout(location = 0) out vec4 out_var_SV_TARGET;

void main()
{
    vec4 _35;
    do
    {
        SPIRV_CROSS_BRANCH
        if (gl_BaryCoordNoPerspEXT.x > 0.100000001490116119384765625)
        {
            _35 = vec4(gl_BaryCoordNoPerspEXT.xy, (1.0 - gl_BaryCoordNoPerspEXT.x) - gl_BaryCoordNoPerspEXT.y, 1.0);
            break;
        }
        _35 = vec4(gl_BaryCoordEXT.xy, (1.0 - gl_BaryCoordEXT.x) - gl_BaryCoordEXT.y, 1.0);
        break;
    } while(false);
    out_var_SV_TARGET = _35;
}

But then it won't for HLSL using this command:
.\spirv-cross.exe .\test.spirv --hlsl --shader-model 61
And it's giving this confusing error:
SPIRV-Cross threw an exception: Unsupported builtin in HLSL: 5286
???? 🤔 ????
Also when I try to generate MSL using this command:
.\spirv-cross.exe .\test.spirv --msl --msl-version 30100
It gives this error:
SPIRV-Cross threw an exception: Cannot declare both BaryCoordNV and BaryCoordNoPerspNV in same shader in MSL.
Well how do you know that? There's nothing in the Metal specs (3.1 here as specified by the profile) that prevents you from doing that. A prove of that is that Apple's own metal-shaderconverter can convert the DXIL generated from DXC and render it correctly:
dxc .\test.hlsl -T ps_6_1 -E PSMain -O3 -Fo test.dxil
And then:
metal-shaderconverter.exe .\test.dxil --validateAll -o test.mil
And it renders correctly.
Until these are fixed for both HLSL and MSL, SPIRV-Cross should be called SPIRV-GLSL.

This fell off the radar. I'll have a look next week.

I implemented HLSL barycentrics here https://github.com/KhronosGroup/SPIRV-Cross/pull/2391
But MSL is a broken language that cannot support Vulkan barycentrics.
Two problems.

PerVertexKHR cannot be implemented in any reasonable way.

Well how do you know that?

Because Metal shader compiler rejects:
#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct main0_out
{
    float3 value [[color(0)]];
};

struct main0_in
{
    float3 gl_BaryCoordEXT [[barycentric_coord, center_perspective]];
    float3 gl_BaryCoordNoPerspEXT [[barycentric_coord, center_no_perspective]];
};

fragment main0_out main0(main0_in in [[stage_in]])
{
    main0_out out = {};
    out.value = in.gl_BaryCoordEXT + in.gl_BaryCoordNoPerspEXT;
    return out;
}

Z:/home/maister/git/SPIRV-Cross/test.msl:14:12: error: declaration with attribute 'barycentric_coord' already specified
    float3 gl_BaryCoordNoPerspEXT [[barycentric_coord, center_no_perspective]];
    ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
Z:/home/maister/git/SPIRV-Cross/test.msl:13:12: note: previous declaration with attribute 'barycentric_coord' here
    float3 gl_BaryCoordEXT [[barycentric_coord, center_perspective]];

The reason Apple's DXIL conversion can work is because they bypass the language they expose to developers and emit IR directly. That's why they can make it work, but we cannot.

I think in https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf
They describe what seems to be a solution to that, if I'm not mistaken, starting page 113.

I think in https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf They describe what seems to be a solution to that, if I'm not mistaken, starting page 113.

I see what you're saying, but the problem with that is that there cannot be more than one [[stage_in]] parameter in a shader function. Presumably, the two structures, FragmentInput0 and FragmentInput1, are for two different fragment shaders. Good try, though.

The reason Apple's DXIL conversion can work is because they bypass the language they expose to developers and emit IR directly. That's why they can make it work, but we cannot.

I'm starting to wonder if MoltenVK really should decompile to HLSL instead of MSL and use Apple's D3D shader converter.  Supporting tessellation would be a problem with that route, however, since AFAIK SPIRV-Cross does not yet support HLSL output from tessellation shaders.
I've also seriously considered building a shader converter of my own that takes SPIR-V and emits AIR directly. Then we could sidestep all the silly restrictions that Apple places on MSL. But that's a lot more work--I'd be starting from scratch.

struct main0_in
{
    float3 gl_BaryCoordEXT [[barycentric_coord, center_perspective]];
    float3 gl_Position [[position]]; // Fragment's position
};

Given this  can't we use the z component to manually calculate

I fixed Centroid/Sample decorations on bary coords now, so I don't think there is anything further that can be done before MSL improves.


markdown.suggest.paths.includeWorkspaceHeaderCompletions default should be onSingleOrDoubleHash

Testing #174884
Could we change the default of markdown.suggest.paths.includeWorkspaceHeaderCompletions to be onSingleOrDoubleHash?
Can we influence the sorting in the suggest widget such that the current file suggestions are sorted to the top?
Or at least that they have a different icon?
My concern is that with the other default this cool feature will be very undiscoverable.
What do you think?

There’s a cost to computing these suggestions, especially the first time before we have cached them. These suggestions can also result in a lot of extra suggestion item
that may confuse users if they aren’t expecting them
That’s why we require a more explicit trigger of these completions by default, while also giving users a setting to enable them more easily


Old versions of example data

The old versions of the example data files do not need to be tracked in the repository and unnecessarily impact the storage quote.  These can be removed using https://rtyley.github.io/bfg-repo-cleaner/

This appears to be impossible with LFS.  See https://github.com/github/git-lfs/issues/1101.  I think we should re-open if I am wrong or can find another solution...

I agree, i would really like to delete old git-lfs stored files from repo's


[local volume provisioner]temp disk may be corrupted if it's used by two drivers or applications.

What happened:
Since for the temp disk, it may be used by other applications or driver, e.g. blob csi driver, if both drivers or applications are writing data to /dev/sdb1 temp disk, that disk would be corrupted easily.
Events:
  Type     Reason       Age               From               Message
  ----     ------       ----              ----               -------
  Normal   Scheduled    22s               default-scheduler  Successfully assigned default/deployment-localdisk-6f95f4f858-w4srx to aks-nodepool1-40942288-vmss000003
  Warning  FailedMount  6s (x6 over 22s)  kubelet            MountVolume.MountDevice failed for volume "local-pv-4b838508" : local: failed to mount device /dev/sdb1 at /var/lib/kubelet/plugins/kubernetes.io/local-volume/mounts/local-pv-4b838508 (fstype: ext4), error format of disk "/dev/sdb1" failed: type:("ext4") target:("/var/lib/kubelet/plugins/kubernetes.io/local-volume/mounts/local-pv-4b838508") options:("defaults") errcode:(exit status 1) output:(mke2fs 1.44.1 (24-Mar-2018)
/dev/sdb1 is mounted; will not make a filesystem here!
)

Events:
  Type     Reason       Age               From               Message
  ----     ------       ----              ----               -------
  Normal   Scheduled    19s               default-scheduler  Successfully assigned default/deployment-localdisk-6f95f4f858-zmbkd to aks-nodepool1-40942288-vmss000003
  Warning  FailedMount  4s (x6 over 19s)  kubelet            MountVolume.MountDevice failed for volume "local-pv-4b838508" : local: failed to mount device /dev/sdb1 at /var/lib/kubelet/plugins/kubernetes.io/local-volume/mounts/local-pv-4b838508 (fstype: ext4), error format of disk "/dev/sdb1" failed: type:("ext4") target:("/var/lib/kubelet/plugins/kubernetes.io/local-volume/mounts/local-pv-4b838508") options:("defaults") errcode:(exit status 1) output:(mke2fs 1.44.1 (24-Mar-2018)
/dev/sdb1 is apparently in use by the system; will not make a filesystem here!
)

# echo a > b
bash: b: Structure needs cleaning

workaround
run umount /mnt on the agent node
What you expected to happen:
How to reproduce it:
Anything else we need to know?:
Environment:

Kubernetes version (use kubectl version):
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Others:

Option#2: discover temp(/dev/sdb1) disk

to make sure temp disk is not used, run following command to unmount temp disk first
kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-volume-drivers/master/local/umount-mnt.yaml

kubectl apply -f https://raw.githubusercontent.com/Azure/kubernetes-volume-drivers/master/local/local-pv-provisioner-tempdisk.yaml


PDT-366 Use mmapped file for Redis circuit breaker

This changes the cacheops circuit breaker to share its "circuit breaker is open" flag with the one used by the core Django cache's circuit breaker as implemented in https://github.com/HearstCorp/rover/pull/909.

@nicwolff
Would be great to have tests for this.

Not sure we can test the IPC but I'll take a look at mocking out the redis-py call to raise TimeoutError and verifying that the flag gets set.

No luck so far on unit tests for setting the flag. We haven't added any tests for our previous changes to cacheops either. @mpmlj can you OK this? @keithmeng @nmohoric @rfein-hearst can you CR this as well? Thanks.

from what I can tell looks ok, but would recommend Rich or Nick to approve over me

^^ @rfein-hearst @nmohoric Can I get one or two more CRs on this?
Note that this and https://github.com/HearstCorp/rover/pull/909 won't work as intended in EC2 until we add something to the deploy scripts (or the Rover code?) to create /tmp/redis_circuit_breaker

why not have your code create it? /tmp should be writeable. Why introduce an additional dependency?

@rfein-hearst OK, it creates the CB mmap file now if it's not found.

:+1:

@rfein-hearst Thanks – can you approve this PR? And https://github.com/HearstCorp/rover/pull/909 too.

Merged, will get tested with https://github.com/HearstCorp/rover/pull/909