Mocked test environment instruction issues

I want to write a PR for this library. I'm struggling to install from a git clone.
I did a git clone. Then I created a virtual env and tried pip install ./ at both the root of the repo, and awswrangler subdirectory.
$ pip3 install -e ./aws-sdk-pandas/
ERROR: File "setup.py" or "setup.cfg" not found. Directory cannot be installed in editable mode: /home/ec2-user/environment/scratch/timestream-null/aws-sdk-pandas
(A "pyproject.toml" file was found, but editable mode currently requires a setuptools-based build.)

$ pip3 install -e ./aws-sdk-pandas/awswrangler/
ERROR: File "setup.py" or "setup.cfg" not found. Directory cannot be installed in editable mode: /home/ec2-user/environment/scratch/timestream-null/aws-sdk-pandas/awswrangler

contributing.md says:

Then run the command bellow to install all dependencies:
poetry install --extras "sqlserver oracle sparql deltalake"

Well I've never used poetry before, so that's not installed. Can we add it to the steps here?
I'm guessing it's just pip3 install poetry. Obviously this is such a trivial change that it's not worth writing a PR for just that. But if we're going to make other changes to these instructions (see below), we should add that.
So now I run:
poetry install --extras "sqlserver oracle sparql deltalake"

Then ./validate.sh, which failed.
awswrangler/oracle.py:43: error: Name "oracledb.Connection" is not defined  [name-defined]
awswrangler/oracle.py:44: error: "Module oracledb" does not explicitly export attribute "Connection"  [attr-defined]
awswrangler/oracle.py:58: error: Name "oracledb.Cursor" is not defined  [name-defined]
awswrangler/oracle.py:74: error: Name "oracledb.Cursor" is not defined  [name-defined]
awswrangler/oracle.py:82: error: Name "oracledb.Cursor" is not defined  [name-defined]
awswrangler/oracle.py:117: error: Name "oracledb.Connection" is not defined  [name-defined]
awswrangler/oracle.py:178: error: "Module oracledb" does not explicitly export attribute "makedsn"  [attr-defined]
awswrangler/oracle.py:180: error: "Module oracledb" does not explicitly export attribute "connect"  [attr-defined]
awswrangler/oracle.py:193: error: Name "oracledb.Connection" is not defined  [name-defined]
awswrangler/oracle.py:259: error: Name "oracledb.Connection" is not defined  [name-defined]
awswrangler/oracle.py:332: error: Name "oracledb.Connection" is not defined  [name-defined]
awswrangler/oracle.py:434: error: "Module oracledb" does not explicitly export attribute "Cursor"  [attr-defined]
awswrangler/oracle.py:437: error: Module has no attribute "DB_TYPE_NUMBER"  [attr-defined]
awswrangler/oracle.py:448: error: "Module oracledb" does not explicitly export attribute "LOB"  [attr-defined]
awswrangler/oracle.py:450: error: "Module oracledb" does not explicitly export attribute "LOB"  [attr-defined]
Found 15 errors in 1 file (checked 83 source files)

Hmm. Ok so why did the validation fail? What do I do if validation fails?
Let's try the next step, which also fails.
$ pytest tests/test_moto.py::test_get_bucket_region_succeed
ImportError while loading conftest '/home/ec2-user/environment/scratch/timestream-null/aws-sdk-pandas/tests/conftest.py'.
tests/conftest.py:12: in <module>
    import awswrangler as wr
awswrangler/__init__.py:10: in <module>
    from awswrangler import (  # noqa
awswrangler/sqlserver.py:22: in <module>
    import pyodbc  # pylint: disable=import-error
E   ImportError: libodbc.so.2: cannot open shared object file: No such file or directory

Is that a related error, or unrelated error?
This looks like I need to yum/apt install some ODBC drivers. But I'm following the instructions for "Mocked test environment", which I expect doesn't involve any real database connectivity.
Playing around, it looks like validation and tests pass if I don't install the extras.
Because there's a try/except somewhere (in the sqlserver library?) which will ignore the absence of pyodbc, but if pyodbc is present, it expects odbc drivers to be present too. I want to write a PR for something that doesn't use ODBC (timestream). I expect that I don't have to install ODBC drivers for every database flavor that this library supports.

Thanks for raising these points @mdavis-xyz, I have gone through the exercise of updating the contributing guide in #2054. I was able to successfully run the validate.sh and moto tests on my local machine by following these instructions. Please let us know if they work for you too and we will merge the PR.

Yep, those instructions work for me.
validate.sh is passing now.


Error when using long Docker Registry passwords

Hi there,
I am not entirely sure if this is the correct repository for this bug report, so feel free to point out where this should go instead!
Now that there is support for private Docker registries, I tried deploying an application to Swisscom Cloudfoundry (they are running CF API 2.84.0) using the cf CLI tool.
Unfortunately, when using Docker registry passwords longer than 175 characters (either with cf push using CF_DOCKER_PASSWORD) or when using the curl method as suggested here, this seems to cause a server-side error, reported as follows:
Using docker repository password from environment variable CF_DOCKER_PASSWORD.
Updating app my-app in org my-org / space my-space as<EMAIL_ADDRESS>FAILED
Server error, status code: 500, error code: 10011, message: Database error

When I trim the password to any length shorter than 175 characters, this database error does not occur but since the credentials generated by AWS are around 2000 characters, authentication obviously fails (as expected) when trying to fetch the image: Failed getting docker image by tag: unknown: Your Authorization Token is invalid.

Hey
Swisscom guy here.
The encrypted_docker_password column in ccdb.packages is varchar(255).
We verified the reported problem with different length passwords and received ERROR 1406 (22001): Data too long for column 'encrypted_docker_password' at row 1. The password is not stored in clear text, there is much chars overhead for the encryption.
@wmnnd  thank you very much for reporting to upstream. Great investigation and report. Can you please report the bug in https://github.com/cloudfoundry/cloud_controller_ng.
Bosh is not involved in Docker Registry.
Here the the error:
{"timestamp":1499406631.9314291,"message":"exception not translated: Sequel::DatabaseError - Mysql2::Error: Data too long for column 'encrypted_docker_password' at row 1:\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.5/lib/mysql2/client.rb:120:in `_query'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.5/lib/mysql2/client.rb:120:in `block in query'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.5/lib/mysql2/client.rb:119:in `handle_interrupt'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/mysql2-0.4.5/lib/mysql2/client.rb:119:in `query'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/adapters/mysql2.rb:136:in `block in _execute'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/logging.rb:48:in `log_connection_yield'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/adapters/mysql2.rb:131:in `_execute'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/adapters/utils/mysql_mysql2.rb:36:in `block in execute'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `block in synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/connection_pool/threaded.rb:103:in `hold'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/adapters/utils/mysql_mysql2.rb:36:in `execute'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/adapters/mysql2.rb:76:in `execute_insert'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/dataset/actions.rb:986:in `execute_insert'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/dataset/actions.rb:338:in `insert'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1997:in `_insert_raw'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1979:in `_insert'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2041:in `block (2 levels) in _save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1243:in `around_create'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2038:in `block in _save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1243:in `around_save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2034:in `_save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1684:in `block (2 levels) in save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2188:in `block in checked_transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:186:in `block in transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `block in synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/connection_pool/threaded.rb:103:in `hold'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:156:in `transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2188:in `checked_transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1684:in `block in save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:2176:in `checked_save_failure'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/model/base.rb:1684:in `save'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/package_create.rb:24:in `block in create'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:186:in `block in transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `block in synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/connection_pool/threaded.rb:103:in `hold'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:156:in `transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/package_create.rb:23:in `create'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/package_create.rb:34:in `create_without_event'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/v2/app_create.rb:71:in `create_lifecycle'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/v2/app_create.rb:19:in `block in create'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:215:in `_transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:190:in `block in transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `block in synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/connection_pool/threaded.rb:107:in `hold'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/connecting.rb:280:in `synchronize'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sequel-4.41.0/lib/sequel/database/transactions.rb:156:in `transaction'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/app/actions/v2/app_create.rb:11:in `create'\\n/var/vcap/data/packages/cloud_controller_ng/853a1d8b1c7be39a15e72854ed8b929688d87c9e/cloud_controller_ng/app/controllers/runtime/apps_controller.rb:261:in `create'\\n/var/vcap/data/packages/cloud_controller_ng/853a1d8b1c7be39a15e72854ed8b929688d87c9e/cloud_controller_ng/app/controllers/base/base_controller.rb:79:in `dispatch'\\n/var/vcap/data/packages/cloud_controller_ng/853a1d8b1c7be39a15e72854ed8b929688d87c9e/cloud_controller_ng/lib/cloud_controller/rest_controller/routes.rb:16:in `block in define_route'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1610:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1610:in `block in compile!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:975:in `block (3 levels) in route!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:994:in `route_eval'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:975:in `block (2 levels) in route!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1015:in `block in process_route'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1013:in `catch'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1013:in `process_route'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:973:in `block in route!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:972:in `each'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:972:in `route!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1085:in `block in dispatch!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `block in invoke'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `catch'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `invoke'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1082:in `dispatch!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:907:in `block in call!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `block in invoke'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `catch'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:1067:in `invoke'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:907:in `call!'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:895:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/xss_header.rb:18:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/path_traversal.rb:16:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/json_csrf.rb:18:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/base.rb:49:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/nulllogger.rb:9:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/head.rb:13:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:182:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/sinatra-1.4.7/lib/sinatra/base.rb:2013:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:66:in `block in call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `each'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_logs.rb:22:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/security_context_setter.rb:19:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/vcap_request_id.rb:15:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:49:in `call_app'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/cors.rb:14:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/middleware/request_metrics.rb:12:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/builder.rb:153:in `call'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/thin-1.7.0/lib/thin/connection.rb:86:in `block in pre_process'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/thin-1.7.0/lib/thin/connection.rb:84:in `catch'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/thin-1.7.0/lib/thin/connection.rb:84:in `pre_process'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/thin-1.7.0/lib/thin/connection.rb:50:in `block in process'\\n/var/vcap/packages/cloud_controller_ng/cloud_controller_ng/vendor/bundle/ruby/2.3.0/gems/eventmachine-<IP_ADDRESS>/lib/eventmachine.rb:1067:in `block in spawn_threadpool'","log_level":"warn","source":"cc.api","data":{"request_guid":"93f6a3b5-c5db-4c43-5a03-72b89db5b66f::d8ab52e8-f9cf-4fd7-b38c-98dc3a27e923"},"thread_id":47357287403320,"fiber_id":47357303324800,"process_id":6524,"file":"/var/vcap/data/packages/cloud_controller_ng/853a1d8b1c7be39a15e72854ed8b929688d87c9e/cloud_controller_ng/app/controllers/base/base_controller.rb","lineno":305,"method":"translate_and_log_exception"}

@GETandSELECT Thank you, I will report it in the cloud_controller_ng repository.


add -v/--version mode to print the current version of FLOSS

so that a user can quickly confirm if they have a feature or not.
also to provide a link to the project and authors pages.

-v is already verbose, so we'll have to do just --version

With 276f4df --version includes a link to the project page.
Need to remember to update version information with upcoming releases.


Usage in a tab navigator

Slightly confused on how to setup fluid transition with a tab navigation that disappears when navigation to a specific screen.
Currently I have something like this:
const PrimaryNav = createStackNavigator({
  LoginStack: { screen: LoginStack },
  Tabs: { screen: Tabs }
}, {
  headerMode: 'none',
  mode: 'modal',
  initialRouteName: 'Tabs',
  navigationOptions: {
    gesturesEnabled: false
  }
})

Tabs:
const SearchTabScreens = createStackNavigator({
  SearchTab: {
    screen: SearchScreen,
    navigationOptions: {
      title: 'Search',
      headerBackTitle: null
    }
  },
  OtherScreen: {
    screen: OtherScreen,
    navigationOptions: {
      headerStyle: styles.headerTransparent,
      headerTransparent: true
    }
  },
},
{
  headerMode: 'screen',
  navigationOptions: {
    headerTintColor: 'red',
    headerTitleStyle: styles.headerTitleStyle,
    headerStyle: styles.header
  }
}
)

SearchTabScreens.navigationOptions = ({ navigation }) => {
  let { routeName } = navigation.state.routes[navigation.state.index]
  let navigationOptions = {}

  if (screenNames.includes(routeName)) {
    navigationOptions.tabBarVisible = false
  }

  return navigationOptions
}

Tab navigator:
const Tabs = createBottomTabNavigator({
  SearchTab: {
    screen: SearchTabScreens,
    navigationOptions: ({ navigation }) => ({
      title: 'Search',
      tabBarIcon: ({ tintColor, focused }) => (
        <Icon
          name={'search'}
          size={32}
          color={tintColor}
        />
      )
    })
  }
},
{
  initialRouteName: 'SearchTab',
  swipeEnabled: false,
  animationEnabled: false,
  tabBarOptions: {
    activeTintColor: 'red',
    inactiveTintColor: 'gray',
    style: styles.tabBarStyle,
    labelStyle: styles.labelStyle,
    showLabel: false
  }
}
)

How am I suppose to setup a fluid transitioner, if I want to have shared element transition between main screen in a searchtab to a screen with more details about the item. I need a header in a search screen, but a screen with details should be without a header since its a dynamic header depending on a scroll position in other words image will scale up a scrollview and a header will appear if user scrolls to the bottom. But thats done inside a OtherScreen.

Did you figure it out? I'm going through the same

@matheushf Nope the issue is there. Decided not to use FluidTransitions until this gets resolved.

And did you find another way to use animations?

Closed due to inactivity.


Fixedpoint element

Started some work on a fixed point element a little bit ago. Here's what I've got so far. Still WIP

seems good to me! let's add it.


ARROW-142 Update documentation to make clear that extension types are not supported in Pandas

There's a docs error:

/home/runner/work/mongo-arrow/mongo-arrow/bindings/python/docs/source/extension_types.rst:document isn't included in any toctree


Wrong furigana for some words

It might be an upstream bug, I'm not sure if it's appropriate to open the ticket here.
Words that contain kanji, end with kana, and where the ending kana is same as the last kana of the preceding kanji.
e.g.:
可愛い (かわいい)
憎く (にくく)
低く (ひくく)

Dictionary.furigana function returns the wrong result:
from yomikata.dictionary import Dictionary

dictreader = Dictionary()
print(dictreader.furigana('可愛い'))

# it returns {可愛/かわ}い
# but the correct result should be {可愛/かわい}い

That's a big issue, thanks for letting me know. I'm looking into it now.

The bug is in the speach library's from_furi function (https://github.com/neocl/speach/blob/main/speach/ttlig.py), which fails on this class of words. I'll open an issue there, but in the meantime I've pushed a quick-but-ugly solution to fix this in yomikata.


Check all analyzer strict modes for interaction with patterns, records

[x] strict_raw_type
[ ] strict_casts
[ ] strict_inference

Anything further being done on this?  Shall I move it to stable or close it?

Oops, I'll move to stable.


Multi-region support

Might just loop in a shell script, not worth cluttering up the api or config but something to note. Basically just the ability to deploy the same function(s) to N regions.
Another aspect is that CloudWatch logs won't tail from multiple regions by default.

easy enough with a little script for now


Problems with GET requests

Hi Marlon,
currently we are facing issues when trying to address the following endpoints:
states
germany
districts
Example error message from states:
{"error":{"message":"There is a problem with the official RKI API.","rkiError":{"code":499,"message":"Token Required","messageCode":"GWM_0003","details":["Token Required"]},"url":"https://services7.arcgis.com/mOBPykOjAyBO2ZKk/arcgis/rest/services/RKI_COVID19/FeatureServer/0/query?where=NeuGenesen IN(1,-1)&objectIds=&time=&resultType=standard&outFields=AnzahlGenesen,MeldeDatum,IdBundeland&returnIdsOnly=false&returnUniqueIdsOnly=false&returnCountOnly=false&returnDistinctValues=false&cacheHint=false&orderByFields=IdBundesland&groupByFieldsForStatistics=IdBundesland&outStatistics=[{"statisticType":"sum","onStatisticField":"AnzahlGenesen","outStatisticFieldName":"recovered"},{"statisticType":"max","onStatisticField":"MeldeDatum","outStatisticFieldName":"date"}]&having=&resultOffset=&resultRecordCount=&sqlFormat=none&f=json&token="}}
It seems as if some tokens are missing.
Can this be corrected from your side?
Thanks and regards,
Chris Franz.

Hi Chris,
this is a issue from the rki, we have to wait.

The same as #399 . I think you could close this issue.

duplicate, close this issue


LEDs flash colors when performing a read of the temperature sensor

Describe the bug
When the temperature sensor is being read, the leds flash random colours.
To Reproduce

Set to solid color
Wait for temperature to be read, default at 20 seconds past each minute 0:20, 1:20, 2:20, etc

Expected behavior
Usermod should not impact colors
WLED version

Board: ESP32 Mini
Version: 0.11.0

Using logic analyser, it seems when the 1 one sensor is being read, it is messing up the sending of the data. Example below the lights are on blue but when sensor is being read, the wled colors are being interfeered with

Was the flashing happening on every read for you? I just disabled the temperature usermod and haven't noticed any flickers. But mine were only happening every 5-10 minutes.

yes on every reading, like clock work.

Demonstration of the problem. https://www.youtube.com/watch?v=F4fBYrZCXz8

Feedback from Michael Miller - https://gitter.im/Makuna/NeoPixelBus

@pbolduc More than likely they are not using the bitbang. On ESP (8266 or 32) bitbang just isn't going to work if you use many of the system features, like PWM, WiFi, Bluetooth. It is not a recommended method as mentioned in the Wiki.
@pbolduc Further, is the the one wire implementation interrupt based or is it also bitbang?
On ESP32 the default method is the RMT (channel 6 or 3) which is mostly async.

strip.CanShow() will return true when it is finished.


Evil window across frames

Describe the feature
When enabling the window-select +numbers module you can SPC w 1 to jump to the first buffer, even if in another frame.
It would be fantastic if this cross-frame functionality also worked for the evil-window-left, and similar, commands. Such that a user could SPC w j to move to a buffer that is on a frame in another monitor.
I realise that based on the configuration of the frames, this may not make sense (where is the next buffer on the left?), but in some configurations, e.g. a fullscreen frame per monitor, it very much does make sensor and could improve workflow.

Personal ideal use case: I would rebind C-j to evil-window-left, and could move between buffers across all monitors really fast.

This can be done with framemove and (setq framemove-hook-into-windmove t) as evil uses windmove internally.


StackdriverTraceExporter exceptions with "Panic! This is a bug!"

What version of OpenCensus are you using?
0.23.0
What JVM are you using (java -version)?
 > java -version
java version "1.8.0_221"
Java(TM) SE Runtime Environment (build 1.8.0_221-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.221-b11, mixed mode)

What did you do?
If possible, provide a recipe for reproducing the error.
I've created a small repository to demonstrate the issue, here. From the root of the repository run bazel run //:echo. You will see something like:
> bazel run //:echo
INFO: Analyzed target //:echo (1 packages loaded, 2 targets configured).
INFO: Found 1 target...
Target //:echo up-to-date:
  bazel-bin/echo.jar
  bazel-bin/echo
INFO: Elapsed time: 0.404s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Aug 18, 2019 12:05:45 AM com.google.auth.oauth2.DefaultCredentialsProvider warnAboutProblematicCredentials
WARNING: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that most server applications use service accounts instead. If your application continues to use end user credentials from Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service accounts, see https://cloud.google.com/docs/authentication/.
Aug 18, 2019 12:05:48 AM io.opencensus.implcore.trace.RecordEventsSpanImpl finalize
SEVERE: Span Sent.com.google.api.client.http.HttpRequest.execute is GC'ed without being ended.

if you have existing Google Credentials (please be aware that this may write traces into your account). Note the last line:
SEVERE: Span Sent.com.google.api.client.http.HttpRequest.execute is GC'ed without being ended.
To trigger the issue either wait for enough traces to buffer or send traffic to force the issue. I do the following:
>  ghz --insecure --proto=proto/echo.proto --call=com.example.echo.service.Echo.Echo localhost:8080

Running ghz from the root of the echo project.
What did you expect to see?
I expected to see traces in the Stackdriver Trace console.
What did you see instead?
Aug 18, 2019 12:07:21 AM io.opencensus.implcore.trace.export.SpanExporterImpl$Worker onBatchExport
WARNING: Exception thrown by the service export io.opencensus.exporter.trace.stackdriver.StackdriverTraceExporter
com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
	at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
	at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
	at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1052)
	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:398)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1030)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:871)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:716)
	at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:507)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:482)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700)
	at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
	at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
	at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:510)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:66)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:630)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:518)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:692)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:681)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
	Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
		at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
		at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
		at com.google.cloud.trace.v2.TraceServiceClient.batchWriteSpans(TraceServiceClient.java:234)
		at com.google.cloud.trace.v2.TraceServiceClient.batchWriteSpans(TraceServiceClient.java:182)
		at io.opencensus.exporter.trace.stackdriver.StackdriverV2ExporterHandler.export(StackdriverV2ExporterHandler.java:444)
		at io.opencensus.implcore.trace.export.SpanExporterImpl$Worker.onBatchExport(SpanExporterImpl.java:248)
		at io.opencensus.implcore.trace.export.SpanExporterImpl$Worker.exportBatches(SpanExporterImpl.java:350)
		at io.opencensus.implcore.trace.export.SpanExporterImpl$Worker.run(SpanExporterImpl.java:290)
		... 1 more
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Panic! This is a bug!
	at io.grpc.Status.asRuntimeException(Status.java:533)
	... 23 more
Caused by: java.lang.AbstractMethodError: Receiver class io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder$NettyTransportFactory does not define or inherit an implementation of the resolved method abstract newClientTransport(Ljava/net/SocketAddress;Lio/grpc/internal/ClientTransportFactory$ClientTransportOptions;Lio/grpc/ChannelLogger;)Lio/grpc/internal/ConnectionClientTransport; of interface io.grpc.internal.ClientTransportFactory.
	at io.grpc.internal.CallCredentialsApplyingTransportFactory.newClientTransport(CallCredentialsApplyingTransportFactory.java:49)
	at io.grpc.internal.InternalSubchannel.startNewTransport(InternalSubchannel.java:246)
	at io.grpc.internal.InternalSubchannel.access$400(InternalSubchannel.java:65)
	at io.grpc.internal.InternalSubchannel$2.run(InternalSubchannel.java:196)
	at io.grpc.SynchronizationContext.drain(SynchronizationContext.java:95)
	at io.grpc.SynchronizationContext.execute(SynchronizationContext.java:127)
	at io.grpc.internal.ManagedChannelImpl$NameResolverListener.onResult(ManagedChannelImpl.java:1365)
	at io.grpc.internal.DnsNameResolver$Resolve.resolveInternal(DnsNameResolver.java:311)
	at io.grpc.internal.DnsNameResolver$Resolve.run(DnsNameResolver.java:213)
	... 3 more

Additional context
I'm unsure if this issue is related to the one I present in #1962. I ran into this issue trying to build a small reproducible case but hit this instead.

There seem to be two issues here:

Span Sent.com.google.api.client.http.HttpRequest.execute is not ended correctly. This is usually because span.end() is not called before program exits. This is being tracked in https://github.com/googleapis/google-http-java-client/issues/788.
The gRPC AbstractMethodError due to shaded netty artifact. My guess is it has something to do with the dependencies in Bazel build. Bazel is not good at resolving shading and transitive dependencies, and if you have incompatible Google Cloud client libraries and gRPC versions you may get this kind of error. Can you paste your Bazel WORKSPACE file here? Also /cc @zhangkun83 @ejona86 any suggestions here?

The AbstractMethodError looks to be because grpc is getting included twice in the build: once via http_archive()/java_grpc_library() and another time via maven_install() (via io.opencensus:opencensus-exporter-*-stackdriver). Since maven_install is the first time there's been transitive dep tracking in Bazel, I've not really thought about this case at all. It seems overriding targets to point to their Bazel equivalents seems the appropriate solution. gRPC could maybe maintain a mapping that could be added to the override_targets.
I tried to reproduce but couldn't. bazel run //:echo ran fine for me. That's not surprising, given it is dependent on the classloading order and a few things that are "random" (not strictly speaking, but in practice). But something like this should fix the issue:
maven_install(
    ...
    override_targets = {
        "io.grpc:grpc-alts": "@io_grpc_grpc_java//alts",
        "io.grpc:grpc-auth": "@io_grpc_grpc_java//auth",
        "io.grpc:grpc-context": "@io_grpc_grpc_java//context",
        "io.grpc:grpc-core": "@io_grpc_grpc_java//core",
        "io.grpc:grpc-grpclb": "@io_grpc_grpc_java//grpclb",
        "io.grpc:grpc-netty-shaded": "@io_grpc_grpc_java//netty",
        "io.grpc:grpc-protobuf": "@io_grpc_grpc_java//protobuf",
        "io.grpc:grpc-protobuf-lite": "@io_grpc_grpc_java//protobuf-lite",
        "io.grpc:grpc-stub": "@io_grpc_grpc_java//stub",
    },
    ...
)

Note that //netty isn't fully equivalent to grpc-netty-shaded. They have different package names (which should generally be okay; people should not be referring to grpc-netty-shaded's package name) but also grpc-netty-shaded includes netty tcnative. So I also needed to add @io_netty_netty_tcnative_boringssl_static//jar to the runtime_deps of //:echo. Note that isn't required in all cases, like if using Java 9+.

@ejona86 your override_targets does appear to work on my system and the target deployment system as well. Thanks very much.

Fix has been made to upstream. Mark this issue as resolved for now.


iOS support

Hi,
is this script supported on iOS browser ?
I'm testing it on iPad (http://auxiliary.github.io/rater/) and if there are multiple rating elements on the same page, each click on one rating element resets the others to their default value.
is it normal ?

Rater hasn't been tested on iOS, however, this is a very strange bug and I haven't seen it before. Did you test the rater homepage or did you use the script and made your own test?

I tested both, rater homepage and my own test.

So I tested on a couple of places and it worked on an iPad and an iPhone but it had problems on certain versions of Firefox on Mac OSX. The problem only occurred for some star symbols, meaning that the UTF8 code for those is probably represented in a different way. Did you also see this occur for the heart rating at the bottom of the page?

Yes, this issue occurs also with the heart rating. My IPADs are in IOS 8, i will try to find IOS 9 devices to test it.

Is there any fix for that issue? I have the same probleme on all mobile iOS devices.

@auxiliary Same problem, iOS multiple raters breaks it

I have the same problem, the reason is that mobile iOS treats clicks as hover event. The solution is described here http://stackoverflow.com/questions/7892863/jquery-click-not-working-with-ipad . In rater.js file you need to replace $(this.element).on("click", $.proxy(this.select, this)); with $(this.element).on("click touchstart", $.proxy(this.select, this));

Got the same problem.

Change "click" to "click touchstart" works but
$(".rating").on("change", function(ev, data){
  console.log(data.from, data.to);
});

shows 0 NaN

Hello. I downloaded examples page. If i touch on one of rating then all other rating to be clean.

Yeah, I'm a little screwed.  I'm building a mobile web app using rater.js.  I had to add multiple raters on one page and I didn't realize until now that mobile safari is broken.  Not much time to roll my own solution, but I can't find any workaround. :-(  suggestions?

I ended up using a different package called RateYo (http://rateyo.fundoocode.ninja/) - apple and Safari are pretty major browsers and really needs corrected.  Sorry for recommending another package @auxiliary but just helping @brandonrich out with a package that works.

Is there a workaround for this issue? Thanks

@rsands2801 same for me and also worked!

@ghost Thanks! It works perfect

I had the same problem on the iPad. I fixed (hack) the problem like this:
    $('.star-rating').rate({
        max_value: 5,
        step_size: 0.5,
        selected_symbol_type: 'fontawesome_star'
    }).on('touchend', function(ev) {
        var offset = $(this).offset();
        var rating = (ev.originalEvent.changedTouches[0].pageX - offset.left) / $(this).width() * 10;
        $(this).rate('setValue', Math.ceil(rating)/2);
    });

It's not a good solution, but it's working for me.


[Bug] Python multiline string in diff hunk header breaks syntax highlighting

Similar to #549 but not quite the same, so I figured it warrants a new issue.
When git diff interprets an object instantiation with a multiline string as a function header and includes the quotation marks in the header of the hunk, syntax highlighting breaks and interprets everything as a string:
Example:

This fixes itself when the quotation marks are not included in the hunk header anymore.
Example:

Same reason as #549

Can the hunk header not be excluded from syntax highlighting?
I understand, that the issue in #549 requires a different approach to syntax highlighting, because the quotation marks are part of the changed code, but in this case the header is not part of the code.
Is the complete diff highlighted instead of just the parts inside the hunks?

I've checked the source code if AvaloniaEdit. To implement this feature, it is probably necessary to rewrite a AvaloniaEdit.TextMate. This is not in the development plan.


iPad Default Look

The default look on the iPad hides the sidebar on portrait mode.
The story is that UISplitViewControllers have two capabilities that control the behavior: preferredSplitBehavior and preferredDisplayMode, documented on the UISplitViewController page
The problem is that these capabilities are not exposed in the automatic SwiftUI system that wires this up, as documented by Hacking with Swift.
The options seem to be:

Rewrite the launcher app to use UIKit instead of SwiftUI, but would be giving up on a lot of wiring up and goodies
Manually configure navigationTitle to provide some breadcrumbs to the user as to what they can do (currently, we just default to "Back", so we would need to change all nested items`)
Use an idle loop hack to poke around values.   The idle loop hack requires on hidden windows on the main surface area to display this.  An example of this is in this post and in this StackOverflow answer

The hack can be made to work, but comes with assorted caveats, like need to take over the various policies, even for those where the defaults of SwiftUI are sensible.

The new landing page avoids the discoverability issues.  Also the swift-introspect can be used to do this with SwiftUI if necessary in the future


monarch_cuda_interface_fwd_bf16.cu failed with invalid argument (1).

ERROR: CUDA RT call "cudaFuncSetAttribute(&monarch_conv_cuda_32_32_32_kernel<32, 8, 32768, 2, 16, false, 2, 8, 8>, cudaFuncAttributeMaxDynamicSharedMemorySize, 135168)" in line 969 of file /root/flash-fft-conv/csrc/flashfftconv/monarch_cuda/monarch_cuda_interface_fwd_bf16.cu failed with invalid argument (1). CUDA Runtime Error at: /root/flash-fft-conv/csrc/flashfftconv/monarch_cuda/monarch_cuda_interface_fwd_bf16.cu:1041 invalid argument
Tried the example code with my_flashfftconv(x, k) and tests/test_flashfftconv.py using the Nvidia PyTorch docker container (23.05). Previously, I used conda with different CUDA versions (12.1, 12.2 and 12.3).
I'm using two NVIDIA RTX 3090 with Driver Version: 535.129.03 and CUDA Version: 12.2
Is there any fix for this problem? (changing tensor types didn't fixed)

Thanks for this bug report! This is because the RTX series has less SRAM than A100/H100 (99 KB vs. 163/227 KB), which I didn't check for during development. You should be good for now for sequences up to 16K, and sequence lengths between 64K and 524K.
We'll try to fill in the rest of the sequence lengths for 3090 & 4090 in the next week or so, up to 2M (it requires some code changes and special-casing for different GPUs).

Thanks for your excellent work on this project! I'm running into the same CUDA error with an RTX 4090.
I specifically need to use sequence lengths between 8K and 64K, which seems to be the current gap in RTX support due to the SRAM limitations compared to A100/H100. Are there any plans to address this particular range for RTX cards? It would be really valuable for my use case.
Thanks for all your work on this!


DOUBLETAP NOT WORKING

Fedoraware Version Date:
Config used:

Description:

Steps to reproduce:

[ ] I am able to consistently reproduce this error

Steps to reproduce the behavior:

Enable X
Do Y
Notice Z

Additional context

[ ] I have included the steps to reproduce to the best I am able to share.
mfed is trans

terry davis


Support Count(Distinct) (and similar) aggregation

What is the problem the feature request solves?
We should also support aggregations such as count(distinct(col)) from tbl. In Spark,
SELECT COUNT(DISTINCT(_1)) FROM tbl

produces a plan like the following:
AdaptiveSparkPlan isFinalPlan=false
+- HashAggregate(keys=[], functions=[count(distinct _1#9)], output=[count(DISTINCT _1)#16L])
   +- Exchange SinglePartition, ENSURE_REQUIREMENTS, [plan_id=40]
      +- HashAggregate(keys=[], functions=[partial_count(distinct _1#9)], output=[count#20L])
         +- HashAggregate(keys=[_1#9], functions=[], output=[_1#9])
            +- Exchange hashpartitioning(_1#9, 5), ENSURE_REQUIREMENTS, [plan_id=37]
               +- HashAggregate(keys=[_1#9], functions=[], output=[_1#9])
                  +- Scan parquet [_1#9] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_1:int>

Describe the potential solution
Add the support for COUNT(DISTINCT) (and similar), so the Spark physical plan can be properly converted to a native plan and executed.
Additional context
No response

cc @huaxingao


feat (connect-nodejs): NodeJS Implementation

The nodejs implementation for hyper connect should work with typescript, esm, commonjs, and vite. This issue is to refactor the current connect repo into two repos, a nodejs and a deno repo, both will be typescript to take advantage of autocomplete for VS Code users and Typescript Users.

[x] Data

[x] Cache

[x] add
[x] get
[x] set
[x] remove
[x] query

[ ] Search

[ ] Migrate Tests

[x] Data
[x] Cache
[ ] Search

[ ] Update Documentation and Re-Record Workshops

I was able to get a build together that supports all four platforms

typescript
esm
cjs
vite/esbuild

Need to migrate to this repository and migrate tests to validate code

The code looks good to me, besides some TS specific things that i'll detail below. I see you've switched to using closures instead of ADTs, which I think is a fine tradeoff for now, until we figure out how to get crocks working. Here are some things that I noticed about the types and some potential solutions:

There is a TS error here because params is of type unknown. But we do know what params will be: undefined | Record<string, any>. So with params typed as params?: Record<string, any>, the error is resolved. We use any here because URLSearchParams does casting of types like number and boolean for us. So I think any is acceptable here. This also solves an issue here

There is a TS error here because options type is:

{
  body?: unknown;
  headers: Headers;
  method: Method; <-- TS Enum
}

and the implicit type returned is:
{
  body?: unknown;
  headers: {};
  method: string;
}

Changing to initializing options like this resolves the issue:
let options = {
  headers: new Headers({
    "Content-Type": "application/json",
  }),
  method: method ? method : Method.GET,
};

Enums are one of the "things" in TypeScript that actually compile down into JS. Enums are compiled into POJOs, so you can use them like a pojo. Though Enums do some magic, which I don't like, and then you have to use that enum everywhere in code, when static strings, with autocomplete, is what you want. For those reasons, I usually avoid TS enums and opt to use literals see here OR arrays declared as const, then use typeof Val[number] see here. Both compile down to JS, and the latter compiles the array down into the JS (because it is just an array), for reference in the JS output. Both have pros and cons, albeit small.

There is a TS error here because body of type unknown is incompatible with BodyInit | undefined. Changing body to be of type BodyInit solves the issue, which seems kosher since you're already using Headers from node-fetch in that same type.

A bunch of TS errors in this file on the promise chains. This is due to h being (hr: HyperRequest) => Promise<Request> and the service methods expecting (hr: HyperRequest) => Request. Changing HyperRequestFunction to return a Promise<Request> solves the issues.

A couple TS errors here because you're using the array as a functor to map over (clever btw). You can:

declare service as const:
const service = "cache" as const;

declare method as const:
method: "PUT" as const,

and declare ttl as string | undefined
const includeTTL = (ttl: string | undefined) =>

Lots of TS errors in the test files, but I think the only thing worth fixing is awaiting the return of the service methods, since they are meant to return promises. You can replace identity with Promise.resolve

You can see these suggestions on tillathehun0/feat-connect-372-ts-amendments if that is easier to grok

One final comment:
in types.ts you've used both Type and T. Then, that type is used sometimes used as the type of argument, sometimes as the output.
I would consider being more explicit. If the type is the return type, name the type something like Result or R. If the type if for the input arg, name it Arg or Input. Or perhaps name it to match the domain ie. data Document and cache Value.
Just a thought.

Awesome, thank you for taking the time to review and the helpful typescript hints


Series with the same name messed up result graphs. append a unique id for duplicates

old

new

will eventually also add the grafana graph

trigger build


Added reactive event handler

@tdiesler I have been thinking a lot about how to make testing easier, especially when working with websockets where there are multiple threads that need to be synchronised. So I had the idea to introduce reactive streams as a solution. In the connect two subwallets test you see how this could look like: https://github.com/hyperledger-labs/acapy-java-client/blob/1ccdf6ddc1d7af9022d9db529ff3179d4714e45a/src/test/java/org/hyperledger/aries/api/multitenant/MultiTenantTest.java#L130
In short it is like:
// create base client
// create two sub wallets
// create clients for sub wallets

// create websocket and reactive listener for sub wallet 2
ReactiveEventHandler rx = ReactiveEventHandler.builder().walletId(wallet2.getWalletId()).build();
AriesWebSocketClient socket = AriesWebSocketClient.builder()
        .apiKey(adminApiKey)
        .url("ws://localhost:" + ariesContainer.getMappedPort(IntegrationTestBase.ARIES_ADMIN_PORT) + "/ws")
        .bearerToken(wallet2.getToken())
        .handler(rx)
        .handler(new TenantAwareEventHandler.DefaultTenantAwareEventHandler())
        .build();

// wallets are prepared
// wallet 2 creates invitation
// wallet 2 receives invitation

// now wallet 2 can do this
rx.connection().filter(ConnectionRecord::stateIsRequest).subscribe(s -> {
    try {
        sub2.connectionsAcceptRequest(s.getConnectionId(), null);
    } catch (IOException e) {
        Assertions.fail("expecting to connect", e);
    }
 });

ConnectionRecord active = rx.connection()
                .filter(ConnectionRecord::stateIsActive)
                .blockFirst(Duration.ofSeconds(5));
Assertions.assertNotNull(active);
Assertions.assertEquals(cr1.getInvitationKey(), active.getInvitationKey());

As the try/catch in the subscriber is not very nice I'm thinking about wrapping the checked exception into a RuntimeException.
Any feedback is welcome.

Yes, we seem to be needing yet another plugin mechanism to the WS event processing chain. The (WS) Listener delegates to the EventHandler, which sees typed representations of the messages. The Listener is scoped on the (multitenant) wallet (when we have the "default" message distribution type). Now, not every wallet wants to see the messages from all (other) wallets nor all emitted message types. We seem to be needing a plugin to the EventHandler that can filter messages by source and/or type.
The reactive API could be used for such a plugin, but might be an engineering overkill for such a (simple) plugin. I believe WS message delivery is already decoupled sufficiently such that we don't need yet another async message processing route to the message consumer. Still, the reactive API defines concepts that are well suited to talk about what is needed here. We have ...

Publisher - publishes messages to interested parties (possibly async)
Subscriber - subscribes to messages from a given Publisher
Subscription - handles lifecycle of the Subscriber and message delivery semantics

From what I can see so far, I'd have these requirements on the message filtering system ...

Allow Subscriptions when the EventHandler is created (so we don't miss messages)
Allow dynamic subscriptions to be added/removed from a given EventHandler
Filter on 0:N source wallet ids - Settings does not have a source wallet - option to listen to all source wallets
Filter on 0:N message types -  option to listen to all message types
Simple message Consumer API that gives access to all aspects of the WS message and can throw checked Exceptions

To help me understand what is needed for the getting started walkthrough (so far) I also prototyped an EventHandler plugin mechanism based on the JDK Reactive API (no extra dependencies). This is mostly throw-away code for you to look at. In the end, I of course prefer this API to live in this project.
Still, here you have ...

An example registering subscriptions at EventHandler build time
Dynamic subscriptions on an existing EventHandler
Filter on multiple wallet ids
Filter on multiple message types
Consumer that creates its AriesClient through the Event and makes the call without exception handling

In case, you already have the above (more or less) I'm happy to try out that API and give specific feedback before the merge if you like.

A simplified approach might be to add filtering to the EventHandler directly and hence requiring WS connections to be short lived and purpose build. In that case the reactive API might not be needed at all. WDYT?

Ok many points here. First a general remark. If you look on how aca-py deals with webhooks vs.  websockets we need to make sure that we are not working around a bug here by putting too much logic into the websocket handler. To make this clearer.
If you are dealing with webhooks in multi-tenant mode aca-py allows to set a wallet dispatch type per sub-wallet and allows you to register webhook handlers on a sub-wallet basis. Like this, events are already separated on the aca-py side, and a sub-wallet handler never sees events from another wallet. The base wallet being the exception here.
If you compare this with what is happening with webhooks my guess is that there are simply some implementation details missing in aca-py, because I would expect that if I connect a websocket with a Bearer token that all events are already filtered by the sub-wallets token. Everything else is a potential security risk. Imagine someone is hosting a multi-tenant enabled aca-py to companies A and B, and B sees all events of A. Long story short I would expect that the websocket client behaves in the same way as the rest client meaning that you need to create a new instance for each sub wallet.
Concerning the other points.
In my understanding the websocket client is the publisher and the handler is the subscriber. In the PR I switched to a handler list so that you can push as many handlers as needed.
AriesWebSocketClient socket = AriesWebSocketClient.builder()
        .handler(h1)
        .handler(h2)
        .build();

I have not thought about making this dynamic, because my use cases are static atm, also usually I simply throw away and recreate, but changing this is not a problem.
Filtering by sub wallet happens in the handler:
ReactiveEventHandler h1 = ReactiveEventHandler.builder().walletId(wallet2.getWalletId()).build();

Meaning one handler per sub wallet, but this can also be extended to a list of ids. But as mentioned above this should not be needed at all imho.
I believe so far this is pretty close to what you are doing, what is different is that my reactive implementation is already pre filtered and one can subscribe for example to a stream of connection events and then filter by event attributes like:
h1.connection().filter(ConnectionRecord::stateIsRequest).subscribe(System.out::println)

The subscribe part can take any consumer.
The main reason for introducing the reactor framework was that this allows for easier thread synchronisation and hence easier testing, because in the raw implementation you either have to manually synch the threads or do all the testing in the consumer. Whereas with reactor you can block and wait and hence synch within the test thread like:
ConnectionRecord active = h1.connection()
                .filter(ConnectionRecord::stateIsActive)
                .blockFirst(Duration.ofSeconds(5));
Assertions.assertEquals("1234", active.getInvitationKey());

Ok where to go from here? As both ways have its merits I think a compromise would be that the websocket clients accepts the following:

the handler interface which is more coarse granular
a tuple of event class and consumer
0:N wallet ids

AriesWebSocketClient socket = AriesWebSocketClient.builder()
        .walletId("1")
        .walletId("2")
        .handler(h1)
        .subscribe(ConnectionRecord.class, System.out::println)
        .build();

Then it is up to the user of the api what implementation to choose. The only thing that I would not do is add more filter options to the websocket client. Also what is totally in the responsibility of the client is the decision to block in the handler/subscriber here you can shoot yourself in the foot pretty quickly. The reactor framework is more lenient here as it allows back pressure in this case.

This looks very good ...

I also believe we are looking at an aca-py bug or at least at a not very well defined area of event visibility
Not having dynamic WS listeners should work fine for now
"easier thread synchronisation" - yes, I didn't think of that - I have latches all over

add more filter options to the websocket client

We don't do that, right? AFAIK, a WS client registers a listener which then delegates to a handler
Regarding event visibility, I have this awkward code here. As you can see, the Inviter listens to events from the Invitee and vice versa. It may only work because of that presumed visibility bug.
Finally, what about throwing checked exceptions from the consumer? Would this be an option?

where to go from here?

I suggest you give me a shout when you're done with your branch. Then, I migrate my code over and after going back and forth a few times we merge to main. You can always meet me in Discord for a quick chat.

I revisited the connection code and as it turns out here to we can filter on the wallet that owns the ws connection. In fact, I could remove the filter by wallet API completely, except for the revoke notify case when Alice wants to see the revoked CredentialExchange - this event is published by Acme.

I played around with this some more, the fine granular handlers are a bit of a pain as Java does not work very well with dynamic types, but in the end it should not really matter because I directly made the client a handler so now it is like:
AriesWebSocketClient ws = AriesWebSocketClient.builder()
        .apiKey("123")
        .walletId("1") // optional 0:N
        .walletId("2")
        .handler(new TenantAwareEventHandler.DefaultTenantAwareEventHandler()) // optional 0:N
        .build();

And all events are directly accessible:
ConnectionRecord active = ws.connection()
                .filter(ConnectionRecord::stateIsActive)
                .blockFirst(Duration.ofSeconds(5));

If you like you can take this for a first test drive.

Ok, thanks - will do. It might take a little while however ...

We could say, if a clients opens a WS it is also responsible for consuming all events and when done, close the WS. Alternatively, a WS connection could (lazily) be associated with the wallet  and the client opens/closes subscriptions on individual event types. Events that the client is not interested in, would be ignored and would hence not fill buffers/sinks

The nice thing here is that multicast sinks do not start producing any events as long as there aren't any subscribers, so events are gone/ignored if no one is there to listen. This is different for replay or unicast sinks where one would receive previous events depending on the buffer size: https://projectreactor.io/docs/core/release/reference/#sinks The subscriber is created together with the flux when calling ws.connection() for example, not before.
One possible issue is if there is a slow subscriber and events are coming in faster than they are taken from the buffer then the backpressure buffer would fill up and events would get lost. And theoretically if there are many slow subscribers and many events with huge payloads memory could indeed file up. But I see this in the responsibility of the clients to handle this correctly, same as calling dispose() on the flux if needed.
But I think it makes sense to make the WebSocketClient implement AutoClosable, like this most IDE's will even warn you if the client is not closed.

Ok, all good then. How about we build the WS client functionality into the WalletRecord or a higher level Wallet thing that can be obtained from the Wallet builder - similar to what I have in NessusWallet? Note, how this is awkwardly constructed with a json roundtrip.

I would preferably do this in a new branch, and merge this one first.

Ok, let's do that. Are you blocked by 0.7.4 for this merge?


production builds using<EMAIL_ADDRESS>fail on ios Safari/UIWebView

Type of issue
:bug:
Description
I have only been able to repro this on an actual physical iphone, iOS simulator does NOT error.
error in console:
Error: Loading chunk 3 failed.
(error: production_url_for_fusion_app/_static/3-8d3fd1c14cbf6fa2672f.js)

Error occurs in a file that looks like:
production_url_for_fusion_app/_static/client-vendor-3249340eddecd23065de.js
I have confirmed that the error DOES NOT occur with version 1.1.0 of fusion-cli.
Current behavior
Client side error occurs in mobile safari + UIWebView causing the page to not finish client-side hydrating.
Expected behavior
It should not do that
Your environment

fusion-cli version: 1.3.0-1

Node.js version (node --version): 8.9.3

npm version (npm --version): 5.6.0

Operating System: iphone 7+, iOS 11.2.6

Thank you for the detailed bug report and sorry you're running into issues.

Client side error occurs in mobile safari + UIWebView

Does this mean both the mobile safari browser, as well as a custom in-app page using UIWebView?
I'm wondering if the bug reproduces with WKWebView as well.

There's an outstanding bug with Safari where it doesn't send cookies for script tags with crossorigin="anonymous" attributes, even when the source url is same-origin. See:

https://bugs.webkit.org/show_bug.cgi?id=171566
https://bugs.webkit.org/show_bug.cgi?id=171550

As a result of this, any chunks loaded by webpack will fail in Safari if they require cookies to be sent (because webpack uses crossorigin="anonymous").

Still some issues from this...


Missing File in Training Process

Hello, I encountered a small issue during the training process. It seems that line 17 in the train.py file attempts to import a function named get_model from models.epsnet, but I couldn't locate the models/epsnet.py file. Could you kindly verify if the file is missing? Thank you very much for your assistance.

Thank you su much for pointing it out. We have uploaded the corresponding file which contains the get_model function. Fell free to reach out to me if you have further questions.


Fix up npm scripts

There were some issues in package.json preventing npm run static from working out of the box, which this PR should resolve:

bower and http-server dependencies are declared
npm run build-static runs before npm run static
';' is replaced with '&&' in shell scripts so that non-zero error codes are not ignored (i.e., if you can't go into a directory, report failure, and don't start deleting things or running http-server)

:+1: love the cleanup! thanks


Support serialise nested object as nil

We have to support sending a nested object as nil so as to delete the object.
  objectUpdate(input: $input) {
        userErrors { field, message }
        object {
          id
          title
          suboject {
            id
            name
          }
        }
      }
    }

To do that we have to modify the serialise function.

This is also affecting the android client. Is there an open issue to fix that as well?

I created the ticket but haven't had a chance to look into it. It's not on my TODO plan now. But I can take a peek next week.

Closing in favour of https://github.com/Shopify/graphql_swift_gen/pull/8


Update quill-jdbc, quill-jdbc-zio to 3.14.1

Updates

io.getquill:quill-jdbc
io.getquill:quill-jdbc-zio

from 3.10.0 to 3.14.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.getquill" } ]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #266.


Jekyll build failure

Hi. Thanks for this great plugin.
I'm trying to figure this out with my blog, but I failed to compile my jekyll blog.
Here is the error log.
  Conversion error: Jekyll::Converters::Markdown encountered an error while converting '_posts/2018-07-08-avl-tree.md':
                    Error retrieving language configuration for scope 'source.cpp': Language not found
                    ------------------------------------------------
      Jekyll 4.1.1   Please append `--trace` to the `serve` command
                     for any additional information or backtrace.
                    ------------------------------------------------

/var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/cache.rb:88:in `[]': unhandled exception                                                                                [0/1513]
        44: from /home/sangwoo-joh/.gems/bin/jekyll:23:in `<main>'
        43: from /home/sangwoo-joh/.gems/bin/jekyll:23:in `load'
        42: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/exe/jekyll:15:in `<top (required)>'
        41: from /var/lib/gems/2.7.0/gems/mercenary-0.4.0/lib/mercenary.rb:21:in `program'
        40: from /var/lib/gems/2.7.0/gems/mercenary-0.4.0/lib/mercenary/program.rb:44:in `go'
        39: from /var/lib/gems/2.7.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `execute'
        38: from /var/lib/gems/2.7.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `each'
        37: from /var/lib/gems/2.7.0/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `block in execute'
        36: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/commands/serve.rb:86:in `block (2 levels) in init_with_program'
        35: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/command.rb:91:in `process_with_graceful_fail'
        34: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/command.rb:91:in `each'
        33: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/command.rb:91:in `block in process_with_graceful_fail'
        32: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/commands/build.rb:36:in `process'
        31: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/commands/build.rb:65:in `build'
        30: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/command.rb:28:in `process_site'
        29: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:80:in `process'
        28: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:207:in `render'
        27: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:515:in `render_docs'
        26: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:515:in `each_value'
        25: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:516:in `block in render_docs'
        24: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:516:in `each'
        23: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:517:in `block (2 levels) in render_docs'
        22: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/site.rb:532:in `render_regenerated'
        21: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:63:in `run'
        20: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:84:in `render_document'
        19: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:100:in `convert'
        18: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:100:in `reduce'
        17: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:100:in `each'
        16: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/renderer.rb:102:in `block in convert'
        15: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/converters/markdown.rb:85:in `convert'
        14: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/cache.rb:108:in `getset'
        13: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/cache.rb:111:in `rescue in getset'
        12: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/converters/markdown.rb:86:in `block in convert'
        11: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/converters/markdown/kramdown_parser.rb:110:in `convert'
        10: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/converters/markdown/kramdown_parser.rb:62:in `to_html'
         9: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/base.rb:107:in `convert'
         8: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:58:in `convert'
         7: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:371:in `convert_root'
         6: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:70:in `inner'
         5: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:70:in `each'
         4: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:71:in `block in inner'
         3: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/html.rb:113:in `convert_codeblock'
         2: from /var/lib/gems/2.7.0/gems/kramdown-2.3.0/lib/kramdown/converter/base.rb:197:in `highlight_code'
         1: from /home/sangwoo-joh/.gems/gems/kramdown-syntax_tree_sitter-0.5.0/lib/kramdown/converter/syntax_highlighter/tree_sitter.rb:22:in `call'
/home/sangwoo-joh/.gems/gems/kramdown-syntax_tree_sitter-0.5.0/lib/kramdown/converter/syntax_highlighter/tree_sitter.rb:22:in `highlight': Error retrieving language configuration for scope 'source.cpp': Language not found (RuntimeError)

It seems like every language shows the same error.
I'm using rustc 1.73.0, ruby  2.7.0p0 and jekyll 4.1.1 version.
Here is my Gemfile:
...
gem "kramdown-syntax_tree_sitter", "~> 0.5.0"

, and my _config.yml:
plugins:
  - jekyll-feed
  - jekyll-seo-tag
  - jekyll-org
  - kramdown/syntax_tree_sitter

markdown: kramdown
highlighter: rouge
kramdown:
  input: GFM
  syntax_highlighter: tree-sitter
  syntax_highlighter_opts:
    css_class: 'highlight'
    span:
      line_numbers: false
    block:
      line_numbers: false

How do I fix this error?
Looking forward to hearing.

Hi! Happy to help.
I believe this is the root issue:

Error retrieving language configuration for scope 'source.cpp': Language not found

I think you still need to add some Tree Sitter language parsers to your project so that this tool can find and use them. Unlike many syntax highlighters with built-in support for many popular languages, Tree Sitter relies on external language parser libraries to understand the grammar of each language. For example, tree-sitter/tree-sitter-cpp is the project for the Tree Sitter C++ parser library. More information on how that applies to this tool can be found here in the README, but feel free to let me know if you're still confused.
Also, I happen to know that it is especially complicated to set up the C++ parser library in particular, because that Tree Sitter grammar has a dependency on the Tree Sitter C grammar. It is possible to get it to work but I might suggest trying to get things working with basically any other language first.

Wow, thank you for the gentle explanation!!
As you said, I missed the README part.
I made ~/tree_sitter_parers directory and cloned tree-sitter-c, tree-sitter-cpp, tree-sitter-python. This time, I got different error message:
       Jekyll Feed: Generating feed for posts
  Conversion error: Jekyll::Converters::Markdown encountered an error while converting '_posts/2018-07-08-avl-tree.md':
                    Failed to read query file "node_modules/tree-sitter-c/queries/highlights.scm": No such file or directory (os error 2)
                    ------------------------------------------------
      Jekyll 4.2.0   Please append `--trace` to the `build` command
                     for any additional information or backtrace.
                    ------------------------------------------------
Traceback (most recent call last):
	46: from /Users/sangwoo-joh/.rbenv/versions/2.7.7/bin/bundle:23:in `<main>'
	45: from /Users/sangwoo-joh/.rbenv/versions/2.7.7/bin/bundle:23:in `load'
	44: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/exe/bundle:33:in `<top (required)>'
	43: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	42: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/exe/bundle:45:in `block in <top (required)>'
	41: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:28:in `start'
	40: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	39: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:34:in `dispatch'
	38: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	37: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	36: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	35: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:491:in `exec'
	34: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:23:in `run'
	33: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:58:in `kernel_load'
	32: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:58:in `load'
	31: from /Users/sangwoo-joh/.gems/bin/jekyll:23:in `<top (required)>'
	30: from /Users/sangwoo-joh/.gems/bin/jekyll:23:in `load'
	29: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/exe/jekyll:15:in `<top (required)>'
	28: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary.rb:21:in `program'
	27: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/program.rb:44:in `go'
	26: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `execute'
	25: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `each'
	24: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `block in execute'
	23: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
	22: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `process_with_graceful_fail'
	21: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `each'
	20: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `block in process_with_graceful_fail'
	19: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:36:in `process'
	18: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:65:in `build'
	17: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:28:in `process_site'
	16: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:80:in `process'
	15: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:210:in `render'
	14: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:530:in `render_docs'
	13: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:530:in `each_value'
	12: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:531:in `block in render_docs'
	11: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:531:in `each'
	10: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:532:in `block (2 levels) in render_docs'
	 9: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:547:in `render_regenerated'
	 8: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:63:in `run'
	 7: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:84:in `render_document'
	 6: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `convert'
	 5: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `reduce'
	 4: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `each'
	 3: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:106:in `block in convert'
	 2: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/converters/markdown.rb:85:in `convert'
	 1: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/cache.rb:109:in `getset'
/Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/cache.rb:88:in `[]': unhandled exception
	59: from /Users/sangwoo-joh/.rbenv/versions/2.7.7/bin/bundle:23:in `<main>'
	58: from /Users/sangwoo-joh/.rbenv/versions/2.7.7/bin/bundle:23:in `load'
	57: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/exe/bundle:33:in `<top (required)>'
	56: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/friendly_errors.rb:117:in `with_friendly_errors'
	55: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/exe/bundle:45:in `block in <top (required)>'
	54: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:28:in `start'
	53: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
	52: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:34:in `dispatch'
	51: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
	50: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
	49: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
	48: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli.rb:491:in `exec'
	47: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:23:in `run'
	46: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:58:in `kernel_load'
	45: from /Users/sangwoo-joh/.gems/gems/bundler-2.4.4/lib/bundler/cli/exec.rb:58:in `load'
	44: from /Users/sangwoo-joh/.gems/bin/jekyll:23:in `<top (required)>'
	43: from /Users/sangwoo-joh/.gems/bin/jekyll:23:in `load'
	42: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/exe/jekyll:15:in `<top (required)>'
	41: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary.rb:21:in `program'
	40: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/program.rb:44:in `go'
	39: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `execute'
	38: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `each'
	37: from /Users/sangwoo-joh/.gems/gems/mercenary-0.4.0/lib/mercenary/command.rb:221:in `block in execute'
	36: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:18:in `block (2 levels) in init_with_program'
	35: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `process_with_graceful_fail'
	34: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `each'
	33: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:91:in `block in process_with_graceful_fail'
	32: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:36:in `process'
	31: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/commands/build.rb:65:in `build'
	30: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/command.rb:28:in `process_site'
	29: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:80:in `process'
	28: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:210:in `render'
	27: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:530:in `render_docs'
	26: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:530:in `each_value'
	25: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:531:in `block in render_docs'
	24: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:531:in `each'
	23: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:532:in `block (2 levels) in render_docs'
	22: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/site.rb:547:in `render_regenerated'
	21: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:63:in `run'
	20: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:84:in `render_document'
	19: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `convert'
	18: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `reduce'
	17: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:104:in `each'
	16: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/renderer.rb:106:in `block in convert'
	15: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/converters/markdown.rb:85:in `convert'
	14: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/cache.rb:108:in `getset'
	13: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/cache.rb:111:in `rescue in getset'
	12: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/converters/markdown.rb:86:in `block in convert'
	11: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/converters/markdown/kramdown_parser.rb:110:in `convert'
	10: from /Users/sangwoo-joh/.gems/gems/jekyll-4.2.0/lib/jekyll/converters/markdown/kramdown_parser.rb:62:in `to_html'
	 9: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/base.rb:107:in `convert'
	 8: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:58:in `convert'
	 7: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:373:in `convert_root'
	 6: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:70:in `inner'
	 5: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:70:in `each'
	 4: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:71:in `block in inner'
	 3: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/html.rb:115:in `convert_codeblock'
	 2: from /Users/sangwoo-joh/.gems/gems/kramdown-2.3.1/lib/kramdown/converter/base.rb:197:in `highlight_code'
	 1: from /Users/sangwoo-joh/.gems/gems/kramdown-syntax_tree_sitter-0.5.0/lib/kramdown/converter/syntax_highlighter/tree_sitter.rb:22:in `call'
/Users/sangwoo-joh/.gems/gems/kramdown-syntax_tree_sitter-0.5.0/lib/kramdown/converter/syntax_highlighter/tree_sitter.rb:22:in `highlight': Failed to read query file "node_modules/tree-sitter-c/queries/highlights.scm": No such file or directory (os error 2) (RuntimeError)

I ran this command in macbook air m1, so maybe this could affect.
All other versions are the same.

I also tried to install npm install tree-sitter tree-sitter-cli with -g and without -g, but it won't work.
Thank you!

I think you're now encountering the issue that makes the Tree Sitter C++ parser particularly complicated to use; namely, that it has a dependency on the Tree Sitter C parser, and that this dependency is not resolved by default. One successful workaround I have personally used is to symbolically link the Tree Sitter C++ node_modules directory to the tree-sitter-parsers directory that you created, for example via the following command:
ln -s .. ~/tree_sitter_parsers/tree-sitter-cpp/node_modules

This is what I do for my own Jekyll-based site, as described here. Hopefully that will work for you too!

Thank you!
Ok.. Sorry to keep bothering you :sweat_smile:
I've referenced your site's config.
kramdown:
    syntax_highlighter: tree-sitter
    syntax_highlighter_opts:
        tree_sitter_parsers_dir: tree_sitter_parsers

I cloned every language I've written into this directory in my project. I then ran ln -s .. ~/tree_sitter_parsers/tree-sitter-cpp/node_modules in the project's root.
Now I got the following message:
....
          3: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/reader.rb:52:in `block in read_directories'
          2: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/utils.rb:141:in `has_yaml_header?'
          1: from /var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/utils.rb:141:in `open'
/var/lib/gems/2.7.0/gems/jekyll-4.1.1/lib/jekyll/utils.rb:141:in `initialize': Too many levels of symbolic links @ rb_sysopen - /home/sangwoo-joh/type/personal/sangwoo-joh.github.io/tree_sitter_parsers/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules/tree-sitter-cpp/node_modules (Errno::ELOOP)

What am I missing? :cry:

Hmm that doesn't look so good.
I'm not exactly sure how your site is structured, but if my hunch is correct then think I encountered a similar issue in the past and solved it like this. Namely, I added the Tree Sitter parsers directory to my Jekyll exclude list so that Jekyll wouldn't traverse it while building my site. Maybe something similar will work for you?

Hmm that doesn't look so good.
I'm not exactly sure how your site is structured, but if my hunch is correct then think I encountered a similar issue in the past and solved it like this. Namely, I added the Tree Sitter parsers directory to my Jekyll exclude list so that Jekyll wouldn't traverse it while building my site. Maybe something similar will work for you?

Wow, thank you very much!!
Finally I successfully built my blog!! And syntax highlighting with your plugin looks so great. Thank you!!


[workbox-window] Waiting event does not trigger sometimes.

Library Affected: workbox-window
workbox-sw, workbox-build, etc.
Browser & Platform: Google Chrome Version 75.0.3770.142
Issue or Feature Request Description:
Your request here. Issue
Steps to reproduce (sometimes)

Open tab and register service worker

const wb = new Workbox('/sw.js')
this.wb = wb
this.registration = await wb.register()
wb.addEventListener('waiting', this.onSWWaiting)

Keep tab open for an hour, lock screen, switch to another tab.
Build your service worker again, open your website in a new tab.
In the new tab it shows that a new version is available, but it does not trigger in the long open tab.

I have the following Chrome extensions, I don't think any of Chrome apps are suspending the tab. (The js in the long running tab is still running)

Reproducing Repo: https://github.com/azizhk/next-workbox/
Can be deployed on Netlify. (Demo: https://mystifying-almeida-8880f8.netlify.com/)
So you can fork and deploy, open a tab, the run deploy again.
Netlify Commands:

Build Command: yarn build
Publish Directory: out

CC: @philipwalton to confirm.
This sounds to me like the scenario described at https://developers.google.com/web/tools/workbox/modules/workbox-window#when_an_unexpected_version_of_the_service_worker_is_found
There's an externalwaiting event that workbox-window fires when the updated service worker originated from a different tab.
Does that address your issue?

Yup, I guess this must be it.
Is there some heuristic which determines whether waiting or externalwaiting gets triggered.
Because I had waiting triggered on me for some cases even when I installed the SW in a different tab.

Yeah, the heuristic is here. You can see the comments in the code: https://github.com/GoogleChrome/workbox/blob/4e770a6ad2a95db6d7195defcbc34b2636dc7f84/packages/workbox-window/src/Workbox.ts#L343-L374
If you were able to deploy a new version of the service worker in less than a minute, you'd get a false positive on the heuristic.


complete work to disable COSE_Sign1 and/or COSE_Sign

This was started in the Mac0 commit, but didn't go through and disable all COSE signing code.

@laurencelundblade May I ask you to provide more information about this issue?
I was looking at the TEST code (CMakeLists.txt:124), by default the Mac0 tests are disabled,
do we want to disable the sign/verification test cases as well - even when a crypro backend is available?
It is not clear for me, thanks.

Yes, definitely wait. Way too much churn in that area right now.

Hi Laurence,
I've returned to this topic and got a bit confused by the changes on the dev branch.
Is the T_COSE_DISABLE_SIGN1 still needed? RSA and ECDSA were separated from EDDSA (sign_main and sign_eddsa).
The Sign1 structure does not belong exclusively to the sign_main part of the code, it's independent of the signature algorithm.
If I understand it correctly the generation of COSE_Sign1 has been integrated with COSE_Sign in cose_sign_sign and
cose_sign1_sign became just a compatibility layer (as the comments say).
We can generate COSE_Sign/Sign1 structures using both the sign_main and sign_eddsa signers (and the same applies for the verifiers). Would it make sense to rename/replace the T_COSE_DISABLE_SIGN1 option with a T_COSE_DISABLE_SIGN_MAIN / T_COSE_DISABLE_SIGN option for example? Or simply remove it and rely on the T_COSE_DISABLE_ options only?
It would be easier to use a common disabling option for the sign_main module.
Should these DISABLE options be added to the tests/examples?
Thank you,
David

Probably next week before I can get to this. One things I'll say is that what's in GitHub right now for these #ifdefs is not thought out and probably not correct.  I'm pretty sure we'll want an option to disable COSE_Sign because lots of people only want COSE_Sign1 and linker dead stripping won't do it.

Hi David, I’d like to change strategy here now that I’ve thought it through.
I don’t think any #ifdef is necessary at all. Anyone serious about object code reduction will use -dead_strip and that will cause the MAC code to be removed all the way into the crypto adaptor and crypto library if they’re not calling any of the public MAC APIs.
Maybe someone might think to disable T_COSE_ALGORITHM_HMAC512 and T_COSE_ALGORITHM_HMAC384 if they are only using T_COSE_ALGORITHM_HMAC256, but all that’s going to do is save 32 bytes of stack.
That leaves the issue of disabling test code when the library or crypto layer doesn’t support HMAC. The short term answer is to run all the mac tests every time. There’s no #ifdef removing them, but each tests will check to see if the algorithm is available by calling t_cose_is_algorithm_supported(). Grep for t_cose_is_algorithm_supported() and see that a lot of tests do this now with some just returning success if the algorithm is not available.  Longer term I want to do a little better perhaps by the test returning a special error code to indicate they didn’t actually run so the print out of tests run is more accurate.
The issue is a little different for COSE_Sign and COSE_Sign1. You can leave that one to me if you want. My thought there is to replace T_COSE_DISABLE_SIGN1 with T_COSE_DISABLE_SIGN because lots of people only want COSE_Sign1. A #ifdef for COSE_Sign will remove a lot of code. Dead-strip won’t remove much COSE_Sign code. Vice versa, disabling COSE_Sign1 won’t remove that much code if you are using COSE_Sign so there’s not that much need to disable COSE_SIGN1.
When support is added for COSE_Mac over and above COSE_Mac0, we’ll need T_COSE_DISABLE_MAC_WITH_RECIPIENTS for reasons similar to COSE_Sign, but that’s not now.
So the only thing to do here is remove T_COSE_DISABLE_MAC0 and make the MAC tests conditional on t_cose_is_algorithm_supported.
Thank you!

Another reason to reduce the number of T_COSE_DISABLE_XXX is that the build check in tdv scales by the square of the number of these because it checks all combinations. It's also just less stuff the user has to worry about.

Thank you very much Laurence for the clarification, now I have 2 separate PRs to cover these tasks.
Removing the _DISABLE_MAC0 option is covered in this PR: #211
Replacing the _DISABLE_SIGN1 option with _DISABLE_SIGN1 is covered here: #222
This one is still in progress, there are still open questions around it.

@laurencelundblade can we close this issue?

One last fix #230 and this is closed.


What is the Turbolinks roadmap?

Hi folks,

Ground-up rewrite of Turbolinks (v5) for performance & extensibility. Unreleased, but you can follow along here: https://github.com/basecamp/turbolinks/tree/v5/lib/assets/javascripts/turbolinks
-- @sstephenson (https://twitter.com/sstephenson/status/645993147399405568)

So v5 will be the next major release of Turbolinks.
But what will be the backwards-incompatible changes of this fifth release?
Is the Turbolinks.replace() will be kept?
Thanks in advance,
Tortue Torche

It is hard to know. The announcement was made yesterday and only Basecamp people have context about it. That said you can ask this question in their repository.

There's no roadmap right now. Turbolinks 5 is still an R&D project here at Basecamp. Continue using the current version, and we'll make sure there's a migration path everyone's happy with when the time comes.