Connecting Elasticsearch with SSL

Hi There,
I am using ssl to connect my elastic cluster on http port (9200)
Certificate is .pfx file, root and sub certificates also included with chain.
Trying different configurations (config.yaml) to connect elastalert to cluster with, everytime i got different errors like,
SSL: EE_KEY_TOO_SMALL
SSL: CERTIFICATE_VERIFY_FAILED
How can i use .pfx file for elastalert as truststore ? What is the best configuration ?
Thanks in advance.

https://github.com/Yelp/elastalert/issues/2235

Hi @nsano-rururu ,
Let me understand clearly. I seperated the .pfx file to cer and key with no password, and write it in 'elastalert config.yaml' file.
However, it also did not work.
Do i need to change it on server-side ? Do i need to change it on elasticsearch.yaml with different certificate ?
Thanks.

Do i need to change it on server-side ? Do i need to change it on elasticsearch.yaml with different certificate ?

yes

I've tried everything with docker and elastalert-server. Below is the log. I don't know if it will be helpful.
chmod 777 rules rule_templates
/home/user/dkwork2/es2
|--.env
|--config
|  |--api.config.json
|  |--elastalert.yaml
|--create-certs.yml
|--docker-compose-praeco.yml
|--docker-compose.yml
|--instances.yml
|--nginx_config
|  |--default.conf
|  |--nginx.conf
|--public
|  |--praeco.config.json
|--rule_templates
|--rules

.env
COMPOSE_PROJECT_NAME=es
CERTS_DIR=/usr/share/elasticsearch/config/certificates
ELASTIC_PASSWORD=PleaseChangeMe

config/api.config.json
{
  "appName": "elastalert-server",
  "port": 3030,
  "wsport": 3333,
  "elastalertPath": "/opt/elastalert",
  "verbose": false,
  "es_debug": false,
  "debug": false,
  "rulesPath": {
    "relative": true,
    "path": "/rules"
  },
  "templatesPath": {
    "relative": true,
    "path": "/rule_templates"
  },
  "es_host": "es01",
  "es_port": 9200,
  "es_username": "elastic",
  "es_password": "xxxxxxxxx
  ",
  "es_ca_certs": "/usr/share/elasticsearch/config/certificates/ca/ca.crt",
  "es_client_cert": "/usr/share/elasticsearch/config/certificates/es01/es01.crt",
  "es_client_key": "/usr/share/elasticsearch/config/certificates/es01/es01.key",
  "es_ssl": true,
  "writeback_index": "praeco_elastalert_status"
}

config/elastalert.yaml
# The elasticsearch hostname for metadata writeback
# Note that every rule can have its own elasticsearch host
es_host: es01

# The elasticsearch port
es_port: 9200

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: rules

# How often ElastAlert will query elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 60

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 1

# Optional URL prefix for elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to elasticsearch
use_ssl: True

# Verify TLS certificates
#verify_certs: True

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for elasticsearch
es_username: elastic
es_password: xxxxxxxxx

# ssl
ca_certs: /usr/share/elasticsearch/config/certificates/ca/ca.crt
client_cert: /usr/share/elasticsearch/config/certificates/es01/es01.crt
client_key: /usr/share/elasticsearch/config/certificates/es01/es01.key

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: praeco_elastalert_status

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

skip_invalid: True

create-certs.yml
version: '2.2'

services:
  create_certs:
    container_name: create_certs
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    command: >
      bash -c '
        if [[ ! -f /certs/bundle.zip ]]; then
          bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;
          unzip /certs/bundle.zip -d /certs;
        fi;
        chown -R 1000:0 /certs
      '
    user: "0"
    working_dir: /usr/share/elasticsearch
    volumes: ['certs:/certs', '.:/usr/share/elasticsearch/config/certificates']

volumes: {"certs"}

instances.yml
instances:
  - name: es01
    dns:
      - es01
      - localhost
    ip:
      - <IP_ADDRESS>

  #- name: es02
  #  dns:
  #    - es02
  #    - localhost
  #  ip:
  #    - <IP_ADDRESS>

docker-compose.yml
version: '2.2'

services:
  es01:
    container_name: es01
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.0
    environment:
      - discovery.type=single-node
      - cluster.name=es01
      #- node.name=es01
      #- discovery.seed_hosts=es01,es02
      #- cluster.initial_master_nodes=es01,es02
      - network.host=<IP_ADDRESS>
      - ELASTIC_PASSWORD=$ELASTIC_PASSWORD
      - "ES_JAVA_OPTS=-Xms256m -Xmx256m"
      - xpack.license.self_generated.type=trial
      - xpack.security.enabled=true
      - xpack.security.http.ssl.enabled=true
      - xpack.security.http.ssl.key=$CERTS_DIR/es01/es01.key
      - xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.http.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.enabled=true
      - xpack.security.transport.ssl.verification_mode=certificate
      - xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
      - xpack.security.transport.ssl.certificate=$CERTS_DIR/es01/es01.crt
      - xpack.security.transport.ssl.key=$CERTS_DIR/es01/es01.key
    volumes: ['data01:/usr/share/elasticsearch/data', 'certs:$CERTS_DIR']
    ports:
      - 9200:9200
    healthcheck:
      test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
      interval: 30s
      timeout: 10s
      retries: 5

volumes: {"data01", "certs"}

docker-compose-praeco.yml
version: '3'

services:
  elastalert:
    image: 'praecoapp/elastalert-server'
    ports:
      - 3030:3030
      - 3333:3333
    volumes:
      - ./config/elastalert.yaml:/opt/elastalert/config.yaml
      - ./config/api.config.json:/opt/elastalert-server/config/config.json
      - ./rules:/opt/elastalert/rules
      - ./rule_templates:/opt/elastalert/rule_templates
      - certs:/usr/share/elasticsearch/config/certificates
    #extra_hosts:
    #  - 'elasticsearch:${PRAECO_ELASTICSEARCH}'
    tty: true
    networks:
      - default
      - es_default

  webapp:
    image: 'praecoapp/praeco'
    ports:
      - 8080:8080
    volumes:
      - ./public/praeco.config.json:/var/www/html/praeco.config.json
      - ./nginx_config/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx_config/default.conf:/etc/nginx/conf.d/default.conf
      - certs:/usr/share/elasticsearch/config/certificates
    tty: true
    networks:
      - default
      - es_default

volumes: { "certs" }

networks:
  es_default:
    external: true

$ docker-compose -f create-certs.yml run --rm create_certs

Creating network "es_default" with the default driver
Creating volume "es_certs" with default driver
Archive:  /certs/bundle.zip
   creating: /certs/ca/
  inflating: /certs/ca/ca.crt
   creating: /certs/es01/
  inflating: /certs/es01/es01.crt
  inflating: /certs/es01/es01.key

$ docker-compose up -d

Creating volume "es_data01" with default driver
Creating es01 ... done

$ docker run --rm -v es_certs:/certs --network=es_default docker.elastic.co/elasticsearch/elasticsearch:7.10.0 curl --cacert /certs/ca/ca.crt -u elastic:PleaseChangeMe https://es01:9200

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (7) Failed to connect to es01 port 9200: Connection refused

$ docker exec es01 /bin/bash -c "bin/elasticsearch-setup-passwords \
auto --batch \
--url https://localhost:9200"

# Should be output with a different value each time

Changed password for user apm_system
PASSWORD apm_system = aaaaaaaaaaaa

Changed password for user kibana_system
PASSWORD kibana_system = bbbbbbbbb

Changed password for user kibana
PASSWORD kibana = ddddddddddd

Changed password for user logstash_system
PASSWORD logstash_system = eeeeeeeeee

Changed password for user beats_system
PASSWORD beats_system = fffffffffff

Changed password for user remote_monitoring_user
PASSWORD remote_monitoring_user = gffffffff

Changed password for user elastic
PASSWORD elastic = xxxxxxxxx

$ curl -u elastic:xxxxxxxxx https://localhost:9200/_cat/indices?v

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

$ curl -u elastic:xxxxxxxxx localhost:9200?pretty

curl: (52) Empty reply from server

$ docker run --rm -v es_certs:/certs --network=es_default docker.elastic.co/elasticsearch/elasticsearch:7.10.0 curl --cacert /certs/ca/ca.crt -u elastic:xxxxxxxxx https://es01:9200/_cat/indices?v

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   214  100   214    0     0    697      0 --:--:-- --:--:-- --:--:--   697
health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .security-7 HVZs78GJTV6A1EbjTG_2Fg   1   0          7            0     25.1kb         25.1kb

$ docker ps

CONTAINER ID        IMAGE                                                  COMMAND                  CREATED             STATUS                   PORTS                              NAMES
65263085b175        docker.elastic.co/elasticsearch/elasticsearch:7.10.0   "/tini -- /usr/local…"   2 minutes ago       Up 2 minutes (healthy)   <IP_ADDRESS>:9200->9200/tcp, 9300/tcp   es01

$ docker-compose -f docker-compose-praeco.yml up -d

$ docker logs -f es_elastalert_1

><EMAIL_ADDRESS>start /opt/elastalert-server
> sh ./scripts/start.sh

15:42:37.418Z  INFO elastalert-server: Config:  No config.dev.json file was found in /opt/elastalert-server/config/config.dev.json.
15:42:37.421Z  INFO elastalert-server: Config:  Proceeding to look for normal config file.
15:42:37.443Z  INFO elastalert-server: Config:  A config file was found in /opt/elastalert-server/config/config.json. Using that config.
15:42:37.506Z  INFO elastalert-server: Router:  Listening for GET request on /.
15:42:37.507Z  INFO elastalert-server: Router:  Listening for GET request on /status.
15:42:37.507Z  INFO elastalert-server: Router:  Listening for GET request on /status/errors.
15:42:37.508Z  INFO elastalert-server: Router:  Listening for GET request on /rules.
15:42:37.511Z  INFO elastalert-server: Router:  Listening for GET request on /rules/:id*.
15:42:37.512Z  INFO elastalert-server: Router:  Listening for POST request on /rules/:id*.
15:42:37.513Z  INFO elastalert-server: Router:  Listening for DELETE request on /rules/:id*.
15:42:37.513Z  INFO elastalert-server: Router:  Listening for GET request on /templates.
15:42:37.513Z  INFO elastalert-server: Router:  Listening for GET request on /templates/:id*.
15:42:37.513Z  INFO elastalert-server: Router:  Listening for POST request on /templates/:id*.
15:42:37.514Z  INFO elastalert-server: Router:  Listening for DELETE request on /templates/:id*.
15:42:37.514Z  INFO elastalert-server: Router:  Listening for PUT request on /folders/:type/:path*.
15:42:37.515Z  INFO elastalert-server: Router:  Listening for DELETE request on /folders/:type/:path*.
15:42:37.515Z  INFO elastalert-server: Router:  Listening for POST request on /test.
15:42:37.515Z  INFO elastalert-server: Router:  Listening for POST request on /silence/:path*.
15:42:37.515Z  INFO elastalert-server: Router:  Listening for GET request on /config.
15:42:37.516Z  INFO elastalert-server: Router:  Listening for POST request on /config.
15:42:37.516Z  INFO elastalert-server: Router:  Listening for POST request on /download.
15:42:37.516Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert.
15:42:37.516Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert_status.
15:42:37.516Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/silence.
15:42:37.524Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/elastalert_error.
15:42:37.525Z  INFO elastalert-server: Router:  Listening for GET request on /metadata/past_elastalert.
15:42:37.525Z  INFO elastalert-server: Router:  Listening for GET request on /indices.
15:42:37.525Z  INFO elastalert-server: Router:  Listening for GET request on /mapping/:index.
15:42:37.527Z  INFO elastalert-server: Router:  Listening for POST request on /search/:index.
15:42:37.528Z  INFO elastalert-server: Router:  Listening for GET request on /config.
15:42:37.535Z  INFO elastalert-server: ProcessController:  Starting ElastAlert
15:42:37.535Z  INFO elastalert-server: ProcessController:  Creating index
15:42:42.772Z  INFO elastalert-server:
    ProcessController:  Elastic Version: 7.10.0
    Reading Elastic 6 index mappings:
    Reading index mapping 'es_mappings/6/silence.json'
    Reading index mapping 'es_mappings/6/elastalert_status.json'
    Reading index mapping 'es_mappings/6/elastalert.json'
    Reading index mapping 'es_mappings/6/past_elastalert.json'
    Reading index mapping 'es_mappings/6/elastalert_error.json'
    New index praeco_elastalert_status created
    Done!

15:42:42.772Z ERROR elastalert-server:
    ProcessController:  /home/node/.local/lib/python3.8/site-packages/elasticsearch/connection/base.py:193: ElasticsearchDeprecationWarning: Camel case format name dateOptionalTime is deprecated and will be removed in a future version. Use snake case name date_optional_time instead.
      warnings.warn(message, category=ElasticsearchDeprecationWarning)
    /home/node/.local/lib/python3.8/site-packages/elasticsearch/connection/base.py:193: ElasticsearchDeprecationWarning: [types removal] Using include_type_name in put mapping requests is deprecated. The parameter will be removed in the next major version.
      warnings.warn(message, category=ElasticsearchDeprecationWarning)

15:42:42.772Z  INFO elastalert-server: ProcessController:  Index create exited with code 0
15:42:42.773Z  INFO elastalert-server: ProcessController:  Starting elastalert with arguments [none]
15:42:42.827Z  INFO elastalert-server: ProcessController:  Started Elastalert (PID: 37)
15:42:42.831Z  INFO elastalert-server: Server:  Server listening on port 3030
15:42:42.833Z  INFO elastalert-server: Server:  Websocket listening on port 3333
15:42:42.835Z  INFO elastalert-server: Server:  Server started


Blinking cursor and exiting

I didn't realize at first when I saw the ball sometimes was 1-char, otherwise 2-char. It turns out the second one was the blinking cursor.
It should be able to be resolved with ^[[?25l / ^[[?25h, from urxvt(7):
"Pm = 25" (DECTCEM)
    h   Visible cursor {cnorm/cvvis}
    l   Invisible cursor {civis}

About the exiting, which seems to be a bit of less elegant, the shell prompt reappeared right after the URL in the title screen, which is just a middle of lines.
A quick fix is using sout.Clear(), but a better one is to save and restore using 1049, that is:
"Pm = 1047" (X11 XTerm alternate screen buffer)
    h   Use Alternate Screen Buffer
    l   Use Normal Screen Buffer - clear Alternate Screen Buffer if returning from it
"Pm = 1048" (X11 XTerm alternate DECSC)
    h   Save cursor position
    l   Restore cursor position
"Pm = 1049" (X11 XTerm 1047 + 1048)
    h   Use Alternate Screen Buffer - clear Alternate Screen Buffer if switching to it
    l   Use Normal Screen Buffer

I'd open a PR, but I don't really sure where I should put these codes in. I thought about the constructor and destructor Sin, but that doesn't sound quite right, so I am creating this issue.
By the way, maybe SOut::Clear() should use escape codes (erase display/move cursor) instead of making system() call.

Thank you. I really appreciate for your help. I will check the problems and fix them.

First, I fixed blinking cursor first. (75022b9) Now ^[[?25l is automatically called when the constructor of SOut is called at first, and ^[[?25h is for the destructor.

@pauis Good, I can see it's fixed now. One down, one to go.

1049 was used to switch buffers, and Ps J was used to replace system("clear").
"ESC [ Ps J"
Erase in Display ( ED )

But 1049 don't do their work at some devices (e.g. Raspberry Pi bash), so I'll locate the cursor end of the screen so that all command prompt print the exiting more appropriately. I'm working on it.


Implement DAOs for Gym and Badge

Implements DAOs for Gym and Badge, uncovered some bugs in the tests, which are reported in #42 and #43 .

Class javadoc will be added in next commit, please ignore that its missing in your reviews for now.


Create Texture from OpenGL texture

Following up on this issue: https://github.com/google-ar/sceneform-android-sdk/issues/81#event-1768673219 I'd like to propose a feature to allow somehow populating a SceneForm texture from a native OpenGL texture handle. The specific use case here would be to texture geometry in a SceneForm scene with a video texture (such as from video rendered onto a SurfaceTexture).
Any means of supporting this would be a fine I think: constructing a Texture from a user-defined GL texture id, or Texture being able to provide a GL id that the user is able to do their own copy to, etc.

In Sceneform 1.4 we did release an api for ExternalTexture's, there is an example here showing how to use it to play chroma keyed videos.

Oh interesting, that might be all we need then. Feel free to close this issue then, thanks!


[ui] Additional Logging Configuration update

An option for JSON parsing needs to be added under the Additional Logging Configuration section when enabling a logging provider.
A new field enableJSONParsing is being added to the logging config.
Related issue: https://github.com/rancher/rancher/issues/23646
backend PR: https://github.com/rancher/rancher/pull/24361

Available in latest2 and latest-2.3

@westlywright it seems we're not sending true in the UI for the toplevel enableJSONParsing ?
https://github.com/rancher/rancher/issues/23646#issuecomment-564723871

The enableJSONParsing  only lives on clusterLogging, not the logging provider configs. It's global for everybody.

Validated this is working at the project level in versions:
2.4 (12/16/2019) Rancher version bbb75e098
2.3-head (12/16/2019) Rancher version 78ed261c6

Cluster level logging I hit the following issue, which blocks me to finish the validation. https://github.com/rancher/rancher/issues/24545
@westlywright @dramich @sangeethah

validated working after fix of #24545
2.3-head (12/27/2019)
2019/12/27 16:05:06 [INFO] Rancher version e9a549e59 is starting

master-head (12/27/2019)
2019/12/27 16:33:53 [INFO] Rancher version 189e4337c is starting


0.7.290  memory leak during apero_database.py --update

We moved from the 288 to the 290 for the quicklook processing, as we couldn't fix the issue wit the obj_db.
When I ran apero_database.py --update to rebuild the db from the old folders (tellu and calib) the RAM usage increased
linearly as shown below.   The full rebuilding took 33h, with 200,000 files in tellu/.
I got no error messages at the end of the apero_database.py and the new quicklook290 seems to work as expected. I'm catching up with the nights of current run.

I mean to me if the RAM goes down without killing python its not a "leak" of memory - its just a lot being kept open (for some reason) that is then closed at some point. Though this is just semantics, I tried with v0.7 to fix this and close more things but it is a complex problem.
I aim in v0.8 to address this and hope that having all this stuff "open" for so long goes away (and will thus fix the problem with using multiprocessing - where this seems to climb way higher).


(PUP-9094) Switch rest client back to our HttpPool

We had been working to have our rest client use the httpclient gem for
cert related HTTP requests. Unfortunately, we weren't able to finish
that work in time for Puppet 6.0.0. This commit switches those requests
back to using Puppet::Network::HttpPool.
Unfortunately, this isn't a clean switch back. We are preserving all the
existing work as much as possible so we can more easily get back to the
httpclient gem in the future. For now, we have to sacrifice some nice
new functionality in favor of the old way of doing things. Notably, if
users have an invalid or out of date cert on their agent, they have to
remove it as the automation will not update it. This is consistent with
how things used to work prior to the httpclient gem.

This is blocked on getting @Magisus 's work in, so we can rebase this work on hers

I think there are several more spots that need to be updated, e.g. when submitting a CSR or downloading a CRL.

@melissa merged the CA removal. Are you working on updates to the other queries?

CLA signed by all contributors.

@Magisus yes, that's currently in progress. Once we get that working, I'll rebase with your stuff

@Magisus okay, this has been updated and rebased against 6.0.x!

I'm a little confused about the different verify modes we're using, but this does seem to work. I tested it against an intermediate CA with no issues.

@Magisus updated all verify_peer calls to false

Updated!


adding ppt presentation of PDO

Signed-off-by: Bruno Vavala<EMAIL_ADDRESS>

@TomBarnes @cmickeyb Do you have an opinion on checking in slides vs continuing to host them on google docs?

Ok, I've been overruled - merged this

later I will add a link to the paper as well, and a note to contact us in case the link does not work.


chore: add token refreshing and refine the fbc test cases

Signed-off-by: Jing Qi<EMAIL_ADDRESS>The fbc test cases need to take several minutes for build pipelineRun and release pipelineRun. Since the access token expired sooner than before, the test cases
are refined and added the token refreshing during the test.
Description
Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.
Issue ticket number and link
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Checklist:

[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added meaningful description with JIRA/GitHub issue key(if applicable), for example HASSuiteDescribe("STONE-123456789 devfile source")
[ ] I have updated labels (if needed)

/approve

/retest

/lgtm

/retest

/retest


Convert uniXXXX glyph names to proper ones when building the charCodeToGlyphId map for TrueType fonts (bug 1132849, issue 6893, issue 6894)

Some PDF generators, e.g. Scribus PDF, use improper uniXXXX glyph names which breaks the glyph mapping. We can avoid this by converting them to "standard" glyph names instead.
Fixes https://bugzilla.mozilla.org/show_bug.cgi?id=1132849.
Fixes #6893.
Fixes #6894.
/cc @timvandermeij

/botio-linux preview

/botio test

This is awesome! I can confirm that this fixes both issues I filed. The original and the reduced test case from the Bugzilla issue already seem to be fixed for me with the current master (on Arch Linux when comparing PDF.js with Okular), but nevertheless it's good to include it as a reduced test case.
@brendandahl Could you please review this?

The original and the reduced test case from the Bugzilla issue already seem to be fixed for me with the current master (on Arch Linux when comparing PDF.js with Okular), but nevertheless it's good to include it as a reduced test case.

For me there's actually a difference on Windows (with HWA on), which is why I included it.

Although there's now a couple of somewhat similar test-cases, which might be unnecessary. The already existing issue6889.pdf could probably be replaced with issue6894.pdf in this patch, since those two appear to be using the exact same font as (but the new one covers more issues).
However, I didn't want to remove any existing tests in this patch.

I've update the patch slightly, to only attempt this conversion for the this.differences case. Since otherwise we just read from one of the standard encodings, which we know don't suffer from this issue.
/botio test

/botio-linux test

/botio-windows test

This is looking good. I was curious how other PDF viewers handle this and found https://pdfium.googlesource.com/pdfium/+/master/core/src/fxge/freetype/fx_freetype.cpp#56 . Seems like it would be a good idea to make a general function for getting the unicode from glyph name. There function handles names like 'u123456' and a few other cases. Now I wonder if we have any broken pdfs with names like the above.

Seems like it would be a good idea to make a general function for getting the unicode from glyph name.

Sure, that sounds fine! The only one I can remember seeing in practice is uniXXXX, but I suppose it can't hurt to support the uXXXX/uXXXXXX case as well.
@brendandahl How closely do you want that function to mimic what the FreeType code does?
Should we enforce only upper-case hexadecimal chars, to prevent accidentally matching things that are not actually hex numbers?

I've added a getUnicodeForGlyph helper function, similarly to the FreeType one, which is used in various places to do the glyphName -> Unicode conversion.
I kept the recoverGlyphName, which is using getUnicodeForGlyph internally, since we may need both the standard/non-standard glyph names when creating the charCodeToGlyphId map for TrueType fonts.
@brendandahl Is this what you had in mind?
/botio-linux preview

@brendandahl Is this what you had in mind?

Yes, looks very good. Just the minor nits above.

Yes, looks very good. Just the minor nits above.

Thanks for the review, I've addressed the comments.
/botio test

r+
/botio makeref

/botio-linux makeref

/botio-linux makeref


How will First Party Sets treat unpartitioned cookies after 3rd party cookie deprecation?

The CHIPS page states that cookies must be set with the Partitioned attribute in order to be sent on cross-party requests once (unpartitioned) third-party cookies are obsoleted.
How does this affect First Party Sets? Assume site test1.com and test2.com are in the same FPS. Test2.com sets an unpartitioned cookie. Will Test1.com still be able to access test2.com's cookie?

How does this affect First Party Sets? Assume site test1.com and test2.com are in the same FPS. Test2.com sets an unpartitioned cookie. Will Test1.com still be able to access test2.com's cookie?

Note that test1.com cannot directly access test2.com's cookie (if it did, this would be a violation of the Same Origin Policy for cookies), even in today's status quo where cross-site cookies exist. However, test2.com can access it's own cookies.
After cross-site cookies are disabled; if test1.com and test2.com are in the same FPS, test2.com can continue to access it's own cookies only after Storage Access API (SAA) is appropriately invoked. See this article for details.
Usually, this means that test1.com can make a credentialed request to test2.com, and the browser will attach test2.com's cookies as long as the storage access permission has been granted to the site (by virtue of the FPS+SAA integration).

Yeah I failed to include, Test2.com is embedded on Test1.com. In another tab, Test2.com is its own top-level site. Test2.com sets an unpartitioned non-CHIPS cookie after 3rd-party cookie deprecation. So in this case, will the embedded test2.com be able to access the unpartitioned cookie set by test2.com in the next tab?

So in this case, will the embedded test2.com be able to access the unpartitioned cookie set by test2.com in the next tab?

Yes! As long as Storage Access API is invoked correctly before the relevant requests to test2.com are made, it will be able to access the unpartitioned cookie.


Ability to sort symbols in the diff view

I was thinking it'd be useful to be able to sort symbols by...

Offset (the current behavior)
Total symbol size
Unmatched bytes remaining (or maybe remaining match percentage? I think byte count would be more useful though)
Alphabetical by name (for finding symbols quickly)

For selecting sort order, I think either a flyout menu from Diff Options (like Recent Projects), or a popup (like Algorithm), would suffice. It should override the project settings as it's a user-level preference. The user selects one sort method, and can select it again for reverse order. All sections are sorted by the same criterion.
Also, it seems like this could replace Reverse function order by just being Offset (descending), but I'm not very familiar with the use case of the existing option.
Given approval of the idea and some pointers on implementation, I wouldn't mind implementing it.

Sure, I like this idea. Though instead of inside Diff Options, maybe it'd make more sense to be a "sort" icon in the symbol list itself (or next to each section header?)
For "Reverse function order": that's specifically for objects compiled with -inline deferred, which reverses the function order in the object. To match the order of the functions in the source file, they're (un)reversed in the UI.
Since that's an object-level property (it's set by dtk-template if -inline deferred is set in that object's cflags), and this new feature is a user view preference, it makes sense for them to remain separate. Though how they interact is another question.

Though how they interact is another question.

I think it'd make sense for the object-level setting to apply first, and the user-level setting has no awareness of it. So the "base order" that the view receives is already reversed if deferred is set.
Where would you want the sorting of the model to happen? I assume it should be persistently sorted so that the render function isn't running the sort every frame.
Is this the line that sorts them currently?
https://github.com/encounter/objdiff/blob/0a85c498c5e38741114761d98bcdaf176aa0ccfd/src/obj/elf.rs#L118

Yes, that's the baseline sort when loading the object. I think this should happen in the view, though. Probably here, alongside the existing logic: https://github.com/encounter/objdiff/blob/0a85c498c5e38741114761d98bcdaf176aa0ccfd/src/views/symbol_diff.rs#L274
It's totally fine to do the sort every frame. The beauty of immediate mode UI!


EXSWHTEC-285 - Implement tests for exponential and power device math functions

Depends on: #260

Accuracy tests for single-precision and double-precision exp/exp2/exp10/expm1/pow/ldexp/powi/scalbn/scalbln/frexp
Negative tests for single-precision and double precision exponential and power math functions

PR has been merged into develop branch via commit 87d601411b1364fc625c59dd4c053f05726c5313.


bug: multiple nested rendering of custom components

When the slots of a custom component are html elements, this results in multiple layers of duplicate rendering!
react

solid

Would you like to send a PR to address this issue?

Would you like to send a PR to address this issue?

I tried to solve this problem but I don't really understand solid.

Hey @arthur-fontaine 👋 ，The main reason for #15 and #16 is that createComponent in createComponent.tsx is directly judged by if (props.children), and since props may be empty, it reports an error, and by changing it to props?.children, the page doesn't report the error. But there are still some problems!


[v2] Refactor configuration API

Context:
https://github.com/swc-project/swc/issues/5720#issuecomment-1236054505
I think something like starlark, which has well-defined semantics, would be much better for users.
Tasks

[ ] Remove support for arrays in .swcrc
[ ] Determine configurations to remove
[ ] Investigate configuration languages like starlark

cc @kwonoj

@kwonoj Are you going to provide alternative if you remove support for arrays in your next release

This never worked correctly, so no at this moment until we have proper RFC to satisfy most of usecases.

Is there any I can use multiple parsers in swc config then?

Use different config file, or use different config object per calling transform API.
Please do not use this meta issue for asking individual questions.

@kwonoj As the support for arrays being removed, could you give me an example of how to specify multiple parsers with .swcrc


Remove unused, deprecated methods

The usage of these methods was removed in https://github.com/exercism/x-api/pull/147

Unlike my other PRs, this one can be merged.

Rebased on master after the merge of #24. Expected conflicts, but didn't get any. Yay.

How did I miss this? I'm so sorry!


add trial_end,trial_period_days on subscription_data

I wanna use trial_end and trial_period_days in subscriptions checkout session.
https://stripe.com/docs/api/checkout/sessions/create#create_checkout_session-subscription_data

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


New version

Hi!
thank You very mutch for your work ))
I have a two questions:

I`s installd BBB-MP4 about two mounth ago, but now exixst a new version. What must I do for update my BBB-MP4 to lasted version?
Can I to convert my old video in BBB to mp4 via  BBB-MP4?
Thank You again!

@Korney99

You can simply install it again ,This is fully dockerized so no need to worry about anything.
You can convert existing recording to mp4 by running sh bbb-mp4.sh meetingID

Thank`s for your help )).


Speed up CI

We can probably do some more caching in CI to speed things up.
Would be best if we nix-ified it as well

nix-ified it as well

We do this for a few of the servers right now and I have not figured out how to make it faster. We cache some of the nix stuff, but I think it pulls in all of the cargo files too. I think if we can cache those too, it could be faster. But right now it's like 15 minutes with the whole fedimint stack that it pulls in.

But right now it's like 15 minutes with the whole fedimint stack that it pulls in

fedimint's CI only takes a couple mins so it should be possible to get it pretty low


Change style of the button

Need to change style of the "CRX Package Manager" button in main menu for general style or more visible

I would like to contribute, please assign me  this issue

Hey, can I try? I am still a newbie to open source as well as web development but I would like to give it a try.

You are welcome)

@AlKaliada assign me


fix reference to Visual Studio IDE

fix reference to Visual Studio IDE
Summary
I believe the reference to Visual Studio Code is actually intended to be a reference to the Visual Studio IDE.
The outlining feature doesn't appear to be work OOTB for the #region preprocessor directive in VS Code.
(I don't think VS Code recognizes C# syntax w/o the use of an add-on)

hahaha
The documentation is referring to the Code Editor in Visual Studio... Not the editor called Visual Studio Code
I bet I'm not the only one that made this mistake...


Mixed Java/Scala build with circular dependencies

I have a project which has mixed scala/java code and they interdepend on each other :
scala Intf : A.scala
java classes : B.java, C.java
scala class : D.scala
class B implements A {} //java deps on scala
class D {
    C c;
}

i have BUILD rules , which build , first scala and then java using deps as the scala lib.
building the scala , requires i specify the java files as srcs for the scala_library rule, which leads to the java files gettign compiled , and then again in the java_library.
My project is pretty big, so specifying individual classes as sources isn't an option.

filegroup (
    name = "main-scala"
    srcs = glob(["src/**/*.java", "src/**/*.scala"])
)

filegroup (
    name = "main-java"
    srcs = glob(["src/**/*.java"])
)

scala_library(
    name = "scalaLib",
    srcs = [":main-scala-sources"],
    deps = [ <list_of_deps>]
)

java_library(
    name = "javaLib",
    srcs = [":main-java-sources"],
    deps = [":scalaLib", <other_deps>]
)

javaLibrary(
    name = "exportedLib"
    exports = [":scalaLib", ":javaLib" ]
)

the ":exportedLib" target doesn't seem to export a single consolidated jar which would encompass bothe the scala and java classes, rather just has a Manifest entry, nothing else.
Am i doing something wrong, or is the only way to build java/scala mixed projects?

Fortunately that’s not the way.
If you have cycles you should specify all of the needed sources (java and
scala) as sources of a scala_library and it will compile all of them
correctly.
You can see our tests for examples
On Mon, 27 Aug 2018 at 12:34 Rahul Roy<EMAIL_ADDRESS>wrote:

I have a project which has mixed scala/java code and they interdepend on
each other :
scala Intf : A.scala
java classes : B.java, C.java
scala class : D.scala
class B implements A {} //java deps on scala
class D {
C c;
}
i have BUILD rules , which build , first scala and then java using deps as
the scala lib.
building the scala , requires i specify the java files as srcs for the
scala_library rule, which leads to the java files gettign compiled , and
then again in the java_library.
My project is pretty big, so specifying individual classes as sources
isn't an option.
filegroup (
name = "main-scala"
srcs = glob(["src//*.java", "src//*.scala"])
)
filegroup (
name = "main-java"
srcs = glob(["src/**/*.java"])
)
scala_library(
name = "scalaLib",
srcs = [":main-scala-sources"],
deps = [ <list_of_deps>]
)
java_library(
name = "javaLib",
srcs = [":main-java-sources"],
deps = [":scalaLib", <other_deps>]
)
javaLibrary(
name = "exportedLib"
exports = [":scalaLib", ":javaLib" ]
)
the ":exportedLib" target doesn't seem to export a single consolidated jar
which would encompass bothe the scala and java classes, rather just has a
Manifest entry, nothing else.
Am i doing something wrong, or is the only way to build java/scala mixed
projects?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/597, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABUIF6ENOnXnG4n12i3gh6F9B_n58xuZks5uU70lgaJpZM4WNeks
.

@ittaiz do you mind putting up a quick bazel sample of a java-scala mix repo that works and putting it on github? I have been struggling to build a  java-scala mix repo (java calls scala and scala calls java) with little success.

https://github.com/ittaiz/bazel-scala-getting-started
On Fri, Nov 22, 2019 at 4:08 AM Punit Agrawal<EMAIL_ADDRESS>wrote:

@ittaiz https://github.com/ittaiz do you mind putting up a quick bazel
sample of a java-scala mix repo that works and putting it on github? I have
been struggling to build a java-scala mix repo (java calls scala and scala
calls java) with little success.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/rules_scala/issues/597?email_source=notifications&email_token=AAKQQF2RCJ5OK42ISKDGGGDQU45JRA5CNFSM4FRV5EWKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE4JA5A#issuecomment-557355124,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKQQF22S26EPMATI3ISVQDQU45JRANCNFSM4FRV5EWA
.

--
Ittai Zeidman
Cell:<PHONE_NUMBER>
40 Hanamal street, Tel Aviv, Israel
http://www.wix.com


WaveFrontReader fails when mtl names have spaces in them

meshconvert.exe fails to process .obj files that refer to .mtl files with spaces in their names, and .mtl files that refer to image files with spaces in their names. It’s a result of the fstream implementation of >> for strings. It delimits based on spaces.
Use cup.obj file as an example. Rename the .mtl, and .jpg files to “microsoft cup.xxx” and edit the .obj and .mtl files to reference the renamed files (both files are pretty short and can be edited with notepad or VS).

I'm not sure that spaces in the filenames specified in a Wavefront OBJ are allowed. Looking for an example of such a file.


[BUG] EnvironmentCredential kicks is when environment variables are empty string?

Describe the bug
Per the doc for  DefaultAzureCredential the order of the underlying credentials is the following:

EnvironmentCredential
ManagedIdentityCredential

What means that if the former kicks-in but fails the the latter would never kick-in and be tried. And that's what apparently happens in our case.
Our service in production is running as a Azure Service Fabric application on VMSS and acquires a token using User-assigned Managed Identity. Locally we're setting the necessary environment variables to mimic the behavior.
Service Fabric application manifest is an XML file that allows to set environment variables to either a value or an empty string. It doesn't allow to set the value to null:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="MyServiceApplicationType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
  <Parameters>
    <Parameter Name="AzureClientId" DefaultValue="" />
    <Parameter Name="AzureTenantId" DefaultValue="" />
  </Parameters>
  <EnvironmentOverrides CodePackageRef="Code">
    <EnvironmentVariable Name="AZURE_CLIENT_ID" Value="[AzureClientId]" />
    <EnvironmentVariable Name="AZURE_TENANT_ID" Value="[AzureTenantId]" />
  </EnvironmentOverrides>
</ApplicationManifest>

What means that the environment variables are set as empty strings. The != checks in EnvironmentCredential.cs are not enough to skip it altogether.
Expected behavior
Token is acquired by ManagedIdentityCredential, not by EnvironmentCredential because it's skipped due to missing environment variables.
Actual behavior (include Exception or Stack Trace)
Invalid tenant id provided. You can locate your tenant id by following the instructions listed here: https://docs.microsoft.com/partner-center/find-ids-and-domain-names

The type initializer for 'Azure.Identity.DefaultAzureCredential' threw an exception.
at Azure.Identity.DefaultAzureCredential.GetDefaultAzureCredentialChain(DefaultAzureCredentialFactory factory, DefaultAzureCredentialOptions options)
at Azure.Identity.DefaultAzureCredential..ctor(DefaultAzureCredentialFactory factory, DefaultAzureCredentialOptions options)
at Azure.Identity.DefaultAzureCredential..ctor(DefaultAzureCredentialOptions options)
at Azure.Identity.DefaultAzureCredential..ctor(Boolean includeInteractiveCredentials)
at Eds.Platform.Identity.AzureAdTokenProvider..ctor(ILogger`1 logger)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitRootCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitConstructor(ConstructorCallSite constructorCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.<>c__4`1.<AddHttpMessageHandler>b__4_1(HttpMessageHandlerBuilder b)
at Microsoft.Extensions.Http.DefaultHttpClientFactory.<>c__DisplayClass17_0.<CreateHandlerEntry>g__Configure|0(HttpMessageHandlerBuilder b)
at Microsoft.Extensions.Http.LoggingHttpMessageHandlerBuilderFilter.<>c__DisplayClass2_0.<Configure>b__0(HttpMessageHandlerBuilder builder)
at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateHandlerEntry(String name)
at Microsoft.Extensions.Http.DefaultHttpClientFactory.<>c__DisplayClass14_0.<.ctor>b__1()
at System.Lazy`1.ViaFactory(LazyThreadSafetyMode mode)
at System.Lazy`1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor)
at System.Lazy`1.CreateValue()
at System.Lazy`1.get_Value()
at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateHandler(String name)
at Microsoft.Extensions.Http.DefaultHttpClientFactory.CreateClient(String name)
at Microsoft.Extensions.DependencyInjection.HttpClientBuilderExtensions.<>c__DisplayClass12_0`2.<AddTypedClientCore>b__0(IServiceProvider s)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitDisposeCache(ServiceCallSite transientCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetService[T](IServiceProvider provider)
at Eds.Cdp.AccountManagementService.Client.AccountManagementServiceCollectionExtensions.<>c__DisplayClass2_0.<AddTenantDataProvider>b__0(IServiceProvider provider)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitFactory(FactoryCallSite factoryCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSiteMain(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitCache(ServiceCallSite callSite, RuntimeResolverContext context, ServiceProviderEngineScope serviceProviderEngine, RuntimeResolverLock lockType)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.VisitScopeCache(ServiceCallSite singletonCallSite, RuntimeResolverContext context)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteVisitor`2.VisitCallSite(ServiceCallSite callSite, TArgument argument)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.CallSiteRuntimeResolver.Resolve(ServiceCallSite callSite, ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.DynamicServiceProviderEngine.<>c__DisplayClass1_0.<RealizeService>b__0(ServiceProviderEngineScope scope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngine.GetService(Type serviceType, ServiceProviderEngineScope serviceProviderEngineScope)
at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
at Microsoft.Extensions.DependencyInjection.ActivatorUtilities.GetService(IServiceProvider sp, Type type, Type requiredBy, Boolean isDefaultParameterRequired)
at lambda_method(Closure , IServiceProvider , Object[] )
at Microsoft.AspNetCore.Mvc.Controllers.ControllerActivatorProvider.<>c__DisplayClass4_0.<CreateActivator>b__0(ControllerContext controllerContext)
at Microsoft.AspNetCore.Mvc.Controllers.ControllerFactoryProvider.<>c__DisplayClass5_0.<CreateControllerFactory>g__CreateController|0(ControllerContext controllerContext)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextExceptionFilterAsync>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ExceptionContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Builder.RouterMiddleware.Invoke(HttpContext httpContext)
at Eds.Platform.Identity.ServiceToServiceAuthMiddleware.InvokeAsync(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

To Reproduce
TBD
Environment:

Name and version of the Library package used:

Azure.Identity 1.3.0

Hosting platform or OS and .NET runtime version (dotnet --info output for .NET Core projects):

.NET SDK (reflecting any global.json):
 Version:   5.0.101
 Commit:    d05174dc5a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.101\

Host (useful for support):
  Version: 5.0.1
  Commit:  b02e13abab

.NET SDKs installed:
  3.0.100 [C:\Program Files\dotnet\sdk]
  5.0.101 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.23 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

IDE and version:

Microsoft Visual Studio Enterprise 2019
Version 16.8.3
VisualStudio.16.Release/16.8.3+30804.86
Microsoft .NET Framework
Version 4.8.04084

Thank you for your feedback, and for your contribution.  Tagging and routing to the team member best able to assist.

Thank you for your feedback, and for your contribution.  Tagging and routing to the team member best able to assist.

@abatishchev sorry for the trouble here. The failure is in a very unfortunate place since it's being invoked by the type initializer. I'm glad you were able to root cause the issue, and I agree with your proposed fix to treat empty string the same as the variable being unset.  As we discussed offline you should be able to work around this issue by reading the environment before referencing the DefaultAzureCredential and set the value to null if it's empty string by calling Environment.SetEnviromentVariable, but since it's in a type initializer you'll need do that before referencing any types which reference DefaultAzureCredential.

@abatishchev sorry for the trouble here. The failure is in a very unfortunate place since it's being invoked by the type initializer. I'm glad you were able to root cause the issue, and I agree with your proposed fix to treat empty string the same as the variable being unset.  As we discussed offline you should be able to work around this issue by reading the environment before referencing the DefaultAzureCredential and set the value to null if it's empty string by calling Environment.SetEnviromentVariable, but since it's in a type initializer you'll need do that before referencing any types which reference DefaultAzureCredential.


Styling and layout - infima defaults fail tests

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Description
Docs link: https://docusaurus.io/docs/styling-layout#styling-your-site-with-infima
Repo link: https://github.com/facebook/docusaurus/blob/main/website/docs/styling-layout.mdx
The defaults for the styling-your-site-with-infima tool to generate a colour palette are failing. As a reader It's difficult to know what the guide expects me to do as the tip suggests to avoid this.

Self-service

[ ] I'd be willing to address this documentation request myself.

Hi! The tip says "AA ratio for the primary color". It has no requirement for the other colors—indeed, those colors may not be used on a white/black background at all, so it's up to you to ensure contrast.


Generate sync webhook payloads in the dataloaders in order taxes.

I want to merge this change because generating sync webhook payloads in the dataloaders for order tax calculation.
⚠ ⚠ ⚠
This PR should be merged after PR#16086.
⚠ ⚠ ⚠

Impact

[ ] New migrations
[ ] New/Updated API fields or mutations
[ ] Deprecated API fields or mutations
[ ] Removed API types, fields, or mutations

Docs

[ ] Link to documentation:

Pull Request Checklist

[ ] Privileged queries and mutations are either absent or guarded by proper permission checks
[ ] Database queries are optimized and the number of queries is constant
[ ] Database migrations are either absent or optimized for zero downtime
[ ] The changes are covered by test cases
[ ] All new fields/inputs/mutations have proper labels added (ADDED_IN_X, PREVIEW_FEATURE, etc.)
[ ] All migrations have proper dependencies
[ ] All indexes are added concurrently in migrations
[ ] All RunSql and RunPython migrations have revert option defined

To pre-generate order payload we need to execute logic from fetch_order_prices_if_expired inside dataloader. We have decided to refactor 'fetch_order_prices_if_expired' before we start duplicating this logic.


Add Wikidata QID based on identifier value

Is your feature request related to a problem or area of OpenRefine? Please describe.
I have a list of identifier values for an external database. Some of them are already present in items in Wikidata. I'd like to reconcile them directly to Wikidata QIDs.
Describe the solution you'd like
A direct API call to retrieve QID or directly reconcile the column to the QIDs.
Describe alternatives you've considered
Currently, I have to regularly use cell.cross to join my project with an updated dump from Wikidata.

Have you tried using reconciliation for this? When reconciling a column, you can add any other column as additional property, and match it to the corresponding identifier property on Wikidata. Because this is an identifier property, it will be used in priority to retrieve reconciliation candidates (so the text in the column you are reconciling does not even need to match the labels on Wikidata).

Have you tried using reconciliation for this? When reconciling a column, you can add any other column as additional property, and match it to the corresponding identifier property on Wikidata. Because this is an identifier property, it will be used in priority to retrieve reconciliation candidates (so the text in the column you are reconciling does not even need to match the labels on Wikidata).

Hmm, is that so? I thought that reconciliation would prioritize labels.
Anyway, reconciliation over 250 000 lines takes ages and I think that an alternative, more straight-forward, pure fetching of QID could be much quicker. Am I mistaken?

Reconciliation should be quicker if used with a unique identifier since it does not need to do any text search (but the Wikidata reconciliation service could still be optimized for speed in many places).
You could also try using other web services to translate unique identifiers to Qids, or build your own.
As far as OpenRefine is concerned, I would say that this translates into the following requests:

better support for joins (without having to use cell.cross)
better UX for reconciliation without labels (which requires changes to the API: see https://github.com/reconciliation-api/specs/issues/27)


[1314] Provider lookup supporting name or code

Context
https://trello.com/c/8urvajGN/1314-m-provider-lookup
Changes proposed in this pull request

Add provider filter section allowing text search using name or code

Guidance to review

Visit /support/providers
Input a name or code
Results should match input

Example

Checklist

[x] Make sure all information from the Trello card is in here
[x] Attach to Trello card
[x] Rebased master
[x] Cleaned commit history
[x] Tested by running locally

 [M] Provider lookup


Register

@JuliaRegistrator register

Error while trying to register: Register Failed
@Jcharis, it looks like you are not a publicly listed member/owner in the parent organization (jcharistech).
If you are a member/owner, you will need to change your membership to public. See GitHub Help

@JuliaRegistrator register

Error while trying to register: Register Failed
@Jcharis, it looks like you are not a publicly listed member/owner in the parent organization (jcharistech).
If you are a member/owner, you will need to change your membership to public. See GitHub Help

@JuliaRegistrator register

Registration pull request created: JuliaRegistries/General/45351
After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.
This will be done automatically if the Julia TagBot GitHub Action is installed, or can be done manually through the github interface, or via:
git tag -a v0.0.1 -m "<description of version>" 932bec1d0d51f43810b924a0bb1b20d8172cab8e
git push origin v0.0.1


[ERROR:flutter/shell/platform/android/platform_view_android_jni_impl.cc(43)] java.lang.ArrayIndexOutOfBoundsException: length=3; index=3

My app developed a hard crash in an arean of the app that has been untouched for some time. The only package used is reorderables however that hasn't been updated since March - and this functionality was built and working since then. The only thing that I can think that has changed has been my flutter version. I've created a reproducable example of the crash.
Reproducable example
flutter_hard_crash.zip
Stack trace
Waiting for another flutter command to release the startup lock...
Launching lib/main.dart on Pixel 3 in debug mode...
✓ Built build/app/outputs/flutter-apk/app-debug.apk.
Installing build/app/outputs/flutter-apk/app.apk...
W/Gralloc3(12146): mapper 3.x is not supported
E/libc    (12146): Access denied finding property "vendor.gralloc.disable_ahardware_buffer"
W/RenderThread(12146): type=1400 audit(0.0:5791157): avc: denied { read } for name="u:object_r:vendor_default_prop:s0" dev="tmpfs" ino=25627 scontext=u:r:untrusted_app_27:s0:c80,c258,c512,c768 tcontext=u:object_r:vendor_default_prop:s0 tclass=file permissive=0
Connecting to VM Service at ws://<IP_ADDRESS>:64178/nBRCnQCMwVs=/ws
E/flutter (12146): [ERROR:flutter/shell/platform/android/platform_view_android_jni_impl.cc(43)] java.lang.ArrayIndexOutOfBoundsException: length=3; index=3
E/flutter (12146): 	at io.flutter.view.AccessibilityBridge.updateCustomAccessibilityActions(AccessibilityBridge.java:1262)
E/flutter (12146): 	at io.flutter.view.AccessibilityBridge$1.updateCustomAccessibilityActions(AccessibilityBridge.java:260)
E/flutter (12146): 	at io.flutter.embedding.engine.FlutterJNI.updateCustomAccessibilityActions(FlutterJNI.java:499)
E/flutter (12146): 	at android.os.MessageQueue.nativePollOnce(Native Method)
E/flutter (12146): 	at android.os.MessageQueue.next(MessageQueue.java:336)
E/flutter (12146): 	at android.os.Looper.loop(Looper.java:174)
E/flutter (12146): 	at android.app.ActivityThread.main(ActivityThread.java:7356)
E/flutter (12146): 	at java.lang.reflect.Method.invoke(Native Method)
E/flutter (12146): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
E/flutter (12146): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:930)
E/flutter (12146):
F/flutter (12146): [FATAL:flutter/shell/platform/android/platform_view_android_jni_impl.cc(977)] Check failed: CheckException(env).
F/libc    (12146): Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 12146 (tter_hard_crash), pid 12146 (tter_hard_crash)
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'google/blueline/blueline:10/QQ3A.200605.001/6392402:user/release-keys'
Revision: 'MP1.0'
ABI: 'arm64'
Timestamp: 2020-07-06 15:54:49+0100
pid: 12146, tid: 12146, name: tter_hard_crash  >>> com.example.flutter_hard_crash <<<
uid: 10592
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: '[FATAL:flutter/shell/platform/android/platform_view_android_jni_impl.cc(977)] Check failed: CheckException(env).
'
    x0<PHONE_NUMBER>000000  x1<PHONE_NUMBER>002f72  x2<PHONE_NUMBER>000006  x3  0000007fdeac4180
    x4<PHONE_NUMBER>000000  x5<PHONE_NUMBER>000000  x6<PHONE_NUMBER>000000  x7<PHONE_NUMBER>008000
    x8<PHONE_NUMBER>0000f0  x9  714a633fe3e836cf  x10<PHONE_NUMBER>000001  x11<PHONE_NUMBER>000000
    x12 fffffff0fffffbdf  x13<PHONE_NUMBER>339500  x14<PHONE_NUMBER>000001  x15 00128bf132d8eefa
    x16<PHONE_NUMBER>30e8c0  x17<PHONE_NUMBER>2ec0c0  x18<PHONE_NUMBER>1f6000  x19<PHONE_NUMBER>002f72
    x20<PHONE_NUMBER>002f72  x21 00000000ffffffff  x22<PHONE_NUMBER>2212d8  x23 00000000ffffffff
    x24<PHONE_NUMBER>000018  x25<PHONE_NUMBER>000018  x26<PHONE_NUMBER>00036e  x27<PHONE_NUMBER>000354
    x28<PHONE_NUMBER>00002e  x29 0000007fdeac4220
    sp  0000007fdeac4160  lr<PHONE_NUMBER>2a0f48  pc<PHONE_NUMBER>2a0f74
backtrace:
      #00 pc<PHONE_NUMBER>081f74  /apex/com.android.runtime/lib64/bionic/libc.so (abort+160) (BuildId: 8de865099c99977483c8947f9b7937e9)
      #01 pc<PHONE_NUMBER>257fcc  /data/app/com.example.flutter_hard_crash-8iCjMuf7d5BKgg_OCBaKog==/lib/arm64/libflutter.so (BuildId: df9d67f61ebd66906967bce5885cff2f6c89d58e)
      #02 pc<PHONE_NUMBER>0019dc  [anon:.bss]
Lost connection to device.
Exited (sigterm)

Flutter doctor output
[✓] Flutter (Channel master, 1.20.0-3.0.pre.130, on Mac OS X 10.14.6 18G1012, locale en-GB)
    • Flutter version 1.20.0-3.0.pre.130 at /Users/jimmyff/flutter-sdk
    • Framework revision a74310242e (20 hours ago), 2020-07-05 14:58:01 -0400
    • Engine revision f8bbcc396b
    • Dart version 2.9.0 (build 2.9.0-20.0.dev c190fc3a31)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
    • Android SDK at /Users/jimmyff/Library/Android/sdk
    • Platform android-30, build-tools 29.0.3
    • Java binary at: /Applications/Development/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 11.3.1, Build version 11C504
    • CocoaPods version 1.9.0

[!] Android Studio (version 3.6)
    • Android Studio at /Applications/Development/Android Studio.app/Contents
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • Java version OpenJDK Runtime Environment (build 1.8.0_212-release-1586-b4-5784211)

[!] IntelliJ IDEA Ultimate Edition (version 2017.2)
    • IntelliJ at /Applications/Development/IntelliJ IDEA.app
    ✗ Flutter plugin version 15.1 - the recommended minimum version is 16.0.0
    • Dart plugin version 172.3317.48
    • For information about installing plugins, see
      https://flutter.dev/intellij-setup/#installing-the-plugins

[✓] Connected device (1 available)
    • Pixel 3 (mobile) • 8B5X140C0 • android-arm64 • Android 10 (API 29)

! Doctor found issues in 2 categories.

I've just experienced this crash again in another part of my app without the reorderables widget, this time with a RangeSlider.
I've moved to the beta channel and the issue is resolved... Problem with flutter master?

Hi @jimmyff
does it reproduces only on real device? I tried to run your project on Pixel 3 Android 10 (API 29) (emulator) and I can't hit the crash. The same about running on my Redmi Note 7 Android 9 (API 28).
Looks like a duplicate of https://github.com/flutter/flutter/issues/60711  - could you check if code sample from there causes the same crash with your device?

Looks like I'm hitting the same hard crash only when I run on a real device (Pixel 2) using the RangeSlider.  Was not experiencing the hard crash with ios Simulator.

Without additional information we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.

I had the same issue while using ReorderableListView on my Xiaomi MI9, meanwhile it has been working correctly in emulator. And the problem was in android/app/build.gradle file. By default boilerplate set sdk versions it to version 28. Android docs recommends to set compileSdkVersion to the maximum available at the moment, for now it is version 30. Also I recommend to set targetSdkVersion to version 30.
This small fixes resolve this problem and everything works properly, I hope it will help you too :)


[Doc] C++ client's doc is poor

Is your enhancement request related to a problem? Please describe.
The C++ client's doc is poor, a lot of interfaces are not well documented. For example, a lot of setters/getters have no document, see http://pulsar.apache.org/api/cpp/2.7.0-SNAPSHOT/classpulsar_1_1_producer_configuration.html for example. Users cannot get some default config values and its definition.
Describe the solution you'd like
Improve the documents. At least we should add documents for those interfaces.
Additional context
I'm working on new API of Producer::send recently (see #9176), when the PR is pushed, it can be treated as an example.

As the generated docs in my local environment, here's an example from #9287

As we can see, we can link ProducerConfiguration's related methods from Producer#send. However it requires these external methods have JavaDoc style comments. If not, these methods cannot be linked, like the current C++ API website:

The methods with dark bold text are not documented and have no link.

As the generated docs in my local environment, here's an example from #9287

As we can see, we can link ProducerConfiguration's related methods from Producer#send. However it requires these external methods have JavaDoc style comments. If not, these methods cannot be linked, like the current C++ API website:

The methods with dark bold text are not documented and have no link.

Hi @BewareMyPower I've added descriptions for the methods in the ProducerConfiguration.h, could you please help review? Thanks
https://docs.google.com/document/d/1SjO8SCVcbCMuEVW-HI1kf_jJjy-Ej5yFZoY9g4jUAQQ/edit

Hi @BewareMyPower, I've added descriptions for the methods in the following files:

ClientConfiguration.h
ConsumerConfiguration.h
ReaderConfiguration.h

Could you please help review?
Feedback before 3/31 is highly appreciated, thanks!

@Anonymitaet OK

@BewareMyPower I've added more descriptions for various classes here.
Could you please review and give feedback before 4/13? Many thanks!

@Anonymitaet OK, I'm a little busy these days and may give some feedbacks on 4/11 or 4/12.

@BewareMyPower OK, take your time and thanks for your help.


Fix xbrl metadata renaming in the electric_plant_depreciation_functional_ferc1 table

PR Overview
mkay i did some more upstream cleaning of the xbrl_factoid names in this table. this was an error bc the columns actually coming from the xbrl data did not line up with the columns in the xbrl metadata so our standing mapping of xbrl name to pudl name was not working.
PR Checklist

[ ] Merge the most recent version of the branch you are merging into (probably dev).
[ ] All CI checks are passing. Run tests locally to debug failures
[ ] Make sure you've included good docstrings.
[ ] For major data coverage & analysis changes, run data validation tests
[ ] Include unit tests for new functions and classes.
[ ] Defensive data quality/sanity checks in analyses & data processing functions.
[ ] Update the release notes and reference reference the PR and related issues.
[ ] Do your own explanatory review of the PR to help the reviewer understand what's going on and identify issues preemptively.

So this is a naming inconsistency that we just hadn't previously identified, and you moved the special case out of process_xbrl_metadata() and into the actual renaming function and made it account for the newly identified inconsistency fully?

Yup! the only factoid w/ a calc was the total and basically everything else was not effectively mapped in our old version. applying the rename in this new way does it earlier in the process and enabled me to remove the bespoke one-factoid rename.


If value of score_saliency_map  is all nagative, how can i solve this?

Hi, I'm interested in Score-CAM and I'm using it.
I use Score-CAM in pytorch model vgg16_bn and ImageNet validation image. Most of the values on the activation map have negative values.
So score_saliency_map has all negative values and F.relu return all zeros.
My question is
(1) If score_sliency_map have all negative values, I cannot use Score-CAM?
(2) Is there any solution?

@DJ-Maeng Thanks for your interest.
I didn't meet this problem before where most values on activation maps are negative, but here are some suggestions. (1) ReLU operation is not necessary, we just follow the setting as previous works to make the saliency map better in visual. You can remove it in your case. (2) Instead of linearly combine the scores/weights with the original activation maps, you can use the normalized activations with values in [0,1].
Please let me know whether these help. If you have further questions, feel free to discuss here.

Thank you for your answer. Your answer was very helpful.
I will apply the answers to the question (2).
I was able to see that the activation maps were created properly in VGG16, but I could see that all values of the score_salency_map were negative values in VGG16_bn.
I use pytorch.models.vgg16_bn(batch normalization) and ILSVRC2012_img_val.
I did the same as possible as the environment I experimented with in your paper.
I may have made a mistake, but I'll show you the image number and the result of the ScoreCAM_map.
ILSVRC2012_val_00003014.JPEG

ILSVRC2012_val_00000293
ILSVRC2012_val_00002138
I haven't checked all the images, but the images I checked are the same as the three above.

I didn't test VGG_BN in the paper. To reproduce the result, try VGG16.
This may be caused by the BN operation after the last convolution layer. Thus, another suggestion is that you can hook the activation map before the BN layer.

Oh I`m really thank you for your answer. I apply your suggestion. I think Score-CAM is very good idea


Are there any limitations to database size? (not max key amount)

Basically I want to store ~10TB. The current DB size is 340GB and growing by 150-170GB per day. The thing is i don't want it to die halfway.
So is there any db size limitations or is storing data as 4GB chunks allows it to scale indefinitely?

hi! There is limit on the number of segments - 32767, with the default max segment size of 4GiB it limits the total database size to ~131TiB.
I'm curious, how many keys do you expect to store? Pogreb uses a 32 bit hash function, so as the number of keys gets close to the limit, the number of collisions increases and the performance degrades

Hi! Glad to hear Pogreb can store that much data, I expect to store ~150mil keys at max, And sorry for the late response


Continuous Integration for MacOS

The circleci continuous integration on MacOS isn't working currently because the build stalls out and doesn't finish within the 5-hour circleci limit.  I expect this is due to the resource limits of the MacOS "medium" machine which only has 4-cores and 8GB of RAM.
We have a request into circleci to get access to the MacOS "large" machine which has 8-cores and 16GB of RAM, hopefully this will take care of the issue.
In the mean time, I'll perform the MacOS builds on a local machine.

MacOS "large" is now available, and the MacOS build is succeeding.


minor bug in stack renderers

Description
Minor bug for stack render to load null image when findImage does not find a suitable match

https://github.com/cornerstonejs/cornerstoneTools/pull/1401


iterative machine in NewtonSVM

[WIP] as state is not updated properly yet so there is nothing meaningful available when we cancel.
we can first merge #4346 first and then rebase here as a test.

Could you create a pr where you don’t apply the autoformatter, otherwise it is very hard to see what you changed. No white spaces changes in the first instance please

@karlnapf this does not have the cleanups we discussed (like removing DEBUG flags) we can have it in another pr and keep this one related to IterativeMachine or ?

absolutely!

yep :)

windwos build is timeout, travis has a java problem


Emulating 32-bit overflow

It's known that the in-game registers during damage calculation can only hold 32-bit values and they make no attempt to account for anything beyond that. In highly contrived circumstances, by having a damage calculation run just beyond the limit of a 32-bit register, it can wrap around to low values and only get credit for a small number, allowing a Pokemon to survive a hit that should handily KO it by a margin of hundreds of thousands of damage.
In a competitive battle where both players aren't in cahoots to achieve one of these results, it will make no difference to any meaningful result whether Showdown adds support for this mechanic or not, with one exception: Custom Game. By faithfully respecting some of the game's limitations (such as the 10000-speed cap which is then truncated to 13 bits), but allowing level (and the corresponding stats it controls) to go far beyond the 1-byte cap that's always been in place for it in the game, the format is in this weird spot where if the overflow mechanic was implemented, Pokemon would constantly run into it without even trying: for example, a level 9999 neutral-natured Mew with stats of 29402 all around would hit a wraparound in damage calculation even if it used a move with just 37 base power, as 4001 * 29402 * 37 = 4,352,583,874, which wraps around to become 57,616,578 before you even take the opponent's defense stat into account. Still, Custom Game is decidedly not a competitive format by any means, so this can understandably be a low-severity issue.
On top of that, a lot of rollovers end up being invisible and irrelevant, such as the oft-cited example of Shuckle with Ice Ball. Even after that rolls over, there's still so much damage left behind, relative to an opponent's health bar, that it romps all over them for a KO anyway. The only cases where this is detectable, and therefore relevant, involve carefully picking numbers that multiply together to land just beyond the rollover point, so that the leftover damage fails to measure up to the opponent's HP. Since these registers hold 32 bits, but no Pokemon's HP bar ever takes more than 10 bits to express (often less, particularly with the kinds of low-level opponents that are even capable of dropping down to 1 defense), this requires precision finer than 1 part in 4 million.
Known Overflow Tests to Date

This battle is captured in the USUM battle video GBXG-WWWW-WWWL-YQNC. A Pikachu with a starting attack stat of 269 gets to +6 Attack with Huge Power, Light Ball, and a Flower Gift bonus in sun, along with four stat boosts elsewhere, and gets a lingering Rollout storage multiplier (see #4592) including the Defense Curl bonus, for 32x. Pikachu then uses Power Trip (normally 220 power with these boosts) after getting the benefit of two Helping Hands and that lingering multiplier, and the idea is that if everything calculates as I expected, then after the L *  P * A steps in the damage formula, the intermediate result there was too big to fit in a register, and would wrap around to just 80,384. The opponent was a Mew transformed into a level 1 Smeargle, so that it could have a decent supply of HP as a buffer zone, and the expectation was that after knocking down that 80384 with things like type resistance, the final damage would be in the range 170-201 so that Mew would survive it. Indeed it did survive, but the observed damage was much lower than expected, at just 74. I still have no explanation for why this case undershot the low end by so much. but it was the earliest known case that seemingly proved the general case mechanic existed. I suspect it's a misbehavior on the part of the move Power Trip in particular, but don't see any plausible mechanism for why it varied in this specific way.

This battle is captured in the USUM battle video AHMW-WWWW-WWWL-YQN7. Similar to the first case, but Pikachu is only level 95 and its attack stat starts at 229 instead of 269, and instead of using Power Trip, it attacks with the more well-behaved Bolt Strike (still with two Helping Hand boosts, but with Pikachu's type change to deprive it of STAB) the turn after using Charge. This case was supposed to roll over at the some point in the formula, but the overrun was pegged as 149,504 instead of 80,384. This was just enough to be tamable to fit within Mew's health bar, as the expected damage was 317-374, and this time it behaved perfectly in line with the projection, dealing 347 damage.

This battle is captured in the USUM battle video 55VG-WWWW-WWWL-UQGX, and explained in more detail at https://www.youtube.com/watch?v=BUthCamWwrc . Unlike the first two, this doesn't involve Rollout at all, and explores running over the limit at a different point. A level 74 Mewtwo-Mega-X (actually a Mew that transformed into L100 MMX to take advantage of the higher stat there) gets +6, Huge Power, and Flower Gift, then uses Me First to copy Flare Blitz from a -6 level 1 Growlithe (in sun, of course, otherwise Flower Gift would have been meaningless). This result is lower through the L * P * A section, and continues through the / D / 50 + 2 section as expected, but the idea here is that the next thing that gets applied is weather, which is a full-scale multiplication by 6144 and then, as a separate transaction, a division by 4096. Multiplying by 6144 in this case causes a wraparound to 81,920, which the rest of the formula acts on so the final damage is just 8-10 (observed: 9), and the level 1 Growlithe's natural health bar can take that without even needing a teammate to transform into it to provide a larger target. This proved the presence of at least two places in the formula that are susceptible to rollover.

This battle is captured in the USUM battle video UGFW-WWWW-WWWQ-NTWU, and was an early attempt to actualize something similar to test 3 but on the special side. A Dialga gets its special attack set to 485 ahead of the usual +6, and gets no item boost but is instead boosted by a teammate's Battery and its own Water Bubble. The addition of rain is designed to cause a rollover to 303,104 in the weather step as Dialga uses Hydro Pump,  and after all the dampening multipliers after that, the expected damage range was 9-11. But thanks to a transforming Mew again, the observed damage could be checked as 382, much larger than the expected range which would have allowed the original Steelix to take the hit. Just like the Power Trip case, this probably has to do with the most recent thorough audit of the damage formula coming at a time when Battery didn't exist as an ability, and thus some uncertainty in what exactly its boost is or where in the modifier order it lands. In cases like a +6 Palkia, whether Battery is treated as 5324 or 5325 would make a difference. This is another one that still needs to be deciphered somehow, before a faithful test of the mechanic can be done.

I'm not aware of any battle video code that's been assigned to this test, but it's captured on a screen recorder at https://www.youtube.com/watch?v=6xfyw67oISQ#t=246s . This time I had reconfigured the numbers and tested them out in single-player before sending out the plan; instead of a 485-stat Dialga using Hydro Pump, it's a 311-stat Kyogre using Water Spout at 149 power (so 223 after the Helping Hand); since it's holding Choice Specs, it has to get its SA boosts passed in by Baton Pass (and then its ability immediately overwritten with Water Bubble) so it can use the attacking move as its first turn. While Water Bubble, like Battery and Power Trip, didn't exist as of BW, these simple 2.0 (8192) modifiers seem to be well-behaved nonetheless, and this scenario was right in line with what the formula expected: after accounting for weather overrun, it was supposed to deal 8-10 damage, and dealt 10.

So far, this test has been conducted entirely in the single-player game, without having a battle video code or encoded video to show for it. The most simplistic form of the test thus far involves some blatant hackmons for ease of setup: a level 94 Haxorus with 383 attack, a Choice Band, the Hustle ability, and two illegal moves in Charge and Me First; and then a level 1 Rattata (species doesn't really matter here) whose moveset consists of four copies of Bolt Strike, ensuring that it will always pick that particular move but is incapable of ever running out of PP and using Struggle instead. I have to run around for wild encounters until I find a Ditto, switch to Rattata on turn 1 so it transforms into that, then I bring in a Sweet Scent user, spam that move, and switch to Smeargle. Smeargle uses Screech twice, Magic Room, a third Screech, Belly Drum, and Baton Pass to Haxorus. Haxorus's first turn out is the last turn of Magic Room, so it can use Charge without getting locked in, then Magic Room ends and the next turn will get the Choice Band boost as it uses a Me First, Charged Bolt Strike (which succeeds even if the opponent's version of the move is set to fail for lack of PP). The first time I tried this, Ditto never called for help and got KO'd by the Bolt Strike, while the second time it did make a call, and on the Bolt Strike turn, Ditto survived after taking just 3 damage, verified by catching it and examining its stats. Subsequent trials tested various combinations of not calling yet, calling with the teammate being KO'd, and calling with the teammate still there at the end, but that proved a red herring: regardless of situation, Ditto usually got KO'd but would occasionally survive, and when it did survive it would always show exactly 3 damage taken. The trick is that these numbers were chosen such that after the "+2" step, multiplying by 4096 would cause a rollover to exactly 3, so that if the formula were applying, say, a "no-op" modifier in the spread-move step for a single target move, I would expect the damage range to be 2-3. Based on these trials, though, it appears that what's actually happening is it's going ahead and applying a default x4096/4096 final modifier after everything else, including the random roll, and even if there are no triggers (such as Life Orb) speaking up in that section of the damage formula. So in those cases where a max damage roll showed up, the damage stayed intact at that exact figure that rolls over to 3, but with anything else, the lower number was able to be put through that final modifier without rolling over.

@Marty-D do you have opinions on this?

I think it's worth simulating the parts that have effect outside of Custom Game, at least.

I agree it's worth simulating, but I personally can't justify spending time on it when there are hundreds of other known bugs or unimplemented mechanics to work on that could actually matter in regular play.
If someone else is interested in doing this then I say have at it. There would still be plenty of research to do on it, especially in gens before 5.

I've since been able to crack test 1: what's happening there is that if a stored Rollout multiplier is in effect, but the next attack is something with variable power, the Rollout multiplier is the only thing that gets applied in the variable power step. (Other modifiers to the power, such as Helping Hand or a plate, still take effect just fine.) A lot of moves with variable power are coded to have an innate power of 1, such as Return, Gyro Ball, and even some moves for which 1 is never supposed to be in the gamut of possibilities at all, like Low Kick or Fling. In those cases, the Rollout multiplier will be applied to that bare 1, instead of running any kind of function to replace the 1 with a more sanitized value before doing so.
In the present case, what that means is that the Rollout-boosted Power Trip was actually being calculated as 20x32 (640), instead of the expected 220x32 (7040). That's enough of a reduction that it no longer overflows a 32-bit datapath at any step, and by all appearances the damage should then be in the 800K-900K range. But the fact remains that 74 was an actual, observed thing, and the explanation for that turns out to be far more sinister.
Test 7. This battle is captured in the USUM battle video UJ8W-WWWW-WWWS-9YPA. A level 94 Mega Lucario with 350 SpA sets up just a single Calm Mind, then starts spamming Aura Sphere against an Aron with no stat modifiers, and its original 4 SpD. It's doing this in search of the maximum random roll, as anything else just runs into Sturdy and gets healed off by Leech Seed, allowing another trial next turn. Basically, these numbers are calibrated so that a max roll will compute out to exactly 65536 damage, but when that max roll finally arrives, it deals 0 damage instead. What this tells me is that at the very end of the formula, the game makes a further truncation, not just to 32 bits but to 16, and that step happens after the minimum damage has been forced up to 1. This probably has something to do with the fact that HP (and other stats) never exceed 2 bytes, so only 2 bytes are allocated to them in the first place, leading them to truncate the number when it comes time to deal damage and avoiding the subtraction of a 3- or 4-byte number which would spill over into other bytes with other unintended effects.
65536 is of course a far easier target to hit, even accidentally, as seen with this case of very light setup vs. unaltered FEARmon. Implementing this cap now that we know how low it really extends could have some meaningful effect on battles that aren't merely cooperative exhibitions, and especially so in Custom Game. This explanation, combined with the knowledge that Power Trip is being taken as 640 power instead of 7040, also sheds the last necessary piece of light on battle 1, revealing it to be the same kind of happy accident as the eventual max roll in battle 7:
Damage rolls prior to the final truncation - (829725, 839486, 849248, 859009, 868771, 878532, 888294, 898055, 907817, 917578, 927340, 937101, 946862, 856624, 866385, 976147)
Those same damage rolls after truncation - (43293, 53054, 62816, 7041, 16803, 26564, 36326, 46087, 55849, 74, 9836, 19597, 29358, 39120, 48881, 58643)


Use same num_proc for dataset download and generation

Use the same num_proc value for data download and generation. Additionally, do not set num_proc to 16 in DownloadManager by default (num_proc now has to be specified explicitly).

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

I noticed this bug the other day and was going to look into it! "Where are these processes coming from?" ;-)


Remove incorrect or useless dependencies

github.com/docker/docker from v1.6.1 to v1.4.2-0.20190924003213-a8608b5b67c7 ( to be consistent with nerdctl)
remove github.com/docker/libcontainer

Signed-off-by: Nan Li<EMAIL_ADDRESS>

Codecov Report

Merging #85 (c6df223) into main (044d191) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #85   +/-   ##
=======================================
  Coverage   28.49%   28.49%
=======================================
  Files          17       17
  Lines        1516     1516
=======================================
  Hits          432      432
  Misses       1020     1020
  Partials       64       64

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 044d191...c6df223. Read the comment docs.

may I ask what we can do for this pr? it'd be much better to have nerdctl support ;)


updateBasicGroupFullInfo object has non-negative basic_group_id_ field

I've been debugging an issue with a client I'm writing: sometimes I get an error when calling getChat due to the chat not existing.
I found out that whenever I get an updateBasicGroupFullInfo Telegram Update, the basic_group_id_ field is positive, and to my understanding, the basic_group_id, if we understand it as an equivalent of a chat ID, should be negative.
Is this by design? should I be multiplying all basic group IDs with -1 to call getChat with the basic group ID? I am doing that as a workaround for the moment, but it feels like a possible mistake in the API.
Thanks

I just had the same issue with getBasicGroupFullInfo, looks like I have to provide the positive chat ID, which reinforces the point that this is by design.
Can the docs reflect this bit? I think many devs would expect the chat ID to be equivalent to the basic group/supergroup ID since those are subsets of Chat.
Thanks

the basic_group_id, if we understand it as an equivalent of a chat ID

This is wrong. You can't manually convert between chat_id/basic_group_id/supergroup_id/secret_chat_id/user_id. They are identifiers of unrelated entities and belong to different sopes.
You need to inspect Chat.type field in order to find type-specific information about the chat.
createPrivateChat/createBasicGroupChat/createSupergroupChat/createSecretChat can be used for backward conversion.

So let's say for instance I get a notification that a new message was sent.
The updateNewMessage contains the chat id
Then lets say I want to get the details from this chat ID
I am already inspecting the chat type, but if I wanted to call say, getBasicGroupFullInfo, or anything similar, I have to provide the basic_group_id, but I only have the chat_id!
So if only have the new message update, how do I convert that to a basic group id, or a supergroup id?

I have the same problem!

Ooooh thank you I didn't know that!
I think the issue has been pretty much addressed so I'll close it.
Thank you!

chat.type.basic_group_id

Maybe i also need this information


Question about usage of _MACHEPS

Hello,
i have trouble understanding this line https://github.com/scipy/scipy/blob/c1ed5ece8ffbf05356a22a8106affcd11bd3aee0/scipy/optimize/_differentialevolution.py#L947
From my understanding, it is used to avoid division by zero. Am I correct about this?
If so, then then shouldn't the machine epsilon be added outside after taking the absolute? So
np.abs(np.mean(self.population_energies)) + _MACHEPS)

instead?
I was just trying to understand the convergence mechanism and thought I'd mention.
Thanks in advance! :)

Hi @stepeos,
yes, you got it right, this + _MACHEPS mechanism prevents a division by zero error.
In my humble opinion you even found a bug:
np.abs(np.mean(self.population_energies) + _MACHEPS)

Let's assume that the mean of the energies is -_MACHEPS since our function to be optimized has a negative optimum so the energies are negative too.
Then we have
  np.abs(np.mean(self.population_energies) + _MACHEPS)
= np.abs(-_MACHEPS + _MACHEPS)
= 0

This leads to a division by zero error.
Thus, we must place the _MACHEPS outside:
  np.abs(np.mean(self.population_energies))  + _MACHEPS
= np.abs(-_MACHEPS) + _MACHEPS
= +_MACHEPS + _MACHEPS
= 2 * _MACHEPS

This prevents the division by zero error since _MACHEPS > 0.
Since you found this bug, do you want to submit a PR with this little fix?
If not, then I can submit one, too.

Awesome, thanks for the elaborate answer! I'll shoot a PR in a minute :)

completed in 18392


qemu: build qemu-5.2.0 from source

QEMU 5.2.0 is much faster than v4 and is being rolled out across
other balena projects. In this case we are still building QEMU
from source as we need the system utilities for x86-64 and not
for arm-static emulation.
Change-type: patch
Signed-off-by: Kyle Harding<EMAIL_ADDRESS>Depends-on: https://github.com/balena-io/autohat/pull/126

@klutchell once you get a new version, please update the e2e-ephemeral job in Jenkins to default to it (parameter).

@ab77 done, I updated this job to use autohat v0.1.2: https://jenkins.dev.resin.io/job/balena-e2e-ephemeral/


Support padding in Display.

fn main() {
    let v = semver::Version::from((0, 0, 0));
    println!("{:<15} {:<15}", v, v);
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=e68ff1fc17bf96e41963222800080d09
Results in:
0.0.0 0.0.0

I'm printing Version in a tabular fashion, and it would help a lot if this was supported.

Looks like this is supported now.

Have not verified, but I'm fine with closing the issue. Thanks!