Plugin crash with Nextcloud

Hi,
Successfully got the plugin working, but as soon as I try to login within nextcloud it crashes.
It doesn't forward the query to modsecurity-crs container.
I've tried with cache disable but got the same error.

traefik config:
commands:
      # Enabling WAF module
      - '--experimental.plugins.traefik-modsecurity-plugin.modulename=github.com/madebymode/traefik-modsecurity-plugin'
      - '--experimental.plugins.traefik-modsecurity-plugin.version=v1.4.5'

labels:
      # WAF enable (This is a very simple plugin that proxies the query to the owasp/modsecurity apache container, and checks in its answer if it has detected something in the query (>400 response) or not). Set max body size to inspect to 1GB.
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.modSecurityUrl=http://waf:80
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.maxBodySize=1073741824
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.cacheEnabled=true

nextcloud labels (waf middleware here):
      - "traefik.http.routers.nextcloud.middlewares=other,nextcloud-redirectregex,security-and-proxy-headers,waf"

modescurity config (added allowed methods especially for Nextcloud):
  # WAF ModSecurity https://plugins.traefik.io/plugins/644d9a72ebafd55c9c740848/mx-m-owasp-crs-modsecurity-plugin / https://github.com/madebymode/traefik-modsecurity-plugin
  # You need to have an always on web service as backend, like a dummy whoami one
  waf:
    image: owasp/modsecurity-crs:apache-alpine
    restart: always
    depends_on:
      - traefik
      - waf_dummy_whoami_for_modsecurity
    # 2 replicas as critical service (needs to be in swarm mode)
    deploy:
      mode: replicated
      replicas: 2
      endpoint_mode: vip
    networks:
      - security_services
    environment:
      - PARANOIA=1
      - ANOMALY_INBOUND=10
      - ANOMALY_OUTBOUND=5
      - BACKEND=http://waf_dummy_whoami_for_modsecurity
      - SERVER_NAME=waf
      - REMOTEIP_INT_PROXY=traefik
      #Native/JSON : Native better parsed from CrowdSec modsecurity parser
      - MODSEC_AUDIT_LOG_FORMAT=Native
      # Nextcloud uses a lot of methods
      - ALLOWED_METHODS=MKCOL GET HEAD POST PUT OPTIONS PROPFIND
  # The dummy whoami for modsecurity
  waf_dummy_whoami_for_modsecurity:
    image: containous/whoami
    security_opt:
      - no-new-privileges:true
    restart: always
    deploy:
      mode: replicated
      replicas: 2
      endpoint_mode: vip
    networks:
      - security_services

Ask me anything you need for further debug if you wish.

hi @f-vt,
hrm, this is an interesting panic - i'd only see this happening if the request passed to the plugin was nil, which means the middleware chain is acting weird
would you mind cloning this tree locally: https://github.com/madebymode/traefik-modsecurity-plugin/tree/handle-nil-requests
and using https://github.com/madebymode/traefik-modsecurity-plugin/blob/handle-nil-requests/docker-compose.local.yml with your next cloud setup to see if traefik logs "Received a nil request" ?

Hi,
Tested just now, and it seems that I'm still getting an error :

happy to help!
hrm, this still looks like it still has something to do with a bad http.Request or http.ResponseWriter  getting passed to the middleware
per your screenshot

it's crashing here: https://github.com/madebymode/traefik-modsecurity-plugin/blob/9b4f1e9adb69355b0defe2f772de4b37d80af497/modsecurity.go#L168
can you paste your nextcloud config (minus the keys) for me so I can take a look

Ha yes I see now !
Here it is:

Compose (I use swarm):

version: '3.6'

############################################
############################################
###   SERVICES
############################################
############################################
services:
  traefik:
    # The official v2.0 Traefik docker image
    image: traefik:latest
    # 2 replicas as critical service (needs to be in swarm mode)
    deploy:
      # to see real IP in logs: Mode global: one (and only one) service per node); to use with HOST networking (so that ports exposed on host through bridge network directly, bypassing mesh overlay. Only port 80 as 443 exposed through SSLH via docker_gwbridge
      mode: global
      # for not putting it behind VIP and then seeing real client IPs
      endpoint_mode: dnsrr
      placement:
        constraints:
          # Traefik instances needs to be scheduled on a manager node to get labels (this reduces availability for example in case the node fails (we'll need to wait for a redeploy on another node) but no other option yet)
          - "node.role==manager"
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: rollback
    # Configure networks for traffic to endpoint to RP
    networks:
      - proxy
      - web_priv_services
      - web_devops_tools
      - security_services
    # Expose following ports to HOST IP (clients to RP)
    ports:
      # HTTPS on port 443 HOST exposure handled by SSLH, and forwarded to port 443 of container private ip in the "proxy" vxlan (added route on host using docker_gwbridge IP)
      # PING SERVICE HTTP on port 443 exposure handled by SSLH also,and forwarded to port 8082 container private ip in the "proxy" vxlan (added route on host using docker_gwbridge IP)
      ## The DNS port / HOST mode. Will listen on HOST IP address. And will redirect to docker internal DNS IP (<IP_ADDRESS>) of this traefik container. So we can resolve Docker Services DNS Names from host. Would like to be Listening on localhost only as we only want to use this from the host itself, but this is not possible to specify this at the moment, so we just block port 53 with firewall...
      - target: 53
        published: 53
        protocol: udp
        mode: host
      ## The HTTP port / HOST mode to see real IP in logs (to bypass overlay/mesh; combined with global deployment instruction)
      - target: 80
        published: 80
        protocol: tcp
        mode: host
    environment:
      - "GANDIV5_API_KEY_FILE=/run/secrets/gandiv5_api_key"
      - LEGO_DISABLE_CNAME_SUPPORT=true # https://community.traefik.io/t/suddenly-experiencing-error-400-invalid-domain-errors-with-le-certs/16366/2 - IF YOU HAVE A WILDCARD CNAME for your DNS Zone - LEGO (backend program used by traefik for adding TXT entries through DNS provider API for automating letsencrypt DNS Challenge) can't check cname if a wildcard is present... Need to disable this
      - TZ=Europe/Paris
      #- PUID=33 # Not handled by traefik container
      #- PGID=33 # Not handled by traefik container
    command:
      # enable traefik dashboard an configuration expose to api@internal
      - '--api.dashboard=true'
      # enable ping feature
      - '--ping=true'
      - '--ping.entryPoint=ping'
      #- '--api.insecure=true' don't needed, it exposes api on 8080, but we are using RP to api@internal not 8080.
      # Logs for Crowdsec
      - '--accesslog=true'
      # Docker provider
      - '--providers.docker'
      - '--providers.docker.endpoint=unix:///var/run/docker.sock'
      - '--providers.docker.watch=true'
      - '--providers.docker.exposedByDefault=false' # Warning: you needs to explicitely set "traefik.enable" flag label to true
      - '--providers.docker.swarmmode=false' # at the moment we use the docker socket, no need to use swarm mode
      # File provider
      - '--providers.file.directory=/traefik-dynamic-configs/'
      - '--providers.file.watch=true'
      # Entrypoints config
      - '--entryPoints.ping.address=:8082'
      - '--entrypoints.http=true'
      - '--entryPoints.http.address=:80'
      - '--entrypoints.https=true'
      - '--entryPoints.https.address=:443'
      - '--entrypoints.udpdns=true'
      - '--entryPoints.udpdns.address=:53/udp'
      # Entrypoints - http to https
      - '--entrypoints.http.http.redirections.entrypoint.to=https'
      - '--entrypoints.http.http.redirections.entrypoint.scheme=https'
      # CERT Resolver
      -<EMAIL_ADDRESS>      #- '--certificatesresolvers.dnschallenge.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory' # letsencrypt staging/testing when needs debugging to not reach letsencrypt rate limit
      - '--certificatesresolvers.dnschallenge.acme.storage=/letsencrypt/acme.json'
      - '--certificatesresolvers.dnschallenge.acme.dnschallenge=true'
      - '--certificatesresolvers.dnschallenge.acme.dnschallenge.provider=gandiv5'
      - '--certificatesresolvers.dnschallenge.acme.dnschallenge.resolvers=<IP_ADDRESS>:53,<IP_ADDRESS>:53' # IMPORTANT: Put some Gandi's NS IP there. If not, letsencypt will have to wait A LOT OF TIME the DNS propagation (DNS challenge use in our case gandi API to create an entry in our DNS zone at our provider to validate ownership)
      # Enabling WAF module
      ## debug load local module
      - '--experimental.localPlugins.traefik-modsecurity-plugin.moduleName=github.com/madebymode/traefik-modsecurity-plugin'
      ## from online repo, prod module
      ###- '--experimental.plugins.traefik-modsecurity-plugin.modulename=github.com/madebymode/traefik-modsecurity-plugin'
      ###- '--experimental.plugins.traefik-modsecurity-plugin.version=v1.4.5'
    volumes:
      # Traefik certificate storage
      - traefik_letsencrypt:/letsencrypt/
      # Traefik dynamic config files (profider.file.directory)
      - traefik_dynconfig:/traefik-dynamic-configs/:ro
      # So that Traefik can listen to the Docker events
      - /var/run/docker.sock:/var/run/docker.sock:ro
      # plugin debugging
      - /home/docker/containers-data/traefik/plugins-local/traefik-modsecurity-plugin-handle-nil-requests:/plugins-local/src/github.com/madebymode/traefik-modsecurity-plugin
    secrets:
      - gandiv5_api_key
    labels:
      # GLOBAL - Auth
      - "traefik.http.middlewares.auth.basicauth.users=Admin:$$apr1$$PI4r4gDw$$AcyfFA4cd/XzqkekNqIl80" # use htpasswd to generate, double the $ signs
      # GLOBAL - Headers
      # Commented cause better conf below - 'traefik.http.middlewares.HSTS.headers.customresponseheaders.Strict-Transport-Security="max-age=6307200; includeSubDomains; preload"'
      - "traefik.http.middlewares.security-and-proxy-headers.headers.accessControlAllowOriginList=*"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.AccessControlMaxAge=100"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.BrowserXssFilter=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.ContentTypeNosniff=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.ForceSTSHeader=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.FrameDeny=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.SSLRedirect=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.STSIncludeSubdomains=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.STSPreload=true"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.STSSeconds=63072000"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.CustomFrameOptionsValue=SAMEORIGIN"
      - "traefik.http.middlewares.security-and-proxy-headers.headers.referrerPolicy=same-origin"
      #- "traefik.http.middlewares.security-and-proxy-headers.headers.hostsProxyHeaders=X-Forwarded-Host"
      #- "traefik.http.middlewares.security-and-proxy-headers.headers.sslProxyHeaders.X-Forwarded-Proto=https"
      # GLOBAL - Other
      - "traefik.http.middlewares.other.buffering.maxRequestBodyBytes=0"
      - "traefik.http.middlewares.other.buffering.memRequestBodyBytes=1000000000"
      - "traefik.http.middlewares.other.buffering.maxResponseBodyBytes=0"
      - "traefik.http.middlewares.other.buffering.memResponseBodyBytes=1000000000"
      - "traefik.http.middlewares.other.compress=false"
      # Endpoint Config for reverse proxying UDP DNS
      - "traefik.udp.routers.udpdnsrt.entrypoints=udpdns"
      - "traefik.udp.routers.udpdnsrt.service=udpdnssvcdocker@file"
      # Endpoint Config for reverse proxying traefik API
      - "traefik.enable=true"
      - "traefik.http.routers.api.entrypoints=https"
      - "traefik.http.routers.api.rule=Host(`traefik.XXXX.XXX)"
      - "traefik.http.routers.api.tls=true"
      - 'traefik.http.routers.api.tls.options=modern@file'
      - "traefik.http.routers.api.middlewares=auth,security-and-proxy-headers"
      - "traefik.http.routers.api.service=api@internal" # Assign api@internal who is pregenerated by traefik for itself to the api router, so nothing to configure for this, rewrites for api, etc. already defined.
      - "traefik.http.routers.api.tls.certresolver=dnschallenge"
      - "traefik.http.routers.api.tls.domains[0].main=XXXX.XXX"
      - "traefik.http.routers.api.tls.domains[0].sans=*.XXXX.XXX"  # this wildcard cert will then be applied to all sites matching it
      # WAF enable (This is a very simple plugin that proxies the query to the owasp/modsecurity apache container, and checks in its answer if it has detected something in the query (>400 response) or not). Set max body size to inspect to 1GB.
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.modSecurityUrl=http://waf:80
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.maxBodySize=1073741824
      - traefik.http.middlewares.waf.plugin.traefik-modsecurity-plugin.cacheEnabled=true

  ############################################
  #   SOME DEVOPS TOOLS
  ############################################
  portainer:
    image: portainer/portainer-ce:latest
    depends_on:
      - traefik
    deploy:
      mode: replicated
      replicas: 1
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: continue
    networks:
      - web_devops_tools
    environment:
      - TZ=Europe/Paris
      - DISABLE_AUTH="true"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:z
      - portainer_data:/data:z
    command:
      -H unix:///var/run/docker.sock
    labels:
      - "traefik.enable=true" # enablig traefik RP for this app
      - "traefik.http.routers.portainer.entrypoints=https" # https listener (config in line below)
      - "traefik.http.routers.portainer.rule=Host(`portainer.XXXX.XXX`)"
      - "traefik.http.routers.portainer.tls=true"
      - 'traefik.http.routers.portainer.tls.options=modern@file'
      - "traefik.http.routers.portainer.middlewares=auth,security-and-proxy-headers,waf"
      - "traefik.http.routers.portainer.service=portainer" # Useless as the command below dynamically creates it, but kept there as an example
      - "traefik.http.services.portainer.loadbalancer.server.port=9000" # RP with LB to ports 9000 of all containers
  whoami:
    image: containous/whoami
    depends_on:
      - traefik
    deploy:
      mode: replicated
      replicas: 1
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: continue
    networks:
      - web_devops_tools
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami.entrypoints=https"
      - "traefik.http.routers.whoami.rule=Host(`whoami.XXXX.XXX`)"
      - "traefik.http.routers.whoami.tls=true"
      - 'traefik.http.routers.whoami.tls.options=modern@file'
      - "traefik.http.routers.whoami.middlewares=auth,security-and-proxy-headers,waf"
      - "traefik.http.services.whoami.loadbalancer.server.port=80"
  # Rolling updates every 10 hours, rollback on failure
  shepherd:
    image: containrrr/shepherd
    deploy:
      # One service only, on manager node
      mode: global
      placement:
        constraints:
          - "node.role==manager"
      restart_policy:
        condition: on-failure
      update_config:
        parallelism: 1
        delay: 5s
        order: stop-first
        monitor: 10s
        failure_action: rollback
    environment:
      - TZ=Europe/Paris
      # Update all (test it with: docker service ls -f name=)
      - FILTER_SERVICES=name=
      # Keeping last 2 images
      - IMAGE_AUTOCLEAN_LIMIT=2
      # every 10 hours
      - SLEEP_TIME=600m
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:z
      - /etc/localtime:/etc/localtime:ro

  ############################################
  #   HOSTING - BACKEND
  ############################################
  mariadb_nextcloud:
    image: mariadb:latest
    deploy:
      mode: replicated
      replicas: 1
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 10s
        failure_action: rollback
    networks:
      - backend_services
    environment:
      - PUID=999
      - PGID=999
      - MARIADB_RANDOM_ROOT_PASSWORD=yes
      - MARIADB_DATABASE=nextcloud
      - MARIADB_USER=nextcloud
      - MARIADB_PASSWORD_FILE=/run/secrets/mariadb_nextcloud_password
      - MARIADB_AUTO_UPGRADE=yes
      - TZ=Europe/Paris
    command:
      - '--transaction-isolation=READ-COMMITTED'
      - '--log-bin=mysqld-bin'
      - '--binlog-format=ROW'
    volumes:
      - mariadb_nextcloud_db:/var/lib/mysql
    secrets:
      - mariadb_nextcloud_password

  redis_nextcloud:
    image: redis:alpine
    deploy:
      mode: replicated
      replicas: 1
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: rollback
    networks:
      - backend_services

  ############################################
  #   HOSTING - WEB
  ############################################
  nextcloud:
    image: nextcloud:latest
    depends_on:
      - mariadb_nextcloud
      - redis_nextcloud
    deploy:
      mode: replicated
      replicas: 1
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: rollback
    networks:
      - web_priv_services
      - backend_services
    volumes:
      - nextcloud_www_html:/var/www/html
      - nextcloud_data:/nextcloud
    environment:
      - TZ=Europe/Paris
      - PUID=33
      - PGID=33
      - NEXTCLOUD_DATA_DIR=/nextcloud/data
      - REDIS_HOST=redis_nextcloud
      - MYSQL_HOST=mariadb_nextcloud
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - MYSQL_PASSWORD_FILE=/run/secrets/mariadb_nextcloud_password
      - NEXTCLOUD_ADMIN_USER=admin
      - NEXTCLOUD_ADMIN_PASSWORD_FILE=/run/secrets/nextcloud_admin_password
      - PHP_MEMORY_LIMIT=8192M
      - PHP_UPLOAD_LIMIT=100000000M
      - NEXTCLOUD_TRUSTED_DOMAINS="localhost cloud.XXXX.XXX"
      #- NEXTCLOUD_INIT_HTACCESS=true
      #- APACHE_DISABLE_REWRITE_IP=true # we want apache to rewrite IP from our forwarded for headers
      - TRUSTED_PROXIES=traefik <IP_ADDRESS>/24 # Add potential traefik proxy address
      - OVERWRITEHOST=cloud.XXXX.XXX
      - OVERWRITEPROTOCOL=https
      - OVERWRITECLIURL="https://localhost/"
    secrets:
      - nextcloud_admin_password
      - mariadb_nextcloud_password
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=web_priv_1"
      - "traefik.http.routers.nextcloud.entrypoints=https"
      - "traefik.http.routers.nextcloud.rule=Host(`cloud.XXXX.XXX`)"
      - "traefik.http.routers.nextcloud.tls=true"
      - 'traefik.http.routers.nextcloud.tls.options=intermediate@file'
      - "traefik.http.routers.nextcloud.middlewares=other,nextcloud-redirectregex,security-and-proxy-headers"
      - "traefik.http.routers.nextcloud.service=nextcloud"
      - 'traefik.http.middlewares.nextcloud-redirectregex.redirectregex.permanent=true'
      - "traefik.http.middlewares.nextcloud-redirectregex.redirectregex.regex=https://(.*)/.well-known/(?:card|cal)dav"
      - "traefik.http.middlewares.nextcloud-redirectregex.redirectregex.replacement=https://$${1}/remote.php/dav"
      - "traefik.http.middlewares.nextcloud-headers.redirectregex.replacement=https://$${1}/remote.php/dav"
      - "traefik.http.services.nextcloud.loadbalancer.server.port=80"

  ############################################
  #   SECURITY
  ############################################

  # WAF ModSecurity https://plugins.traefik.io/plugins/644d9a72ebafd55c9c740848/mx-m-owasp-crs-modsecurity-plugin / https://github.com/madebymode/traefik-modsecurity-plugin
  # You need to have an always on web service as backend, like a dummy whoami one
  waf:
    image: owasp/modsecurity-crs:apache-alpine
    depends_on:
      - traefik
      - waf_dummy_whoami_for_modsecurity
    # 2 replicas as critical service (needs to be in swarm mode)
    deploy:
      mode: replicated
      replicas: 2
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 15s
        order: start-first
        monitor: 30s
        failure_action: rollback
    networks:
      - security_services
    environment:
      - TZ=Europe/Paris
      - PARANOIA=1
      - ANOMALY_INBOUND=10
      - ANOMALY_OUTBOUND=5
      - BACKEND=http://waf_dummy_whoami_for_modsecurity
      - SERVER_NAME=waf
      - REMOTEIP_INT_PROXY=traefik
      # Nextcloud uses a lot of methods
      - ALLOWED_METHODS=MKCOL GET HEAD POST PUT OPTIONS PROPFIND
  # The dummy whoami for modsecurity
  waf_dummy_whoami_for_modsecurity:
    image: containous/whoami
    deploy:
      mode: replicated
      replicas: 2
      endpoint_mode: vip
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 15s
        order: start-first
        monitor: 60s
        failure_action: rollback
    networks:
      - security_services

  #crowdsec - debian one for systemd and journalctl support
  crowdsec:
    image: crowdsecurity/crowdsec:latest-debian
    deploy:
      # One service only, on manager node
      mode: global
      placement:
        constraints:
          - "node.role==manager"
      restart_policy:
        condition: any
      update_config:
        parallelism: 1
        delay: 5s
        order: start-first
        monitor: 30s
        failure_action: rollback
    networks:
      - security_backend_services
    ports:
      - 8080:8080
    environment:
      - TZ=Europe/Paris
      #this is the list of collections we want to install
      - COLLECTIONS=crowdsecurity/whitelist-good-actors crowdsecurity/nginx crowdsecurity/traefik crowdsecurity/modsecurity crowdsecurity/sshd crowdsecurity/linux crowdsecurity/linux-lpe crowdsecurity/base-http-scenarios crowdsecurity/http-cve crowdsecurity/http-dos crowdsecurity/nextcloud crowdsecurity/auditd crowdsecurity/mysql crowdsecurity/mssql crowdsecurity/pgsql crowdsecurity/postfix crowdsecurity/smb crowdsecurity/iptables
      #this is the list of parsers we want to install
      - PARSERS=crowdsecurity/docker-logs
      - GID=1000
      - CUSTOM_HOSTNAME=crowdsec_XXXX
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:z
      - crowdsec_db:/var/lib/crowdsec/data/
      - crowdsec_config:/etc/crowdsec/
      - /var/log/journal:/run/log/journal:ro
      - /var/log:/var/log/from_host:ro

  ############################################
  #   CONTAINER WITH DOCKER TO RUN COMMANDS ON STARTED CONTAINERS
  ############################################
  # only try 3 restarts if fails
  afterstartcommands_netxcloud:
    image: docker:latest
    depends_on:
      - nextcloud
    deploy:
      # One service only, on manager node
      mode: global
      placement:
        constraints:
          - "node.role==manager"
      restart_policy:
        condition: on-failure
        delay: 10s
        max_attempts: 3
      update_config:
        parallelism: 1
        delay: 5s
        order: stop-first
        failure_action: continue
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:z
    command: >
              sh -c "sleep 10 &&
              echo "" | docker exec -i $$(docker ps -q -f name=stack-server_nextcloud) apt-get update &&
              docker exec -i $$(docker ps -q -f name=stack-server_nextcloud) apt-get -y install ffmpeg &&
              exit 0"

############################################
############################################
###   VOLUMES & SECRETS
############################################
############################################
volumes:
  traefik_letsencrypt:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-data/traefik/letsencrypt
  traefik_dynconfig:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-config/traefik/dynamic-configs
  mariadb_nextcloud_db:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-data/nextcloud/db
  nextcloud_data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /mnt/data/nextcloud
  nextcloud_www_html:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-data/nextcloud/www/html
  portainer_data:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-data/portainer
  crowdsec_db:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-data/crowdsec/db
  crowdsec_config:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /home/docker/containers-config/crowdsec

secrets:
  nextcloud_admin_password:
    file: /home/docker/containers-secrets/nextcloud_admin_password.txt
  mariadb_nextcloud_password:
    file: /home/docker/containers-secrets/mariadb_nextcloud_password.txt
  gandiv5_api_key:
    file: /home/docker/containers-secrets/gandiv5_api_key.txt

############################################
############################################
###   NETWORKS
############################################
############################################
# docker network prune" command is useful to cleanup old stuff
networks:
  proxy:
    name: proxy
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  web_devops_tools:
    name: web_1
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  web_priv_services:
    name: web_priv_1
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  backend_services:
    name: back_1
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  security_services:
    name: itsec_1
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  security_backend_services:
    name: itsec_back_1
    driver: overlay
    ipam:
      driver: default
      config:
        - subnet:  "<IP_ADDRESS>/24"
  macvlan:
    name: macvlan
    driver: macvlan
    driver_opts:
      parent: eth0
    ipam:
      config:
        - subnet: "<IP_ADDRESS>/24"

Hey @f-vt this is very helpful.
I've confirmed that Nexcloud works with our plugin:
https://github.com/madebymode/traefik-modsecurity-plugin/assets/129784/15b4608a-2918-4549-bb4f-b1551d2dc89d
so the issue is either your swam config or your additional traefik.http.middlewares.security-and-proxy-headers middleware, I would try removing  your security and proxy headers first, then if that doesn't work, it's your swam config
Unfortunately, we don't support chained middleware or swarm, but i've created a branch for you here that should help you debug your setup further: https://github.com/madebymode/traefik-modsecurity-plugin/tree/debug-nextcloud
docker-compose -f docker-compose.nextcloud.local.yml up

that branch will remain on this upstream for the next 14 days and then I will delete, hopefully that will assist you with your setup
good luck!

Hi,
Thanks a lot for your time,
You just gave me the final hint !
It seems that the middleware which cause issues it the "other" one (i tested them one by one):
      - "traefik.http.middlewares.other.buffering.maxRequestBodyBytes=0"
      - "traefik.http.middlewares.other.buffering.memRequestBodyBytes=1000000000"
      - "traefik.http.middlewares.other.buffering.maxResponseBodyBytes=0"
      - "traefik.http.middlewares.other.buffering.memResponseBodyBytes=1000000000"
      - "traefik.http.middlewares.other.compress=false"

However reading again the traefik's middlewares documentation: https://doc.traefik.io/traefik/middlewares/overview/
The middlewares are evaluated in the order you specify them, so I just moved the "waf" one at the beggining of the list, which is now:       - traefik.http.routers.nextcloud.middlewares=waf,other,nextcloud-redirectregex,security-and-proxy-headers"
And then it works !
Thanks for this great plugin and your help

Hi,
Finally got fully working with nextcloud (had few modsecurity rules to exclude).
Main issue was with the "whoami" backend, it has crashed randomly with big / long uploads.
I had to use nginx which performs better and doesn't crash:

Nginx config:


Add delete property editor value endpoint

https://github.com/systeminit/si/assets/227823/7828ebbb-e175-4cf0-85d2-700a9ec826d5

LGTM 👍 working exactly as expected with the minor nit of the trash cans being slightly misaligned.

bors try

bors merge


Allow for images output by one Task to be used by steps of downstream Tasks

Expected Behavior
Sometimes folks need to be able to build an image, which later they need to run as part of their CI/CD. For example, say you need an image (like we did for https://github.com/tektoncd/pipeline/pull/632) that has:

gcloud
ko
golang

You might not be able to find a published and supported image that meets your exact needs, so you might need to build and publish your own. It would be great to have the option to do this as part of your CI/CD pipeline, e.g. something like:
apiVersion: tekton.dev/v1alpha1
kind: Pipeline
metadata:
  name: demo-pipeline
spec:
  resources:
  - name: source-repo
    type: git
  - name: ko-image
    type: image
  tasks:
  - name: build-ko-image
    taskRef:
      name: ko-image-builder
    resources:
      inputs:
      - name: workspace
        resource: source-repo
      outputs:
      - name: builtImage
        resource: ko-image
  - name: use-ko-image
    taskRef:
      name: run-ko
    resources:
      inputs:
      - name: workspace
        resource: source-repo
      - name: builtImage
        resource: web-image
        from: [build-ko-image]

Where the run-ko Task looks something like:
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: run-ko
spec:
  inputs:
    resources:
    - name: workspace
      type: git
    - name: ko-image
      type: image
  steps:
  - name: build-and-push
    image<EMAIL_ADDRESS>    command:
       ...

image<EMAIL_ADDRESS>would allow the Task to use an image which is actually an input resource
Actual Behavior
The only way to do this now would be to make the above Task something like:
```yaml
apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: run-ko
spec:
  inputs:
    resources:
    - name: workspace
      type: git
  steps:
  - name: build-and-push
    image: my-ko-image
    command:
       ...

And hope that `my-ko-image:latest` happens to be the one that was built by the previous Task.

# Additional Info

* https://github.com/tektoncd/pipeline/issues/216 is about adding Image outputs

We may want to wait on immutable resource before we tackle this one.

We may want to wait on immutable resource before we tackle this one.

I think we can actually do this today, judging by the golang catalog tasks:
https://github.com/tektoncd/catalog/blob/947198561e74b3a022e35ead29e01680e6634304/task/golang-build/0.1/golang-build.yaml#L41


Add pkill signals to custom modules

This implements pkill signals for custom modules (see #170). You can define it with "signal" in the config and test it with pkill -RTMIN+8 waybar (for example signal 8).
In addition there is now also the setting "interval": "once", to load the module only once at startup.
edit: I already updated the wiki for this PR. I didn't know they were directly live. If you need time for the review, I can reverse it. Just let me know.

I don't think it's the best way for the `once' interval but for now i'll leave it like that, Thanks!


Implementing Arctan function using Python

🚀 Feature
I'll demonstrate how scientific calculators can calculate the inverse of the Tan function. (Arctan)
Have you read the Contributing Guidelines on Pull Requests?

[x] Yes

Motivation
I am a GSSOC participant who like writing clean readable Python code, and I have interest in Math, Data Structures and Algorithms.
Pitch
Many developers don't like using things without knowing how it works or at least get a general idea about how It operates. One of the things we student developers use without paying atenttion about how it works is the scientific calculator, and in this script, I will focus some light on explaining one of its functionalities.

/assign


Fix asserts

The surrounding if-clauses prevent the checks from actually guarding against anything.

My theory on how this happened is that guards like this can be useful if they avoid a string.Format for the message. But that's not the case for these instances.
LGTM

I've updated the PR to remove the src/OrcasEngine change from it.

:shipit:


Error boundary and Context

Fix #11
Fix #10

@alfonsogarciacaro , thanks, it was anticipated so much! :)


RenderBox was not laid out

I can't see the app I am creating, I just see the background color and I get this error:
RenderBox was not laid out: RenderPadding#7dc5d relayoutBoundary=up10 NEEDS-PAINT
'package:flutter/src/rendering/box.dart':
Failed assertion: line 1929 pos 12: 'hasSize'
It was working perfectly fine till now

w

Hi @tamara-argentina, Thanks for filing the issue.

Failed assertion: line 1929 pos 12: 'hasSize'

From the error, it looks like you have not provided size constraint to your widget.
Please share a minimal and complete reproducible code along with the output of flutter doctor -v to better address the issue.
Thanks


MAINT: Replace all uses of NumarrayTypes in ni_morphology.c

I think this is the last of these...

Last one, phew:) Thanks Jaime.

Note that NI_NormalizeType is now unused, you can delete it in a next PR.


swift: command not found

The image says it all:

Also Swift is pretty outdated.

Swift is only available through running remotely.
You have to press the run button

Swift is only available through running remotely.
You have to press the run button

That doesn’t work in TestFlight builds. I tried already.

Swift is only available through running remotely.
You have to press the run button

That doesn’t work in TestFlight builds. I tried already. Plus that’s not what I’m trying to do. I’m trying to create a Package.swift file.

I don't think the ide has swift or a swift terminal command

There isn’t a swift compiler available in Code App. Use Swift Playground
for that.
On Thu, 30 Jun 2022 at 22:49, hg0428 @.***> wrote:

Swift is only available through running remotely.
You have to press the run button
That doesn’t work in TestFlight builds. I tried already. Plus that’s not
what I’m trying to do. I’m trying to create a Package.swift file.
I don't think the ide has swift or a swift terminal command
—
Reply to this email directly, view it on GitHub
https://github.com/thebaselab/codeapp/issues/505#issuecomment-1171317005,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJE6T2YCUVYV2D2CJKK5YJTVRWXWFANCNFSM5ZX5SX6Q
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

There isn’t a swift compiler available in Code App. Use Swift Playground
for that.
On Thu, 30 Jun 2022 at 22:49, hg0428 @.***> wrote:

Swift is only available through running remotely.
You have to press the run button
That doesn’t work in TestFlight builds. I tried already. Plus that’s not
what I’m trying to do. I’m trying to create a Package.swift file.
I don't think the ide has swift or a swift terminal command
—
Reply to this email directly, view it on GitHub
https://github.com/thebaselab/codeapp/issues/505#issuecomment-1171317005,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJE6T2YCUVYV2D2CJKK5YJTVRWXWFANCNFSM5ZX5SX6Q
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Ok, thanks for the info (sorry for the late response)


[BACKLOG-18169] Added pentaho server to run configuration

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -fn -f 'pom.xml' -pl 'plugins/engine-configuration/ui,ui,engine,plugins/engine-configuration/impl,plugins/engine-configuration/api' -P '!assemblies' -amd clean install && mvn -B -f 'pom.xml' -pl 'plugins/engine-configuration/ui,ui,engine,plugins/engine-configuration/impl,plugins/engine-configuration/api' -P '!assemblies' -amd site

Cleanup Commands
mvn -B -f 'pom.xml' -pl 'plugins/engine-configuration/ui,ui,engine,plugins/engine-configuration/impl,plugins/engine-configuration/api' -P '!assemblies' -amd build-helper:remove-project-artifact

Changed files
  engine/src/main/java/org/pentaho/di/ExecutionConfiguration.java
  engine/src/main/java/org/pentaho/di/job/JobExecutionConfiguration.java
  engine/src/main/java/org/pentaho/di/job/entries/job/JobEntryJob.java
  engine/src/main/resources/org/pentaho/di/job/entries/job/messages/messages_en_US.properties
  engine/src/main/resources/org/pentaho/di/job/entries/trans/messages/messages_en_US.properties
  plugins/engine-configuration/api/pom.xml
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfiguration.java
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfigurationDialog.java
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfigurationExecutor.java
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfigurationFactory.java
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfigurationProvider.java
  plugins/engine-configuration/api/src/main/java/org/pentaho/di/engine/configuration/api/RunConfigurationUI.java
  plugins/engine-configuration/impl/pom.xml
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/MetaStoreRunConfigurationFactory.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/RunConfigurationManager.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/extension/RunConfigurationExtensionPoint.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/extension/RunConfigurationRunExtensionPoint.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/DefaultRunConfiguration.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/DefaultRunConfigurationExecutor.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/DefaultRunConfigurationProvider.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/DefaultRunConfigurationUI.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/scheduler/SchedulerRequest.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/pentaho/scheduler/SpoonUtil.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/spark/SparkRunConfiguration.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/spark/SparkRunConfigurationExecutor.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/spark/SparkRunConfigurationProvider.java
  plugins/engine-configuration/impl/src/main/java/org/pentaho/di/engine/configuration/impl/spark/SparkRunConfigurationUI.java
  plugins/engine-configuration/impl/src/main/resources/org/pentaho/di/engine/configuration/impl/pentaho/messages/messages_en_US.properties
  plugins/engine-configuration/impl/src/main/resources/org/pentaho/di/engine/configuration/impl/spark/messages/messages_en_US.properties
  plugins/engine-configuration/impl/src/test/java/org/pentaho/di/engine/configuration/impl/pentaho/DefaultRunConfigurationExecutorTest.java
  plugins/engine-configuration/ui/src/main/java/org/pentaho/di/engine/ui/RunConfigurationDialog.java
  plugins/engine-configuration/ui/src/main/resources/org/pentaho/di/engine/ui/messages/messages_en_US.properties
  ui/src/main/java/org/pentaho/di/ui/core/dialog/ConfigurationDialog.java
  ui/src/main/java/org/pentaho/di/ui/job/dialog/JobExecutionConfigurationDialog.java
  ui/src/main/java/org/pentaho/di/ui/job/entries/job/JobEntryJobDialog.java
  ui/src/main/java/org/pentaho/di/ui/job/entries/trans/JobEntryBaseDialog.java
  ui/src/main/java/org/pentaho/di/ui/job/entries/trans/JobEntryTransDialog.java
  ui/src/main/java/org/pentaho/di/ui/spoon/delegates/SpoonJobDelegate.java
  ui/src/main/java/org/pentaho/di/ui/spoon/delegates/SpoonTransformationDelegate.java
  ui/src/main/java/org/pentaho/di/ui/trans/dialog/TransExecutionConfigurationDialog.java

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.injection.bean.BeanInjector

Branch Change: -.79%:small_red_triangle_down:
Complexity Change: -1.30%:small_red_triangle_down:

org.pentaho.di.core.lifecycle.KettleLifecycleSupport

Branch Change: -8.33%:small_red_triangle_down:
Complexity Change: -9.09%:small_red_triangle_down:

org.pentaho.di.job.entries.job.JobEntryJob

Branch Change: -.08%:small_red_triangle_down:
Complexity Change: -.13%:small_red_triangle_down:
Instruction Change: -.35%:small_red_triangle_down:
Line Change: -.39%:small_red_triangle_down:
Method Change: -.72%:small_red_triangle_down:

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change:  + .66%
Complexity Change:  + .93%
Instruction Change:  + .13%

org.pentaho.di.job.JobExecutionConfiguration

Complexity Change: -.48%:small_red_triangle_down:
Instruction Change: -.27%:small_red_triangle_down:
Line Change: -.51%:small_red_triangle_down:
Method Change: -.88%:small_red_triangle_down:

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change: -.10%:small_red_triangle_down:

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change:  + 1.92%
Complexity Change:  + 1.69%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change: -.11%:small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change:  + 2.00%
Complexity Change:  + 1.89%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change: -1.18%:small_red_triangle_down:
Complexity Change: -1.35%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change:  + 1.79%
Complexity Change:  + 1.19%
Instruction Change:  + .07%

org.pentaho.di.engine.configuration.impl.pentaho.DefaultRunConfiguration

Complexity Change: -3.85%:small_red_triangle_down:
Instruction Change: -4.50%:small_red_triangle_down:
Line Change: -2.27%:small_red_triangle_down:
Method Change: -3.85%:small_red_triangle_down:

org.pentaho.di.engine.configuration.impl.pentaho.DefaultRunConfigurationExecutor

Branch Change: -1.14%:small_red_triangle_down:
Complexity Change: -2.38%:small_red_triangle_down:
Instruction Change:  + 3.39%
Line Change:  + 3.45%

org.pentaho.di.engine.configuration.impl.pentaho.DefaultRunConfigurationProvider

Branch Change: -25.00%:small_red_triangle_down:
Complexity Change: -19.48%:small_red_triangle_down:
Instruction Change: -13.08%:small_red_triangle_down:
Line Change: -7.41%:small_red_triangle_down:
Method Change: -18.18%:small_red_triangle_down:

org.pentaho.di.engine.configuration.impl.pentaho.scheduler.SchedulerRequest

Complexity New Coverage: 100.00%
Instruction New Coverage: 81.08%
Line New Coverage: 80.00%
Method New Coverage: 100.00%

org.pentaho.di.engine.configuration.impl.pentaho.scheduler.SchedulerRequest.Builder

Branch New Coverage: 50.00%
Complexity New Coverage: 60.00%
Instruction New Coverage: 51.28%
Line New Coverage: 57.14%
Method New Coverage: 100.00%

org.pentaho.di.engine.configuration.impl.pentaho.scheduler.SpoonUtil

Branch New Coverage: 50.00%
Complexity New Coverage: 75.00%
Instruction New Coverage: 61.36%
Line New Coverage: 60.00%
Method New Coverage: 85.71%

org.pentaho.di.engine.configuration.impl.RunConfigurationManager

Branch Change: -16.91%:small_red_triangle_down:
Complexity Change: -14.01%:small_red_triangle_down:
Instruction Change: -19.75%:small_red_triangle_down:
Line Change: -18.36%:small_red_triangle_down:
Method Change: -13.82%:small_red_triangle_down:

org.pentaho.di.engine.configuration.impl.spark.SparkRunConfiguration

Complexity Change: -8.18%:small_red_triangle_down:
Instruction Change: -11.50%:small_red_triangle_down:
Line Change: -5.64%:small_red_triangle_down:
Method Change: -8.18%:small_red_triangle_down:

org.pentaho.di.engine.configuration.impl.spark.SparkRunConfigurationExecutor

Branch Change: -4.55%:small_red_triangle_down:
Complexity Change: -7.14%:small_red_triangle_down:
Instruction Change: -1.04%:small_red_triangle_down:

org.pentaho.di.engine.configuration.impl.spark.SparkRunConfigurationProvider

Complexity Change: -27.78%:small_red_triangle_down:
Instruction Change: -23.91%:small_red_triangle_down:
Line Change: -14.77%:small_red_triangle_down:
Method Change: -20.83%:small_red_triangle_down:

org.pentaho.di.trans.steps.gpload.GPLoadMeta

Instruction Change: -.09%:small_red_triangle_down:

org.pentaho.di.ui.repository.pur.repositoryexplorer.model.UIEERepositoryUser

Branch Change: -8.33%:small_red_triangle_down:
Complexity Change: -6.67%:small_red_triangle_down:
Instruction Change: -.85%:small_red_triangle_down:
Line Change: -3.03%:small_red_triangle_down:

org.pentaho.di.ui.repository.pur.repositoryexplorer.model.UISecurityRole

Branch Change:  + 1.79%
Complexity Change:  + 1.54%
Instruction Change:  + .16%
Line Change:  + .58%

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change:  + .15%

org.pentaho.di.ui.job.entries.job.JobEntryJobDialog

Complexity Change:  + .08%
Instruction Change: -.26%:small_red_triangle_down:
Line Change: -.17%:small_red_triangle_down:

org.pentaho.di.ui.job.entries.trans.JobEntryBaseDialog

Complexity Change:  + .55%
Instruction Change:  + .03%
Line Change:  + .05%
Method Change:  + 3.33%

org.pentaho.di.ui.job.entries.trans.JobEntryTransDialog

Instruction Change: -.01%:small_red_triangle_down:

org.pentaho.di.ui.spoon.delegates.SpoonJobDelegate

Instruction Change: -.07%:small_red_triangle_down:
Line Change: -.07%:small_red_triangle_down:

org.pentaho.di.ui.spoon.delegates.SpoonTransformationDelegate

Instruction Change: -.01%:small_red_triangle_down:
Line Change: -.01%:small_red_triangle_down:

@ddiroma


Add more DLL support

vcruntime140
vcruntime140_1
msvcp140 (Needs explicit definition in UniversalProxyDLL
msvcp140_1
msvcp140_2

I could not get vcruntime140 or vcruntime140_1 to work when I tested with several games and I could not get msvcp140 to build:

For the rest I don't think I found a game that imported them.

In my case it was the operation harsh doorstop dedicated server running on UE4

Whats wrong?

It's been a while since I tested but I couldn't really get any of the added DLLs to work.
The vcruntime ones built but only caused a crash when trying to use them. I can't really see any obvious reason why they shouldn't work.
MSVCP didn't build for me and I suspect it's got to do with the export names, as far as I can see the export names are not supposed to have spaces in them. Looking in PE-bear I see the names are displayed as for example:
??$_Getvals@_W@?$time_get@DV?$istreambuf_iterator@DU?$char_traits@D@std@@@std@@@std@@IEAAX_WAEBV_Locinfo@1@@Z
And I couldn't easily find any games that imported msvcp140_1 or msvcp140_2.

So everyone who needs them needs to use my fork then, i see. Thanks anyway


Nested selector workaround in react

Same problem as here but with simplebar-placeholder selector. Parent simplebar selects simplebar-placeholder from it's nested child simplebar here.
Here is an example: https://jsfiddle.net/e5hufs3o/17/

@Grsmto Hi! Can you look at this issue? Should we update snapshots or maybe make fixes in core package and don't touch simplebar-react?

Hey Odrin, yes I suppose that would make sense to fix this the same way you did on your first PR?
I see how changing the order fixes it. Maybe it's not a really trustful solution and the same fix as you did last time would be preferable?

Ok, I will create another PR today.


PLAT-10144: Adding built-in help command

Ticket
PLAT-10144
Description
Adding built-in help command
Dependencies
List the other pull requests that should be merged before/along this one.
Checklist

[x] Referenced a ticket in the PR title and in the corresponding section
[x] Filled properly the description and dependencies, if any
[x] Unit tests updated or added
[x] Javadoc added or updated
[ ] Updated the documentation in docs folder

Otherwise it looks ok to me

Otherwise it looks ok to me


Dissertation available?

Hi Le Tuan Anh,
PhD Dissertation: Developing and Applying an Integrated Semantic Framework for Natural Language Understanding
Are you planning to make your dissertation available ?

It is not submitted yet.
On Fri, Aug 24, 2018 at 11:45 AM Alexandre Rademaker <
<EMAIL_ADDRESS>wrote:

PhD Dissertation: Developing and Applying an Integrated Semantic Framework
for Natural Language Understanding
Are you planning to make your dissertation available ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/letuananh/intsem.fx/issues/4, or mute the thread
https://github.com/notifications/unsubscribe-auth/ABD8xp3r5rCt2maJpm7x1BPgamTnKbgoks5uT1qzgaJpZM4WKnLN
.

--
Francis Bond http://www3.ntu.edu.sg/home/fcbond/
Division of Linguistics and Multilingual Studies
Nanyang Technological University


Frame animation are scattered in images folder

That's not an issue, please stop opening issues or I'll have to block your account.

How is that not an issue?

Because it's intended behaviour

What am I supposed to do with that then?

Open the mfa, extract it using Clickteam

And what if I don't have Clickteam or don't want to extract hundreds of images manually?

Well the first ones your problem, the second one, Clickteam lets you extract entire animations at once.

And what if there's a lot of animations to export? Can I export them all at once?

Nope, but it's either that, or coding your own decompiler

Ok. Fine. Sorry to have bothered you.


jest-enzyme: vulnerability related to unset-value

We are seeing vulnerability in jest-enzyme version 7.1.2 which is using<EMAIL_ADDRESS>The fix for this is to upgrade unset-value to 2.0.1
https://security.snyk.io/vuln/SNYK-JS-UNSETVALUE-2400660
here's the dependency tree:
└─┬<EMAIL_ADDRESS> └─┬<EMAIL_ADDRESS>   └─┬<EMAIL_ADDRESS>     └─┬<EMAIL_ADDRESS>       └─┬<EMAIL_ADDRESS>         └─┬<EMAIL_ADDRESS>           └─┬<EMAIL_ADDRESS>             └─┬<EMAIL_ADDRESS>               └─┬<EMAIL_ADDRESS>                 └──<EMAIL_ADDRESS>Can you help with this please?

Prototype pollution isn't really an attack vector when it's in your test framework - anyone who has the authority to write tests already can do far more dangerous things.


add more common file suffixes

[x] Closes #142
[ ] Tests added
[ ] Changes are documented in docs/releases.rst
[ ] New functions/methods are listed in api.rst

I would suggest considering just not trusting file extensions at all and only using the magics to identify file types.

I took a stab at just using magic bytes in the last commit.  Note that I changed the 'netcdf4' mapping to 'hdf5' since that is the format behind the scenes based on magic_bytes.  Tests seem to pass locally, I haven't added new tests, but did try the follow 'test urls':
import fsspec
# https://en.wikipedia.org/wiki/List_of_file_signatures
examples = {
    'grib':'https://github.com/pydata/xarray-data/raw/master/era5-2mt-2019-03-uk.grib',
    'netcdf3':'https://github.com/pydata/xarray-data/raw/master/air_temperature.nc',
    'netcdf4':'https://github.com/pydata/xarray-data/raw/master/ROMS_example.nc',
    'hdf4':'https://github.com/corteva/rioxarray/raw/master/test/test_data/input/MOD09GA.A2008296.h14v17.006.2015181011753.hdf',
    'tif':'https://github.com/corteva/rioxarray/raw/master/test/test_data/input/cog.tif',
     #from https://github.com/astropy/astropy/blob/4d034aa7e27e31cb0241cc01bbe76eab47406a91/astropy/io/fits/tests/test_fsspec.py#L73
    'fits':'https://mast.stsci.edu/api/v0.1/Download/file/?uri=mast:HST/product/ibxl50020_jif.fits',
    'jpg': 'https://github.com/rasterio/rasterio/raw/main/tests/data/389225main_sw_1965_1024.jpg',

}

for file_type,url in examples.items():
    with fsspec.open(url) as f:
        magic_bytes = f.read(8)
        print(file_type, magic_bytes)

    if magic_bytes.startswith(b"CDF"):
        print('CDF!')
    elif magic_bytes.startswith(b"\x89HDF"):
        print('HDF5!')
    elif magic_bytes.startswith(b"\x0e\x03\x13\x01"):
        print('HDF4!')
    elif magic_bytes.startswith(b'GRIB'):
        print('GRIB!')
    elif magic_bytes.startswith(b'II*'):
        print('TIFF!')
    elif magic_bytes.startswith(b'SIMPLE'):
        print('FITS!')
    else:
        raise NotImplementedError(f"Unrecognised file based on header bytes: {magic_bytes}")

That list of examples is cool - is there any point adding tests of opening those files to the test suite?

I think it's useful. I could add a @network marker as Xarray does? Should have some time to do that later this week
https://github.com/pydata/xarray/blob/be8e17e4dc5da67d7cbb09db87d80c1bbc71a64e/conftest.py#L10

I could add a @network marker as Xarray does?

Yes nice. This test of reading from s3 could also use that marker.

Worked on this a bit more @TomNicholas and I think it's ready to go. Added tests for all formats and linked to example files for all kerchunk-supported formats... But it seems like some additional work is needed to fully support TIFF, FITS, and HDF5.

Thanks so much @scottyhq !


Failing build

Hi. It seems that there is something in InternetTest::testIpv4NotLocalNetwork or ipv4() which is making the build to fail.
Details below:
There was 1 failure:
1) Faker\Test\Provider\InternetTest::testIpv4NotLocalNetwork
Failed asserting that '<IP_ADDRESS>' does not match PCRE pattern "/\A1\./".
/home/travis/build/fzaninotto/Faker/test/Faker/Provider/InternetTest.php:113

I've run the test on the PCRE pattern and it seems to be fine.

Ok, looks like I do know where the problem is.
Function assertNotRegExp: Asserts that a string does not match a given regular expression.
This IP address does match, so the test fails.
So it certainly looks like that ipv4() function has generated a failing IP address.

I added another method, testPhoneNumberFormat(), to the unit test class. Build passed, issue resolved. How ever: since the InternetTest::testIpv4NotLocalNetwork method uses mt_rand() random number generator, virtually any commit can fail the build. There must be some way to amend the unit test mechanism so it won't randomly fail the commits.

I think this issue can be closed for now.


any_instance_of does not restore method on JRuby

Subject of the issue
A special combination of any_instance_of calls can prevent rspec-mocks from properly restoring the originally mocked method method during it cleanup step. I'm only able to observe this on JRuby.
Your environment

Ruby version: jruby <IP_ADDRESS>
rspec-mocks version: 3.9.1

Steps to reproduce
# frozen_string_literal: true

begin
  require "bundler/inline"
rescue LoadError => e
  $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
  raise e
end

gemfile(true) do
  source 'https://rubygems.org'

  gem 'rspec', '3.9.0'
  gem 'rspec-mocks', '3.9.1'
end

require 'rspec/autorun'

puts "Ruby version is: #{RUBY_VERSION}"
STDERR.puts 'You need JRuby to reproduce this issue!' unless RSpec::Support::Ruby.jruby?

RSpec.describe do
  it do
    clazz = Class.new do
      prepend Module.new

      def foo
      end
    end

    allow_any_instance_of(clazz).to receive(:foo)

    RSpec::Mocks.space.reset_all

    expect_any_instance_of(clazz).to_not receive(:foo)

    RSpec::Mocks.space.reset_all

    clazz.new.foo
  end
end

Expected behavior
No failures:
1 example, 0 failures

Actual behavior
Failures:

  1)
     Failure/Error: clazz.new.foo

     NoMethodError:
       undefined method `foo' for #<#<Class:0x1d008e61>:0x29db008c>
       Did you mean?  fork
     # rspec-mock-issue.rb:39:in `block in rspec-mock-issue.rb'

Finished in 0.02769 seconds (files took 0.15631 seconds to load)
1 example, 1 failure

Preliminary investigation
My semi-educated guess is that the issue lies in this code block: https://github.com/rspec/rspec-mocks/blob/v3.9.1/lib/rspec/mocks/any_instance/recorder.rb#L202-L222
Forcing alias_method method_name, alias_method_name to when running on JRuby in that code block "solves" the problem from my example. I unfortunately wasn't able process how the comment included in that snippet linked is applying to our case, so I'm not 100% confident that this is the correct solution.

Nice find, thanks for reporting.
Would it make sense to add JRuby check to RUBY_VERSION < "2.3"?
Is there a more real-life spec that we could add to prevent regression?
Does it make sense to open a JRuby ticket to eventually address this?

@marcotc ping

For context, I spent around 2 tries trying to fix this issue, but I wasn't able to figure out how the internals of rspec-mocks should work around the involved the area. My goal was to open a PR to fix the issue, and the only reason I'm opening an issue instead because I wasn't able to come up with a solution.

Would it make sense to add JRuby check to RUBY_VERSION < "2.3"?

It would this specific issue, but I don't have enough confidence to say if that's the right solution.

Is there a more real-life spec that we could add to prevent regression?

I tried my best to reduce the test to its minimal form, and I wasn't able to simplify it any further than the reported version above.
In our use-case it boiled down to:

A test calls allow_any_instance_of(clazz).to receive(:foo).
Subsequent test calls expect_any_instance_of(clazz).to_not receive(:foo).
After that, clazz does not respond to :foo anymore.

Trying to simplify/remove any of these steps did not allow me to reproduce the issue anymore.

Does it make sense to open a JRuby ticket to eventually address this?

I don't have the confidence to know if the issue lies in this repo or JRuby. I noticed there's an issue linked from rspec-mocks about an MRI but that I thought was affecting JRuby, but JRuby actually behaves as expected in that scenario, so it's not that same issue.

- if RUBY_VERSION < "2.3" || backed_up_method_owner[method_name.to_sym] == self
+ if RUBY_VERSION < "2.3" || RSpec::Support::Ruby.jruby? || backed_up_method_owner[method_name.to_sym] == self

made your spec green on jruby-head (jruby <IP_ADDRESS>-SNAPSHOT (2.6.5) 2020-08-25 2f0c49000a OpenJDK 64-Bit Server VM 14.0.1+14 on 14.0.1+14 +jit [darwin-x86_64])
I propose the following:

test on earlier rubies available in RVM, check if the above change works on all
open a ticket for JRuby
add a spec and a note to the above change mentioning JRuby ticket
send a pull request with the change

Ah JRuby, I haven't missed your quirks, is this something thats happening only on the current head? Why haven't our builds caught it @pirj?

The example https://github.com/rspec/rspec-mocks/blob/018d0d54a0ecaed259fd4eac5f151c51abb57b1d/spec/rspec/mocks/any_instance_spec.rb#L204 is a bit different. Here, there's no even a parent class involved.
But there is:

prepend
double stubbing
double reset

without any one of those the issue doesn't reproduce.

Care to add an example to our build that fails as a draft as a first step?

Closing due to inactivity / lack of a reproduction during the monorepo migration.


Embedded service URL different from server URL

I think I found a little glitch in using the project when deployed in complex networking environments (e.g. virtual machine hosts/guests, Docker containers, NAT port forwarding, etc.).
In such environments I was able to access the base OData feed but not the entities within. I changed the code such that the URL embedded in the served XML is now configurable. I added an entry service_advertise_root in the [server] section of the configuration file and modified server.py:
def configure_app(c, doc):
    service_root = c.get('server', 'service_advertise_root')
    logger.info("Advertising service at %s" % service_root)
    app = ReadOnlyServer(serviceRoot=service_root)
    app.SetModel(doc)
    return app

Once I did the above, I was able to access our data with OData dev tools (e.g. Sesame) and ultimately able to connect up Power BI with our production system. In our usage Influx and odata-influxdb are both running in Docker containers with NAT port forwarding to the host machine.

thanks for this one too @mkarlesky  :+1: - reckon you could double check my additions works for your situation?

Thanks for putting in the effort to figure out a proper conditional for the new option. However, there are syntax errors in configure_app() (two mismatched parens). In addition, I also found that the new start_server() is no longer configurable but hardcoded for localhost:8080.
I would have submitted a pull request for the fixes, but I was not able to get my server to start up reliably with the new werkzeug wrapper approach. Anything other than localhost yielded socket errors about an address already being in use. In the end I grabbed the few lines of the new conditional code, corrected the syntax issues, and kept on using our modified version of a previous commit.
Incidentally, you may want to update get_sample_config() with a comment for the new configuration option.
Thanks again for the project. Saved us a ton of effort.
p.s. Related to configuration options… we found the envtpl templating tool to be great for building production.conf from configurable environment variables in our Docker container. Thought you might like to know how your project has been deployed by one of its users.

Ah, yeah whoops, got my wires crossed on the version i was testing and didn't consider the new way it's going to be done to have auth. thinking i'll wait till we figure that out properly first then get to this

I think in my development of the auth feature I removed the previous code that was parsing the service_root config option. I like your option name, service_advertise_root, but I think it is synonymous with the previous service_root option so I'm going to clean that up.


Refactored ajax to use promises

I refactored the ajax call to use promises instead of callbacks. Why bother? Using promises make another pull request I'm working on very simple. It also removes the callbacks from the options, which prevents a developer from breaking them in the pjax:beforeSend event. In my opinion, the syntax is also a little cleaner.

Sorry that I never got around to doing this. Pjax is now largely unmaintained. I've made some fixes to it, but I'm refraining from pulling this because it only changes things syntax-wise and doesn't result in a tangible benefit. If there will be a future version of pjax, it will be actual Promise-based and not dependent on jQuery.


CDC USB shell receives garbage when application starts

Describe the bug
When the shell is enabled, with serial backend, using USB CDC directly without console, at startup the application seems to receive what can only be described as complete garbage:
^[[m
^[[m: command not found
uart:~$
uart:~$
uart:~$
*** Booting Zephyr OS build v3.3.0-rc2-111-g226062396ec4 ***
[00:00:00.003,021] <inf> littlefs: LittleFS version 2.5, disk version 2.0
[00:00:00.003,204] <inf> littlefs: FS at flash-controller@4001e000:0xdc000 is 4 0x1000-byte blocks with 512 cycle
[00:00:00.003,234] <inf> littlefs: sizes: rd 16 ; pr 16 ; ca 64 ; la 32
[00:00:00.003,540] <inf> littlefs: /lfs1 mounted
[00:00:00.006,622] <inf> smp_bt_sample: Advertising successfully started
[00:00:00.006,683] <err> smp_sample: Failed to enable USB
uart:~$ ^[[1;32mu

This seemingly seems to be data that the USB CDC/shell has put into the transmit buffer as it looks to be a colour escape code.
To Reproduce
Build samples/mgmt/mcumgr/smp_svr for nrf52840dongle_nrf52840 using USB CDC
Expected behavior
No garbage to be present
Impact
Annoyance. Possibly indicative of a larger memory issue.
Environment (please complete the following information):

Commit SHA or Version used: a5e04ccd4ab976b99f45d106e3de873fc8dd882a

It is already described and fixed here https://github.com/zephyrproject-rtos/zephyr/issues/36948
Is CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR set?
Please also note https://github.com/zephyrproject-rtos/zephyr/issues/47120

CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR is set (it can't be disabled apparently). This is not echoed by the terminal, and it actually feeds itself back to the USB CDC port's input (verified with wireshark that when the module boots, it sends the characters then sends the command not valid response indicating that it gave input to itself without a terminal being attached)

See trace here usb.zip, terminal is connected (which sends nothing to the module, just receives), raw output is:
\1b[m

\1b[1;32muart:~$ \1b[m^[[m
\1b[1;31m^[[m: command not found
\1b[m\1b[1;32muart:~$ \1b[m
\1b[1;32muart:~$ \1b[m
\1b[1;32muart:~$ \1b[m
\1b[1;32muart:~$ \1b[m^[\1b[10D\1b[J*** Booting Zephyr OS build v3.3.0-rc2-111-g226062396ec4 ***
\1b[1;32muart:~$ \1b[m^[\1b[10D\1b[J[00:00:00.002,197] \1b[0m<inf> littlefs: LittleFS version 2.5, disk version 2.0\1b[0m

CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR is set (it can't be disabled apparently).

To Reproduce
Build samples/mgmt/mcumgr/smp_svr for nrf52840dongle_nrf52840 using USB CDC

I am not convinced of this. Please provide exact information how to reproduce.

I am not convinced of this. Please provide exact information how to reproduce.

None of this images enables CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR.
Can you please try samples/subsys/shell/shell_module/ using cmake -DBOARD=nrf52840dongle_nrf52840 -DDTC_OVERLAY_FILE=usb.overlay -DOVERLAY_CONFIG=overlay-usb.conf ../ ?

Same output (on the second ttyACM device):
[1;32muart:~$ ^[[m
^[[m: command not found
uart:~$
uart:~$
uart:~$
uart:~$ ^[
^[: command not found

And CONFIG_SHELL_BACKEND_SERIAL_CHECK_DTR is enabled for this

Same output/behaviour if that Kconfig is disabled too

I can't reproduce this on an nrf52840dk without mcuboot, only on the dongle, wonder if it's something to do with that but can't think why that would have any impact on it

See trace here usb.zip, terminal is connected (which sends nothing to the module, just receives), raw output is:

The data is coming out from host. Packet 213 contains data from device 1b 5b 6d 0d 0a 0d 0a 1b and  then hosts sends data in packet 215 and following ones. Unfortunately Wireshark CDC ACM dissector doesn't support follow stream feature and hence there is no easy way to see all the data flowing in and out on the interface.
When ttyACM is opened in terminal application the "garbage" quickly ends. However, then the output is simply read with cat /dev/ttyACM0, there is endless feedback where the Linux host driver sends some data to device. The stream can be stopped by running stty -F /dev/ttyACM0 raw in another terminal.

This seems to be related to echoing control characters. Doing stty -F /dev/ttyACM0 -echo followed by cat /dev/ttyACM0 gives expected result.

I believe the problem is in Linux kernel actively reading IN endpoint immediately after ttyACM open() call before application has a chance to set TTY parameters, i.e. disable ECHO. This seems really like a race condition and the solution would be to make Linux read IN endpoint only after user-space application issues read(). Note that the race is between userspace open() and ioctl() and is completely unrelated to when enumeration happens. If ttyACM device is not opened, kernel does not read IN endpoint. Device has no reliable way to determine when open() call happens.
The workaround on device side would be to arm IN endpoint with ZLP once interface is enabled and once the host reads the ZLP add some timeout before real data can be armed. But this is really trying to solve the issue not where it exists and as such is prone to fail.
CDC ACM here is used as a "virtual COM" that does not care about baud rate and simply provides data transport mechanism. If the CDC ACM interface is connected to device that does not transmit on its own (e.g. only replies to commands received) the issue is not there at all. Also the baud rate minimizes the "race window" - in Zephyr case we can consider the baud rate to be infinite as it is not arbitrarily limited.

The issue does not happen on Windows because it does not ECHO characters received by default.
It is not easily observable on other boards, especially on targets that transmit with real UART to JLink because:

default UART baud rate in Linux is 9600 but Zephyr uses 115200
JLink stays connected across target device resets (i.e. kernel reboot cold will not show the issue, because it is the same ttyACM instance and not a new one)

The easiest way to observe the issue on nRF52840DK is to change default baud rate to 9600, e.g. with following patch:
diff --git a/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts b/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
index bc87fa7077..f6c4bb38eb 100644
--- a/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
+++ b/boards/arm/nrf52840dk_nrf52840/nrf52840dk_nrf52840.dts
@@ -149,7 +149,7 @@
 &uart0 {
        compatible = "nordic,nrf-uarte";
        status = "okay";
-       current-speed = <115200>;
+       current-speed = <9600>;
        pinctrl-0 = <&uart0_default>;
        pinctrl-1 = <&uart0_sleep>;
        pinctrl-names = "default", "sleep";

After flashing, the steps to reproduce are:

disconnect JLink
stop all terminal processes that could try to auto configure tty once it becomes available (e.g. stop GTKTerm)
connect JLink, see kernel log for ttyACM number (it is ttyACM0 if no other devices are present)
cat /dev/ttyACM0 (note that this command does not disable echo but does call open() on ttyACM)
press RESET button on the DK (this resets the target, leaves JLink running)
observe endless feedback loop

To break out of the loop, disable echo using stty -F /dev/ttyACM0 -echo. If you call stty without stopping cat you will observe that eventually the queue will empty. Then you can send some command with e.g. echo "help" > /dev/ttyACM0

Related patch in linux-usb Re: [PATCH] cdc-acm: acm_init: Set initial BAUD to B0 led me to check SHELL_BACKEND_SERIAL_CHECK_DTR and it seems that SHELL_BACKEND_SERIAL_CHECK_DTR is not working as expected.
SHELL_BACKEND_SERIAL_CHECK_DTR is introducting changes only in uart_callback. However during initialization shell_start() just calls z_shell_vt100_color_set() which after few calls goes down to z_shell_write() that does call shell->iface->api->write (write() function in shell_uart.c) which calls irq_write() that simply puts the data to tx ringbuf and calls uart_irq_tx_enable(). This arms the endpoint really early, but the data is only transmitted after Linux user-space opens ttyACM device (before host does not try to read the IN endpoint). The actual IN data packet response is sent before host sets DTR.
SHELL_BACKEND_SERIAL_CHECK_DTR is obviously not respected in Zephyr and it must be fixed before proceeding further on this issue.

CC @jakub-uC

I believe the problem is in Linux kernel actively reading IN endpoint immediately after ttyACM open() call before application has a chance to set TTY parameters, i.e. disable ECHO. This seems really like a race condition and the solution would be to make Linux read IN endpoint only after user-space application issues read().

That is indeed the problem.  There is an inherit race that's been in there since before POSIX.  It's not possible to configure the tty parameters until after the device is opened.  But local processing of the tty input starts at open, e.g. ECHO, ISIG.
Not doing input processing until read would be a change to POSIX semantics.  I don't think that would ever be acceptable.  Consider the situation where Linux has a serial console.  There'd be no echo back on the console until a process reads from it.
The ESP-C3 has a dedicated USB CDC ACM serial/debug device (not an external chip as used on e.g. ESP32 and not a general purpose USB end-point peripheral that uses the zephyr usb end-point stack).  It has this same problem and obviously changes the cdc acm driver aren't going to fix it.
While this has always existed, it's hard to hit on traditional RS-232 ports for a couple reasons.
One is that RS-232 transmitters (not using flow-control) don't buffer output until the receiver is ready to receive it.  This means a character must be transmitted during the small window between when the process opens the tty and when it calls ioctl() to set the desired termios parameters.
On the other hand Zephyr's CDC ACM driver buffers output indefinitely until a receiver is ready to read it.  But only in interrupt mode, in polling mode it doesn't.  This is non-bug #53319
Because the zephyr ACM driver effectively has non-optional DTR/DSR hardware flow always enabled, and because USB always gets to transmit an URB when the port is opened, data will always arrive during the window between open() and ioctl().  So instead of narrow race that rarely happens it's always a problem.
Another reason this was less of problem with non-USB ports, is that termios parameters persist across open() calls.  So the very first open of /dev/ttyS0 might have echo enabled, all opens afterward will not and not have this race.  But the USB CDC tty device is removed and re-created on the host for every zephyr boot.  So it is constantly reset to the racy default of echo enabled each time.
IMHO, the right place to solve this is by fixing the POSIX race.  The no shell output until DTR feature doesn't work for anything outputting serial data that is not the shell.  And it doesn't work for other USB serial devices like the serial/debug peripheral in the ESP32-C3 or an external USB-RS232 chip.


Fresh --style=stylus install does not work / runtime error

Please provide us with the following information:

OS?  Windows 7, 8 or 10. Linux (which distribution). Mac OSX (Yosemite? El Capitan?)

os: darwin x64

Versions. Please run ng --version. If there's nothing outputted, please run
in a Terminal: node --version and paste the result here:

angular-cli: 1.0.0-beta.14
node: 6.1.0

Repro steps. Was this an app that wasn't created using the CLI? What change did you
do on your code? etc.

ng new notworkingapp --style=stylus

The log given by the failure. Normally this include a stack trace and some
more information.

It says that style.css needs to be an array of strings or something. Just run the install and open the app. Adding a style to styles.styl results in another error.

Mention any other details that might be useful.

One of the problems is that the application is generated with a app.component.stylus file and the webpack configuration is looking for .styl files. Renaming app.component.stylus to app.component.styl and changing the string in app.component.ts fixes the issue.

After re-reading the README, I believe this is a documentation error. In the CSS Preprocessor integration, it lists the three preprocessors supported by name (sass/scss, less, and stylus) and then gives an example of sass. This example works great if you substitute in the preprocessor names in the list above except stylus. The extension that angular-cli's expects for stylus files is styl. There are two ways this could be solved: either update the README to document the valid values for the --style argument or when the user passes --style=stylus, set styleExt in the new project to styl. These two options aren't mutually exclusive, either.

+1 Same issue on me


Release?

Does it make sense to make a release under the new ruby-jwe name?

'ruby-jwe' is just the github repo name. The actual gem name is specified in the jwe.gemspec file.
The same is done in the ruby-jwt gem.

Gotcha. I guess I wasn't paying attention. Thanks!


Implement /ivp/meters/readings as it's an official documented API by Enphase

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Add endpoint ivp/meters/readings to allow reporting of this data. The endpoint is documented by Enphase, see attached. Probably needs to implement /ivp/meters as well to grab the actual configured data from the readings
Additional context
Tech Brief-IQ Gateway Local API or UI Access using Token-EN-US-12-14-2022.pdf

to support future switch to HA Core one integration is using this naming scheme:
    SensorEntityDescription(
        key="import_index",
        name="Index Import",
        native_unit_of_measurement=UnitOfEnergy.WATT_HOUR,
        state_class=SensorStateClass.TOTAL_INCREASING,
        device_class=SensorDeviceClass.ENERGY,
    ),
    SensorEntityDescription(
        key="export_index",
        name="Index Export",
        native_unit_of_measurement=UnitOfEnergy.WATT_HOUR,
        state_class=SensorStateClass.TOTAL_INCREASING,
        device_class=SensorDeviceClass.ENERGY,
    ),

using
        ENDPOINT_URL_METERS = "http{}://{}/ivp/meters/readings"

        raw_json = self.endpoint_meters_json_results.json()

        index_imp = raw_json[1]["actEnergyDlvd"]

        index_exp = raw_json[1]['actEnergyRcvd']

To implement:

Add dataclass EnvoyMeterData in the meters model for ct meter data with from_api and from_phase methods.

@dataclass(slots=True)
class EnvoyMeterData:
    """Model for the Envoy's CT meter data."""

    eid: str  #: CT meter identifier
    timestamp: int  #: Time of measurement
    energy_delivered: int  #: Lifetime Energy delivered by CT
    energy_received: int  #: Lifetime Energy received by CT
    active_power: int  #: Current power measured, received when positive, delivered when negative
    power_factor: float  #: Power factor reported for CT measurement
    voltage: float  #: Voltage on circuit, when multi-phase sum of voltage of individual phases
    current: float  #: current measured by CT
    frequency: float  #: frequency measured on CT the circuit
    state: CtState | None  #: Actual State of CT
    measurement_type: CtType | None  #: Measurement type configured for CT
    metering_status: CtMeterStatus | None  #: CT Measurement status
    status_flags: list[CtStatusFlags] | None  #: CT status flags.

Add production and consumption CT attributes to Envoy.Data

    ctmeter_production: EnvoyMeterData | None = None  #: Production CT Meter data
    ctmeter_consumption: EnvoyMeterData | None = None  #: Consumption CT Meter data
    ctmeter_production_phases: dict[
        str, EnvoyMeterData
    ] | None = None  #: Production CT Meter Individual phase data
    ctmeter_consumption_phases: dict[
        str, EnvoyMeterData
    ] | None = None  #: Consumption CT Meter Individual phase data

Add code to update stub in meters updater to report meters data for production and consumption CT and their phases [^1][^2] .

 for index in range(len(meters_readings)):
     if meter["eid"] == self.production_meter_eid and self.production_meter_type:

        # if production meter was enabled (type known) store ctmeter production data
        envoy_data.ctmeter_production = EnvoyMeterData.from_api(
            meter, meters_status[index]
        )

        # if more then 1 phase configured store ctmeter phase data
        for phase in range(self.phase_count if self.phase_count > 1 else 0):
            production: EnvoyMeterData | None = EnvoyMeterData.from_phase(
                meter, meters_status[index], phase
            )

    if ( meter["eid"] == self.consumption_meter_eid and self.consumption_meter_type:

        # if consumption meter was enabled (type known) store ctmeter consumption data
        envoy_data.ctmeter_consumption = EnvoyMeterData.from_api(
            meter, meters_status[index]
        )

        # if more then 1 phase configured store ctmeter phase data
        for phase in range(self.phase_count if self.phase_count > 1 else 0):
            consumption: EnvoyMeterData | None = EnvoyMeterData.from_phase(
                meter, meters_status[index], phase
            )

Update tests
Update Documentation [^3]

[^1]: example only showing core steps
[^2]: pending merging #126
[^3]: pending merging #129

This is delivered in 1.18.0.


STRIP_TOOLS not working

The piece of code to strip the .so is not working as expected.
#!/bin/bash

STRIP_PATH="/NDK/toolchains/llvm/prebuilt/linux-x86_64/bin"
STRIP_TOOLS=(
    ['armeabi-v7a'] = "$STRIP_PATH/arm-linux-androideabi-strip.exe"
    ['arm64-v8a']   = "$STRIP_PATH/aarch64-linux-android-strip.exe"
    ['x86']         = "$STRIP_PATH/x86_64-linux-android-strip.exe"
    ['x86_64']      = "$STRIP_PATH/x86_64-linux-android-strip.exe"
)

ABI=('armeabi-v7a' 'arm64-v8a' 'x86' 'x86_64')

for abi in "${ABI[@]}"
do

  echo "=> Compiling Dlib for ABI: '$abi'..."
  $STRIP_TOOLS[$abi] --strip-unneeded dlib/libdlib.so

done

Output
=> Compiling Dlib for ABI: 'armeabi-v7a'...
./x.sh: line 17: [armeabi-v7a][armeabi-v7a]: command not found
=> Compiling Dlib for ABI: 'arm64-v8a'...
./x.sh: line 17: [armeabi-v7a][arm64-v8a]: command not found
=> Compiling Dlib for ABI: 'x86'...
./x.sh: line 17: [armeabi-v7a][x86]: command not found
=> Compiling Dlib for ABI: 'x86_64'...
./x.sh: line 17: [armeabi-v7a][x86_64]: command not found

To get it working change to:
# Declare the array
declare -A STRIP_TOOLS

STRIP_TOOLS=(
        ['armeabi-v7a']=$STRIP_PATH/arm-linux-androideabi-strip
        ['arm64-v8a']=$STRIP_PATH/aarch64-linux-android-strip
        ['x86']=$STRIP_PATH/x86_64-linux-android-strip
        ['x86_64']=$STRIP_PATH/x86_64-linux-android-strip
)

and
${STRIP_TOOLS[$abi]} --strip-unneeded dlib/libdlib.so

Best Regards.

https://github.com/Luca96/dlib-for-android/pull/4


Ml engine rework

feature: grpc based python ml engine
todo: implement ML engine grpc client for sentryflow
todo: add sqlite metrics

todo: add ai-engine build CI as well
todo: add pylint for CI as well


immediate selectorGraph() change logging

Currently, changes in state will not immediately be reflected in reselect-tools unless selectorGraph() is fired (i.e. on clicking "refresh" in the extension). I've overcome this by adding some redux middleware:
const reselectTools = store => next => action =>{

  const nodes = selectorGraph().nodes

  let result = next(action)

  if (typeof action === "function") return result

  const nextNodes = selectorGraph().nodes

  const nnKeys = Object.keys(nextNodes) // ["data$", "ui$"]

  // log if action changed any selectors
  nnKeys.forEach(k=>{
    if (nextNodes[k].recomputations!= nodes[k].recomputations) console.log({
      ...checkSelector(k), action
    })

  })

  return result

}

Now I can at least search in the console for a selector and see the action that triggered it whenever it's recomputed. It'd be great to combine this with the extension search filter.
It seems like this should be the default, or maybe there is a better way to do this? Also, any way to remotely update the extension?

Hey Matt - your solution looks good to me! Technically this library does not depend on redux, so I'd be reluctant to add any kind of redux-specific middleware. However, this looks like a simple recipe for getting the desired behavior.
I'm not sure I understand your question about 'remotely updating the extension'. Could you clarify?


Reproducing WMT14

I want to reproduce and then adjust the data of the WMT2014 Benchmark. Therefore I cant use helm directly (if I understand it correctly). Therefore i want to use the dataset and the way how the prompt is constructed in my own pipeline. (same applies to the evaluation code)
Unfortunately I cant understand and find my way through the code.
I believe the PromptRunExpander is responsible for creating the prompt but i cant figure out what it exactly does and what values are used and where they are obtained from.
I would happy for any explenation on how the flow of information is and how the final prompt is constructed

Hi @MaxHahnbueck, here's some pointers to the code:
The main flow for the prompt generation happens in Runner.run_one here. Specifically, this calls WMT14Scenario.get_instances (link) to download and load instances into memory, and it calls GenerationAdapter.generate_requests (link) to turn them into prompt strings, which are placed in RequestState.request.input.text.
You may also want to check out the documentation if you haven't already. Hope this helps.


Malicious contract v3 - FP

attacker - 0x947b05e9ac793ad8acb326f965b9641529b818b6
AD alert - https://app.forta.network/alerts/0xe450f3055c1249685a12ecc31926728139edd92055341081d44dfa7f717ba934
Base bots - malicious-smart-contract, suspicious-contract-creation and Asset Drained
potential cause -  the contract is non-malicious and verified.

Early-AD does not trigger with this transaction


Make ReturnHttpResponse action a free function

Instead of ReturnHttpResponse being a static member function of a
NetworkMock class make it a free function.
Relates-to: OLPEDGE-768
Signed-off-by: Serhii Lysenko<EMAIL_ADDRESS>

Do we really need to clutter global namespace? Having ReturnHttpResponse as a static member of NetworkMock syntactically ties mocks of the class and its response.


add mkdocs documentation skeleton

Preview: https://jothepro.github.io/djinni-documentation/djinni-support-lib/install/
This is part of the global documentation. The resulting website will look like this: https://jothepro.github.io/djinni-documentation
djinni-documentation will include djinni-support-lib as a submodule. All support-lib related information can be documented here inside the docs folder.
See https://github.com/cross-language-cpp/djinni-documentation/pull/3 for more details about the new documentation structure.

The only question I have is, why is the mkdocs.yml in this (sub) project so much more than in other (sub) projects ?

Umph, that's the wrong yaml. Thx!

Thanks for the fast update!


Tweak proguard rules and add test

[Current coverage][cc-pull] is 95.48%

Merging [#52][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master        #52   diff @@
==========================================
  Files            15         15
  Lines           620        620
  Methods           0          0
  Messages          0          0
  Branches        129        129
==========================================
  Hits            592        592
  Misses           13         13
  Partials         15         15

[][cc-pull]

Powered by Codecov. Last updated by 097d496
[cc-base-branch]: https://codecov.io/gh/vanniktech/OnActivityResult/branch/master?src=pr
[cc-pull]: https://codecov.io/gh/vanniktech/OnActivityResult/pull/52?src=pr


Remove multimapping reads

It's generally considered best practice to remove multimapping reads for ChIP-seq analysis (eg. the classic bowtie -m 1 for bowtie1). Most modern mappers, including bwa mem that we use here, don't have an equivalent option. However, samtools view can be used to filter out reads with low mapping scores. See biostars posts herez and here (and many other places I'm sure).
It would be good to add in an extra filter step like this, possibly making it optional.

Removing secondary alignments is not removing multimapping reads - their primary alignments are still retained in the bam file (re link #1). I only use bwa very occasionally, but this is a filtering strategy I would go for. Does the solution have to use samtools? I like ngsutils for filtering, and there is a module available now at Uppmax.
bamutils filter $filename $file_uniq -failed $failed_reads -eq XT:A U
I used this command on output of bwa aln. I am not sure if bwa mem outputs the XT:A flags though, the link above suggests that it does.
One can add additional filter for MAPQ if desired - not sure how much that will change, though.

As of yesterday, the pipeline doesn't use UPPMAX environment modules any more - instead it uses singularity containers ported from a docker container. This makes the pipeline much more portable and reproducible 🎉
So yes, we could certainly add ngstools if we need it. But it'll make this singularity image larger, so if we can do it with samtools then it's probably better to stick with what we already have.
Phil


Consider using git for the undo stack

Currently, cluster changes are saved in a Python list: (spike_ids, cluster_ids). Undoing = replaying the entire past sequence except the last item. This is lost when closing the session.
Another option would be to save the entire spike_clusters array in a text file, with one line per spike, containing the cluster id for that spike (this is the old .clu file). This text file would be saved in a user temporary directory, and it would be version-controlled with git. It may be part of the cluster store. The user would never have to see this.

Location: ~/.phy/my_file/clustering/0/main/spike_clusters.txt and cluster_groups.txt (0 is the shank)
Every clustering change (merge, split, etc.) = 1 commit
Undo = go back in the branch
How to deal with branching? Example: merge, merge, undo, split. How does git deal with that?
cluster_groups can contain 3 noise or 7 good on each line, with the cluster_id and group name on every line.
This is saved into the kwik file only when closing the dataset (or when manually clicking on the "Save" button?). Saving = copying the contents of these files into the kwik file.

Pros

Never loose your data again. Every action is saved and reasonably safe from data corruption. If the computer crashes, everything has been automatically saved.
Performance?
Unlimited undo stack, even across multiple sessions.
Does not take any RAM (but does take disk space).

Cons

Hard dependency on git, harder to distribute, Windows users, etc.

Ping @nippoo @ctaf42

What does git brings, that simple texts files doesn't if we are speaking of temporary files not shared?
IE:
do we need compression? we may. but its very to do file by file too.
do we need to exchange the history with others or share it accros boxes?
do we need to merge histories together?

History!

flat history is easy with files:
h.1
h.2
h.3
tree is history is a little bit harder, but still very easy to do =>
h.hash1
h.hash2
h.hash3
h.index
h.index define the tree that refer to all temporary files.
Very simple and straightforward to implement, without the complexity of git, nor the dependency.

That's a possibility indeed. But the point with using git was that it was already implemented. If we have to implement it ourselves (even if it's simple) we may as well keep things as they are for now.

I'am pretty sure, the implementation for what I propose is shorter than the one using git, and more maintainable. It's a matter of using pickle.
you dont need the feature of git at all.  to me using git here feels like using a bazouka to kill a moskito.

Yeah you may be right. Instead of pickle I would probably use text files though, unless the perf is too bad.

was speaking of pickle in textfile (unless perf are too bad).
for example :
pickle.dump(mynode, open( "file.hash", "wb" ) )
pickle.dump(mytree, open( "tree", "wb" ) )

YAGNI


Add install step to start docker daemon

https://github.com/docker/docker/issues/12427

Can you please sign your commits following these rules:
https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "12427-add-install-step" git@github.com:gabesullice/docker.git somewhere
$ cd somewhere
$ git commit --amend -s --no-edit
$ git push -f

This will update the existing PR, so you do not need to open a new one.

It's convention in debian based distributions for services to start automatically upon package installation. Thus manual start should not be necessary.

@gabesullice I tested the installation when I rewrote this page. A daemon start wasn't required as per @phemmer .  Are you finding that is when you test? What are the steps you are following? I'd like to duplicate it.

Yeah, I'm going to retest this again tonight too, but I did have to start
the docker daemon by hand when I filed the issue. Additional context: I was
installing on Ubuntu 13.10.
On Thu, Apr 16, 2015 at 9:59 AM, moxiegirl<EMAIL_ADDRESS>wrote:

@gabesullice https://github.com/gabesullice I tested the installation
when I rewrote this page. A daemon start wasn't required as per @phemmer
https://github.com/phemmer . Are you finding that is when you test?
What are the steps you are following? I'd like to duplicate it.
—
Reply to this email directly or view it on GitHub
https://github.com/docker/docker/pull/12429#issuecomment-93771245.

@gabesullice I believe ya had to start by hand. Thanks for doing the further testing.  I'm gonna test on my end.  That's a pretty popular page but 13.10 not sure how that stacks up so you may have run into something.  I retested on 13.10. Here is my transcript.
https://gist.github.com/moxiegirl/9fb8f0f846afca92ed3f

Closing as @gabesullice's testing proved this unable to dup.


Add Clob support

OK. I will see that tomorrow.

For information, i have just added a new page in the documentation to list the releases with the contributing on these releases : http://joel-costigliola.github.io/assertj/assertj-db-news.html

I start to see the pull request but i am not sure to finish tonight.

Hello @regis1512 thanks for the mention on the page but my name is wrong there. The correct is Otoniel Isidoro. Thx.

I forgot some tests to other mapped SQL types on AbstractDbData class. When I have some time I'll do this.

I fix the name (sorry usually i copy/paste and it is better ;-)).
For the tests, wait because i saw you added many types.
But i prefer one type per pull request : it will be easier to validate.
After i am not yet sure but i think you fork before last pull request validation because the comparison is strange. I will see that this evening (in France it is 08:15am and i need to go to work).


chore(main): release 1.0.5

:robot: I have created a release beep boop
1.0.5 (2022-05-30)
Bug Fixes

tmp (7d96491)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/SecretHouseGame/ngx-design-system/releases/tag/v1.0.5 :sunflower:


Add Cargo.lock

To allow Dependabot to operate correctly.

Waiting to merge this until I see if the Dependabot kickoff generates any PRs on its own.


[v4 JS] ion-ripple effect broken

Bug Report
Ionic Info
Directly using @ionic/core version 4.0.0-beta.1 from CDN (e.g. unpkg or jsdelivr)
Tried on Chrome and Firefox.
Describe the Bug
In MD mode, all ion elements have lost their ripple effect (<ion-button>, <ion-item button>…).
In iOS mode, buttons have a hover effect, but they no longer have their subtle "taint" effect.
Steps to Reproduce
Steps to reproduce the behavior:

Go to https://beta.ionicframework.com/docs/api/ripple-effect/ (defaults in MD mode)
Click on any of the buttons of the device simulator.
No ripple effect on click.

Related Code

v4.0.0-beta.1 (MD mode): https://next.plnkr.co/edit/dXIqMtK1jX91BuLS (no visual effect at all)
v4.0.0-beta.1 (iOS mode): https://next.plnkr.co/edit/CkjufAerAIEYj7QH (no tainting)
v4.0.0-beta.0 (MD mode): https://next.plnkr.co/edit/ghWyz0OyaT2bKkSA (works in Chrome, slight tainting in FF)
v4.0.0-beta.0 (iOS mode): https://next.plnkr.co/edit/LuYKBQxtZ6myXQ7f (works)
v4.0.0-alpha.8 (MD mode): https://next.plnkr.co/edit/5efKvrZLaPjTOCiY (works in Chrome, slight tainting in FF)
v4.0.0-alpha.8 (iOS mode): https://next.plnkr.co/edit/aK27GGzQctBq4Saj (works)
v4.0.0-alpha.7 (MD mode): https://next.plnkr.co/edit/ZjmATa7Q5eKtci8f (works)
v4.0.0-alpha.7 (iOS mode): https://next.plnkr.co/edit/ZPsoaLpNEebkPQga (works)

Expected Behavior
On user click / tap, buttons should have their ripple effect in MD mode, and a "taint" effect in iOS mode.
Additional Context
N/A

This is fixed in master! Thanks for reporting the issue