Simple Authentication always fails

I am running kops on AWS. I can get the mongo  cluster to work without authentication, but as soon as I try to secure it with simple user/password the nodes fail to connect.
I am creating the user like
db.createUser( { user: "cesco", pwd: "mypassword", roles:[{role:"root", db:"admin"}] })
then deleting the statefulset and creating back again uncommenting the below commented lines
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongod
spec:
  serviceName: mongodb-service
  replicas: 3
  template:
    metadata:
      labels:
        role: mongo
        environment: development
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongod-container
          image: mongo:3.6
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--bind_ip"
            - <IP_ADDRESS>
            - "--smallfiles"
            - "--noprealloc"
            # - "--auth"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-persistent-storage
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo"
            - name: KUBERNETES_MONGO_SERVICE_NAME
              value: "mongodb-service"
            # - name: MONGODB_USERNAME
            #   value: "cesco"
            # - name: MONGODB_PASSWORD
            #   value: "mypassword"
            # - name: MONGODB_DATABASE
            #   value: "admin"
  volumeClaimTemplates:
  - metadata:
      name: mongo-persistent-storage
      annotations:
        volume.beta.kubernetes.io/storage-class: "fast"
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 4Gi

On the sidecar-log i get
Db.prototype.authenticate method will no longer be available in the next major release 3.x as MongoDB 3.6 will only allow auth against users in the admin db and will no longer allow multiple credentials on a socket. Please authenticate using MongoClient.connect with auth credentials.
Error in workloop { MongoError: Authentication failed.
    at Function.MongoError.create (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/error.js:31:11)
    at /opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/pool.js:497:72
    at authenticateStragglers (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/pool.js:443:16)
    at Connection.messageHandler (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/pool.js:477:5)
    at Socket.<anonymous> (/opt/cvallance/mongo-k8s-sidecar/node_modules/mongodb-core/lib/connection/connection.js:333:22)
    at Socket.emit (events.js:180:13)
    at addChunk (_stream_readable.js:269:12)
    at readableAddChunk (_stream_readable.js:256:11)
    at Socket.Readable.push (_stream_readable.js:213:10)
    at TCP.onread (net.js:578:20)
  name: 'MongoError',
  message: 'Authentication failed.',
  ok: 0,
  errmsg: 'Authentication failed.',
  code: 18,
  codeName: 'AuthenticationFailed',
  operationTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_:<PHONE_NUMBER> },
  '$clusterTime':
   { clusterTime: Timestamp { _bsontype: 'Timestamp', low_: 1, high_:<PHONE_NUMBER> },
     signature: { hash: [Binary], keyId: [Long] } } }

I can exec into the mongod-0 pod and run mongo -u cesco -p mypassword and log in just right.As well as running use admin db.auth("cesco","mypassord") I get 1
Logs show
2018-05-02T19:13:22.851+0000 I ACCESS   [conn12] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-2.mongodb-service.default.svc.cluster.local:27017", fromId: 2, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.039+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 817) to mongod-2.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.063+0000 I NETWORK  [listener] connection accepted from <IP_ADDRESS>:54200 #13 (11 connections now open)
2018-05-02T19:13:23.063+0000 I NETWORK  [conn13] received client metadata from <IP_ADDRESS>:54200 conn13: { driver: { name: "nodejs", version: "2.2.35" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.4.115-k8s" }, platform: "Node.js v9.8.0, LE, mongodb-core: 2.1.19" }
2018-05-02T19:13:23.064+0000 I ACCESS   [conn13] SCRAM-SHA-1 authentication failed for cesco on admin from client <IP_ADDRESS>:54200 ; UserNotFound: Could not find user cesco@admin
2018-05-02T19:13:23.122+0000 I REPL_HB  [replexec-2] Error in heartbeat (requestId: 819) to mongod-1.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.191+0000 I ACCESS   [conn6] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-1.mongodb-service.default.svc.cluster.local:27017", fromId: 1, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.352+0000 I ACCESS   [conn12] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-2.mongodb-service.default.svc.cluster.local:27017", fromId: 2, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.539+0000 I REPL_HB  [replexec-1] Error in heartbeat (requestId: 821) to mongod-2.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.623+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 823) to mongod-1.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.692+0000 I ACCESS   [conn6] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-1.mongodb-service.default.svc.cluster.local:27017", fromId: 1, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:23.852+0000 I ACCESS   [conn12] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-2.mongodb-service.default.svc.cluster.local:27017", fromId: 2, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.040+0000 I REPL_HB  [replexec-2] Error in heartbeat (requestId: 825) to mongod-2.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.124+0000 I REPL_HB  [replexec-1] Error in heartbeat (requestId: 827) to mongod-1.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.193+0000 I ACCESS   [conn6] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-1.mongodb-service.default.svc.cluster.local:27017", fromId: 1, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.353+0000 I ACCESS   [conn12] Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-2.mongodb-service.default.svc.cluster.local:27017", fromId: 2, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.541+0000 I REPL_HB  [replexec-0] Error in heartbeat (requestId: 829) to mongod-2.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }
2018-05-02T19:13:24.625+0000 I REPL_HB  [replexec-2] Error in heartbeat (requestId: 831) to mongod-1.mongodb-service.default.svc.cluster.local:27017, response status: Unauthorized: not authorized on admin to execute command { replSetHeartbeat: "rs0", configVersion: 5, hbv: 1, from: "mongod-0.mongodb-service.default.svc.cluster.local:27017", fromId: 0, term: 1, $replData: 1, $clusterTime: { clusterTime: Timestamp(1525288116, 1), signature: { hash: BinData(0, 259E9B3067A93809C76C1DC5B31CA15C0B2814DF), keyId:<PHONE_NUMBER>938735618 } }, $db: "admin" }

What I am doing wrong? I need to grant external access to this database, otherwise i would not care about authentication. Help would be very much appreciated

Strange thing I notice is that if I enable auth on the sidecar and not on mongod, the replicas are able to connect, but the logs get full of
Db.prototype.authenticate method will no longer be available in the next major release 3.x as MongoDB 3.6 will only allow auth against users in the admin db and will no longer allow multiple credentials on a socket. Please authenticate using MongoClient.connect with auth credentials.

As soon as I enable --auth flag on the mongod I replicas fail to connect to master

Finally got it to work with https://gist.github.com/thilinapiy/0c5abc2c0c28efe1bbe2165b0d8dc115
Only thing I change is the password and the replica count. I using 3 without any issues.
I just cannot realize why it does not work the way I was doing. Enabling with after setting a sido user seems to be the mongo way to do this 🤷♂🤷♂🤷♂

Strange thing I notice is that if I enable auth on the sidecar and not on mongod, the replicas are able to connect,

@cescoferraro  I meet the same problem, how do you enable auth on the sidecar?

Maybe it is caused by the bug https://jira.mongodb.org/browse/SERVER-31916, my MongoDB is 3.6.4.

The gist on my last comment works just fine. I cannot find the reason why

Replica sets and sharded clusters require internal authentication between members when access control is enabled. For more details, please see Internal Authentication.


🐛 Bug Report: Google based OIDC provider from scratch redirect to login page when I refresh the page after I logged in

📜 Description
I am redirecting to the login page on refresh, I am aware that there are several discussions regarding redirecting to the login page in this channel. But I am not sure if it is a new issue or an issue due to configuration.
I am facing a similar issue for my "OIDC provider from scratch" as others. I have followed the steps mentioned in https://backstage.io/docs/auth/oidc. But instead of azureOIDC, I am using a custom google provider (called "googlecustom") for testing. After the implementation, I can login to the portal, but when I refresh it, it redirects to the login page.
What I have observed is, I am not getting a refresh token after I logged in (googlecustom-refresh-token). So whenever I refresh the page I get 401 Unauthorized.
Request URL: http://localhost:7007/api/auth/googlecustom/refresh?optional&env=development
I have followed the following steps as mentioned by @Rugvip in GitHub.
https://github.com/backstage/backstage/pull/13695?#pullrequestreview-1123807546
👍 Expected behavior
Refresh page should not redirect to login page instead it has to stay on that page.
👎 Actual Behavior with Screenshots

{
"error": {
"name": "AuthenticationError",
"message": "Refresh failed; caused by InputError: Missing session cookie",
"cause": {
"name": "InputError",
"message": "Missing session cookie",
"stack": "InputError: Missing session cookie
at OAuthAdapter.refresh (...../backstage-google-auth-latest/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:446:15)
at OAuthEnvironmentHandler.refresh (...../backstage-google-auth-latest/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:206:58)
at handleReturn (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
at bound refresh (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:64:7)
at handleReturn (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
at bound refresh (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:64:7)
at Layer.handle [as handle_request] (...../backstage-google-auth-latest/node_modules/express/lib/router/layer.js:95:5)
at next (...../backstage-google-auth-latest/node_modules/express/lib/router/route.js:144:13)
at Route.dispatch (...../backstage-google-auth-latest/node_modules/express/lib/router/route.js:114:3)
at Layer.handle [as handle_request] (...../backstage-google-auth-latest/node_modules/express/lib/router/layer.js:95:5)"
},
"stack": "AuthenticationError: Refresh failed; caused by InputError: Missing session cookie
at OAuthAdapter.refresh (...../backstage-google-auth-latest/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:464:13)
at OAuthEnvironmentHandler.refresh (...../backstage-google-auth-latest/node_modules/@backstage/plugin-auth-backend/dist/index.cjs.js:206:58)
at handleReturn (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
at bound refresh (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:64:7)
at handleReturn (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:24:23)
at bound refresh (...../backstage-google-auth-latest/node_modules/express-promise-router/lib/express-promise-router.js:64:7)
at Layer.handle [as handle_request] (...../backstage-google-auth-latest/node_modules/express/lib/router/layer.js:95:5)
at next (...../backstage-google-auth-latest/node_modules/express/lib/router/route.js:144:13)
at Route.dispatch (...../backstage-google-auth-latest/node_modules/express/lib/router/route.js:114:3)
at Layer.handle [as handle_request] (...../backstage-google-auth-latest/node_modules/express/lib/router/layer.js:95:5)"
},
"request": {
"method": "GET",
"url": "/api/auth/googlecustom/refresh?optional&env=development"
},
"response": {
"statusCode": 401
}
}
👟 Reproduction steps
I have followed the section OIDC provider from scratch and instead of azureOIDC I have used google.
My implementation is similar to this:
https://github.com/backstage/backstage/pull/13695?#pullrequestreview-1123807546
📃 Provide the context for the Bug.
Page refresh is not working.
🖥 Your Environment
OS:   Darwin 21.6.0 - darwin/x64
node: v16.14.2
yarn: 1.22.18
cli:  0.22.0 (installed)
backstage:  1.9.0
👀 Have you spent some time to check if this bug has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

How come you need to implement a custom google provider, what's wrong with the existing one? 😁
Regarding the actual issue, verify that you're actually getting a refresh token back in the auth result. You might for example need to request offline scope or similar.

I am testing how we can use the OIDC provider instead of implementing a new one. So thought of implementing google one as you have shown in the https://github.com/backstage/backstage/pull/13695 to understand what's happening. I agree that issue is due to missing refresh token. Even after put access_type='offline' I am not getting refresh token. Tried via rest client too. I think I need to do bit research on google oauth.
Anyway thanks for the help ;)

May I ask if you have solved this problem, do you have any ideas?

I have the same problem with cognito. Want to use this provider with Kubernetes Plugin. But on every refresh I have to login again. With local running kubelogin I see cognito providing refresh tokens - but don't know how to make this work in backstage.


How to add LST low level calibration test resources?

I wonder about how to test code in ctapipe.
To test my code (low level calibration for LST) I need at least two file:

1 fits file ~ 30 MB
File with events ~ 2 GB.

Should I use ctapipe-extra? Aren't files too big?

Yes, the vent file is too big.
You should try to produce a smaller file with just a couple of events. Like 5 or 10.
Then you can add those to ctapipe-extra

I Think this issue is about LST measured data not simulations and there are several small prod3b files included for testing in ctapipe

LST low level calibration is no longer in scope of ctapipe, this  is handled in ctapipe_io_lst and cta-lstchain


Django templates intermittently throw failed key lookup exceptions

This issue has been moved from a ticket on Developer Community.

When rendering a template in Visual Studio's Django test server, I intermittently observe the following error:
Message=Failed lookup for key [addRowButton] in [...]
The line this issue triggers on looks like this (lightly modified to hide structure):
{% include '.../components/gen_formset_table_form.html' with form=next formset=formset addRowButton=addRowButton addSortable=addSortable %}
It is true that addRowButton is undefined in this template. However, only on some runs is this exception thrown, and having references to undefined keys should not throw exceptions in Django templates.
Once this issue is observed on a given run of the test server, it will consistently reappear every time this template is rendered. And, vice-versa, if the issue does not appear, it never appears. Furthermore, simply pressing the "restart" button on the toolbar is likely to fix the issue.
I also observe on most runs of the test server the following, possibly related issue:
----------------------------------------
Exception happened during processing of request from ('<IP_ADDRESS>', 59880)
[20/Feb/2020 20:57:16] "GET /ajax/ing/setup/json/ HTTP/1.1" 200 712
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\socketserver.py", line 650, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\socketserver.py", line 360, in finish_request
    self. RequestHandlerClass(request, client_address, self)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\socketserver.py", line 720, in __init__
    self.handle()
  File "[venvpath]\lib\site-packages\django\core\servers\basehttp.py", line 174, in handle
    self.handle_one_request()
  File "[venvpath]\lib\site-packages\django\core\servers\basehttp.py", line 182, in handle_one_request
    self.raw_requestline = self.rfile.readline(65537)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
----------------------------------------

Original Comments
Visual Studio Feedback System on 2/20/2020, 11:15 PM:
We have directed your feedback to the appropriate engineering team for further evaluation. The team will review the feedback and notify you about the next steps.
Visual Studio Feedback System on 3/26/2020, 06:52 PM:
I have detected that for the last 35 days, this issue didn't have much product team activity and a very small amount of new votes or comments. Based on this, its severity, and affected area, it’s my experience that this issue is very unlikely to be fixed.
Hugues Valois [MSFT] on 3/27/2020, 10:27 AM:
Thanks for the feedback.
We’ve just released Visual Studio 2019 16.5 and in which we’ve changed the “launch browser” action to wait until the console shows that web url, meaning it is ready to accept connections. What we had noticed in the past was that if it didn’t wait long enough, it could cause weird trouble in the connection leading to page errors.
There’s a small chance that it might help in your scenario, it would be worth trying when you get a chance to update VS.
If it doesn’t help, then I’d try to do those same things from command line rather than in VS and see if the issue still occurs.
thanks
Visual Studio Feedback System on 4/7/2020, 07:18 PM:
We will close this report in 14 days because we don’t have enough information to investigate further. To keep the problem open, please provide the requested details.
sirbayer on 4/12/2020, 00:43 PM:
Unfortunately, I can confirm that this hasn't resolved the issue; I'm still seeing the same exceptions after updating to 16.5.3.I've never observed these issues when running from the Ubuntu on Windows terminal. In fact, the exceptions generated by VS do not appear in the VS terminal, either - they only seem to exist in the VS debug popups.
Hugues Valois on 4/16/2020, 01:49 PM:
Thanks sirbayer.
I’ve set up a repro for this and I can see that the behavior has changed between VS 2017 and VS 2019.
In VS 2019, even though I have unchecked all Python exceptions in the Exception Settings tool window, it still breaks on that exception. It doesn’t cause any problems with the application, I press F5 to continue and some code must be handling that exception since things work fine, no error in terminal.
However, the debugger should not be breaking when the exception is thrown, if I have all of them disabled in settings, and there is code that handles the exception. In VS 2017, it did not break under the same circumstances.
I was worried this was broken everywhere but fortunately this seem to only be an issue when debugging django templates.
I’ll transfer this over to our bug repo on github. Thank you for filing this.

Original Solutions
(no solutions)

to repro, starting with the VS template for Django project:
in layout.html, add this in the
            {% include "app/copyright.html" with hello=foo %}

add new item, copyright.html to the app/templates/app folder

Copyright 2020

Just my Code setting doesn't seem to make any difference.
Disable all Python exceptions in the Exception Settings tool window.
Debug the application
Debugger breaks on this exception
If you do the same with VS 2017/ptvsd 4.3.2, it does not break on that exception.
  Message=Failed lookup for key [foo] in [{'True': True, 'False': False, 'None': None}, {'csrf_token': <SimpleLazyObject: <function csrf.<locals>._get_val at 0x0000023F7CE86CA8>>, 'request': <WSGIRequest: GET '/about/'>, 'user': <SimpleLazyObject: <django.contrib.auth.models.AnonymousUser object at 0x0000023F7D1E7FC8>>, 'perms': <django.contrib.auth.context_processors.PermWrapper object at 0x0000023F7D1D7F48>, 'messages': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x0000023F7CE32348>, 'DEFAULT_MESSAGE_LEVELS': {'DEBUG': 10, 'INFO': 20, 'SUCCESS': 25, 'WARNING': 30, 'ERROR': 40}}, {}, {'title': 'About', 'message': 'Your application description page.', 'year': 2020}]
  Source=C:\Users\huvalo\source\repos\DjangoWebProject4\app\templates\app\layout.html
  StackTrace:
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\app\templates\app\layout.html", line 40, in Django Template
    {% include "app/copyright.html" with hello=foo %}
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 850, in _resolve_lookup
    (bit, current))  # missing attribute
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 796, in resolve
    value = self._resolve_lookup(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 671, in resolve
    obj = self.var.resolve(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 904, in render_annotated
    return self.render(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 937, in render
    bit = node.render_annotated(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 163, in _render
    return self.nodelist.render(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 904, in render_annotated
    return self.render(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 937, in render
    bit = node.render_annotated(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 163, in _render
    return self.nodelist.render(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\env\Lib\site-packages\django\template\base.py", line 171, in render
    return self._render(context)
  File "C:\Users\huvalo\source\repos\DjangoWebProject4\app\views.py", line 43, in about
    'year':datetime.now().year,

This is the code throwing that exception:

And a few frames up the call stack, the code that catches it:

I tested with more versions, ptvsd 4.2.4 worked fine, ie it detects that it is a handled exception, and ptvsd 4.2.7 does not work as expected, it thinks it is unhandled.


Install pip for Python3.4 for gcov tests

Followup for https://github.com/grpc/grpc/pull/15312.
Gcov tests are now broken with
+ python3.4 -m pip install virtualenv
/usr/bin/python3.4: No module named pip
2018-05-09 09:14:18,696 FAILED: /var/local/git/grpc/tools/run_tests/helper_scripts/build_python.sh [ret=1, pid=2028, time=0.0sec]

https://source.cloud.google.com/results/invocations/51a10c7b-2652-4d33-8d16-dec1e844241e/targets/grpc%2Fcore%2Fmaster%2Flinux%2Fgrpc_coverage/log
I think we will need a round at cleaning up python docker images (I'm getting lost with which are used for what - e.g. are we currently using python_jessie_x64?).

****************************************************************

libgrpc.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

****************************************************************

libgrpc++.so

     VM SIZE        FILE SIZE
 ++++++++++++++  ++++++++++++++

  [ = ]       0        0  [ = ]

[trickle] No significant performance differences

[microbenchmarks] No significant performance differences


OAuth token expired event for IRC and TwitchAPI

Hey swiffty,
Is is possible to get an event when the irc login or an api call failed bcs the oauth token that is used is expired?
This would help alot!
Regards
Syzuna

Do you have an oauth that has expired or sample message that is received on expired oauth?
I don't have either so it's hard to test functionality for it.

twitch docs say:
When you make a request with expired or incorrect authorization credentials, the API returns a WWW-Authenticate header (with an invalid_token error) and a 401 Unauthorized status:
DELETE /kraken/users/test_user1/follows/channels/test_channel HTTP/1.1
Host: api.twitch.tv
User-Agent: curl/7.43.0
Accept: application/vnd.twitchtv.v3+json
Authorization: OAuth <access_token>
Client-ID: retgzhvpsxjwun0rvrb1rfwheegu1yw
HTTP/1.1 401 Unauthorized
Accept-Ranges: bytes
Access-Control-Allow-Headers:
Access-Control-Allow-Methods:
Access-Control-Allow-Origin: *
Age: 0
Cache-Control: max-age=0, private, must-revalidate
Content-Type: application/json; charset=utf-8
Date: Wed, 12 Oct 2016 22:25:10 GMT
Server: nginx
Status: 401 Unauthorized
Vary: Accept-Encoding
Via: 1.1 varnish
WWW-Authenticate: OAuth realm='TwitchTV', error='invalid_token'
Content-Length: 89
Connection: keep-alive
{
"error": "Unauthorized",
"message": "Token invalid or missing required scope",
"status": 401
}
Sadly I dont have any information how the IRC one looks

"On seeing a 401 error with a WWW-Authenticate header, an application should try to refresh the session if a refresh token is present. "

Ya, I do think it's a good idea to get some kind of event for the client when using an expired token, and an exception for the API. As for automatically refreshing the token, i don't think that's really possible without providing the twitch app secret + refresh token, which is nasty.

ye I dont need an auto refresh but sth to detect that I need to make a refresh call inside the lib would be nice

IRC only prints login failed into the console it seems

Since the referenced pull request is closed, just an update: This detection is completed for API, still needs to be done for TwitchClient.

This issue was moved to TwitchLib/TwitchLib.Client#41


[SFTTrainer] Fix non packed dataset

What does this PR do?
This PR properly educates users on how to correctly use formatting_func method when someone uses a non-packed dataset.
Since the dataset processing calls dataset.map(xxx, batched=True) under the hood, it is important to return an array of processed texts to properly process all texts from the dataset example batch, otherwise it will lead to silent bugs that are hard to understand such as the one described in https://github.com/lvwerra/trl/issues/439
from datasets import load_dataset
from trl import SFTTrainer
import transformers

dataset = load_dataset("tatsu-lab/alpaca", split="train")

model = transformers.AutoModelForCausalLM.from_pretrained("facebook/opt-350m")
tokenizer = transformers.AutoTokenizer.from_pretrained("facebook/opt-350m")

def formatting_prompts_func(examples):
    output_text = []
    for i in range(len(examples["instruction"])):
        instruction = examples["instruction"][i]
        input_text = examples["input"][i]
        response = examples["output"][i]

        if len(input_text) >= 2:
            text = f'''Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

            ### Instruction:
            {instruction}

            ### Input:
            {input_text}

            ### Response:
            {response}
            '''
        else:
            text = f'''Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

            ### Instruction:
            {instruction}

            ### Response:
            {response}
            '''
        output_text.append(text)

    return output_text

trainer = SFTTrainer(
    model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    formatting_func=formatting_prompts_func,
    max_seq_length=256,
    packing=False,
)

trainer.train()

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

Will this fine-tune the complete model end-to-end, or will this example fine-tune just a portion of it, like in Lora?

The above example will train the full model but there are also options to use LoRA.

Is this merged in main and then reverted? because padding is still False in _prepare_non_packed_dataloader in main (0.7.9)

Is this merged in main and then reverted? because padding is still False in _prepare_non_packed_dataloader in main (0.7.9)

Actually padding was turned off by this PR: https://github.com/huggingface/trl/pull/512
Now running SFT on alpaca gives
ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length. Perhaps your features (labelsin this case) have excessive nesting (inputs typelistwhere typeint is expected).

Any update on this issue?


Fail to use apko to install a package from 3rd party repository

I would like to install zulu-openjdk from their repository.
From their docker build file, I notice it requires to install their PGP keys
RUN wget --quiet<EMAIL_ADDRESS>-P /etc/apk/keys/ &&
RUN apk --repository https://repos.azul.com/zulu/alpine --no-cache add zulu17-ca-jdk
wondering how is this possible with apko tools.
I tried: 1, download the public key, 2, add -k "path_to_keys" in apko build command
like:
docker run -v "$PWD":/work cgr.dev/chainguard/apko build alpine-base.yaml apko-alpine:edge apko-alpine-new.tar   -k<EMAIL_ADDRESS>and this is my alpine-test.yaml
_contents:
repositories:
- https://dl-cdn.alpinelinux.org/alpine/edge/main/
- https://repos.azul.com/zulu/alpine
packages:
- alpine-baselayout
- zulu17-ca-jdk~=<IP_ADDRESS>-r1
entrypoint:
command: /bin/sh -l
_
However things does not work..
Also can someone nicely present the full document that I can use?
Nov  9 07:38:32.713 [INFO] loading config file: alpine-base.yaml
Nov  9 07:38:32.714 [INFO] [arch:x86_64] building image 'apko-alpine:edge'
Nov  9 07:38:32.714 [INFO] [arch:x86_64] build context:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   working directory: /tmp/apko-3356677548
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   tarball path:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   use proot: false
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   source date: 1970-01-01 00:00:00 +0000 UTC
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   Docker mediatypes: false
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   SBOM output path: /work
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   arch: x86_64
Nov  9 07:38:32.714 [INFO] [arch:x86_64] image configuration:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   contents:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main/ https://repos.azul.com/zulu/alpine]
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     keyring:      []
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     packages:     [alpine-baselayout zulu17-ca-jdk~=<IP_ADDRESS>-r1]
Nov  9 07:38:32.714 [INFO] [arch:x86_64]   entrypoint:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     type:
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     command:     /bin/sh -l
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     service: map[]
Nov  9 07:38:32.714 [INFO] [arch:x86_64]     shell fragment:
Nov  9 07:38:32.714 [INFO] [arch:x86_64] doing pre-flight checks
Nov  9 07:38:32.714 [INFO] [arch:x86_64] building image fileystem in /tmp/apko-3356677548
Nov  9 07:38:32.714 [INFO] [arch:x86_64] initializing apk database
Nov  9 07:38:32.714 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk add --initdb --arch x86_64 --root /tmp/apko-3356677548
Nov  9 07:38:32.729 [INFO] [arch:x86_64] initializing apk world
Nov  9 07:38:32.729 [INFO] [arch:x86_64] initializing apk keyring
Nov  9 07:38:32.729 [INFO] [arch:x86_64] initializing apk repositories
Nov  9 07:38:32.730 [INFO] [arch:x86_64] synchronizing with desired apk world
Nov  9 07:38:32.730 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk fix --root /tmp/apko-3356677548 --no-scripts --no-cache --update-cache --arch x86_64
Error: failed to build layer image: initializing apk: failed to fixate apk world: exit status 5
2022/11/09 07:38:36 error during command execution: failed to build layer image: initializing apk: failed to fixate apk world: exit status 5

i think you're missing keyring in your apko file. you can try building with --debug the error will be clearer i believe

there's also --keyring-append documented here https://github.com/chainguard-dev/apko/blob/71cdb2460aece396d00dd4ea47d7d7b5eab3a37d/NEWS.md#changes-from-012-to-020

hi, @tuananh
actually, I already add -k to append the additional keyring.
this works the same as --keyring-append
as in the source file: https://github.com/chainguard-dev/apko/blob/64613494cbfaad30b9eb4bf02124e6a51f1cbd6f/internal/cli/build.go
I turned on the debug message:
some observation:
1, it did install the addtinoal keys
2, however, during fetching, it fails, it does not gave too much information, complaining the package is not there.
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] appending 1 extra keys to keyring
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key<EMAIL_ADDRESS>Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key<EMAIL_ADDRESS>Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key<EMAIL_ADDRESS>Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key<EMAIL_ADDRESS>Nov 10 02:47:25.508 [INFO] [arch:x86_64] synchronizing with desired apk world
Nov 10 02:47:25.508 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk fix --root /tmp/apko-398846510 --no-scripts --no-cache --update-cache --arch x86_64
Nov 10 02:47:25.516 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
Nov 10 02:47:25.632 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] fetch https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
Nov 10 02:47:25.783 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] ERROR: unable to select packages:
Nov 10 02:47:25.799 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:]   java-common (no such package):
Nov 10 02:47:25.799 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:]     required by: zulu17-ca-jre-headless-17.0.5-r1[java-common]
What I did next is, I run up a alpine latest docker container,
run the following command:
wget --quiet<EMAIL_ADDRESS>-P /etc/apk/keys/
apk --repository https://repos.azul.com/zulu/alpine --no-cache add zulu17-ca-jdk
the logs shows that installation is fully successfully.
Guess:
1, does  the additional key not working?
2, or I need some specific yaml lines to force the key usage on the repo?
[root@coordinator apko-test]# docker run -it alpine
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob 213ec9aee27d done
Copying config 9c6f072447 done
Writing manifest to image destination
Storing signatures
/ # wget --quiet<EMAIL_ADDRESS>-P /etc/apk/keys/
/ # apk --repository https://repos.azul.com/zulu/alpine --no-cache add zulu17-ca-jdk
fetch https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
(1/14) Installing java-common (0.5-r0)
(2/14) Installing zulu17-ca-jre-headless (17.0.5-r1)
Executing zulu17-ca-jre-headless-17.0.5-r1.post-install
(3/14) Installing libxau (1.0.9-r0)
(4/14) Installing libxdmcp (1.1.3-r0)
(5/14) Installing libxcb (1.15-r0)
(6/14) Installing libx11 (1.8-r0)
(7/14) Installing libxext (1.3.4-r0)
(8/14) Installing libxi (1.8-r0)
(9/14) Installing libxrender (0.9.10-r3)
(10/14) Installing libxtst (1.2.3-r3)
(11/14) Installing alsa-lib (<IP_ADDRESS>-r0)
(12/14) Installing zulu17-ca-jre (17.0.5-r1)
(13/14) Installing zulu17-ca-jdk-headless (17.0.5-r1)
(14/14) Installing zulu17-ca-jdk (17.0.5-r1)
Executing busybox-1.35.0-r17.trigger
Executing java-common-0.5-r0.trigger
OK: 280 MiB in 28 packages

Your config pins to version <IP_ADDRESS>, while their repository only offers <IP_ADDRESS>.  You will need to update the version and then it will work.On Nov 9, 2022, at 8:59 PM, Xiaobo Jiang @.> wrote:
hi, @tuananh
actually, I already add -k to append the additional keyring.
this works the same as --keyring-append
as in the source file: https://github.com/chainguard-dev/apko/blob/64613494cbfaad30b9eb4bf02124e6a51f1cbd6f/internal/cli/build.go
I turned on the debug message:
some observation:
1, it did install the addtinoal keys
2, however, during fetching, it fails, it does not gave too much information, complaining the package is not there.
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] appending 1 extra keys to keyring
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key @.
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key @.***
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key @.***
Nov 10 02:47:25.277 [DEBUG] [arch:x86_64] installing key @.***
Nov 10 02:47:25.508 [INFO] [arch:x86_64] synchronizing with desired apk world
Nov 10 02:47:25.508 [INFO] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] running: /sbin/apk fix --root /tmp/apko-398846510 --no-scripts --no-cache --update-cache --arch x86_64
Nov 10 02:47:25.516 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
Nov 10 02:47:25.632 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] fetch https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
Nov 10 02:47:25.783 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:] ERROR: unable to select packages:
Nov 10 02:47:25.799 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:]   java-common (no such package):
Nov 10 02:47:25.799 [DEBUG] [arch:x86_64] [cmd:apk] [use-proot:false] [use-qemu:]     required by: zulu17-ca-jre-headless-17.0.5-r1[java-common]
What I did next is, I run up a alpine latest docker container,
run the following command:
wget --quiet @.*** -P /etc/apk/keys/
apk --repository https://repos.azul.com/zulu/alpine --no-cache add zulu17-ca-jdk
the logs shows that installation is fully successfully.
Guess:
1, does  the additional key not working?
2, or I need some specific yaml lines to force the key usage on the repo?
@.*** apko-test]# docker run -it alpine
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
Resolved "alpine" as an alias (/etc/containers/registries.conf.d/000-shortnames.conf)
Trying to pull docker.io/library/alpine:latest...
Getting image source signatures
Copying blob 213ec9aee27d done
Copying config 9c6f072447 done
Writing manifest to image destination
Storing signatures
/ # wget --quiet @.*** -P /etc/apk/keys/
/ # apk --repository https://repos.azul.com/zulu/alpine --no-cache add zulu17-ca-jdk
fetch https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz
(1/14) Installing java-common (0.5-r0)
(2/14) Installing zulu17-ca-jre-headless (17.0.5-r1)
Executing zulu17-ca-jre-headless-17.0.5-r1.post-install
(3/14) Installing libxau (1.0.9-r0)
(4/14) Installing libxdmcp (1.1.3-r0)
(5/14) Installing libxcb (1.15-r0)
(6/14) Installing libx11 (1.8-r0)
(7/14) Installing libxext (1.3.4-r0)
(8/14) Installing libxi (1.8-r0)
(9/14) Installing libxrender (0.9.10-r3)
(10/14) Installing libxtst (1.2.3-r3)
(11/14) Installing alsa-lib (<IP_ADDRESS>-r0)
(12/14) Installing zulu17-ca-jre (17.0.5-r1)
(13/14) Installing zulu17-ca-jdk-headless (17.0.5-r1)
(14/14) Installing zulu17-ca-jdk (17.0.5-r1)
Executing busybox-1.35.0-r17.trigger
Executing java-common-0.5-r0.trigger
OK: 280 MiB in 28 packages
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

@kaniini
I forget to update my latest yaml in the ticket.
I removed the version in the yaml file, trying to let APKO to download the latest version.
which will lead to use the latest version, which is 17.0.5
my updated yaml:
contents:
repositories:
- https://dl-cdn.alpinelinux.org/alpine/edge/main/
- https://repos.azul.com/zulu/alpine
packages:
- alpine-baselayout
- zulu17-ca-jdk
- tzdata
entrypoint:
command: /bin/sh -l

@kaniini can you help use my yaml and reproduce the issue? appreciate your help.

@xiaobojiang
contents:
  repositories:
    - https://dl-cdn.alpinelinux.org/alpine/edge/main
    - https://dl-cdn.alpinelinux.org/alpine/edge/community
    - https://repos.azul.com/zulu/alpine
  packages:
    - alpine-baselayout
    - zulu17-jre-headless~=<IP_ADDRESS>
    - tzdata

entrypoint:
  command: /bin/sh -l

environment:
  LANG: en_US.UTF-8
  LANGUAGE: en_US:en
  LC_ALL: en_US.UTF-8
  TZ: Etc/UTC
  JAVA_HOME: /usr/lib/jvm/zulu17

archs:
  - arm64
  - amd64

Builds successfully with latest version of apko.

@farcop  thanks for updating with solution.
It is making some differences.

My apko version:
_      ____    _  __   ___
/ \    |  _ \  | |/ /  / _
/ _ \   | |) | | ' /  | | | |
/ ___ \  |  __/  | . \  | || |
//   _\ ||     |_|_\  ___/
apko
GitVersion:    v0.10.0
GitCommit:     764deeccdb363b11f02231a3fe457dc74fb2c644
GitTreeState:  clean
BuildDate:     '2023-09-07T16:03:16Z'
GoVersion:     go1.21.1
Compiler:      gc
Platform:      linux/amd64

@xiaobojiang Sorry. My bad. Please use symlink instead hardlink.
contents:
  keyring:
    -<EMAIL_ADDRESS>  repositories:
    - https://dl-cdn.alpinelinux.org/alpine/edge/main
    - https://dl-cdn.alpinelinux.org/alpine/edge/community
    - https://repos.azul.com/zulu/alpine
  packages:
    - alpine-baselayout
    - zulu17-jre-headless~=<IP_ADDRESS>
    - tzdata

entrypoint:
  command: /bin/sh -l

environment:
  LANG: en_US.UTF-8
  LANGUAGE: en_US:en
  LC_ALL: en_US.UTF-8
  TZ: Etc/UTC
  JAVA_HOME: /usr/lib/jvm/zulu17-ca

paths:
  - path: /usr/lib/jvm/default-jvm
    type: symlink
    source: /usr/lib/jvm/zulu17-ca
    permissions: 0o777

archs:
  - arm64
  - amd64

Then it builds successfull.
% apko build zulu.yaml apko-alpine:edge apko-alpine-zulujdk.tar
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | Building images for 2 architectures: [arm64 amd64]
ℹ  aarch64   | building tags [apko-alpine:edge]
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | doing pre-flight checks
ℹ  aarch64   | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64
ℹ  aarch64   | initializing apk database
2023/10/07 14:19:15 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:19:16 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
2023/10/07 14:19:16 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
ℹ  aarch64   | finished initializing apk database
ℹ  aarch64   | setting apk world
ℹ  aarch64   | initializing apk keyring
2023/10/07 14:19:16 [DEBUG] GET<EMAIL_ADDRESS>ℹ  aarch64   | setting apk repositories
ℹ  aarch64   | build context:
ℹ  aarch64   | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "arm64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  aarch64   | image configuration:
ℹ  aarch64   |   contents:
ℹ  aarch64   |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  aarch64   |     keyring<EMAIL_ADDRESS>ℹ  aarch64   |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  aarch64   |   entrypoint:
ℹ  aarch64   |     type:
ℹ  aarch64   |     command:     /bin/sh -l
ℹ  aarch64   |     service: map[]
ℹ  aarch64   |     shell fragment:
ℹ  aarch64   | synchronizing with desired apk world
ℹ  aarch64   | determining desired apk world
ℹ            | loading config file: zulu.yaml
2023/10/07 14:19:16 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | doing pre-flight checks
ℹ  x86_64    | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/x86_64
ℹ  x86_64    | initializing apk database
2023/10/07 14:19:16 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:19:17 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
2023/10/07 14:19:17 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
ℹ  x86_64    | finished initializing apk database
ℹ  x86_64    | setting apk world
ℹ  x86_64    | initializing apk keyring
2023/10/07 14:19:17 [DEBUG] GET<EMAIL_ADDRESS>ℹ  x86_64    | setting apk repositories
2023/10/07 14:19:17 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | build context:
ℹ  x86_64    | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/x86_64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "amd64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  x86_64    | image configuration:
ℹ  x86_64    |   contents:
ℹ  x86_64    |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  x86_64    |     keyring<EMAIL_ADDRESS>ℹ  x86_64    |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  x86_64    |   entrypoint:
ℹ  x86_64    |     type:
ℹ  x86_64    |     command:     /bin/sh -l
ℹ  x86_64    |     service: map[]
ℹ  x86_64    |     shell fragment:
ℹ  x86_64    | synchronizing with desired apk world
ℹ  x86_64    | determining desired apk world
2023/10/07 14:19:18 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/aarch64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] GET https://repos.azul.com/zulu/alpine/aarch64/APKINDEX.tar.gz
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/java-common-0.5-r0.apk
2023/10/07 14:19:18 [DEBUG] GET https://repos.azul.com/zulu/alpine/aarch64/zulu17-ca-jre-headless-<IP_ADDRESS>-r3.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/musl-1.2.4_git20230717-r3.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/tzdata-2023c-r1.apk
2023/10/07 14:19:18 [DEBUG] GET https://repos.azul.com/zulu/alpine/aarch64/zulu17-jre-headless-<IP_ADDRESS>-r3.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/busybox-1.36.1-r11.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/busybox-binsh-1.36.1-r11.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/alpine-baselayout-3.4.3-r2.apk
2023/10/07 14:19:18 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/alpine-baselayout-data-3.4.3-r2.apk
2023/10/07 14:19:19 [DEBUG] GET https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
2023/10/07 14:19:19 [DEBUG] GET https://repos.azul.com/zulu/alpine/x86_64/zulu17-jre-headless-<IP_ADDRESS>-r3.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/busybox-binsh-1.36.1-r11.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/alpine-baselayout-data-3.4.3-r2.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/musl-1.2.4_git20230717-r3.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/alpine-baselayout-3.4.3-r2.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/java-common-0.5-r0.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/busybox-1.36.1-r11.apk
2023/10/07 14:19:19 [DEBUG] GET https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/tzdata-2023c-r1.apk
2023/10/07 14:19:19 [DEBUG] GET https://repos.azul.com/zulu/alpine/x86_64/zulu17-ca-jre-headless-<IP_ADDRESS>-r3.apk
Error: failed to build layer image for "arm64": failed to mutate paths: link /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64/usr/lib/jvm/zulu17-ca /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64/usr/lib/jvm/default-jvm: operation not permitted
2023/10/07 14:19:36 error during command execution: failed to build layer image for "arm64": failed to mutate paths: link /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64/usr/lib/jvm/zulu17-ca /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-2203190632/aarch64/usr/lib/jvm/default-jvm: operation not permitted
farcop@MacBook-Pro Sources % vim zulu.yaml
farcop@MacBook-Pro Sources % apko build zulu.yaml apko-alpine:edge apko-alpine-zulujdk.tar
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | Building images for 2 architectures: [arm64 amd64]
ℹ  aarch64   | building tags [apko-alpine:edge]
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | doing pre-flight checks
ℹ  aarch64   | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64
ℹ  aarch64   | initializing apk database
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
ℹ  aarch64   | finished initializing apk database
ℹ  aarch64   | initializing apk keyring
ℹ  aarch64   | setting apk world
ℹ  aarch64   | setting apk repositories
2023/10/07 14:20:07 [DEBUG] GET<EMAIL_ADDRESS>ℹ  aarch64   | build context:
ℹ  aarch64   | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "arm64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  aarch64   | image configuration:
ℹ  aarch64   |   contents:
ℹ  aarch64   |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  aarch64   |     keyring<EMAIL_ADDRESS>ℹ  aarch64   |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  aarch64   |   entrypoint:
ℹ  aarch64   |     type:
ℹ  aarch64   |     command:     /bin/sh -l
ℹ  aarch64   |     service: map[]
ℹ  aarch64   |     shell fragment:
ℹ  aarch64   | synchronizing with desired apk world
ℹ  aarch64   | determining desired apk world
ℹ            | loading config file: zulu.yaml
2023/10/07 14:20:07 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | doing pre-flight checks
ℹ  x86_64    | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64
ℹ  x86_64    | initializing apk database
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
2023/10/07 14:20:07 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/APKINDEX.tar.gz
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
ℹ  x86_64    | finished initializing apk database
ℹ  x86_64    | setting apk world
ℹ  x86_64    | initializing apk keyring
ℹ  x86_64    | setting apk repositories
2023/10/07 14:20:07 [DEBUG] GET<EMAIL_ADDRESS>2023/10/07 14:20:07 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | build context:
ℹ  x86_64    | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "amd64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  x86_64    | image configuration:
ℹ  x86_64    |   contents:
ℹ  x86_64    |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  x86_64    |     keyring<EMAIL_ADDRESS>ℹ  x86_64    |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  x86_64    |   entrypoint:
ℹ  x86_64    |     type:
ℹ  x86_64    |     command:     /bin/sh -l
ℹ  x86_64    |     service: map[]
ℹ  x86_64    |     shell fragment:
ℹ  x86_64    | synchronizing with desired apk world
ℹ  x86_64    | determining desired apk world
2023/10/07 14:20:08 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
2023/10/07 14:20:08 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
2023/10/07 14:20:08 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
⚠  aarch64   | did not find /etc/os-release at etc/os-release
⚠  aarch64   | distro ID not specified and /etc/os-release does not already exist
ℹ  aarch64   | generating supervision tree
ℹ  aarch64   | finished building filesystem in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64
ℹ  aarch64   | built image layer tarball as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-2107292138/apko-aarch64.tar.gz
ℹ  aarch64   | building image from layer
ℹ  aarch64   | OCI layer digest: sha256:02714be5799e64b38211c7de0515ac031b3d6d982c9281408ccb26e8ac847574
ℹ  aarch64   | OCI layer diffID: sha256:7d5cc9351813bde9038d43232c9d11f1f7717bb7a8fee0c54709164f351a0c25
⚠  x86_64    | did not find /etc/os-release at etc/os-release
⚠  x86_64    | distro ID not specified and /etc/os-release does not already exist
ℹ  x86_64    | generating supervision tree
ℹ  x86_64    | finished building filesystem in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64
ℹ  x86_64    | built image layer tarball as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-1366844051/apko-x86_64.tar.gz
ℹ  x86_64    | building image from layer
ℹ  x86_64    | OCI layer digest: sha256:a2a625eef12f3a2a792f5e6a7dadebd92047cac60ed25b0da81adbd6ca8d815a
ℹ  x86_64    | OCI layer diffID: sha256:34294cfe292d735f7bad09334bcd91682ac5cc1064f41dbafe633be653c81dbc
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | doing pre-flight checks
ℹ  aarch64   | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907
ℹ  aarch64   | initializing apk database
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
ℹ  aarch64   | finished initializing apk database
ℹ  aarch64   | setting apk world
ℹ  aarch64   | initializing apk keyring
ℹ  aarch64   | setting apk repositories
2023/10/07 14:20:10 [DEBUG] GET<EMAIL_ADDRESS>ℹ  aarch64   | built index file as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-100493398/index.json
INFO[0003] Generating arch image SBOMs
ℹ  x86_64    | Generating image SBOM for amd64
ℹ  aarch64   | Generating image SBOM for arm64
ℹ  aarch64   | Generating index SBOM
INFO[0003] Final index tgz at: apko-alpine-zulujdk.tar

@xiaobojiang Sorry. My bad. Please use symlink instead hardlink.
contents:
  keyring:
    -<EMAIL_ADDRESS>  repositories:
    - https://dl-cdn.alpinelinux.org/alpine/edge/main
    - https://dl-cdn.alpinelinux.org/alpine/edge/community
    - https://repos.azul.com/zulu/alpine
  packages:
    - alpine-baselayout
    - zulu17-jre-headless~=<IP_ADDRESS>
    - tzdata

entrypoint:
  command: /bin/sh -l

environment:
  LANG: en_US.UTF-8
  LANGUAGE: en_US:en
  LC_ALL: en_US.UTF-8
  TZ: Etc/UTC
  JAVA_HOME: /usr/lib/jvm/zulu17-ca

paths:
  - path: /usr/lib/jvm/default-jvm
    type: symlink
    source: /usr/lib/jvm/zulu17-ca
    permissions: 0o777

archs:
  - arm64
  - amd64

Then it builds successfull.
% apko build zulu.yaml apko-alpine:edge apko-alpine-zulujdk.tar
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | Building images for 2 architectures: [arm64 amd64]
ℹ  aarch64   | building tags [apko-alpine:edge]
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | doing pre-flight checks
ℹ  aarch64   | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64
ℹ  aarch64   | initializing apk database
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
ℹ  aarch64   | finished initializing apk database
ℹ  aarch64   | initializing apk keyring
ℹ  aarch64   | setting apk world
ℹ  aarch64   | setting apk repositories
2023/10/07 14:20:07 [DEBUG] GET<EMAIL_ADDRESS>ℹ  aarch64   | build context:
ℹ  aarch64   | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "arm64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  aarch64   | image configuration:
ℹ  aarch64   |   contents:
ℹ  aarch64   |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  aarch64   |     keyring<EMAIL_ADDRESS>ℹ  aarch64   |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  aarch64   |   entrypoint:
ℹ  aarch64   |     type:
ℹ  aarch64   |     command:     /bin/sh -l
ℹ  aarch64   |     service: map[]
ℹ  aarch64   |     shell fragment:
ℹ  aarch64   | synchronizing with desired apk world
ℹ  aarch64   | determining desired apk world
ℹ            | loading config file: zulu.yaml
2023/10/07 14:20:07 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | doing pre-flight checks
ℹ  x86_64    | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64
ℹ  x86_64    | initializing apk database
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
2023/10/07 14:20:07 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/aarch64/APKINDEX.tar.gz
2023/10/07 14:20:07 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-6165ee59.rsa.pub
ℹ  x86_64    | finished initializing apk database
ℹ  x86_64    | setting apk world
ℹ  x86_64    | initializing apk keyring
ℹ  x86_64    | setting apk repositories
2023/10/07 14:20:07 [DEBUG] GET<EMAIL_ADDRESS>2023/10/07 14:20:07 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/aarch64/APKINDEX.tar.gz
ℹ  x86_64    | build context:
ℹ  x86_64    | build options:
{
	"withVCS": true,
	"workDir": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64",
	"tags": [
		"apko-alpine:edge"
	],
	"sourceDateEpoch": "1970-01-01T00:00:00Z",
	"sbomPath": "/var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/image",
	"sbomFormats": [
		"spdx",
		"cyclonedx"
	],
	"arch": "amd64",
	"Log": {
		"Out": {},
		"Fields": null,
		"Level": 4
	}
}
ℹ  x86_64    | image configuration:
ℹ  x86_64    |   contents:
ℹ  x86_64    |     repositories: [https://dl-cdn.alpinelinux.org/alpine/edge/main https://dl-cdn.alpinelinux.org/alpine/edge/community https://repos.azul.com/zulu/alpine]
ℹ  x86_64    |     keyring<EMAIL_ADDRESS>ℹ  x86_64    |     packages:     [alpine-baselayout zulu17-jre-headless~=<IP_ADDRESS> tzdata]
ℹ  x86_64    |   entrypoint:
ℹ  x86_64    |     type:
ℹ  x86_64    |     command:     /bin/sh -l
ℹ  x86_64    |     service: map[]
ℹ  x86_64    |     shell fragment:
ℹ  x86_64    | synchronizing with desired apk world
ℹ  x86_64    | determining desired apk world
2023/10/07 14:20:08 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
2023/10/07 14:20:08 [DEBUG] HEAD https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
2023/10/07 14:20:08 [DEBUG] HEAD https://repos.azul.com/zulu/alpine/x86_64/APKINDEX.tar.gz
⚠  aarch64   | did not find /etc/os-release at etc/os-release
⚠  aarch64   | distro ID not specified and /etc/os-release does not already exist
ℹ  aarch64   | generating supervision tree
ℹ  aarch64   | finished building filesystem in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/aarch64
ℹ  aarch64   | built image layer tarball as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-2107292138/apko-aarch64.tar.gz
ℹ  aarch64   | building image from layer
ℹ  aarch64   | OCI layer digest: sha256:02714be5799e64b38211c7de0515ac031b3d6d982c9281408ccb26e8ac847574
ℹ  aarch64   | OCI layer diffID: sha256:7d5cc9351813bde9038d43232c9d11f1f7717bb7a8fee0c54709164f351a0c25
⚠  x86_64    | did not find /etc/os-release at etc/os-release
⚠  x86_64    | distro ID not specified and /etc/os-release does not already exist
ℹ  x86_64    | generating supervision tree
ℹ  x86_64    | finished building filesystem in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907/x86_64
ℹ  x86_64    | built image layer tarball as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-1366844051/apko-x86_64.tar.gz
ℹ  x86_64    | building image from layer
ℹ  x86_64    | OCI layer digest: sha256:a2a625eef12f3a2a792f5e6a7dadebd92047cac60ed25b0da81adbd6ca8d815a
ℹ  x86_64    | OCI layer diffID: sha256:34294cfe292d735f7bad09334bcd91682ac5cc1064f41dbafe633be653c81dbc
ℹ            | loading config file: zulu.yaml
ℹ  aarch64   | doing pre-flight checks
ℹ  aarch64   | building apk info in /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-133640907
ℹ  aarch64   | initializing apk database
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/releases.json
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
2023/10/07 14:20:10 [DEBUG] GET https://alpinelinux.org/keys/alpine-devel%40lists.alpinelinux.org-616ae350.rsa.pub
ℹ  aarch64   | finished initializing apk database
ℹ  aarch64   | setting apk world
ℹ  aarch64   | initializing apk keyring
ℹ  aarch64   | setting apk repositories
2023/10/07 14:20:10 [DEBUG] GET<EMAIL_ADDRESS>ℹ  aarch64   | built index file as /var/folders/bb/_k53lmmn03vcfj7vsgxxy4_w0000gn/T/apko-temp-100493398/index.json
INFO[0003] Generating arch image SBOMs
ℹ  x86_64    | Generating image SBOM for amd64
ℹ  aarch64   | Generating image SBOM for arm64
ℹ  aarch64   | Generating index SBOM
INFO[0003] Final index tgz at: apko-alpine-zulujdk.tar

It works! @farcop  thanks a lot!!!
I will close the ticket


Automated the manifest file

We having a redundancy between the chart file and the meilisearch.yaml file in Manifest. Which is not a bad thing. Although, it may be good to have the manifest file created automatically when a change is done in the charts, or at least compared, during a pipeline.
What do you guys think? We having now the possibility that we changing the charts, and forgetting about the manifest, and this one will not correspond to it any more.

Hi @HelmutLety
This seems to be a duplicate of https://github.com/meilisearch/meilisearch-kubernetes/issues/12
Totally agree with this, but would be better to discuss it there? :)

Oh yes! sorry didn't saw that :)


Fatal because of conflicting interfaces

Description
When symfony/console:2.8.45 is analysed, the dependency guard stops with a fatal error.
[30-Aug-2018 10:48:00 Europe/Berlin] PHP Fatal error:  Class Symfony\Component\Console\Style\SymfonyStyle contains 4 abstract methods and must therefore be declared abstract or implement the remaining methods (Symfony\Component\Console\Output\OutputInterface::isQuiet, Symfony\Component\Console\Output\OutputInterface::isVerbose, Symfony\Component\Console\Output\OutputInterface::isVeryVerbose, ...) in /builds/NotRelevant/vendor/symfony/console/Style/SymfonyStyle.php on line 33

These methods do not exist in the given interfaces in this version of symfony console.
Steps to Reproduce

create a project requiring symfony/console:2.8.45
composer install
run dependency-guard in project root

Expected behavior: continues, since there is no issue
Actual behavior: dies with a fatal error
Reproduces how often: 100%
Versions
mediact/dependency-guard: 1.0.3 (seperate)
symfony/console: 2.8.45 (included in a project)

I guess it should be scoped and then shipped as a .phar file to work around this.

would be sufficient if all files are only analysed statically, I assume somewhere in the code they are still loaded in some way.
https://github.com/Roave/BetterReflection could be a solution to that if it's the case - haven't had time to dig through DG's code to check if that's possible.

@Idrinth indeed, Dependency Guard uses ReflectionClass to determine if a class exists and whether the class lives in user land. Using the BetterReflection package would definitely solve this issue. I encourage you to introduce this package and solve the problem.
There are only 2 places where Reflection is used. The \Mediact\DependencyGuard\Candidate\CandidateExtractor, which uses it to determine the filename of the symbol and \Mediact\DependencyGuard\Php\Filter\UserDefinedSymbolFilter, which uses it to determine whether the symbol exists and whether it is an internal or user defined class. It seems that these features are present within BetterReflection.

will hopefully have time this weekend @johmanx10 and will have a look then.

@Idrinth I have provided a fix in the branch issue/21. Could you verify that this solves your problem?

working on it @johmanx10 - give me 15mins

The issue does not seem to be fixed yet, seems the newer interface is still in use.
Did shorten the paths, dependency-guard is a freshly cloned one from issue/21, folder contains a composer package with symfony/console as dependency.
$ php ../dependency-guard/bin/dependency-guard
PHP Fatal error:  Class Symfony\Component\Console\Style\SymfonyStyle contains 4 abstract methods and must therefore be declared abstract or implement the remaining methods (Symfony\Component\Console\Output\OutputInterface::isQuiet, Symfo
ny\Component\Console\Output\OutputInterface::isVerbose, Symfony\Component\Console\Output\OutputInterface::isVeryVerbose, ...) in folder\vendor\symfony\console\Style\SymfonyStyle.php on line 33
PHP Stack trace:
PHP   1. {main}() dependency-guard\bin\dependency-guard:0
PHP   2. Composer\Console\Application->run() dependency-guard\bin\dependency-guard:20
PHP   3. Composer\Console\Application->run() dependency-guard\vendor\composer\composer\src\Composer\Console\Application.php:103
PHP   4. Composer\Console\Application->doRun() depePHP   5. Composer\Console\Application->doRun() dependency-guard\vendor\composer\composer\src\Composer\Console\Appl
ication.php:254
PHP   6. Composer\Console\Application->doRunCommand() dependency-guard\vendor\symfony\console\Application.php:262
PHP   7. Mediact\DependencyGuard\Composer\Command\DependencyGuardCommand->run() dependency-guard\vendor\symfony\console\Application.php:886
PHP   8. Mediact\DependencyGuard\Composer\Command\DependencyGuardCommand->execute() dependency-guard\vendor\symfony\console\Command\Command.php:251
PHP   9. Mediact\DependencyGuard\Composer\Command\Exporter\ViolationExporterFactory->create() dependency-guard\src\Composer\Command\DependencyGuardCommand.php:94
PHP  10. spl_autoload_call() dependency-guard\src\Composer\Command\Exporter\ViolationExporterFactory.php:44
PHP  11. Composer\Autoload\ClassLoader->loadClass() dependency-guard\src\Composer\Command\Exporter\ViolationExporterFactory.php:44
PHP  12. Composer\Autoload\includeFile() dependency-guard\vendor\composer\ClassLoader.php:322
PHP  13. include() dependency-guard\vendor\composer\ClassLoader.php:444

Fatal error: Class Symfony\Component\Console\Style\SymfonyStyle contains 4 abstract methods and must therefore be declared abstract or implement the remaining methods (Symfony\Component\Console\Output\OutputInterface::isQuiet, Symfony\Co
mponent\Console\Output\OutputInterface::isVerbose, Symfony\Component\Console\Output\OutputInterface::isVeryVerbose, ...) in folder\vendor\symfony\console\Style\SymfonyStyle.php on line 33

Call Stack:
    0.1923     384512   1. {main}() dependency-guard\bin\dependency-guard:0
    0.3669    3263376   2. Composer\Console\Application->run() dependency-guard\bin\dependency-guard:20
    0.3669    3263376   3. Composer\Console\Application->run() dependency-guard\vendor\composer\composer\src\Composer\Console\Application.php:103
    0.4319    3346432   4. Composer\Console\Application->doRun() dependency-guard\vendor\symfony\console\Application.php:145
    2.5720    7107960   5. Composer\Console\Application->doRun() dependency-guard\vendor\composer\composer\src\Composer\Console\Application.php:254
    2.5724    7107960   6. Composer\Console\Application->doRunCommand() dependency-guard\vendor\symfony\console\Application.php:262
    2.5726    7107960   7. Mediact\DependencyGuard\Composer\Command\DependencyGuardCommand->run() dependency-guard\vendor\symfony\console\Application.php:886
    2.5828    7115176   8. Mediact\DependencyGuard\Composer\Command\DependencyGuardCommand->execute() dependency-guard\vendor\symfony\console\Command\Command.php:251
    2.6603   10466688   9. Mediact\DependencyGuard\Composer\Command\Exporter\ViolationExporterFactory->create() dependency-guard\src\Composer\Command\DependencyGuardCommand.php:94
    2.6625   10494096  10. spl_autoload_call() dependency-guard\src\Composer\Command\Exporter\ViolationExporterFactory.php:44
    2.6625   10494160  11. Composer\Autoload\ClassLoader->loadClass() dependency-guard\src\Composer\Command\Exporter\ViolationExporterFactory.php:44
    2.6631   10494288  12. Composer\Autoload\includeFile() dependency-guard\vendor\composer\ClassLoader.php:322
    2.6638   10553280  13. include('folder\vendor\symfony\console\Style\SymfonyStyle.php') dependency-guard\vendor\composer\ClassLoader.php:444

I had hoped this would not be required, but it seems we have to configure a different source locator for the reflector, in order to achieve what is desired. Either that or there is something we're overlooking. When next I work on this, I'll reproduce the environment you pointed out, instead of using what I already have, locally. Please allow a number of days to pass for this to be picked up again.

this is not a service I'm paying for @johmanx10 - I'm positively surprised at how fast you did get startet. On top of that it currently breaks only a single library's ci for us, it's not too hard to manually check that one.

@Idrinth I have strengthened the patch with your regression test and am now pretty confident that this should fix your issue. Could you verify once more?
A pull request is open for review by my co-workers. Once the pull request is accepted, a new patch version, 1.0.4, will follow.

I can confirm it no longer produces a fatal with the current patch. Thank you for your quick work.


Launchers not in library

Non of the launchers show up in the Steam library.

I'm also seeing this issue. Latest stable version of SteamOS.

Same thing for me. Three days ago was on latest Preview version, no launchers in Steam after install. Yesterday switched to latest Beta, same result - installer sees launchers installed, but no launchers in Steam.

Same here ... Installed it for the first time on a deck with nearly no customisations on the os itself

@vralfy @Skyline969 @RaejenXaxios @sagman76 @doejohnjr3 I've updated and added a fix to 2.75 that may help some people, I'll be adding the rest of the code in a couple of days to 2.8

HI i am on SteamOS 3.4.6
i downloaded SteamDeck 2.8 for the first time
i tried installing twice, and got the yoda contemplation.
then i start fresh..
all attempts fail to get the launcher in the library
please help @moraroy

https://github.com/moraroy/NonSteamLaunchers-On-Steam-Deck/issues/27
It was resolved within the last push (2.83)


Add style option to center textarea

Feature Request
When using the following style options:
sendBoxTextWrap: true,
sendBoxHeight: 55,

you get a textarea instead of an input for user input.
This textarea alligns with the top.
There's also a div behind it, probably for accessibility, with the same dimensions.
Current:

Desired:

The <div> is a doppelganger and use for auto-resizing the <textarea>.
Are there any specific options you want to customize? We will rework the send box in #2211.

I want the textarea to be in the vertical center of the sendbox.
This is how we've implemented our textarea in V3:

In V4 the textarea doesn't scale out from the center:

Just to get a field sendBoxCenterInput: true in styleOptions.
The default behavior should be to have it render as it does currently.
When set to true the textarea becomes centered vertically in the sendBox area.

I think we should center it by default. 😉

Awesome!

Some preliminary study on the layout here. You can study the code to understand how the doppelganger work.
https://compulim.github.io/playpen-textarea-doppelganger/
Still need to clean up the CSS a bit. Feel free to file PR over there for suggestions.
Will eventually move the JS/CSS there back to this repository.

I cleaned up the CSS over there. Need help from someone who can help copying the styles here.

Added to backlog - will look to prioritize for an upcoming release.

Writing a whole bunch of 'k's in our current chat:

In our V4 implementation:

In the playpen:

Setting "word-break", "overflow-wrap" on the textarea through chrome dev tools yielded no positive results.

The icon of send box button is not centered vertically. It is filed as #3672.

The icon of send box button is not centered vertically. It is filed as #3672.


Debug output written to stdout

I am currently testing your libary in a project and it works great so far, thanks for your work!
I noticed that it writes to stdout (via log in request.go/proxy(), see here), breaking the output of programs using it.
Is it possible to remove that or to make it configurable?

@svent I'd be happy to merge a PR that adds a logger *log.Logger to the Config, and using that throughout instead of the global logger. That way you can use a logger that discards output if you'd like.

Thanks for the quick feedback - that sounds good, I will create a PR for that.

Closed by #5


support metrics API querying

This PR adds function and a data struct for querying Solr Metrics API.

Pull Request Test Coverage Report for Build 322

3 of 3 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 98.571%

Totals

Change from base Build 319:
0.03%

Covered Lines:
138

Relevant Lines:
140

💛  - Coveralls


Fix/calibration publisher frame

Status
PRODUCTION / DEVELOPMENT
Description
This PR solves the issue reported on #945
It also support multiple cameras.
Todos

[X] Tests
[X] Documentation

Steps to Test or Reproduce

Open Runtime Manager
Select Sensing Tab
Launch Calibration publisher
Select "Register tf between lidar and camera"
Write desired target frame

The tf will be registered on all the cameras to the specified target frame.

Tested on Ubuntu 16.04


[d3d9] Mercenaries 2 World in flame: game crashes after ~5 seconds of gameplay.

It's working smooth without any wrapper on Win10, but decided to report it anyway.
System information

GPU: GTX 1050 Ti
Driver: 531.79 (and early)
OS: Windows 10 2H22
DXVK version: 2.1, 2.2
(Note: It seems that v2.0 does not crash, but stutters as hell )

Apitrace file(s)
Apitrace DOESN'T work with this game! It returns me a error with apitrace dll's:

Log files
2.2 logs:
Mercenaries2_d3d9.log
d3d.log
2.0 logs (does not crash but stutters):
Mercenaries2_d3d9.log
d3d.log

Haven't looked at this yet but note that the above config doesn't do anything with this dxvk
That's because I've tested it with differenr versions, including "async" one. Or whatever it called, the entirely different github page. Config is just a "leftovers".

I did some testing on Linux with AMD and radv but didn't encounter any crashing issues. Gonna try Windows later

Works fine fingers crossed with this build: https://github.com/doitsujin/dxvk/actions/runs/5372617181 But there is still some very annoying "bug": if you alt-tab too much, game stops capturing by OBS and the picture freezes in OBS preview. Other than this everything seems fine.

Game was crashing after cutscenes (some time after you exit quest buildings). if you need it, I can record an apitrace later.

Yes please.

Apitrace file(s)
Can't apitrace because of this error:

Oh, yes, I forgot about this error. So no apitrace, sorry :( Only can record a video of a glitch.

Can you also share a save game if it's in a specific location? :slightly_smiling_face:

@CortexReaver Friendly ping

@CortexReaver Friendly ping

Yeah, I am so sorry. I've been really busy lately. I'll provide all necessary files when I can.

@CortexReaver Might be interesting to give it another shot with current master when you have time
https://github.com/doitsujin/dxvk/actions/runs/7797876671

@CortexReaver Friendly ping. Could you retry with 2.3.1


SSL Error

I updated the certificate on my Linux machine, but it did not work. Any ideas?

$ pip install .
Processing /home/jennifer/git/ctcdecode
Complete output from command python setup.py egg_info:
zip_safe flag not set; analyzing archive contents...

Installed /tmp/pip-_3xikmb0-build/.eggs/wget-3.2-py3.6.egg
Traceback (most recent call last):
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/home/jennifer/anaconda3/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/home/jennifer/anaconda3/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/home/jennifer/anaconda3/lib/python3.6/ssl.py", line 814, in __init__
    self.do_handshake()
  File "/home/jennifer/anaconda3/lib/python3.6/ssl.py", line 1068, in do_handshake
    self._sslobj.do_handshake()
  File "/home/jennifer/anaconda3/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-_3xikmb0-build/setup.py", line 55, in <module>
    os.path.join(this_file, "build.py:ffi")
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/home/jennifer/anaconda3/lib/python3.6/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/setuptools/dist.py", line 372, in __init__
    _Distribution.__init__(self, attrs)
  File "/home/jennifer/anaconda3/lib/python3.6/distutils/dist.py", line 281, in __init__
    self.finalize_options()
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/setuptools/dist.py", line 528, in finalize_options
    ep.load()(self, ep.name, value)
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 204, in cffi_modules
    add_cffi_module(dist, cffi_module)
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
    execfile(build_file_name, mod_vars)
  File "/home/jennifer/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 25, in execfile
    exec(code, glob, glob)
  File "/tmp/pip-_3xikmb0-build/build.py", line 24, in <module>
    'third_party/boost_1_63_0.tar.gz')
  File "/tmp/pip-_3xikmb0-build/build.py", line 14, in download_extract
    out=dl_path)
  File "/tmp/pip-_3xikmb0-build/.eggs/wget-3.2-py3.6.egg/wget.py", line 526, in download
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 248, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/home/jennifer/anaconda3/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)>

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-_3xikmb0-build/

You can download the dependencies manually if absolutely necessary. This is a configuration issue with your python environment.


patch profile rather than use temp file.

this not make all xfail disappear. some test, which currently marked as xfail still only working when testing on single module. my guess is , that have something to do with profile module which use global variable.
but it mean future test which use profile module  will not have different result in all situation

Current coverage is 45.86% (diff: 100%)

Merging #147 into master will decrease coverage by 1.18%

@@             master       #147   diff @@
==========================================
  Files            29         29
  Lines          1016       1016
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
- Hits            478        466    -12
- Misses          538        550    +12
  Partials          0          0

Powered by Codecov. Last update 8e29dda...9756538


Test for Auth Method fails for some private keys

Key was generated with
ssh-keygen -t ed25519 -C<EMAIL_ADDRESS>
Signed-off-by: Somtochi Onyekwere<EMAIL_ADDRESS>

What does this PR do? By the looks of it, this is not just adding a test but a new feature?

No, it improves the previously added feature in #338 to support more key formats.

Thanks, I missed the previous PR.


Custom dataset

Hello,
I'm interested in running GeoMVSNet on my own dataset but am a bit unclear on how to proceed. I've read the README.md, but some parts are not entirely clear to me. My custom dataset doesn't include ground truth; it's purely for testing and consists of both indoor and outdoor scenes, covering roughly a city block's worth of distance.
You mentioned in section 3.3 (Custom Data) of the README that:
GeoMVSNet can reconstruct on custom data. At present, you can refer to MVSNet to organize your data, and refer to the same steps as above for depth estimation and point cloud fusion.
I understand that I need to organize my data according to MVSNet. However, could you please clarify what you mean by "the same steps as above"? Are you referring to the steps outlined in section 3.2 (Tanks and Temples)?
I'm currently unsure about how to apply your code to my dataset, so a detailed guide on the necessary steps would be greatly appreciated.
Thank you.

Hi, sorry for marking that part as @TODO in README since I am busy these days.
I can give you a general process here:

Pre-process of your images. (if you captured video, cut frames then)
Use Colmap to estimate camera pose. (Here is an instruction in Chinese )
Use this colmap2mvsnet script to transfer the camera pose from Colmap format to MVSNet series format.

Organize the image, camera parameters, and depth range into the style of the public dataset we provide.

Modify the test script (test_dtu.sh & fusion_dtu.sh).
Use our pre-trained model for estimating and fusion.

I hope you can apply your own dataset through my hints.
After that, when I have time, I will record a video to show how to construct custom datasets.

Hello, I'm back again.
I've tried working with the hints you kindly provided over the past few days, but the results have been really disappointing.
My dataset consists of videos shot in a small rectangular room, which I've cut into frames, totaling about 300 images.
The room is shaped like the picture below.
In the picture, the left wall is sky blue, and the right wall is brown (actually, this is a wall made of a glass door, but it appears yellow due to the outside lighting).
Inside the room, there are two sets of small chairs joined together to form couches, which are facing each other, with a table in between them.
There's also a refrigerator by the glass door.
Excluding the left and right walls, the floor is a dark cement color, and the remaining walls are white (gray).
The ceiling is also white (gray) with rectangular lights attached.

I ran your code with this dataset and got the following results, but I don't really understand why the results are like this.
Can you tell me what might be causing this based on what you see?

Even when I input the camera parameters correctly, the outcome is like this, and when I generate the colmap without any processing, it still comes out like this. I've made sure that the parameters are formatted correctly, even though I don't know the values compared to the DTU dataset.
Below are depth_est and confidence,
For depth_est, all images turn out like the one on the bottom left, could this be the problem?
The confidence seems to be coming out well though...

Looking at the results, it seems like your model could yield very good outcomes, but I'm sad that I can't find a solution.
I would appreciate it if you could share your thoughts on this issue.

Hi @vietpho.
Firstly, I have to congratulate that the results of the point cloud you presented represent that you have successfully run our model. And honestly, I think the result is acceptable since the checkpoint we released is trained on the DTU dataset which mainly contains small objects.
However, the data you faced is a large room (open, with glasses, light change, etc.). Maybe you should try finetuning our pre-trained model on large-scale datasets like BlendedMVS or other datasets (7-scenes?).
There are several other reasons for the above results, including:

The camera parameters obtained by Colamp can not guarantee the accuracy.
The video shot may not have controlled exposure, etc., and the frames are blurred.
The depth range assumption is not accurate. This is why depth map visualization failed. Try to specify the v_min and v_max of visualization.

I don't think using the model we provided directly is suitable for your use case. You can try to finetune our base model or use other methods (NeuralRecon and SimpleRecon are used to reconstruct rooms).

Thank you for your advice. I'll give it another try following your suggestions!


MatCheckbox not rendering check for undefined color when using default options

Reproduction
https://stackblitz.com/edit/angular-yyan4s?file=src%2Fapp%2Fcheckbox-overview-example.html
Steps to reproduce:

Adding the default behavior provider     {   provide: MAT_CHECKBOX_DEFAULT_OPTIONS,   useValue: { clickAction: "noop" } as MatCheckboxDefaultOptions }
Setting any checkbox via input, for example: <mat-checkbox [checked]="true">Checked</mat-checkbox>

Expected Behavior

Actual Behavior
What behavior did you actually see?

Environment

Angular: 11
CDK/Material: 11
Browser(s): Firefox 82
Operating System (e.g. Windows, macOS, Ubuntu):  Windows 10

@crisbeto this was supposed to be fixed in #18467 but the stackblitz above still seems to have the problem behavior


add scaledown, target cpu scaling, and update modules

Update modules to latest version, configure target cpu for scaling to 80% instead of 50%. Also added scale down and up. CPU and memory metrics look ok in aws for the different environments.

CI: No IaC security warnings found

CI: No IaC security warnings found

CI: No IaC security warnings found


TRIO100 does not detect async context managers

Noticed this in my work on https://github.com/python-trio/trio/pull/2947, it appears that TRIO100 does not detect async context managers.
minimal.py:
import trio

async def test_reentry_doesnt_deadlock() -> None:
    # Regression test for issue noticed in GH-2827
    # The failure mode is to hang the whole test suite, unfortunately.
    # XXX consider running this in a subprocess with a timeout, if it comes up again!

    async def child() -> None:
        while True:
            await trio.to_thread.run_sync(trio.from_thread.run, trio.sleep, 0, abandon_on_cancel=False)

    with trio.move_on_after(2):
        async with trio.open_nursery() as nursery:
            for _ in range(4):
                nursery.start_soon(child)

> ruff check minimal.py --isolated --select TRIO
minimal.py:12:5: TRIO100 A `with trio.move_on_after(...):` context does not contain any `await` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint.

> ruff --version
ruff 0.2.1

Thanks!


Handle broken Accept headers somehow

see http://www.gethifi.com/blog/browser-rest-http-accept-headers

So it looks like accept headers are broken in every possible way, with no obvious not-broken approach.
Did you have any notion of what the most correct approach would be for us? Clearly not just ignoring the accept header completely, as Rails does. =^)

I am not sure. I was thinking of somehow detecting if this is a "normal" browser request and replacing the Accept header with something appropriate. But this is complicated, as the value of IE's Accept header depends on the programs installed on the system. And the User-Agent is generally not to be trusted, I guess. Recent Chrome browsers do know how to behave.

As part of our maintenance of this gem, we are closing any feature tickets without associated patch or discussion in < 2 years.
If anyone is willing to take this on, please feel free! <3