Retry failed workflow with ttl  deleted after initial secondsAfterFailure while still running

Pre-requisites

[X] I have double-checked my configuration
[X] I can confirm the issue exists when I tested with :latest
[X] I have searched existing issues and could not find a match for this bug
[X] I'd like to contribute the fix myself (see contributing guide)

What happened/what did you expect to happen?
I am retrying a failed worflow that has steps set up with manual approval and ttl secondsAfterFailure .
Step-0 failed in the first execution  , i  tried a call to /retry method , the failed step has now gone fine and I manage to advance our workflow correctly.

The workflow is now waiting for the next step with manual approval as you can see in the screenshot .
But once the TTL secondsAfterFailure is completed, "workflow gone" message is displayed and the workflow status shown in the UI is the one before the /retry action.
"workflow gone" image :

"archived workflows" image :

I think ttl secondsAfterFailure is not takig into account that the workflow is running, ~as it has a "suspend" step waiting for approval.~ edited by agilgur5: suspend is unrelated / red herring in this case, see below
Version
v3.4.11
Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: "ttl-workflow-deploy-template"
spec:
 activeDeadlineSeconds: 1814400
 archiveLogs: true
 ttlStrategy:
   secondsAfterCompletion: 300 # Time to live after workflow is completed, replaces ttlSecondsAfterFinished
   secondsAfterSuccess: 300   # Time to live after workflow is successful
   secondsAfterFailure: 300    # Time to live after workflow fail
 entrypoint: deploy-workflow
 onExit: exit-handler
 templates:
    - name: exit-handler
      steps:
        - - name: send-report
            templateRef:
              name: workflow-template-whalesay-template
              template: whalesay-template
            arguments:
              parameters:
                 - name: message
                   value: "{{workflow.status}}"

    - name: deploy-workflow
      dag:
        tasks:
        - name: step-provision
          templateRef:
              name: workflow-template-whalesay-template
              template: whalesay-template
          arguments:
              parameters:
                - name: message
                  value: "step-provision"

        - name: step-0
          depends: "step-provision"
          templateRef:
            name: workflow-template-random-fail-template
            template: random-fail-template

        - name: step-0-approval
          depends: "step-0.Succeeded"
          template: step-approval

        - name: step-1
          depends: "step-0-approval"
          when: '{{tasks.step-0-approval.outputs.parameters.action}} == DEPLOY'
          templateRef:
            name: workflow-template-random-fail-template
            template: random-fail-template

        - name: step-1-approval
          depends: "step-1.Succeeded"
          template: step-approval

        - name: step-2
          depends: "step-1-approval"
          when: '{{tasks.step-1-approval.outputs.parameters.action}} == DEPLOY'
          templateRef:
            name: workflow-template-random-fail-template
            template: random-fail-template

        - name: step-2-approval
          depends: "step-2.Succeeded"
          template: step-approval

        - name: step-3
          depends: "step-2-approval"
          when: '{{tasks.step-2-approval.outputs.parameters.action}} == DEPLOY'
          templateRef:
            name: workflow-template-random-fail-template
            template: random-fail-template

    - name: step-approval
      suspend: {}
      inputs:
        parameters:
        - name: action
          default: 'DEPLOY'
          enum:
          - 'DEPLOY'
          - 'ROLLBACK'
        - name: skipAutovalidation
          default: false
      outputs:
        parameters:
          - name: action
            valueFrom:
              supplied: {}
          - name: skipAutovalidation
            valueFrom:
              supplied: {}

Logs from the workflow controller
time="2024-02-07T09:58:53.901Z" level=info msg="Updated phase Running -> Failed" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:53.901Z" level=info msg="Marking workflow completed" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:53.901Z" level=info msg="Marking workflow as pending archiving" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:53.907Z" level=info msg="cleaning up pod" action=deletePod key=argo-workflows-des/ttl-workflow-deploy-template-h557k-1340600742-agent/deletePod
time="2024-02-07T09:58:53.922Z" level=info msg="Workflow update successful" namespace=argo-workflows-des phase=Failed resourceVersion=2206381733 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:53.949Z" level=info msg="archiving workflow" namespace=argo-workflows-des uid=7a114a25-3f2d-42af-b128-f647bb1a3e59 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:53.953Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-workflows-des/ttl-workflow-deploy-template-h557k-whalesay-template-1434212105/labelPodCompleted
time="2024-02-07T09:58:53.991Z" level=info msg="Queueing Failed workflow argo-workflows-des/ttl-workflow-deploy-template-h557k for delete in 5m0s due to TTL"
time="2024-02-07T09:58:57.983Z" level=info msg="Processing workflow" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:57.983Z" level=info msg="Task-result reconciliation" namespace=argo-workflows-des numObjs=0 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:57.983Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:58:57.983Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4259206060, taskName step-0"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4259206060, taskName step-0"
time="2024-02-07T09:58:57.984Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4259206060, taskName step-0"
time="2024-02-07T09:58:57.984Z" level=info msg="All of node ttl-workflow-deploy-template-h557k.step-0 dependencies [step-provision] completed" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:57.984Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:57.987Z" level=info msg="Pod node ttl-workflow-deploy-template-h557k-4259206060 initialized Pending" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:58.039Z" level=info msg="Created pod: ttl-workflow-deploy-template-h557k.step-0 (ttl-workflow-deploy-template-h557k-random-fail-template-4259206060)" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:58.039Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:58:58.039Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:58:58.040Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:58:58.040Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:58.040Z" level=info msg=reconcileAgentPod namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:58:58.040Z" level=warning msg="Coudn't obtain child for ttl-workflow-deploy-template-h557k-11827592, panicking"
time="2024-02-07T09:58:58.062Z" level=info msg="Workflow update successful" namespace=argo-workflows-des phase=Running resourceVersion=2206381913 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.014Z" level=info msg="Processing workflow" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.014Z" level=info msg="Task-result reconciliation" namespace=argo-workflows-des numObjs=0 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.014Z" level=info msg="node changed" namespace=argo-workflows-des new.message=PodInitializing new.phase=Pending new.progress=0/1 nodeID=ttl-workflow-deploy-template-h557k-4259206060 old.message= old.phase=Pending old.progress=0/1 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:04.014Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:04.015Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:04.015Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.015Z" level=info msg=reconcileAgentPod namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:04.015Z" level=warning msg="Coudn't obtain child for ttl-workflow-deploy-template-h557k-11827592, panicking"
time="2024-02-07T09:59:04.030Z" level=info msg="Workflow update successful" namespace=argo-workflows-des phase=Running resourceVersion=2206382142 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.319Z" level=info msg="Processing workflow" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.319Z" level=info msg="Task-result reconciliation" namespace=argo-workflows-des numObjs=1 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.319Z" level=info msg="task-result changed" namespace=argo-workflows-des nodeID=ttl-workflow-deploy-template-h557k-4259206060 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.319Z" level=info msg="node changed" namespace=argo-workflows-des new.message= new.phase=Succeeded new.progress=0/1 nodeID=ttl-workflow-deploy-template-h557k-4259206060 old.message=PodInitializing old.phase=Pending old.progress=0/1 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:16.320Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3686633912, taskName step-0-approval"
time="2024-02-07T09:59:16.320Z" level=info msg="All of node ttl-workflow-deploy-template-h557k.step-0-approval dependencies [step-0] completed" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.320Z" level=warning msg="Node was nil, will be initialized as type Skipped" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=info msg="Suspend node ttl-workflow-deploy-template-h557k-3686633912 initialized Pending" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=info msg="node ttl-workflow-deploy-template-h557k.step-0-approval suspended" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=info msg="node ttl-workflow-deploy-template-h557k-3686633912 phase Pending -> Running" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:16.323Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:16.323Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=info msg=reconcileAgentPod namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.323Z" level=warning msg="Coudn't obtain child for ttl-workflow-deploy-template-h557k-11827592, panicking"
time="2024-02-07T09:59:16.349Z" level=info msg="Workflow update successful" namespace=argo-workflows-des phase=Running resourceVersion=2206382789 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:16.355Z" level=info msg="cleaning up pod" action=labelPodCompleted key=argo-workflows-des/ttl-workflow-deploy-template-h557k-random-fail-template-4259206060/labelPodCompleted
time="2024-02-07T09:59:26.341Z" level=info msg="Processing workflow" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=info msg="Task-result reconciliation" namespace=argo-workflows-des numObjs=1 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=info msg="task-result changed" namespace=argo-workflows-des nodeID=ttl-workflow-deploy-template-h557k-4259206060 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:26.341Z" level=info msg="node ttl-workflow-deploy-template-h557k.step-0-approval suspended" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4275983679, taskName step-1"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-3362124077, taskName step-1-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-4292761298, taskName step-2"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-2029873582, taskName step-2-approval"
time="2024-02-07T09:59:26.341Z" level=warning msg="was unable to obtain the node for ttl-workflow-deploy-template-h557k-14571621, taskName step-3"
time="2024-02-07T09:59:26.341Z" level=info msg="TaskSet Reconciliation" namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=info msg=reconcileAgentPod namespace=argo-workflows-des workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T09:59:26.341Z" level=warning msg="Coudn't obtain child for ttl-workflow-deploy-template-h557k-11827592, panicking"
time="2024-02-07T09:59:26.351Z" level=info msg="Workflow update successful" namespace=argo-workflows-des phase=Running resourceVersion=2206382789 workflow=ttl-workflow-deploy-template-h557k
time="2024-02-07T10:03:14.000Z" level=info msg="Deleting garbage collected workflow 'argo-workflows-des/ttl-workflow-deploy-template-h557k'"
time="2024-02-07T10:03:14.012Z" level=info msg="Successfully deleted 'argo-workflows-des/ttl-workflow-deploy-template-h557k'"

Logs from in your workflow's wait container
kubectl logs -n argo -c wait -l workflows.argoproj.io/workflow=${workflow},workflow.argoproj.io/phase!=Succeeded

The workflow is now waiting for the next step with manual approval as you can see in the screenshot .
But once the TTL secondsAfterFailure is completed, "workflow gone" message is displayed and the workflow status shown in the UI is the one before the /retry action.
[...]
"archived workflows" image :

So using archived workflows is an important piece of this. Updates to the live and archived workflows naturally have a race condition -- the archived version may lag behind the live one.
Does the archived workflow's old status persist after, say, 10 minutes? Or does it then match the new status?

I think ttl secondsAfterFailure is not takig into account that the workflow is running, as it has a "suspend" step waiting for approval.

Not sure if you have a typo here or not -- it sounds like it hit the TTL during the suspend based on your description, no?
The TTL should take into account suspended time, I believe it is a pure calculation from startedAt and not how much time the Workflow has spent actively running.

Once a workflow fails and has configured the value ttlStrategy - secondsAfterFailure: 300 , the workflow is queued for deletion .
You can see an example in attached traces :
time="2024-02-07T09:58:53.991Z" level=info msg="Queueing Failed workflow argo-workflows-des/ttl-workflow-deploy-template-h557k for delete in 5m0s due to TTL"
And the workflow is deleted 5 minutes later,
time="2024-02-07T10:03:14.012Z" level=info msg="Successfully deleted 'argo-workflows-des/ttl-workflow-deploy-template-h557k'"
But if we launch a retry command ,and the workflow goes to "running" state ,the workflow is deleted even if it is in running state as you can see in the logs .
This behaviour occurs when archiving is enabled and disabled.
I would expect that if a retry is performed on a failed wf and the workflow goes to "running" state, the workflow should not be deleted because the retry operation has fixed the error and the workflow is running.

Thanks for clarifying. Yes, in the case of a secondsAfter* TTL, it should be reset when there is a retry. I suspect that the Workflow is just never removed from the queue when a retry is triggered (retries are also currently handled a bit incorrectly: #12538)
Could you answer the question I had regarding the Workflow Archive? Does it eventually update to the retried Workflow? If not, that might be another bug, an unhandled race

Per your analysis, that sounds like the suspend step is unrelated then, right?

Correct , suspend step is unrelated .
Could you also answer the question I had regarding the Workflow Archive? Does it eventually update to the retried Workflow? If not, that might be another bug, an unhandled race

It is not updated to the retry workflow, it is shown in archived when the workflow fails. I attached two images related in my initial comment
,

I attached two images related in my initial comment

Yes I saw those, but I was wondering if the Archived Workflow might update after, say, 10 more minutes. If the last screenshot stays the same or changes to match the retried Workflow.
Per your response, it stays the same, so it seems like there's a secondary unhandled race condition here. Although fixing the TTL issue might resolve that race as well; the TTL GC is not anticipating an incomplete Workflow, so in this case GC is happening before archiving (since only completed Workflows get archived)

@agilgur5 @manuelbmar We are also facing the same issue. We have set a ttl of 7 days for the workflow. Let's Workflow has failed 7 days, the workflow was retried 2 days ago and it is in Running state. Still the workflow has deleted.
Any resolution for this issue. It seems a critical issue

@agilgur5  Any updates? Are there any open issues or pull requests being worked on to resolve the problem?

It seems a critical issue

This only occurs as a race condition with a combination of several features. Retries in particular should not be used frequently (as that would suggest there is an issue with the tasks themselves that should be fixed) and are also one of the most complex areas of the codebase. I.e. this is a low frequency + high complexity issue.

@agilgur5 Any updates? Are there any open issues or pull requests being worked on to resolve the problem?

If there were updates, they would already be in the thread. Please follow proper open source etiquette.

[x] I'd like to contribute the fix myself (see contributing guide)

You are also more than welcome to contribute as you checked that you'd like to do.

Although fixing the TTL issue might resolve that race as well; the TTL GC is not anticipating an incomplete Workflow, so in this case GC is happening before archiving (since only completed Workflows get archived)

Clarification here, archiving only occurs for a completed Workflow, so this is a single bug where the solution is likely to be to remove the Workflow from the TTL queue when it is retried.

hi, @agilgur5 @manuelbmar We encountered a similar issue where we couldn't find a good solution for removing retried workflows from the TTL queue, as there seems to be no built-in method for deleting elements from the delayed queue.
As a temporary workaround, we have implemented a somewhat inelegant bypass, as in https://github.com/argoproj/argo-workflows/pull/12905, which incurs a query cost before the deletion operation, but it effectively addresses our current issue.
Regarding the idea of removing elements from the queue:

Adding a label named workflows.argoproj.io/retried to workflows that have been retried.
Adding logic in the handler function of the gc_controller.go informer to handle workflows with the 'retried' label, such as keeping a record of retried workflows and skipping them during dequeuing from the delayed queue.

would like to ask if this direction is feasible?

We encountered a similar issue where we couldn't find a good solution for removing retried workflows from the TTL queue, as there seems to be no built-in method for deleting elements from the delayed queue.

The client-go workqueue doesn't have a Remove function, but can't you just call Done early basically?

As a temporary workaround, we have implemented a somewhat inelegant bypass, as in #12905, which incurs a query cost before the deletion operation, but it effectively addresses our current issue.

Yea that could potentially add quite a lot of queries, since it's one more for every deletion 😕

Adding a label named workflows.argoproj.io/retried to workflows that have been retried.

We actually are implementing something similar in #12734 (see also #12538)


Associations don't work when using slice of struct pointers

Hello
If I change the example in the documentation to use a slice to struct pointers, Gorm complains.
I am using Postgres 9.3 if that helps
type User struct {
    Id           int64

    Emails            []*Email         // This doesn't work
}

Thanks
-Rv

Hi @rvenugopal
Sorry for the long time no response, this should be fixed now.
Thank you for your report.


Highlight sterling performance work

I'd just like to extend my personal thanks to @rynowak and team for the amazing performance strides being made in MVC.

cc @DamianEdwards @SajayAntony
:smiley_cat:

@danroth27 should we move this to the Backlog? :smile:

Oh, and where do I log the same bug but to thank @benaadams for all his hard work and contributions???

:blush:

:+1:

:fireworks:  :+1:  :heart_eyes:

@Eilon my great hope is one day people will move past the defeatist argument that it doesn't matter because "my database is slow". That's like saying I might as well use php in interpreted mode because "my database is slow". There is so much more potential to the platform!
We were going custom for the performance; but we've come back to aspnet because of the improvements in Kestrel; and now we are coming back to to the full stack for the great strides in performance and direction of MVC. So I just wanted to say thank you.
Also saves us a bunch of work :wink:

It begins... https://stevedesmond.ca/blog/performance-is-paramount /cc @stevedesmond-ca

:+1: :yum:

LOL @ the timing of this -- I was trying to find the time to finish that post for days!
But seriously, thanks! I wish I had the cycles to contribute more directly, but you're all doing great work!

@stevedesmond-ca great blog post!

:blush:

Closing because this is an old discussion. We always appreciate the kind words 😄


feat: upgrade dev deps

Description
upgrade Dev dependencies
Linked Issues/Tasks
Type of change
Refactoring
Tests
No test
QA Notes
No special check required

Абсолютно безопасно

Regarding socket Security issues - we already have those dependencies in our package.json.
We can't get rid of tiny-secp256k1 or protobufjs. But we should try to remove mongodb-memory-server
I will open another issue about it


How can I have the dropped file?

I try to use this library with Blazor and here you fine the repo. It is working.
The problem I get is to upload a file. Using only a simple HTML page
<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <meta http-equiv="X-UA-Compatible" content="ie=edge" />
        <title>Example / Preview</title>
        <link rel="stylesheet" href="../src/css/easymde.css" />
        <script src="../src/js/easymde.min.js"></script>
    </head>

    <body>
        <textarea></textarea>
        <script>
            const easyMDE = new EasyMDE({
                uploadImage: true,
                imageMaxSize: 2097152,
                imageAccept: "image/png, image/jpeg",
                imageUploadEndpoint: "https://localhost:7013/api/Image",
                imageUploadFunction: (file, onSuccess, onError) => {
                    console.log(file);
                },
            });
        </script>
    </body>
</html>

The result is this one (there some same Xs)

I drop and image on the textbox, I don't see any request from the page to the API.

What I expect is that the EasyMDE calls the API to save the file. So, the API return the path of the image.
Then, my question. All my APIs have a strict security, so I have to customize the call to the API. For that, I like to have the content of the file. With the content of the file (in a byte[] format or in base64) I can write a custom call and then give to the EasyMDE the path of the image.
Is it possible to have something like that?
Thank you in advance,
Enrico

Check the documentation for imageUploadFunction. If that option is given, you should write a custom upload function in it, and it will replace the default uploading logic. The editor will only show the result when onSuccess is called.
The file in your "console.log(file);" is this type: https://developer.mozilla.org/en-US/docs/Web/API/File, I think file.arrayBuffer() will be useful to you.

Thank you. I found how to have the file!

I have a working example! Thank you


[PART-2] GPUGridAggregation : Add experimental GPUGridLayer.

For #1556

Copy existing GridLayer and GridCellLayers to experimental.
Update layers to use GPUGridAggregator for world space aggregation.
Add additional utility methods required to enable GridCellLayer to consume aggregated data.
Add  unit tests.
Add examples to layer-browser.
2 to 3X boost in FPS when using GPU Aggregation.
Picking and Color/Elevation customizations done on CPU and are not supported in this new experimental layer, those feature need to added by moving more calculation to GPU or by using TF.

Change List

Add new experimental GPUGridLyaer, uses GPU Aggregation

@ibgreen

Before reviewing in detail: Was the decision in today's meeting that we would update the official layers instead, assuming that the GPU aggregation can easily be turned off with a single prop?

Yes thats the plan for ScreenGridLayer. For GridLayer, GPU version is not on parity with CPU version (picking, color/elevation scale need to re-implemented for GPU), so GPUGridLayer will remain experimental until those features ported into shaders.
For ScreenGridLayer, there is some work needed (colorDomain/range) , aiming to finish that this week, so we can replace existing ScreenGridLayer with GPUScreenGridLayer (in experimental), please note there are changes to CPU path as well, due to attribute data format has to match between CPU and GPU.


Remove old hack.

Codecov Report

Merging #144 (b69cc5d) into main (bb0d0fe) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #144      +/-   ##
==========================================
- Coverage   43.15%   43.13%   -0.03%
==========================================
  Files          83       82       -1
  Lines        4187     4185       -2
==========================================
- Hits         1807     1805       -2
  Misses       2027     2027
  Partials      353      353

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bb0d0fe...b69cc5d. Read the comment docs.


Improve build-environment for GNU/Linux users

Hi,
please find some conversions of your batch scripts, to bash-shell scripts.
Usage of the compiler for AVR is very easy under a Debian-based system (Debian, Ubuntu, Mint).
Simply do
sudo apt-get install gcc-avr avr-libc make

Just added some commits for CMake.
I'm running Linux here, so I don't know if some modifications to CMakeLists.txt are needed for Windows (you might possibly need to uncomment this line for example)
Anyway, to build:
mkdir build
cd build
cmake ..
make

Examples of nice CMakeLists.txt for close-to-hardware-projects can be found here:
https://github.com/arobenko/embxx_on_rpi
https://github.com/mkleemann/cmake-avr
https://github.com/projectNe10/Ne10

Nice. Thanks for the *nix additions.


Injection custom headers

Hi,
When I add headers with:
msg = Message(headers=headers)

They are not added to the message that is created. Am I doing something wrong, or... should I use:
msg.message.add_header('test', 'hi')

The add_header works, but I think it's a bit weird to use it when there is a headers parameter :)
I've tried adding the header with a list of string, and also as tuple, but both do not end up in the final message...
For now I will use the add_headers. Was just wondering if I was doing something strange when using the headers param.
Thanks for the reaction.

Fixed in the latest version 0.0.6. Thanks for pointing it out.

I think this still not working. can you do some test again?

I just checked and it is working. Maybe you're not using the latest version? You can check which version of the library using using the following command
(venv) sumit@HAL9000:~$ pip3 show drymail
Name: drymail
Version: 0.0.6
Summary: Drymail is a minimalist wrapper over Python’s existing smtplib and email libraries, designed to be friendly but unrestrictive.
Home-page: https://github.com/SkullTech/drymail
Author: Sumit Ghosh
Author-email<EMAIL_ADDRESS>License: MIT
Location: /home/sumit/Coding/drymail/venv/lib/python3.7/site-packages
Requires: mistune, beautifulsoup4
Required-by:

It should show version 0.0.6

I can also confirm that version 0.0.6 fixes the problem.
Bit late test/upgrade/reaction :(
@SkullTech: Thanks for fixing this.

Sorry for being late. I have to work on this, my response time :) Anyway glad that it helped you. Cheers!!


Support Fn::If Intrinsic

Hello!

Vote on this issue by adding a 👍 reaction
If you want to implement this feature, comment to let us know (we'll work with you on design, scheduling, etc.)

Issue details

The Fn::If intrinsic is used in some constructs (e.g. kinesis.Stream)
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-conditions.html#intrinsic-function-reference-conditions-if
Affected area/feature

Added to epic https://github.com/pulumi/home/issues/2191


Operating System Concepts: Non Preemptive Scheduling

Description
Non-preemptive scheduling algorithms
Overview

First come first serve
Shortest job first
Priority scheduling(non-preemptive)

Domain
C/CPP
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign


After implementing locales, middleman config.rb won't allow nested loops.

We recently implemented locales in Contentful, and are pulling the .yaml files into Middleman just fine.
Now we need to rework all of our templates with the localization helpers as every single object in the .yaml file now has a :en-US: level...
But one issue we have encountered is that when we were proxying pages in a nested loop, these are no longer working.  We have spent ~16 people hours on this and can't get it to work.
I am hoping somebody may have run into something similar and can either sympathize or better still offer up a solution? :-)
Is there possibly a conflict between middleman_contentful and the i18n middleman module?
Here are examples of our previous and new version of the code:
# Build james pages
    data.oursite.alljamess.each do |id, james|
      proxy "vs/#{james.slug}/index.html", "james.html", locals: { james: james }, :ignore => true
      data.oursite.alljamess.each do |id, james2|
        if james.name != james2.name
          proxy "vs/#{james.slug}/#{james2.slug}/index.html", "james_james.html", locals: { james: james, james2: james2 }, :ignore => true
        end
      end
    end

And the new version:
# Build james pages
    data.oursite.alljamess.each do |id, james|
      proxy "vs/#{james['slug']['en-US']}/index.html?l288=true", "james.html", locals: { james: james }, :ignore => true
      data.oursite.alljamess.each do |id, james2|
        if james.name['en-US'] != james2.name['en-US']
          proxy "vs/#{james['slug']['en-US']}/#{james2['slug']['en-US']}/index.html", "james_james.html", locals: { james: james, james2: james2 }, :ignore => true
        end
      end
    end

From testing, we see the pages in the outer loop are generated, but the inner ones don't.
simply put:
# Build james pages
    data.oursite.alljamess.each do |id, james|
      puts "a"
      data.oursite.alljamess.each do |id, james2|
      	puts "b"
      end
    End

does not return abbbbbabbbbbabbbb...  but returns aaaaaaaaaaaa

I think we found a way to get this working, although either this fix or simply the upgrade to the new Middleman version has caused our site to grind to a halt during middleman build
https://stackoverflow.com/questions/62976636/how-to-get-nested-loops-in-config-rb-to-work-in-middleman


GitHub link doesn't work when it's "deep"

For example, on https://pub.dartlang.org/packages/meta, the link is in the pubspec is https://github.com/dart-lang/sdk/tree/master/pkg/meta, but the site removed the /pkg/meta part.

The current page has:

the Homepage link: the original URL, which is deep into a repository
the (inferred) Repository link: the actual repository root
the (inferred) Issue tracker link: relative to the repository root

What would you expect instead? How would you like to see the distinction between the (now-Homepage) deep location and the repository root?

This works fine now we show an explicit Homepage link; closing


Don't auto sync. Add sync button.

Our cosmos backend started getting a lot more traffic after the last release. I believe it's due to a change I made to do a cloud sync whenever an Arcade tab gains focus. This backs that out and adds a "Sync" button to the home screen, allowing the user to choose when to sync cloud projects.
This change should cut down the number of cross-tab conflicts as well.

What if the user doesn't click the button ever? Is that ok?  Do we atleast sync once we load the page?
Instead of the button, can this be done based on the time heuristics? Like if there was sync in last 10 minutes we don't trigger another one, otherwise we do?

What if the user doesn't click the button ever? Is that ok? Do we atleast sync once we load the page?
Instead of the button, can this be done based on the time heuristics? Like if there was sync in last 10 minutes we don't trigger another one, otherwise we do?

We sync when the page loads, and we do a sync on a project when it's opened. I want to stay away from any kind of auto sync until we have a more efficient API for it. It needs to be a super lightweight check that doesn't hit cosmos.


fix compile with gcc 6.x on armv7hl

The default mode in gcc 6 for C++ is now -std=gnu++14 instead of -std=gnu++98.
Fixes #15.

hm, looks like there are some issues with adding parameters to $CXXFLAGS on ruby < 2 :(.

got already fixed in master


Last volume won't mark as read when completed

Steps to reproduce
read except last volume in tachiyomi.
Mark as read in kavita/read the volume
Sync tachiyomi. It marks as completed but last volume not marked as read
Expected behavior
NA
Actual behavior
NA
Crash logs
No response
Tachiyomi version
ddd180e56aef69380b1da3137dca8aed590b68e1
Android version
NA
Device
NA
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with an extension, I should be opening an issue in the extensions repository.
[X] I have tried the troubleshooting guide.
[X] I have updated the app to version 0.13.6.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

also applies to chapters

Fixed in https://github.com/Kareadita/Kavita/pull/1503. Nightly: v<IP_ADDRESS>


Add shell integration for Fish

Current ShellGPT ingeration works only with Zsh and Bash. Add shell integration for Fish as well.
Originally posted by @endolith in https://github.com/TheR1D/shell_gpt/discussions/268#discussioncomment-7777678

I have try wrote this:
function _sgpt_fish
    set -l cmd (echo "$line" | sgpt --shell --no-interaction)
    commandline -r $cmd
end

bind \cl _sgpt_fish

But the $line is empty, don't know how to get the input buffer in fish

I get, it work now
function _sgpt_fish
    set -l cmd (commandline | sgpt --shell --no-interaction)
    commandline -r $cmd
end

bind \cl _sgpt_fish

@onewesong I'd like to try it out. Where do I need to place this snipped?
Does it go to ~/.config/fish/completions/  ?

@andyfr If you just want to try,  you can just paste it to fish terminal

Didn't work for me:
change git branch Usage: sgpt [OPTIONS] [PROMPT]
Try 'sgpt --help' for help.
╭─ Error ───────────────────────────────────────────────────────────────────────────────────────────────────╮
│ No such option: --no-interaction (Possible options: --install-integration, --no-install-functions,        │
│ --no-install-integration)                                                                                │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────╯

@hugows maybe upgrade your fish version, my version is 3.4.1

 pip install shell-gpt --upgrade


Package Explorer Limit is now 15

From https://npe.codeplex.com/workitem/167

I am unable to see more than 15 packages when using a filter. When I type "StructureMap", I get a lot of packages, but can only see 15. Evening trying to sort by Id doesn't seem to work when I use a filter.

How do you reproduce this issue? I see 153 packages for StructureMap, with 15 per page

Indeed, this seems to be fixed


Add different platform paths to .zshrc setup instructions

Annoyed this has to be spelled out, but this is documentation after all!

I'm actually not sure we should be documenting this. After all, for a user to successfully use homebrew at all, they have to put it in their path, and telling them how to do that is probably outside the scope of the just readme.

OK, that's fair. You're right, this should already be done. As long as they put the README code at the bottom of their .zshrc/.zprofile, it should work.


add google-maps dependancies into component maps

Hi Julien,
Just modify, the injection dependancy for map's component.
Angular2_Google_maps is now fully into maps component.

Good stuff bro :)


Remove MakePayment and introduce SignedExtension docs/examples

Closes #3299
Also adds some documentation about the replacement of MakePayment: SignedExtensions to the example module.

[clabot:check]


Improve CLI instructions in Things Join Server documentation

Summary
While registering a device in the Things Join Server, CLI Configuration With Cloud Cluster can be used. Only if a third-party LoRaWAN network is used, the CLI Configuration With Join Server Only has to be used.
Why do we need this ?
To make the CLI Configuration part more clear to the customers when using Things Join Server.
What is already there? What do you see now?
In the Register Devices section, it is mentioned to use a configuration file with Join Server only.
Ref: https://www.thethingsindustries.com/docs/getting-started/cloud-hosted/tti-join-server/register-devices/#prerequisites
What is missing? What do you want to see?

...
How do you propose to document this?
Update the CLI Configuration instruction point as below in the Register Devices section as part of the PR here.

Command-line interface (CLI) configuration file with Cloud cluster. If you are working with a third-party LoRaWAN network, use the Command-line interface (CLI) configuration file with Join Server Only.

Ref: https://www.thethingsindustries.com/docs/getting-started/cloud-hosted/tti-join-server/register-devices/#prerequisites
Can you do this yourself and submit a Pull Request?
No

Closed with #667.


use newer version of fsnotify for receiver to work correctly with cgroup v2

[x] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes
This change will use the newer version of fsnotify that is already in go.mod.
If possible, I would appreciate a cherry-pick of this and a 0.32.0 release.

Verification

Looks good to me, we have been missing watch events too, out of curiosity what changed in the newer version? i couldnt find a smoking gun in the changelog.

Don't hold me to this but I think it is this PR which was released with 1.5.4 that contained the relevant fix.

Please move forward with #6432 instead. As it turns out my initial analysis of my issue was incorrect and this ended up being the solution to my problems.


Update README.org

I haven't tested it but I'm guessing that I would include the dependency if I want the feature...

Thanks, merged!


Copying and pasting token path when remapping a token adds extra lines

Is your feature request related to a problem? Please describe.
When double clicking to copy and paste a token path in the remap modal, the plugin adds extra lines which forces the user to delete the extra lines first before remapping.
Describe the solution you'd like
Allow for users to also double click, copy and paste the path without the extra lines being added.
Describe alternatives you've considered
As seen in the recording the current workaround is to drag and select the path and paste it which does not cause for the extra lines error.
Recording:

Definition of done:

Double clicking the token name in the inspect panel copies the token name
Pasting it in the remap input field does not add extra lines in the input field

This request has been moved to our new feedback tool. Go check out https://tokensstudio.featurebase.app/ to create a free account. You can upvote and comment on existing posts, make a new request, and subscribe to be notified when requests move into our roadmap.

Added to Featurebase Roadmap : 🪙 Token organization enhancements


Unit test

These test cases will cover 92 % of the code base.
it helped detect some Bugs, I will open the found issues:

the title of the number annotation is ignored and not found in the json schema
the "readOnly" attribute is sometimes written "readonly" and other times "readOnly", maybe we need to define constants.

All test cases are failing, they were running fine in my local repo, I am closing this pull request, i will update with the latest changes and check them again.


Getting Wrong Value 😓

give value=1000
getting - 8108080

Same issue.

Import CSS file in your app :

Yes, you didn't applied CSS. Closing issue


Where can I get your paper?

Your paper is great, but I can't find it. Can you provide a paper?

Your paper is great, but I can't find it. Can you provide a paper?

https://dl.acm.org/doi/pdf/10.1145/3404835.3462915


Notebook: KNN regression

See the spreadsheet for details

Content
Programming

DB
TS

Ideas/prereqs:
Regression vs classification, regression and prediction, causality and experiments, Functions, estimating functions, supervised vs unsupervised,
Direct link
https://jupyter.olney.ai/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fmemphis-iis%2Fdatawhys-content-notebooks&subPath=KNN-regression.ipynb&app=lab

v1 with https://github.com/memphis-iis/datawhys-content-notebooks/pull/39

Added pipeline and scaling
Cut loop plots put kept loop rmse for optimal k
Loop plots are beautiful and should be brought in again later

v2 with https://github.com/memphis-iis/datawhys-content-notebooks/pull/40

Partly put the loop plots back in, with the following differences

Not in loop
Removed need to sort dataframe by using markers

This gives them exposure to putting multiple layers on the same plot, which I think is really important and achievable for them at this stage :smile_cat:

PS version with https://github.com/memphis-iis/datawhys-content-notebooks/pull/41

Plots for increasing K back in, though without some of the pretty axis labels
Changed mpg (mtcars) so ? is nan
Added mpg-nona with all nan rows removed to avoid complexity

closing for now to clear the board; will reopen for discussion at future date


[rtext] DrawTextCodepoint() doesn't check if font is valid, which causes SIGSEGV

Issue description
Calling DrawTextCodepoint (and other functions working on fonts, I assume) with an invalid Font structure causes SIGSEGV, presumably due to the following line in the GetGlyphIndex function:
if ((index == 0) && (font.glyphs[0].value != codepoint)) index = fallbackIndex;
Causing this SIGSEGV is easy, as LoadFont doesn't indicate that it failed with an error, and IsFontValid is not used in the examples after loading fonts.
Possible solutions:

DrawTextCodepoint and similar functions call IsFontValid instead of assuming that font is valid.
DrawTextCodepoint and similar functions check font.glyphCount>0 instead of assuming that font is valid.
A boolean flag called isValid is added to the Font structure, indicating that the font is valid, and functions like DrawTextCodepoint check it each time they're called.
LoadFont returns an int which indicates that loading the font failed.
IsFontValid is shown in all examples.

Environment
Linux 6.11
Issue Screenshot

Code Example
int main(void) {
    Font font = LoadFont("font.ttf"); // a non-existent font
    printf("Glyph count = %d\n", font.glyphCount); // 0
    InitWindow(800, 600, "title");
    while (!WindowShouldClose()) {
        BeginDrawing();
            DrawTextCodepoint(
                font,
                'a',
                (Vector2){100,100},
                100,
                GREEN
            );
        EndDrawing();
    }
}

@DenizBasgoren thanks for reporting, fixed!


[ 1.6.0 ] Error with some devices (Android 5.0 and 6.0)

Hi.
I notice some devices, specially between Android 5.0 (Lollilop) API 21 and 6.0 (Marshmallow), the ViroViewScene just doesn't works.
The layout for it, that works flawless in Android 7.0 and 8.0, for devices with same capabilities and specs, stay white in 5/6 versions.
Some devices for quick info: Lenovo Vibe B (6.0), Samsung J5 and Samsung J5 Pro (5.0/60) and few others that are not bad phones.
I found that Lenovo and Samsung uses Mali as GPU.
This output appears every second in Android Studio "Run" window:
I/BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) queueBuffer: slot 0 is dropped, handle=0xaf850640
I/BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) queueBuffer: slot 1 is dropped, handle=0xaf850780
I/BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) queueBuffer: slot 0 is dropped, handle=0xaf850640
I/BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) queueBuffer: slot 1 is dropped, handle=0xaf850780
I/BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) queueBuffer: slot 0 is dropped, handle=0xaf850640

In the logcat, some info:
04-29 18:21:20.892 13087 13148 I Viro    : GPU vendor [ARM], renderer [Mali-T720]
04-29 18:21:20.892 13087 13148 I Viro    :    Detected antiquated Mali-T720 GPU, rendering will be limited

P.S.: This Mali-T720 support even OpenGL | ES 3.1, it's not so antiquated.
And more logcat:
04-29 18:21:20.910 13087 13148 I Viro    : Creating render targets with configuration:
04-29 18:21:20.910 13087 13148 I Viro    : [MRT supported:   0]
04-29 18:21:20.910 13087 13148 I Viro    : [Shadows enabled: 0]
04-29 18:21:20.910 13087 13148 I Viro    : [HDR supported:   0, HDR enabled:   0]
04-29 18:21:20.910 13087 13148 I Viro    : [PBR supported:   0, PBR enabled:   0]
04-29 18:21:20.910 13087 13148 I Viro    : [Bloom supported: 0, Bloom enabled: 0]
04-29 18:21:20.911 13087 13148 I BufferQueueProducer: [SurfaceTexture-0-13087-0](this:0x9eb7f400,id:0,api:1,p:13087,c:13087) new GraphicBuffer needed

So, the screen/layout stays with the Android layout background color and doesn't shows nothing. Using PBR, HDR or not using it at all, just a cube, nothing.
Do Viro uses some mapping for GPU devices that can run or not run, or this is made by runtime?
How can I trust that someone with a device using a different GPU will be able to purchase and run the app in Google Play Store and will work?
Or Android 5.0 and 6.0 is (and won't) be supported by Viro yet?
My target is API 21+, I will only use ViroViewScene, no AR or VR. Just 3d scene with HDR/PBR.
Thanks!

Hi @rgipd, we've identified some issues with our Mali support and are actively looking into improving our support for Mali now, in time for our next release.
The target minimum version for Viro is Android 5.0 (API 21). Viro does perform some mapping of GPU devices at runtime, but it doesn't stop any device from running. At most, Viro will disable certain features (e.g. HDR, PBR) for older GPUs.

Hi @rgipd, that shouldn't happen with Mali GPUs -- is there a test scene you can send me that's failing on your end? So far we've fixed one Mali driver bug, in that Mali cards don't correctly report when they've acquired an sRGB framebuffer, resulting in over-saturation. However, the bug you're describing seems different. I'm testing currently with a Mali-T830.

@radvani I just initialize the ViroView (in layout xml or in the source code, same issue in both cases).
My test code is nothing different from the "Getting Started" code, just positioned the camera, a box and setup the scene to use HDR background and PBR chromed material in the box.
When I run it, just show blank/white (that's the layout background).
But happened in all devices with Android 5.0 and Android 6.0 that I tested (all devices have Mali).
In the Android ARM emulator (that runs after 1 hour to boot the apk), I could see that ViroView has initialized. So it's a problem in real devices. Just with Mali, or maybe (hope not) with another GPU models.

Hi @rgipd, I think I know what's wrong there: are all those devices using the Mali-T720 specifically? We had an errant check in our code (which you noticed above, in your initial post) where we disable a lot of rendering features for that GPU. I believe that was causing the issues; we've removed this for our next release.

@radvani  Not just Mali T720. Mali-T830 MP1 and Mali-T830MP2 as well in another 2 devices I could test.
There are any way in code to know when all rendering is disabled so I can put a message to the user that the device is not supported?
When do you think the next release will be out?
Thanks.

@rgipd The rendering features disabled is being removed for all cards except old Adreno GPUs, which have significant driver issues (that is, Adreno330 and earlier), so you won't run into this after the next release. In the current release this only occurs with Mali T720 and those older Adreno GPUs. I'm testing other Mali drivers now to ensure they work correctly. The next release is targeted for mid to late May.

@radvani Ok, please let me know :)

This was fixed in ViroCore 1.8. Please re-open if you are running into issues


Arkansas deployment issue

Description
Last week, Arkansas had an issue where they were unable to deploy the most recent version of CiviForm.
First, they had an error because they were on the "latest" version of the cloud deployment infra repo, which was failing. We had them pull the most recent version of the fork and then change their version to the same one as the CIVIFORM_VERSION.
They upgraded the postgres version as part of the release and that went fine and we confirmed that the secret variables were the correct length.
We were unable to reproduce the issue, but they saw this both on staging and production. For each instance, the deployment would start and then they'd see an error about the health checks failing. The deployment would keep trying until it eventually would fail and rollback to the last release.

@nb1701 @rockycodes and I worked with Terry to troubleshoot this issue and we eventually figured out that the timeout for the health checks weren't long enough.
After merging these PRs, we had Terry change the deployment version to "latest" and deploy and the issue was resolved:

https://github.com/civiform/cloud-deploy-infra/pull/331
https://github.com/civiform/cloud-deploy-infra/pull/333
https://github.com/civiform/cloud-deploy-infra/pull/334

We then had him change it back to the CIVIFORM_VERSION for the next release.
We have since created a couple of additional issues related to this:

https://github.com/civiform/civiform/issues/7698
https://github.com/civiform/civiform/issues/7697


How to validate covid cert?

In our flutter app we need to validate covid cert. But I’m a bit lost. Is there an guideline or how to validate these? Maybe an piece of code.

Hi @aBuder , Business rules form only a small part of the process of validating a DCC (and it's pretty much right at the end). You might want to start here: https://github.com/ehn-dcc-development/overview
(It's a work-in-progress, so still a bit rough around the edges, but the detailed content behind the links is solid, and helpful.)

Since you're talking about Flutter specifically, I think I can help you a bit further. We have the package on pub.dev https://pub.dev/packages/certlogic_dart
How we handle it in our app is doing the following

validationRules is a List<ValidationRule>? containing a list of certlogic json data example of rules
actualValidationDateTime is today, but we can switch it for debugging purposes
data is a Map<String, dynamic> containing the valuesets etc
We first check if the rule is still valid (we can update these remotely so we can switch on a specific date if the law changes).
We throw away all null's (passed rules)
In the end, we have an array with all failed rules. These contain human-friendly descriptions that we can show depending on the use-case (for example police can see why it failed, but someone at a bar can't).

    final data = {
      'payload': payload,
      'external': {
        'valueSets': _valueSets.toJson(),
        'validationClock': validationDateTime.toIso8601String(),
      },
    };
    final failedRules = validationRules
        ?.map((rule) {
          if (rule.validFrom.isAfter(actualValidationDateTime) || rule.validTo.isBefore(actualValidationDateTime)) return null;
          return CertLogic.evaluate(rule.logic, data) as bool == true ? null : rule;
        })
        .whereType<ValidationRule>()
        .toList();
    return failedRules ?? [];

Hope this helps

I'm closing this issue. If you have any further questions, you know where to find us, I assume.


Cannot find the file specified on install

When I tried to install with pip I got the following error.
C:\Users\User\Downloads>pip install oursql
Collecting oursql
  Using cached https://files.pythonhosted.org/packages/8c/88/9f53a314a2af6f56c0a1249c5673ee384b85dc791bac5c1228772ced3502/oursql-<IP_ADDRESS>.tar.gz
Installing collected packages: oursql
  Running setup.py install for oursql ... error
    Complete output from command c:\python27\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\user\\appdata\\local\\temp\\pip-install-_ufglh\\oursql\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\user\appdata\local\temp\pip-record-r9ceez\install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    skipping 'oursqlx\oursql.c' Cython extension (up-to-date)
    error: [Error 2] The system cannot find the file specified

    ----------------------------------------
Command "c:\python27\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\user\\appdata\\local\\temp\\pip-install-_ufglh\\oursql\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\user\appdata\local\temp\pip-record-r9ceez\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\user\appdata\local\temp\pip-install-_ufglh\oursql\

You need Cython to compile oursql because the author has forgotten to commit generated modules.
As the author of oursql seems to disappear I forked the module, committed missed generated files and applied pull requests. I doubt I will develop it further but I can accept pull requests.


helm repo upadte error when chart appVersion is a quoted scientific number

When running command helm repo update, an error occurs , the output as follows:
Unable to get an update from the "market-test" chart repository (http://${ip:port}/chartrepo/opensource-stable):
error unmarshaling JSON: while decoding JSON: json: cannot unmarshal number into Go struct field ChartVersion.entries.appVersion of type string
I found this issue is induced by a chart uploaded before, because the appVersion in chart.yalm is a quoted scientific value.
the content of Chart.yaml is :
apiVersion: v2
appVersion: "1826e29"
description: A Helm chart for Kubernetes
name: admin
type: application
version: 0.1.0

I want the appVersion to be 1826e29,  if it's not qouted,  helm push will change the value to 1.826e+32.
Is there a way to keep the wanted version, and no errors ?
Output of helm version:
version.BuildInfo{Version:"v3.6.0", GitCommit:"7f2df6467771a75f5646b7f12afb408590ed1755", GitTreeState:"clean", GoVersion:"go1.16.3"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.6", GitCommit:"72c30166b2105cd7d3350f2c28a219e6abcd79eb", GitTreeState:"clean", BuildDate:"2020-01-18T23:31:31Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.6", GitCommit:"72c30166b2105cd7d3350f2c28a219e6abcd79eb", GitTreeState:"clean", BuildDate:"2020-01-18T23:23:21Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.):

This issue has to do with YAML. The YAML spec covers floating point numbers using scientific notation.
appVersion: 1826e29

The above example should be converted to a number. This is proper YAML. The below example should be a string...
appVersion: "1826e29"

If you are not seeing this than an issue needs to be filed against the YAML parser. Helm uses the same one Kubernetes does. It is maintained by the Kubernetes project.

@mattfarina  , As you said , appVersion: "1826e29"  should be a string , however, the error  " cannot unmarshal number into Go struct field ChartVersion.entries.appVersion of type string"   hints the yaml parser view it as a number.
Is the version of yaml used by helm 3.6 still older ?
there is a similar case and solution  at https://github.com/helm/chartmuseum/issues/290

The version of yaml used by Helm is from the k8s project and the latest version they have released.
Are you seeing the issue with helm chart push?

Firstly , the issue in  helm/chartmuseum#290  happened with  helm repo add
Secondly,  do you read  codes and locate where this error occurs?
helm repo update : https://github.com/helm/helm/blob/v3.6.0/cmd/helm/repo_update.go#L98
helm repo add:  https://github.com/helm/helm/blob/v3.6.0/cmd/helm/repo_add.go#L183
Both errors throws by the same function   DownloadIndexFile() at
https://github.com/helm/helm/blob/v3.6.0/pkg/repo/chartrepo.go#L117

The metadata from the Chart.yaml is also in the index.yaml file. In the index.yaml file is the appVersion quoted for those with a scientific number? If not, I'd like to know what version of Helm produced the index.yaml file.

Yeah, thank you for giving me an clue. I tried locally with helm repo index (helm v3.6) , the  appVersion   in  index yaml is quoted, this is we expected, then I checked the index.yaml in real chart repo  ,it's not quoted.  I will inspect the  chart repo server proplem.


Subscripts and superscripts show too big in Firefox

Hi,
I'm trying to upgrade to Mathjax 3 in my Jekyll website, in which I'm currently using Mathjax 2.7.6. My javascript is:
<script>
MathJax = {
  tex: {
    tags: "ams",
    packages: {'[+]': ['ams', 'noerrors', 'color']}
  },
  chtml: {
    scale: 1.05,
    minScale: 1
  },
  options: {
    ignoreHtmlClass: 'tex2jax_ignore',
    processHtmlClass: 'tex2jax_process',
    renderActions: {
      find_script_mathtex: [10, function (doc) {
        for (const node of document.querySelectorAll('script[type^="math/tex"]')) {
          const display = !!node.type.match(/; *mode=display/);
          const math = new doc.options.MathItem(node.textContent, doc.inputJax[0], display);
          const text = document.createTextNode('');
          node.parentNode.replaceChild(text, node);
          math.start = {node: text, delim: '', n: 0};
          math.end = {node: text, delim: '', n: 0};
          doc.math.push(math);
        }
      }, '']
    }
  },
  loader: {
    load: ['[tex]/ams', '[tex]/noerrors', '[tex]/color']
  }
};
</script>
<script src="https://polyfill.io/v3/polyfill.min.js?features=es6"></script>
<script type="text/javascript" id="MathJax-script" async<EMAIL_ADDRESS>
Maths look good in Chrome and Safari, but not Firefox. Without setting the cthml option parameters, math font size shows extremely small in Firefox (but not in other browsers). The font size can be enlarged with those parameters (I've tried other combinations, such as  minScale=0.5 and matchFontHeigh=false) but then the font size for subscripts and superscripts is just too large, and looks ugly.
For instance:
In all browsers, including Firefox, when using MathJax 2.7.6 with the following config:
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({ TeX: { extensions: ["color.js"], equationNumbers: { autoNumber: "AMS" }}, CommonHTML: {scale: 105, minScaleAdjust: 100}});
</script>

Same equation when changing to Mathjax 3, in Firefox 73 for macOS:

This latter equation, with Mathjax 3, looks just as it did with MathJax 2.7.6 in Chrome and Safari, so it seems a Firefox-only issue.
Any idea on how to fix that?

I am not able to reproduce the result.  Can you give an example page that exhibits the problem?

Hi @dpvc,
The page is at https://gnss-sdr.org
The code is at https://github.com/gnss-sdr/geniuss-place

Sorry, I meant a page that is running MathJax v3.  This one is at 2.7.6, which doesn't exhibit the problem.

I don't have an example online, since I couldn't fix this issue for Firefox.
I found this issue by replacing in the online example the call to MathJax 2.7.6 by MathJax 3. If it is of help, I can create a simple demo page and put it online.

If it is of help, I can create a simple demo page and put it online.

Yes, that would help, thanks.

Ok, there it is. I've reproduced the same website, but now using MathJax 3.
Check it online, for instance: https://carlesfernandez.github.io/web-test/docs/sp-blocks/observables/
In this instance, I still can see differences in the subscript/superscript font size in Firefox with respect to other browsers.

OK, thanks.  I see the same thing now, but haven't yet figured out what is the cause.  Can you leave the site up for a bit?

Yes, no problem, I will leave it online.
Thanks for looking into it!

OK, I've figured out what is going on.
MathML (the internal format for MathJax) has a value that specifies the minimum font size that MathJax will use for super- and subscripts, and it is set to 8px by default.  Now, MathJax does all its measurements internally in relative units of ems, and number of pixel per em is determined by matching the MathJax fonts to the surrounding font by making their ex-heights the same.  MathJax does that by measuring the ex-height of the surrounding font (in pixels).  That measurement is "fragile" in some ways, and it does rely on the surrounding font having a correct ex-height and MathJax being able to measure it properly.
It turns out that either your main text font has a bad ex-height, or Firefox is getting the ex-height wrong, because the ex-height is not correct for that font in Firefox (but is own other browsers).  Font metrics are a real mess, and most fonts include separate metrics for Windows and Mac environments, and it may be that these are not consistent in this font, and Firefox is using one of them while other browses are using the other.  I haven't tried to look any further than that.  I have seen similar issues in Google fonts in the past.
In any case, one solution is to change the script minimum size to be in ems rather than pixels, so that the px-per-em value isn't used.  That is probably the easiest work around for now.  Use
<script>
MathJax = {
  startup: {
    ready() {
      const MmlMath = MathJax._.core.MmlTree.MmlNodes.math.MmlMath;
      MmlMath.defaults.scriptminsize = '.1em';
      MathJax.startup.defaultReady();
    }
  }
}
</script>

I think this will work around the problem for now.

I'm marking this as "upstream" as I suspect it is really the ex-height setting in the Ovo font that is the problem, and I don't really have another label that makes sense.

This solution works great. Thank you!


local nav server - added driving directions

Description [](A description of the refactor, including the goals it accomplishes.)
Added Navigation driving directions to the local nav web server.
Saves a lightweight json version of the navd mapbox routing pulling out unnecessary components.
Adds to the json, Current Destination and Current Step.
New tpl for javascript to display information on the screen.
After entering destination through webserver, driving directions will immediately show up.
If refreshed, webpage will display driving directions if param NavDestination is populated, in the case where the user enters the address directly onto the Comma device.
Displays Current Destination, and clearer step text while removing previous steps once completed.
Refreshes every 5 seconds for new data.
Known Limitations:
Does not decrease distance as you drive during that step, I did not want to add additional code to support this to make it lighter.
No dark mode, so the webpage is still using white background and black text. A bit jarring
Without Comma device in the onroad state, entering new destination will not refresh destination or directions since navd does not attempt to pull new mapbox api until device is in the onroad state.
I am not a programmer by trade, this code works for me but can always be optimized.
Verification [](Explain how you tested the refactor for regressions.)
Works fine on my end during my testing period of over a week. Used mobile iOS safari browser as well as chrome loaded onto my infotainment system.
-->

Seems like force pushing got this automatically closed again... Will ping you and open another PR with this change once I get to this on dev-c3 (cherry-picking should retain the commit history in the new PR). Thanks for the PR! 🎉


Asynchronous error handling

According to final specification (2015-05-08):
2.5.6 Error handling
<IP_ADDRESS> Queue error handling
<IP_ADDRESS> Buffer Synchronization Rules
<IP_ADDRESS> Image Synchronization Rules
3.6.1 Error Handling Rules

I don't believe this feature to be instrumental to the basic working implementation - synchronous errors can be used instead - so I'm reducing it's importance.


Tuple documentation mentions deconstructing declaration but not deconstructing assignment

In both articles about tuples and tuple deconstruction at docs.microsoft.com (https://docs.microsoft.com/en-us/dotnet/csharp/deconstruct, https://docs.microsoft.com/en-us/dotnet/csharp/tuples) all the examples about tuple deconstruction are indeed deconstruction declarations, there is no single example on deconstruction assignment. There is actually no mention that deconstruction is possible into existing variables, so the new users don't expect it to be possible.
I would also suggest including the canonical example from other languages with tuples: swap one-liner
(a, b) = (b, a);

This is a great suggestion. The updates should also mention it's great for constructors in record-like types:
public class Person
{
    public string LastName  { get; }
    public string FirstName { get; }

    public Person(string first, string last) => (FirstName, LastName) = (first, last);
}


⚠ Mobitopup has degraded performance

In ed225f4, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 4195 ms

Resolved: Mobitopup performance has improved in ea259de after 423 days, 2 hours, 53 minutes.


The "-legacy" option does not appear to work with the "-version" option.

Using the "-legacy" and "-version" options together does not work.  It always outputs all legacy versions.  Either these options should actually work together -OR- using them together should produce an error.

-latest and -version do work together but only if you have VS installed because the parser for the version range is implemented by the COM API that the VS installer installs. -latest works by sorting the legacy versions in descending order as seen in https://github.com/Microsoft/vswhere/blob/develop/src/vswhere.lib/InstanceSelector.cpp#L145-L156.
At this time, I have no plans to duplicating the version range parsing logic in vswhere to handle when the COM API is not installed. What scenario are you trying to support that you need to specify a version range with only legacy products installed?

I was trying to query for only Visual Studio 2015 using "-version 14.0"; instead it returned 2010 to 2015.  Perhaps you could add a small check to see if the COM API is installed and disallowed -version with -legacy in that case?


add php 8.3 & Sylius 1.13.0 compatibility

I'm not able to install the plugin using php 8.3 and Sylius 1.13.0.

hi @netrising
The last version https://github.com/monsieurbiz/SyliusMenuPlugin/releases/tag/v1.4.0 might solve your issue

I confirm, this is working fine with Php 8.3 and Sylius 1.13.7


Automatically download correct bootloader based on selected firmware

@Jason2866 wrote here:

A nice feature would be: WebTools scans firmware (magic first bytes https://github.com/esphome/esphome-flasher/blob/main/esphomeflasher/common.py#L143-L153) which Bootloader file and other files are needed and downloads it from espressif github repo. There is no need to provide this file from user side. Only partitions file is specific
Example for https://github.com/esphome/esphome-flasher/tree/main/esphomeflasher

Websites cannot randomly download files from other websites and process the content due to CORS security restrictions. Our ESPHome Flasher doesn't have that restriction because it's in Python.
Since each project defines their own manifest to use ESP Web Tools, they are also able to include the other files.
Maybe someone could make a CLI tool to generate a manifest based on an ESP32 firmware and download the right files?

Or the files should just be included in a single file, just as I do for ESPEasy.
https://github.com/letscontrolit/ESPEasy/blob/mega/tools/pio/post_esp32.py
It is just included in the PlatformIO build script.

@TD-er I "borrowed" your platformio script. Will try to use with web-tools.
Every flash and speed option should be possible this way.

@balloob  wrote

Websites cannot randomly download files from other websites and process the >content due to CORS security restrictions in the browser. Our ESPHome Flasher doesn't >have that restriction because it's in Python.

Since the files are always the same. It must not be a random website where to download. I am a total Web coding and restrictions noob. Is there no way to download files from a hardcoded repo?
User specific file is only the partition.bin. This file needs to be provided in any case from the user as the firmware. All other files are static

We can only download files from the same host. It means you have to host them easier. I think it’s better if ESP Web Tools not dive into knowing how to find the right files and bundle them. That is up to the user to gather the right files to install.
Open a new issue for feature requests.

Or the files should just be included in a single file, just as I do for ESPEasy<EMAIL_ADDRESS>It is just included in the PlatformIO build script.

I have just added this to ESPHome via https://github.com/esphome/esphome/pull/3003

I just linked in that PR to the latest version of the script, now actually working on flash chips other than 4M in size ;)  (or at least with partitions extending the 4M limit)

Going to close this. As it’s not ESP Web Tools responsibility. It could be a responsibility of an ESP Web Tools manifest creator tool, but that’s out of scope for this repo.

100% fine with closing, since it is possible and working to flash a one file firmware
(since no file patching is done anymore).

@TD-er sorry if I resume this old issue but I found it very useful, your solution with platformio is smart. :)
I can create the factory firmware and I can flash a single binary file with ESP Web Tools no problem but I have problems when I try to flash that firmware via OTA.
ESP32 complains about a "Wrong Magic Byte"...
Any idea on how to fix this using your POST script?

@sblantipodi You can not flash via OTA the factory firmware. You have to use the firmware without factory in the name

No, since the OTA part is Arduino core. If you want a workaround you have to change the OTA code there.