Specifying delivery delays per subscription

Is your enhancement request related to a problem? Please describe.
I am unable to setup delayed deliveries per subscription. In other words, the current approach to delaying messages does not conform to following diagram (taken from official docs).

This diagram suggests that delayed messages are tracked per subscription. However, it seems that the delay can only be set by producer.
producer.newMessage().deliverAfter(3L, TimeUnit.Minute).value("Hello Pulsar!").send();

This, IMO, is a contradiction in the design, as the producer in bound a topic, while the delay tracker is bound to a subscription.
This poses a problem to pull-based (as opposed to push-based) architectures, where producers are not aware of the consumption logic (producers are just announcers). There might be many different logics that are bound to the same message topic, some of them may need to be delayed, some may not. It's the consumer (subscription to be exact) that knows whether the messages should be delayed or not, as the diagram correctly suggests.
Describe the solution you'd like
It'd be nice to be able to configure a default delay per subscription. The subscription delay can be overridden using per message delay.
Describe alternatives you've considered
In order to emulate the desired behavior, the consumer has to receive every message and then dispatch them to a retry topic. This introduces unnecessary processing overhead, assuming that delayed deliveries are in fact tracked per subscription which makes pulsar capable to handling this internally.

Closed as stale and no one worked on it. Please open a new issue if it's still relevant to the maintained versions.


Consider listing limitations in the module documentation or readme

Hey, cool project!   Unfortunately, after sinking in quite a bit of effort, figured out it has some issues that make it unusable for my usecase.  It would be good to document these in the haddock / readme:

Polymorphism doesn't work (#1)

Looks like dependencies check will infinite loop on mutual recursion

The minor issue of quotes in the datatype is definitely a problem, but probably easily resolved #4

It would be good to document that the package is work-in-progress or partially complete, because, well, someone like me might come along and spend hours trying to get it to work for their usecase.
I would try to fix it (already fixed some things in https://github.com/mikeplus64/aeson-flowtyped/pull/3), but the use of generics / functor fixpoints / functor composition make it a real head scratcher.  I think it would be better to:

Use TH instead of generics. Using aeson's TH would be a good starting point, because then you can be sure the logic is related.
Use a plain ol recursive datatype for flow types, and avoid pattern synonyms.  Using fancy stuff makes it much less likely that others will understand and contribute.

Use TH instead of generics. Using aeson's TH would be a good starting point, because then you can be sure the logic is related.

It would be nice to reuse TH code from aeson, but I don't really see any exports from there that can be used?

Use a plain ol recursive datatype for flow types, and avoid pattern synonyms. Using fancy stuff makes it much less likely that others will understand and contribute.

Doable but likely a lot of effort... Right now FlowTypeF is only put to real use with the Info tags that are passed around, which are only used for detecting nullary constructor alternatives like data A = A | B | C.

It would be nice to reuse TH code from aeson, but I don't really see any exports from there that can be used?

It'd be more of a copy+modify kind of re-use.
RE the fixpoint stuff I was mostly just a bit grumpy because I couldn't figure out how to get this working for my usecase.  It seems likely that I will revisit generating flow types sometime in the next month or two, and at that point will either try to get this working with polymorphism or use TH

hm, I can't remember why I didn't use this in the first place, but recursion-schemes has got a makeBaseFunctor  which would generate FlowTypeF from a simpler/more idiomatic data FlowType = ... definition. That would probably simplify all the code that doesn't need "extra stuff" like FlowTypeI does.


Consider running the Infra: CVE checks with every commit as well

Issue submitter TODO list

[X] I've searched for an already existing issues here
[X] I'm running a supported version of the application which is listed here and the feature is not present there

Is your proposal related to a problem?
Currently, the "Infra: CVE checks" check is configured to run twice per month, and while that is great, it does not raise the constant awareness that CVE should raise.
For example, the latest run failed  but it is only known if we navigate to the specific build check while main is considered to be healthy
Describe the feature you're interested in
We should extend the cve_checks.yml workflow to also run on pull requests and merges to main. It should continue to be a separate check

Pull requests: Because we should not be introducing new dependencies with CVEs
On Main : To raise awareness and serve as a continuous reminder that actions may be needed

Describe alternatives you've considered
Use the existing schedule and remember to check manually
Version you're running
318bcc91ddcca72b6a280d20f5f51ea85dac77c0
Additional context
No response

Pull requests: Because we should not be introducing new dependencies with CVEs

I don't mind this one, would be nice to know if there's anything new in case of newly introduced dependencies.

On Main : To raise awareness and serve as a continuous reminder that actions may be needed

This one is questionable, considering we don't have much bandwidth to fix these issues too often. Like, #256 was open for half a year. Running this on every commit sounds like an extra noise.
@yeikel what do you think?

Pull requests: Because we should not be introducing new dependencies with CVEs

I don't mind this one, would be nice to know if there's anything new in case of newly introduced dependencies.

On Main : To raise awareness and serve as a continuous reminder that actions may be needed

This one is questionable, considering we don't have much bandwidth to fix these issues too often. Like, #256 was open for half a year. Running this on every commit sounds like an extra noise.
@yeikel what do you think?

I agree that it would be noise if it not actioned, but I think that this is good noise. We should be more than aware than issues with CVE exist and I think that every commit is a good way to show that

Pull requests: Because we should not be introducing new dependencies with CVEs

I don't mind this one, would be nice to know if there's anything new in case of newly introduced dependencies.

On Main : To raise awareness and serve as a continuous reminder that actions may be needed

This one is questionable, considering we don't have much bandwidth to fix these issues too often. Like, #256 was open for half a year. Running this on every commit sounds like an extra noise.
@yeikel what do you think?

I agree that it would be noise if it not actioned, but I think that this is good noise. We should be more than aware than issues with CVE exist and I think that every commit is a good way to show that

aight, @yeikel wanna raise a PR for these changes?


Services are shown as child devices

Services are shown as child devices

It's weird though that the bridge is show correctly as a service:

To Reproduce
I used the tedge-rugpi-image repo to create a pi3 image, lashed it to an sd card and followed the instruction to register the device.
Expected behavior
After registration there are child device that should be displayed as services.
Environment (please complete the following information):

OS rugpi image release 20231221.2255
pi3b
arm64
thin-edge.io

@mstoffel-sag Can you please add the output of the MQTT messages using the following command (wait 5 seconds, then kill the command, then post the console output)
tedge mqtt sub 'te/+/+/+/+'

We suspect that there is something incorrect with the order of messages which is causing the services to be registered a child devices.

Here's the output:
root@rpi3-b827eb596700 ~ # tedge mqtt sub 'te/+/+/+/+'
INFO: Connected
[te/device/main/service/tedge-mapper-collectd] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-agent] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-mapper-c8y] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/mosquitto-c8y-bridge] {"@id":"rpi3-b827eb596700:device:main:service:mosquitto-c8y-bridge","@parent":"device/main//","@type":"service","name":"mosquitto-c8y-bridge","type":"service"}

Thanks @mstoffel-sag. The team has been able to reproduce the issue and analyzing the results.

The issue is triggered by SmartREST message 104. Despite being described as "Set a status for given software service." , if 104 is sent before registering a service, it actually creates a child device.

The issue is triggered by SmartREST message 104. Despite being described as "Set a status for given software service." , if 104 is sent before registering a service, it actually creates a child device.

Good find. So we're going to have to make sure we hold off sending 104 messages until the 102 is sent.

@Bravo555 @albinsuresh
After investigating the image created by tedge-rugpi-image, it seems that disable the services before tedge connect c8y is run seems to fix some of the services being created, however there is an additional side effect now where an additional child device called operations was created erroneously, though I created a separate ticket for this: https://github.com/thin-edge/thin-edge.io/issues/2606
So it seems that starting up the services before tedge connect c8y is run is definitely causing unexpected issues (that need to be fixed).

Procedure
The following repositories/branches were used to build test images.

Name
Branch

tedge-rugpi-image
feat-debug-example

tedge-rugpi-core
feat-debug-logger

Then the following procedure was used for each test case:

Delete the existing device

c8y devices delete --id rpi4-d83add90fe56 --cascade

Delete entries from the known_hosts file
sed -i '/rpi4-d83add90fe56/d' /Users/reubenmiller/.ssh/known_hosts

Create the rugpi image

Flash image to SD card

SSH onto device and check that services are either started/stopped
ssh<EMAIL_ADDRESS>
Check service status
systemctl status tedge-agent
systemctl status tedge-mapper-c8y
systemctl status tedge-mapper-collectd
systemctl status "tedge-*"

Create a bootstrapping entry (to make it visible in the logs when bootstrapping is occurring)
tedge mqtt pub 'debug/bootstrapping' 'bootstrapping soon'

Bootstrap device
c8y tedge bootstrap<EMAIL_ADDRESS>

Copy the mqtt log output to your local machine
scp root@rpi4-d83add90fe56.local:/data/mqtt.log ./mqtt-autostart-disabled.log

Test case 1: Auto start disabled
mqtt-autostart-disabled.log
Before bootstrapping
# tedge mqtt sub '#'
INFO: Connected
[te/device/main///twin/c8y_Hardware] {"sysClockSync":"no","model":"Raspberry Pi 4 Model B Rev 1.5","revision":"-d03115","serialNumber":"1000000044d70329"}
[te/device/main///twin/firmware] {"name":"tedge_rugpi_tryboot","version":"20240122.1045","updated":"Mon 11 Dec 04:55:24 GMT 2023"}
[te/device/main///twin/device_OS] {"displayName":"Debian GNU/Linux 12 (bookworm)","version":"12","hostname":"rpi4-d83add90fe56","arch":"aarch64","family":"GNU/Linux","kernel":"#1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24)","sysClockSync":"no"}

Note tedge-inventory service is still configured for autostart
Test case 2: Auto start enabled
mqtt-autostart-enabled.log
Before bootstrapping
# tedge mqtt sub '#'
INFO: Connected
[te/device/main///cmd/config_snapshot] {"types":["tedge-configuration-plugin","tedge-log-plugin","tedge-mosquitto.conf","tedge.toml"]}
[te/device/main///cmd/config_update] {"types":["tedge-configuration-plugin","tedge-log-plugin","tedge-mosquitto.conf","tedge.toml"]}
[te/device/main///cmd/log_upload] {"types":["c8y_Command","firmware_update","software-management"]}
[te/device/main///cmd/firmware_update] {}
[te/device/main///cmd/restart] {}
[te/device/main///cmd/software_list] {}
[te/device/main///cmd/software_update] {}
[te/device/main///twin/device_OS] {"displayName":"Debian GNU/Linux 12 (bookworm)","version":"12","hostname":"rpi4-d83add90fe56","arch":"aarch64","family":"GNU/Linux","kernel":"#1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24)","sysClockSync":"no"}
[te/device/main///twin/firmware] {"name":"tedge_rugpi_tryboot","version":"20240122.1021","updated":"Mon 11 Dec 04:55:20 GMT 2023"}
[te/device/main///twin/c8y_Hardware] {"sysClockSync":"no","model":"Raspberry Pi 4 Model B Rev 1.5","revision":"-d03115","serialNumber":"1000000044d70329"}
[te/device/main/service/tedge-agent] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-agent/status/health] {"pid":772,"status":"up","time":1702270504.030202}
[te/device/main/service/tedge-mapper-collectd] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-mapper-collectd/status/health] {"pid":794,"status":"up","time":1702270504.0413673}

After bootstrapping
# tedge mqtt sub '#'
INFO: Connected
[te/device/main///cmd/config_snapshot] {"types":["tedge-configuration-plugin","tedge-log-plugin","tedge-mosquitto.conf","tedge.toml"]}
[te/device/main///cmd/config_update] {"types":["tedge-configuration-plugin","tedge-log-plugin","tedge-mosquitto.conf","tedge.toml"]}
[te/device/main///cmd/log_upload] {"types":["c8y_Command","firmware_update","software-management"]}
[te/device/main///cmd/firmware_update] {}
[te/device/main///cmd/restart] {}
[te/device/main///cmd/software_list] {}
[te/device/main///cmd/software_update] {}
[te/device/main///twin/device_OS] {"displayName":"Debian GNU/Linux 12 (bookworm)","version":"12","hostname":"rpi4-d83add90fe56","arch":"aarch64","family":"GNU/Linux","kernel":"#1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24)","sysClockSync":"no"}
[te/device/main///twin/firmware] {"name":"tedge_rugpi_tryboot","version":"20240122.1021","updated":"Mon 11 Dec 04:55:20 GMT 2023"}
[te/device/main///twin/c8y_Hardware] {"sysClockSync":"no","model":"Raspberry Pi 4 Model B Rev 1.5","revision":"-d03115","serialNumber":"1000000044d70329"}
[te/device/main///twin/c8y_Agent] {"name":"thin-edge.io","url":"https://thin-edge.io","version":"1.0.0-rc.2~116+g02d20ea"}
[te/device/main/service/tedge-agent] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-agent/status/health] {"pid":772,"status":"up","time":1705916358.6633723}
[te/device/main/service/tedge-mapper-collectd] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-mapper-collectd/status/health] {"pid":794,"status":"up","time":1705916358.663361}
[te/device/main/service/tedge-mapper-c8y] {"@parent":"device/main//","@type":"service","type":"service"}
[te/device/main/service/tedge-mapper-c8y/status/health] {"pid":2349,"status":"up","time":1705916358.6610012}
[te/device/main/service/mosquitto-c8y-bridge] {"@id":"rpi4-d83add90fe56:device:main:service:mosquitto-c8y-bridge","@parent":"device/main//","@type":"service","name":"mosquitto-c8y-bridge","type":"service"}
[te/device/main/service/mosquitto-c8y-bridge/status/health] 1

file: entity_store.jsonl
{"version":"1.0"}
{"topic":{"name":"te/device/main///twin/c8y_Agent"},"payload":"{\"name\":\"thin-edge.io\",\"url\":\"https://thin-edge.io\",\"version\":\"1.0.0-rc.2~116+g02d20ea\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main///twin/c8y_Hardware"},"payload":"{\"model\":\"Raspberry Pi 4 Model B Rev 1.5\",\"revision\":\"-d03115\",\"serialNumber\":\"1000000044d70329\",\"sysClockSync\":\"no\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main///twin/c8y_Firmware"},"payload":"{\"name\":\"tedge_rugpi_tryboot\",\"updated\":\"Mon 11 Dec 04:55:24 GMT 2023\",\"version\":\"20240122.1045\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main///twin/device_OS"},"payload":"{\"arch\":\"aarch64\",\"displayName\":\"Debian GNU/Linux 12 (bookworm)\",\"family\":\"GNU/Linux\",\"hostname\":\"rpi4-d83add90fe56\",\"kernel\":\"#1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24)\",\"sysClockSync\":\"no\",\"version\":\"12\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main/service/mosquitto-c8y-bridge"},"payload":"{\"@id\":\"rpi4-d83add90fe56:device:main:service:mosquitto-c8y-bridge\",\"@parent\":\"device/main//\",\"@type\":\"service\",\"name\":\"mosquitto-c8y-bridge\",\"type\":\"service\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main/service/tedge-mapper-c8y"},"payload":"{\"@parent\":\"device/main//\",\"@type\":\"service\",\"type\":\"service\"}","qos":1,"retain":true}
{"topic":{"name":"te/device/main/service/tedge-agent"},"payload":"{\"@parent\":\"device/main//\",\"@type\":\"service\",\"type\":\"service\"}","qos":1,"retain":true}

Just wondering, is that maybe also a "wrong" behaviour of core?

Just wondering, is that maybe also a "wrong" behaviour of core?

I wouldn't say so, but yes the core's behaviour makes this more difficult than it should be

I observed the same issue on a device where thin-edge was running for a while without being connected to any c8y endpoint. When finally connected to a new tenant (as I did after the staging tenant has been wiped) or to the same tenant but with a new device id (as I tested), then the service health status messages are forwarded by the mapper to c8y (104) without any service registration (102).
This is not an expected use-case, but this highlights that the registration mechanism is really fragile.
Would it make sense to send a smartrest 102 message before all 104 messages?

The issue should be fixed in #2620 by changing service status smartrest message from 104 to 102, with the 102 actually creating a service if it didn't exist yet.
A new test suite bootstrap.robot was created with 2 tests checking if no unexpected child devices are created. The first case relates to this ticket, and the second one to #2606, although the second case doesn't work correctly because fs events are not properly detected by inotify inside the test device container.

QA has thoroughly checked the bug and here are the results:

[x] Test for ticket exists in the test suite.
tests/RobotFramework/tests/cumulocity/bootstrap.robot
[x] QA has tested the bug and it's not reproducable anymore


tilt: fix duplicates in LastDeployedEdits

Problem:
After a failed build, changed files were showing up twice in "Last deployed edits"
This was supposed to be a one-line change, plus tests :(. ManifestState had three lists of files, and I figured we were just incorrectly populating one.
The three lists were:

CurrentlyBuildingFileChanges
LastSuccessfulDeployEdits
PendingFileChanges

It turns out (1) was both "file changes included in the current build", "files that have changed since the last image we pushed (i.e., changes since the last successful build)". We weren't clearing it after failed builds because we needed to ensure we copy those files on subsequent image builds, which meant the changes from those failed builds got listed as currently building changes and last deployed changes (and, if any of those files were changed again, they'd get listed twice).
One option is to just dedupe (1) and (2) above. This leaves us with a UI question: do we want to show new changes that we just tried to build for the first time, or do we want to show all changes that we had not successfully built?
If the user changes one file, gets a failure, changes another file, gets a success, it's probably fine to show both.
If I edit 10 files, causing failures all the way, and then tabs back to tilt when they're done, I think it'd be better to show only the more recent changes, since what I want to know is whether tilt has caught up to me, and the full list would just scroll of the screen (given current implementation) or wrap, either of which are ugly.
So what I've done here is turn these three into four lists of files, which are hopefully reasonably described here. I'm not in love with the names: they don't capture that we exclude unmounted config files and they are coupled to the current implementation that the last image content is identical to the content used for the last successful build, but those are problems with the previous names as well.
A good middle ground I hadn't considered until posting this is to dedupe and sort the changes in reverse chron, which would solve the "I'm most interested in confirming the changes I just made have been picked up" problem, but not the "long file lists are ugly" problem. I prefer the UI implemented in this PR.

hey, i have a pr out that moves all this code around. can we merge my change first?

ya, i'm deeply worried that this is quickly going to be unmanageable
i think the core problem is that there's a lot of state to maintain, and a lot of the state updates are destructive and non-monotonic. e.g., "move file from slice A to slice B" or "oops, the build failed, so we have to move file from slice C back to slice A"
what if we just used monotonic clocks and file timestamps? e.g., just use a single map[string]time.Time . and then FilesChangedSinceLastSuccessfulBuild, etc could be implemented as methods. it would also allow us to experiment with different UIs (like reverse chron) without drastically changing the data model every time.

Yeah, you're right that this is getting too hairy. This PR probably isn't a good starting point for another solution, so just going to close it.


TF 1.8 benchmark distributed run failed

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
Linux Ubuntu 16.04
TensorFlow installed from source with verbs support
TensorFlow version:
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
('v1.8.0-1-g8753e2e', '1.8.0')
Python version: 2.7
Bazel version: 0.9 ,0.10
GCC/Compiler version: 5.4
CUDA/cuDNN version: 9.1 ,7.0
GPU model and memory: P100, 16GB
Exact command to reproduce:

On first node:
CUDA_VISIBLE_DEVICES='' python ~/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --model inception3 --batch_size 64 --ps_hosts <IP_ADDRESS>:10001,<IP_ADDRESS>:10002 --worker_hosts <IP_ADDRESS>:20001,<IP_ADDRESS>:20002 --task_index 1 --job_name ps --num_gpus 1 --variable_update=distributed_replicated --cross_replica_sync=True &
python ~/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --model inception3 --batch_size 64 --ps_hosts <IP_ADDRESS>:10001,<IP_ADDRESS>:10002 --worker_hosts <IP_ADDRESS>:20001,<IP_ADDRESS>:20002 --task_index 1 --job_name worker --num_gpus 1 --variable_update=distributed_replicated --cross_replica_sync=True
On second node:
CUDA_VISIBLE_DEVICES='' python ~/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --model inception3 --batch_size 64 --ps_hosts <IP_ADDRESS>:10001,<IP_ADDRESS>:10002 --worker_hosts <IP_ADDRESS>:20001,<IP_ADDRESS>:20002 --task_index 0 --job_name ps --num_gpus 1 --variable_update=distributed_replicated --cross_replica_sync=True &
python ~/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py --model inception3 --batch_size 64 --ps_hosts <IP_ADDRESS>:10001,<IP_ADDRESS>:10002 --worker_hosts <IP_ADDRESS>:20001,<IP_ADDRESS>:20002 --task_index 0 --job_name worker --num_gpus 1 --variable_update=distributed_replicated --cross_replica_sync=True
Problem
I'm running benchmark ( master branch) on two hosts with error:
2018-05-08 14:55:30.598759: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:04:00.0
totalMemory: 15.90GiB freeMemory: 15.61GiB
2018-05-08 14:55:30.979461: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 1 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:06:00.0
totalMemory: 15.90GiB freeMemory: 15.61GiB
2018-05-08 14:55:31.325185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 2 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:07:00.0
totalMemory: 15.90GiB freeMemory: 15.61GiB
2018-05-08 14:55:31.676291: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 3 with properties:
name: Tesla P100-PCIE-16GB major: 6 minor: 0 memoryClockRate(GHz): 1.3285
pciBusID: 0000:08:00.0
totalMemory: 15.90GiB freeMemory: 15.61GiB
2018-05-08 14:55:31.685309: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0, 1, 2, 3
2018-05-08 14:55:32.760916: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-05-08 14:55:32.760978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929]      0 1 2 3
2018-05-08 14:55:32.760986: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0:   N Y Y Y
2018-05-08 14:55:32.760991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 1:   Y N Y Y
2018-05-08 14:55:32.760997: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 2:   Y Y N Y
2018-05-08 14:55:32.761002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 3:   Y Y Y N
2018-05-08 14:55:32.762303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:worker/replica:0/task:0/device:GPU:0 with 15133 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:04:00.0, compute capability: 6.0)
2018-05-08 14:55:33.097816: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:worker/replica:0/task:0/device:GPU:1 with 15133 MB memory) -> physical GPU (device: 1, name: Tesla P100-PCIE-16GB, pci bus id: 0000:06:00.0, compute capability: 6.0)
2018-05-08 14:55:33.431088: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:worker/replica:0/task:0/device:GPU:2 with 15133 MB memory) -> physical GPU (device: 2, name: Tesla P100-PCIE-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0)
2018-05-08 14:55:33.735570: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:worker/replica:0/task:0/device:GPU:3 with 15133 MB memory) -> physical GPU (device: 3, name: Tesla P100-PCIE-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0)
2018-05-08 14:55:34.010006: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job ps -> {0 -> <IP_ADDRESS>:10001, 1 -> <IP_ADDRESS>:10002}
2018-05-08 14:55:34.010056: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job worker -> {0 -> localhost:20001, 1 -> <IP_ADDRESS>:20002}
2018-05-08 14:55:34.016202: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:332] Started server with target: grpc://localhost:20001
TensorFlow:  1.8
Model:       inception3
Dataset:     imagenet (synthetic)
Mode:        training
SingleSess:  False
Batch size:  128 global
64 per device
Num batches: 100
Num epochs:  0.01
Devices:     ['/job:worker/task:0/gpu:0']
Data format: NCHW
Layout optimizer: False
Optimizer:   sgd
Variables:   distributed_replicated
Sync:        True
Generating model
W0508 14:55:41.537986<PHONE_NUMBER>28896 tf_logging.py:126] From /root/benchmarks/scripts/tf_cnn_benchmarks/benchmark_cnn.py:1525: init (from tensorflow.python.training.supervisor) is deprecated and will be removed in a future version.
Instructions for updating:
Please switch to tf.train.MonitoredTrainingSession
2018-05-08 14:55:53.359721: I tensorflow/core/distributed_runtime/master.cc:221] CreateSession still waiting for response from worker: /job:worker/replica:0/task:1
2018-05-08 14:56:00.360362: E tensorflow/core/distributed_runtime/master.cc:269] Master init: Unavailable: OS Error
I0508 14:56:00.440299<PHONE_NUMBER>28896 tf_logging.py:116] Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.UnavailableError'>, OS Error
Traceback (most recent call last):
File "/root/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py", line 60, in
app.run(main)  # Raises error on invalid flags, unlike tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 274, in run
_run_main(main, argv)
File "/usr/local/lib/python2.7/dist-packages/absl/app.py", line 238, in _run_main
sys.exit(main(argv))
File "/root/benchmarks/scripts/tf_cnn_benchmarks/tf_cnn_benchmarks.py", line 56, in main
bench.run()
File "/root/benchmarks/scripts/tf_cnn_benchmarks/benchmark_cnn.py", line 1306, in run
return self._benchmark_cnn()
File "/root/benchmarks/scripts/tf_cnn_benchmarks/benchmark_cnn.py", line 1535, in _benchmark_cnn
start_standard_services=start_standard_services) as sess:
File "/usr/lib/python2.7/contextlib.py", line 17, in enter
return self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 1000, in managed_session
self.stop(close_summary_writer=close_summary_writer)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 828, in stop
ignore_live_threads=ignore_live_threads)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/coordinator.py", line 389, in join
six.reraise(*self._exc_info_to_raise)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 989, in managed_session
start_standard_services=start_standard_services)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/supervisor.py", line 726, in prepare_or_wait_for_session
init_feed_dict=self._init_feed_dict, init_fn=self._init_fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/session_manager.py", line 285, in prepare_session
sess.run(init_op, feed_dict=init_feed_dict)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnavailableError: OS Error

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
OS Platform and Distribution

Ubuntu 16.04 , grpc

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there.
If you think we've misinterpreted a bug, please comment again with a clear explanation, as well as all of the information requested in the issue template. Thanks!

This problem still exists on 1.13 branch


chore: add bootc to all images

39 cannot use the bootupctl command with the updated installer.
40 needs the bootupctl command  and the updated installer.
40 currently has an issue with rpmfusion. I would like to confirm that a downstream image doesn't have to convert again.
Thank you for contributing to the Universal Blue project!
Please read the Contributor's Guide before submitting a pull request.

While bootc will be the future, I'm wondering if we should hold off on having these on all images while the supporting infrastructure is still being put in place.
The command to make the iso work on F40 causes it to break on F39. This can be handled via an if statement during build. But maybe a point of concern for downstream images where if it isn't rerun the iso will break.

While bootc will be the future, I'm wondering if we should hold off on having these on all images while the supporting infrastructure is still being put in place.
The command to make the iso work on F40 causes it to break on F39. This can be handled via an if statement during build. But maybe a point of concern for downstream images where if it isn't rerun the iso will break.

I support adding bootc ... and I think we had this in the past, no?
Users of bootc would be experimental/early-adopters who are fine with it's explicit lack of support for local layering.
Thus, all our docs which talk about command-line stuff should continue to reference rpm-ostree operations.
I don't know how ISO building has anything to do with having the binary on an image to support the bootc effort.

bootc is not in F41, only it's dependency bootupd is in it.
Just noting if this PR is still followed or considered.


toggle not sized properly when in a collapsed parent

The toggle is not initialized properly when parent is first collapsed :
See sample :

    RTT

        <input
          type="checkbox"
          class="form-check-input d-inline-block"
          data-toggle="toggle"
          data-onstyle="success"
          data-offstyle="danger"
          data-on="Oui"
          data-off="Non"
        />

This same issue occurs when you have a hidden toggle when the page loads. Once you show the hidden toggle it doesn't have the 'height' setting applied to the toggle labels. The HTML code renders without the 'line-height: **px'.

I've run into this issue with my own solutions and usually get around it by placing the toggle in a col and/or adding the w-50 or w-100 class.
I'm unsure how to code around it other than to watch for it to become visible and re-render it...


Change default networkType to OVNKubernetes

With OCP 4.12, the default network type is OVNKubernetes
Reference: https://github.com/openshift/installer/pull/6014
Signed-off-by: Pravin D'silva<EMAIL_ADDRESS>

@pravin-dsilva: PR is not mergeable.

The PR state is: BLOCKED
Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


tests: kernel.timer.timer: frdm_ke17z512: timer_jitter_drift fails

Describe the bug
timer_jitter_drift test failure on frdm_ke17z512
Please also mention any information which could help others to understand
the problem you're facing:

What target platform are you using?
frdm_ke17z512
What have you tried to diagnose or workaround this issue?
NA
Is this a regression?
Running TESTSUITE timer_tick_train
===================================================================
START - test_one_tick_timer_train
Initializing 4 Timers, Tick Rate 10000Hz, Expecting 25000 callbacks in 11000 ms
Starting Timers with Skews
Faking busy work, remaining timers is 4, timer callbacks 1
Faking busy work, remaining timers is 4, timer callbacks 638
Faking busy work, remaining timers is 4, timer callbacks 1275
Faking busy work, remaining timers is 4, timer callbacks 1912
Faking busy work, remaining timers is 4, timer callbacks 2550
Faking busy work, remaining timers is 4, timer callbacks 3187
Faking busy work, remaining timers is 4, timer callbacks 3825
Faking busy work, remaining timers is 4, timer callbacks 4462
Faking busy work, remaining timers is 4, timer callbacks 5100
Faking busy work, remaining timers is 4, timer callbacks 5737
Faking busy work, remaining timers is 4, timer callbacks 6374
Faking busy work, remaining timers is 4, timer callbacks 7012
Faking busy work, remaining timers is 4, timer callbacks 7649
Faking busy work, remaining timers is 4, timer callbacks 8286
Faking busy work, remaining timers is 4, timer callbacks 8924
Faking busy work, remaining timers is 4, timer callbacks 9561
Faking busy work, remaining timers is 4, timer callbacks 10199
Faking busy work, remaining timers is 4, timer callbacks 10837
Faking busy work, remaining timers is 4, timer callbacks 11475
Faking busy work, remaining timers is 4, timer callbacks 12112
Faking busy work, remaining timers is 4, timer callbacks 12750
Faking busy work, remaining timers is 4, timer callbacks 13388
Faking busy work, remaining timers is 4, timer callbacks 14026
Faking busy work, remaining timers is 4, timer callbacks 14663
Faking busy work, remaining timers is 4, timer callbacks 15301
Faking busy work, remaining timers is 4, timer callbacks 15939
Faking busy work, remaining timers is 4, timer callbacks 16577
Faking busy work, remaining timers is 4, timer callbacks 17215
Faking busy work, remaining timers is 4, timer callbacks 17852
Faking busy work, remaining timers is 4, timer callbacks 18490
Faking busy work, remaining timers is 4, timer callbacks 19128
Faking busy work, remaining timers is 4, timer callbacks 19766
Faking busy work, remaining timers is 4, timer callbacks 20404
Faking busy work, remaining timers is 4, timer callbacks 21041
Faking busy work, remaining timers is 4, timer callbacks 21679
Faking busy work, remaining timers is 4, timer callbacks 22317
Faking busy work, remaining timers is 4, timer callbacks 22955
Faking busy work, remaining timers is 4, timer callbacks 23593
Faking busy work, remaining timers is 4, timer callbacks 24231
Faking busy work, remaining timers is 4, timer callbacks 24869
One Tick Timer Train Done, took 10202 ms, busy loop ran 40 times
Perfect delta 19200 cycles or 400 us
Timer 0 max delta 19690 cycles or 410 us, 0 late callbacks (0.0%)
RECORD: {"testcase":"one_tick_timer_train", "timer":0, "max_delta_cycles":19690, "max_delta_us":410, "late_callbacks":0, "perfect_delta_cycles":19200, "perfect_delta_us":400, "train_time_ms":10202, "busy_loops":40, "timers":4, "expected_callbacks":25000, "expected_time_ms":11000, "CONFIG_SYS_CLOCK_TICKS_PER_SEC":10000}
Timer 1 max delta 19664 cycles or 410 us, 0 late callbacks (0.0%)
RECORD: {"testcase":"one_tick_timer_train", "timer":1, "max_delta_cycles":19664, "max_delta_us":410, "late_callbacks":0, "perfect_delta_cycles":19200, "perfect_delta_us":400, "train_time_ms":10202, "busy_loops":40, "timers":4, "expected_callbacks":25000, "expected_time_ms":11000, "CONFIG_SYS_CLOCK_TICKS_PER_SEC":10000}
Timer 2 max delta 19953 cycles or 416 us, 0 late callbacks (0.0%)
RECORD: {"testcase":"one_tick_timer_train", "timer":2, "max_delta_cycles":19953, "max_delta_us":416, "late_callbacks":0, "perfect_delta_cycles":19200, "perfect_delta_us":400, "train_time_ms":10202, "busy_loops":40, "timers":4, "expected_callbacks":25000, "expected_time_ms":11000, "CONFIG_SYS_CLOCK_TICKS_PER_SEC":10000}
Timer 3 max delta 19845 cycles or 413 us, 0 late callbacks (0.0%)
RECORD: {"testcase":"one_tick_timer_train", "timer":3, "max_delta_cycles":19845, "max_delta_us":413, "late_callbacks":0, "perfect_delta_cycles":19200, "perfect_delta_us":400, "train_time_ms":10202, "busy_loops":40, "timers":4, "expected_callbacks":25000, "expected_time_ms":11000, "CONFIG_SYS_CLOCK_TICKS_PER_SEC":10000}
PASS - test_one_tick_timer_train in 10.356 seconds
===================================================================
TESTSUITE timer_tick_train succeeded
------ TESTSUITE SUMMARY START ------
SUITE FAIL -   0.00% [timer_jitter_drift]: pass = 0, fail = 2, skip = 0, total = 2 duration = 15.416 seconds
- FAIL - [timer_jitter_drift.test_jitter_drift_timer_period] duration = 7.708 seconds
- FAIL - [timer_jitter_drift.test_jitter_drift_timer_startdelay] duration = 7.708 seconds
SUITE PASS - 100.00% [timer_tick_train]: pass = 1, fail = 0, skip = 0, total = 1 duration = 10.356 seconds
- PASS - [timer_tick_train.test_one_tick_timer_train] duration = 10.356 seconds
------ TESTSUITE SUMMARY END ------
===================================================================
RunID: 10fa5bd6de98a1a3f520501918bdebae
PROJECT EXECUTION FAILED

Environment (please complete the following information):

OS: (e.g. Linux, )
Toolchain (e.g Zephyr SDK)
Commit SHA or Version used: v3.7.0-504-g2d2ce4e78d00

@ankeXiao

Okay, I'll take a look at it!


Review UI Getting Started page for docs material

Ref: https://github.com/rancher/dashboard/issues/7413
The dashboard team is in the process of removing written material, e.g. release notes and a getting started page, that's bundled in the UI. For this ticket, the docs team should review https://github.com/rancher/dashboard/blob/master/shell/content/docs/en-us/getting-started.md for any new information that can be added to the docs.

I did an inventory of the items listed in the "Migrating from Cluster Manager" table last night. About half of the listed items have corresponding pages that contain instructions for how to find them in the Rancher UI. Others only seem to have CLI commands included in our docs, or are so frequently navigated to (the clusters list) that there's no particular page to link users to.

Feature
Page
Note

Cluster drivers
https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-provisioning-drivers/manage-cluster-drivers#docusaurus_skipToContent_fallback
/

RKE (Cluster) Templates
https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/authentication-permissions-and-global-configuration/about-rke1-templates/manage-rke1-templates
/

RKE Cloud Credentials
https://ranchermanager.docs.rancher.com/reference-guides/user-settings/manage-cloud-credentials
The instructions on this page aren't RKE1-specific. The original table indicates that this feature can be found under RKE1 Configuration but I don't see that listed in 2.7. The 2.6 version of the page also lists the alternate, non-RKE1 path. I think the table might be outdated

RKE Node Templates
https://ranchermanager.docs.rancher.com/reference-guides/user-settings/manage-node-templates
/

Pod Security Policy
https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/manage-projects/manage-pod-security-policies#docusaurus_skipToContent_fallback
/

Cluster (functionality)
https://ranchermanager.docs.rancher.com/pages-for-subheaders/access-clusters
This page doesn't really describe how to navigate to the feature but it contains a subheading that would be a natural place to correct that

Projects/Namespaces
https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces#docusaurus_skipToContent_fallback
/

Namespaces
https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/manage-clusters/projects-and-namespaces#docusaurus_skipToContent_fallback

Alerts
https://ranchermanager.docs.rancher.com/reference-guides/monitoring-v2-configuration/receivers
This isn't along the legacy path suggested by the original table

Monitoring
https://ranchermanager.docs.rancher.com/how-to-guides/advanced-user-guides/monitoring-alerting-guides/enable-monitoring#docusaurus_skipToContent_fallback
/

CLI Download
https://ranchermanager.docs.rancher.com/reference-guides/cli-with-rancher/rancher-cli#download-rancher-cli
/

API and Keys
https://ranchermanager.docs.rancher.com/reference-guides/user-settings/api-keys#api-keys-and-user-authentication
/

Preferences
https://ranchermanager.docs.rancher.com/reference-guides/user-settings/user-preferences#docusaurus_skipToContent_fallback

@btat For the info regarding features that are now hidden by the Legacy feature flag in the UI -- should we add instructions to /docs, /2.7, /2.6, etc. on the relevant feature pages (Example for Alerts: "You can turn on v1 Alerts by . Once you do that, you can find controls for them under "), or simply point to the /versioned_docs pages where this information is already recorded ("V1 Alerts are considered a legacy feature. If you're looking for information, ")?

@btat For the info regarding features that are now hidden by the Legacy feature flag in the UI -- should we add instructions to /docs, /2.7, /2.6, etc. on the relevant feature pages (Example for Alerts: "You can turn on v1 Alerts by instructions. Once you do that, you can find controls for them under instructions"), or simply point to the /versioned_docs pages where this information is already recorded ("V1 Alerts are considered a legacy feature. If you're looking for information, link to page")?

I think option 2 is fine if the navigation/steps (after enabling the legacy feature) are the same on versions where it's considered legacy and versions where it's not (e.g. <=2.5).

V1 (cluster) Alerts: https://ranchermanager.docs.rancher.com/v2.0-v2.4/pages-for-subheaders/cluster-alerts#adding-cluster-alerts
V1 Notifiers: https://ranchermanager.docs.rancher.com/v2.0-v2.4/explanations/integrations-in-rancher/notifiers#adding-notifiers
V1 (cluster) Monitoring: https://ranchermanager.docs.rancher.com/v2.0-v2.4/pages-for-subheaders/cluster-monitoring#enabling-cluster-monitoring
V1 (cluster) Metrics: https://ranchermanager.docs.rancher.com/v2.0-v2.4/explanations/integrations-in-rancher/cluster-monitoring/cluster-metrics <-- need to add explicit UI instructions, buy already has instructions for other, more deeply nested, metrics
V1 (node) Metrics: https://ranchermanager.docs.rancher.com/v2.0-v2.4/explanations/integrations-in-rancher/cluster-monitoring/cluster-metrics#finding-node-metrics
V1 (workload) Metrics: https://ranchermanager.docs.rancher.com/v2.0-v2.4/explanations/integrations-in-rancher/cluster-monitoring/cluster-metrics#finding-workload-metrics
V1 (pod) Metrics: Completely missing

Closing as review was performed for #739


container-specific dashboard

[ ] container cpu utilization widget
[ ] container memory utilization widget
[ ] container network io utilization widget
[ ] container disk io utilization widget

@brendangregg let me know if anything else comes to mind in terms of widgets/metrics

@hc000 from the widgets mentioned above, which ones you guys already use at att?

container cpu utilization widget
container memory utilization widget
container network io utilization widget
i will be pushing them later this week.


Update use-sso-in-event-based-activation.md

I probably spent 2 hours trying to debug this after adding it to my webserver before I googled the Office samples github repo and found out how it was supposed to be implemented :)

@ElizabethSamuel-MSFT thanks for the suggestion! LGTM

@1TheMuffinMan Sorry to hear that you faced this issue. We appreciate your taking the time to propose this update.
Currently, we're verifying the update with the feature crew. We'll merge if the crew confirms this change.
Thanks.


removing wrong copy of file.

Two copies of the Readme are present in the PsfLauncher folder. This removes the "wrong" one.

Good job on catching that.


Add information about method return type changes

This changes fix the issue:
https://github.com/tomzx/php-semver-checker/issues/54

I will create PR for a function at this weekend.

Hello, I've added the function functionality here, please check it again. If there is some other change what I should do, write me here.

Hi @tomzx,
Is there anything we can do to help with approving this PR?


Use outside Sketchup

Hi,
Is there any way to use the script directly? If I have an STL file, from Thingiverse and want to modify it for my needs, the Sketchup online app does not let me import stl files, if I can convert to .skp I can open it.
thanks

I don't know about the SketchUp online app (I didn't even know there was one), but for the regular Sketchup program, I've used this plug-in: https://github.com/SketchUp/sketchup-stl

This project is an STL importer/exporter to be plugged into SketchUp, not a stand-alone "converter". It requires the SketchUp API to work, which is unfortunately not available as a stand-alone ruby gem.
Unfortunately the only way to produce SKP files from STL is to use the SketchUp desktop app (which foild the web-app's beautiful advantage of platform-independence).


Random tile coordinates

when we compute the random coordinate of a tile in the Random tiler, based on binary mask, we are considering random coordinates (x,y). Precisely, we are taking independently x and y as random position where the binary mask is True, however (x,y) could be taken in a position where the mask is False. We should instead fix x or y where the binary mask is True and then take the corresponding y or x.

it should be replaced with something like
x_ul_lvl = np.random.choice(true_mask[1])
idx = np.random.choice(np.where(true_mask[1]==x_ul_lvl)[0])
y_ul_lvl = true_mask[0][idx]


Unable to access admin-api via loopback in Kong DB-Less on Kubernetes

Summary
I'm trying to implement Kong 2.1 in DB Less mode on Kubernetes. I can access the admin API through port forwarding but not via proxy loopback.
Kong Ingress controller version
image: "kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.1"
Kong version
image: kong:2.1
Kubernetes version
paste kubectl version output
Client Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:26:26Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.10", GitCommit:"f3add640dbcd4f3c33a7749f38baaac0b3fe810d", GitTreeState:"clean", BuildDate:"2020-05-20T13:51:56Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Environment

Cloud provider: AWS

What happened
When I access the URL, following response is returned
{ "message": "An invalid response was received from the upstream server"}

The logs for ingress-kong pod show the following
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxx HTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=Q4ZPt6RiW02cep95kRC3", host: "localhost:8081"
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxxHTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=Q4ZPt6RiW02cep95kRC3", host: "localhost:8081"
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxxHTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=xxxxxx", host: "localhost:8081"
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxxHTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=xxxxxx", host: "localhost:8081"
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxxHTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=xxxxxx", host: "localhost:8081"
2020/08/16 13:38:52 [error] 22#0: *652 connect() failed (111: Connection refused) while connecting to upstream, client: <IP_ADDRESS>, server: kong, request: "GET /kong-admin/?apikey=xxxxxxHTTP/1.1", upstream: "http://<IP_ADDRESS>:8001/kong-admin/?apikey=xxxxxx", host: "localhost:8081"
<IP_ADDRESS> - - [16/Aug/2020:13:38:52 +0000] "GET /kong-admin/?apikey=xxxxxxHTTP/1.1" 502 75 "-" "PostmanRuntime/7.25.0"

Expected behavior
Steps To Reproduce
Apply the following yaml files.
all-in-one-dbless.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: kong
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongclusterplugins.configuration.konghq.com
spec:
  additionalPrinterColumns:
    - JSONPath: .plugin
      description: Name of the plugin
      name: Plugin-Type
      type: string
    - JSONPath: .metadata.creationTimestamp
      description: Age
      name: Age
      type: date
    - JSONPath: .disabled
      description: Indicates if the plugin is disabled
      name: Disabled
      priority: 1
      type: boolean
    - JSONPath: .config
      description: Configuration of the plugin
      name: Config
      priority: 1
      type: string
  group: configuration.konghq.com
  names:
    kind: KongClusterPlugin
    plural: kongclusterplugins
    shortNames:
      - kcp
  scope: Cluster
  validation:
    openAPIV3Schema:
      properties:
        config:
          type: object
        configFrom:
          properties:
            secretKeyRef:
              properties:
                key:
                  type: string
                name:
                  type: string
                namespace:
                  type: string
              required:
                - name
                - namespace
                - key
              type: object
          type: object
        disabled:
          type: boolean
        plugin:
          type: string
        protocols:
          items:
            enum:
              - http
              - https
              - grpc
              - grpcs
              - tcp
              - tls
            type: string
          type: array
        run_on:
          enum:
            - first
            - second
            - all
          type: string
      required:
        - plugin
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongconsumers.configuration.konghq.com
spec:
  additionalPrinterColumns:
    - JSONPath: .username
      description: Username of a Kong Consumer
      name: Username
      type: string
    - JSONPath: .metadata.creationTimestamp
      description: Age
      name: Age
      type: date
  group: configuration.konghq.com
  names:
    kind: KongConsumer
    plural: kongconsumers
    shortNames:
      - kc
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        credentials:
          items:
            type: string
          type: array
        custom_id:
          type: string
        username:
          type: string
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongcredentials.configuration.konghq.com
spec:
  additionalPrinterColumns:
    - JSONPath: .type
      description: Type of credential
      name: Credential-type
      type: string
    - JSONPath: .metadata.creationTimestamp
      description: Age
      name: Age
      type: date
    - JSONPath: .consumerRef
      description: Owner of the credential
      name: Consumer-Ref
      type: string
  group: configuration.konghq.com
  names:
    kind: KongCredential
    plural: kongcredentials
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        consumerRef:
          type: string
        type:
          type: string
      required:
        - consumerRef
        - type
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongingresses.configuration.konghq.com
spec:
  group: configuration.konghq.com
  names:
    kind: KongIngress
    plural: kongingresses
    shortNames:
      - ki
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        proxy:
          properties:
            connect_timeout:
              minimum: 0
              type: integer
            path:
              pattern: ^/.*$
              type: string
            protocol:
              enum:
                - http
                - https
                - grpc
                - grpcs
                - tcp
                - tls
              type: string
            read_timeout:
              minimum: 0
              type: integer
            retries:
              minimum: 0
              type: integer
            write_timeout:
              minimum: 0
              type: integer
          type: object
        route:
          properties:
            headers:
              additionalProperties:
                items:
                  type: string
                type: array
              type: object
            https_redirect_status_code:
              type: integer
            methods:
              items:
                type: string
              type: array
            path_handling:
              enum:
                - v0
                - v1
              type: string
            preserve_host:
              type: boolean
            protocols:
              items:
                enum:
                  - http
                  - https
                  - grpc
                  - grpcs
                  - tcp
                  - tls
                type: string
              type: array
            regex_priority:
              type: integer
            strip_path:
              type: boolean
        upstream:
          properties:
            algorithm:
              enum:
                - round-robin
                - consistent-hashing
                - least-connections
              type: string
            hash_fallback:
              type: string
            hash_fallback_header:
              type: string
            hash_on:
              type: string
            hash_on_cookie:
              type: string
            hash_on_cookie_path:
              type: string
            hash_on_header:
              type: string
            healthchecks:
              properties:
                active:
                  properties:
                    concurrency:
                      minimum: 1
                      type: integer
                    healthy:
                      properties:
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        successes:
                          minimum: 0
                          type: integer
                      type: object
                    http_path:
                      pattern: ^/.*$
                      type: string
                    timeout:
                      minimum: 0
                      type: integer
                    unhealthy:
                      properties:
                        http_failures:
                          minimum: 0
                          type: integer
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        tcp_failures:
                          minimum: 0
                          type: integer
                        timeout:
                          minimum: 0
                          type: integer
                      type: object
                  type: object
                passive:
                  properties:
                    healthy:
                      properties:
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        successes:
                          minimum: 0
                          type: integer
                      type: object
                    unhealthy:
                      properties:
                        http_failures:
                          minimum: 0
                          type: integer
                        http_statuses:
                          items:
                            type: integer
                          type: array
                        interval:
                          minimum: 0
                          type: integer
                        tcp_failures:
                          minimum: 0
                          type: integer
                        timeout:
                          minimum: 0
                          type: integer
                      type: object
                  type: object
                threshold:
                  type: integer
              type: object
            host_header:
              type: string
            slots:
              minimum: 10
              type: integer
          type: object
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: kongplugins.configuration.konghq.com
spec:
  additionalPrinterColumns:
    - JSONPath: .plugin
      description: Name of the plugin
      name: Plugin-Type
      type: string
    - JSONPath: .metadata.creationTimestamp
      description: Age
      name: Age
      type: date
    - JSONPath: .disabled
      description: Indicates if the plugin is disabled
      name: Disabled
      priority: 1
      type: boolean
    - JSONPath: .config
      description: Configuration of the plugin
      name: Config
      priority: 1
      type: string
  group: configuration.konghq.com
  names:
    kind: KongPlugin
    plural: kongplugins
    shortNames:
      - kp
  scope: Namespaced
  validation:
    openAPIV3Schema:
      properties:
        config:
          type: object
        configFrom:
          properties:
            secretKeyRef:
              properties:
                key:
                  type: string
                name:
                  type: string
              required:
                - name
                - key
              type: object
          type: object
        disabled:
          type: boolean
        plugin:
          type: string
        protocols:
          items:
            enum:
              - http
              - https
              - grpc
              - grpcs
              - tcp
              - tls
            type: string
          type: array
        run_on:
          enum:
            - first
            - second
            - all
          type: string
      required:
        - plugin
  version: v1
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: tcpingresses.configuration.konghq.com
spec:
  additionalPrinterColumns:
    - JSONPath: .status.loadBalancer.ingress[*].ip
      description: Address of the load balancer
      name: Address
      type: string
    - JSONPath: .metadata.creationTimestamp
      description: Age
      name: Age
      type: date
  group: configuration.konghq.com
  names:
    kind: TCPIngress
    plural: tcpingresses
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata:
          type: object
        spec:
          properties:
            rules:
              items:
                properties:
                  backend:
                    properties:
                      serviceName:
                        type: string
                      servicePort:
                        format: int32
                        type: integer
                    type: object
                  host:
                    type: string
                  port:
                    format: int32
                    type: integer
                type: object
              type: array
            tls:
              items:
                properties:
                  hosts:
                    items:
                      type: string
                    type: array
                  secretName:
                    type: string
                type: object
              type: array
          type: object
        status:
          type: object
  version: v1beta1
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kong-serviceaccount
  namespace: kong
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: kong-ingress-clusterrole
rules:
  - apiGroups:
      - ""
    resources:
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - networking.k8s.io
      - extensions
      - networking.internal.knative.dev
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - networking.k8s.io
      - extensions
      - networking.internal.knative.dev
    resources:
      - ingresses/status
    verbs:
      - update
  - apiGroups:
      - configuration.konghq.com
    resources:
      - tcpingresses/status
    verbs:
      - update
  - apiGroups:
      - configuration.konghq.com
    resources:
      - kongplugins
      - kongclusterplugins
      - kongcredentials
      - kongconsumers
      - kongingresses
      - tcpingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
      - get
      - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kong-ingress-clusterrole-nisa-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kong-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: kong-serviceaccount
    namespace: kong
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "8443"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "60"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:xxxxxxx"
  name: kong-proxy
  namespace: kong
  labels:
    app: kong-proxy
spec:
  externalTrafficPolicy: Local
  ports:
    # - name: proxy
    #   port: 80
    #   protocol: TCP
    #   targetPort: 8000
    - name: proxy-ssl
      port: 8443
      protocol: TCP
      targetPort: 8000
  selector:
    app: ingress-kong
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: kong-admin
  namespace: kong
  labels:
    app: kong-admin
spec:
  type: ClusterIP
  ports:
    - name: admin
      port: 8001
      protocol: TCP
      targetPort: 8001
    - name: admin-ssl
      port: 8444
      targetPort: 8444
      protocol: TCP
  selector:
    app: ingress-kong
---
apiVersion: v1
kind: Service
metadata:
  name: kong-validation-webhook
  namespace: kong
spec:
  ports:
    - name: webhook
      port: 443
      protocol: TCP
      targetPort: 8080
  selector:
    app: ingress-kong
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ingress-kong
  name: ingress-kong
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-kong
  template:
    metadata:
      annotations:
        kuma.io/gateway: enabled
        prometheus.io/port: "8100"
        prometheus.io/scrape: "true"
        traffic.sidecar.istio.io/includeInboundPorts: ""
      labels:
        app: ingress-kong
    spec:
      containers:
        - env:
            - name: KONG_PROXY_LISTEN
              value: <IP_ADDRESS>:8000, <IP_ADDRESS>:8443 http2 ssl
            - name: KONG_ADMIN_LISTEN
              value: <IP_ADDRESS>:8001, <IP_ADDRESS>:8444 http2 ssl
            - name: KONG_STATUS_LISTEN
              value: <IP_ADDRESS>:8100
            - name: KONG_DATABASE
              value: "off"
            - name: KONG_NGINX_WORKER_PROCESSES
              value: "1"
            - name: KONG_ADMIN_ACCESS_LOG
              value: /dev/stdout
            - name: KONG_ADMIN_ERROR_LOG
              value: /dev/stderr
            - name: KONG_PROXY_ERROR_LOG
              value: /dev/stderr
          image: kong:2.1
          lifecycle:
            preStop:
              exec:
                command:
                  - /bin/sh
                  - -c
                  - kong quit
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /status
              port: 8100
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: proxy
          ports:
            - containerPort: 8001
              name: admin
              protocol: TCP
            - containerPort: 8444
              name: admin-ssl
              protocol: TCP
            - containerPort: 8000
              name: proxy
              protocol: TCP
            - containerPort: 8443
              name: proxy-ssl
              protocol: TCP
            - containerPort: 8100
              name: metrics
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /status
              port: 8100
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          securityContext:
            runAsUser: 1000
        - env:
            - name: CONTROLLER_KONG_ADMIN_URL
              value: https://<IP_ADDRESS>:8444
            - name: CONTROLLER_KONG_ADMIN_TLS_SKIP_VERIFY
              value: "true"
            - name: CONTROLLER_PUBLISH_SERVICE
              value: kong/kong-proxy
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
          image: kong-docker-kubernetes-ingress-controller.bintray.io/kong-ingress-controller:0.9.1
          imagePullPolicy: IfNotPresent
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          name: ingress-controller
          ports:
            - containerPort: 8080
              name: webhook
              protocol: TCP
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 5
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
      serviceAccountName: kong-serviceaccount

admin-loopback.yaml
---
# Key Auth Plugin
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: loopback-key-auth-plugin
  namespace: kong
plugin: key-auth

---
# ACL Plugin
apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
  name: loopback-acl-plugin
  namespace: kong
plugin: acl
disabled: false
config:
  whitelist:
    - admin-group
  hide_groups_header: true

---
# Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: kong-loopback
  namespace: kong
  annotations:
    plugins.konghq.com: loopback-key-auth-plugin, loopback-acl-plugin
spec:
  rules:
    - http:
        paths:
          - path: /kong-admin
            backend:
              serviceName: kong-admin
              servicePort: 8001

---
# Create Admin Consumer
apiVersion: configuration.konghq.com/v1
kind: KongConsumer
metadata:
  name: my-admin
username: my-admin
custom_id: my-admin-id

---
# ACL credentials
apiVersion: configuration.konghq.com/v1
kind: KongCredential
metadata:
  name: admin-group
consumerRef: my-admin
type: acl
config:
  group: admin-group

---
# Key-auth apikey credentials
apiVersion: configuration.konghq.com/v1
kind: KongCredential
metadata:
  name: my-admin-apikey
consumerRef: my-admin
type: key-auth
config:
  key: xxxx
---

Port forward or go to loadbalancer URL
kubectl port-forward -n kong svc/kong-proxy 8081:8443

The following URL should redirect to admin route.
http://localhost:8081/kong-admin/?apikey=xxxxx
Let me know if you have any suggestions on this issue.
Thank you

Port 8001 and 8444 are not exposed in the ingress-kong deployment. You will have to do that.
You also need to change KONG_ADMIN_LISTEN to listen on <IP_ADDRESS> instead of loopback interface.

Do note that any code running inside your k8s cluster will have access to kong's admin api without authn/authz with this setup.

Thanks for looking into this @hbagdi

Port 8001 and 8444 are not exposed in the ingress-kong deployment. You will have to do that.

Just to be clear, are you suggesting to modify the docker image for ingress-kong or simple expose the ports in yaml?  I'm exposing the container ports for the ingress-kong deployment and enabling those in the kong-admin service.

apiVersion: v1
kind: Service
metadata:
  name: kong-admin
  namespace: kong
  labels:
    app: kong-admin
spec:
  type: ClusterIP
  ports:
    - name: admin
      port: 8001
      protocol: TCP
      targetPort: 8001
    - name: admin-ssl
      port: 8444
      targetPort: 8444
      protocol: TCP
  selector:
    app: ingress-kong
---

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: ingress-kong
  name: ingress-kong
  namespace: kong
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ingress-kong
  template:
    metadata:
      annotations:
        kuma.io/gateway: enabled
        prometheus.io/port: "8100"
        prometheus.io/scrape: "true"
        traffic.sidecar.istio.io/includeInboundPorts: ""
      labels:
        app: ingress-kong
    spec:
      containers:
        - env:
            - name: KONG_PROXY_LISTEN
              value: <IP_ADDRESS>:8000, <IP_ADDRESS>:8443 http2 ssl
            - name: KONG_ADMIN_LISTEN
              value: <IP_ADDRESS>:8001, <IP_ADDRESS>:8444 ssl
            - name: KONG_STATUS_LISTEN
              value: <IP_ADDRESS>:8100
            - name: KONG_DATABASE
              value: "off"
            - name: KONG_NGINX_WORKER_PROCESSES
              value: "1"
            - name: KONG_ADMIN_ACCESS_LOG
              value: /dev/stdout
            - name: KONG_ADMIN_ERROR_LOG
              value: /dev/stderr
            - name: KONG_PROXY_ERROR_LOG
              value: /dev/stderr
          image: kong:2.1
          ...
          name: proxy
          ports:
            - containerPort: 8001
              name: admin
              protocol: TCP
            - containerPort: 8444
              name: admin-ssl
              protocol: TCP
            - containerPort: 8000
              name: proxy
              protocol: TCP
            - containerPort: 8443
              name: proxy-ssl
              protocol: TCP
            - containerPort: 8100
              name: metrics
              protocol: TCP
    ...

I also tried implementing the following env parameter for ingress-kong deployment.
- name: KONG_ADMIN_LISTEN
value: <IP_ADDRESS>:8001, <IP_ADDRESS>:8444 ssl

If it helps troubleshoot, when accessing the loopback, it responded with 404.
{ "message": "Not found" }
And the admin API endpoint - {{kong-admin-url}}/services/ responded with
{
    "next": null,
    "data": [
        {
            "host": "kong-admin.kong.8001.svc",
            "id": "a1aec58d-f0ee-5d2e-a258-c4e3a263f255",
            "protocol": "http",
            "read_timeout": 60000,
            "tls_verify_depth": null,
            "port": 80,
            "updated_at":<PHONE_NUMBER>,
            "ca_certificates": null,
            "created_at":<PHONE_NUMBER>,
            "connect_timeout": 60000,
            "write_timeout": 60000,
            "name": "kong.kong-admin.8001",
            "retries": 5,
            "path": "/",
            "tls_verify": null,
            "client_certificate": null,
            "tags": null
        }
    ]
}

Thank you again for your help!

And the admin API endpoint - {{kong-admin-url}}/services/ responded with

Where are you invoking this from? Reading this line in your message makes me wonder if you have achieved what you set out to do and we can close this issue.
If not, what is the error you receive after the changes you did in your last message?

Where are you invoking this from?

To access the Admin endpoint, I port-forwarded svc/kong-admin to my localhost
kubectl  port-forward -n kong svc/kong-admin 8080:8001

Then I made a GET request to http://localhost:8080/services
{
    "next": null,
    "data": [
        {
            "host": "kong-admin.kong.8001.svc",
             ...
        }
    ]
}

If not, what is the error you receive after the changes you did in your last message?

I'm accessing the Loopback endpoint via proxy, I port-forwarded svc/kong-proxy to my localhost
kubectl port-forward -n kong svc/kong-proxy 8081:8443

I made the GET request to http://localhost:8081/kong-admin/services and it threw the following error. Accessing it via the AWS loadbalancer also provided the same response.
{
  "message": "Not found"
}

Can you do kubectl  port-forward -n kong svc/kong-admin 8080:8001 and share the output of http://localhost:8080/routes?

Sure.
http://localhost:8080/routes
{
    "next": null,
    "data": [
        {
            "id": "fc583924-ecef-5687-94e6-fa6b384631f3",
            "path_handling": "v0",
            "paths": [
                "/kong-admin"
            ],
            "destinations": null,
            "headers": null,
            "protocols": [
                "http",
                "https"
            ],
            "created_at":<PHONE_NUMBER>,
            "snis": null,
            "service": {
                "id": "a1aec58d-f0ee-5d2e-a258-c4e3a263f255"
            },
            "name": "kong.kong-loopback.00",
            "hosts": null,
            "preserve_host": true,
            "regex_priority": 0,
            "methods": null,
            "sources": null,
            "updated_at":<PHONE_NUMBER>,
            "https_redirect_status_code": 426,
            "tags": null,
            "strip_path": false
        }
    ]
}

Ah, add konghq.com/strip-path: "true" annotation to the Ingress resource kong-loopback. That should fix this problem.

@hbagdi Oh! That worked. Thank you so much for taking the time to help with this.


get_cn_states() output is misleading for missing samples

The get_cn_states() has a misleading feature in the code that needs to be dropped and deprecated. When the sample id does not have the CNV data, the output of get_cn_states() returns the matrix with neutral CN states instead of accurately reporting the missing value as NA.
The minimal reproducible example that illustrates this problem:
my_regions <- grch37_lymphoma_genes_bed %>%
	head %>%
	mutate(region = paste0(
		chromosome_name,
		":",
		start_position,
		"-",
		end_position
	)) %>%
	pull(region)

my_meta <- get_gambl_metadata() %>%
	filter(sample_id %in% c(
		"10-18191T", # This is chromium sample omitted from CNV workflows and missing the CNV results
		"HTMCP-01-06-00485-01A-01D", # Some random ids
		"012-19-01TD"
	)
)

# Now running the get_cn_states produces this matrix
get_cn_states(
	regions_list = my_regions,
	these_samples_metadata = my_meta
)

                          1:2487078-2496821 1:6581407-6614595 1:6650784-6674667 1:9711790-9789172 1:11166592-11322564 1:12227060-12269285
10-18191T                                 2                 2                 2                 2                   2                   2
012-19-01TD                               2                 1                 1                 1                   1                   2
HTMCP-01-06-00485-01A-01D                 2                 2                 2                 2                   2                   2

# But the real reason the first sample has all diploids is because it does not have CNV outputs
# We can double-check this by running
get_sample_cn_segments(
	this_sample_id = my_meta$sample_id
) %>%
	pull(ID) %>%
	unique

[1] "HTMCP-01-06-00485-01A-01D" "012-19-01TD"

# More wild example where one of the IDs is clobbered
clobbered_meta <- my_meta %>%
	mutate(sample_id = c("Imaginary_sample", my_meta$sample_id[2:3]))

get_cn_states(
	regions_list = my_regions,
	these_samples_metadata = clobbered_meta
)

                          1:2487078-2496821 1:6581407-6614595 1:6650784-6674667 1:9711790-9789172 1:11166592-11322564 1:12227060-12269285
10-18191T                                 2                 2                 2                 2                   2                   2
Imaginary_sample                          2                 2                 2                 2                   2                   2
HTMCP-01-06-00485-01A-01D                 2                 2                 2                 2                   2                   2

This part has to be updated to keep the NA values instead of replacing them.

I think this is a good suggestion to have this configurable so we respect the legacy behavior but also have the flexibility to return NAs. Thanks!

As get_cn_states is, it cannot differentiate samples that don't contain CN from samples that don't exist --- for both cases, the function generates NA values. Also, I think getting a zero for the first case would be more informative than a NA. Would it be interesting to make this change? For example, we could get outputs like this:
                          1:2487078-2496821 1:6581407-6614595 1:6650784-6674667 1:9711790-9789172 1:11166592-11322564
10-18191T                                 0                 0                 0                 0                   0
Imaginary_sample                         NA                NA                NA                NA                  NA
HTMCP-01-06-00485-01A-01D                 2                 2                 2                 2                   2
                          1:12227060-12269285
10-18191T                                   0
Imaginary_sample                           NA
HTMCP-01-06-00485-01A-01D                   2

0 in this case would mean that there are 0 copies of DNA in that region (or gene), so would be interpreted as deletions. Returning 2 for samples that don't contain CN changes would just mean that they have a diploid state, and returning NA for those that don't exist would mean that this information is not available. So the 2 vs NA will be the distinction. So the table in this small example would be
                          1:2487078-2496821 1:6581407-6614595 1:6650784-6674667 1:9711790-9789172 1:11166592-11322564
10-18191T                                 NA                 NA                 NA                 NA                   NA
Imaginary_sample                         NA                NA                NA                NA                  NA
HTMCP-01-06-00485-01A-01D                 2                 2                 2                 2                   2
                          1:12227060-12269285
10-18191T                                   NA
Imaginary_sample                           NA
HTMCP-01-06-00485-01A-01D                   2

It will mean that there is no data available for 10-18191T and Imaginary_sample, and these regions in HTMCP-01-06-00485-01A-01D are all diploid.

My interpretation is that 10-18191T exists and there are no copies of DNA in that region (deletions). Isn't this right?
If 10-18191T exists, the function shouldn't return NA for it, should it?

The sample itself exists but the copy number data for it is not - because it is intentionally deleted from the workflows. So for this application, it is the same as Imaginary_sample - the NA should be returned. It is easy to check whether or not the data is available, in this example it is
get_sample_cn_segments(
	this_sample_id = my_meta$sample_id
) %>%
	pull(ID) %>%
	unique

The code you posted in the earlier comment regarding the missing_data_as_diploid parameter is good and can be implemented as is 👍

I completely understood. Thank you so much for the explanation. I'm going to make the Pull Request.

Thanks for looking at it!

The issue was fixed by PR #210.
I'm going to close the issue.


Provide the size of the disk image, along with the checksum and url

Description
I know that one can ask the server, but maybe it is down or slow.
When we automate the checksumming ("digest"), we could have it populate the byte size as well ?
images:
# Try to use release-yyyyMMdd image if available. Note that release-yyyyMMdd will be removed after several months.
- location: "https://cloud-images.ubuntu.com/releases/22.10/release-20230413/ubuntu-22.10-server-cloudimg-amd64.img"
  arch: "x86_64"
  size: 762445824
  digest: "sha256:d46438111d7a6dfcdfb2e1fb76762a79d56f09dc308cc9ab2300effc0078fd5b"
- location: "https://cloud-images.ubuntu.com/releases/22.10/release-20230413/ubuntu-22.10-server-cloudimg-arm64.img"
  arch: "aarch64"
  size: 739573760
  digest: "sha256:e1865c2ebc531600ae6012d86fa2dae5846e7801cdcdbcf90d9c830218ef4ae9"
# Fallback to the latest release image.
# Hint: run `limactl prune` to invalidate the cache
- location: "https://cloud-images.ubuntu.com/releases/22.10/release/ubuntu-22.10-server-cloudimg-amd64.img"
  arch: "x86_64"
- location: "https://cloud-images.ubuntu.com/releases/22.10/release/ubuntu-22.10-server-cloudimg-arm64.img"
  arch: "aarch64"

A disk size of "0" would just mean that we don't know, before contacting (or that the image is generic)

https://github.com/lima-vm/lima/issues/1347

The reason for this was the question of "which template has the bigger images, ubuntu or debian"

Any reason not to just use comment lines?

Wouldn't that make the parsing harder ? I'm imaging some kind of tool or visualization, similar to what we currently do for du

Like if you want to use some parsing construct, avoid hitting the remote server for known information....
limactl info | yq '.templates[] | select(.name == "debian") | .location' | xargs yq .images

I'm fine to add the size field, but I expect it to happen after this:

https://github.com/lima-vm/lima/issues/1347

Otherwise maintaining the templates will be even much harder.

I'm not sure how important disk size really is, these days.
If one really wanted to complicate things, it could even record the uncompressed size (like apt shows you)
But in general, I think you want to have plenty of disk space (10G+) available when running virtual machines.


Node device could not get IP address from Root when closing The Access Point (IDFGH-7741)

Environment

Development Kit:      ESP32-C3-DevKitM-1
IDF version:  framework-espidf @ 3.40302.0 (4.3.2)
Build System:         [Make|CMake|idf.py]
Compiler version : xtensa-esp32s2 @ 8.4.0+2021r2-patch2
Operating System:     Windows
Environment type: MSYS2 mingw32
Using an IDE: Platformio on VS code
Power Supply:         USB

Problem Description
Using The source code of Mesh IP Internal Networking example  the Node device in Mesh network couldn't get IP address from the root (DHCP server) after closing the Access point.
Expected Behavior
After closing the AP and press the reset button of the Node device, The root which is the DHCP server forward an IP address to the Node.
Actual Behavior
The node couldn't have an IP address even after opening the AP and this message on the pc terminal side of The node is diplayed : "dhcp_select: could not allocate DHCP request".
Steps to reproduce

Power up  two dev kits
Open an Acess point
Mesh network established and each device obtained an IP address 5 one device is elected as node and the other one as node)
Turn OFF the AP
Press the Reset button of the NODE device

Code to reproduce this issue
https://github.com/espressif/esp-idf/blob/master/examples/mesh/ip_internal_network/main/mesh_main.c
Ps:  When we comment these lines in the source code below the problem is solved:

Debug Logs

We will describe this situation in the documentation. Thank you for your contribution.

I found the solution, when The AP is closed, we must stop the DHCP server on root node  using esp_netif_dhcps_stop function and start it when the AP reported

I found the solution, when The AP is closed, we must stop the DHCP server on root node using esp_netif_dhcps_stop function and start it when the AP reported


[BUG] Autoscroll spams notify when there are too many logs

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
As we all have experienced, sometimes the Flutter dev log (especially in Android) can be spammed with messages. When this happens, the autoscroll behavior starts spamming vim.notify, which eventually makes my IDE unusable. I'm unable to get past the endless notifications from the autoscroll to even kill Flutter.
50,000+ lines of ADB Log Spam

vim.notify Spam

Expected Behavior
I would like to be able to suppress all of those notifications. I get it, ADB is spamming the console output. But it shouldn't hang up my entire IDE.
I don't know that the autoscroll should even ping vim.notify. Maybe suppress this message entirely. Maybe only log it to the dev logs? Maybe allow someone to suppress this particular message through configuration settings.
Steps To Reproduce
Config
  flutter_path = "/usr/local/lib/fvm/default",
  fvm = true,
  lsp = {
    on_attach = on_attach,
  },

Example Code?

while (true) {
  await Future.delayed(Duration(milliseconds: 1));
  print('SPAM');
}

Run some code which spams output to ADB. Maybe something like the above? For me it happens with Flutter's Google Maps library (I get :  BufferQueueProducer ... BufferQueue has been abandoned spam).
Run Flutter using flutter-tools.
Keep the __FLUTTER_DEV_LOG__ buffer open with autoscroll enabled.
Move your cursor to some line on the buffer.
Get spammed with notifications and the IDE becomes unusable.

Environment
- OS:Pop!_OS
- Flutter version: 3.7.9
- Is flutter in $PATH: FVM is
- neovim version: v0.10.0-dev

Anything else?
I love this project. Also, I'm using NVChad if that's relevant? If there's anything else I can help with or if you need additional info, please let me know!

Hi,
Thanks for the report. I think this might be able to be fixed by using
vim.notify_once instead of vim.notify there.
https://github.com/akinsho/flutter-tools.nvim/blob/b3854c50aef753ed52bf15517af7eaa82b46a339/lua/flutter-tools/log.lua#L65
can you see if changing that line to use that function fixes the issue for you locally.
If so would appreciate a PR with that change 🙏🏾


fix(cache): ResourceType::GUILD required for other resources

Closes: #1553

not the best place to suggest this but how about separating GUILDS to ROLES, NICKS etc?

not the best place to suggest this but how about separating GUILDS to ROLES, NICKS etc?

For roles we already have the ROLE resource type and nicknames should be covered by the MEMBERS one.

This PR should include a review of documentation stating that ResourceType::GUILD is required (For example, InMemoryCachePermissions::in_channel requires ResourceType::GUILD for checking if user_id is the owner, but functions otherwise)


feat: add link to Twilio Console logs to deploy/activate

This adds a link to the logs in the Twilio Console for increased visibility of the current solution. If the browser supports terminal links (like iTerm) it will display a clickable link "Open the Twilio Console" otherwise it will display the very long URL.
Contributing to Twilio

All third-party contributors acknowledge that any contributions they provide will be made under the same open-source license that the open-source project is provided under.

[x] I acknowledge that all my contributions will be made under the project's license.

Ping :)


FEATURE-RELEASE: add nodejs 14 support

Description
Recently AIO made a switch to default custom apps to use nodejs:14 as the kind. We need our cli plugin to also support this.

added nodejs:14 to list of accepted kinds
updated aio dependencies to new versions that support node 14
added ci run in node 14
added test-worker test case with a worker that uses nodejs:14 in the manifest.yml

Fixes # [no jira] failing integration tests: https://github.com/adobe/asset-compute-integration-tests/actions/runs/879540103
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

note: future improvement, to find kind dynamically: https://github.com/apache/openwhisk-wskdebug/blob/master/src/invoker.js#L89-L114


Move streams and query expressions to language concepts

Purpose

Move streams and query expressions to language concepts

Check List

[ ] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #21963 into stage will not change coverage by %.
The diff coverage is n/a.

@@           Coverage Diff           @@
##            stage   #21963   +/-   ##
=======================================
  Coverage   14.59%   14.59%
=======================================
  Files          51       51
  Lines        1411     1411
  Branches      219      219
=======================================
  Hits          206      206
  Misses       1189     1189
  Partials       16       16

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 102275b...014d50d. Read the comment docs.


Extra characters added during commit messages

Hi,
Thank you for maintaining this neovim plugin. I didn't know that I needed lazygit in my life until I tried it and it has helped me greatly.
I have recently updated my lunarvim config (opinionated wrapper around neovim) and now when I add a space to the commit message it automatically adds " and + as if it concatenate the string.
I am unsure, if you have experiences that error on your end or if I need to raise on the lazyGit repo. Please find attached a video showing my problem.

nvim --version:

NVIM v0.6.1
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by runner@fv-az87-780
Thank you for your time

I don't know what happened but the bug stopped.
Sorry for the churn.


Fix #1748: Return canonical files from CrossType.sharedSrcDir().

Some tools, such as scoverage, are not happy if they receive paths to source files that contain '..' in them. So we avoid creating such paths using .getParentFile instead of / "..".

Review by @gzm0

LGTM

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/1876/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/2424/
Test FAILed.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://scala-webapps.epfl.ch/jenkins/job/scalajs-pr/1886/
https://scala-webapps.epfl.ch/jenkins/job/scalajs-matrix-build/2434/
Test PASSed.


Task :app:bundleReleaseJsAndAssets : Module mattermost-redux/actions/users does not exist in the Haste module map

Summary
When building release, during bundleReleaseJsAndAssets task, process fails with error:
> Task :app:bundleReleaseJsAndAssets
warning: the transform cache was reset.
Loading dependency graph, done.
BUILDING MODULES FOR android
error Unable to resolve module `mattermost-redux/actions/users` from `/root/mattermost-mobile/app/mattermost.js`: Module `mattermost-redux/actions/users` does not exist in the Haste module map

This might be related to https://github.com/facebook/react-native/issues/4968
To resolve try the following:
  1. Clear watchman watches: `watchman watch-del-all`.
  2. Delete the `node_modules` folder: `rm -rf node_modules && npm install`.
  3. Reset Metro Bundler cache: `rm -rf /tmp/metro-bundler-cache-*` or `npm start -- --reset-cache`.
  4. Remove haste cache: `rm -rf /tmp/haste-map-react-native-packager-*`.. Run CLI with --verbose flag for more details.

The suggested solution doesn't improve anything.
Environment Information

OS Version: Debian 10.0
Mattermost App Version: master branch

Steps to reproduce
my env:
export APP_NAME="mattermost-univnantes"
export MAIN_APP_IDENTIFIER=com.univnantes.mattermost
export BUILD_FOR_RELEASE=true
export BETA_BUILD=false
export MATTERMOST_RELEASE_STORE_FILE=~/mattermost-univnantes-release.keystore
export MATTERMOST_RELEASE_KEY_ALIAS=mattermost-univnantes
export MATTERMOST_RELEASE_PASSWORD=xxxxxxxxx
export BRANCH_TO_BUILD=master
export REPLACE_ASSETS=true

nodejs: 10.16.0
npm: 6.9.0
I start building with the following command
make build-android

am I the only one having the problem?

@ArnY I have the same problem after fetching current code from upstream

same issue. new repo doesn't include this file mattermost-redux/actions/users

I'm also having this issue trying to build the app for iOS. Did any of you who had this before manage to solve it?

rm -rf node_modules
npm cache clean --force
npm i
This one worked for me

Hello @valentijnnieman
The make build-android process already remove the modules and clears the cache. This doesn't fix anything for me, unfortunately...

Going to close this issue as it hasn’t had any recent activity