Does TFoS support Spark2 Livy?

I installed Sparkmagic on jupyter and want to submit the yarn task to Spark cluster through Livy service, but TFoS doesn't work and the cluster can only get one node's information.The exception information is as follows:

17/10/31 19:01:19 INFO RMProxy: Connecting to ResourceManager at idap-agent-19.idap.com/<IP_ADDRESS>:8030
17/10/31 19:01:19 INFO YarnRMClient: Registering the ApplicationMaster
17/10/31 19:01:19 INFO YarnAllocator: Will request 2 executor container(s), each with 1 core(s) and 1408 MB memory (including 384 MB of overhead)
17/10/31 19:01:19 INFO YarnAllocator: Submitted 2 unlocalized container requests.
17/10/31 19:01:19 INFO ApplicationMaster: Started progress reporter thread with (heartbeat : 3000, initial allocation : 200) intervals
17/10/31 19:01:20 INFO AMRMClientImpl: Received new token for : idap-agent-19.idap.com:45454
17/10/31 19:01:20 INFO YarnAllocator: Launching container container_e11_1509069317335_0063_01_000002 on host idap-agent-19.idap.com
17/10/31 19:01:20 INFO YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them.
17/10/31 19:01:20 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
17/10/31 19:01:20 INFO ContainerManagementProtocolProxy: Opening proxy : idap-agent-19.idap.com:45454
17/10/31 19:01:21 INFO YarnAllocator: Launching container container_e11_1509069317335_0063_01_000003 on host idap-agent-19.idap.com
17/10/31 19:01:21 INFO YarnAllocator: Received 1 containers from YARN, launching executors on 1 of them.
17/10/31 19:01:21 INFO ContainerManagementProtocolProxy: yarn.client.max-cached-nodemanagers-proxies : 0
17/10/31 19:01:21 INFO ContainerManagementProtocolProxy: Opening proxy : idap-agent-19.idap.com:45454
17/10/31 19:01:23 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (<IP_ADDRESS>:36418) with ID 1
17/10/31 19:01:23 INFO BlockManagerMasterEndpoint: Registering block manager idap-agent-19.idap.com:36105 with 366.3 MB RAM, BlockManagerId(1, idap-agent-19.idap.com, 36105, None)
17/10/31 19:01:23 INFO YarnSchedulerBackend$YarnDriverEndpoint: Registered executor NettyRpcEndpointRef(null) (<IP_ADDRESS>:36420) with ID 2
17/10/31 19:01:23 INFO BlockManagerMasterEndpoint: Registering block manager idap-agent-19.idap.com:39620 with 366.3 MB RAM, BlockManagerId(2, idap-agent-19.idap.com, 39620, None)
17/10/31 19:01:23 INFO YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8
17/10/31 19:01:23 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook done
17/10/31 19:01:24 INFO SharedState: Warehouse path is 'file:/hadoop/yarn/local/usercache/livy/appcache/application_1509069317335_0063/container_e11_1509069317335_0063_01_000001/spark-warehouse'.
17/10/31 19:01:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@49c40604{/SQL,null,AVAILABLE,@Spark}
17/10/31 19:01:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@71a48037{/SQL/json,null,AVAILABLE,@Spark}
17/10/31 19:01:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@153ae97a{/SQL/execution,null,AVAILABLE,@Spark}
17/10/31 19:01:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@23b9714a{/SQL/execution/json,null,AVAILABLE,@Spark}
17/10/31 19:01:24 INFO ContextHandler: Started o.s.j.s.ServletContextHandler@6c9d2a74{/static/sql,null,AVAILABLE,@Spark}
17/10/31 19:01:24 INFO HiveUtils: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes.
17/10/31 19:01:24 INFO YarnAllocator: Received 1 containers from YARN, launching executors on 0 of them.
17/10/31 19:01:24 INFO metastore: Trying to connect to metastore with URI thrift://idap-agent-19.idap.com:9083
17/10/31 19:01:24 INFO metastore: Connected to metastore.
17/10/31 19:01:24 INFO SessionState: Created local directory: /hadoop/yarn/local/usercache/livy/appcache/application_1509069317335_0063/container_e11_1509069317335_0063_01_000001/tmp/yarn
17/10/31 19:01:24 INFO SessionState: Created local directory: /hadoop/yarn/local/usercache/livy/appcache/application_1509069317335_0063/container_e11_1509069317335_0063_01_000001/tmp/0cde2f14-e7e7-4d3b-8d0f-011957cd9865_resources
17/10/31 19:01:24 INFO SessionState: Created HDFS directory: /tmp/hive/livy/0cde2f14-e7e7-4d3b-8d0f-011957cd9865
17/10/31 19:01:24 INFO SessionState: Created local directory: /hadoop/yarn/local/usercache/livy/appcache/application_1509069317335_0063/container_e11_1509069317335_0063_01_000001/tmp/yarn/0cde2f14-e7e7-4d3b-8d0f-011957cd9865
17/10/31 19:01:24 INFO SessionState: Created HDFS directory: /tmp/hive/livy/0cde2f14-e7e7-4d3b-8d0f-011957cd9865/_tmp_space.db
17/10/31 19:01:24 INFO HiveClientImpl: Warehouse location for Hive client (version 1.2.1) is file:/hadoop/yarn/local/usercache/livy/appcache/application_1509069317335_0063/container_e11_1509069317335_0063_01_000001/spark-warehouse
17/10/31 19:11:56 INFO SparkContext: Starting job: foreachPartition at /opt/conda/lib/python3.5/site-packages/tensorflowonspark/TFCluster.py:253
17/10/31 19:11:56 INFO DAGScheduler: Got job 0 (foreachPartition at /opt/conda/lib/python3.5/site-packages/tensorflowonspark/TFCluster.py:253) with 3 output partitions
17/10/31 19:11:56 INFO DAGScheduler: Final stage: ResultStage 0 (foreachPartition at /opt/conda/lib/python3.5/site-packages/tensorflowonspark/TFCluster.py:253)
17/10/31 19:11:56 INFO DAGScheduler: Parents of final stage: List()
17/10/31 19:11:56 INFO DAGScheduler: Missing parents: List()
17/10/31 19:11:56 INFO DAGScheduler: Submitting ResultStage 0 (PythonRDD[1] at foreachPartition at /opt/conda/lib/python3.5/site-packages/tensorflowonspark/TFCluster.py:253), which has no missing parents
17/10/31 19:11:56 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 64.0 KB, free 912.2 MB)
17/10/31 19:11:56 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 17.7 KB, free 912.2 MB)
17/10/31 19:11:56 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on <IP_ADDRESS>:43198 (size: 17.7 KB, free: 912.3 MB)
17/10/31 19:11:56 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:996
17/10/31 19:11:56 INFO DAGScheduler: Submitting 3 missing tasks from ResultStage 0 (PythonRDD[1] at foreachPartition at /opt/conda/lib/python3.5/site-packages/tensorflowonspark/TFCluster.py:253)
17/10/31 19:11:56 INFO YarnClusterScheduler: Adding task set 0.0 with 3 tasks
17/10/31 19:11:57 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, idap-agent-19.idap.com, executor 1, partition 0, PROCESS_LOCAL, 6183 bytes)
17/10/31 19:11:57 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, idap-agent-19.idap.com, executor 2, partition 1, PROCESS_LOCAL, 6183 bytes)
17/10/31 19:11:57 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on idap-agent-19.idap.com:39620 (size: 17.7 KB, free: 366.3 MB)
17/10/31 19:11:57 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on idap-agent-19.idap.com:36105 (size: 17.7 KB, free: 366.3 MB)

I don't see any errors in those logs... what errors/exceptions are you seeing?

@leewyang Thank you for your reply.  HDFS doesn't have permission to read the file. Now it's ok. Thanks again


error:  no reads in first split: bad BAM file or tiny split size?

I keep receiving the following error while trying to convert a bam file into adam format.
java.io.IOException: 'file:/my.bam': no reads in first split: bad BAM file or tiny split size?

Additional info:

I'm able to run the small.sam test set
I've regenerated my bam file multiple times and checked for any kind of corruption, etc
I've repeated this experiment on both a local environment (spark in single machine mode), and in EMR.
spark-1.5.2-bin-hadoop2.6
adam-distribution_2.10-0.18.2
scala version 2.10.6

Full stack trace: https://gist.github.com/astewart-twist/ceb36e492f96fe2a2500

@fnothaft @jpdna I was never able to reproduce this, might it be related to #962?

Nah, #962 would be output only, not input.

Ah right, sorry, wasn't looking closely enough

Fixed by https://github.com/HadoopGenomics/Hadoop-BAM/pull/80 upstream, will have to wait for an update of Hadoop-BAM dependency version to 7.4.1 (or later as appropriate) to pick it up.

See #989 which uses Hadoop-BAM version 7.4.1-SNAPSHOT to pick up the fix for this issue.  Currently blocked on Jenkins environment changes.

@shaneknapp should be back online tomorrow. I looked and alas, I do not have permissions to make the @AmplabJenkins changes.

May be fixed by #989, please retest if possible.

Closing as fixed by #989.


RaspiBlitz in a Proxmox VM

Check this discussion for a general VM/Proxmox:
Discussed in https://github.com/raspiblitz/raspiblitz/discussions/4282

Originally posted by openoms November 29, 2023
Just had a successful RaspiBlitz install in a Proxmox VM.  Install went very smooth and had no issues.  I'm running Proxmox VE 8.1.3 and used Debian 12.2.0 for the VM.  Hardware is a Ryzen 5700g, 32GB DDR4 RAM, and a Samsung 990Pro 4TB NVME SSD.
Steps taken

Setup the VM in Proxmox to install Debian, CPU set to Host, and allotted a 64GB drive on the 990Pro to emulate the SD card.
Started the VM and installed Debian with no GUI.
Booted into the VM, switched to root and installed sudo with ' apt install sudo '
Downloaded the RaspiBlitz build script with ' wget https://raw.githubusercontent.com/rootzoll/raspiblitz/dev/build_sdcard.sh '
Ran the build script with ' sudo bash build_sdcard.sh -f true -b dev -d headless -t false -w off '
When the script finished the install I shut down the VM.
Configured a second 1.5TB SSD on the 990Pro for the VM to emulate the data drive and restarted the VM.
SSH into the VM and the setup took off just like on the Raspberry Pi 4.

So far I have Electrs, Mempool, JoininBox, and C-Lightning running successfully.  I have Zeus connected to the node as well.  Happy to say this install was completely painless and it runs MUCH faster than on the Pi.  Just thought I'd share my experience.
Originally posted by @Silarous in https://github.com/raspiblitz/raspiblitz/issues/2104#issuecomment-1828956785

One solution was: echo "nameserver <IP_ADDRESS>" | sudo tee /etc/resolv.conf

Will be fixed with #4715

#4715 was not enough ... when installing on Proxmox following:
https://cercatrova.blog/raspiblitz-auf-proxmox-installieren/
the build script fails:
Reading state information...
The following NEW packages will be installed:
  resolvconf
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 55.6 kB of archives.
After this operation, 184 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bookworm/main amd64 resolvconf all 1.91+nmu1 [55.6 kB]
Fetched 55.6 kB in 0s (707 kB/s)
Preconfiguring packages ...
/var/cache/debconf/tmp.ci/resolvconf.config.NcD1pJ: 13: ifquery: not found
/var/cache/debconf/tmp.ci/resolvconf.config.NcD1pJ: 13: ifquery: not found
Selecting previously unselected package resolvconf.
(Reading database ... 28393 files and directories currently installed.)
Preparing to unpack .../resolvconf_1.91+nmu1_all.deb ...
Unpacking resolvconf (1.91+nmu1) ...
Setting up resolvconf (1.91+nmu1) ...
Created symlink /etc/systemd/system/sysinit.target.wants/resolvconf.service → /lib/systemd/system/resolvconf.service.
Created symlink /etc/systemd/system/systemd-resolved.service.wants/resolvconf-pull-resolved.path → /lib/systemd/system/resolvconf-pull-resolved.path.
Unit /lib/systemd/system/resolvconf-pull-resolved.path is added as a dependency to a non-existent unit systemd-resolved.service.
Created symlink /etc/systemd/system/systemd-resolved.service.wants/resolvconf-pull-resolved.service → /lib/systemd/system/resolvconf-pull-resolved.service.
Unit /lib/systemd/system/resolvconf-pull-resolved.service is added as a dependency to a non-existent unit systemd-resolved.service.
Processing triggers for man-db (2.11.2-2) ...
Processing triggers for resolvconf (1.91+nmu1) ...
build_sdcard.sh: line 384: resolvconf: command not found
Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  sudo
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,889 kB of archives.
After this operation, 6,199 kB of additional disk space will be used.

FAIL! apt-get failed to install package: sudo

to things to target here:

build_sdcard.sh: line 384: resolvconf: command not found
FAIL! apt-get failed to install package: sudo

Looks good for the first fix - still one thing to fix: even its started with -d headless it tries to install lcd driver in the end:
   Compiling electrs-rocksdb v0.19.0-e3
    Finished release [optimized] target(s) in 20m 27s
### blitz.display.sh set-display
# raspi_configfile(/boot/config.txt)
# raspi_commandfile(/boot/cmdline.txt)
# blitz.display.sh set-display lcd
baseimage(debian)
# old(headless)
# new(lcd)
err='baseimage not supported'

Tested minimal build - that stays with the headless setting. Might just be fatpack that is doing the lcd install.


Sema: Try a little harder to infer associated types to generic parameters if all else fails

If we have an abstract witness, we don't attempt a generic parameter
binding at all. But if simplifying the abstract witness failed, we
should still attempt it.
This would be cleaner as a disjunction in the solver but I want
to change behavior as little as possible, so this adds a new fallback
that we run when all else fails.
Fixes rdar://problem/123345520.

@swift-ci Please smoke test

@swift-ci Please smoke test

@swift-ci Please test source compatibility

@swift-ci Please smoke test


autest: log port # when port is recycled

I think someone will need to login to the autest box and add the --debug --verbose (not --trace) to the autest script or the script that invokes autest.sh

[approve ci autest]


Is it possible to give color gradient in Heatmap

Hello, is there a way to specify a range of colors(with shades of colors) based on value range(0-10 is one color and 10-20 is another color etc) to be used within the heatmap.

We don't support any smart type of binning like that, but I think you can do something like manually like
<HeatmapSeries data={data} colorType="literal" getColor={d => {
   if (d.color < 10) {
       return 'red';
   } else if (d.color < 20) {
        return 'rgba(255, 255, 0, 1)';
   } else {
        return '#ff0196';
   }
}}/>

Yeah will try that , Thank you


Fonts used by Wagtail doesn't look the same for all languages

Fonts used by Wagtail doesn't look the same for all languages. As example, take a look at how it looks like when mixing english words and russian:

Note "email" between other words.
For example, Ubuntu font looks the same for all languages!

The font is Open Sans, which I believe should support the Russian alphabet https://www.google.com/fonts/specimen/Open+Sans however comparing the size of the font files in the repo, against a fresh download of Open Sans from Google Fonts with all the extended characters enabled, it suggests we're not currently included the necessary Cyrillic characters. They were probably omitted originally to save on file-size.

Also note that field labels are not translated somehow, but I'm submitted russian translation with 100% coverage. Didn't checked out if these words are actually not enabled for translation or translations didn't synced yet.

Translations don't get synced with Transifex automatically - we need to pull them manually. We'll do this before the 1.0 release.

Hi @MechanisM there's another custom text font used in the page editor interface: Bitter. It's used for any charFields where you've applied the "title" or "full" classnames configurations in your model (See http://wagtail.readthedocs.org/en/latest/pages/editing_api.html?highlight=title#defining-panels)
As far as I can tell this font is not available with cyrillic characters, which is a bit of a show-stopper. It would really help if you could try out title or full and screenshot the result with Russian characters.
Its only while procuring the cyrillic version of Open Sans that I've realised Bitter might not be international enough for use in Wagtail.

A close alternative to Bitter, one of only 5 serif fonts that Google Fonts provides with Extended Cyrillic characters, is https://www.google.com/fonts/specimen/Roboto+Slab
I'm tempted to remove the Serif font completely, given how little it's used. It could still be added back to the page editor interface on a per-implementation basis with wagtail's hooks if so desired.

Merged in e08e35d + parents.


Reposense fix

Added author name for afiqlattif

Coverage remained the same at 71.675% when pulling 633ce618f23153ad7cb7b91809ace1bbb0d38ab8 on afiqlattif:reposense-fix into c8b3e53254cbbb0b46a5143806dcb14d6a572f01 on CS2103-AY1819S1-W16-2:master.


Possible to force umask 0002 when installing npm packages

Is there a way when installing things with npm to set the umask. We wish to force the umask to be 0002.

isaacs,
So first npm does not seem to respect the sticky bit (SGID) for groups. If we create a directory in a directly that has the SGID set for group deploy:
$ mkdir test
drwxr-sr-x 2 root deploy 4096 Oct  7 15:07 test/

But when we install a package with npm in that same directory with the SGID set to the deploy group:
$ npm install socket.io
drwxr-sr-x 3 root root   4096 Oct  7 15:01 node_modules/
drwxr-xr-x 7 root root   4096 Oct  7 15:01 socket.io/

Notice the group owner is root, naughty, should be deploy right?

Sorry, the group and SGID problem we fixed with simply:
npm config set group deploy

Sorry, should have read your documentation better.
But we still have the problem with permissions. The folders are be created as:
drwxr-sr-x 3 root deploy 4096 Oct  7 17:09 node_modules/
    drwxr-xr-x 7 root deploy 4096 Oct  7 17:09 socket.io/

Need to be:
drwxrwsr-x 3 root deploy 4096 Oct  7 17:09 node_modules/
    drwxrwxr-x 7 root deploy 4096 Oct  7 17:09 socket.io/

Is there a way to automate this, so NPM automatically sets the umask to 0002, instead of manually having to set it everytime?

Aha, I see.
npm always creates folders and binary files as 0755, and other files
as 0644.  (Except for the cache, which is 100% 0644.)
It seems like it would be worthwhile to make the umask configurable,
and then mask 0777 or 0666 against that umask config, rather than
hard-coding 0755 and 0644.
On Fri, Oct 7, 2011 at 17:13, NodeSocket
<EMAIL_ADDRESS>wrote:

Sorry, the group and SGID problem we fixed with simply:
npm config set group deploy

Sorry, should have read your documentation better.
But we still have the problem with permissions. The folders are be created as:
drwxr-sr-x 3 root deploy 4096 Oct  7 17:09 node_modules/
   drwxr-xr-x 7 root deploy 4096 Oct  7 17:09 socket.io/

Need to be:
drwxrwsr-x 3 root deploy 4096 Oct  7 17:09 node_modules/
   drwxrwxr-x 7 root deploy 4096 Oct  7 17:09 socket.io/

Is there a way to automate this, so NPM automatically sets the umask to 0002, instead of manually having to set it everytime?
--
Reply to this email directly or view it on GitHub:
https://github.com/isaacs/npm/issues/1509#issuecomment-2329781

issacs,
Yeah would we awesome to be able to do something like:
npm config set umask 0002

Which would set the following permissions:
Directories/Binary: 775
Files: 664

So, the config is there now, but it's not working 100%, because tar creates most of those files, and I don't go through and re-chmod them.

Easier than I'd thought.  There was a stray process.umask() in there.  Fixed on c0df906b4f35d5f8ab74eb345f695ce53aff938d

isaacs,
Your awesome, when will this go live and into production?

todayish.

Beautiful, thanks for the taking the time to implement this. Confirmed it works.

Big problem, when we login to a virtual machine:
ls /root

The permissions are rwxr-x--- as they should be. Then we execute:
npm config get umask

02

Now when we do
ls /root

The permissions got changed to:
rwxrwxrwx. This is a problem for a lot of reasons, but one being that public ssh keys no longer work, since the permissions on that folder are insecure.

I see.  Very trickly.  Fixed in 1.0.101

Appreciate your speedy response. Timetable for 1.0.101 to go prime-time?

In teh next hour.  This is the only change from 100, so it's pretty easy.

published now.

Confirmed fixed. Thanks.


issue with Scratch image

This examples in the page don't work. The C code referenced is not a valid C program(no main) and the example of building the C code with Docker doesn't work. However, the description does give some insight into building a base image.
File: develop/develop-images/baseimages.md, CC @johndmulhausen

@mistyhacks , do you have time to take a look or recommend someone?

The hello.c is not complete on its own. The link should point to https://github.com/docker-library/hello-world instead of straight to the hello.c.


[Broker] Skip response when producerFuture is already complete

Motivation
This PR fixes an issue introduced in branch-2.7 by one of my commits. @codelipenghui has some context in this PR https://github.com/apache/pulsar/pull/13245 and proposes the opposite solution.
The behavior here aligns with https://github.com/apache/pulsar/commit/d12486b9b012fc8264fa0d4cb7f4e3e148c30187. Because branch-2.7 does not have this commit, one of my tests in the cherry picked commits is failing.
I think we need to update branch-2.7 to be like master and to only respond if the future is completed by this exception. From my perspective, the contract in the class is that if a thread completes the future, it should also send (or intentionally not send) a response. In the failing test case, the producer future is completed by a CloseProducer command.
Alternatively, we could update the test if we don't want to change this client behavior.
Modifications

If the producer fails to get created, only send a response to the client if the producerFuture is not already completed.

Verifying this change
I verified this test by running the ServerCnxTest locally with the proposed change. It passed. This change is also already i master.
Does this pull request potentially affect one of the following parts:
This change affects the binary protocol because it skips sending a response to the client. However, it only skips it when the future is already completed, and the completion implies that the ServerCnx has already handled the request.
Documentation

[x] no-need-doc

I talked this through with @codelipenghui over slack, and we agree that we should not change the behavior in branch 2.7. He committed the test fix here: https://github.com/apache/pulsar/commit/e99777a55740470aa2d807ecc32093b871d28f6e#diff-b73845cd7706e03b89122a21b3e60c36c95ce6a0c7dd4574f8eda3fc67dd02b1R875-R878.


Support for Azure function CosmosDBTrigger

I want to migrate my existing codebase to an AKS cluster, but I've noticed you don't have support for CosmosDBTrigger according to https://docs.microsoft.com/en-us/azure/azure-functions/functions-kubernetes-keda. If possible, please add support for this trigger.
Use-Case
By using a CosmosDBTrigger, I can listen to messages in Change Feed and create an event-driven architecture that leverages Cosmos DB. An ideal implementation would create as many leases as there are active instances running. Leases should be load-balanced as active instance count changes.

Definitely a good use-case, we are already tracking it on https://github.com/kedacore/keda/issues/232.
Thanks for suggesting!

Definitely a good use-case, we are already tracking it on https://github.com/kedacore/keda/issues/232.
Thanks for suggesting!


How get blob?

Hi
I try to export blob from an oracle database with this piece of code :
rows, err := db.Query(`select hash, content from "BLOBSTORE"."BLOB" where rownum < 10`)
	if err != nil {
		log.Fatal(err)
	}

	defer rows.Close()

    for rows.Next() {
        //row structure
		var hash string
        var content []byte
		err = rows.Scan(&hash, &content)
        checkErr(err)
        ...
```

And at the execution, I've got this :

```
➜  gotothefs ./blober -alsologtostderr=true
connection
query
I0406 10:48:27.675358   40107 util.go:187] [Drv.Open]
I0406 10:48:27.675751   40107 env.go:299] E1 [Env.OpenCon]
I0406 10:48:27.675790   40107 env.go:299] E1 [Env.OpenSrv]
I0406 10:48:27.765079   40107 srv.go:322] E1S1 [Srv.OpenSes]
I0406 10:48:27.849781   40107 ses.go:637] E1S1S1 [Ses.PrepAndQry]
I0406 10:48:27.849821   40107 ses.go:639] E1S1S1 [Ses.Prep] SELECT property_value FROM database_properties WHERE property_name = 'NLS_CHARACTERSET'
I0406 10:48:27.850010   40107 stmt.go:1154] E1S1S1S1 [Stmt.qry]
I0406 10:48:27.850027   40107 stmt.go:1167] E1S1S1S1 [Stmt.bind] Params 0
I0406 10:48:27.868394   40107 rset.go:772] E1S1S1S1R1 [Rset.open]
I0406 10:48:27.868498   40107 rset.go:785] E1S1S1S1R1 [Rset.open] 1. PROPERTY_VALUE/1
I0406 10:48:27.868712   40107 rset.go:785] E1S1S1S1R1 [Rset.open] []ora.def{(*ora.defString)(0xc820071410)}
I0406 10:48:27.868838   40107 conn.go:157] E1S1S1C1 [Con.Prepare]
I0406 10:48:27.868856   40107 ses.go:639] E1S1S1 [Ses.Prep] select hash, content from "BLOBSTORE"."PJ_BLOB" where rownum < 10
I0406 10:48:27.868987   40107 drvStmt.go:108] E1S1S1S2 [DrvStmt.Query]
I0406 10:48:27.869000   40107 stmt.go:1154] E1S1S1S2 [Stmt.qry]
I0406 10:48:27.869009   40107 stmt.go:1167] E1S1S1S2 [Stmt.bind] Params 0
I0406 10:48:27.887039   40107 rset.go:772] E1S1S1S2R2 [Rset.open]
I0406 10:48:27.887122   40107 rset.go:785] E1S1S1S2R2 [Rset.open] 1. HASH/1
I0406 10:48:27.887154   40107 rset.go:785] E1S1S1S2R2 [Rset.open] 2. CONTENT/113
I0406 10:48:27.887201   40107 rset.go:785] E1S1S1S2R2 [Rset.open] []ora.def{(*ora.defString)(0xc820071590), (*ora.defLob)(0xc820071620)}
I0406 10:48:27.948790   40107 drvStmt.go:108] E1S1S1S2 [DrvStmt.Close]
I0406 10:48:27.948823   40107 stmt.go:1154] E1S1S1S2 [Stmt.close]
I0406 10:48:27.948838   40107 rset.go:772] E1S1S1S2R2 [Rset.close]
I0406 10:48:27.966613   40107 conn.go:157] E1S1S1C1 [Con.close]
I0406 10:48:27.966638   40107 ses.go:637] E1S1S1 [Ses.close]
I0406 10:48:27.966648   40107 stmt.go:1154] E1S1S1S1 [Stmt.close]
I0406 10:48:27.966656   40107 rset.go:772] E1S1S1S1R1 [Rset.close]
I0406 10:48:27.985902   40107 srv.go:322] E1S1 [Srv.close]
panic: sql: Scan error on column index 1: unsupported Scan, storing driver.Value type *ora.lobReader into type *[]uint8

goroutine 1 [running]:
panic(0x4228420, 0xc82006f580)
	/usr/local/Cellar/go/1.6/libexec/src/runtime/panic.go:464 +0x3e6
main.checkErr(0x8896000, 0xc82006f580)
	/Users/nabil/projects/gotothefs/blober.go:126 +0x4b
main.main()
	/Users/nabil/projects/gotothefs/blober.go:65 +0x52b
```

How I can get blob?

What do you want to do with the blob?
See ExampleWriteLOB in z_example_test.go - writes and reads black LOBs.

I want to write the blob to the file system.

Thanks with this example, I succeed to write.
Many thanks.


tests fail on windows

Describe the bug
test_cataloging.py fails on Win32 Python 3.7.3
To Reproduce
run pytest in root directory produces this:

Additional context
I noticed that Travis was only set up to run in a linux env, so feel free to close this issue if you aren't worried about testing on Windows, just wanted you all to be aware that I had tried to run them and what the result was.
Note: Submitting this issue as part of the JOSS review process (openjournals/joss-reviews#2075)

It looks like you're running it in a Conda environment which likely has an
older version of pandas. Have you tried it with pipenv, making sure to
include the project directory in your PYTHONPATH environment variable?
On Thu, Feb 20, 2020, 4:52 PM Katherine Ratliff<EMAIL_ADDRESS>wrote:

Describe the bug
test_cataloging.py fails on Win32 Python 3.7.3
To Reproduce
run pytest in root directory produces this:
[image: image]
https://user-images.githubusercontent.com/12630578/74981825-f7a8f600-5400-11ea-8b98-ff4a6b7dcd01.png
Additional context
I noticed that Travis was only set up to run in a linux env, so feel free
to close this issue if you aren't worried about testing on Windows, just
wanted you all to be aware that I had tried to run them and what the result
was.
Note: Submitting this issue as part of the JOSS review process (
openjournals/joss-reviews#2075
https://github.com/openjournals/joss-reviews/issues/2075)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/UNCG-DAISY/psi-collect/issues/102?email_source=notifications&email_token=AAYU3IKZTQH2Y5DA6GLE2NDRD33Q5A5CNFSM4KYXRKNKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IPEDPSQ,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAYU3IPFDSHD6I3M3Q4Q4ALRD33Q5ANCNFSM4KYXRKNA
.

@katmratliff, I was able to get pytest running in my Windows environment so I think you're experiencing some issue with your local environment? Perhaps check you're using the latest version of numpy (see this issue).

@Matmorcat yes I was testing in a conda env. If @chrisleaman had no issues running the tests on windows then I will close this - thanks!


Updating comfy dependencies causes double VRAM usage on Q8?

I had been using comfy quite happily with some pretty old packages, however today I updated the dependencies and the Q8 quant is suddenly taking the same space as fp16, and I cannot figure out why could that be, maybe the bf16 has something to do with it?:
Warning torch.load doesn't support weights_only on this pytorch version, loading unsafely.
C:\ComfyUI\custom_nodes\ComfyUI-GGUF\nodes.py:79: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:212.)
  torch_tensor = torch.from_numpy(tensor.data) # mmap

ggml_sd_loader:
 1                             476
 8                             304
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX

Okay so this seems to be caused by --highvram parameter, on normal vram the usage never surpasses 15GB, so it is probably something related to high vram loading

I haven't tested with highvram so that could definitely be it lol. I also saw people reporting issues with pytorch 2.4.x on regular comfy with flux, so that could be adding to it?


Request: If PowerApps/Power Automate Owner is not found copy file to SharePoint

Describe the bug
Hi @JeneferM-MSFT
recently I faced an issue where the power automates creator left the organization and his team was still using the power automate since the creator did not found it came to admin and the admin deleted the flow because the creator was not found. in the early CoE kit, there was a feature where if the owner is not found it and the admin approves it before it gets deleted it will store a copy of Power Automate/PowerApps to the SharePoint site so we can retrieve it back. Can the CoE team add this feature back?
Also, another question is - Are Power Automate flagged to be deleted when the flow creator is terminated, even if there are other owners of the Power Automate who are still active employees?
I had this discussion with @topness-msft.
Component (please tell us which flow or app you are experiencing issues with):

Solution [e.g. Core, Nurture]
Version [e.g. 1.29]
App or Flow [e.g. Sync Template (v2)]

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

Hello. I'm afraid we cannot add that feature back, we removed the support due to some product changes. We are no longer able, at this time, to reliably restore flows created by other people programmatically. This is because of the way that connections are now stored. We have given the product team feedback and if the capability returns we will add this back in.
The new archival flows instruct owners to back up their own flows before the system deletes them (if deletion approved).
People who have user permission should also be able to create a backup.
We do not automatically mark flows for deletion if they are orphaned.

Hello. I'm afraid we cannot add that feature back, we removed the support due to some product changes. We are no longer able, at this time, to reliably restore flows created by other people programmatically. This is because of the way that connections are now stored. We have given the product team feedback and if the capability returns we will add this back in.
The new archival flows instruct owners to back up their own flows before the system deletes them (if deletion approved).
People who have user permission should also be able to create a backup.
We do not automatically mark flows for deletion if they are orphaned.


DEVOPS-1827 pt2

Restoring step I'd accidentally deleted in merge conflict resolution.

This is fairly low priority, but if future proofs the script for platforms outside of ARM and Intel.


Python/NumPy support matrix

It would be good to support the same Python/NumPy as CIL
https://github.com/TomographicImaging/CIL/issues/1719
Python:

3.10
3.11
3.12

NumPy:

1.23
1.24
1.25
1.26

related: https://github.com/TomographicImaging/CIL/issues/1719


DOC: Use real civis_file_to_table example

We were using a csv_to_civis example in the civis_file_to_table definition Switching to a correct example.

@jkulzick - quick doc fix for your new civis_file_to_table - mind doing a quick review?

@seancquinn i'll add this change to [DE-108] Fix use of TemporaryDirectory in civis.io.file_to_civis. there is a CI error with the code i need to fix for the changes to go in


Token refreshed each time the JS UI thread is stucked for more than 1 sec

What is the problem
I expect the SDK to refresh the token before it will expire, but not so much, because very frequent token refresh can cause deconnexions.
But I see that if my main UI thread is stucked for more than 1 second (usecase : can happen with a 200+ elements list, with DOM recycling, even with a good CPU), then the SDK will refresh the token.
Maybe the issue is here ?

How to reproduce
Just run this code in your console with the network debugger opened, it will calculate PI decimals for 5 seconds. Then you can see that the token is refreshed.
function * generateDigitsOfPi() {
    let q = 1n;
    let r = 180n;
    let t = 60n;
    let i = 2n;
    while (true) {
        let digit = ((i * 27n - 12n) * q + r * 5n) / (t * 5n);
        yield Number(digit);
        let u = i * 3n;
        u = (u + 1n) * 3n * (u + 2n);
        r = u * 10n * (q * (i * 5n - 2n) + r - t * digit);
        q *= 10n * i * (i++ * 2n - 1n);
        t *= u;
    }
}

let iter = generateDigitsOfPi();
let digits = "";
let endAt = Date.now()+1000*5; //5 seconds

do{
    digits += iter.next().value
}while(Date.now() < endAt)

Hello,
Can't reproduce with the latest version of the SDK. Could you try and tell me if you're facing the same issue?
FYI the one-second timer now checks the interval between two timestamps: the current time and the JWT expiration time. We are not using relative values such as the previous attempt (e.g. increment one second per iteration)

No news, closing for now.
Don't hesitate to ping us with additional info and we'll reopen the issue.
Thanks!


User isInEditMode() in show/hideKeyboard()

This fixes
The following classes could not be instantiated: - com.lapism.searchview.SearchView
error in the Layout editor

Thanks. I will add it to next version.


Can't run dnx commands from a solution root

Every time I try to run a command from a project using dnx -p SomeProject some-command it complains about project.lock.json being missing even tho it's present. When I run the command by going to the directory of the project with dnx some-command it works.
Here's an example: https://gist.github.com/beraboris/6c1437de9d0ffbbc5355
In this case I have a global.json file pointing to the test/ directory. I also have a test project with xunit called SomeTest. This project depends on xunit and its dnx runner. You should notice that I don't have a test/SomeTest/project.lock.json file.
I try to run dnx -p SomeTest test it complains about test/SomeTest/project.lock.json being missing. This is perfectly normal since the file is not there. The error also tells me to run dnu restore.
Following the advice from the error, I run dnu restore which creates test/SomeTest/project.lock.json. This is illustrated in the following command.
I try to run dnx -p SomeTest test again and I get the same error. This time it's not expected behaviour because the required file does exist.
I then try to go to the test/SomeTest directory and run the test command directly (dnx test) and the command runs fine.
I should note that I'm running Arch Linux 64 bit with dnx v1.0.0-beta7, and mono v4.0.3.

@moozzyk might be able to help

Considering that dnx is dead now and that I don't need this anymore I'm just going to close this


Uploaded audio files that go through multimedia processing fail when reported mime-type does not actually match format

An audio file, such as an M4A file that reports its mime-type to be video/mp4, can be uploaded and go through multimedia processing for videos, which results in failure as it tries to get processed as video when it is in fact, audio.
Overall, a different processing path to detect the streams should be incorporated when media files are encountered.

https://github.com/GCRC/nunaliit/pull/1115


[feat:discord] Discord bridge plugin

Force pushed to refactor the commit names a bit.

Force pushed to refactor the commit names a bit.


Update sbt, sbt-dependency-tree to 1.6.0 in series/0.23

Updates

org.scala-sbt:sbt
org.scala-sbt:sbt-dependency-tree

from 1.5.8 to 1.6.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt" } ]

labels: library-update, early-semver-minor, semver-spec-minor, commit-count:1

Superseded by #5790

Superseded by #5797.


Metrics dashboard not working with kube-prometheus-stack

Describe the bug
Dashboard infinite load metrics but not showing

To Reproduce
Steps to reproduce the behavior:

Create EKS Cluster
Add repository https://github.com/prometheus-community/helm-charts
Add custons on values https://github.com/lensapp/lens/blob/master/troubleshooting/custom-prometheus.md
Install Helm Chart kube-prometheus-stack version 45.24.0

Expected behavior

Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):

Lens Version: 2023.4.141316-latest
OS: [e.g. OSX] Windows 10
Installation method (e.g. snap or AppImage in Linux):

Kubernetes version 1.24

same here

Server Version: v1.26.2-eks-a59e1f0

Guys, i solve my problem.
At first it was believed to be a problem in the helm, however, the real problem was in a network connectivity that prevented both lens and prometheus from accessing the metrics server.
In my case, using EKS, it was only necessary to create an egress rule for the cluster sg allowing access to port 443

@STV-MFJ If you have self-managed nodes or managed nodes, the node security group needs to allow TCP 9090 the cluster security group.
It's solved my issue.

@STV-MFJ EKS Managed node group require 3 sgs for node and 2 to cluster.
Cluster Required:
SG 1: To node 1025-65535
Protocol: TCP
Ports: 1025-65535
Type: egress
source: source_node_security_group
SG 2: Node all egress
Protocol: TCP
Ports: 443
Type: egress
source: source_node_security_group
Node SG Requierd:
SG 1: Node to node all ports/protocols
Ports:  ALL TRAFFIC
Type: ingress
source: self sg node groups
SG 2: Control Plane to Nodegroup all traffic
Ports:  ALL TRAFFIC
Type: ingress
source: CLUSTER  SG
SG 3: Node all egress
Protocol: TCP
Ports: ALL TRAFIC
Type: egress
cidr_blocks      = ["<IP_ADDRESS>/0"]
I use terraform do create my cluster and terraform module aws eks.
`
Extend cluster security group rules
cluster_security_group_additional_rules = {
egress_nodes_ephemeral_ports_tcp = {
description                = "To node 1025-65535"
protocol                   = "tcp"
from_port                  = 1025
to_port                    = 65535
type                       = "egress"
source_node_security_group = true
}
cluster_egress_https = {
description      = "Node all egress"
protocol         = "tcp"
from_port        = 443
to_port          = 443
type             = "egress"
source_node_security_group = true
}
}
Extend node-to-node security group rules
node_security_group_additional_rules = {
ingress_self_all = {
description = "Node to node all ports/protocols"
protocol    = "-1"
from_port   = 0
to_port     = 0
type        = "ingress"
self        = true
}
egress_all = {
description      = "Node all egress"
protocol         = "-1"
from_port        = 0
to_port          = 0
type             = "egress"
cidr_blocks      = ["<IP_ADDRESS>/0"]
ipv6_cidr_blocks = ["::/0"]
}
ingress_cluster_to_node_all_traffic = {
description                   = "Cluster API to Nodegroup all traffic"
protocol                      = "-1"
from_port                     = 0
to_port                       = 0
type                          = "ingress"
source_cluster_security_group = true
}
}
`
I hope it help you.

@STV-MFJ EKS Managed node group require 3 sgs for node and 2 to cluster.
Cluster Required:
SG 1: To node 1025-65535 Protocol: TCP Ports: 1025-65535 Type: egress source: source_node_security_group
SG 2: Node all egress Protocol: TCP Ports: 443 Type: egress source: source_node_security_group
Node SG Requierd:
SG 1: Node to node all ports/protocols Ports: ALL TRAFFIC Type: ingress source: self sg node groups
SG 2: Control Plane to Nodegroup all traffic Ports: ALL TRAFFIC Type: ingress source: CLUSTER SG
SG 3: Node all egress Protocol: TCP Ports: ALL TRAFIC Type: egress cidr_blocks = ["<IP_ADDRESS>/0"]
I use terraform do create my cluster and terraform module aws eks. `
Extend cluster security group rules
cluster_security_group_additional_rules = { egress_nodes_ephemeral_ports_tcp = { description = "To node 1025-65535" protocol = "tcp" from_port = 1025 to_port = 65535 type = "egress" source_node_security_group = true } cluster_egress_https = { description = "Node all egress" protocol = "tcp" from_port = 443 to_port = 443 type = "egress" source_node_security_group = true } }
Extend node-to-node security group rules
node_security_group_additional_rules = { ingress_self_all = { description = "Node to node all ports/protocols" protocol = "-1" from_port = 0 to_port = 0 type = "ingress" self = true } egress_all = { description = "Node all egress" protocol = "-1" from_port = 0 to_port = 0 type = "egress" cidr_blocks = ["<IP_ADDRESS>/0"] ipv6_cidr_blocks = ["::/0"] } ingress_cluster_to_node_all_traffic = { description = "Cluster API to Nodegroup all traffic" protocol = "-1" from_port = 0 to_port = 0 type = "ingress" source_cluster_security_group = true } } `
I hope it help you.

Big thanks! It really helped!

@STV-MFJ If you have self-managed nodes or managed nodes, the node security group needs to allow TCP 9090 from the cluster security group.
It's solved my issue.

That solved my issue as well, thank you very much!

@STV-MFJ EKS Managed node group require 3 sgs for node and 2 to cluster.
Cluster Required:
SG 1: To node 1025-65535 Protocol: TCP Ports: 1025-65535 Type: egress source: source_node_security_group
SG 2: Node all egress Protocol: TCP Ports: 443 Type: egress source: source_node_security_group
Node SG Requierd:
SG 1: Node to node all ports/protocols Ports: ALL TRAFFIC Type: ingress source: self sg node groups
SG 2: Control Plane to Nodegroup all traffic Ports: ALL TRAFFIC Type: ingress source: CLUSTER SG
SG 3: Node all egress Protocol: TCP Ports: ALL TRAFIC Type: egress cidr_blocks = ["<IP_ADDRESS>/0"]
I use terraform do create my cluster and terraform module aws eks. `
Extend cluster security group rules
cluster_security_group_additional_rules = { egress_nodes_ephemeral_ports_tcp = { description = "To node 1025-65535" protocol = "tcp" from_port = 1025 to_port = 65535 type = "egress" source_node_security_group = true } cluster_egress_https = { description = "Node all egress" protocol = "tcp" from_port = 443 to_port = 443 type = "egress" source_node_security_group = true } }
Extend node-to-node security group rules
node_security_group_additional_rules = { ingress_self_all = { description = "Node to node all ports/protocols" protocol = "-1" from_port = 0 to_port = 0 type = "ingress" self = true } egress_all = { description = "Node all egress" protocol = "-1" from_port = 0 to_port = 0 type = "egress" cidr_blocks = ["<IP_ADDRESS>/0"] ipv6_cidr_blocks = ["::/0"] } ingress_cluster_to_node_all_traffic = { description = "Cluster API to Nodegroup all traffic" protocol = "-1" from_port = 0 to_port = 0 type = "ingress" source_cluster_security_group = true } } `
I hope it help you.

Really Thank You! you saved me a lot of time. i added the code formatted for reference below.
 cluster_security_group_additional_rules = {
    egress_nodes_ephemeral_ports_tcp = {
      description                = "Nodes on ephemeral ports"
      protocol                   = "tcp"
      from_port                  = 1025
      to_port                    = 65535
      type                       = "egress"
      source_node_security_group = true
    }
    cluster_egress_https = {
      description                = "All Internal Nodes to All Internal Nodes on ephemeral ports"
      protocol                   = "TCP"
      from_port                  = 443
      to_port                    = 443
      type                       = "egress"
      source_node_security_group = true
    }
  }
  node_security_group_additional_rules = {
    ingress_self_all = {
      description = "Node to node all ports/protocols"
      protocol    = "-1"
      from_port   = 0
      to_port     = 0
      type        = "ingress"
      self        = true
    }
    egress_all = {
      description = "Node all egress"
      protocol    = "-1"
      from_port   = 0
      to_port     = 0
      type        = "egress"
      cidr_blocks = ["<IP_ADDRESS>/0"]
      ipv6_cidr_blocks = ["::/0"]
    }
    ingress_cluster_to_node_all_traffic = {
      description                   = "Cluster API to Nodegroup all traffic"
      protocol                      = "-1"
      from_port                     = 0
      to_port                       = 0
      type                          = "ingress"
      source_cluster_security_group = true
    }
  }


Test cli commands refactoring

Codecov Report

:exclamation: No coverage uploaded for pull request base (test-cli-commands@3f1ad5f). Click here to learn what that means.
The diff coverage is 80%.

@@                 Coverage Diff                  @@
##             test-cli-commands     #125   +/-   ##
====================================================
  Coverage                     ?   94.92%
====================================================
  Files                        ?       49
  Lines                        ?     1577
  Branches                     ?        0
====================================================
  Hits                         ?     1497
  Misses                       ?       80
  Partials                     ?        0

Impacted Files
Coverage Δ

cli-commands/etherlime-test/evm-commands.js
100% <ø> (ø)

cli-commands/etherlime-test/time-travel.js
100% <ø> (ø)

...nds/compiler/etherlime-compile/compilerSupplier.js
69.65% <80%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3f1ad5f...a31ff71. Read the comment docs.


Add 'warp' to the libraries page

I believe this closes #106.

This structure implies that http-client and wreq are alternatives to warp. What about putting warp in its own section on web services? I could imagine that section having servant and Yesod in there as well

Bump

Thanks, I'll be back to this!
On Tue, Dec 27, 2016, 2:07 AM Michael Snoyman<EMAIL_ADDRESS>wrote:

Bump
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/haskell-lang/haskell-lang/pull/114#issuecomment-269280851,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAYZZstUlUNkwhDfuGepxf0CcSEIqUlFks5rMLkugaJpZM4K-jIR
.

Wow, has it been nearly a year already? :)
Updated to add a web servers section which lists wai, warp, and servant. I haven't added Yesod (yet) because I'm not sure what to say about it - what differentiates it from scotty, spock, etc?

I'd differentiate Yesod by saying it's a full stack framework with a scaffolding and support for databases, auth, and forms out of the box. It's comparable to Rails in that sense, as opposed to a lighter weight system like Sinatra.


#183 - Add core logic for the DelayedStart option support

Fixes #183 . The only problem is that MMC and Service are too smart and cache the option internally (see this article), so the setting does not get displayed there until the system restart.

[x] - Internal installation logic
[x] - Documentation update
[x] - Samples update
[x] - Functional tests

CC @mtman

@reviewbybees JENKINS-29496

This pull request originates from a CloudBees employee. At CloudBees, we require that all pull requests be reviewed by other CloudBees employees before we seek to have the change accepted. If you want to learn more about our process please see this explanation.

CC @jtnord

@jtnord Agreed. First time I was unable to find a proper API call in the documentation, should be fixed now.
I have also created #208 for the Service description field

Am left wondering if delayed start is correct, as opposed to setting correct service dependencies, but should work all the same (just with more delay)

WinSW allows defining service dependencies. IMHO extra feature should not hurt here, especially since it's hard to define which services may be actually required by Jenkins jobs, for example.

@reviewbybees done


To change password, the current one != the login one

Trying to change my own password on the Asia IPT.
I logged in with passworda, and would like to change it to passwordb. In the 'Current Password' field, I suppose I should use passworda. But the IPT gives me 'The password provided doesn't match.'

The error message can be more explicit. Against what does it try to match?
If it indeed should be the login password, which I assume so, then something wrong happened.

Thank you @ckotwn I'll check it

Thank you for spotting this quickly @ckotwn !
This is fixed now, I re-deployed cloud IPTs.
Since field is called 'Current password', it of course compares with the current login password, but we can improve wording if you find this confusing.

@mike-podolskiy90 I think the wording does make sense - I was just making logical assumptions.
I've just successfully changed my password. Thank you so much!


[WIP] Introduce whyNot API

What is the context for this pull request?

Tracking Issue: n/a
Parent Issue:  #253
Dependencies: n/a

What changes were proposed in this pull request?

This PR introduces whyNot API which analyzes why the given index is not applied to the given query.
Scope - check logical relation only

Get latest stable index log entry of "indexName"

invalid index name
no available stable version

Collect all logical relations in the given query

no HadoopFsRelation

Schema check - exclude relations by comparing column names (relations & index included columns & indexed columns)

column schema mismatch

Signature comparison

if signature mismatch - print number of common files, appended files, deleted files

Index application check

if the index is applied, show query plan.
if not, show the list of the possible causes.

Example scenario (term or msg can be revised - result is from the draft change):
val hs = new Hyperspace(spark)val df2 = Seq((1, 2), (3, 4)).toDF("id", "id2")
val df2 = Seq((1, 2), (3, 4)).toDF("id", "id2")
hs.whyNot(df2, "testIndex2")

No applicable relation type in query. Hyperspace only supports HadoopFsRelation.

df2.write.parquet("testda")
val df2 = spark.read.parquet("testda")
hs.whyNot(df2, "testIndex2")

Not found relation including all indexed/included columns in the given index. Column Names: List(id, name)

val df = spark.read.parquet("table2")
val query = df.filter("id == 1")
hs.whyNot(query2, "testIndex2")

Inapplicable relations in the given query:
root
 |-- id: integer (nullable = true)
 |-- name: string (nullable = true)

  - No common source file.
    - Relation files sample: file:/C:/path/to/table2/part-00002-2a4a0234-dd64-4075-802c-72284aff5705-c000.snappy.parquet
    - Index files sample: file:/C:/path/to/table/part-00000-6646a693-a8fa-4d69-93df-8bf68c1a1d5a-c000.snappy.parquet

hs.createIndex(df, IndexConfig("testIndex2", Seq("id"), Seq("name")))
hs.whyNot(query, "testIndex")

Candidate logical relations after signature match:
StructType(StructField(id,IntegerType,true), StructField(name,StringType,true))

The index is applied to the given query. Plan:
*(1) Project [id#0, name#1]
+- *(1) Filter (isnotnull(id#0) && (id#0 = 1))
   +- *(1) FileScan parquet [id#0,name#1] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/C:/path/to/spark-warehouse/indexes/testIndex2/v..., PartitionFilters: [], PushedFilters: [IsNotNull(id), EqualTo(id,1)], ReadSchema: struct<id:int,name:string>

val query2 = df.filter("name == 'name3'") // indexed column mismatch
hs.whyNot(query2, "testIndex2")

Candidate logical relations after signature match:
StructType(StructField(id,IntegerType,true), StructField(name,StringType,true))

Given query plan using Hyperspace:
*(1) Project [id#0, name#1]
+- *(1) Filter (isnotnull(name#1) && (name#1 = name3))
   +- *(1) FileScan parquet [id#0,name#1] Batched: true, Format: Parquet, Location: InMemoryFileIndex[file:/C:/path/to/table2], PartitionFilters: [], PushedFilters: [IsNotNull(name), EqualTo(name,name3)], ReadSchema: struct<id:int,name:string>

There can be several possible causes:
  - Filter condition: filter column should be the first "indexed" column.
  - Join condition: only support Equi-join and ANDs.
  - Join condition: both left and right child should have a candidate index.
  - Rule order: Join Rule is prior to Filter Rule.
  - Index rank: another index is applied to the relation.

df.write.mode("append").parquet("table2")
val df = spark.read.parquet("table2")
val query = df.filter("id == 1")
hs.whyNot(query, "testIndex2") // source file change

Inapplicable relations in the given query:
root
 |-- id: integer (nullable = true)
 |-- name: string (nullable = true)

  - Detected 8 of common source file(s) between 8 of index files and 16 of relation files.
  - Detected 8 of appended file(s).
    - Appended files sample: file:/C:/path/to/table2/part-00004-5839d8a7-cc14-4cfd-8e50-b0478c5b1f23-c000.snappy.parquet

Does this PR introduce any user-facing change?

Yes, introduces a new API described above section.
How was this patch tested?

TODO - unit test

will be delivered via #449


Revamp Chatbox | ADD Appealing and Interactive user Features

fix: #229

Hey @Harshit7492 please review my PR, also can you please level up if you like the implementation design

Well done @shubhagarwal1. Can you give star to this repo?

Hey @Anuj3553 can you help me with more tasks i can revamp more pages and make them appealing,
it would be a great approtubity for me

Well done @shubhagarwal1. Can you give star to this repo?

Hey @Anuj3553 can you help me with more tasks i can revamp more pages and make them appealing, it would be a great approtubity for me

Yes you can proceed. You can raise your PR.

Well done @shubhagarwal1. Can you give star to this repo?

Hey @Anuj3553 can you help me with more tasks i can revamp more pages and make them appealing, it would be a great approtubity for me

Yes you can proceed. You can raise your PR.

Thankyou so much, can you list me some specific pages, i will start working onthem

Well done @shubhagarwal1. Can you give star to this repo?

Hey @Anuj3553 can you help me with more tasks i can revamp more pages and make them appealing, it would be a great approtubity for me

Yes you can proceed. You can raise your PR.

Thankyou so much, can you list me some specific pages, i will start working onthem

Okay, i will tell you soon. Stay tuned ✌


Remove "upload" and "register" commands.

Summary of changes
The upload and register commands were deprecated over a year ago, in July 2018 (PR #1410, discussed in issue #1381). It is time to actively remove them in favor of twine.
I think the most controversial part of this might be that I've added setuptools.errors. It wasn't clear what the best error class to use would be. distutils will use raise SystemExit for a missing command; that didn't seem quite like the right thing to do, but I'm willing to be convinced otherwise.
CC @pypa/setuptools-developers @di
Pull Request Checklist

[X] Changes have tests
[x] News fragment added in changelog.d. See documentation for details

Oops, turns out that if you name your classes UploadTest instead of TestUpload it doesn't get detected by pytest. I guess that's why they were originally named Test(Upload|Register)Test...
Yet another time when I am appreciative that we include the tests in the coverage, and that we enforce diff coverage...

OK, merging this, which means that next release should be a breaking one!


Import Account/Region error: CloudFormation stack fails on ConfigBucketPolicy - This policy contains invalid Json

Describe the bug
CloudFormation Stack for importing another account/region fails, and then fails to roll back
To Reproduce
Set up AWS Perspective.
Import another region from the same account, run the CloudFormation template
Expected behavior
CloudFormation stack to create successfully
Screenshots

Browser (please complete the following information):

Name [e.g. chrome, firefox]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

I think this is (part of) the CloudTrail record for the failed call, so the policy was blank?
"errorMessage": "This policy contains invalid Json",
"requestParameters": {
    "bucketPolicy": {},
    "bucketName": "aws-perspective-configbucket-qaq1ed68eo44",
    "Host": "aws-perspective-configbucket-qaq1ed68eo44.s3.eu-west-2.amazonaws.com",
    "policy": ""
},

Hi, if you go to the CloudFormation console and click on this stack. You should be able to see the template under the 'Template' tab. Is that blank?

After some investigation, we found that this bug occurs when importing an account that does not already have AWS Config set up (or when you select No for the AlreadyHaveConfigSetup CloudFormation parameter) It is caused by invalid JSON in the ConfigBucketPolicy. The section that is causing the problem is
"Sid": "HttpsOnly"
"Action": "*"
"Effect": "Deny",
"Resource": "arn:aws:s3:::${ConfigBucket}/*",
"Principal": '*'
"Condition": {
   "Bool": {
        "aws:SecureTransport": "false"
    }
}

As a workaround you can edit the CloudFormation template before deploying. It should look like this
  "Sid": "HttpsOnly",
  "Action": "*",
  "Effect": "Deny",
  "Resource": "arn:aws:s3:::${ConfigBucket}/*",
  "Principal": "*",
  "Condition": {
    "Bool": {
      "aws:SecureTransport": "false"
    }
  }

We are working on a fix for this and will update this ticket when it is available.

An update on this one...We have released v1.0.1 that fixes this issue. If you want you can tear down the v1.0.0 stack and replace with v1.0.1.
If that is going to be a pain (which we understand completely), then we are working on a set of instructions to upgrade. These aren't available yet as we are still working them out, but will update the README when they are complete.

#86

Fixed in v1.0.1


Great Project - License?

This project is fantastic and much needed, what license is it under?
I also noticed that there are some translation keys missing in english:
days: {
    ...
    days: "day(s)",

Hello,
Thank !
It will be an MIT license.
I will update it soon to add the license, update the README, and fix translations

Thanks, much appreciated!


[Bug]: Updating branding even as admin results in a 401 error

Preflight Checklist

[x] I could not find a solution in the documentation, the existing issues or discussions
[ ] I have joined the ZITADEL chat

Environment
Self-hosted
Version
v2.63.4
Database
PostgreSQL
Database Version
Latest (Docker container)
Describe the problem caused by this bug
When I spin up a fresh instance using docker compose and going through the initial steps, I try to go to settings and change the branding but results in a 401 Unauthorized every time. I've removed the container, started from scratch but the issue still persists. I have done nothing else apart from logging in and going straight to the branding section to update it.
To reproduce

Use docker-compose
docker up
Login
Go to settings
Change branding logo/icon
Get an error

Screenshots

Expected behavior
To be able to change the branding with a fresh new spin up of the application
Operating System
Device: PC
OS: ZorinOS 17
Browser: Zed
Relevant Configuration
No response
Additional Context
No response

I thought we had something similar reported in the past but I couldn't find an issue.
From the top of my head the problem was that the database for the assets was not created or added, or something like that.
@stebenz you might be able to tell it better

Able to reproduce this issue in the latest version(v2.64.0) as well.

can you have a look if the table assets exists in the system schema?

It's there

@hifabienne Just want to make sure, when setting up Zitadel with docker-compose, will it be fine if I change the db name from zitadel to something else? Or is preferred to have it as zitadel?

also facing this issue

also seeing this image ghcr.io/zitadel/zitadel:stable (which is v2.54.8)

Also facing same issue

This can be fixed by going to the Zitadel project and adding your Zitadel URL to the allowed URLs list.

Kindly share a screenshots of allowed urls section on a project. we can't seem to find

I'm on mobile, but find the Zitadel project and select the Console application; then update your origin with your Zitadel URL
This should really be done by default for us.

@rawkode I'll have a look and see if that works thanks.

what @rawkode suggested worked for me on my local instance, thanks!

Hi @rawkode @Divaaaaaan @startsUp
Quick question to this, how did you setup Zitadel in general specificly with the domains?
The allowed origins, to which you add here, are computed with the configured redirect URIs as a base and then all additional origins.
Which could indicate that there were changes to the application in general, or a change in the domain sphere which is not propagated correctly.

@rawkode Thanks for the help, however this only works within the scope of an organisation, not the default settings. Any clue as to why that could be?


bug: nested a tag

removed nested a tag

That has to be there because of next I think :/

You are right! I'm surprised I didn't test it or read the next docs... anyways here's the updated PR, just removed the nested link and updated the styles :)


Fix avatar

Closes #4

Your fork is very outdated and reverts many new changes haha

Your fork is very outdated and reverts many new changes haha

Whoops, those weren't working on my end, lemme go fix that lol

Your fork is very outdated and reverts many new changes haha

Alright, problems fixed.

By the way @notnotnotswipez, any idea why the avatars are stuck T-posing? I'm wondering if it relates to this change.

Im currently working on a large refactor of the mod, which will fix these issues. These changes havent been pushed to github yet however, so im not going to merge this. Apologies

Avatars would get stuck tposing if the bone indexes were never populated on either the client or the rep

Im currently working on a large refactor of the mod, which will fix these issues. These changes havent been pushed to github yet however, so im not going to merge this. Apologies

All good, thanks for taking a look!


reduce interpolated string nodes

a fix for #29

before this change,
this
$"my shell is ($shell)"

produces this tree
(nu_script [0, 0] - [1, 0]
  (pipeline [0, 0] - [1, 0]
    (pipe_element [0, 0] - [0, 23]
      (val_interpolated [0, 0] - [0, 23]
        (escaped_interpolated_content [0, 2] - [0, 3])
        (escaped_interpolated_content [0, 3] - [0, 4])
        (escaped_interpolated_content [0, 4] - [0, 5])
        (escaped_interpolated_content [0, 5] - [0, 6])
        (escaped_interpolated_content [0, 6] - [0, 7])
        (escaped_interpolated_content [0, 7] - [0, 8])
        (escaped_interpolated_content [0, 8] - [0, 9])
        (escaped_interpolated_content [0, 9] - [0, 10])
        (escaped_interpolated_content [0, 10] - [0, 11])
        (escaped_interpolated_content [0, 11] - [0, 12])
        (escaped_interpolated_content [0, 12] - [0, 13])
        (escaped_interpolated_content [0, 13] - [0, 14])
        expr: (expr_interpolated [0, 14] - [0, 22]
          (pipeline [0, 15] - [0, 21]
            (pipe_element [0, 15] - [0, 21]
              (val_variable [0, 15] - [0, 21]
                name: (identifier [0, 16] - [0, 21])))))))))

but after, this is the tree
(nu_script [0, 0] - [1, 0]
  (pipeline [0, 0] - [1, 0]
    (pipe_element [0, 0] - [0, 23]
      (val_interpolated [0, 0] - [0, 23]
        (escaped_interpolated_content [0, 2] - [0, 14])
        expr: (expr_interpolated [0, 14] - [0, 22]
          (pipeline [0, 15] - [0, 21]
            (pipe_element [0, 15] - [0, 21]
              (val_variable [0, 15] - [0, 21]
                name: (identifier [0, 16] - [0, 21])))))))))

i don't really have a way to test for this

ok, thanks for the before and after, that's always a good thing to put in a PR. let's move forward with this.

you should also probably close #29


Add cargo-make dependency and makefiles

This PR suggest to use cargo-make as our primary build tool. It has a slightly nicer interface than build.rs, supports auto-installing dependencies and is fully integrated with Github actions (not implemented in this PR).
I've added several tasks that can be used to validate code before submitting a PR - these tasks run the same set of checks as Github CI, so it could be convenient for contributors to make sure their code is correct before getting an official feedback from the CI tool.

Not sure about the grcov stuff as that seems like a bit of a white elephant (for example, we're already at 1.60 but it has a bunch of code for the 1.59->1.60 transition; also, it's a bit short on variable quoting). I guess we can just pull that in occasionally.

I agree, it does look scary. I looked into it a bit more and it seems there is a better way to produce coverage reports. Will be removing grcov shortly

it does not seem that we're in favor of pulling cargo make, I will probably just create a bash script that runs the same checks as GH workflow instead.


Can I submit tag helpers for "readonly" and "disabled" attributes?

Is it acceptable for me to put in a PR for a couple new tag helpers? I have written custom ones locally to handle readonly and disabled tags on elements. I think these would be widely used and helpful for many people if we had asp-readonly and asp-disabled helpers.
Currently, the following will NOT work as it outputs an empty disabled attribute (which is valid for disabling):
disabled="@(someBool ? "disabled" : "")"

However, this does work fine and does not output any disabled attribute:
disabled="@(someBool ? "disabled" : null)"

IMHO, it would be a better developer experience if we (a) did not have to use the longwinded ternary operator and (b) did not have to remember to use null instead of "". Instead, it would be great to have something like (that I have written locally)...
asp-disabled="someBool"

If this is good, let me know, and I'll put in a PR.

I think all you have to do for minimizable properties is set disabled="@isDisabed" and Razor does the right thing, no? I thought it's smart with bool values and if it's false it omits the attribute entirely, and if it's true is sets it to foo="foo"?

Wow, just tested, and you're right. TIL something new. Sweet, closing this (and deleting my tag helpers). Carry on...

Awesome! 😄

@Eilon On a similar note, what about a asp-roles-include and asp-roles-exclude? I wrote this up locally also, and it has served well. Of course, you might surprise me and say there's already a built-in way.

@robertmiles3 do they conditionally show/hide an HTML element based on the role(s) the user is in? I think it's been considered before and we didn't feel it was a high enough value, mostly because it's not too different from a regular @if(...) statement. We had a discussion on this a while back in this issue: https://github.com/aspnet/Mvc/issues/6750
But, feel free to log a new suggestion if you have any specific thoughts.

Yes, they operate like that. I actually took the existing environment tag helper and reworked it for roles. IMO, I would rather see something like this in certain scenarios rather than curly braces everywhere which also annoy my layout OCD, haha.
But, I understand. Just thought I'd throw it out there in case it was deemed useful.

Understood, thanks!


Android Keyboard doesn't appear .

The android keyboard doesn't appear on editing text in svg.

It has not been coded for mobiles, sorry. Editing text in SVG is pure javascript and doesn't use native html inputs.

Thank you so so much,
I appreciate your information.
Thank you again.
With my best regards.
On Tue, Dec 17, 2019 at 11:38 AM ybochatay<EMAIL_ADDRESS>wrote:

It has not been coded for mobiles, sorry. Editing text in SVG is pure
javascript and doesn't use native html inputs.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/YannickBochatay/JSYG.FullEditor/issues/2?email_source=notifications&email_token=AFKQLGWKZU236VOF7LCEDL3QZCFWRA5CNFSM4I24PW5KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHBTB3A#issuecomment-566440172,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFKQLGXWLXYB7VOH5DZNSG3QZCFWRANCNFSM4I24PW5A
.


add  rewayat.club

Source name
rewayat.club
Source link
https://rewayat.club/
Language
Arabic
Other details
i hope to add this source or at least tell me how to do it.thank
you for your time.
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open issue.
[X] I have written a title with source name.
[X] I have checked that the extension does not already exist by searching the GitHub repository and verified it does not appear in the code base.
[X] I will fill out all of the requested information in this form.

please solve this problem.


WIP: config parser

First version of the config parser.
See issue #1 for details.
This PR will be updated as features are added to the library.

Also put this file under the config submodule, not config_parser.


flag: increase coverage on TestSetOutput

I didn't realize I needed a gmail account to contribute

Closing older PRs that didn't sign the CLA. Feel free to reopen or re-send if you can sign it.

@mvdan any way you could raise the Gmail requirement issue?

I don't think you need a gmail account, you just need a Google account. I don't use gmail at all, for example.

@mvdan I think you should revisit the process in a browser you aren't logged into already. It 100% requires you have to login with a google account.

Yeah don't re-open it. I left it open in the hopes someone, like you 😉, would post the patch, but I guess I can submit it to the mailing list to find someone.


Rename helium to t4 and pyarrow to parquet

Rename the metadata header and xattr from "helium" to "t4"
Rename "PYARROW" to "PARQUET", since Parquet is the name of the format

NOTE: This breaks metadata on all existing objects out there! I can support the old metadata if needed.

@ResidentMario will this break Rob's upcoming notebook? hurdat?

Can you create an alias in the enum (i.e. that would continue to support PYARROW)?
I am fine with this change but want to note that T4 will probably be rolled into a bigger project so not sure how useful it is to rename; but if it makes the code more readable, do it.

@akarve It looks like accessing old T4 objects in the new API would break, which is a background change that doesn't affect anything because we always access newly created objects in all of our tutorial etc. materials.

Decided to wait on this.


Adding a dialog for selecting services by type (Kinetic)

This dialog is similar to the ones for listing topics or TF frames, but it is
a little different under the hood.  Notably:

It relies on the rosapi node in order to be able to search for services
Since searching is done via a service call, ROS communication is handled
on a separate thread that will not block the GUI
Unlike topics, only searching for a single service type is supported

Conflicts:
mapviz/package.xml

Apparently rosapi hasn't been released into kinetic yet.  Here's the appropriate ticket: https://github.com/RobotWebTools/rosbridge_suite/issues/214
I suppose this PR will have to wait until then.

rosbridge is up in shadow-fixed for kinetic.


Sometimes pressing a button in the shop does not purchase anything with no indication as to why

I think this may just be an issue with some hidden condition not being visually or audibly shown but it seems sometimes when buying nothing happens

Issue resolved this was actually just a visual Cue issue


Editing the pattern does not seem to update the pattern setting 0.14.5 in docker

Thank you for developing this plugin, seems like plugin for everyone to use.
I tried to change the pattern multiple times.
Then I noticed, that even though the newer value is visible in the settings, if I press the edit value button by the IPN pattern I still get the previous pattern appearing:

Expected:
I should see the editor pop up with the SAME value as is visible at the settings table below.
(Note: I was unable to get
DS-001-0001 ... DS-001-9999 kind of pattern with (DS-){3}(-){4} which should work in theory. Maybe the dash - sign is treated as a negative zero number?)

Thank you for the report.
From what I gather, you're saying the displayed pattern is different between the settings page, and the "edit plugin setting" modal?
The picture seems to show the same value boht places, not different ones?
I'll run some tests and see if I've missed something with dashes. In theory, (DS-001-){4} should've yielded the range DS-001-0001 ... DS-001-9999, so I'll look into that.

Wow, talk about demo effect.   I tried like 8 times to edit that value, then decided it's worth reporting, and just when I wasn't checking anymore (just snapped the pic) it started to behave.
But back to the issue: yet, the Edit Plugin Setting modal had a different (older) value in the inputbox than what was visible at the Settings table "IPN pattern" row.
I'm not perfectly sure that it was not a general UI issue (independent from any plugins), but I have reloaded the page, even SHIFT-reload (so any caches should be cleared).
I'll try it again next time and document it better, so I can demonstrate the behaviour better.

I'll deploy the fix to dashes and a couple of other special characters.
In general, InvenTree should handle all settings updates, so I don't think the display issue is directly related to the plugin. I've had a few weird things happen with the Classic UI settings myself, so it wouldn't surprise me if that's the core of the problem there.
I'll close this with the new release.
Feel free to drop a new comment here or raise a new issue if you find something of interest.


Open quickfix when there are new errors automatically

@w0rp I don't understand why you're saying that my comments are in-constructive. I thought I was very clear about what I'm suggesting.
What I'm trying to communicate is that if you would give ability to open quickfix automatically when errorlist have changed (because it means that either new errors have been added or old errors have been removed, then because this plugin already can close quickfix when there are no errors it could give instant feedback without vertical jumpiness of signs, with ability to ignore errors by closing quickfix (that signs and highlights prevent) and with ability to see error messages right away without moving the cursor to the line with the error or opening quickfix manually.
I hope now it's clear enough.

See :help g:ale_open_list. This is supported already. Please read the documentation.
If there's something more specific about the behaviour you would like to change, then open a pull request.

https://github.com/w0rp/ale#5ix-how-can-i-use-the-quickfix-list-instead-of-the-loclist This is mentioned in the FAQ.


Update Enterprise Metrics documentation, Agent-->Alloy

This issue tracks the work for updating Agent-->Alloy in the Enterprise Metrics documentation.
### Tasks
- [ ] Identify the topics that reference or document using Agent with Alloy
- [ ] Assess the contents for minor update, rewrite, or remove.

Enterprise Metrics topics that link to Agent Cloud:
WARN  /docs/enterprise-metrics/latest/operations/self-monitoring/: [render-link] page links to the Grafana Cloud Agent page /docs/grafana-cloud/agent/.
WARN  /docs/enterprise-metrics/v2.12.x/operations/self-monitoring/: [render-link] page links to the Grafana Cloud Agent page /docs/grafana-cloud/agent/.

Some more potential links hat need to be checked:
/docs/enterprise-metrics/latest/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v1.4.x/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v1.5.x/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v1.6.x/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v1.7.x/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.0.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.1.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.10.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.11.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.12.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.2.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.3.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.4.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.5.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.6.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.7.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.8.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.
/docs/enterprise-metrics/v2.9.x/operations/self-monitoring/ -> /docs/grafana-cloud/agent/.

If you have any PRs open for this Issue, make sure to mention this Issue in the PR description somewhere, or link to the PR here.... just so everything links up :-)


Mod eating up 8GB allocated RAM in seconds

We use many mods, most of them are functional mods and scripts yet we had no issues so far. I was working on a huge hall and I was so happy to find this mod, happy enough to throw out block carpentry (since placing and then breaking a block carpentry slab together with another slab made the game crash, which isn't the issue right now) but after building up one wall this mod became extremely RAM hungry.
I had 6GB RAM allocated to Minecraft and first, it took like 4 minutes to munch down all the RAM while working on my building before freezing for app. 30 seconds to clear occupied RAM. I kept going but the timeframe it took to consume all the RAM got shorter with every freeze. After just 10 minutes in-game I need to restart Minecraft as it becomes unplayable, freezing every 5 seconds while freezes take longer and longer to a point where I actually get thrown from the server due to a time out.
I try to keep an eye on the RAM usage while f3 is enabled and I can't get less than 70% RAM usage when near my building. I'm fine outside of the chunk or at my friend's location and he has a lot of stuff going on with the create mod, applied energetics and immersive engineering and we don't have a problem.
I think a mod adding the possibility to merge slabs together shouldn't consume more RAM than a mod that allows building heavy machinery with animated visuals, then again I don't really know what makes this mod eat so much RAM.
I have allocated 8GB RAM to Minecraft now with the newest version of this mod for 1.16.4 but nothing much changes. I might be able to play a little longer (like 5 minutes) before it gets unplayable but otherwise, It doesn't change much and I actually don't want to allocate more RAM to Minecraft. I even disabled my shader, resource pack and set all my graphic settings to potato but it's not helping much. I don't have anything else going on in my chunk, I have no types of machinery working, nothing that needs a lot of render or RAM usage. It's just a big ol' castle on a big ol' mountain that's not even furnished yet so idk what is consuming so much RAM :(
I hope this can be fixed somehow because without the freezes and the RAM munchies this is one of my fav mods and I wouldn't want to throw it out of the list at all.

Hi there, thanks for the report. I'm currently experimenting with ways to improve performance. I believe I understand where the performance impact is coming from and so I am trying to address this. Please try 3.5.0rc2 to see if this improves performance further. Please backup your world before upgrading as these are experimental versions. If you could potentially send your world file and mod list / mod pack link so I can test it directly on my machine, that would be amazing?

I'll try with the new version. If it keeps munching down on the RAM so badly I'll try to send you my world file and mod list :)

Okay brilliant! Thank you :)

Hi there! I've made some further improvements and so I'm going to close this issue. Thank you again for reporting this :) For now the update is only for 1.16 but I will be backporting to older versions as soon as I can :)


Remove mesh and particles RD dependencies from canvas rendering server

The rendering server should not depend on any RD specific classes.
Including the RD storage classes here meant that they were being wrongly included in the GLES3 rasterizer classes.

Thanks!


Is it possible to send a stream?

Hi, Thanks for the cool framework! Makes it so much easier to connect devices :) ..
I was wondering if it is possible to send a stream between the connected devices?
So like, one device is sending its audio (microphone) and the other devices are receiving and listening.
If its possible, then I think I need to use the sendFrameWithSource but i'm not quite sure how to use it.
Thanks 😉

@mikkeldamm Hello!
sendFrameWithSource is experimental API, but it does the same thing as sendFrame.
The latter already sends frames sequentially.
But only TCP-like sending is supported, so all frames are guaranteed to be received (except due to disconnect), but for real-time audio often UDP is needed.

@virl So what you are saying is that its not possible as of right now?

@mikkeldamm It depends on what you need in your audio transfer task.

I'm trying to stream live voice audio from one device to multiple other devices. E.g. one person talks via the microphone and that audio stream is sent to other phones that listens to that stream live.
So you can say its kinda like an voice group call to multiple phones.

@mikkeldamm Great idea!
Bluetooth on iOS have roughly 200 KBytes/sec throughput, but I don't know is it per device or shared among connected devices (probably the latest).
The problem with TCP streaming (as opposed to UDP) is that if there is some temporary connection hiccup (due to nearby working microwave, for example), the most recent audio will not be sent until earlier audio data is received.


/plugins-storage volume

I was hoping to try out the new plugin system, but looks traefik has ./plugins-storage hardcoded as the storage location, and I havn't figured out how to mount something there via the current helm chart.
│ traefik-b789945b8-d9zq4:time="2020-10-12T18:41:16Z" level=info msg="Configuration loaded from flags."                                                                │ │ traefik-b789945b8-d9zq4:2020/10/12 18:41:16 traefik.go:76: command traefik error: mkdir plugins-storage: read-only file system                                       │ │ traefik-b789945b8-d9zq4:stream closed                                                                                │

Hey @sretlawd,
currently, the location is not configurable. We did this within a scope of a first iteration for security reasons. Therefore, its obviously also not configurable within the helm chart.
I've raised #282 to address that issue. As this is just the question, I'll close it.

@SantoDE how can i use this plugin https://plugins.traefik.io/plugins/6335346ca4caa9ddeffda116/crowdsec-bouncer-traefik-plugin then?

I get the same error:
time="2023-01-25T23:05:04Z" level=error msg="plugins-storage/sources/gop-3757122034/src/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/bouncer.go:169:6: panic" plugin=plugin-crowdsec-bouncer-traefik-plugin module=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
time="2023-01-25T23:05:05Z" level=error msg="Plugins are disabled because an error has occurred." error="mkdir plugins-storage: read-only file system"
time="2023-01-25T23:05:09Z" level=error msg="Plugins are disabled because an error has occurred." error="mkdir plugins-storage: read-only file system"
time="2023-01-25T23:09:13Z" level=error msg="plugins-storage/sources/gop-2210337619/src/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/bouncer.go:169:6: panic" plugin=plugin-crowdsec-bouncer-traefik-plugin module=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
time="2023-01-25T23:09:13Z" level=error msg="plugins-storage/sources/gop-2210337619/src/github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin/bouncer.go:169:6: panic" plugin=plugin-crowdsec-bouncer-traefik-plugin module=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin

@Y0ngg4n I added an example on how to provide a directory for a plugin in #782 .
Does it work for you ?

@mloiseleur Nop it did not work for me. I had to set plugins.enabled to false because it defaults to emptydir. But i f i disable it i can not mount the volume because it always states that ""traefik" is invalid: spec.template.spec.containers[0].volumeMounts[2].name: Not found: "plugins".
I have added it like this:
    additionalVolumes:
      - name: plugins
        persistentVolumeClaim:
          claimName: traefik-plugins-pvc
    additionalVolumeMounts:
      - name: plugins
        mountPath: /plugins-storag

Yes, if you need it to be persistent, it's better to use a PVC than an emptyDir.

@mloiseleur yes but i cant use it with pvc because i get this error
``""traefik" is invalid: spec.template.spec.containers[0].volumeMounts[2].name: Not found: "plugins"

@Y0ngg4n Then I invite you to check your indentation. This mean that the volumeMount is declared with a plugin Volume, but this volume is not declared in the Deployment spec.
The example provided work out of the box on a kubernetes cluster with an emptyDir:
helm template traefik -f crowdsec-example.yaml  traefik/traefik

---
# Source: traefik/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: traefik
  namespace: default
  labels:
    app.kubernetes.io/name: traefik
    app.kubernetes.io/instance: traefik-default
    helm.sh/chart: traefik-20.8.0
    app.kubernetes.io/managed-by: Helm
  annotations:
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: traefik
      app.kubernetes.io/instance: traefik-default
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  minReadySeconds: 0
  template:
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/path: "/metrics"
        prometheus.io/port: "9100"
      labels:
        app.kubernetes.io/name: traefik
        app.kubernetes.io/instance: traefik-default
        helm.sh/chart: traefik-20.8.0
        app.kubernetes.io/managed-by: Helm
    spec:
      serviceAccountName: traefik
      terminationGracePeriodSeconds: 60
      hostNetwork: false
      containers:
      - image: traefik:v2.9.6
        imagePullPolicy: IfNotPresent
        name: traefik
        resources:
[...]
        volumeMounts:
          - name: data
            mountPath: /data
          - name: tmp
            mountPath: /tmp
          - mountPath: /plugins-storage
            name: plugins
        args:
[...]
          - "--experimental.plugins.bouncer.moduleName=github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin"
          - "--experimental.plugins.bouncer.version=v1.1.9"
      volumes:
        - name: data
          emptyDir: {}
        - name: tmp
          emptyDir: {}
        - name: plugins
      securityContext:
        fsGroup: 65532

You may check with helm template -f your-values.yaml xxx what yaml will be produced.

@mloiseleur thank you. it was the indentation.


[BUG] CAPR machine-provisioned etcd nodes are never drained before deletion

Rancher Server Setup

Rancher version: v2.7.5
Installation option (Docker install/Helm Chart): N/A

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): N/A

Proxy/Cert Details: N/A

Information about the Cluster

Kubernetes version: N/A
Cluster Type (Local/Downstream): Downstream CAPR
User Information
What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom)

If custom, define the set of permissions:

Describe the bug
CAPR machine provisioned etcd nodes are never drained on deletion. Apparently this was added as a "workaround" in the past but this was not the correct thing to do.
See: https://github.com/rancher/rancher/blob/8e8c62c86d53690a77260884e6fe1afcdafaafc4/pkg/controllers/provisioningv2/provisioningcluster/template.go#L356 and the corresponding PR here: https://github.com/rancher/rancher/pull/36411
To Reproduce
Create a CAPR-powered cluster with provisioned machines
Delete an etcd node
Result
Observe the etcd node is not actually drained even if you specify "drain on delete"
Expected Result
If you specify drain on delete, the node should be drained before deletion.
Screenshots

Additional context

This was merged in as part of PR: https://github.com/rancher/rancher/pull/42228/files#diff-2b509028a39574459f6e6e8dc738e9a1dc9be1c1d20deba00a7abd1f66c927adL356-L357 and as such, will be within the Q3 release.

QA TEST PLAN
Scenarios

Scenario
Test Case

1
Using QA's testing framework, provision downstream custom RKE1/RKE2/K3S clusters as a standard user (1 etcd, 1 cp, 1 worker)

2
Using QA's testing framework, provision downstream custom RKE1/RKE2/K3S clusters as a standard user (1 etcd+1 cp, 2 workers)

3
Using QA's testing framework, upgrade downstream node driver RKE1/RKE2/K3S clusters as a standard user (1 etcd, 1 cp, 1 worker)

4
Using QA's testing framework, upgrade downstream node driver RKE1/RKE2/K3S clusters as a standard user (1 etcd+1 cp, 2 workers)

5
Using QA's testing framework, run through etcd backup and restore test suite as a standard user

6
Setup Rancher v2.6.13, provision downstream node driver RKE1/RKE2/K3S clusters as a standard user, upgrade Rancher to v2.7-head

7
As a standard user, provision a downstream RKE2 node driver cluster cluster (2 etcd, 1 cp, 2 workers) w/drain on delete specified, delete an etcd node and validate it's drained before deleted

8
As a standard user, provision a downstream RKE2 node driver cluster cluster (2 etcd+ cp, 3 workers) w/drain on delete specified, delete an etcd node and validate it's drained before deleted

@markusewalker can you add some cases like drain etcd on delete -> wait for new etcd to come up -> drain again ... until all nodes have cycled 2x? and some other similar use cases given the latest etcd snapshot issues
also can you add some around testing etcd behavior after drain on delete since this probably hasn't been tested through rancher in the last 5 years, at least.

Validated this against v2.7-head https://github.com/rancher/rancher/commit/10d02ae6827fd549ae0f373be84a5e327999925b. See details below:
ENVIRONMENT DETAILS

Rancher install: Docker
Rancher version: v2.7-head https://github.com/rancher/rancher/commit/10d02ae6827fd549ae0f373be84a5e327999925b

TEST RESULT

Scenario
Test Case
Result

1
Using QA's testing framework, provision downstream custom RKE1/RKE2/K3S clusters as a standard user (1 etcd, 1 cp, 1 worker)
:white_check_mark:

2
Using QA's testing framework, provision downstream custom RKE1/RKE2/K3S clusters as a standard user (1 etcd+1 cp, 2 workers)
:white_check_mark:

3
Using QA's testing framework, upgrade downstream node driver RKE1/RKE2/K3S clusters as a standard user (1 etcd, 1 cp, 1 worker)
:white_check_mark:

4
Using QA's testing framework, upgrade downstream node driver RKE1/RKE2/K3S clusters as a standard user (1 etcd+1 cp, 2 workers)
:white_check_mark:

5
Using QA's testing framework, run through etcd backup and restore test suite as a standard user
:white_check_mark:

6
Setup Rancher v2.6.13, provision downstream node driver RKE1/RKE2/K3S clusters as a standard user, upgrade Rancher to v2.7-head
:white_check_mark:

7
As a standard user, provision a downstream RKE2 node driver cluster cluster (2 etcd, 1 cp, 2 workers) w/drain on delete specified, delete an etcd node and validate it's drained before deleted
:white_check_mark:

8
As a standard user, provision a downstream RKE2 node driver cluster cluster (2 etcd+ cp, 3 workers) w/drain on delete specified, delete an etcd node and validate it's drained before deleted
:white_check_mark:

VALIDATION STEPS
Scenario 1

Successfully ran the following test cases using QA's automated test framework:

Provision RKE1 v1.26.7-rancher1-1 custom cluster (1 etcd, 1 cp, 1 worker) as a standard user
Provision RKE2 v1.26.7+rke2r1 custom cluster (1 etcd, 1 cp, 1 worker) as a standard user
Provision K3S v1.26.7+k3s1 custom cluster (1 etcd, 1 cp, 1 worker) as a standard user

Scenario 2

Successfully ran the following test cases using QA's automated test framework:

Provision RKE1 v1.26.7-rancher1-1 custom cluster (1 etcd+1 cp, 2 workers) as a standard user
Provision RKE2 v1.26.7+rke2r1 custom cluster (1 etcd+1 cp, 2 workers) as a standard user
Provision K3S v1.26.7+k3s1 custom cluster (1 etcd+1 cp, 2 workers) as a standard user

Scenario 3

Successfully ran the following test cases using QA's automated test framework:

Upgrade RKE1 v1.25.12-rancher1-1 -> v1.26.7-rancher1-1 downstream node driver cluster (1 etcd, 1 cp, 1 worker) as a standard user
Upgrade RKE2 v1.25.12+rke2r1 -> v1.26.7+rke2r1 downstream node driver cluster (1 etcd, 1 cp, 1 worker) as a standard user
Upgrade K3S v1.25.12+k3s1 -> v1.26.7+k3s1 downstream node driver cluster (1 etcd, 1 cp, 1 worker) as a standard user

Scenario 4

Successfully ran the following test cases using QA's automated test framework:

Upgrade RKE1 v1.25.12-rancher1-1 -> v1.26.7-rancher1-1 downstream node driver cluster (1 etcd+1 cp, 2 workers) as a standard user
Upgrade RKE2 v1.25.12+rke2r1 -> v1.26.7+rke2r1 downstream node driver cluster (1 etcd+1 cp, 2 workers) as a standard user
Upgrade K3S v1.25.12+k3s1 -> v1.26.7+k3s1 downstream node driver cluster (1 etcd+1 cp, 2 workers) as a standard user

Scenario 5

Successfully ran the following test cases using QA's automated test framework:

Provision RKE2 v1.25.12+rke2r1, took a local etcd snapshot, upgrade to v1.26.7+rke2r1, restore to v1.25.12+rke2r1 using all restore options
Provision RKE2 v1.25.12+rke2r1, took an S3 etcd snapshot, upgrade to v1.26.7+rke2r1, restore to v1.25.12+rke2r1 using all restore options

Scenario 6

Setup Rancher v2.6.13 and logged in as a standard user.
Provisioned the following downstream node driver clusters:

v1.24.16-rancher1-1
v1.24.16+rke2r1
v1.24.16+k3s1

Upgraded Rancher v2.6.13 -> v2.7-head; validated that no cluster auto-upgraded their K8s version.
Upgraded each cluster to the highest K8s versions successfully; validated that each cluster and nodes were in a state of Active.

Scenario 7

As a standard user, provisioned a downstream RKE2 v1.26.7+rke2r1 node driver cluster (2 etcd, 1 cp, 2 workers).

Ensured that drain on delete is specified.

Deleted an etcd node; ensured that it is drained before it was deleted and a replacement came up in the cluster Active.
Repeated step 2 two more times, each with the same results.

Scenario 8

As a standard user, provisioned a downstream RKE2 v1.26.7+rke2r1 node driver cluster (2 etcd+ cp, 3 workers).

Ensured that drain on delete is specified.

Deleted an etcd node; ensured that it is drained before it was deleted and a replacement came up in the cluster Active.
Repeated step 2 two more times, each with the same results.