max_blocks constraint not enforced (HTEX+SlurmProvider)

Describe the bug
Hi there, and thanks for your support. While using HighThroughputExecutor+SlurmProvider+SrunLauncher to distribute tasks on a SLURM cluster, parsl opens more blocks than I expect, given my configuration. I set init_blocks=min_blocks=max_blocks=1 in SlurmProvider, but I obtain 2 blocks. When inspecting the block-0 and block-1 folders, I see that all workers in block-0 received (and completed) a task, while workers in block-1 were only started but they did nothing.
Perhaps I simply misunderstood what parsl is doing, but it is unclear to me why the second block should exist at all.

Script to reproduce issue (click to expand)
import platform

import parsl
from parsl.app.app import python_app
from parsl.config import Config
from parsl.channels import LocalChannel
from parsl.providers import SlurmProvider
from parsl.executors import HighThroughputExecutor
from parsl.launchers import SrunLauncher
from parsl.addresses import address_by_hostname

slurm = SlurmProvider(
  nodes_per_block=1,
  init_blocks=1,
  min_blocks=1,
  max_blocks=1,
  cores_per_node=16,
  parallelism=0,
  partition='main',
  mem_per_node=1,
  cmd_timeout=60,
  channel=LocalChannel(),
  walltime='01:00:00',
  launcher=SrunLauncher(),
  )
htex = HighThroughputExecutor(
  max_workers=16,
  provider=slurm,
  label='dummy',
  address=address_by_hostname(),
  )
config = Config(executors=[htex])
parsl.load(config)

@python_app
def just_sleep():
  import time
  time.sleep(5)
  return 0

tasks = []
for i in range(16):
  tasks.append(just_sleep())
[i.result() for i in tasks]

print(sys.version)
print(platform.uname())
print(parsl.__version__)

Part of runinfo/000/parsl.log (click to expand)
2022-05-06 09:31:02.520 parsl.providers.slurm.slurm:216 [DEBUG]  Writing submit script
2022-05-06 09:31:02.521 parsl.providers.slurm.slurm:220 [DEBUG]  moving files
2022-05-06 09:31:02.535 parsl.executors.status_handling:178 [DEBUG]  Launched block 0->9081009
2022-05-06 09:31:02.536 parsl.dataflow.task_status_poller:114 [DEBUG]  Adding executor dummy
2022-05-06 09:31:02.540 parsl.dataflow.dflow:821 [DEBUG]  Task 0 will be sent to executor dummy
2022-05-06 09:31:02.540 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.540 parsl.dataflow.dflow:901 [INFO]  Task 0 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.540 parsl.dataflow.dflow:909 [DEBUG]  Task 0 set to pending state with AppFuture: <AppFuture at 0x7f2780563ee0 state=pending>
2022-05-06 09:31:02.540 parsl.dataflow.memoization:229 [DEBUG]  Task 0 will not be memoized
2022-05-06 09:31:02.540 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.541 parsl.dataflow.dflow:610 [INFO]  Task 0 launched on executor dummy
2022-05-06 09:31:02.541 parsl.dataflow.dflow:821 [DEBUG]  Task 1 will be sent to executor dummy
2022-05-06 09:31:02.541 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.541 parsl.dataflow.dflow:901 [INFO]  Task 1 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.541 parsl.dataflow.dflow:909 [DEBUG]  Task 1 set to pending state with AppFuture: <AppFuture at 0x7f2780578a90 state=pending>
2022-05-06 09:31:02.541 parsl.dataflow.memoization:229 [DEBUG]  Task 1 will not be memoized
2022-05-06 09:31:02.541 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.541 parsl.dataflow.dflow:610 [INFO]  Task 1 launched on executor dummy
2022-05-06 09:31:02.541 parsl.dataflow.dflow:821 [DEBUG]  Task 2 will be sent to executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.542 parsl.dataflow.dflow:901 [INFO]  Task 2 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.542 parsl.dataflow.dflow:909 [DEBUG]  Task 2 set to pending state with AppFuture: <AppFuture at 0x7f2780578c70 state=pending>
2022-05-06 09:31:02.542 parsl.dataflow.memoization:229 [DEBUG]  Task 2 will not be memoized
2022-05-06 09:31:02.542 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.542 parsl.dataflow.dflow:610 [INFO]  Task 2 launched on executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:821 [DEBUG]  Task 3 will be sent to executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.542 parsl.dataflow.dflow:901 [INFO]  Task 3 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.542 parsl.dataflow.dflow:909 [DEBUG]  Task 3 set to pending state with AppFuture: <AppFuture at 0x7f2780578fa0 state=pending>
2022-05-06 09:31:02.542 parsl.dataflow.memoization:229 [DEBUG]  Task 3 will not be memoized
2022-05-06 09:31:02.542 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.542 parsl.dataflow.dflow:610 [INFO]  Task 3 launched on executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:821 [DEBUG]  Task 4 will be sent to executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.542 parsl.dataflow.dflow:901 [INFO]  Task 4 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.542 parsl.dataflow.dflow:909 [DEBUG]  Task 4 set to pending state with AppFuture: <AppFuture at 0x7f2780588310 state=pending>
2022-05-06 09:31:02.542 parsl.dataflow.memoization:229 [DEBUG]  Task 4 will not be memoized
2022-05-06 09:31:02.542 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.542 parsl.dataflow.dflow:610 [INFO]  Task 4 launched on executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:821 [DEBUG]  Task 5 will be sent to executor dummy
2022-05-06 09:31:02.542 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.542 parsl.dataflow.dflow:901 [INFO]  Task 5 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.542 parsl.dataflow.dflow:909 [DEBUG]  Task 5 set to pending state with AppFuture: <AppFuture at 0x7f2780588640 state=pending>
2022-05-06 09:31:02.542 parsl.dataflow.memoization:229 [DEBUG]  Task 5 will not be memoized
2022-05-06 09:31:02.542 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.543 parsl.dataflow.dflow:610 [INFO]  Task 5 launched on executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:821 [DEBUG]  Task 6 will be sent to executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.543 parsl.dataflow.dflow:901 [INFO]  Task 6 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.543 parsl.dataflow.dflow:909 [DEBUG]  Task 6 set to pending state with AppFuture: <AppFuture at 0x7f2780588970 state=pending>
2022-05-06 09:31:02.543 parsl.dataflow.memoization:229 [DEBUG]  Task 6 will not be memoized
2022-05-06 09:31:02.543 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.543 parsl.dataflow.dflow:610 [INFO]  Task 6 launched on executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:821 [DEBUG]  Task 7 will be sent to executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.543 parsl.dataflow.dflow:901 [INFO]  Task 7 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.543 parsl.dataflow.dflow:909 [DEBUG]  Task 7 set to pending state with AppFuture: <AppFuture at 0x7f2780588ca0 state=pending>
2022-05-06 09:31:02.543 parsl.dataflow.memoization:229 [DEBUG]  Task 7 will not be memoized
2022-05-06 09:31:02.543 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.543 parsl.dataflow.dflow:610 [INFO]  Task 7 launched on executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:821 [DEBUG]  Task 8 will be sent to executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.543 parsl.dataflow.dflow:901 [INFO]  Task 8 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.543 parsl.dataflow.dflow:909 [DEBUG]  Task 8 set to pending state with AppFuture: <AppFuture at 0x7f2780588fd0 state=pending>
2022-05-06 09:31:02.543 parsl.dataflow.memoization:229 [DEBUG]  Task 8 will not be memoized
2022-05-06 09:31:02.543 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.543 parsl.dataflow.dflow:610 [INFO]  Task 8 launched on executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:821 [DEBUG]  Task 9 will be sent to executor dummy
2022-05-06 09:31:02.543 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.543 parsl.dataflow.dflow:901 [INFO]  Task 9 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.543 parsl.dataflow.dflow:909 [DEBUG]  Task 9 set to pending state with AppFuture: <AppFuture at 0x7f278058e340 state=pending>
2022-05-06 09:31:02.544 parsl.dataflow.memoization:229 [DEBUG]  Task 9 will not be memoized
2022-05-06 09:31:02.544 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.544 parsl.dataflow.dflow:610 [INFO]  Task 9 launched on executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:821 [DEBUG]  Task 10 will be sent to executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.544 parsl.dataflow.dflow:901 [INFO]  Task 10 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.544 parsl.dataflow.dflow:909 [DEBUG]  Task 10 set to pending state with AppFuture: <AppFuture at 0x7f278058e670 state=pending>
2022-05-06 09:31:02.544 parsl.dataflow.memoization:229 [DEBUG]  Task 10 will not be memoized
2022-05-06 09:31:02.544 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.544 parsl.dataflow.dflow:610 [INFO]  Task 10 launched on executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:821 [DEBUG]  Task 11 will be sent to executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.544 parsl.dataflow.dflow:901 [INFO]  Task 11 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.544 parsl.dataflow.dflow:909 [DEBUG]  Task 11 set to pending state with AppFuture: <AppFuture at 0x7f278058e9a0 state=pending>
2022-05-06 09:31:02.544 parsl.dataflow.memoization:229 [DEBUG]  Task 11 will not be memoized
2022-05-06 09:31:02.544 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.544 parsl.dataflow.dflow:610 [INFO]  Task 11 launched on executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:821 [DEBUG]  Task 12 will be sent to executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.544 parsl.dataflow.dflow:901 [INFO]  Task 12 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.544 parsl.dataflow.dflow:909 [DEBUG]  Task 12 set to pending state with AppFuture: <AppFuture at 0x7f278058ecd0 state=pending>
2022-05-06 09:31:02.544 parsl.dataflow.memoization:229 [DEBUG]  Task 12 will not be memoized
2022-05-06 09:31:02.544 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.544 parsl.dataflow.dflow:610 [INFO]  Task 12 launched on executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:821 [DEBUG]  Task 13 will be sent to executor dummy
2022-05-06 09:31:02.544 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.545 parsl.dataflow.dflow:901 [INFO]  Task 13 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.545 parsl.dataflow.dflow:909 [DEBUG]  Task 13 set to pending state with AppFuture: <AppFuture at 0x7f2780594040 state=pending>
2022-05-06 09:31:02.545 parsl.dataflow.memoization:229 [DEBUG]  Task 13 will not be memoized
2022-05-06 09:31:02.545 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.545 parsl.dataflow.dflow:610 [INFO]  Task 13 launched on executor dummy
2022-05-06 09:31:02.545 parsl.dataflow.dflow:821 [DEBUG]  Task 14 will be sent to executor dummy
2022-05-06 09:31:02.545 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.545 parsl.dataflow.dflow:901 [INFO]  Task 14 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.545 parsl.dataflow.dflow:909 [DEBUG]  Task 14 set to pending state with AppFuture: <AppFuture at 0x7f2780594370 state=pending>
2022-05-06 09:31:02.545 parsl.dataflow.memoization:229 [DEBUG]  Task 14 will not be memoized
2022-05-06 09:31:02.545 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.546 parsl.dataflow.dflow:610 [INFO]  Task 14 launched on executor dummy
2022-05-06 09:31:02.546 parsl.dataflow.dflow:821 [DEBUG]  Task 15 will be sent to executor dummy
2022-05-06 09:31:02.546 parsl.dataflow.dflow:647 [DEBUG]  Adding output dependencies
2022-05-06 09:31:02.546 parsl.dataflow.dflow:901 [INFO]  Task 15 submitted for App just_sleep, not waiting on any dependency
2022-05-06 09:31:02.546 parsl.dataflow.dflow:909 [DEBUG]  Task 15 set to pending state with AppFuture: <AppFuture at 0x7f278058eee0 state=pending>
2022-05-06 09:31:02.546 parsl.dataflow.memoization:229 [DEBUG]  Task 15 will not be memoized
2022-05-06 09:31:02.546 parsl.executors.high_throughput.executor:557 [DEBUG]  Pushing function <function just_sleep at 0x7f2780579700> to queue with args ()
2022-05-06 09:31:02.547 parsl.dataflow.dflow:610 [INFO]  Task 15 launched on executor dummy
2022-05-06 09:31:03.507 parsl.providers.slurm.slurm:143 [DEBUG]  Executing sqeueue
2022-05-06 09:31:03.522 parsl.providers.slurm.slurm:145 [DEBUG]  sqeueue returned
2022-05-06 09:31:03.522 parsl.providers.slurm.slurm:158 [DEBUG]  Updating job 9081009 with slurm status RUNNING to parsl status 0
2022-05-06 09:31:03.522 parsl.dataflow.strategy:188 [DEBUG]  Executor dummy has 16 active tasks, 0/0 running/pending blocks, and 0 connected workers
2022-05-06 09:31:03.523 parsl.dataflow.strategy:253 [DEBUG]  Requesting single slot
2022-05-06 09:31:03.523 parsl.providers.slurm.slurm:200 [DEBUG]  Requesting one block with 1 nodes
2022-05-06 09:31:03.523 parsl.providers.slurm.slurm:216 [DEBUG]  Writing submit script
2022-05-06 09:31:03.524 parsl.providers.slurm.slurm:220 [DEBUG]  moving files
2022-05-06 09:31:03.537 parsl.executors.status_handling:178 [DEBUG]  Launched block 1->9081010
2022-05-06 09:31:08.507 parsl.dataflow.strategy:188 [DEBUG]  Executor dummy has 16 active tasks, 0/1 running/pending blocks, and 32 connected workers
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 0 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 2 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 1 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 10 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 3 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 4 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 5 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 7 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 8 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 11 completed (launched -> exec_done)
2022-05-06 09:31:08.807 parsl.dataflow.dflow:461 [INFO]  Task 6 completed (launched -> exec_done)
2022-05-06 09:31:08.808 parsl.dataflow.dflow:461 [INFO]  Task 9 completed (launched -> exec_done)
2022-05-06 09:31:08.808 parsl.dataflow.dflow:461 [INFO]  Task 12 completed (launched -> exec_done)
2022-05-06 09:31:08.817 parsl.dataflow.dflow:461 [INFO]  Task 13 completed (launched -> exec_done)
2022-05-06 09:31:08.817 parsl.dataflow.dflow:461 [INFO]  Task 14 completed (launched -> exec_done)
2022-05-06 09:31:08.817 parsl.dataflow.dflow:461 [INFO]  Task 15 completed (launched -> exec_done)
2022-05-06 09:31:08.818 parsl.dataflow.dflow:1057 [INFO]  DFK cleanup initiated
2022-05-06 09:31:08.818 parsl.dataflow.dflow:947 [INFO]  Summary of tasks in DFK:
2022-05-06 09:31:08.818 parsl.dataflow.dflow:961 [INFO]  Tasks in state States.exec_done: 16
2022-05-06 09:31:08.818 parsl.dataflow.dflow:967 [INFO]  End of summary
2022-05-06 09:31:08.818 parsl.dataflow.dflow:1081 [INFO]  Terminating flow_control and strategy threads
2022-05-06 09:31:08.818 parsl.executors.high_throughput.executor:663 [DEBUG]  Current blocks : {'0': '9081009', '1': '9081010'}
2022-05-06 09:31:08.818 parsl.executors.high_throughput.executor:521 [DEBUG]  [HOLD_BLOCK]: Sending hold to manager: bc4956735467
2022-05-06 09:31:08.819 parsl.executors.high_throughput.executor:490 [DEBUG]  Sent hold request to worker: bc4956735467
2022-05-06 09:31:08.819 parsl.executors.high_throughput.executor:521 [DEBUG]  [HOLD_BLOCK]: Sending hold to manager: 2df21f357f4b
2022-05-06 09:31:08.819 parsl.executors.high_throughput.executor:490 [DEBUG]  Sent hold request to worker: 2df21f357f4b
2022-05-06 09:31:08.832 parsl.executors.high_throughput.executor:698 [INFO]  Attempting HighThroughputExecutor shutdown
2022-05-06 09:31:08.832 parsl.executors.high_throughput.executor:700 [INFO]  Finished HighThroughputExecutor shutdown attempt
2022-05-06 09:31:08.832 parsl.executors.threads:101 [DEBUG]  Done with executor shutdown
2022-05-06 09:31:08.832 parsl.dataflow.dflow:1111 [INFO]  DFK cleanup complete

Output (click to expand)
[...]
3.8.12 (default, Oct 12 2021, 13:49:34)
[GCC 7.5.0]
uname_result(system='Linux', node='XXXXX-worker-001', release='4.4.0-166-generic', version='#195-Ubuntu SMP Tue Oct 1 09:35:25 UTC 2019', machine='x86_64', processor='x86_64')
1.2.0

To Reproduce

Setup Parsl 1.2.0 with Python 3.8.12
Run the test script on the login node of a SLURM cluster (you will need to replace partition='main' with the correct name for your system).
Monitor the queue with squeue, and notice that there are two SLURM jobs.
After execution (~8 seconds, in my case), in runinfo/000/parsl.log has a line like 2022-05-06 09:23:51.533 parsl.dataflow.strategy:253 [DEBUG]  Requesting single slot.
After execution, there are two folders block-0 and block-1 in runinfo/000/dummy. Files like runinfo/000/dummy/block-0/bc4956735467/worker_0.log report that worker 0 received and completed a task, while the same kind of files in the block-1 folder only report that a worker has started (without any task received or completed).

Expected behavior
By looking at https://github.com/Parsl/parsl/blob/master/parsl/dataflow/strategy.py#L251-L255, I expect the test in line 253 to be False, since one block is already active and max_blocks=1. But I should me missing something, because the following lines in parsl.log suggest that one more block has been asked for:
2022-05-06 09:31:03.523 parsl.dataflow.strategy:253 [DEBUG]  Requesting single slot
2022-05-06 09:31:03.523 parsl.providers.slurm.slurm:200 [DEBUG]  Requesting one block with 1 nodes
2022-05-06 09:31:03.523 parsl.providers.slurm.slurm:216 [DEBUG]  Writing submit script
2022-05-06 09:31:03.524 parsl.providers.slurm.slurm:220 [DEBUG]  moving files
2022-05-06 09:31:03.537 parsl.executors.status_handling:178 [DEBUG]  Launched block 1->9081010

Environment

OS: Ubuntu 16
Python version: 3.8.12
Parsl version: 1.2.0

Distributed Environment

Where are you running the Parsl script from ? Login node
Where do you need the workers to run ? Compute nodes

This looks suspicious in your log file:
2022-05-06 09:31:03.507 parsl.providers.slurm.slurm:143 [DEBUG]  Executing sqeueue
2022-05-06 09:31:03.522 parsl.providers.slurm.slurm:145 [DEBUG]  sqeueue returned
2022-05-06 09:31:03.522 parsl.providers.slurm.slurm:158 [DEBUG]  Updating job 9081009 with slurm status RUNNING to parsl status 0
2022-05-06 09:31:03.522 parsl.dataflow.strategy:188 [DEBUG]  Executor dummy has 16 active tasks, 0/0 running/pending blocks, and 0 connected workers
2022-05-06 09:31:03.523 parsl.dataflow.strategy:253 [DEBUG]  Requesting single slot
2022-05-06 09:31:03.523 parsl.providers.slurm.slurm:200 [DEBUG]  Requesting one block with 1 nodes

It's requesting that second block because it thinks there are 0 blocks available (rather than seeing the 1 block it submitted earlier)
In this line, it has asked slurm to give status for all the blocks it knows about using squeue, and is given the status slurm RUNNING which is then translated to parsl status 0 (UNKNOWN).
2022-05-06 09:31:03.522 parsl.providers.slurm.slurm:158 [DEBUG]  Updating job 9081009 with slurm status RUNNING to parsl status 0

That is surprising.
It looks like the parsl slurm provider is expecting to see short status codes. Here is the table that translates slurm status codes to parsl codes, at the top of slurm.py https://github.com/Parsl/parsl/blob/ccd787e285123aba61b279aeedf77f154e35e856/parsl/providers/slurm/slurm.py#L20
translate_table = {
    'PD': JobState.PENDING,
    'R': JobState.RUNNING,
    'CA': JobState.CANCELLED,
    'CF': JobState.PENDING,  # (configuring),
    'CG': JobState.RUNNING,  # (completing),
    'CD': JobState.COMPLETED,
    'F': JobState.FAILED,  # (failed),
    'TO': JobState.TIMEOUT,  # (timeout),
    'NF': JobState.FAILED,  # (node failure),
    'RV': JobState.FAILED,  # (revoked) and
    'SE': JobState.FAILED   # (special exit state)
}

The mapping R to JobState.RUNNING is what is going wrong here.
So maybe/probably your squeue command is returning status in an unexpected format.
The SlurmProvider runs squeue like this: https://github.com/Parsl/parsl/blob/ccd787e285123aba61b279aeedf77f154e35e856/parsl/providers/slurm/slurm.py#L141
        job_id_list = ','.join(self.resources.keys())
        cmd = "squeue --job {0}".format(job_id_list)

Can you paste what that command output looks like on your system?  (eg   squeue --job 123 for random job you have launched, id 123). You don't need to run it inside parsl.
Maybe also interesting to paste slurm version info.

I made PR #2257 to fix that misspelling of squeue

for comparison, running on cori.nersc.gov, I see this:
 Updating job 58928398 with slurm status R to parsl status 2

with slurm returning R, not RUNNING.

This seems to be the problem, indeed.
Our Slurm version is 15.08.7, which is perhaps a bit old.
The output of squeue is:
$ squeue --job 9101064
JOBID        USER    ACCOUNT       NAME    STATE       TIME TIME_LIMIT CPUS MIN_MEMORY NODELIST(REASON) DEPENDENCY
9101064        joe     joe           XXX  RUNNING    3:57:36   12:00:00    1      8000M slurm-worker-002

Do you have any advice on how to treat this, apart from modifying translate_table?

I confirm that editing parsl/providers/slurm/slurm.py locally (by adding 'RUNNING': JobState.RUNNING, to translate_table) actually fixes this issue on our system.
Another simple fix is to play with the default of  the --format option of squeue. By making sure that job status is printed as %t (its compact  form, like R) and not as %T (the extended version, like RUNNING), parsl will correctly parse the output of squeue. This is straightforward to achieve by setting the SQUEUE_FORMAT environment variable in the worker_init field of SlurmProvider (I just realized that for some reason on our system this variable currently uses %T). I tried it, and I confirm that it works as expected (i.e. it does not open a second block).
On our side, this second option already fixes the issue in practice (with no need for a local patch of parsl), although including also the extended job-status strings in translate_table seems something useful to do in general.

thanks for the update - so I think this comes down to site configuration being unexpected, rather than the particular version of slurm version as we suspected earlier.


Question: Scaling data when using LightGBM cross-validation

I was wondering whether it is possible to scale data when using the LightGBM cross-validation. So far, it seems that I have to program the cross-validation manually such that I can standardize the training data and apply the same transformation to the validation data in each iteration.
Am I missing something, or is it simply not possible?
Thanks!

Keep in mind that scaling the features shouldn't have any effect on tree-based models like lightgbm, however if you want to do it you can use the scikit-learn API with something like the following:
import lightgbm as lgb
import pandas as pd
from sklearn.datasets import make_regression
from sklearn.model_selection import GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

X, y = make_regression(n_samples=1_000, n_features=4, n_informative=2, random_state=0)
estimator = Pipeline([
    ('scaler', 'passthrough'),
    ('model', lgb.LGBMRegressor(random_state=0))
])
search = GridSearchCV(
    estimator,
    {'scaler': ['passthrough', StandardScaler()], 'model__': [0.1, 0.01]},
    scoring='neg_mean_squared_error',
)
search.fit(X, y)
search_df = pd.DataFrame(search.cv_results_['params'])
search_df['score'] = search.cv_results_['mean_test_score']
search_df

model__learning_rate
scaler
score

0.1
passthrough
-92.827

0.1
StandardScaler()
-94.8684

0.01
passthrough
-1311.86

0.01
StandardScaler()
-1312.23

@jameslamb do you know why the scores aren't identical? I just tried this example with xgboost and sklearn.ensemble.GradientBoostingRegressor and they get the exact same scores with and without scaling.

@Dr4cky

I was wondering whether it is possible to scale data when using the LightGBM cross-validation.

I guess you need fpreproc argument.
https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.cv.html
https://github.com/microsoft/LightGBM/blob/d05f54701ee88a1373bd9e0a1ea4317827015c9c/tests/python_package_test/test_engine.py#L1589-L1607

@jmoralez I came here because I have a similar question. I get that for tree models the scale shouldn't have an impact (see the discussion here https://datascience.stackexchange.com/questions/60950/is-it-necessary-to-normalize-data-for-xgboost for example). However I don't really see how it would hold with l1 or l2 regularisation. Scale usually matter a lot when there is regularisation. For exemple L1 regularisation might drop features with more important scales first. I don't really understand why 'scaling data is not necessary' would hold when there is regularisation in GBT models. This might exeplain the difference here if there is a small regularisation enabled by default.

I think you may be thinking of L1 and L2 regularization in the context of penalizing coefficients as with Lasso or Ridge. In GBDT the regularization is applied to the split gain. So for example if you have a split at x = 100 with some gain g and you try scaling your feature, the split may now be at x = 50 but the gain should be the same.

Yes I was interpreting the l1-l2 penalisation as penalising coefficients. Do you have a source that explain the penalisation of the gain in details ? I have found : Regularized Gradient Boosting but it is a bit stale in terms of illustrations.

Hi @lcrmorin, sorry for the late response.
In LightGBM there are two parameters that control the regularization: reg_alpha and reg_lambda and they roughly work like in the following equation:

Note that I said roughly because the numerators are actually lower bounded by zero. So as you can see reg_alpha can turn a small gain into zero, as in L1 regularization.
On the other hand reg_lambda acts on the denominator, making the gain smaller in a smoother way but hardly making it zero, as in L2 regularization.
Here's a small example showing this for the L2 objective (mean squared error), where the gradients are y - pred and the hessians are ones:
import lightgbm as lgb
import numpy as np

def get_split_gain(x, gradients, threshold, l1=0, l2=0):
    gs = gradients.sum()
    hs = gradients.size
    left_mask = x <= threshold
    right_mask = ~left_mask
    gls = gradients[left_mask].sum()
    grs = gradients[right_mask].sum()
    hls = left_mask.sum()
    hrs = right_mask.sum()
    return (gls - l1)**2 / (hls + l2) + (grs - l1)**2 / (hrs + l2) - (gs - l1)**2 / (hs + l2)

def get_threshold_and_gain_from_model(X, y, l1=0, l2=0):
    model = lgb.LGBMRegressor(n_estimators=1, num_leaves=2, reg_alpha=l1, reg_lambda=l2)
    model.fit(X, y, init_score=np.zeros(y.size))
    trees_df = model.booster_.trees_to_dataframe()
    return trees_df.loc[trees_df.node_depth == 1, ['threshold', 'split_gain']].iloc[0]

np.random.seed(0)
x = np.linspace(-1, 1, 100)
X = x.reshape(-1, 1)
y = x**2 + 0.1*np.random.rand(x.size)

threshold, gain = get_threshold_and_gain_from_model(X, y)
threshold, gain
# (-0.5858585858585857, 2.7980101108551025)

gradients = y # init_score is 0
get_split_gain(x, gradients, threshold)
# 2.798012322171269

# L1
_, gain = get_threshold_and_gain_from_model(X, y, l1=0.1)
# 2.673409938812256
get_split_gain(x, gradients, threshold, l1=0.1)
# 2.673407082375469

# L2
_, gain = get_threshold_and_gain_from_model(X, y, l2=0.1)
# 2.7535200119018555
get_split_gain(x, gradients, threshold, l2=0.1)
# 2.753522844719676

Note that this only involves the target (through the gradients) so scaling the features shouldn't have any effect on the gain, even with regularization. Hope this gives you a better idea of how the regularization works.


Ionicons with -outline suffix are not rendering with the 5.0.0 release.

Several icons I use with are Ionicons, I've been using the -outline suffix, so (example, ios-pricetag-outline). Which have rendering fine with the 4.x version. I upgrade to 5 and now getting question marks (instead of desired icon) for all of them when adding the -outine.

Why did new release version remove these icons?

They have probably been removed from the font, check the website. I can only find solid pricetags for iOS.

Super annoying! I tried installing older versions but I must've done something wrong because they still weren't rendering. Finally, I found an old release from January 2018 in in the @expo module, so I just replaced the react-native-vector-icons in node_modules with that one. And bingo -- it worked again. Remind me never to upgrade this library.

Are you linking the library after upgrading? It is needed to replace the font files and has to be done every time you upgrade the library and the fonts has been updated 🙂

The fonts are generated thanks to the glypthmap (Ionicons.json) and Ionicons.tiff. **If you want to keep the nice-looking, thin "outline" style icons, DO NOT UPGRADE THIS LIBRARY.
Below is a side-by-side comparison of the source code! In the latest update you can clearly see that the ios-home-outline was totally removed.

I have no clue why this happened but I'm very lucky I had saved some older versions. If you need the files let me know here and I can create a gist explaining how to revert this change.

https://github.com/ionic-team/ionicons/issues/550

@daviscabral Thank you, everyone should be respected until they did something to deserve otherwise 🙂 If I can help give back to the community that helped me learn a lot then I think I should but yes, people seem to equate open source with a proprietary company that can afford dedicating people to support. A lot of us are doing open-source on our spare-time and I would like more people to appreciate that but this was kinda the first time 🙃 Thank you for real 🙏

People are annoyed because the -outline icons used to be there, and were removed without notice. It was extra work to remove them. If a dependency removed them in an update, don't include the ^ in the package.json for that dependency.

To people still coming here and liking the reply about not using ^, it does not impact the library for users since it is a devDependency! Fonts are checked into the repo along with the glyphmap meaning that the font file is linked to the version of this library instead of the actual dependency version. Whoever the ghost is was wrong, unnecessary hostile and does not understand how this library works. There was no "extra work" to remove any icons.
If you are worried about upgrading, check the release notes before doing so to make sure that you know what you are getting 🙂
I'm closing this now because it is not really an issue that can be fixed currently.

Here is how I fixed this problem.
My example is for the Ionicons pack (import { Ionicons } from '@expo/vector-icons';) but you can apply it to any icon pack.
There are 3 problems to fix.
1. Import can't be done the same way
Fix it like this:
In your component, replace
import { Ionicons } from '@expo/vector-icons';
with
import Ionicons from 'react-native-vector-icons/Ionicons';
2. All '-outline' icons have been removed from the list of valid icon names
(this is why you get Warning: Failed prop type: Invalid prop 'name' of value 'ios-paper-outline' supplied to 'Icon')
Fix it like this:
In a temp folder, checkout your code from an earlier version, when outline icons worked for you, and grab this file:
<EMAIL_ADDRESS>Then paste it into your project.
It contains the mapping between outline icons and the integer code.
Now your 'ios-xxx-outline' icon name is valid again, but it's still not present in the actual font file.
3. All '-outline' icons have been removed from the font file
Fix it like this:
From your temp folder with the old code version, grab this file:
<EMAIL_ADDRESS>Then paste it into your project.

^--- make sure to use patch-package after following the advise above

You can just download the .SVG icon from https://ionicons.com/ so it could be any outline or filled or shape and after downloading (example : home-outline.SVG) move it to Your_Project\node_modules\ionicons\dist\ionicons\svg directory... and then you are Ready To Use it ... i tried and it displays the icons !


use latest nightly compiler with new hermit target

bors r+

bors r-

bors r+

bors r+

bors retry

bors retry

bors r-

bors r+


Changing appearances of widgets

This is an amazing project! Clean code, nice documentation, extremely supportive on issues, etc. Impressive work.
I have a question regarding form widget annotations. The API to add new annotations is pretty straightforward.
My use case however is to manipulate existing ones.
I didn't find much available on the documentation, but digging through the code, I started doing this:
const PDFLib = require('pdf-lib')
const fs = require('fs').promises

const { rgb } = PDFLib

const run = async () => {
  const buffer = await fs.readFile(process.argv[2])
  const doc = await PDFLib.PDFDocument.load(buffer)

  const form = doc.getForm()

  for(field of form.getFields()) {
    if (field.constructor.name !== 'PDFTextField')
      continue

    field.setText('one punch man')
    const [ widget ] = field.acroField.getWidgets()
    const ac = widget.getAppearanceCharacteristics()
    ac.setBackgroundColor(rgb(0,1,0))

    widget.getBorderStyle().setWidth(10)
    ac.setBorderColor(rgb(1,1,0))
  }

  await fs.writeFile(process.argv[3], await doc.save())
}

run()

The purpose of this script is to iterate through all forms and change the background color and border color of text fields.
The problem im facing is that the pdf saves fine, but none of the changes i made to widgets are effective. The input pdf and output pdf's widget background colors and border colors are the same.
Any ideas?

Can you try form.markFieldAsDirty(field.ref)?

Hey @ajaishankar I just gave it a try. No luck unfortunately. No errors, but the output is still the same.

For some further clarification, I did use your above code to set my border width to 0 and that did correctly remove the border. Color changes didn't work for me either, though.

Hello @emilsedgh!
You need to use the colorToComponents(...) function. Note also that the best way to check what subtype of PDFField you're working with is instanceof.
I've updated the snippet you shared below:
import { colorToComponents, PDFDocument, PDFTextField, rgb } from 'pdf-lib';

(async () => {
  const dodBytes = await fetch(
    'https://pdf-lib.js.org/assets/dod_character.pdf',
  ).then((res) => res.arrayBuffer());

  const pdfDoc = await PDFDocument.load(dodBytes);

  const form = pdfDoc.getForm();

  for (const field of form.getFields()) {
    if (field instanceof PDFTextField) {
      field.setText('one punch man');

      const [widget] = field.acroField.getWidgets();
      const ac = widget.getAppearanceCharacteristics();
      ac?.setBackgroundColor(colorToComponents(rgb(0, 1, 0)));

      widget.getBorderStyle()?.setWidth(5);
      ac?.setBorderColor(colorToComponents(rgb(1, 0, 0)));
    }
  }

  const pdfBytes = await pdfDoc.save();
})();

This script produces a document that looks like the following:

As you observed, pdf-lib does not provide high-level APIs for modifying the widget properties of existing form fields. So it is necessary to use low-level APIs that are not well documented. But I would very much like to add high-level APIs for this. If you're interested in submitting a PR, I'd be happy to provide guidance!

Hey @Hopding
Thank you so much for the fantastic library and great support.


Create fixed point math library

Only add, sub, mul and div.
Use base64 format from ABDK and SafeMath from OpenZeppelin

Should we just reference ABDK, or create a stripped down version of their contract?


Explanation regarding multiplying linear biases with q.k^T

From the paper:

But the README recommends multiplying the linear biases with the mask: https://github.com/ofirpress/attention_with_linear_biases/blob/master/fairseq/models/transformer.py#L1022
What I'm struggling to understand is how does that translate to multiplying the linear biases before softmaxing the output of q.k^T.
Thanks in advance.

Ya it's just an implementation trick to make runtime faster.
What we want to do is:
attn = q*k + alibi_bias + mask

but to make it faster, basically what we do is:
mask = mask + alibi_bias
attn = q*k + mask

So both approaches are equivalent, but the second runs faster.

Appreciate the clarification thank you.

Oh I didn't write this, but the reason this runs faster is because we run the
mask = mask + alibi_bias
line just once and re-use it many times.

@ofirpress I have another similar question but it's about an implementation detail.
My parameters are the following:

Number of attention heads: 8
Projection dimension: 768
Maximum sequence length of training data: 512
Maximum tokens: 1024

Now, the q.k^T should be (for a batch size of 1): (1, 8, 512, 512) while alibi would be: (16, 1, 512) (as per the calculations shown in the repository).
If I were to do: attention_scores = attention_scores + alibi in this case it won't be broadcastable and naturally will result in an error. Am I missing out on anything here?


README Improvments

Hey! I have made more README improvements.
Here is the overview of the changes I made:

Added a project logo (I hope you like it !)
Added a short description below the logo for introducing the project to the user
Added a TOC-like feature below the intro to the important sections of the Document
Added a related links section which is designed for beginner users if they are looking for more details about the features, functionalities, and concepts

Thanks for your help! Love what you did! I will add the program files if you would like to work on those.


OAK-10253: Option to only collect references when calling checkConsis…

…tency

Add markOnly option for the check-consistency command

LGTM - just one thing - could we add the new option with description to the readme of oak-run ?

Thanks!! Yes, I missed the doc update


libfilezilla: 0.37.2 -> 0.38.1

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/libfilezilla/versions.
meta.description for libfilezilla is: "A modern C++ library, offering some basic functionality to build high-performing, platform-independent programs"
meta.homepage for libfilezilla is: "https://lib.filezilla-project.org/"
meta.changelog for libfilezilla is: ""
Updates performed

Version update

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.38.1 with grep in /nix/store/wg47b4zwmnxgshqqhgdy9lhzk1ll01a6-libfilezilla-0.38.1
found 0.38.1 in filename of file in /nix/store/wg47b4zwmnxgshqqhgdy9lhzk1ll01a6-libfilezilla-0.38.1

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath
filezilla
libfilezilla

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/wg47b4zwmnxgshqqhgdy9lhzk1ll01a6-libfilezilla-0.38.1 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A libfilezilla https://github.com/r-ryantm/nixpkgs/archive/79e590cac00ef0e0464bc70398fe03e77a02f9ae.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/wg47b4zwmnxgshqqhgdy9lhzk1ll01a6-libfilezilla-0.38.1
ls -la /nix/store/wg47b4zwmnxgshqqhgdy9lhzk1ll01a6-libfilezilla-0.38.1/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

filezilla
libfilezilla

Maintainer pings
cc @pSub for testing.

Result of nixpkgs-review pr 184294 run on x86_64-linux 1

2 packages built:

filezilla
libfilezilla


I am unable to install

I am unable to install. He's sending it to a private directory on his github. The error in the created log record.
.
"install" "electron-router"
28 verbose node v10.15.3
29 verbose npm  v6.11.3
30 error code ENOENT
31 error syscall spawn git
32 error path git
33 error errno ENOENT
34 error enoent Error while executing:
34 error enoent undefined ls-remote -h -t https://github.com/m0n0l0c0/eventemitter3.git
34 error enoent
34 error enoent
34 error enoent spawn git ENOENT
35 error enoent This is related to npm not being able to find a file.
36 verbose exit [ 1, true ]

the problem was git on windows, but getting it fixed. It is a tip for those who have the same problem.

Hello @lukexp, thanks for reporting it anyway!


Add Logger isEnabledFor

Fixes https://github.com/microsoft/picologging/issues/138

Codecov Report

Merging #139 (94932c8) into main (783acf1) will decrease coverage by 0.05%.
The diff coverage is 71.42%.

@@            Coverage Diff             @@
##             main     #139      +/-   ##
==========================================
- Coverage   89.05%   88.99%   -0.06%
==========================================
  Files          11       11
  Lines        2119     2126       +7
==========================================
+ Hits         1887     1892       +5
- Misses        232      234       +2

Impacted Files
Coverage Δ

src/picologging/logger.cxx
84.34% <71.42%> (-0.26%)
:arrow_down:

src/picologging/handlers.py
94.55% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

thanks!


Add uprezzing

This works in my CLI code, so just need to port and bundle the model (or allow download I guess?)

Probably do this as part of export as an option?
Could have it be a per frame option, but when would someone want to do that only on some frames? Think it's yes or no for the whole thing.

Related to #54

Probably add a menu to export button

Export and 'Uprez & Export'
Which is button vs menu item?


Allow for more than one user in Authorization

Collab sessions can result in more than one user being responsible for a PUT - this PR makes it so that we da-admin allows more than one entry in the Authorization header and checks for all if the respective users is authorized. As a special case, it allows for an empty entry in the list of tokens which is interpreted as an anon user being present as well.

I think this raises an interesting point. I'm almost thinking we should say, if you want to write something, you have to be logged in. Even for projects that are not logged down.


Host an open-elevation service

For getting elevation data from specific latitude/longitude coordinates an service needs to be set up which should also be located on our h4nsolo server.
As entrance point the following repository provides information for setting up the service:
https://github.com/Jorl17/open-elevation

The above mentioned repository was forked (under https://github.com/dephiloper/open-elevation) to modify the data retrieving scripts that they only download relevant data (germany).

Like in the tileserver-gl the open-elevation api is available under https://h4nsolo.f4.htw-berlin.de/elevation/ followed by the default api route.

To install the open-elevation endpoint these instructions were followed:
https://github.com/Developer66/open-elevation
Additionally gdal was installed via custom pip command, which gathers the currently installed gdal version itself.
pip install GDAL==$(gdal-config --version) --global-option=build_ext --global-option="-I/usr/include/gdal"

The endpoint can now be accessed via (example query for center of berlin):
https://h4nsolo.f4.htw-berlin.de/elevation/api/v1/lookup?locations=52.520008,13.404954


Add JSDoc support

Adds utilities to parse and render JSDoc comments. Also:

Extends the AstNodeDescription for range information
Modifies the default hover implement to render {@link} inline JSDoc tags.
Improves the langium language configuration to better deal with JSDoc.

I was expecting some new service for the Langium module. Something like this:
interface DocumentationCommentHandler {
  // extracts the comment string from the hidden CST in front of the given node
  getDocumentation(node: AstNode): string|undefined;
}

Which can be used by JSDOC parser…

@Lotes I've tried to keep the JSDoc implementation to be as independent of any services as possible. It's just a function, not a whole service.


refactor(patients): Refactor patient related persons to use react query

Fixes #2336
Changes proposed in this pull request:

Added useAddPatientRelatedPerson hook
Added useRemovePatientRelatedPerson hook
Refactored RelatedPersonTab to use the usePatientRelatedPersons and useRemovePatientRelatedPerson hooks
Refactored AddRelatedPersonModal to use the useAddPatientRelatedPerson hook
Removed addRelatedPerson and removeRelatedPerson from patient-slice.ts

Thanks for the feedback, @jackcmeyer. I'll make the changes.

Hey @jackcmeyer, I made the changes you requested. Using the state variable for the query to get patients from the usePatients hook mostly works, but it seems a little inconsistent. Let me know what you think about it and if there is something that I could do differently to make it behave more like the original onSearch.

Hey @jackcmeyer, I made the changes you requested. Using the state variable for the query to get patients from the usePatients hook mostly works, but it seems a little inconsistent. Let me know what you think about it and if there is something that I could do differently to make it behave more like the original onSearch.

Can you describe the iconsistencies?

Can you describe the inconsistencies?
Not well, unfortunately. What I saw with my test patients was that some would have suggestions with fewer letters than others, and this would sometimes change based on the patient the related person was being added to. Sometimes the next letter would make the suggestion disappear altogether, with the next letter or a space restoring it.


Add UTF-8 support

This is now available on the Steam Workshop: https://steamcommunity.com/sharedfiles/filedetails/changelog/109643223


"Izvoru", "Moșia Mică" are listed in the drop-down menu

Description:
When user writes ” Izvoru” and "Moșia Mică" the locations appear in the drop-down menu next to the comune and the county which is a part according to the law.
Precondition:
The website is up an running.

Step 1
Write in the search bar "Izvoru"

Expected results
The location appears in the drop-down menu as "Sat Izvoru, DÂMBOVIȚA (Valea Lungă)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Write in the search bar "Moșia Mică"

Expected results
The location appears in the drop-down menu as "Sat Moșia Mică, DÂMBOVIȚA (Valea Lungă)".


glibcLocales: cleaner environment handling

A continuation/re-imagining of #4318, trying to make it cleaner.

make pkgs attribute null on unsupported Darwin platform
add setup hook to provide much-duplicated environment variable

This way, builds should find the locale archive when they add pkgs.glibcLocales to buildInputs.
My only problem is that I wasn't able to test this on Linux yet. On Darwin it works fine, since it just removes everything :grin:. I mocked the glibcLocales build and it did create the proper setup-hook with the correct path.

Merged some part of it manually. What would be a good regular expression to grep for for the rest of NixPkgs?

@7c6f434c just git grep LOCALE_ARCHIVE or git grep glibcLocales should do the trick. I think everything that's on master now is properly shielded to eval only on Linux. I wonder if I should change those remaining usages, they're mostly in build helpers etc so will cause massive rebuild and I can't test some of them (e.g. Haskell).

@7c6f434c just git grep LOCALE_ARCHIVE or git grep glibcLocales should do the trick. I think everything that's on master now is properly shielded to eval only on Linux. I wonder if I should change those remaining usages, they're mostly in build helpers etc so will cause massive rebuild and I can't test some of them (e.g. Haskell).

Maybe these could be updated together with the normal updates of such
expressions then.
I don't like mass rebuilds, either.

The Haskell uses glibcLocales on Linux only.

I wonder how do you plan to get locales in darwin libc (in future)? If such things should be handled purely, i.e. compiling them from Apple-provided sources, then we might better have a general libcLocales or something. Of course, using system-provided libc and locales might be easiest...

This is now partially upstream: we have locales package only for Linux.
What is missing is the setupHook, I think it's a good addition save some typing on each package. I propose we make a separate commit to add that and remove redundant exports in packages.

This seems to have died…


Don't recompile assembly if it already is compiled (same as #17)

Don't recompile assembly if it already is compiled (same as #17)
Perhaps I misunderstand the resolved #17 but an assembly is always compiled,
i.e. js scripts are generated on each run.
After looking at the code, I started to think that I should tell to call
getCompiled instead of compile, that is use the option scripted=false for
Warp.RunAndWaitForInput. But this scenario always fails with
"Assembly not compiled. When using Scripted = false, the application must be precompiled."

It is not clear how it is supposed to be precompiled. It is already
"precompiled", in a way, I do not change the assembly, all scripts are
generated at the last run. But they are not "discovered and reused".
I started to think that something has changed since #17. Is this the case?
Anyway, I tried to solve the issue by adding the following code to getCompiled.
When it is about to fail with the above message, I check time stamps of the
assembly and existing scripts. If scripts are up to date, I use them, otherwise
I call compile instead. Now it works. But I am not sure if this is quite right.
Instead of the original code I use
| _ ->
    if asm.IsDynamic then
        eprintfn "Assembly not compiled. When using Scripted = false, the application must be precompiled."
        None
    else
        let (+/) x y = Path.Combine(x, y)
        let dir = rootDir +/ "Scripts" +/ "WebSharper"
        let pathReadableJavaScript = dir +/ "WebSharper.EntryPoint.js"
        let pathCompressedJavaScript = dir +/ "WebSharper.EntryPoint.min.js"
        let toCompile =
            if not (File.Exists pathReadableJavaScript) || not (File.Exists pathCompressedJavaScript) then true
            else
                let asmTime = File.GetLastWriteTimeUtc asm.Location
                asmTime > (File.GetLastWriteTimeUtc pathReadableJavaScript) || asmTime > (File.GetLastWriteTimeUtc pathCompressedJavaScript)
        if toCompile then
            Debug.WriteLine "Compiling..."
            compile asm
        else
            Debug.WriteLine "Using compiled..."
            let opts = { FE.Options.Default with References = refs }
            let compiler = FE.Prepare opts (eprintfn "%O")
            Some {
                ReadableJavaScript = pathReadableJavaScript
                CompressedJavaScript = pathCompressedJavaScript
                Info = compiler.GetInfo()
                References = refs
            })

If the solution is fine I will make a PR. If something else is wrong (or I get this wrong) please let me know.

Hi @nightroman , sorry for the delay.  Warp needs a heavy revision and updating to the latest Zafir (WebSharper 4), and a lot of the shorthands it provides are already in the Zafir sitelets library. What's your scenario?  Are you doing interactive scripting with Warp?

My scenario is somewhat "a friction-less scripted and standalone full-stack F#
client-server applications", as Warp describes itself. It is useful and easy to
use for prototyping in the first place. For prototyping, it is probably fine to
compile each time, even if the sources do not change.  But some prototypes may
stay is use. And some lightweight applications may be based on Warp.
It would be nice to avoid compilation if sources do not change.
This seems to be doable in some cases, and something was done in the past (#17).
Regardless of the scenario, it would be useful to know the purpose of the
option scripted=false.
If Warp needs a heavy revision then this is probably not a big deal now.


tests fail on Debian Stretch

=========================================== ERRORS ============================================
_____________________ ERROR collecting secret_handshake/test_boxstream.py _____________________
.eggs/pytest-3.1.3-py3.5.egg/_pytest/python.py:408: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
.eggs/py-1.4.34-py3.5.egg/py/_path/local.py:662: in pyimport
    __import__(modname)
secret_handshake/__init__.py:22: in <module>
    from .network import SHSClient, SHSServer
E     File "/home/elektron/code/PySecretHandshake/secret_handshake/network.py", line 48
E       yield msg
E       ^
E   SyntaxError: 'yield' inside async function
______________________ ERROR collecting secret_handshake/test_crypto.py _______________________
.eggs/pytest-3.1.3-py3.5.egg/_pytest/python.py:408: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
.eggs/py-1.4.34-py3.5.egg/py/_path/local.py:662: in pyimport
    __import__(modname)
secret_handshake/__init__.py:22: in <module>
    from .network import SHSClient, SHSServer
E     File "/home/elektron/code/PySecretHandshake/secret_handshake/network.py", line 48
E       yield msg
E       ^
E   SyntaxError: 'yield' inside async function
!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!
=================================== 2 error in 0.28 seconds ===================================

I'm actually only targetting 3.6+ right now. I'd have to figure out the cost of maintaining a 3.5-compatible version, since a few things changed between those.

word, I'll check back in on your projects in 2019 when Debian stable is upgraded ;-)

:laughing:
Let me have a look at the changes when I have some time. I just started this way because it was easier.


Is assignable

Purpose

Describe the problems, issues, or needs driving this feature/fix and include links to related issues.

Fixes #
Approach

Describe how you are implementing the solutions along with the design details.

Samples

Provide high-level details about the samples related to this feature.

Remarks

List any other known issues, related PRs, TODO items, or any other notes related to the PR.

Check List

[ ] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #10 (c89a61d) into master (2a8cf8c) will increase coverage by 55.15%.
The diff coverage is 57.10%.

@@              Coverage Diff              @@
##             master      #10       +/-   ##
=============================================
+ Coverage     15.71%   70.87%   +55.15%
- Complexity        0    38675    +38675
=============================================
  Files            47     2863     +2816
  Lines          1266   158301   +157035
  Branches        197    19943    +19746
=============================================
+ Hits            199   112190   +111991
- Misses         1054    39566    +38512
- Partials         13     6545     +6532

Impacted Files
Coverage Δ

...va/io/ballerina/shell/cli/BShellConfiguration.java
0.00% <0.00%> (ø)

...in/java/io/ballerina/shell/cli/BallerinaShell.java
0.00% <0.00%> (ø)

.../java/io/ballerina/shell/cli/PropertiesLoader.java
0.00% <0.00%> (ø)

...a/io/ballerina/shell/cli/ReplShellApplication.java
0.00% <0.00%> (ø)

...ava/io/ballerina/shell/cli/ShellExitException.java
0.00% <0.00%> (ø)

...n/java/io/ballerina/shell/cli/TerminalAdapter.java
0.00% <0.00%> (ø)

.../ballerina/shell/cli/handlers/AbstractCommand.java
0.00% <0.00%> (ø)

...o/ballerina/shell/cli/handlers/CommandHandler.java
0.00% <0.00%> (ø)

...io/ballerina/shell/cli/handlers/DeleteCommand.java
0.00% <0.00%> (ø)

...a/io/ballerina/shell/cli/handlers/ExitCommand.java
0.00% <0.00%> (ø)

... and 2974 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1e067e...c89a61d. Read the comment docs.


[ZEPPELIN-5354]  Inconsistent bold markdowns of keywords in 'binding mode" document

What is this PR for?
Fix bold markdowns of keywords in "interpreter binding mode" document for consistency
What type of PR is it?
[Documentation]
What is the Jira issue?

https://issues.apache.org/jira/browse/ZEPPELIN-5354

How should this be tested?

Checked updated document locally

Questions:

Does the licenses files need update? No
Is there breaking changes for older versions? No
Does this needs documentation? No

Thanks @cuspymd for the contribution, BTW could you create an umbrella ticket for the document related issues ? so that you can put all the documentation fix as the subtask of this umbrella ticket

Thanks @cuspymd for the contribution, BTW could you create an umbrella ticket for the document related issues ? so that you can put all the documentation fix as the subtask of this umbrella ticket

Can you tell me how to create an umbrella ticket? I will refer to it in the next PR.

@cuspymd It is the same as general jira ticket, just naming it as umbrella ticket and link other tickets as its subtask. https://issues.apache.org/jira/browse/ZEPPELIN-4936


QUESTION: Which method is used to determine the distance.

Hello,
in the documentation for the Microsoft Azure Kinect I read that the depth camera uses the time of flight method to measure the distances in the scene. But in another article I read thet the TOF camera uses the Phase shifting or Phase Coherence method to determine the distances in the scene. So my question is: Does the Azure Kinect use both method to avoid ambiguity in the distance values if the movements was bigger than the have of the phase? Or does it only use one of the methods? If the second is the case: Which method is used?
Thank you in advance.

Depth Sensor in Azure Kinect DK uses Indirect Time of Flight Technique. Read this paper for additional information https://ieeexplore.ieee.org/document/8310200

Depth Sensor in Azure Kinect DK uses Indirect Time of Flight Technique. Read this paper for additional information https://ieeexplore.ieee.org/document/8310200


Linked List :1: Remove Nth Node From End of List

Given the head of a linked list, remove the nth node from the end of the list and return its head.
Example 1:
Input: head = [1,2,3,4,5], n = 2
Output: [1,2,3,5]
Example 2:
Input: head = [1], n = 1
Output: []
Example 3:
Input: head = [1,2], n = 1
Output: [1]

@Anjali-Chaudhary3 I would like to solve this issue under GSSOC'23. Can you please assign me this issue?


add override cra paths

Added CRA paths override to resolve issues with not being able to modify the webpack entry and changing the app src folder.
/* cra-paths-plugin.js */

module.exports = {
    overrideCraPathsConfig: ({ context, pathsConfig, pluginOptions }) => {
        return Object.assign(pathsConfig, pluginOptions);
    }
}

/* craco.config.js  */

const path = require('path');
const OverridePaths = require('./cra-paths-plugin');
module.exports = {
    paths: {
        appBuild: path.resolve(__dirname, 'dist'),
        appSrc: path.resolve(__dirname, 'app'),
        // appIndexJs: path.resolve(__dirname, 'app/index.tsx'),
        // testsSetup: path.resolve(__dirname, 'app/setupTests.ts'),
        // proxySetup: path.resolve(__dirname, 'app/setupProxy.ts'),
        // appTsConfig: path.resolve(__dirname, 'tsconfig.json')
    },
    // Or
    plugins: [
        {
            plugin: OverridePaths,
            options: {
                appBuild: path.resolve(__dirname, 'dist'),
                appSrc: path.resolve(__dirname, 'app')
            }
        }
    ]
}

#46 #66 #169 #168
Note that the src matching problem in the reportFiles parameter of the webpack.config.js file is not resolved.
new ForkTsCheckerWebpackPlugin({
          typescript: resolve.sync('typescript', {
            basedir: paths.appNodeModules,
          }),
          async: false,
          checkSyntacticErrors: true,
          tsconfig: paths.appTsConfig,
          compilerOptions: {
            module: 'esnext',
            moduleResolution: 'node',
            resolveJsonModule: true,
            isolatedModules: true,
            noEmit: true,
            jsx: 'preserve',
          },
          reportFiles: [
            '**',
            '!**/*.json',
            '!**/__tests__/**',
            '!**/?(*.)(spec|test).*',
            '!**/src/setupProxy.*',
            '!**/src/setupTests.*',
          ],
          watch: paths.appSrc,
          silent: true,
          formatter: typescriptFormatter,
        }),

Thanks for the PR @LuHugo
I'll try to review it soon. I would love to quickly merge the PR but since the PR update a few key parts of CRACO I must take my time to thoroughly review what you did.
I also need to figure out if this is something I want CRACO to support. I did implement this feature a year ago and reverted it at last minute because it seems like something that could easily breaks and would be a nightmare to support.
Could you please briefly explain the thinking behind your implementation to help me review the code?
Thank you,
Patrick

Thanks, the implementation I didn't find a simple magic way to do it, so I used a replacement involving the src path. If the user uses the paths configuration or the plugin sets up appSrc then use the src folder set by the user Replace the original src with the name of the package.
packages/craco/index.js
Add the createCraPathsConfig method.
packages/craco/lib/cra.js
Adds overrideCraPathsConfig, getAppSrcName, getOriginAppSrcName methods.
packages/craco/lib/features/cra-paths
New module for implementing path overrides.
packages/craco/lib/features/plugins.js
Added overrideCraPaths and applyCraPathsConfigPlugins methods to implement plugins.
packages/craco/lib/features/test/jest-cra-paths-override.js
Used to override path issues in jest.
packages/craco/lib/features/test/merge-jest-config.js
Import overrideJestAppSrcPath
packages/craco/scripts/build.js, packages/ craco/scripts/start.js, packages/craco/ scripts/test.js calls the overrideCraPaths method to override the paths respectively.

Really looking forward to getting this reviewed/merged

This also seems to break on Windows platform for some reason. Here is a screen-shot:

This also seems to break on Windows platform for some reason. Here is a screen-shot:

I'm very sorry, but since I've been using macOS for development, I neglected to mention that the file paths on Windows are different from macOS or Linux. I'll take the time to do a path fix for Windows systems.

Thanks for the work on this. Also looking forward to it.

Bump while waiting for the folks at create-react-app to merge https://github.com/facebook/create-react-app/pull/8986

facebook/create-react-app#8986 was merged
Just saying :P

Very nice

Is there anyway I can change the src source path to app?
I was trying different receipts here, and now stuck at configuring the appIndexJs path.
I cant make it using craco.config.js only.
Please advise if it is possible.

Only one idea I have, it is a problem with jest.config.js paths in Windows. Try to play there with backslashes


feat: job cancellation capability for IRS & utility class for test data generation

Job cancellation in IRS using Java interrupts
This PR addresses issue https://github.com/eclipse-tractusx/item-relationship-service/issues/411.

AASTransferProcessManager

When a new thread is taken from the thread pool to execute a new transfer process, instead of using execute, the code now calls submit for the created Runnable. This returns a Future, which is then stored in a ConcurrentHashMap. The map's keys are the uuid strings of the transfer processes, thus making it possible to associate a transfer process id with the task it represents.

To make use of this, there is now a cancelRequest method which retrieves a Future from the map using a transfer process uuid string argument.  Then, cancel(true) is called on that Future, which sets the interrupt flag for the thread executing the transfer process. Afterwards, the entry for that transfer process is removed from the map.

Before initiating a request, this interrupt flag is checked. If it is set, the transfer process is not started and a TransferInitiateResponse is returned with a newly added ResponseStatus of NOT_STARTED_JOB_CANCELLED.

In case of the unhappy path, where the map somehow doesn't contain an entry for a transfer process or something goes wrong when setting the interrupt flag, JobExceptions  are created using a newly added constructor. This constructor allows for directly setting the error detail.

JobOrchestrator

A cancelJob method has been added which is called when the IRS receives a job cancellation request. It uses a passed MultiTransferJob argument to get all transfer process ids for the job that is to be cancelled and iterates over the collection of ids calling AASTransferProcessManager's cancelRequest method. In each iteration, if cancelRequest throws a JobException, the method evaluates what happened based on the error detail that was specified when creating the Exception in cancelRequest. This detail is then used to construct an error message, which is uploaded to the job store using markJobInError.

BaseJobStore's writeLock now reliably triggers an InterruptedException, which allows to cancel any change to a job from one of the interrupted threads running the transfer processes.

IrsItemGraphQueryService's cancelJobById now calls cancelJob in JobOrchestrator.

Unit tests for AASTransferProcessManager and JobOrchestrator have been modified accordingly.

IrsWireMockIntegrationTest now has a method to test IRS job cancellation. The test uses the utility described below.

Utility for test data generation

This PR contains a new utility class called SingleLevelTestDataGenerator which can be used to generate single-level test data for a (theoretically, but practically infeasible) arbitrary recursion depth. The utility allows for choosing the number of relationships that each asset holds, if any at all. It uses another class which parses data needed for generation from a JSON file.

Closing this PR since we cannot merge this at the moment and will not add this within consortia phase.
Might re-visit this topic in the future


DMP-157 create audio metadata table

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/DMP-157
Change description
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x ] No

Apologies, it wasn't clear on the ticket, but could the name of the table please be moj_media and the column names match those in the entity class, ie. using the prefixes.

are we keeping the moj_ prefix??


If it's convenient for you, could you kindly provide the data you are using?

While reading papers, I am unsure about the form in which data is provided to LLMs. Is the OCR output directly provided to the model as a whole, asking it to make improvements, or are the errors manually identified first, and then the model is tasked with correcting only the erroneous parts?

This is for correction of OCR error only, so you provide the LLM with the OCR output as is, and it attempts to correct it.
For example paste the following into one of the Frontier models (Chatgpt, sonnet 3.5, Mistral Large, Gemini, etc)
"""
You are an expert at OCR extraction please correct the corrupted OCR below
FOR CHEAP WATCHES , Clocks , Gold Chains , and Jewellery , go to Lombard KIBBLE ' street S , 33 , G and racechurch 51 Ludgate street hill , , op one posite door the from Old fr si B lver ailey om Nine . ditto Gold Shillings , One watches Pound and . T Sixpence wo Five Pourids Shillings each Fifteen . ; Every time Shillings -pieces article , ; exchanird warranted . . List Plate of , prices watches post , and freft jewellery bought or ;
"""
The LM will correct the text, this is the basic idea of CLOCR-C. It is useful in the case you have a large amount of text from OCR process, and don't have the resources to extract the text again.
However, this paper is somewhat out of date now. I have a newer paper called scrambledtext. The rate of change in the LM space is such that I think the whole CLOCR-C concept will be outdated soon. But it is interesting and may be useful to you depending on your use case

If this is the Boros et al paper. It is difficult to know. I first thought that the differences were due to older versions of the language model. However, I now think that perhaps it is because the distribution of correction improvement is quite skewed. In my paper I used the median, whilst Boros et al used the mean. In certain cases particularly when there is hallucination, the "corrected" error can be massive compared to the original. This causes a divergence between the mean and median result.
I checked my results and using the mean instead of the median and it shows generally very poor results. Although I haven't checked the code of Boros et al, I would assume they used the mean instead of the median, and their results were overly influenced by the very poor results that can sometimes be produced. This has a particularly big impact on the normalised levenshtien distance they use, which would disproportionally return a value of -1, pushing their mean values negative.
I assumed that the results would be highly skewed due to the tendency of the LM's to hallucinate particularly corrupted data so I checked the distributions, and chose the median to be robust to skew.
I think this would explain the majority of the difference between our results.

I have submitted the data repository to be published, it has to be reviewed first, and as it is Christmas this may not happen until the new year. However, the scrambled text repo is already public and contains the same data in huggingface format you can access it here
https://rdr.ucl.ac.uk/articles/dataset/Scrambled_text_training_Language_Models_to_correct_OCR_errors_using_synthetic_data/27108334
When the CLOCR-C repo is made public the data is stored as text files which you may find more convenient.

Thank you for all your explanations; they have been very helpful to me. I will also take some time to read your latest work. If you have any new progress you'd like to share, feel free to reach out to me via this email. Lastly, I wish you a Merry Christmas and a wonderful holiday season!
At 2024-12-21 01:31:16, "Jonathan Bourne" @.***> wrote:
I have submitted the data repository to be published, it has to be reviewed first, and as it is Christmas this may not happen until the new year. However, the scrambled text repo is already public and contains the same data in huggingface format you can access it here
https://rdr.ucl.ac.uk/articles/dataset/Scrambled_text_training_Language_Models_to_correct_OCR_errors_using_synthetic_data/27108334
When the CLOCR-C repo is made public the data is stored as text files which you may find more convenient.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


Simplify Travis setup

I see that one of the travis builds keeps failing (and the one with allowed failure passing): https://travis-ci.org/xapi-project/xen-api-sdk/builds/547159404
Use just one remote, and use an https URL.
opam update seems to fail with a git URL.

@edwintorok travis is still failing with this PR.

I've fixed this upstream https://github.com/ocaml/ocaml-ci-scripts/pull/299, restarted the travis job now


fix bug + optimizations

Feel free to ask any question about this code. Understanding the code of the others is a great way to improve your rust mastering ;-)


Add Getting Started section to README

README Update
please add a _getting started _ section to the README so folks know how to start with this project!

Added Getting Started section in README file.

issue is now fixed


bug #3525, added default borderWidth for stacked bars

Thank you for creating chart.js, this fixes issue #3525.

Looks good. We can merge this once 2.4 releases
Thanks @melhoos

Isn't it a breaking change?

I think it's ok because all of the other elements have a border by default

That means every charts without an explicit borderWidth (so currently without border) will receive borders after this commit.

That's right. If that's going to cause too many problems we should close the original issue as 'wont fix'

I'm fine to merge it, I agree it's more consistent with other elements. It's not a bug fix though but a change of the default bar chart behavior which is currently defined as not displaying bar borders. So if we merge it, we should not forget to list this change in the "Breaking Changes" section of the 2.5 release notes.

Ok, lets think on this for a bit just to be sure before we merge it.

Closing since this is a breaking change that we don't want to merge


Applying the latest dcgm-exporter some issues with the exporter container

On the DCGM-exporter getting the following on the exporter container
gpu-operator   nvidia-container-toolkit-daemonset-gc4bl                          1/1     Running            0             43s
gpu-operator   nvidia-cuda-validator-5g9k2                                       0/1     Completed          0             39s
gpu-operator   nvidia-dcgm-exporter-pxl49                                        0/1     CrashLoopBackOff   2 (20s ago)   43s
gpu-operator   nvidia-device-plugin-daemonset-ss8fj                              1/1     Running            0             43s
gpu-operator   nvidia-device-plugin-validator-ddssg                              0/1     Completed          0             23s

When I look at the logs I see:
kubectl logs --previous --tail 100 nvidia-dcgm-exporter-pxl49 -n gpu-operator
Defaulted container "nvidia-dcgm-exporter" out of: nvidia-dcgm-exporter, toolkit-validation (init)
time="2022-08-18T23:45:25Z" level=info msg="Starting dcgm-exporter"
time="2022-08-18T23:45:25Z" level=info msg="DCGM successfully initialized!"
time="2022-08-18T23:45:25Z" level=info msg="Collecting DCP Metrics"
time="2022-08-18T23:45:25Z" level=info msg="No configmap data specified, falling back to metric file /etc/dcgm-exporter/dcgm-metrics.csv"
time="2022-08-18T23:45:25Z" level=fatal msg="Could not find Prometheus metry type label"

nvcr.io/nvidia/k8s/dcgm-exporter:2.4.5-2.6.7-ubuntu20.04

@amrragab8080,
You need to update the dcgm-exporter container. The latest version is 2.4.6-2.6.10.
It looks like you got CSV with labels from a newer version of the dcgm-exporter.
That feature was recently added.

That is weird - because that is after a helm repo update
helm search repo nvidia/gpu-operator
NAME                    CHART VERSION   APP VERSION     DESCRIPTION
nvidia/gpu-operator     v1.11.1         v1.11.1         NVIDIA GPU Operator creates/configures/manages

The GPU operator may not yet be updated to include the latest dcgm-exporter version.
The CSV file for the older dcgm-exporter should not have "labels" like these: https://github.com/NVIDIA/dcgm-exporter/blob/bdcfaecbcfba2e3a48bf96231edcbd91d109f000/etc/dcp-metrics-included.csv#L69

This was previously reported on the gpu-operator project along with a work-around: https://github.com/NVIDIA/gpu-operator/issues/363

thanks - confirmed its working now with the change


Make Datastore doctests use a namespace.

Fixes #3232.
Running nox -s doctests passes locally (I expect CI will not check it until post-merge):
$ nox -s doctests
nox > Running session doctests
nox > The virtualenv name was hashed to avoid being too long.
nox > virtualenv /usr/local/google/home/lukesneeringer/Code/Google/google-cloud-python/datastore/.nox/765eb60c -p python3.6
nox > chdir /usr/local/google/home/lukesneeringer/Code/Google/google-cloud-python/datastore
nox > pip install --upgrade mock pytest sphinx ../core/
nox > pip install --upgrade ../test_utils/
nox > pip install --upgrade .
nox > py.test --quiet tests/doctests.py
.
1 passed in 12.24 seconds
nox > Session doctests successful. :)

P.S. It is really hard to edit anything with three-space indenting.


chore(support): delete test FormR PartB

Form was submitted by dev team to validate user data.
NO-CARD: Support query tracked in spreadsheet

I think we should either include S3 deletion in this migration, or do it all off the books to keep some form of consistency in what we action.

Drats... distraction meant I didn't mention I've removed from S3.


schema: use examples to skip validation

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 17 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-1.3%) to 76.882%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

action.go
0
17
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-1.3%

Covered Lines:
286

Relevant Lines:
372

💛  - Coveralls


Disable to print color if --no-color is specified

Pull Request Checklist

[x] implement the feature
[ ] write the documentation
[x] extend the test coverage
[ ] update the specification
[ ] adjust plugin docstring
[ ] modify the json schema
[ ] mention the version
[ ] include a release note

Fixes: #2759

Hi @happz, please help complete the remaining work of this PR as I no longer have the permission to update it, thanks very much!
huanli@FOO:~/dev/2759/dev$ git remote -v
origin	git@github.com:teemtee/tmt.git (fetch)
origin	git@github.com:teemtee/tmt.git (push)

huanli@FOO:~/dev/2759/dev$ git branch -a | grep huanli
* dev-huanli-2759-20240326
  remotes/origin/dev-huanli-1062-20240320-fips
  remotes/origin/dev-huanli-1918-20230316-login-t
  remotes/origin/dev-huanli-2759-20240326

huanli@FOO:~/dev/2759/dev$ git push -f
ERROR: Permission to teemtee/tmt.git denied to idorax.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.

Hi @happz, please help complete the remaining work of this PR as I no longer have the permission to update it, thanks very much!

That's weird, I don't recall any change that would suddenly prevent you from pushing to branches like you used to. Could it be just a temporary glitch? "fatal: Could not read from remote repository." suggests it might have been a network issue or Github outage or something like that. Have you tried pushing to teemtee/tmt since then?
I can definitely finish the PR if needed, but I'd prefer getting your workflow fixed so you could contribute even more :)

hmm, I don't see @idorax in https://github.com/orgs/teemtee/people. Maybe it's best to just use a fork going forward.

Hi @martinhoyer, thank you very much for checking it. Yes, I'm not in the group. When I open link https://github.com/orgs/teemtee/people, such a message in the following is seen:
This organization has no public members.

@happz, please help to  complete the remaining work of this PR, thank you very much!

@idorax You can use fork branches :)

/packit build

/packit build

/packit build

Pushing out to 1.36, there seems to be failing tests as a side-effect of my changes.

/packit build


Nokogiri gem install [help]

What problem are you trying to solve?
Hello, I want to know if all Nokogiri versions require Ruby version >= 2.7, < 3.3.dev. In my gemfile, I specified that I want to install nokogiri 1.13.10, but it still shows me the following error message when I try to run gem install bundler -v 1.17.3 && gem install rails -v 6.1.3
Building native extensions. This could take a while...
ERROR:  Error installing rails:
	The last version of nokogiri (>= 1.8.5) to support your Ruby & RubyGems was 1.13.10. Try installing it with `gem install nokogiri -v 1.13.10` and then running the current command again
	nokogiri requires Ruby version >= 2.7, < 3.3.dev. The current ruby version is <IP_ADDRESS>.
Successfully installed racc-1.6.2
The command '/bin/sh -c gem install bundler -v 1.17.3 && gem install rails -v 6.1.3' returned a non-zero code: 1

Please show your code!
Gemfile
ruby '2.6.6'

gem 'nokogiri', '1.13.10'

Environment
Nokogiri (1.13.10)

Additional context

Sorry you're having problems. I'll try to help.
(In the future, please fill out the  "Installation difficulties" issue template when reporting installation problems. That template prompts you for more information about your system which helps us diagnose problems. Thank you!)
In summary, I'd advise you to upgrade rubygems to the most recent version possible and try again.
The details behind this recommendation:

When you run gem install X you're not running bundler, so your Gemfile is irrelevant, unfortunately. No matter what it says, gem doesn't look at it, so you're experiencing behavior specific to the gem command (and not the bundler command).
Nokogiri v1.13.x supports Ruby 2.6. Nokogiri v1.14.x does not. This error message is likely reporting problems installing Nokogiri v1.14.x with Ruby 2.6.
There have been many rubygems bugfixes over the last few months related to choosing the right gem for your version of Ruby and your architecture. The version you're running is likely not new enough to have these bugfixes. I want to suggest that you run gem update --system and try again.

I'll keep this open for a bit in case you have more questions, but please do let me know if this advice works for you or not.

the issue is fixed now. thank you very much!


Entity config functions

Extend events/actions system with functions, for reusable config.
Add field functions with key/value pairs of function names and action configs.
Function actions can be called from any action by its function name.
Example config:
events: {
    OnSpawn: Call("GreetPlayer"),
},
functions: {
    "GreetPlayer": Echo("Hello Player!"),
},

Don't have functions be defined in separate field, instead just make functions be events.
Config example:
(
    events: {
        // Function definition is simply an event.
        Function("Greet"): Echo("Hello!"),

        // Call function with `Call` action.
        Interval(1000): Call("Greet"),
    },
)

Example usage with custom entity HealthConsumable and Player:
https://github.com/Noah2610/deathfloor/commit/7ab2bba8db1acdb7a368885bdaae7b0d0d36335b


ValueError: No JSON object could be decoded

Hello,
I just installed Djrill and try to send test message:
from django.core.mail import send_mail
send_mail("It works!",
          "This will get sent through Mandrill",
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
          html_message="This will get sent through Mandrill")

The message is sent successfully to user but the code raise the next error:
/home/user/venvpath/local/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/user/venvpath/local/lib/python2.7/site-packages/django/core/mail/__init__.py", line 62, in send_mail
    return mail.send()
  File "/home/user/venvpath/local/lib/python2.7/site-packages/django/core/mail/message.py", line 303, in send
    return self.get_connection(fail_silently).send_messages([self])
  File "/home/user/venvpath/local/lib/python2.7/site-packages/djrill/mail/backends/djrill.py", line 81, in send_messages
    sent = self._send(message)
  File "/home/user/venvpath/local/lib/python2.7/site-packages/djrill/mail/backends/djrill.py", line 153, in _send
    message.mandrill_response = response.json()
  File "/home/user/venvpath/local/lib/python2.7/site-packages/requests/models.py", line 819, in json
    return json.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

Python == 2.7.6
Django == 1.8.2
djrill == 1.4.0
Many thanks for any help.

This can occur if the response from Mandrill is being truncated or altered. I'd guess the problem may be related to the urllib3 warning you're seeing:

InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from  configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.

The link they provide has some suggestions.

Next steps solved the issue:
apt-get install libffi-dev libssl-dev
pip install requests[security] --upgrade

Thanks!

Glad you found a solution.
For anyone else reading this issue because you're seeing an InsecurePlatformWarning when using Djrill, the solution is either:

upgrade to Python 2.7.9 or later
or install the optional requests[security] package extras (and any additional OS support that requires) as in the comment above

More details here.

data = urllib.urlencode(parameters).encode('utf-8')
content = urllib2.urlopen(url=token_url, data=data)
try:
string = content.read()
json_obj = json.loads(string.decode('utf-8'))
print json_obj
except ValueError as a:
print a
else:
print json_obj
any one have idea it gives value error like " No JSON object could be decoded"

@aisayko  i tried this but it give me still error.....

@McaDipali are you having a problem with the Djrill package? The code you posted doesn't look like anything from Djrill.
If your question is not about Djrill, please ask it on StackOverflow.
If you are having problems with Djrill, it might be better to open a new issue. Be sure to say what versions of Djrill, Django, requests, and Python you are using; what specifically you are trying to do that's not working; and include any stack trace from the failure. (Also, please note that Djrill is no longer being supported.)


Gather more metrics on certificate reissue/rotate jobs

Coverage remained the same at 60.529% when pulling c9c782684d1c1d9a27d2d32634be1c2a9ebabfd3 on castrapel:add_metrics_reissue_rotate into 9cfdf55d9ed9d42861dfdc3ed6d2f5fb7780eccd on Netflix:master.


First run results in ModuleNotFoundError: No module named 'seaserv'

Running this command:
docker run --rm -e "SEAFILE_NAME=Seafile" -e "SEAFILE_ADDRESS=seafile.example.com" -e<EMAIL_ADDRESS>-e "SEAFILE_ADMIN_PW=SuperSecretPassword" gronis/seafile

produces the following log:
Python 3.9.2
ldd (Debian GLIBC 2.31-13+deb11u2) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
ls: cannot access '/opt/haiwen/ccnet': No such file or directory
/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Seahub is not running
Done.

/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

seafile server not running yet
Done.
Copying /opt/haiwen/seafile-server-9.0.9/seahub/media/avatars => /seafile/seahub-data/avatars
Dropping /opt/haiwen/seafile-server-9.0.9/seahub/media/avatars
Linking /opt/haiwen/seahub-data => /seafile/seahub-data
Linking /opt/haiwen/seafile-server-9.0.9/seahub/media/avatars => /seafile/seahub-data/avatars
/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Error: there is no conf/ directory.
Have you run setup-seafile.sh before this?

setup_sqlite
/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Checking packages needed by seafile ...

Checking python on this machine ...
Find python: python3

  Checking python module: python-sqlite3 ... Done.

Checking for sqlite3 ...Done.

Checking Done.

This is your config information:

server name:        Seafile
server ip/domain:   seafile.example.com
seafile data dir:   /opt/haiwen/seafile-data
fileserver port:    8082
Generating ccnet configuration in /opt/haiwen/ccnet...

Generating seafile configuration in /opt/haiwen/seafile-data ...

Creating database now, it may take one minute, please wait...

/seafile

Done.

creating seafile-server-latest symbolic link ... done

-----------------------------------------------------------------
Your seafile server configuration has been completed successfully.
-----------------------------------------------------------------

run seafile server:     ./seafile.sh { start | stop | restart }
run seahub  server:     ./seahub.sh  { start <port> | stop | restart <port> }

-----------------------------------------------------------------
If the server is behind a firewall, remember to open these tcp ports:
-----------------------------------------------------------------

port of seafile fileserver:   8082
port of seahub:               8000

When problems occur, refer to

      https://download.seafile.com/published/seafile-manual/home.md

for more information.

/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)

Starting seafile server, please wait ...
** Message: 09:03:36.204: seafile-controller.c(621): No seafevents.

Seafile server started

Done.
/tmp/seafile.env: line 2: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
Traceback (most recent call last):
  File "/opt/haiwen/seafile-server-9.0.9/check_init_admin.py", line 19, in <module>
    from seaserv import ccnet_api
ModuleNotFoundError: No module named 'seaserv'

I've tried using the older 9.0.* versions, but they all throw the same error.

The error is fixed by adjusting PYTHONPATH:
export PYTHONPATH="/opt/haiwen/seafile-server-${VERSION}/seafile/lib/python3/site-packages"

However, another error appears:
Linking /opt/haiwen/ccnet => /seafile/ccnet
Linking /opt/haiwen/conf => /seafile/conf
Linking /opt/haiwen/seafile-data => /seafile/seafile-data
Linking /opt/haiwen/seahub-data => /seafile/seahub-data
ln: failed to create symbolic link '/opt/haiwen/seahub-data/seahub-data': Permission denied

The reason is that /opt/haiwen/seahub-data/seahub-data already exists at the moment of linking. If I add a line to seafile-entrypoint.sh that deletes the directories before linking, the script finishes successfully and Seafile seems to work fine.
My final entrypoint in docker-compose looks like this:
entrypoint: /bin/sh -c '
      export PYTHONPATH="/opt/haiwen/seafile-server-$${VERSION}/seafile/lib/python3/site-packages"
      && sed "256 i rm -rf \$${ARGS[1]}" -i /usr/local/bin/seafile-entrypoint.sh
      && /usr/local/bin/seafile-entrypoint.sh'

Thanks for reporting! I have changed the setup script accordingly:

Only link files if there is no link already.
PYTHON_PATH added


Call Workspace.load_schemes() to load workspace contained schemes

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass
[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[x] I've read the Contribution Guidelines
[x] I've updated the documentation if necessary.

Motivation and Context
Fixes #12113
The xcodeproj library did not see schemes that are in the .xcworkspace (as opposed to a project), so when we get a list of schemes, it doesn't include them.
Description
I PR'd a change to Xcodeproj (https://github.com/CocoaPods/Xcodeproj/issues/557) to add this behavior to load_schemes (it's in 1.5.7) , so we just need to call it in fastlane when we make a workspace object.
I added a .xcworkspace with this kind of setup and added tests to make sure its schemes are read correctly. We already had tests for workspaces without workspace schemes and they still pass.

@DanToml Sorry to bug you -- it looks like tests failed because it's not using xcodeproj 1.5.7 -- when I ran locally, I did a bundle update and it picked it up.  Do I need to do something special in fastlane for it to update?

@loufranco Update the fastlane.gemspec to...
spec.add_dependency('xcodeproj', '>= 1.5.7', '< 2.0.0') # Needed for commit_version_bump action and gym code_signing_mapping

Thanks @loufranco! ✨


PEP8

Gradually cleaning up a lot of the code. Will refactor the variables and classes to follow best practices. Present changes deal with quite a bit.

So the non Raven models is code from another repo that i put there to test
the WPS interface. I'm not sure we should invest time to clean it up. I'm
sorry i did not make that clear earlier. So i suggest you commit your
cleanup, but that we archive these models for the time being.
Le mar. 20 nov. 2018 16:35, Trevor James Smith<EMAIL_ADDRESS>a
écrit :

@Zeitsperre https://github.com/Zeitsperre requested your review on: #50
https://github.com/Ouranosinc/raven/pull/50 PEP8.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/Ouranosinc/raven/pull/50#event-1978535545, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAE9Q6GikYxJBUCUPD_8A8PmLMhPoDk-ks5uxHW5gaJpZM4Yr8Py
.