fairseq stuck during training

Since last fairseq versions, during the training of a transformer_vaswani_wmt_en_de_big the process gets stuck, normally after an OOM batch but not necessarily.
It is reproduceable with pytorch 1.0.1, 1.1.0 and nightly as of today, all with either CUDA 9 or CUDA 10, and the latest master of fairseq (39cd4ce2723d550dd54f6b14b0ed2878e10427f8).

Python version is 3.6. OS is Ubuntu 16.04.2 on one machine and 18.04 in the other one. GPUs are 1080Ti's.
After getting stuck for an while with no new log lines, I CTRL+C it, getting this stack trace:
WARNING: ran out of memory with exception: CUDA out o
f memory. Tried to allocate 354.00 MiB (GPU 0; 10.91 GiB total capacity; 9.27 GiB already allocated; 207.38 MiB free; 913.54 MiB cached);
 Skipping batch

^CTraceback (most recent call last):
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/bin/fairseq-train", line 11, in <module>
    load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq_cli/train.py", line 419, in cli_main
    nprocs=args.distributed_world_size,
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 167, in spawn
    while not spawn_context.join():
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 73, in join
    timeout=timeout,
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

After CTRL+C, I systematically need to manually kill the children processes, which are still occupying GPU memory.
When I run with --ddp-backend no_c10d, the process does not get stuck but crashes with the following stack trace:
WARNING: ran out of memory with exception: CUDA out of memory. Tried to allocate 332.00 MiB (GPU 0; 10.91 GiB total capacity; 9.33 GiB already allocated; 299.38 MiB free; 756.70 MiB cached);
 Skipping batch
                                                                                /mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/multiprocessing/semaphore_tracker.py:143: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
  len(cache))
Traceback (most recent call last):
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/bin/fairseq-train", line 11, in <module>
    load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq_cli/train.py", line 439, in cli_main
    nprocs=args.distributed_world_size,
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 167, in spawn
    while not spawn_context.join():
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 114, in join
    raise Exception(msg)
Exception:

-- Process 0 terminated with the following error:
Traceback (most recent call last):
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq/distributed_utils.py", line 169, in all_gather_list
    result.append(pickle.loads(bytes(out_buffer[2 : size + 2].tolist())))
_pickle.UnpicklingError: pickle data was truncated

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/md0/home/noe/miniconda3/envs/nlp_pytorch/lib/python3.6/site-packages/torch/multiprocessing/spawn.py", line 19, in _wrap
    fn(i, *args)
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq_cli/train.py", line 406, in distributed_main
    main(args, init_distributed=True)
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq_cli/train.py", line 100, in main
    train(args, trainer, task, epoch_itr)
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq_cli/train.py", line 159, in train
    log_output = trainer.train_step(samples)
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq/trainer.py", line 253, in train_step
    [logging_outputs, sample_sizes, ooms, self._prev_grad_norm],
  File "/mnt/md0/home/noe/devel/nmt-word-subword/3party/fairseq/fairseq/distributed_utils.py", line 173, in all_gather_list
    'Unable to unpickle data from other workers. all_gather_list requires all '
Exception: Unable to unpickle data from other workers. all_gather_list requires all workers to enter the function together, so this error usually indicates that the workers have fallen out of sync somehow. Workers can fall out of sync if one of them runs out of memory, or if there are other conditions in your training script that can cause one worker to finish an epoch while other workers are still iterating over their portions of the data.

The last message is clear:
            'Unable to unpickle data from other workers. all_gather_list requires all '
            'workers to enter the function together, so this error usually indicates '
            'that the workers have fallen out of sync somehow. Workers can fall out of '
            'sync if one of them runs out of memory, or if there are other conditions '
            'in your training script that can cause one worker to finish an epoch '
            'while other workers are still iterating over their portions of the data.'

So, if a batch causes OOM then the distributed training is doomed? This wasn't happening a few weeks ago.

I have a similar problem to yours, however when I ctrl+c I get a different error:
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 219, in <module>
    main()
  File "/usr/local/lib/python3.6/dist-packages/torch/distributed/launch.py", line 215, in main
    process.wait()
  File "/usr/lib/python3.6/subprocess.py", line 1477, in wait
    (pid, sts) = self._try_wait(0)
  File "/usr/lib/python3.6/subprocess.py", line 1424, in _try_wait
    (pid, sts) = os.waitpid(self.pid, wait_flags)```

@noe I have also encountered the problems you described above . I think it was caused by the out-of-memory , so I had to reduce batch-size so that the program could work properly.
How can such problem be avoided ?

I also reduce the batch size until I get absolutely no OOM error, so that I can avoid training to hang/crash. Nevertheless, not all OOM seem to be fatal.

We try to catch OOM by skipping the batch, but sometimes it doesn't work (often in the multi GPU case). Usually this causes it to become stuck when the workers are not in sync.

Yes, in trainer.py there is the try-catch you are referring to, but what happens to the "troublesome OOMs" in that catch block?

what happens to the "troublesome OOMs" in that catch block?

If you're using --ddp-backend=c10d then troublesome OOMs can cause hangs. This is because the c10d DistributedDataParallel module communicates gradients during the backward pass, so we can't really recover from an OOM during the backward pass. The no_c10d backend is more robust since it only communicates at the end of the backward pass, but there are still limits to this kind of recovery.
The solution is usually to reduce batch size (and possibly compensate for this with --update-freq).

If you're using --ddp-backend=c10d then troublesome OOMs can cause hangs... The no_c10d backend is more robust since it only communicates at the end of the backward pass, but there are still limits to this kind of recovery.

I'm experiencing a similar issue to this bug. If I change to --ddp-backend=no_c10d, should I expect the same results? (AKA, are models trained with and without c10d equivalent?)

Yes, no_c10d is equivalent, just a slightly more robust DDP backend (and a small amount slower).

Ok - do you also recommend no_c10d on a single GPU?  I'm going to run one GPU with --update-freq 4 -- am trying to avoid the frequent freezes I saw on 2 GPUs.

It's just for distributed training, so it's irrelevant on a single GPU :)

+1. still get stuck in --ddp-backend=no_c10d
^CTraceback (most recent call last):
  File "/usr/local/bin/fairseq-train", line 11, in <module>
    load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
  File "/home/xusong28/project/fairseq/fairseq_cli/train.py", line 441, in cli_main
    distributed_utils.call_main(cfg, main)
  File "/home/xusong28/project/fairseq/fairseq/distributed_utils.py", line 320, in call_main
    cfg.distributed_training.distributed_world_size,
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 200, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 78, in join
    timeout=timeout,
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt

^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

+1. still get stuck in --ddp-backend=no_c10d
^CTraceback (most recent call last):
  File "/usr/local/bin/fairseq-train", line 11, in <module>
    load_entry_point('fairseq', 'console_scripts', 'fairseq-train')()
  File "/home/xusong28/project/fairseq/fairseq_cli/train.py", line 441, in cli_main
    distributed_utils.call_main(cfg, main)
  File "/home/xusong28/project/fairseq/fairseq/distributed_utils.py", line 320, in call_main
    cfg.distributed_training.distributed_world_size,
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 200, in spawn
    return start_processes(fn, args, nprocs, join, daemon, start_method='spawn')
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 158, in start_processes
    while not context.join():
  File "/usr/local/lib/python3.6/dist-packages/torch/multiprocessing/spawn.py", line 78, in join
    timeout=timeout,
  File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
    ready = selector.select(timeout)
  File "/usr/lib/python3.6/selectors.py", line 376, in select
    fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt

^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt

For future reference, I encountered the same issue with PyTorch 1.5.1 and was sure that I don't have any OOM issues (issue persists at batch_size=1). However, upgrading to PyTorch 1.7.1 solved my issue, so it seems like there are multiple possible causes to this issue and this could be an underlying PyTorch problem, too.


Feature/fix set value

setValue was not converting user value to ratio when called

Woops. If you are right, this is because latest BIG changes in code.
Its really time to write tests to avoid that kind of bugs in feature.
I will look at your PR tomorrow (utc+1).
If its critical it will be immediately merged into new version, and published.

I looked into the code (its not easy on phone) and it seems that this bug is present for a very long time.

Your fix was released into npm


[MWPW-139193] - Fix Duplicated Blog Posts

Resolves: MWPW-NUMBER
Description:

There are duplicate URLs/blog posts, this ticket addresses the duplicates.

Test URLs:

Before: https://main--express--adobecom.hlx.page/express/learn/blog
After: https://mwpw-139193--express--adobecom.hlx.page/express/learn/blog

Codecov Report
Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (8d25bcb) 31.12% compared to head (e80a1b5) 31.71%.
Report is 8 commits behind head on stage.

Files
Patch %
Lines

express/blocks/blog-posts/blog-posts.js
20.00%
4 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##            stage     #475      +/-   ##
==========================================
+ Coverage   31.12%   31.71%   +0.59%
==========================================
  Files         123      124       +1
  Lines       23631    23885     +254
==========================================
+ Hits         7354     7576     +222
- Misses      16277    16309      +32

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Use gmake on DragonFly as well

Note: This applies to all other BSDs as well. Hard coding it in this way
should be generally avoided if possible.

Thanks!


[BEAM-6056] Rename vendored grpc relocation to v1p13p1

The current naming scheme for the vendored grpc package relocation uses underscores between the version parts (v1_13_1); this breaks JNI package name parsing, which is used by the default netty SSL provider. When we disabled conscrypt SSL for BEAM-6182, this broke streaming Dataflow pipelines.
This PR changes the naming scheme to use 'p' between identifiers. We also point projects at the project version of vendored grpc rather than the released artifact so that we can test the fix; we should follow-up after releasing a new vendored artifact to point projects at it.
This is a continuation of #7255, trying to fix build issues.
/cc @kennknowles @aaltay

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.
Post-Commit Tests Status (on master branch)

Lang
SDK
Apex
Dataflow
Flink
Gearpump
Samza
Spark

Go

---
---
---
---
---
---

Java

Python

---

---
---
---

Thank you @swegner and @kennknowles !

@swegner it seems that there are some problems with relocating protobuf dependency in RunnerApi - more info in the above-mentioned issue.

@kennknowles when trying to run a python streaming pipeline via docker the grpc is unable to find the module:
[grpc-default-executor-1] ERROR org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService - Encountered Unexpected Exception for Invocation job_d71449f9-6817-4fe5-ba15-a0993cecf604 org.apache.beam.vendor.grpc.v1p13p1.io.grpc.StatusException: NOT_FOUND         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Status.asException(Status.java:534)         at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getInvocation(InMemoryJobService.java:341)         at org.apache.beam.runners.fnexecution.jobsubmission.InMemoryJobService.getStateStream(InMemoryJobService.java:262)         at org.apache.beam.model.jobmanagement.v1.JobServiceGrpc$MethodHandlers.invoke(JobServiceGrpc.java:693)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onHalfClose(Contexts.java:86)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:707)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)         at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748)

@0xmax Can you file an issue to track and discuss this at https://issues.apache.org/jira/projects/BEAM/issues/ ?

@kennknowles FTR I found a related issue already here: https://issues.apache.org/jira/browse/BEAM-6661
same error.


Armenia download died after downloading 2316 files with a generic ["Connection error"] error.

Found during #100 work
The URL - https://armeps.am/ocds/release?limit=100&offset=1519400579553 - opens fine in a browser -  maybe just a temp glitch?

I manually edited the SQLite database and cleared the error and fetch_finished_datetime. I then ran it again, and it picked up fine and is now on file 2329. So it was just an intermittent error.
So this issue could become - how can we stop intermittent errors killing a download entirely?
We already have several retries, but maybe we need more, or more sleep between retries? Or maybe after an operator has examined the problem, a tool so they can easily say "I think that is intermittent, just clear all fetch errors and try again"?

For relatively short-running scrapers, it's probably fine to just start over. For longer-running scrapers, I can see this being a really useful thing to have!

@yolile also commented about this;
I think a (different) approach would require the digestor to extract "item" per item, which then would be processed. Seems like a significant refactor but would enable even more parallelism as well as induce more complexity. Would that merit its own issue branch of course, if it were to be considered...

I think the priority here is making sure that we're able to re-run scrapers and have them just 'fill in the gaps' if there was some kind of issue with the system that they're talking to. From the analyst's perspective, getting the data loaded and being confident that it's a true reflection of what the API/other system is serving are the important things. Not having to babysit too much is also important.
If we can make them faster (eg through more parallelism) for a relatively low cost at the same time, I'm happy for that to be in scope, but not if it's a large job.

For this, I'm seeing a new sub command.
It would take a run that is currently in a fetch stage with some errors, and clear those errors from the meta DB.
The operator would run the new command to clear errors.
They would then run the normal run command again to try again.
ps. parallelism is a different issue

Why a new subcommand and not an option on the run subcommand?
For the use case of, "the run is broken – please just let me reset and start over", I can see the sense of a new subcommand (though, it can also just be an option on the run subcommand). This is separate from other use cases like, "I limited the initial run to 1,000 pages of API results, and I now want to resume from page 1001" or "I had to stop the run for some reason (closed my laptop, lost internet connection, etc.), and I now want to resume where it left off."
The title of this issue is about 'resuming', but I think it's more accurately about 'starting over', in which case a separate 'reset' (or similar) subcommand might be a solution. We should probably rename this issue and split out 'resuming' into a separate issue (I haven't checked if one already exists).


Reject requests to create packages having the name 'default'

Updates to existing packages named "default" are still permitted.
Fixes #1785

pg3/1333

Hmm I think this update logic is incorrect - it would allow me to bypass the reserved names by just doing a PUT force=true. I believe the intention was to allow this only if a package already exists with that name and otherwise to reject it. @mdeuser can you confirm and open a defect if you agree.

@rabbah - yep.  a package update will allow the "default" name for the package.  need to add a check to confirm whether or not the package exists.  hopefully something simple exists :-)

Looks like we can still create a package with the default name if the package is created by wsk action update.

@mdeuser, seems like if we won't let a package to be created named as default we shouldn't let packages with the default name be updated either?

@dubeejw - yep.  @rabbah mentioned the same issue.  I opened https://github.com/apache/incubator-openwhisk/pull/2964 to address it

@dubeejw - if a package exists with a reserved name, i believe the intent is not to block updates to that package.


Fix/allow feature flags to fail gracefully

Changes in this PR
Previously if our calls to the Feature Flag service failed then the call would throw an error and the page could crash. Now we just set the value to false.

@edavey I have tested this locally now and it works as expected. I am not sure how to test it in the cloud env as we don't own the infrastructure Flipt runs on


Added shields.io badges

Added shields.io badges for swiftpackageindex

Why did you remove the Requirements section of the README?

I removed the Requirements because that will now be displayed in the shields.io.


feat: Improve performance

[x] Upgrade major libraries such as antd, react, nextJS& works as expected (see package.json)
[x] Update the design as per the current view
[x] Remove certain (redundant) and unused libraries as react-feather (see package.json)
[x] Check performance increased to ~70%

desktop

mobile

[x] bundle sizes (using analyzer)

QA on Goerli
I was able to mint a component, agent and service.
All items loaded. Search worked on all lists.
Performance is slightly lower than description on desktop:


Unhandled: Not found mavenPOMFile: /home/vsts/work/1/s/pom.xml

Required Information
Question, Bug, or Feature? Bug
Enter Task Name: Maven
https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/MavenV3/task.json
Environment

Server - Azure Pipelines
Agent - Hosted

Issue Description
Some Maven goals do not require a pom.xml. For example, some script task may want to run a java command on a specific JAR file, which can be downloaded from a Maven repository.

Downloads a single artifact transitively from the specified remote repositories.

https://maven.apache.org/plugins-archives/maven-dependency-plugin-2.6/get-mojo.html
azure-pipelines.yml:
variables:
  MAVEN_CACHE: '$(Pipeline.Workspace)/.m2/repository'
  MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE)'
  PLANTUML_VERSION: '1.2022.6'

steps:
- task: Maven@3
  displayName: 'PlantUML Installer'
  inputs:
    goals: 'dependency:get'
    options: |
      -Dartifact='net.sourceforge.plantuml:plantuml:$(PLANTUML_VERSION)'
      -Ddest='$(Build.SourcesDirectory)'
      -DremoteRepositories='https://repo1.maven.org/maven2'
      -Dtransitive=false
    mavenOptions: '$(MAVEN_OPTS)'
- script: java -jar plantuml-$(PLANTUML_VERSION).jar *.puml
  displayName: 'PlantUML Export'

Task logs
==============================================================================
Task         : Maven
Description  : Build, test, and deploy with Apache Maven
Version      : 3.206.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/maven
==============================================================================
##[error]Unhandled: Not found mavenPOMFile: /home/vsts/work/1/s/pom.xml

Troubleshooting
Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting
Error logs
[Insert error from the logs here for a quick overview]

I'm familiar with Maven and POMs, but I have a use case where PlantUML says to call it from the command-line as such.

The most basic way to run it is:
java -jar plantuml.jar file1 file2 file3

https://plantuml.com/command-line
And recommends one way to retrieve the JAR is from the Maven repository.

You can use PlantUML directly with Maven integration.

https://plantuml.com/download
I don't think it's fair to say it's "not recommended" since there are Maven goals that explicitly work without a POM.
https://maven.apache.org/plugins-archives/maven-dependency-plugin-2.6/get-mojo.html
You're recommending to download the file directly from the website or the GitHub Releases, that's fine, but can you recommend a task? I see the cURL task is for upload only.
https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/curl-upload-files?view=azure-devops

this is my pipleline.yaml,    mavenPomFile: '$(system.defaultWorkingDirectory)/scim-web/pom.xml'
Starting: Maven
Task         : Maven
Description  : Build, test, and deploy with Apache Maven
Version      : 3.211.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/maven
##[error]Unhandled: Not found mavenPOMFile: /home/vsts/work/1/s/scim-web/pom.xml
Finishing: Maven
my project sructure:  scim-parent/scim-web/pom.xml , why its prefixing /home/vsts/work/1/s/
1 error(s), 1 warning(s)
| Unhandled: Not found mavenPOMFile:   /home/vsts/work/1/s/scim-web/pom.xml

Maven
Build your Java project and run tests with Apache Maven.
Add steps that analyze code, save build artifacts, deploy, and more:
https://docs.microsoft.com/azure/devops/pipelines/languages/java
trigger:

master

pool:
vmImage: ubuntu-latest
steps:

script: echo $(system.defaultWorkingDirectory)
task: Maven@3
inputs:
mavenPomFile: '$(system.defaultWorkingDirectory)/scim-web/pom.xml'
mavenOptions: '-Xmx3072m'
javaHomeOption: 'JDKVersion'
jdkVersionOption: '1.8'
jdkArchitectureOption: 'x64'
publishJUnitResults: true
testResultsFiles: '**/surefire-reports/TEST-*.xml'
goals: 'package

why its  prefixing /home/vsts/work/1?

Hi @rammaddela,  /home/vsts/work/1s is the default prefix for ms-hosted agents, $(system.defaultWorkingDirectory) just defaults predefined variable, in your case, if pom.xml file exists inside you repo and your "scim-web" inside root you can use task without $(system.defaultWorkingDirectory):
task: Maven@3
inputs:
  mavenPomFile: 'scim-web/pom.xml'
  mavenOptions: '-Xmx3072m'
  javaHomeOption: 'JDKVersion'
  jdkVersionOption: '1.8'
  jdkArchitectureOption: 'x64'
  publishJUnitResults: true
  testResultsFiles: '**/surefire-reports/TEST-*.xml'
  goals: 'package


Performance

The load time for the linter is rather long (typically of the order of 100 ms). This could be because the constructor of the LinterJslint class is quite busy, loading files etc, although this is just an assumption. It's also the case that performance suffers when there are a large number of lint errors, but from what I can tell this is more to do with atom linter than the code that's written.

The default for the new version of JSLint is to report an unlimited number of errors, I've now limited it to 10.


Cannot read property 'active_index' of undefined

Issue description
When I upgrade AmplitudeJS to version 5.3 I get  'Uncaught TypeError: Cannot read property 'active_index' of undefined' in the console when I try to play my playlist. It looks as though it's not loading the playlist. When you hit play, you can see the progress bar is not functional and the Play button is missing.
Previously when using version 5.2 this worked great as far as playing and skipping tracks. But the Repeat and Shuffle don't work. Which is why I was trying to upgrade.
Environment
This is in Wordpress hosted by Kinsta.

What's My Browser Support link: {{ https://www.whatsmybrowser.org/b/YJHK5 }}

Amplitude.js Version:
5.3.1

Steps to reproduce the issue

go to this url: http://cbhdev2.kinsta.cloud/login - login with username<EMAIL_ADDRESS>Password: TempPass392!
Navigate to 'The Healing Room' in the left sidebar menu. and scroll down to the bottom of the page.
Click on the Play button under My Deliverance Prayer List. You'll see the player 'load' in a popup but missing the Play button in the middle there between the forward and backward skips. If you click in that area you'll get the error in the console.

What is expected?
The playlist should be loading dynamically. When the page loads it checks if the user has any saved playlists and inserts the playlists and songs into Amplitude.init(). when the inital Play button is clicked on the page, it loads the player in the popup and when the Play button in the popup is clicked it plays through the playlist.
Link to where issue can be reproduced

http://cbhdev2.kinsta.cloud/the-healing-room/
username<EMAIL_ADDRESS>Password: TempPass392!
Additional details / screenshots

If you look in the console when that page is loaded you will see the object playlist being logged with the Playlist name and the songs.

Thanks! I assigned this to @danpastori, but he's been a bit busy lately. He'll review and get back to you as soon as he can!

Thanks @mcirami for the detailed bug report! I hope to have some time near the end of the week and dive into the issue. I'll let you know what I find.

Hi @danpastori.  I copied everything over to a staging environment so I can revert the 'live' url back to the previous version of amplitude. At least it works for the most part, besides the repeat and shuffle functionality.
So for the error I described on this ticket for the newest version of Amplitude, if you could follow the same instructions but use this url:
https://staging-cbhdev2.kinsta.cloud/
Of course, feel free to check out the live one as well to see the difference if u need to.
Thank you!

Hi @danpastori, just wondering the status of this. Any updates?

Thanks for the follow up @mcirami. Dan's been very busy on some client projects. Hopefully he'll have some free time open up soon!

Also, if it is something where you need direct consulting for your own project, we do have professional support available: https://521dimensions.com/open-source/amplitudejs/docs/get-help/#professional-support

Hi @mcirami ,
I know it's been awhile, but the test links aren't working. If you send over a way for me to test the bug, I can get a quick fix deployed. I have an idea where it's at from the detailed description (thank you) but want to confirm.

Hi @danpastori,
I just tried and didn't have any issues. not sure if there was a problem earlier but if you could give these a try again:
This staging site has the newest version of Amplitude:
https://staging-cbhdev2.kinsta.cloud/login
username<EMAIL_ADDRESS>Password: TempPass392!
and if needed, this is using previous version and you can use same login, same instructions:
https://cbhdev2.kinsta.cloud/login
Let me know if you're still having trouble.
Thank you

Hi @mcirami ,
So I was able to log in, replicate the bug and find a fix. The issue is the same bug referenced here: https://github.com/serversideup/amplitudejs/issues/440. The problem with this bug, is it requires quite a hefty refactor. It's on my list for AmplitudeJS 6.0, but affects the entire library so I will have to do extensive testing.
In the meantime, you can get around the bug by implementing an empty songs array when you call Amplitude.init(). To do that, you'd still set your playlist as is, but make sure your call on init looks like this:
Amplitude.init({
    songs: [{
        url: ''
    }],
    playlists: {
        // RESULT FROM SERVER
    }
});

In earlier versions of AmplitudeJS, I required the songs array and you'd reference those indexes from the playlist. I then made playlists their own feature but didn't account for a "Playlist only" implementation. I really want to make this happen, but for now this is the work around. The bug that you are experiencing is because AmplitudeJS checks to see if there are songs present. If there aren't, it skips initialization. You can init with an empty song and it will reference everything correctly.
I tested this on your player, and everything initialized as it should.
The only other suggestion I have is to make sure when you set the index of your playlist, you remove spaces from the name. I do reference playlists in the code via their object. So instead of My Sample Playlist, it'd be my_sample_playlist as the key. You can then use any meta data field to store the pretty print title for the playlist: https://521dimensions.com/open-source/amplitudejs/docs/configuration/playlists.html.
Hope this helps and let me know if you have any other questions!

Hey @danpastori,
Thanks so much! That worked great! I fixed that and changed the playlist name to use underscores. Still having issues with the repeat and shuffle buttons not working. The classes change when they are clicked to on and off but the functionality doesn't work. Any idea about this?  This is how I'm displaying my controls and the forward and back buttons work as expected:
`

`

Hey @danpastori,
Sorry to bother you again, I know you're busy. But my original issue was never fixed about the repeat and shuffle buttons not working. Doesn't look like they even work on the examples on your website. Wondering if this is something that will be fixed?

On our demo player (https://521dimensions.com/open-source/amplitudejs) I am able to use shuffle and repeat just fine.
Can you post your “What’s My Browser Link”  (https://www.whatsmybrowser.org/) and a link to where you cannot shuffle or repeat?

So if you go here:
https://staging-cbhdev2.kinsta.cloud/login
login with:
username<EMAIL_ADDRESS>Password: TempPass392!
then go to this page:
https://staging-cbhdev2.kinsta.cloud/the-healing-room/
Scroll all the way down to the bottom and you'll see a Play button under 'My Deliverance Prayer List'
That will pop up the player with all the controls.

I will have to leave this to @danpastori for further diagnostics but few things...
To your comment from earlier:

Doesn't look like they even work on the examples on your website.

I cannot replicate this on our demo player at all. We would need your support link from https://www.whatsmybrowser.org/ to diagnose.

Here's the copy from that support link:
https://www.whatsmybrowser.org/b/U9XBT
You're right that player is fine. I think most of the examples do work. The multiple playlist one is the one that I saw the issues with:
https://521dimensions.com/open-source/amplitudejs/docs/examples/multiple-playlists.html
when I set either one to repeat it just stops at the end of the playlist.

Thanks!
And while you are waiting for Dan to get back... did you try upgrading to v5.3.2 just to see if that would fix any issues? I noticed you were running v5.3.1 on your staging site.

just changed it to 5.3.2, no difference.


Fix serialization of 'ocpp_csms_url'

'camelCasing' ocpp_csms_url ends up as ocppCSMSURL.
This is wrong. The spec uses ocppCsmsUrl.
This commit fixes the conversion.

@OSkrk Can you have a look as well?


invalid syntax                                                                                val_Select  = f"\t{r}[$] Please Use The Index Value From The List\n\t\t[+] Not By Your Own :/\n\t\t\t ~ An0n 3xPloiTeR  \n"                                                                                ^                                                                                SyntaxError: invalid syntax

File "SiteBroker.py", line 38
    val_Select  = f"\t{r}[$] Please Use The Index Value From The List\n\t\t[+] Not By Your Own :/\n\t\t\t ~ An0n 3xPloiTeR  \n"
                                                                                ^
SyntaxError: invalid syntax

Hi @Azurai1EZ,
Can you please confirm the version of Python you're using? The script is designed to work on Python >= 3.6

Python 3.8.4

Strange, it works fine for me.
Could you please share the output for the following commands?

Also, if possible create an asciinema recording or a GIF file for me to debug the issue better since I can't reproduce it on my side.

Closing due to no activity.


add a more complex table test for a future bug fix

incorporates a more complete test for problems found by Zoffix

Thank you!
The test itself should be expanded as well. As my attempt at fixing this showed, it's important to check the cell values themselves are all correct too.

On Tuesday, June 28, 2016, Zoffix Znet<EMAIL_ADDRESS>wrote:

Thank you!
The test itself should be expanded as well. As my attempt at fixing this
https://github.com/perl6/doc/pull/614#issuecomment-228729889 showed,
it's important to check the cell values themselves are all correct too.

Absolutely!


form builder Repeater when or hidden not work

Package
filament/forms
Package Version
v2.0.0
Laravel Version
v8.6
Livewire Version
v2.0
PHP Version
PHP 8.1
Bug description

Forms\Components\Repeater::make('field')->schema([
        Forms\Components\Select::make('type')
                                ->options([
                                    'input' => 'input something',
                                    'textarea' => 'text area',
                                ])
                                ->reactive()
                                ->required()
                                ->default('input'),
       Forms\Components\TextInput::make('max')
                                ->when(fn(callable $get) => $get('type') === 'textarea'),
])

The filed max not show when type is textarea
Steps to reproduce
No response
Relevant log output
No response

Hey - someone had issues with this the other day. Can you make sure you're on the latest version of Filament? Also - please swap that == for ===, that weirdly caused someone else issues as well.

Yes, I'm sure I'm on the latest version.
Forms\Components\Repeater::make('field')->schema([
        Forms\Components\Select::make('type')
                                ->options([
                                    'input' => 'input something',
                                    'textarea' => 'text area',
                                ])
                                ->required()
                                ->default('input'),
       Forms\Components\TextInput::make('max')
                                ->when(fn(callable $get) => $get('type') === 'textarea'),
])

that's not work, but
Forms\Components\Repeater::make('field')->schema([
        Forms\Components\Select::make('type')
                                ->options([
                                    'input' => 'input something',
                                    'textarea' => 'text area',
                                ])
                                ->required()
                                ->default('input'), // notice
       Forms\Components\TextInput::make('max')
                                ->when(fn(callable $get) => $get('type') === 'input'), // notice
])

work on fine.
The difference between them is that when the default is displayed then it works fine, and when the default is not displayed then it is not responsive！

I upgrade Filament version problem solved.
But I found some new problem.
When filed is TagsInput, js was an errors
Forms\Components\Repeater::make('field')->schema([
        Forms\Components\Select::make('type')
                                ->options([
                                    'input' => 'input,
                                    'select' => 'select',
                                ])
                                ->reactive(),
       Forms\Components\TagsInput::make('options')
                                ->when(fn(callable $get) => $get('type') === 'select'),
])

Livewire Entangle Error: Livewire property 'data.fields.c06e037c-c966-4f79-9dcd-6d38260dc20f.options' cannot be found

Alpine Expression Error: Cannot read properties of undefined (reading 'length')
Expression: "state.length || true"

module.esm.js:1653
Uncaught TypeError: Cannot read properties of undefined (reading 'length')
    at eval (eval at <anonymous> (module.esm.js:1697), <anonymous>:3:38)

@febelery - See line 4: 'input' => 'input**'**

Fixed by #1008.


 It prints rerun-if-changed=postgresql, not the
actual directory. Interestingly, this causes Cargo to just
unconditionally recompile the crate, regardless of whether anything has
changed. That is undesirable. Instead, let's only recompile if there is
a new migration. That sounds like a good plan.

Interestingly, this causes Cargo to just
unconditionally recompile the crate, regardless of whether anything has
changed.

It's because cargo looks for a file at ./postgresql and when it's not there it assumes it's been deleted: https://github.com/rust-lang/cargo/issues/4213


Results are not consistent in CIS audit output

Describe the Issue
Some checks are not reported because of how goss works - see https://github.com/goss-org/goss/issues/743
Expected Behavior
I'm not sure if there is anything you can do on your side except completely redo some of the checks to avoid overlapping names
Actual Behavior
goss produces output with different checks on different machines.
Control(s) Affected
have not time to go through all but eg. in section 1.4 and 1.6 the checks that are testing same grub.cfg file are overlapping
Environment (please complete the following information):

branch being used: I have seen this issue in eg main and benchmark-v1.0.0
goss v0.3.23

Possible Solution
If we do not want to consolidate the checks (and I dont believe thats a good idea) then only workaround I was able to come up with is to run goss independently for each section(s) we know are overlapping and combine all the results

hi @GhuulCZ
Thank you for taking the time to raise this issue, i believe this has now been addressed for both the devel and the benchmark branches. We have moved to the newer version of goss which enables this tests to take place.
Please let us know if you see anything else that is not as expected.
many thanks
uk-bolly


Error when trying to use Turbo mode. Will there be a fix?

Issue running the program. After running turbo mode this error appears. Seems similar to the person having issues on EUW this is for Na. If the program has an issue will there be a fix for it in the new future? Thanks in advance

having the same error on euw currently, would love help on this.


[Bug]: Receiving error on image.CreateAsync

Testcontainers version
4.0.0
Using the latest Testcontainers version?
Yes
Host OS
Windows
Host arch
x64
.NET version
.NET 9
Docker version
4.36.0

Docker info
Docker.DotNet  3.125.15
Docker.DotNet.X509 3.125.15
[testcontainers.org 00:00:00.09] Connected to Docker:
   Host: npipe://./pipe/docker_engine
   Server Version: 27.3.1
   Kernel Version: <IP_ADDRESS>-microsoft-standard-WSL2
   API Version: 1.47
   Operating System: Docker Desktop
   Total Memory: 15.46 GB
   Labels:
     com.docker.desktop.address=npipe://\\.\pipe\docker_cli
 [testcontainers.org 00:00:00.15] Docker network ecf067c70e51 created
 [testcontainers.org 00:00:00.26] Docker container 46d9852d92dd created
 [testcontainers.org 00:00:00.28] Start Docker container 46d9852d92dd
 [testcontainers.org 00:00:01.57] Wait for Docker container 46d9852d92dd to complete readiness checks
 [testcontainers.org 00:00:01.58] Docker container 46d9852d92dd ready
 [testcontainers.org 00:00:01.65] Docker container e5af65fdf53c created
 [testcontainers.org 00:00:01.66] Start Docker container e5af65fdf53c
 [testcontainers.org 00:00:01.93] Wait for Docker container e5af65fdf53c to complete readiness checks
 [testcontainers.org 00:00:01.94] Execute "/bin/sh -c true && (grep -i ':0*6989' /proc/net/tcp* || nc -vz -w 1 localhost 27017 || /bin/bash -c '</dev/tcp/localhost/27017')" at Docker container e5af65fdf53c
 [testcontainers.org 00:00:02.99] Execute "/bin/sh -c true && (grep -i ':0*6989' /proc/net/tcp* || nc -vz -w 1 localhost 27017 || /bin/bash -c '</dev/tcp/localhost/27017')" at Docker container e5af65fdf53c
 [testcontainers.org 00:00:03.04] Docker container e5af65fdf53c ready
 [testcontainers.org 00:00:03.43] Docker network 8f8159a92a44 created
 [testcontainers.org 00:00:03.49] Docker container 7133ecbde58f created
 [testcontainers.org 00:00:03.49] Start Docker container 7133ecbde58f
 [testcontainers.org 00:00:03.75] Wait for Docker container 7133ecbde58f to complete readiness checks
 [testcontainers.org 00:00:03.76] Execute "/bin/sh -c true && (grep -i ':0*1538' /proc/net/tcp* || nc -vz -w 1 localhost 5432 || /bin/bash -c '</dev/tcp/localhost/5432')" at Docker container 7133ecbde58f
 [testcontainers.org 00:00:04.80] Execute "/bin/sh -c true && (grep -i ':0*1538' /proc/net/tcp* || nc -vz -w 1 localhost 5432 || /bin/bash -c '</dev/tcp/localhost/5432')" at Docker container 7133ecbde58f
 [testcontainers.org 00:00:04.83] Docker container 7133ecbde58f ready
 [testcontainers.org 00:00:04.94] Docker network 7b8e72531597 created
 [testcontainers.org 00:00:04.99] Docker container df8c37b5ede4 created
 [testcontainers.org 00:00:05.00] Start Docker container df8c37b5ede4
 [testcontainers.org 00:00:06.36] Couchbase container is starting, performing configuration.
 [testcontainers.org 00:00:11.64] Wait for Docker container df8c37b5ede4 to complete readiness checks
 [testcontainers.org 00:00:13.75] Docker container df8c37b5ede4 ready
 [testcontainers.org 00:00:13.75] Couchbase container is ready! UI available at http://<IP_ADDRESS>:8091/.
 [testcontainers.org 00:00:13.80] Searching Docker registry credential in CredsStore
 [testcontainers.org 00:00:13.80] Searching Docker registry credential in CredHelpers
 [testcontainers.org 00:00:13.80] Searching Docker registry credential in Auths
 [testcontainers.org 00:00:13.80] Searching Docker registry credential in Auths
 [testcontainers.org 00:00:13.86] Docker registry credential mcr.microsoft.com not found
 [testcontainers.org 00:01:05.41] EOF
 [testcontainers.org 00:01:05.41] Docker image mcr.microsoft.com/dotnet/runtime:9.0 created

What happened?
I upgrade to .net 9
Getting the following error when the code hits for image.CreateAsync()
var location = CommonDirectoryPath.GetSolutionDirectory( "../../" );
        var image = new ImageFromDockerfileBuilder()
            .WithDeleteIfExists( true )
            .WithCleanUp( true )
            .WithName( "couchbase-db-migrations" )
            .WithDockerfile( "samples/Hyperbee.MigrationRunner.Couchbase/Dockerfile" )
            .WithDockerfileDirectory( location.DirectoryPath )
            .Build();

        await image.CreateAsync( CancellationToken.None )
            .ConfigureAwait( false ); //ERR HAPPENS HERE

        return image;

Test method Hyperbee.Migrations.Integration.Tests.MongoDBRunnerTest.Should_Fail_WhenMigrationHasLock threw exception:
Docker.DotNet.DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get "https://mcr.microsoft.com/v2/\": EOF"}
Relevant log output
Test method Hyperbee.Migrations.Integration.Tests.MongoDBRunnerTest.Should_Fail_WhenMigrationHasLock threw exception:
    Docker.DotNet.DockerApiException: Docker API responded with status code=InternalServerError, response={"message":"Get \"https://mcr.microsoft.com/v2/\": EOF"}

Additional information
No response

I'm sorry, but I cannot reproduce the issue. I created a simple Dockerfile using mcr.microsoft.com/dotnet/runtime:9.0 along with .NET 9.0 without any problems. The error message seems like a server error. Could you please try again? Do you have a reproducer?

I see this when I look at the properties for the image variable from this piece of code:

Of course, you will see this. If you are attached to the debugger and try to watch a property, it executes the code associated with the property. In this case, the property delegates to the built image. Since the image hasn't been built yet, you will see the described message. This is not an issue.

Does it still happen? I checked the sources and was able to build the image without any issues.
REPOSITORY                         TAG       IMAGE ID       CREATED         SIZE
couchbase-db-migrations            latest    20fa02074e1d   5 minutes ago   209MB

No, I'm good.  Thank you for helping.

👍 Perhaps you could share some information about how you resolved the issue? It might help others who encounter similar challenges.


Drop yarn classic in favour of pnpm

Motivation
In an effort to reduce the collective time spent by API3 devs waiting for dependencies to install or scripts to run, as well as the amount of drive space required, I've switched from Yarn classic (which is now unmaintained) to pnpm. Pnpm is mature and significantly faster than most other tools. Dependencies are hard-linked which saves on drive space.
I realise that there are now many different projects under the API3 umbrella and this is exactly why I think we should consider migrating to pnpm. I don't particularly want 100 copies of Ethers.js on my machine.
Migrating is as simple as installing pnpm and replacing instances of yarn with pnpm. There are some small differences here and there, but for the most part it's exactly the same API.
Let me know what you think

I think it's unreasonable to ask people to install the package manager we prefer

Not sure I agree with this. It's a pretty common thing to do and pnpm is already roughly about as popular as yarn classic.

trouble with symlinked dependencies

I think that was with Yarn 2? I know a lot of people have had issues with Yarn 2 in the past. For what it's worth, two of the most popular frontend frameworks use pnpm for managing their monorepos (Vue and Next.js)
That being said, if you're not comfortable switching and having some inconsistencies between how API3 projects manage dependencies, then I'm happy to close this PR. There are more important things to focus on than this. We can look at this later.

pnpm is already roughly about as popular as yarn classic

They may be equally popular in terms of active users but I'm referring to it being pre-installed on the user's machine. For example, even if the user normally uses npm, yarn commands work out of the box.

I think that was with Yarn 2

That had to do with Serverless and how the deployment zip was packed. I mainly think that this will need to be updated.

That being said, if you're not comfortable switching and having some inconsistencies between how API3 projects manage dependencies, then I'm happy to close this PR. There are more important things to focus on than this. We can look at this later.

I think we should definitely not have our example projects etc. use pnpm. If you'd rather have us switch over altogether or not at all, then we shouldn't switch. Otherwise, it's a very small amount of extra work for a very small amount of benefit so I don't mind either way.

I think we should definitely not have our example projects etc. use pnpm. If you'd rather have us switch over altogether or not at all, then we shouldn't switch

Yeah this was my main concern - having a mix of yarn & pnpm. I still think we should migrate, but maybe now is not the time.


Extract a fragment from the resource before inserting

If there was a way to insert only a fragment of the intended included resource then @src could reference complete HTML pages rather than just partials. This might be modelled on the  fragment option in jquery-pjax which can be set to a CSS selector.

https://github.com/gustafnk/h-include/pull/11
@shogun70 What do you think?

Did some other PRs that are based on this. Will merge to master.
We can always change the protocol before release, so not a big deal.

@shogun70 Feel free to review and reopen issue if solution is not inline with what you thought.

Improved in https://github.com/gustafnk/h-include/pull/21

Fragment inclusion seems cool, but if it crashes ie9, maybe not.
Other patch looks reasonable. Thanks!

@mnot Ok, I'll send the other patch as a PR and figure out a better way to implement fragment inclusion.


coredns: forward request as it-is when using host's resolver i.e /etc/resolv.conf

Using trust-dns-resolver forces us to read ip , while we don't need to
do it since we should be acting as a proxy. Following PR makes sure that
we forward DNS response as it-is from host's resolver.

Fixes the issue which was discussed where we were not honoring actual
TTL.

@mheon @Luap99 @baude @rhatdan This fixes the TTL issue which was reported yesterday and also removes trust-dns-resolver which is not solving our use-case.

Changes LGTM

@Luap99 @baude Please verify patch and merge me.

/lgtm


Frontend: Build the Product Carousel/Slider component

Issue Summary
The design for Product Carousel/Slider as part of the Product component has been completed and now it needs to be built using our chosen tech stacks. You Product Carousel/Slider has to be inside the Product component story
Figma link
The design for the Product Carousel/Slider can be found here. Please make sure what you build follows the design.
Suggested fix

Build the Product Carousel/Slider component following the design in the Figma file.
Make sure your codes are maintainable and legible.
After coding the component using React and scss, create storybook for the component.
Test it out on your end and make sure it's working properly.

Coding Rules

Let your codes look neat and readable
No unnecessary spacing and always leave one empty line after each code block
Separate each word with a hyphen - in your scss classes, ids.

PR Rules

Link the issue your PR is for by saying Fixes #PRNumber
Add a description of what your PR is about
Add screenshots or video of the working implementation of your PR

pl assign me .

Please work on the other issue assigned to you and create a PR for it before showing interest on other issues
Thanks

Hi @Lovelyfin00, i would like to work on this issue, can you assign it to me ?
Also there is no Product component currently. As of now, the master branch has only Button and Form Component.

Hi , I am contributor under GSSOC'23 and I am eager to showcase my skills and make a meaningful contribution to this project.
Kindly assign me this issue. @Lovelyfin00

Hi @Lovelyfin00 yes, could you assign it on my name, i'll continue working on it

Please put up a PR soon for the issue
I'll assign it to you now

Is this issue available?

Is this issue still available?

there isnt any design for this?


Added TIdentityHashMap

It is a copy of THashMap using identityHashCode and reference equality.
Runtime has been changed to have a better distribution for objects ids returned by identityHashCode.

Using original xorshift approach from Marsaglia paper "Xorshift RNGs" [¹] with seed generated using Math.random. There are more sophisticated ways, don't think they are needed in this case.
Roughly 3x faster than previous approach. Any feedback on that? Tests will come.
[1] http://www.jstatsoft.org/v08/i14/paper

As for performance. Do you have any benchmark that shows performance degradation with incrementing id generator? Does this benchmark show any boost after changing id generation algorithm? What is the performance of xorshift compared to the simple increment?

Added few very specific tests. Regarding raw id generation performance I ran this snippet in Firefox 59 and Chromium 65.

var $rt_lastObjectId = 1;
function $rt_nextIdIncr() {
    var current = $rt_lastObjectId;
    var next = (current + 1) | 0;
    if (next === 0) {
        next = (next + 1) | 0;
    }
    $rt_lastObjectId = next;
    return current;
}

var $rt_seed =<PHONE_NUMBER>;
function $rt_nextId() {
    var x = $rt_seed;
    x ^= x << 13;
    x ^= x >> 17;
    x ^= x << 5;
    $rt_seed = x;
    return x;
}

for (var i = 0; i < 20; i++) {
    var times = 100000000;
    console.time('xorshift')
    for (var j = 0; j < times; j++) {
        $rt_nextId();
    }
    console.timeEnd('xorshift')

    console.time('incr')
    for (var j = 0; j < times; j++) {
        $rt_nextIdIncr();
    }
    console.timeEnd('incr')
}

After warmup I got a median:
firefox (xorshift ~1.6x slower than increment):
xorshift: 444ms
incr: 268ms

chromium (xorshift ~1.8x slower than increment):
xorshift:<PHONE_NUMBER>09375ms
incr:<PHONE_NUMBER>ms

In this content performance is about a balance between raw unique id generation and performance improvement of TIdentityHashMap due to improved distribution of ids.
What do you think?


Any way to run android gradle "install" commands locally?

One side effect of running ./gradlew install{flavor/buildtype/whatever} is that the install task fails with a DeviceException: No connected devices! error (because the install task is running on the remote machine, which isn't running the emulator).
An obvious workaround is to just use an assembe task instead and then manually install/run the APK locally after the Gradle task is finished, but I'm curious if there's a workaround or if you guys have a better solution. Thanks!

Thinking on this more: would it make sense to disable remote execution if the script could detect that (1) a device-specific gradle task is being run (e.g installWhatever vs assembleWhatever) and (2) the build/output APK directory already contained an APK from a previous remote execution command?
That way, you could run something like ./gradlew aSD iSD and let the assembly task run remotely and the install command run locally?

Hey guys, finally I managed to implement this future, if you are still interested please take a look at PR #49.
Your feedback will be very helpful!
@erawhctim @ishaan1995 @MikhailZisman @andrey-bolduzev


fix: Reduce unnecessary interpolation and function calls

Went through the styles and tried to reduce complexity by removing unnecessary interpolation and opting for verbosity rather than mixins (i.e., pfe-print-local) so that sass is easier to parse.
Preview

Link(s) to demo page(s) where this element can be viewed:

Link

Testing instructions

Needs regression testing to ensure no changes.

Browser requirements
Your component should work in all of the following environments:

[ ] Latest 2 versions of Edge
[ ] Internet Explorer 11 (should be useable, not pixel perfect)
[ ] Latest 2 versions of Firefox (one on Mac OS, one of Windows OS)
[ ] Firefox 68 (or latest version for Red Hat Enterprise Linux distribution)
[ ] Latest 2 versions of Chrome (one on Mac OS, one of Windows OS)
[ ] Latest 2 versions of Safari
[ ] Android mobile device (such as the Galaxy S9)
[ ] Apple mobile device (such as the iPhone X)
[ ] Apple tablet device (such as the iPhone Pro)

Ready-for-merge Checklist
Check off items as they are completed.  Feel free to delete items if they are not applicable.

[x] Expected files: all files in this pull request are related to one request or issue (no stragglers or scope-creep).
[ ] Browser testing passed.
[x] Repository compiles and tests pass.
[x] Changelog updated (not needed for documentation updates).

Merging
Please squash when merging and ensure your commit message uses conventional commit formatting.
Be sure to share your updates with the<EMAIL_ADDRESS>mailing list!

On hold pending the base class updates


Add support for Host header

If I want to provide an IP and give a custom Host header (like in cases of hosts behind Cloudflare for example), I have to provide a custom Host header. However, it's passed into the headers argument and is ignored by Go's http lib. See: https://github.com/golang/go/issues/7682
This issue can usually be worked around by adding the IP to your hosts file, but when proxying the requests, this won't help. I have fixed this issue in a way that every Host header passed to -H will be passed as .Host in the requests.

You created the same branch as before. You can see here it's still based off of master:
https://github.com/Vigorwasp/gobuster/network
You need to create a new branch from v3.1-working and apply the patches to it

@FireFart My branch is merged with v3.1-working (which is obviously based off master). I see no conflicts and neither does GitHub's compare tool - https://github.com/OJ/gobuster/compare/v3.1-working...Vigorwasp:bugfix/host-header-and-flags

@FireFart My branch is merged with v3.1-working (which is obviously based off master). I see no conflicts and neither does GitHub's compare tool - v3.1-working...Vigorwasp:bugfix/host-header-and-flags

yeah but github means it can not be merged so there is something wrong

Any chance you can fix the merge conflict @yanir3? ✌ I would love to see this feature!

@n-thumann fixed, but it doesn't seem like it will be merged soon.

@n-thumann fixed, but it doesn't seem like it will be merged soon.

Thank´s for the fast fix 👏 Why is that?

@n-thumann fixed, but it doesn't seem like it will be merged soon.

Thank´s for the fast fix 👏 Why is that?

Last version release was almost a year ago.

I´m running this branch for a while now and it works like a charm 👏
Maybe @FireFart can have a look, now that the merge conflict is solved?


Training data for DIBCO

Could you please point me the right/absolute links to model the DIBCO data? I tried to follow the link DIBCO as suggested but it ends showing me too many resources and most of the resource links seem to only have a small set of test samples. TIA!

Hello.
You can download the DIBCO data from the link.
You can refer to the training and test dataset splitting details in our Arxiv paper.
Thanks.

Thanks much for the quick response and providing me necessary tips.


add SQL exporter to vector

ref https://github.com/neondatabase/neon/pull/5949
close https://github.com/neondatabase/autoscaling/issues/872
This pull request makes the LFC metrics available at the vector metrics endpoint. We use the default scrape interval to avoid overload the database.

Welcome to Alpine!
 ~ This is the VM :) ~
neonvm:~# curl <IP_ADDRESS>:9100/metrics | grep lfc
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 26016  100 26016    0     0  12.1M      0 --:--:-- --:--:-- --:--:-- 24.8M
# HELP lfc_cache_size_limit lfc_cache_size_limit
# TYPE lfc_cache_size_limit gauge
lfc_cache_size_limit<PHONE_NUMBER>2<PHONE_NUMBER>821
# HELP lfc_hits lfc_hits
# TYPE lfc_hits gauge
lfc_hits 0<PHONE_NUMBER>821
# HELP lfc_misses lfc_misses
# TYPE lfc_misses gauge
lfc_misses<PHONE_NUMBER>391821
# HELP lfc_used lfc_used
# TYPE lfc_used gauge
lfc_used<PHONE_NUMBER>391821
# HELP lfc_writes lfc_writes
# TYPE lfc_writes gauge
lfc_writes<PHONE_NUMBER>391821
neonvm:~#

mark ready for review

@skyzh pls also add the working set size in pages to
https://github.com/neondatabase/neon/blob/8dfe3a070cd04dd2310ed07e1f38f4257dd43cd8/vm-image-spec.yaml#L188
select approximate_working_set_size(false);

I'm using the default scrape interval, which is usually 15 secs (1 sec is for host information).

select approximate_working_set_size(false);

will have a separate pull request on the compute side and make it into the next release.

It is not just using resources in compute but also in our victoriametrics database if we scrape so frequently?

These metrics are exclusively consumed by the autoscaler-agent and not persisted anywhere.
The autoscaler-agent fetches metrics (and potentially makes a scaling decision) every 5 seconds.
IMO it's worthwhile to not need to worry about whether those metrics are stale; I don't immediately see a reason not to fetch sql-exporter metrics every 5 seconds (or even every 1 second) - presumably it's pretty cheap?

There are some aggregations and joins on the system catalog, and therefore, if there are a lot of tables, it might be slow to execute.
https://github.com/neondatabase/neon/blob/47d2b3a4830f6d5ecb84086e785ec0f913390176/vm-image-spec.yaml#L162-L172
A better approach is to separate two sql exporters: the ones that are cheap to scrape, and the expensive ones. All LFC metrics are basically O(1) operations when being retrieved from the database.

This pull request will be merged once the compute + console release is done next week. There are still some leftover works from upgrading the compute, and I will ensure that at the time this pull request gets merged, the metrics I put in this vector config will be available from all neon user projects.

I think this can also be implemented by telling the autoscaler-agent another port to fetch metrics from (per each VM).

Do you mean that the autoscaler-agent can directly query LFC data by logging in to the database using Postgres protocol + cloud_admin? In this case, do we need to store that data? I thought the reason that we have vector.dev is to keep some history of the data and avoid storing them by autoscaling-agent itself, but I could be wrong...

Okay I just re-read https://github.com/neondatabase/autoscaling/blob/main/ARCHITECTURE.md and I think I had some misunderstanding there. So I feel the best approach is:

Scrape the data directly using SQL. The autoscaling-agent will connect to the database using Postgres protocol and run SQL queries to collect the data.
Find a way to access these data. As the agent runs outside of the compute pod, it cannot directly log in to cloud_admin. There are 3 approaches: assign a temporary password for cloud_admin; modify ip list so that the agent can connect without a password; create a proxy inside the pod on some port to delegate SQL queries.

create a proxy inside the pod on some port to delegate SQL queries

can we make this into vm-monitor

Having the autoscaler-agent fetch the data via SQL is an interesting idea — hadn't thought of that.
I was thinking more like having it fetch metrics from sql-exporter via prometheus (http). It's already exposed from every VM, so the change would be entirely within the autoscaler-agent.

Having the autoscaler-agent fetch the data via SQL is an interesting idea — hadn't thought of that.

That feels like smashing abstraction layers. Does it yield any practical advantage over below?

I was thinking more like having it fetch metrics from sql-exporter via prometheus (http). It's already exposed from every VM, so the change would be entirely within the autoscaler-agent.

This also, but less so than above. I see now a practical advantage (no bufferization) of bypassing vector. Also, @neondatabase/billing folks say vector is unreliable :slightly_smiling_face: so I guess this is my preference.

I have a demo pull request ready but not tested yet, feel free to leave some comments before we finalize the code + method of exposing the metrics. https://github.com/neondatabase/neon/pull/7302


Typical workflow for GraphMB

Hi,
Thanks for developing this tool.
I want to try your tool in a PacBio data. After reading the document, I am still confused about the preprocessing step. Like Filter and polish assembly and how to generate the assembly graph.
Can you give me more details about how to generate the inputs?
Sincerely
Shaojun

Hi Shaojun,
Filtering and polishing the assembly depends on your data, but it might not be necessary if the error rate of your reads is low. The assembly graph is generated with flye. since you have pacbio data instead of nanopore, you have to change the --nano-raw parameter to one of the --pacbio-* params. Then you can follow step 4 in our pre-processing guide.

Thanks for your reply.
After running flye, what I got is:
assembly_graph.gfa,
assembly_graph.gv,
...
Which one is the assembly should be the input?
Sincerely
Shaojun

Hi, it's the .gfa file. You also need to create a fasta file with the edge sequences if you don't have that already:
awk '/^S/{print ">"$2"\n"$3}' assembly_graph.gfa | fold > edges.fasta
This part was missing from the readme, I'll add it now.

OK. Thanks for your reply!

Hi Shaojun,
to generate that file we used CheckM, as in step 4 of the preprocessing guide:
`mkdir edges
cd edges; cat ../assembly.fasta | awk '{ if (substr($0, 1, 1)==">") {filename=(substr($0,2) ".fa")} print $0 > filename }'; cd ..
find edges/ -name "* *" -type f | rename 's/ /_/g'
evaluate edges
checkm taxonomy_wf -x fa domain Bacteria edges/ checkm_edges/
checkm qa checkm_edges/Bacteria.ms checkm_edges/ -f checkm_edges_polished_results.txt --tab_table -o 2`
the marker_gene_stats.csv file will be in checkm_edges/storage/

OK. I know. Thank you!
Sincerely
Shaojun


Made the Bot Controller look nicer

shouldn't the coords button be some sort of input? we want the bot to travel to a particular set of coordinates right?

shouldn't the coords button be some sort of input? we want the bot to travel to a particular set of coordinates right?

I don't know what @nice6599's plans are for that button


Backup and restore

Hi,
I am using a persistent volume with an Azure disk in an Azure Kubernetes Service AKS cluster.
How do I backup and restore that disk?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d1ac572f-c613-71fc-580a-2805b740b793
Version Independent ID: b4aeef15-7b0f-abc7-872e-28051e10d6ca
Content: Create persistent volumes with Azure Kubernetes Service
Content Source: articles/aks/azure-disks-dynamic-pv.md
Service: container-service
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@spacecat Thanks for the feedback. We are actively investigating and will get back to you soon.

Hi, @spacecat. I don't believe we currently have built-in integration with Azure Backup, but I'll follow up with the engineering team to see if this is something on their roadmap.
As one possible workaround for now since the underlying disk is just a regular Azure managed disk, you can create a snapshot of the disk and use that as a recovery point if needed. First, get the volume name, such as for the PVC named azure-managed-disk:
kubectl get pvc azure-managed-disk

This volume name forms the underlying Azure disk name, so query for the disk ID and provide your PVC volume name, such as:
az disk list --query '[].id|[?contains(@,`<pvcVolume>`)]' -o tsv

With that disk ID, you can then take a snapshot of the disk as follows:
az snapshot create -g myResourceGroup -n pvcSnapshot --source <diskId>

I'll follow back up if there's a cleaner approach and on integration with Azure Backup.
@MicrosoftDocs/azure-cxp-triage Please assign this issue to me for docs follow-up as needed.
#in-progress

Thanks, yes, some managed service for backup and restore would be great.

@spacecat After discussion with the product team, they don't currently have integrations with Azure Backup on their roadmap. The suggested approach is as noted here to create a snapshot of the underlying managed disk. I've created a backlog work item to update the docs to show this approach in case other customers have the same question.
You're welcome to file a feature request on further backup integration as https://github.com/azure/aks/issues. That may help prioritize future planning, and others can comment if they would also like to see this feature.
@MicahMcKittrick-MSFT For now, #please-close

Creating a snapshot seems pretty straightforward by either using the CLI as you explained here or through the portal https://docs.microsoft.com/en-us/azure/virtual-machines/windows/snapshot-copy-managed-disk
Tried to find some info on how to restore my data disk. I can only find info on how to restore a VM OS disk. So I'm wondering how to do the same but for my scenario with a managed Azure data disk for my AKS cluster (best practice). I'm assuming it's basically the same procedure as for restoring a VM OS disk but it would be very helpful if you could maybe share some commands or point me to the relevant article.
Also, I'm thinking consistency. Do I need to stop/dismount my data disk/app before taking the snapshot or what is best practice?

az disk list --query '[].id|[?contains(@,<pvcVolume>)]' -o tsv
did not work for me.
It gives me a:
'[?contains' is not recognized as an internal or external command, operable program or batch file.
I still managed to create the snapshot by just manually searching for the id: az disk list

The commit I was looking for. Docs have been updated with the backup + restore info. Great. Thanks for this.
https://github.com/MicrosoftDocs/azure-docs/commit/5a94058cc317c657d8d0f42550b58bef66d497a4#diff-48570476536f0322a439b7e746d36d28


mypy type checking review, workflow, hook

This PR:

resolves #131
fixes all typing rough edges currently in the code base
adds a mypy workflow & pre-commit hook (willing to remove the hook if we think it's too much)

[re: type checking] – I think we probably only need one or the other in terms of the workflow or pre-commit hook. I'd be in favor of the hook, but I can understand if others think that a bit more tedious.

No mypy in pre-commit pls. Too restrictive and flow disrupting.


Bug When running the example

(winrm work fine for me) but when I try to run the example of winrmcp this error occurred:
out, _ = shell.check_cmd(['ipconfig', '/all'])

File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/winrmcp/client.py", line 41, in check_cmd
return self._check(self.cmd(cmd, *args))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/winrmcp/client.py", line 35, in cmd
command_id = self.protocol.run_command(self.shell_id, cmd, args)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/winrm/protocol.py", line 359, in run_command
res = self.send_message(xmltodict.unparse(req))
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmltodict.py", line 450, in unparse
_emit(key, value, content_handler, full_document=full_document,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmltodict.py", line 407, in _emit
_emit(child_key, child_value, content_handler,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmltodict.py", line 407, in _emit
_emit(child_key, child_value, content_handler,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmltodict.py", line 407, in _emit
_emit(child_key, child_value, content_handler,
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/xmltodict.py", line 412, in _emit
content_handler.characters(cdata)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/xml/sax/saxutils.py", line 212, in characters
content = str(content, self._encoding)
TypeError: decoding to str: need a bytes-like object, list found
(running from mac)

+1 same problem here

The issue is due to the upload_chunks function.
The line below needs to be changed from:
shell.check_cmd(['echo', chunk, '>>', file_path])
to: (removing list), I did this and it worked
shell.check_cmd('echo', chunk, '>>', file_path)


get_kline function isn't returning specified dates/times

import datetime
import time
t_start = datetime.datetime(2021, 1, 1)
t_start = time.mktime(t_start.timetuple())
t_stop = datetime.datetime(2021, 2, 1)
t_stop = time.mktime(t_start.timetuple())
data = marketclient.get_kline('REQ-BTC', '1min', startAt=t_start, endAt=t_stop)
all returned data is the present day - 1500 pieces of data, whether that be 1min or 1day etc...

Yeah I have the exact same problem, except it is 100 items being returned for me. Also the docs shows :param kwargs: [Optional] startAt, endAt, currentPage, pageSize where the paging does not exist.

I am also experiencing this issue. If anyone has any information please update


Instruction copy button doesn't work

Tried both:
remix prod
remix-alpha

the new UI does not include this action anymore


Unable to start generated project on Android

I created a new project using:
motion create --template=motion-game HelloWorld
cd HelloWorld
rake android:device

I get the following error when it tries to run the app on the device (installation succeeded):
E com/yourcompany/helloworld: Exception raised: NoMethodError: undefined method `start' for #<Class:0x100aa6>

I am using motion-game 1.1.4 with Ruby 2.3.1 on Mac OS X 10.11.6
Edit:
The output of rake android:config is
api_version            : "23"
application_class      : nil
archs                  : ["armv5te", "x86"]
assets_dirs            : ["resources"]
build_dir              : "build/android"
features               : []
files                  : ["./app/application.rb", "./app/main_scene.rb", "/Users/me/ruby-2.3.1/lib/ruby/gems/2.3.0/gems/motion-game-1.1.4/lib/motion-game/android/main_activity.rb"]
icon                   : nil
logs_components        : ["com/yourcompany/helloworld:I", "AndroidRuntime:E", "chromium:E", "dalvikvm:E", "Bundle:E", "art:E"]
main_activity          : "MainActivity"
manifest               : {"xmlns:android"=>"http://schemas.android.com/apk/res/android", "package"=>#<Proc:0x007f9853402548@/Library/RubyMotion/lib/motion/project/template/android/config.rb:145 (lambda)>, "android:versionCode"=>#<Proc:0x007f98534024d0@/Library/RubyMotion/lib/motion/project/template/android/config.rb:147 (lambda)>, "android:versionName"=>#<Proc:0x007f9853402458@/Library/RubyMotion/lib/motion/project/template/android/config.rb:148 (lambda)>}
motiondir              : "/Library/RubyMotion"
name                   : "HelloWorld"
ndk_path               : "/Users/me/.rubymotion-android/ndk"
optional_features      : []
package                : "com.yourcompany.helloworld"
permissions            : []
resources_dirs         : []
sdk_path               : "/Users/me/.rubymotion-android/sdk"
services               : []
specs_dir              : "./spec"
sub_activities         : []
support_libraries      : []
target_api_version     : "24"
theme                  : "@android:style/Theme.Material.Light"
version_code           : "1"
version_name           : "1.0"

I have the exact same issue with the default project.  As I believe sparkymat mentioned elsewhere, config/android.rb is missing a line of
app.package = 'com.example'
I corrected this issue, but then ran into the problem described above.

Tried on a few different devices:
Android: 4.4.4 gives a different error:
--------- beginning of main
--------- beginning of system
E/art     ( 8478): Failed writing handshake bytes (-1 of 14): Broken pipe
E/art     ( 8478): Throwing OutOfMemoryError "PushLocalFrame"
E/com/example( 8478): Java exception raised: java.lang.OutOfMemoryError: PushLocalFrame
Android 5 works correctly without errors.
Android 6 gives the original error (though emulation of Android 6 works properly)

Hi,
I am experiencing this issue as well, and it's already building with armv7:
api_version            : "23" application_class      : nil archs                  : ["armv7", "x86"] assets_dirs            : ["./assets", "/Library/RubyMotion/lib/motion/project/template/android/launch_image", "resources"] build_dir              : "build/android" features               : [] files                  : ["./app/application.rb", "./app/game_scene.rb", "/Library/Ruby/Gems/2.0.0/gems/motion-game-1.1.5/lib/motion-game/android/main_activity.rb"] icon                   : nil logs_components        : ["com/yourcompany/flappybird:I", "AndroidRuntime:E", "chromium:E", "dalvikvm:E", "Bundle:E"] main_activity          : "MainActivity" manifest               : {"xmlns:android"=>"http://schemas.android.com/apk/res/android", "package"=>#<Proc:0x007fd92aaa13f0@/Library/RubyMotion/lib/motion/project/template/android/config.rb:145 (lambda)>, "android:versionCode"=>#<Proc:0x007fd92aaa1378@/Library/RubyMotion/lib/motion/project/template/android/config.rb:147 (lambda)>, "android:versionName"=>#<Proc:0x007fd92aaa1300@/Library/RubyMotion/lib/motion/project/template/android/config.rb:148 (lambda)>} motiondir              : "/Library/RubyMotion" name                   : "FlappyBird" ndk_path               : "/Users/TLuong/.rubymotion-android/ndk" optional_features      : [] package                : "com.yourcompany.flappybird" permissions            : [] resources_dirs         : [] sdk_path               : "/Users/TLuong/.rubymotion-android/sdk" services               : [] specs_dir              : "./spec" sub_activities         : [] support_libraries      : [] target_api_version     : "25" theme                  : "@android:style/Theme.Material.Light" version_code           : "1" version_name           : "1.0"
Any thoughts?

@friendlyuis Unfortunately, I can't reproduce this issue.
Can you tell me more information to reproduce ? (ie, your android device name, etc)

@Watson1978 it's a samsung s7 edge, what other information do you want?

@friendlyuis Thank you for helpful information!
I will try it with that device.

@friendlyuis filed your issue in #58


Fix binary search implementation

Code just doesn't "remember" the actual input's lenght
https://github.com/emil14/ts-algorithms/blob/945ce2a2d8b9435f0365c248bc133042647640ee/src/binarySearch.ts#L15

fixed


Revert "fix gitignore"

Reverts lnbits/lnbits#1429

Codecov Report

Merging #1432 (f1ef9a5) into main (b2a9636) will decrease coverage by 0.13%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##             main    #1432      +/-   ##
==========================================
- Coverage   48.20%   48.08%   -0.13%
==========================================
  Files         274      274
  Lines       16393    16393
==========================================
- Hits         7903     7883      -20
- Misses       8490     8510      +20

Impacted Files
Coverage Δ

lnbits/extensions/lnurlp/migrations.py
81.48% <0.00%> (-14.82%)
:arrow_down:

lnbits/db.py
81.20% <0.00%> (-10.07%)
:arrow_down:

lnbits/commands.py
61.40% <0.00%> (-1.76%)
:arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Add support for literals

Currently utils [not shipped] provides a hack to be able to express floats, ints and some strings, but it's clumsy and unacceptable.


Create faster-rcnn-resnet101-coco-sparse-60-0001.json

Hi Hemanth,
Do you have any descriptions for this request?
Mark Liu


RipMe 1.6.9 not downloading all images from reddit user page

Ripme version: 1.6.9
Java version:  (build 1.8.0_151-b12)
Operating system:  Win 10 Pro
Exact URL you were trying to rip when the problem occurred: https://www.reddit.com/user/venusfoxx
Please include any additional information about how to reproduce the problem: Click "rip"

Expected Behavior
Rips all images by user as displayed/linked on user page
Actual Behavior
Skips considerable images and gifycat links

Can you provide logs and more information about which images were skipped that you think should be present? My rip grabbed 236 items, and the user has posted 279 links (many of which are cross-post/duplicate). https://www.reddit.com/user/VenusFoxx/submitted/
Logs: https://pastebin.com/FF7eKFH8

Not OP, but issue persists in 1.6.10. I've noticed it misses gfycat links that start with https://gfycat.com/gifs/detail/ which seems to be the new default share link when creating a gfy. It also misses erome links.

@mne41906 Thanks!

https://github.com/RipMeApp/ripme/issues/90

Sorry it took so long to reply.
While I do get a lot of the users posted images when scraping their user page - some gifycat links do not rip and seem to be ignored. I don't see references to the ones I would expect to see.
 This issue was a duplicate of #90 so we're tracking there now.


Request for resource defs, mediatypes, and doc browser to support an 'internal' directive

In some cases, a resource may be internal and we do not wish it to show up in the generated documentation. It would be nice if we could add an internal directive to the DSL (or something) to flag certain resources/mediatypes as internal.
Some additional considerations off the top of my head:

Do we still generate docs for this in the json files and just filter those resources in the doc browser?
Do we provide a doc browser generation option for whether to include internal resources?

@justingaylor can we please consider updating this issue desc to also include a way for us to mark certain actions as internal? I described uses cases in https://github.com/rightscale/praxis/issues/90 - will close #90 as this issue captures the requirements

@alikhajeh1 Updated, sir.


Full Path in Title Bar

This is a feature request.
I think that it would be useful, if not necessary to have the full path displayed in the title bar of the application in the situation where two or more files of the same basename are open. Comparing two files, it is very confusing as to which file I am looking at any given time. I attached a file of the behavior in question.
I could probably submit a PR at some point, but right now I have my hands full with other things so it would take a while. I would welcome others submitting a PR if this is deemed important.

I tested with TextEdit and it works the same. Since this appears to be default Cocoa NSDocument behavior, I'd prefer to keep it as it is.


Fix a lot locale strings

add functions using
fix identity for meat spike and creaming

:cl:

fix: Fix identity reveal for creaming and meat spike popups.

fixed checks and uploaded media


Confirm before delete

Hey shacker, incredible work on this lightweight but powerful todo app! Thank you.
I had one suggestion for future improvements, which is to add a confirm before deleting the task. Keep it up, really appreciate the work.

Good suggestion @dalinxz . I've retitled this issue and marked it "Good first issue" for anyone who drives by and wants to contribute the feature. Thanks.

Hi @dalinxz , I would like to work on this, if the issue is still open.

Sure thing @sum35h - this still needs to be done, and I welcome your pull request any time. Thanks in advance.

@shacker can i take up this issue

@shacker can i take up this issue

@sum35h Have you started on this? If not, do you mind of @aaroosh-07 works on it?

@sum35h Have you started on this? If not, do you mind of @aaroosh-07 works on it?

@shacker I want to fix this issue. Assign it to me

@shacker I want to fix this issue. Assign it to me

@1hanzla100 I don't assign things here - most OSS projects don't. Either communicate with others in this thread, or just be first with a good pull request!

@1hanzla100 I don't assign things here - most OSS projects don't. Either communicate with others in this thread, or just be first with a good pull request!

This improvement is already done. -Shouldn't this be marked as closed?
Great project. Thanks

Thanks @EdgarPrada, right you are. Closing. Glad you enjoy the project!


feat: update lemongrass hardfork to remove CIP 12 relayer incentivized middleware

Overview
This is a brief update the remove a CIP from the lemongrass upgrade. This still needs discussion before merging.
Checklist

[x] New and updated code has appropriate documentation
[x] New and updated code has new and/or updated testing
[x] Required CI checks are passing
[x] Visual proof for any user facing features like CLI or documentation updates
[x] Linked issues closed with keywords

https://github.com/celestiaorg/CIPs/pull/106#issuecomment-2010359529

closing this PR for now as its this is not the best place to discuss the inclusion of CIP12


On screen DPI

For some reason DPI on screen are different from original file, so on mobile at the moment I am viewing a low res.
I saw that in pdfjsWrapper I can change DPI but only for the printer output...how could I change the on screen one?
Thanks

Oh just found out that overwrting var scale (in my case to 1.7) in the this.renderPage function works, not sure if it's the best way though


Is there a support German, Spanish, French TTS(Text to speech)

Describe the issue
Is there a support German, Spanish, French TTS(Text to speech)
Feature area
Is there a support German, Spanish, French TTS(Text to speech)
Existing doc link
https://docs.microsoft.com/en-us/dotnet/api/microsoft.mixedreality.toolkit.audio.texttospeech?view=mixed-reality-toolkit-unity-2020-dotnet-2.7.0
Additional context
Currently there are only 3 English voices, how can we select a German or Spanish voice?

Duplicate of https://github.com/microsoft/MixedRealityToolkit-Unity/issues/10395


Add small offset to timestamp before fetch

We expect the oracle to attest to the price at timestamp but due to spacetime distortion we give it another minute before we fetch. This should reduce the error below because we will only fetch the attestation if it is very likely to have occurred.
{"timestamp":"2022-05-10T07:00:03.759931111Z","level":"DEBUG","fields":{"message":"Failed to fetch attestation: Failed to deserialize as Attestation: error decoding response body: attestation missing: attestation missing"},"target":"daemon::oracle"}

Resolves #1966
What do you think @luckysori ?

bors r+


Users may want that listening never stops unless they stop it by hand or command

We could add a config which, if set to yes restarts the recognition automatically when it would end by timeout. Then it would only stop if it's stopped by hand via button-click or keyboard shortcut or via voice-command stop listening

I've already implemented this :+1:

I was thinking of a similar idea. Maybe just some sort of selection that does it?
Was also wondering if you could share some code to get the current language of TW (not the plugin), TW itself.

@flancast90 , you can always get the text of a tiddler by $tw.wiki.getTiddlerText("title of tiddler");
Then it's a question in which tiddler tiddlywiki stores the language


Compose button broken on 6.900

Checklist

[X] I have used the search function to see if someone else has already submitted the same bug report.
[X] I will describe the problem with as much detail as possible.

App version
6.900
Where did you get the app from?
Other
Android version
LineageOS 21 based on Android 14
Device model
Oneplus 8T kb2003
Steps to reproduce

Open app
Press compose button
K9-Mail crashes

Expected behavior
You should be able to compose an email.
Actual behavior
It started crashing.
Logs
k9mail-logs.txt

Duplicate of #7842


Allow empty deprecation macros to be passed as macro arguments

The OSSL_DEPRECATEDIN_3_0 macro introduced in PR #13074 is intended to
be passed as a parameter to the various PEM declaration macros. However,
in some cases OSSL_DEPRECATEDIN_3_0 is defined to be empty, and it is
not allowed to pass empty macro arguments in C90. Therefore we ensure
these macros are always defined. In the case where they were empty
previously we use a no-op value instead.

For an example of how this can go wrong, see:
https://travis-ci.com/github/openssl/openssl/jobs/403268332

I can see why... It has a drawback, that we won't be able to use the attribute with types, should we ever decide we want to...
I believe it's possible to have a no-attributes attribute with gcc (i.e. __attribute__ (())) but haven't experimented with __declspec enough to know if MSVC supports something similar. If possible, that would be a better no-op...

I believe it's possible to have a no-attributes attribute with gcc (i.e. attribute (())) but haven't experimented with __declspec enough to know if MSVC supports something similar. If possible, that would be a better no-op...

I don't think that helps though. What do we do with a compiler where we don't know how to do deprecations?

@mattcaswell @levitte I can test a branch with WIP fixes on it using c99 on our "exotic" platform ;)

I don't think that helps though. What do we do with a compiler where we don't know how to do deprecations?

Argh, you're right...
Please amend the comment to say that the macro is unsuitable to use with typedef, and that will have to be good enough for now.

Perhaps at the time we decide to start marking deprecation of types, we will also start requiring C99. :grin:

Please amend the comment to say that the macro is unsuitable to use with typedef, and that will have to be good enough for now.

Done. Fixup pushed.

Not seeing any issues with PR on NonStop

Not seeing any issues with PR on NonStop

Good to know. Thanks.

What does it mean to deprecate a type?  Why isn't wrapping it in the ifndef guard enough, since you already have to do that in header files anyway?

What does it mean to deprecate a type?

In GCC terms, this:
__attribute__ ((deprecated)) typedef rsa_st RSA;

We currently don't do this, no one has really thought of that possibility...  well, before now.

Thanks for the explanation about deprecating types, @levitte !

24 hours has passed since 'approval: done' was set, but as this PR has been updated in that time the label 'approval: ready to merge' is not being automatically set.  Please review the updates and set the label manually.

It just dawned on me that, while perhaps less elegant, and alternative would be this:
DECLARE_PEM_rw_attr(extern OSSL_DEPRECATEDIN_3_0, DHparams, DH)

That means that the argument will never be empty, even if the definition of OSSL_DEPRECATEDIN_3_0 is

I'm not sure if that is neater or not.  It's a nice thought.

I'm not sure if that is neater or not

Depends on from which angle you view it, I guess...

It just dawned on me that, while perhaps less elegant, and alternative would be this:

I had considered that option, but thought the approach in this PR was better.

I had considered that option, but thought the approach in this PR was better.

I was asked to expand on this a bit more.

It just dawned on me that, while perhaps less elegant, and alternative would be this:

It does indeed seem less elegant - basically requiring us to repeat the "extern" bit everywhere we need to do this. This code duplication seems undesirable since:

on most platforms we don't need it anyway
it is less than clear to readers of the code why it is necessary at all (and do we really want to add a comment explaining it everywhere)
we can just incorporate it into the definition of OSSL_DEPRECATEDIN_3_0 as proposed in this PR

Pushed. Thanks.


fix: 🐛 Transfer Ownership method

Pads the ticker that is passed to the token.transferOwnership method to
prevent an incorrect length error from being thrown

:tada: This PR is included in version 12.0.0-beta.2 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 13.0.0-beta.1 :tada:
The release is available on:

npm package (@beta dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 13.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[5.2] Fix for deprecated ValidationException

In the throwValidationException function in the Illuminate\Foundation\Validation\ValidatesRequests class the ValidationException thrown is a referral to a deprecated class.

This needs reverting.

We throw the original one for BC. This broke everything.

Is this not just a doc block change?

Ah I see because of the import.


OKE-27716: Duplicate LB Bug

Added additional checks for lb exceptions

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

10 of 10 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 57.238%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
2143

Relevant Lines:
3744

💛  - Coveralls


Elliot wood/fix azure cognitive search index

Update to AzureCognitiveSearchUploadStager based on feedback. For AzureCognitiveSearch we desire the index to reuse items in the index, and update them if there is a change, therefore we will keep the element_id as the key for the index instead of a unique id each time which adds duplicate entries.
def conform_dict(data: dict) -> dict:
    """
    updates the dictionary that is from each Element being converted into a dict/json
    into a dictionary that conforms to the schema expected by the
    Azure Cognitive Search index
    """

    # https://docs.unstructured.io/open-source/concepts/document-elements#element-id
    # By default, the element ID is a SHA-256 hash of the element’s text, its position on the page, page number it’s on, and the name of the document file - this is to ensure that the ID is deterministic and unique at the document level.
    # To obtain globally unique IDs in the output (UUIDs), you can pass unique_element_ids=True into any of the partition functions. This can be helpful if you’d like to use the IDs as a primary key in a database, for example.
    # For AzureCognitiveSearch we desire the index to reuse items in the index, and update them if there is a change, therefore we will keep the element_id as the key for the index instead of a unique id each time which adds duplicate entries.
    data["id"] = data.get("element_id")
    #data["id"] = str(uuid.uuid4()) // Consider using this if unique_element_ids=True

Ready for review - I copied v2 over v1 by accident but now fixed PR.
This PR contains a fix to the Azure Search Index for reuse of index elements rather than adding new element on each run causing many duplciates.

The issue with this approach is that the element id is not unique. Meaning there might be two elements in the same document with the same id, and this would cause one of them to be lost when uploading.

Oh interesting I had used the assumption based on this line from doco "the element ID is a SHA-256 hash of the element’s text, its position on the page, page number it’s on, and the name of the document file - this is to ensure that the ID is deterministic and unique at the document level".
Can I confirm that it won't be unique at the document level?

I actually have an open PR to solve this exact issue for pinecone by persisting an id associated with the original document: PR219. Until we have a more deterministic approach for elements, this would alleviate duplicates if the same document gets processed. This approach can be extended to azure cognitive search and any other databases that store element-level records.

Can anyone please reivew this branch and accept on the basis that elementID is deterministic and unique at the document level.
I have tested this and confirm it works as intended.

@ElliotWood can we close this PR. Are your findings fixed. Also ACS was renamed to Azure AI Search.

Closed because solved with record_id fix