500+ multi-labels only predicts zeros

This might not be an issue related to fast-bert, but I give it a shot here either way. I now have a dataset of 500+ labels. At first, fast-bert predicts various values between 0-1 for every label which seems fine, but the more I train it the more it predicts only zeros for everything. Logically, it seems wise as only 1/500 is a positive label while the rest are zeros. Is there a way to fix this? Can I change the loss function somehow? Possibly introduce class weights to really penalize false-negatives?

You could use the pos_weight argument to the loss function (BCEWithLogitsLoss or BCELoss) to give greater weight to to positive instances than to negative instances, increasing recall at the expense of precision.

Ah that's great! Although pos_weight seems to be a fixed vector you set from the start to boost up certain label weights if you have an imbalanced dataset. What I want is an adaptive pos_weight that heavily penalizes the loss as soon as fast-bert predicts a 0 instead of a 1 for a label. I don't know if that's possible though.

pos_weight differs from regular class weights, and I think that it should do exactly what you want it to do. You have very sparse labels where for each sample the label mostly contains 0s, and one (or a few?) 1s, so naturally the model learns to guess 0 all the time, getting good (binary) accuracy but poor recall. pos_weight will weight up the loss so that predictions in the label whose ground truth equals one count more than those that equal 0 - so that that the model would care more about getting the few positive labels right, then getting the many negative labels right.
Finding the correct value for pos_weight might require some experimentation, though. If in addition to multilabel sparsity you also have class imbalance (some classes are equal 1 more often than others) than it's worth reflecting this in the pos_weight vector. But it's still a question by how much the positive values should be up-weighted.
BTW, I had a similar problem and pos_weight was able to solve it for a reduced problem set (if I only used 200 labels instead of 600+) but was unable yet to find a good values that make working with 600+ labels work. Let me know if you manage this.

Oh I see! My bad for misunderstanding, very grateful for your help! You're precisely right on what I'm trying to achieve. I'm not quite sure how I would set up this pos_weight parameter though. Is it a vector? How do I calculate it? I'm still a bit confused even after googling up on it.

It's a vector the size of your label, but can also be a scalar if you want the same positive weight for all categories in the label (scalar should simply be broadcast in the calculation). I've seen others use a default value of 10.0 (has to be float), log(num_categories), or number_neg_samples / number_pos_samples (for a vector of varying weights, if you also have class imbalance). It's worth playing around with.

Ah that makes sense. Hmm doesn't seem to work as I notice no difference in the loss when I add the pos_weight parameter, as such: BCEWithLogitsLoss(pos_weight=torch.tensor(np.ones(670)*10.0)). This line does run, as I get errors if I provide another label length other than 670, it's just that nothing seems to happen. Any ideas what could be wrong?
Example of the data:
"Spare Parts for Commissioning and start up each item", 0,0, ... ,1, ... ,0,0 (670 labels)
"100% Project Specification HS84218-00-PIP-SPE-0007-", 0,0, ... ,1, ... ,0,0 (670 labels)
...

Printout of the training process:
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 32768.0
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 16384.0-------------------------------| 0.15% [1/689 00:00<08:33]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 8192.0--------------------------------| 0.29% [2/689 00:01<07:53]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 4096.0--------------------------------| 0.44% [3/689 00:02<07:39]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 2048.0--------------------------------| 0.58% [4/689 00:02<07:31]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 1024.0--------------------------------| 0.73% [5/689 00:03<07:27]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 512.0---------------------------------| 0.87% [6/689 00:03<07:24]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 256.0---------------------------------| 1.02% [7/689 00:04<07:22]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 128.0---------------------------------| 1.16% [8/689 00:05<07:20]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 64.0----------------------------------| 1.31% [9/689 00:05<07:19]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 32.0----------------------------------| 1.45% [10/689 00:06<07:17]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 16.0----------------------------------| 1.60% [11/689 00:07<07:16]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 8.0-----------------------------------| 1.74% [12/689 00:07<07:15]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 4.0-----------------------------------| 1.89% [13/689 00:08<07:14]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 2.0-----------------------------------| 2.03% [14/689 00:08<07:13]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 1.0-----------------------------------| 2.18% [15/689 00:09<07:12]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.5-----------------------------------| 2.32% [16/689 00:10<07:11]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.25----------------------------------| 2.47% [17/689 00:10<07:10]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.125---------------------------------| 2.61% [18/689 00:11<07:09]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.0625--------------------------------| 2.76% [19/689 00:12<07:08]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.03125-------------------------------| 2.90% [20/689 00:12<07:08]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.015625------------------------------| 3.05% [21/689 00:13<07:07]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.0078125-----------------------------| 3.19% [22/689 00:14<07:06]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.00390625----------------------------| 3.34% [23/689 00:14<07:05]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.001953125---------------------------| 3.48% [24/689 00:15<07:04]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.0009765625--------------------------| 3.63% [25/689 00:15<07:04]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.00048828125-------------------------| 3.77% [26/689 00:16<07:03]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.000244140625------------------------| 3.92% [27/689 00:17<07:02]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 0.0001220703125-----------------------| 4.06% [28/689 00:17<07:02]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 6.103515625e-05-----------------------| 4.21% [29/689 00:18<07:01]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 3.0517578125e-05----------------------| 4.35% [30/689 00:19<07:00]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 1.52587890625e-05---------------------| 4.50% [31/689 00:19<06:59]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 7.62939453125e-06---------------------| 4.64% [32/689 00:20<06:59]
Gradient overflow.  Skipping step, loss scaler 0 reducing loss scale to 3.814697265625e-06--------------------| 4.79% [33/689 00:21<06:58]

This loss continues to dive pretty fast down to 10^-200, which seems really strange. Maybe it's something else I'm missing?

I'm not sure, but it could very well be that 10 isn't a significant enough weight for the positive values since 0s outnumber 1s so much in your data. I suggest trying different, higher values. If this experimentation is too computationally costly perhaps try to overfit a smaller sample of the training data. PS instead of log(num_categories) I should have written sqrt(num_categories), which in your case is substantially higher than the former. But if I understand your data correctly number_neg_samples / number_pos_samples should produce even higher weights.

I am observing very similar issues. Multi-label classification with 5 labels. Each data point has a minimum of 1 label and maximum of 2 or 3 labels (encoded as 1s). It is not that sparse like 600+ labels in the case of @irdeen. I see that the predictor acts as it is binary. It returns two classes with high probability and rest are rather close to zero regardless of the number of  labels. The label distribution is skewed too (two labels 500 samples, others are around 80), so I have a imbalance dataset as @guy-mor mentioned.
The question is about the pos_weight argument. Do we have this pos_weight argument to the loss function (BCEWithLogitsLoss or BCELoss) exposed directly in the learner  object in fast-bert? Or you guys are talking modifying the loss function in the source code? There are no documentations and the conversations above were not clear.
And would you @irdeen explain the BCEWithLogitsLoss(pos_weight=torch.tensor(np.ones(670)*10.0))  in your case. You have 670 labels. Doesn't torch.tensor(np.ones(670)*10.0) construct a tensor of size 670 with all 10.0 values? How does this work as a pos_weight? All your labels are equally weighted, aren't they? I thought the point is to weigh according to their counts or distributions. Your feedback is appreciated.


Pin django version in tox docs

Coverage remained the same at 95.301% when pulling a66371fc92c13074ebb701817d48ba7b6d2071e3 on hotfix/rtd into 24a77632deeb8880efa9fb50e670c4dccf29aeb3 on develop.


Fatal deepcopy error on operator-sdk add api

While following the getting started guide (created a memcached operator), and reaching the part of the guide that stated add a new custom-resource-definition, I issued the command operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=Memcached
Some files have changed but eventually the command failed like so:
memcached-operator (master) $ operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=Memcached
INFO[0000] Generating api version cache.example.com/v1alpha1 for kind Memcached.
INFO[0000] Created pkg/apis/cache/group.go
INFO[0000] Created pkg/apis/cache/v1alpha1/memcached_types.go
INFO[0000] Created pkg/apis/addtoscheme_cache_v1alpha1.go
INFO[0000] Created pkg/apis/cache/v1alpha1/register.go
INFO[0000] Created pkg/apis/cache/v1alpha1/doc.go
INFO[0000] Created deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml
INFO[0000] Created deploy/crds/cache.example.com_memcacheds_crd.yaml
INFO[0000] Running deepcopy code-generation for Custom Resource group versions: [cache:[v1alpha1], ]
F1227 15:01:21.313323    2077 deepcopy.go:885] Hit an unsupported type invalid type for invalid type, from ./pkg/apis/cache/v1alpha1.Memcached

I'm not sure if I did something wrong but I followed the tutorial to the letter and tried recreating the project from scratch just in case, but ran into the same issue. Additionally, I could not find any similar case while googling the error
Versions:
$ operator-sdk version
operator-sdk version: "v0.13.0", commit: "1af9c95bb51420c55a7f7f2b7fabebda24451276", go version: "go1.13.3 linux/amd64"

$ go version
go version go1.13.5 linux/amd64

$ git version
git version 2.17.1

Okay, so about 60 seconds after posting this, a very helpful person on the kubernetes slack pointed me to this issue.
tl;dr: the environment variable GOROOT should have been set like so
export GOROOT=$(go env GOROOT)

@tomklino I assume you were able to resolve your issue using the suggested workaround. If not, feel free to re-open this issue.


Minister issue action mandatory fields not consistent on when turning red.

Describe the bug
Minister issue action mandatory fields not consistent on when turning red.

Expected Behaviour
They should all behave the same.
Actual Behaviour
typing in description nothing turned red.  typing in objective it turned red and stayed red even after leaving the field.
To Reproduce
Steps to reproduce the behavior:

Make a rangeland health action
type in description
type in objective
both fields are required, one is red one is not.

Screenshots
see above
Additional context

Can't reproduce with the latest changes in dev:

I'll check the pipeline

It LGTM.  I'll leave it open until someone else (who has also cleared their cache) runs through it in Test and sees it work like your screen.


Request for versioning/history from Educators

Request from Educator on Forum:
Dear MakeCode Arcade Developers,
High School Computer Science Teachers would like to have a File History Feature like Replit File History (Replit File History) so we can verify projects were created by our students without copying and pasting code elsewhere which could be potential plagiarism.
Sincerely,
High School Teachers who teach CS
https://forum.makecode.com/t/dear-makecode-arcade-developers-high-school-wants-a-file-history-feature/21868

Version history is in the /beta but can't be recovered from the hex file or the share link.


[suggestion] Added ability to work with FtbChunk to delete unprotected chunks

A suggestion，Added ability to work with FtbChunk to delete unprotected chunks.
https://www.curseforge.com/minecraft/mc-mods/ftb-chunks-forge

whoops, sorry. i clicked on the wrong issue


oidentd does not work with znc

Hi,
My configuration in /etc/oidentd.conf is:
`default {
    default {
            allow spoof
            deny spoof_all
            deny spoof_privport
            allow random
            allow random_numeric
            allow numeric
            allow hide
    }

} `
My file .oidentd.conf location is ~/.oidentd.conf and permission is -rw-r--r-- (644) ans
My $HOME direction permission is drwxr-xr-x
The configuration os setfile is the same ~/.oidentd.conf and setformat is global { reply "%user%" }
I have this same configuration in other vps and work. I do not know what is a problem. Cloud you help me?

Per https://wiki.znc.in/Identfile#Using_identfile_with_oidentd the /etc/oidentd.conf needs to contain
user "znc" {     default {         allow spoof         allow spoof_all     } }
as the default {} block is insufficient by itself.

Per https://wiki.znc.in/Identfile#Using_identfile_with_oidentd the /etc/oidentd.conf needs to contain
user "znc" {     default {         allow spoof         allow spoof_all     } }
as the default {} block is insufficient by itself.

I appreciate your help.
I had already switched to this configuration that you suggested, but it didn't work. But I changed now again, it didn't work again. I want to mention that my znc is running with my username and not with "znc" username. The configuration that i put now, was
user "myuser" {     default {         allow spoof         allow spoof_all     } }
default {         default {                 allow spoof                 deny spoof_all                 deny spoof_privport                 allow random                 allow random_numeric                 allow numeric                 allow hide         } }
but does not work.

I appreciate your help.
I had already switched to this configuration that you suggested, but it didn't work. But I changed now again, it didn't work again. I want to mention that my znc is running with my username and not with "znc" username. The configuration that i put now, was
user "myuser" {     default {         allow spoof         allow spoof_all     } }
default {         default {                 allow spoof                 deny spoof_all                 deny spoof_privport                 allow random                 allow random_numeric                 allow numeric                 allow hide         } }
but does not work.

The oidentd daemon needs to be restarted when you edit the oidentd.conf file, and you need to make sure port 113 is accessible remotely.

The oidentd daemon needs to be restarted when you edit the oidentd.conf file, and you need to make sure port 113 is accessible remotely.

Yes,
I restaed  daemon oidentd with command sudo systemctl restart oidentd and my port 113 is open. I make command netstat -anp | grep 113 and have the response:
tcp        0      0 <IP_ADDRESS>:113             <IP_ADDRESS>:*               LISTEN      24094/oidentd
tcp6       0      0 :::113                  :::*                    LISTEN      24094/oidentd
I already restart znc and even so, does not work.

Yes,
I restaed  daemon oidentd with command sudo systemctl restart oidentd and my port 113 is open. I make command netstat -anp | grep 113 and have the response:
tcp        0      0 <IP_ADDRESS>:113             <IP_ADDRESS>:*               LISTEN      24094/oidentd
tcp6       0      0 :::113                  :::*                    LISTEN      24094/oidentd
I already restart znc and even so, does not work.

Just because a port is being listened on doesn't mean it's accessible .. you even quoted "No ident response; username prefixed with ~" which means port 113 isn't open.  Check your firewall rules.

Just because a port is being listened on doesn't mean it's accessible .. you even quoted "No ident response; username prefixed with ~" which means port 113 isn't open.  Check your firewall rules.

Ok. Work now. Was the port 113 that still close.
I guess that I had opened.
Thank you so much even.

Ok. Work now. Was the port 113 that still close.
I guess that I had opened.
Thank you so much even.


⚠ ClamAV has degraded performance

In 9532561, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 69 ms

Resolved: ClamAV performance has improved in 19d4a5b after 11 minutes.


Failure when creating NSG for AzureBastionSubnet

Please provide us with the following information:

This issue is for a: (mark with an x)
- [ X ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Apply terraform-azurerm-vnet-shared configuratino

Any log messages given by the failure

Network security group nsg-vnet-shared-01.AzureBastionSubnet does not have necessary rules for Azure Bastion Subnet AzureBastionSubnet."

Expected/desired behavior

No error expected.

OS and Version?

N/S

Versions

AzureSandbox v2.8.6

Mention any other details that might be useful

Thanks! We'll be in touch soon.

Rebuilding to see if this is a transient error

This is a transient failure will reopen issue if it comes back.


📖 docs: Note on 'host.docker.internal' for Ollama Config

Summary
The docs suggest a URL to access ollama ("http://localhost:11434/v1/") that is not working. This pull request fixes the URL to  "http://localhost:11434/v1/chat/completions"
Additionally there is an issue on Mac and Windows when accessing localhost from a docker container. It will not be found as the container is run in a VM. So in this case, you should use 'host.docker.internal' instead. I added this as a comment.
Thanks to Berry from the discord for his support!
Change Type

[x] Documentation update

Testing
I tested the URL "http://host.docker.internal:11434/v1/chat/completions" on my machine, with the ollama version 0.1.30

Thanks for this, glad you figured it out
relevant resources on docker networking:
https://huzaima.io/blog/connect-localhost-docker
https://stackoverflow.com/questions/64763147/access-a-localhost-running-in-windows-from-inside-wsl-2 (mainly for windows, since docker users wsl2 and works similar to how wsl2 routes the network)

In my logs it appears that baseURL is being changed to remove the /chat/completions portion
 - name: "Ollama"
      apiKey: "ollama"
      # use 'host.docker.internal' instead of localhost if running LibreChat in a docker container
      baseURL: "http://host.docker.internal:11434/v1/chat/completions"
      models:
        default: [
          "tinyllama",
          "llama2",
          "mistral",
          "codellama",
          "dolphin-mixtral"
          ]
        fetch: false # fetching list of models is not supported
      titleConvo: true
      titleModel: "llama2"
      summarize: false
      summaryModel: "llama2"
      forcePrompt: false
      modelDisplayLabel: "Ollama"

2024-04-14 02:24:33   baseURL: "http://host.docker.internal:11434/v1",
2024-04-14 02:24:33     modelOptions.model: "tinyllama",
2024-04-14 02:24:33     modelOptions.temperature: 1,
2024-04-14 02:24:33     modelOptions.top_p: 1,
2024-04-14 02:24:33     modelOptions.presence_penalty: 0,
2024-04-14 02:24:33     modelOptions.frequency_penalty: 0,
2024-04-14 02:24:33     // 3 stop(s)
2024-04-14 02:24:33     modelOptions.stop: ["||>","\nUser:","<|diff_marker|>"],
2024-04-14 02:24:33     modelOptions.user: "661ba0caf661a27520b22034",
2024-04-14 02:24:33     modelOptions.stream: true,
2024-04-14 02:24:33     // 1 message(s)
2024-04-14 02:24:33     modelOptions.messages: [{"role":"user","content":"TEST"}],

I'm not certain if that is intentional, but I cannot get Ollama to work with LibreChat (running both in docker containers on windows).


Does it work without flow control pin?

Hi,
having a look at the example.yaml I can see, there is a flow control pin defined.
My ttl rs485 board has no flow control pin. How do I have to modify the yaml config to work without flow control?
Thanks in advance for answer

Just remove the flow_control line.

The flow_control acts like a PTT (push-to-talk) of a radio transmitter. You have to press a button if you want to transmit. The attached GPIO gets pulled in every transmission. If you don't touch/remove the line you only loose a GPIO which gets triggered periodically.


icy_term clears scroll back when a screen clear occurs

this is counter productive just strip the clear and keep buffering the input so you can still scroll back if the screen clears

What should happen on a screen clear?


Unable to download llvm when building from source.

Hi I am seeing this error when building triton from source. How to resolve this?
downloading and extracting https://tritonlang.blob.core.windows.net/llvm-builds/llvm-657ec732-ubuntu-x64.tar.gz ...
      error: HTTP Error 403: This request is not authorized to perform this operation.

I'm seeing the same. Looks like there is an auth issue with tritonlang.blob.core.windows.net.

Yes, something is weird, some people are able to download it and others aren't.
@ptillet is working on a fix.

i meet the same issue and debug for long time :cry:

+1

Same issue

same issue

same issue

I have implemented what should be a temporary fix on the storage side. It may take a few minutes to propagate, but if someone who was blocked before could verify they have access again now I would appreciate it. Thanks!

https://tritonlang.blob.core.windows.net/llvm-builds/llvm-657ec732-ubuntu-x64.tar.gz

Seems to be working now.

I'm going to close this, please let me know if anyone else runs into any issues.


bug: error when open file .env

Did you check docs and existing issues?

[X] I have read all the LazyVim docs
[X] I have searched the existing issues of LazyVim
[X] I have searched the existing issues of plugins related to this issue

Neovim version (nvim -v)
NVIM v0.10.0-dev-468292d
Operating system/version
linux mint 22
Describe the bug
error treesitter parser when opening .env file

Steps To Reproduce

open neo-tree
open .env file
got error

Expected Behavior
no error when open env file
Repro

Ah, apparently you have to install bash treesitter, TSInstall bash


US427002: Removing all previous changes

https://internal.almoctane.com/ui/entity-navigation?p=131002/6001&entityType=work_item&id=427002

A developer build has not yet been created for this branch. Click here to go ahead and create the build...

CI Build Link:
http://sou-jenkins2.swinfra.net:8080/job/MicroFocus/job/MicroFocus~fortify-issue-manager~US427002a~CI


Avoid fetching all finality signatures concurrently

This PR changes the BlockSynchronizer to avoid creating all fetch requests for finality signatures in bulk.
The change should have the effect of requesting only max_simultaneous_peers (i.e. 5) at a time, and rather than asking 5 peers for the same signature, each peer will be asked for a different signature.
While trying to acquire weak finality, rather than stopping as soon as we have weak finality we continue until we have at least requested every potential signature.
There is also a fix for PeerList::need_peers() whereby it would previously return Sufficient even if it exclusively contained peers which were marked dishonest, meaning a subsequent call to PeerList::qualified_peers() would return an empty vector.
Related to #3578.

bors merge

bors merge

bors merge

bors try

bors try

bors merge


Update themes

Updates themes to meteor/hexo-theme-meteor/apollo latest

TODO:

[ ] If this PR is a new feature, reference an issue where a consensus about the design was reached (not necessary for small changes)
[ ] Make sure all of the significant new logic is covered by tests
[ ] Rebase your changes on master so that they can be merged easily
[ ] Make sure all tests and linter rules pass
[ ] Update CHANGELOG.md with your change

Thanks again!

Wow, thanks!


Bug report: Failed when reference a custom fields

The model definition seems like:
class MySQLTimestampField(TimestampField):
    field_type = 'timestamp'

class Table1(Model):
    timestamp_field = MySQLTimestampField()

and the generated code seems like:
import peewee as pw

...
@migrator.create_model
class Table1(pw.Model):
    timestamp_field = pw.MySQLTimestampField()

When I execute router.run(), it raise AttributeError: module 'peewee' has no attribute 'MySQLTimestampField'.
The suggestion is, scan all the referenced fields first, and get module by class.__module__, then generate codes by this relation.

should be fixed in the latest versions


Restructure akka-streams to fs2 DSL to prevent dropping Mat value

Continuation of https://github.com/krasserm/streamz/pull/55

I think we may need to update the documentation as well

By the way - Thank you for being super responsive and submitting this! I hope this helps people out using the library.
I do wonder whether we can (easily) support a period of @deprecated for the old names. Don't feel obligated to keep it, I might think about how to provide that later - maybe a replacement import or something.

Sorry I dropped the ball on this, I had notifications off for this repo for some reason.
Looks good, and I'll probably release it as a milestone build soon


Introduce Google Could Storage submitter

What is the context of this PR?
Card: https://trello.com/c/hT1VML3Y/2752-census-submissions-to-use-google-cloud-storage
Some additions/modification to Davids’s initial work.
Submits completed questionnaires to GCS for downstream.
How to review

Create a GCP project.
Create a storage bucket. https://console.cloud.google.com/storage/

or set up a project using https://github.com/ONSdigital/census-eq-terraform
then your bucket id would be: census-eq-<your-env>-survey-runner-submission
Set the following ENV variables.
EQ_SUBMITTER = 'gcs'
EQ_GCS_SUBMITTER_PROJECT_ID = 'your-project-id'
EQ_GCS_SUBMITTER_BUCKET_ID = 'your-bucket-id'

https://github.com/ONSdigital/eq-survey-runner/blob/46990cf653cae2179691e24474df91013f9d7119/app/settings.py#L48-L51
Ensure:

Submissions are successfully stored in your bucket.
Object/Blob name inside the bucket is same as submitted case_id
Object/Blob metadata is created using the tx_id. You can view it in the GCP console:

1. Open the Cloud Storage browser in the Google Cloud Platform Console.
OPEN THE CLOUD STORAGE BROWSER

2. In the list of buckets, click on the name of the bucket that contains the desired object, and navigate to the object.
Certain pieces of object metadata, such as the object's size and storage class, are displayed along with the object's name.

3. Click the more options button (More actions icon.) associated with the object.

4. Click Edit metadata.

The overlay window that appears shows the current values for the object's editable metadata.

or you can use the gsutil ls -L command, replacing [VALUES_IN_BRACKETS] with the appropriate values:
gsutil ls -L  gs://[BUCKET_NAME]/[OBJECT_NAME]

@jonnyshaw89 As discussed, have added metadata directly to the Object. Updated the PR description also.

Should we be using tx_id rather than case_id for the object name now that we can put case_id in the metadata?
@ajmaddaford @pricem14pc

@bitdivision we need to have a conversation with the RM team about how they will receipt, for now I'd say we shouldn't block this as we can easily swap the case_id and tx_id any time.

@bitdivision @jonnyshaw89 @ajmaddaford
Have changed the renamed the env variable and as per Jon's suggestion, case_id and tx_id have been added to the metadata.


e-b-z.de

https://www.e-b-z.de/

@Kapsonfire Pull request merged.
Please notice that free educational subscription for JetBrains tools is for students and teachers only, and is valid for using the tools for educational purpose only. If employees of this school who are not students, nor teachers, also need a license for JetBrains tools, they can be eligible for an academic discount for the tools. They can get the discount by email to<EMAIL_ADDRESS>Thank you!


feat(popup): 支持动态设置 trigger/placement & add zIndex api

🤔 这个 PR 的性质是？

[x] 日常 bug 修复
[x] 新特性提交
[ ] 文档改进
[ ] 演示代码改进
[ ] 组件样式/交互改进
[ ] CI/CD 改进
[ ] 重构
[x] 代码风格优化
[ ] 测试用例
[ ] 分支合并
[ ] 其他

🔗 相关 Issue

💡 需求背景和解决方案

📝 更新日志

fix(popup): 修复初始化 visible 为 true 时的定位抖动

fix(popup): add zIndex api

feat(popup): 支持动态设置 trigger & placement

[ ] 本条 PR 不需要纳入 Changelog

☑ 请求合并前的自查清单
⚠ 请自检并全部勾选全部选项。⚠

[x] 文档已补充或无须补充
[x] 代码演示已提供或无须提供
[x] TypeScript 定义已补充或无须补充
[x] Changelog 已提供或无须提供

Codecov Report

Merging #950 (5c6101b) into develop (a0db0df) will increase coverage by 2.17%.
The diff coverage is n/a.

:exclamation: Current head 5c6101b differs from pull request most recent head 59a1541. Consider uploading reports for the commit 59a1541 to get more accurate results

@@             Coverage Diff             @@
##           develop     #950      +/-   ##
===========================================
+ Coverage    57.56%   59.73%   +2.17%
===========================================
  Files          346      378      +32
  Lines        12828    14473    +1645
  Branches      2405     2693     +288
===========================================
+ Hits          7384     8646    +1262
- Misses        5200     5563     +363
- Partials       244      264      +20

Impacted Files
Coverage Δ

src/affix/affix.tsx
28.57% <0.00%> (-37.51%)
:arrow_down:

src/swiper/props.ts
68.75% <0.00%> (-31.25%)
:arrow_down:

src/dialog/props.ts
70.00% <0.00%> (-30.00%)
:arrow_down:

src/calendar/props.ts
70.00% <0.00%> (-30.00%)
:arrow_down:

src/popup/props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

src/slider/props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

src/message/props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

src/tooltip/props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

src/pagination/props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

src/steps/step-item-props.ts
75.00% <0.00%> (-25.00%)
:arrow_down:

... and 142 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 298ed5a...59a1541. Read the comment docs.


All fields should be possible to be set using Option

We need to update the procedural derive macro to make all fields possible to use Option<T> if we for example have set the table definition for allowing NULL. If no Option is set, it's automatically NOT NULL.

When the macro is implemented we need to filter out those who have None as value here:
https://github.com/njord-rs/njord/blob/master/njord/src/sqlite/insert.rs#L32


As a GPU researcher I would like the ability to read thermal information from LevelZero

Is your feature request related to a problem? Please describe.
I'm always frustrated when I am unable to read temperature on LevelZero enabled GPUs.
Describe the solution you'd like
Addition of support for LevelZero thermals as described here
Describe alternatives you've considered
Using sysfs or alternative interfaces.

Closed by https://github.com/geopm/geopm/pull/2743


args should come after the last positional argument

Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
[x] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?
[ ] If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?
Because *args came after the last kwarg in load_from_checkpoint, you couldn't pass in positional arguments correctly to the constructor of your model. This fixes that.
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

+1. Recently faced this issue, it would be nice to have this behavior:
class Module(...):
    def __init__(self, hparams, param1, param2, param3):
        ...

param1 = ...
param2 = ...
param3 = ...

ckpt_path = ...
my_module = Module.load_from_checkpoint(ckpt_path, param1, param2, param3)

@Borda let's add this to 0.7.6? need to get these tests to pass

unfortunately saw this too late but docs were not updated.
Docs should reflect changes like this, especially when they break backwards compat.
@yukw777 would you be willing to make a follow up?
I am also surprised the tests don't fail, especially because this is a breaking change. Or do I miss something?

@awaelchli I couldn't find any test cases for this, so I just left it at that. 🤷♂ I didn't think to update it b/c no one could really use *args at all before this change, so this fix would actually make the code consistent with the doc we have now. As @festeh said, some code may break, but I didn't think this would be a big issue, b/c that is a wrong way to call this method anyway, mixing args and kwargs like that. Having said that, I'm happy to send you a PR to fix the doc. Can you advise me on which doc I should update?

ok, got it. yes if it could not be used before then I understand why there is no test.
I think now that it is working properly, we should have a test that calls the load function with at least one arg and one kwarg and check that the model is correctly initialized.
In the docs, I saw there is just a *model_args but no kwargs. The docs are in the header of the function declaration, so we can directly update it there.


[WIP] Ask and tell for runtime optimization

This PR aims to solve the issue discussed in #560.
As a prerequisite #572 is required.
Current state:

I provided (or rather extended) a test which demonstrates the failure

Problems:

During the lying step the time values are not considered at all (correct me if i am wrong)
Within the tell step they are somewhat considered, but the result creation fails.

Additional Suggestion: The result object built by create_result (skopt.utils) should probably contain information about the runtimes as well.

The diff coverage is 66.66%.

@@           Coverage Diff            @@
##           master   #574      +/-   ##
========================================
- Coverage   86.25%    86%   -0.26%
========================================
  Files          23     23
  Lines        1717   1729      +12
========================================
+ Hits         1481   1487       +6
- Misses        236    242       +6

Impacted Files
Coverage Δ

skopt/optimizer/optimizer.py
91.21% <66.66%> (-2.57%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f35a17...05aca73. Read the comment docs.

@chschroeder is this ready for review or do you plan on working on the create_result bit in this PR?

This is not finished yet. The checks should not be green either :sweat:. I will try to keep the changes as small as possible, however, at first glance it looked like i need to get most of it working in order to pass the test (except for the create_result extension maybe).

Seems i messed up the rebase. Fixing this asap. (This was automatically closed somehow by my push).

Hello @chschroeder! Thanks for updating the PR.

In the file skopt/tests/test_parallel_cl.py, following are the PEP8 issues :

Line 20:80: E501 line too long (84 > 79 characters)

Looks like you are back on track :) CircleCI mostly just generates the webpage so the fact that it is green means our examples and docs work, not that your test isn't leading to a failure (as you'd expect). For the tests travis is the one that counts.

I have some time planned for this during the next week. Will keep you updated :)

The Problem seems to be that there are different understandings of how to handle the time values in different parts of the code in skopt/optimizer/optimizer.py. Mostly, the objective values and their corresponding time values are packed as tuples and unpacked again whenever needed. This was probably done to keep the interface as it is. However i feel both from the code and from a usability perspective we should handle the time values separately because as it is, it does not match the docs either (yi is a list of scalars).
For now, i will try to do the non API-breaking fix but i would like to propose to think about the interface here.


Data transmission Delay

It takes too much time to get/receive data in the PES device. Operators gets frustrated for this.
After punching one card, operator needs to wait for a significant time to punch again which hampers their production

This happens when there is no internet. The issue does not occur when there is slow internet or the internet goes offline after the device is started. When the internet goes offline after the device is already running, the issue starts again if the device restarts when the internet is offline.

I think this is happening because of NTP time acquisition during boot. Without internet, it fails to sync time. we need to investigate why it slows down card reading.

FW version 1.13.241015.122816 solves the issue. we are testing this firmware in production.
NTP is the problem. we will need to implement time updating from mqtt and add RTC in hardware for time keeping.


Better cohorts (or "near duplicate set") detection

Our current algorithm is approximately quadratic in number of cohorts. In particular this is not good for the full US NWM groupby.
I've been trying to learn about other methods to do so: .e.g clustering, and finding approximate duplicates
The most promising idea so far is "hashing to detect near duplicate sets". The set here is the set of chunks occupied by a particular group. In particular the MinHashLSH from the datasketch package works well
from datasketch import MinHash, MinHashLSH

sorted_label_chunks = dict(
    sorted(label_chunks.items(), key=lambda kv: len(kv[1]), reverse=True)
)
lsh = MinHashLSH(threshold=0.4, num_perm=128)
hashes = {}
for label, chunks in sorted_label_chunks.items():
    if label == 0:
        continue
    hash_ = MinHash(num_perm=128)
    [hash_.update(c) for c in chunks]
    hashes[label] = hash_
    lsh.insert(label, hash_)

chunk_cohorts = {}
for label in label_chunks.keys():
    if label == 0:
        continue
    cohort = lsh.query(hashes[label])
    if not cohort:
        # print("skipping")
        continue
    # if label not in cohort:
    #    cohort.append(label)
    chunk = []
    for c in cohort:
        lsh.remove(c)
        chunk.extend(label_chunks.loc[c])
    chunk_cohorts[tuple(chunk)] = cohort

assert lsh.is_empty()
assert len(np.concatenate(tuple(chunk_cohorts.values()))) == len(label_chunks) - 1

flox.visualize._visualize_cohorts(zone.data, chunk_cohorts.values())

This runs in 800ms (compare to ~3.5s!) to detect

Challenges are:

external dependency
some tuning might be helpful (e.g. high threshold -> more cohorts), so maybe we want a CohortsGrouper type of class like pandas.Grouper that lets users customize how the cohorts detection works.

For now, I've switched to "containment" (#291). This seems to  be the right metric.

Other options:

https://github.com/ekzhu/josie
vector search: https://unum-cloud.github.io/usearch/python/#
https://github.com/ekzhu/SetSimilaritySearch
https://github.com/nmslib/hnswlib

I think a lot of these optimize for larger sets than we might see usually. And we care about removing a result from the "index" once its returned.

https://github.com/nicodv/kmodes is k-means adapted to categorical variables. Would be good to try out.

Seems to be solved for now using the software matrix approach.
https://flox.readthedocs.io/en/latest/implementation.html#heuristics


Deterministic ClientIDs in a pool

Hello,
I just wondered what would be the best way to have a non-random, deterministic ID created for a worker pool. This could be as simple as just a counter.
How would you go about this? Let's say you have a pool with 10k workers, a connect function (that needs to  give a client ID) and you want the IDs to be named "1" to "10000".

You can use worker id I I suppose. Get it from Meta which is passed to every worker function.
    ID = proplists:get_value(worker_id, Meta)

I'm not sure what the first Id is 0 or 1. But you can easily check it.

Ah, when it's that simple and you get offered the solution on a silver tablet! :)
Thanks a lot!


NIOHTTP1Server can not serve video to Safari (Ranges: handling)

Expected behavior
Be able to serve video files to any major browser.
(Handling ranged requests or at least respond in a way that the browser will accept)
Actual behavior
I have modified NIOHTTP1Server to return Content-type: video/mp4 to serve video files
from a macOS app (I would love to replace CocoaHTTPServer with swift-nio).
Firefox and Chrome do show the video, Safari (11.1 on macOS 10.13.4) does show a broken play button.
Safari seems to use Ranges no matter if the server includes Accept-Range:.
NIOHTTP1 responds with the whole file to a Range: bytes=0-1 request.
Safari first requests (GET) the file without a range - receives the whole file and then requests it again (probably via AVFoundation).
I tried to add code to send "Accept-Range: bytes" but it didn't change Safaris behaviour (which is - at least for iOS - documented to depend on working Range: headersI and I didn't find anything in the NIOHTTP1 code implementing them.
I would like to know if someone is already working on supporting Ranges on NIOHTTP1. If so, I am willing to help. If not I would appreciate pointers on where to best start implementing them as I am not yet familiar with the code.
NIOHTTP2 does not seem to be a viable option yet for production code, so I'd like to get it working with NIOHTTP1.
Below I attach a dump of the request and response headers for the Safari case.
(The results do not change for fileio vs. sendfile).
Thank you very much, kind regards, Bjoern
Request: GET /fileio/rabbithole.mp4
Host: localhost:4200
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15
Accept-Language: de-de
DNT: 1
Accept-Encoding: gzip, deflate
Response:
Content-Length: 328686
Content-Type: text/plain; charset=utf-8
Content-Type: video/mp4
Request: GET /fileio/rabbithole.mp4
Host: localhost:4200
DNT: 1
X-Playback-Session-Id: A42130E1-B461-4BE7-99E8-FAF674864965
Accept-Language: de-de
Range: bytes=0-1
Accept: /
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1 Safari/605.1.15
Referer: http://localhost:4200/fileio/rabbithole.mp4
Accept-Encoding: identity
Connection: keep-alive
Response:
Content-Length: 328686
Content-Type: text/plain; charset=utf-8
Content-Type: video/mp4
Steps to reproduce
If possible, minimal yet complete reproducer code (or URL to code)
SwiftNIO version/commit hash
bf8c64289654d57fd2a9e9405c99798f9515f3c4
Swift & OS version (output of swift --version && uname -a)
Apple Swift version 4.1.2 (swiftlang-902.0.54 clang-9<IP_ADDRESS>)
Target: x86_64-apple-darwin17.5.0
Darwin lucy.local 17.5.0 Darwin Kernel Version 17.5.0: Mon Mar  5 22:24:32 PST 2018; root:xnu-4570.51.1~1/RELEASE_X86_64 x86_64

Yours responses carry multiple content-type headers? First text/plain, then video/mp4. Maybe you just need to fix that ...

Oops. Thank you, seems like I had a severe case of snow blindness.
I fixed it and re-ran the test but it didn't change the result, I edited the issue accordingly.

@bkrpub SwiftNIO is really a framework to built networking libraries and applications. NIOHTTP1Server literally just serves as an example of how to deal with certain HTTP1 things (and as a server for our integration tests).
There's lots of stuff missing that would make this example a real HTTP server. For your application I would suggest using one of the existing Swift webservers that are written with SwiftNIO. For example Vapor, Kitura (there's an option to use SwiftNIO), swift-web or some of the others.
If you prefer to go low-level and use NIO directly it would be your responsibility to handle the range requests, you will be able to implement them but it'll be more work than you'd like. Therefore I'd suggest using something higher level that's meant to be used as a HTTP server and obviously I'd recommend using one that's written with SwiftNIO ;).
Please let me know if I'm missing something, otherwise I'd close this issue.

Does any one of those frameworks actually handle the issue?

I actually implemented the most simple form of range requests (only one range) by slightly extending FileRegion (to keep around the file size) and got video to work but also realized how much is missing, so basically I am done with the issue and thank you very much for the pointers.
What I would still be interested in are opinions on which HTTP implementation to use for running a server inside a macOS GUI app. The next experiment will be using Vapor. Is anyone here sharing that use case?
Thanks a lot, Bjoern

SwiftNIO? http://www.alwaysrightinstitute.com/nio-on-the-client/


More schedule requirements

https://trello.com/c/a4sDMtqD/844-ensure-publishers-meet-requirements-before-they-schedule
This ensures the user can see publishing issues (e.g. no summary) and scheduling issues (e.g. publish time has drifted into the past) on the document summary, and that they meet both before they attempt to schedule a document. Part of this involved a refactoring of the existing ScheduleDatetimeChecker - please see the commits for more details.

@kevindew I think I've addressed all your comments now, except for the one where where I haven't 😜 .


Add a test for rendering default views for the missing views in a subset custom composition

As explained in https://github.com/Starcounter/starcounter-include/pull/123#discussion_r278310702, the starcounter-include tests do not catch if this works correctly.

Related: https://github.com/Starcounter/Blending/issues/571

Won't be fixed, since the component is not relevant for Starcounter 3.


onnx file won't work in deepstream_python_apps without EXPLICIT_BATCH flag set

I have a trained detection model that I trained in this excellent Jetson-Inference series. In that process instructed by pytorch-ssd.md page, training produces a .pth file that is then converted by the onnx_export.py utility to an onnx model. I want to use that trained model in one of the deepstream_python_apps that are perfect for my needs. To use an onnx model, you change the config file to comment out the tlt-encoded-model and the tlt-model-key, add a line with the name of the onnx file, then use trtexec to create an engine file from the onnx file and put that in the model-engine-file entry in place of the sample that was there. With that setup, it starts up great, but then hits an error saying that the onnx model must have been created with NetworkDefinitionCreationFlag.EXPLICIT_BATCH, since trt no longer accepts implict batch size. Adding the argument --explictBatch to the trtexec command doesn't solve it. The documentation at https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#explicit-implicit-batch also doesn't help.
I've tried to get around that by going all the way back to the training and the conversion to onnx and setting the batch size explicitly, but that does not fulfill the requirement. Is there any way to use an onnx network created by the training or the onnx_export.py conversion to be able to run in the deepstream_python_apps? Perhaps some way something could be added to the onnx_export code to set that parameter?
I've got such a nice trained onnx network and the deepstream_python_app examples are perfect for what I want to do if I can solve this.

It does seem like a major flaw to have the main network training method in the dusty-nv/jetson-inference tutorials produce an onnx file that can't be used in the main sample apps in the deepstream_python_apps....  How can that be remedied?

@whutchi the deepstream models are typically trained with TAO (or YOLO iirc) and if you were to get past the explicit batch thing, I'm not sure those other samples support ssd-mobilenet and the pre/post-processing for it.  Supporting models requires more than just the ONNX, and the pytorch training scripts in jetson-inference are basically for educational examples.

From: whutchi @.>
Sent: Friday, June 14, 2024 11:28:30 AM
To: dusty-nv/jetson-inference @.>
Cc: Subscribed @.***>
Subject: Re: [dusty-nv/jetson-inference] onnx file won't work in deepstream_python_apps without EXPLICIT_BATCH flag set (Issue #1864)
It does seem like a major flaw to have the main network training method in the dusty-nv/jetson-inference tutorials produce an onnx file that can't be used in the main sample apps in the deepstream_python_apps.... How can that be remedied?
—
Reply to this email directly, view it on GitHubhttps://github.com/dusty-nv/jetson-inference/issues/1864#issuecomment-2168272971, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADVEGK2SYOVYPJM4YR5RXMTZHMDZ5AVCNFSM6AAAAABJDL6222VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRYGI3TEOJXGE.
You are receiving this because you are subscribed to this thread.Message ID: @.***>


Possible decoupling of services handled

See how practical it is to decouple services from utils/core. Idea is, user only has to provide a service config file and core will simply detect all that's inside the services config directory.
Maybe, move utils/core into utils/kernel/core and services config into utils/kernel/config.

Fully implemented.


Create persistent custom command script for startup

As proposed in issue mattrayner#112
For the user to be able to create custom commands at container startup there should be a persistend script or folder, where we can add our commands like cron startup and such
As the script resides in /run, users can easily bind mount this directory to persistent storage
If the folder or script doesn't exist, we do nothing

Hi. if you want to make some experimental changes, I think you should switch to another branch in your forked repo, otherwise the update will always be sent to all subscribers.
I also note that you are working on the "master" branch. You may need to create a new branch and then change the default branch to it.

omg, sorry :)
I didn't mean to spam your inboxes.

Don't worry. Happy coding :)

I have this working now with my own repo :)
For the new nextcould version I needed some PHP packages.


Parser: read optional PROJECTCOMPATVERSION Record.

Hi @tim-weis ,
I'm the author of an R package to create and modify Office Open XML files and have written various not so safe C++ parsers, including one for the XLSB format. Yesterday I tried to look into the vbaProject.bin file and came across your ovba project. I tried to embed it in an R package, but had no luck. Given that I have no prior knowledge of either rust development or rust development in R, this didn't surprise me. Still I tried to toy around with it and this toying around results in this PR. But as stated, until yesterday I hadn't written a single line of rust code. Therefore ...
The PR added the CompatVersionRecord which is an optional field that is defined as "A 32-bit number that identifies the Office Model version used by a VBA project." This record is optional and according to the OBVA documentation not written by VBA 5.0. Previously, if this record was found, the parser would simply panic. (There is another optional field ConstantsRecord, but this is at the end of the PROJECTINFORMATION record, therefore it didn't bother me).
The second minor change replaces the backslash to slash. This allows using ovba on Mac (and probably Linux). I ran the code on an M1 Mac and the two files I used for testing are created with MS365 on Mac.
A file which has this record can be found here, not sure anymore when or why this was created. Well this file loads now, and even works in my R rust package.
PS: I looked into your ovba-cli file, but this uses some Windows exclusive APIs and I couldn't convince ChatGPT to provide me with a drop in replacement for this.
PPS: Because I initially didn't know what caused the panic I updated the cfb crate to 0.10, hoping that this might solve some hidden issue. Since I'm not aware how the dependency updates work, I left this out of the PR.

Hi,
thanks for the quick response!
When I said panicked, I was referring to the output I'm seeing without my fix (this is with the backslash -> slash correction):
> library(ovbars)
> fl <- "https://github.com/JanMarvin/openxlsx-data/raw/refs/heads/main/gh_issue_416.xlsm"
> wb <- openxlsx2::wb_load(fl)
> vba <- wb$vbaProject
> code <- ovbars::ovbar_out(name = vba)
thread '<unnamed>' panicked at /Users/janmarvingarbuszus/Source/ovbars/src/.cargo/registry/src/index.crates.io-6f17d22bba15001f/extendr-api-0.7.1/src/robj/into_robj.rs:71:13:
called `Result::unwrap()` on an `Err` value: Parser
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread '<unnamed>' panicked at src/lib.rs:12:1:
explicit panic
Error in ovbars::ovbar_out(name = vba) :
  User function panicked: ovbar_out

I assume that the issue is with the R side of things due to called 'Result::unwrap()' on an 'Err' value: Parser
Updating cfb did not change a thing. I have reverted it for my package implementation and it still works fine. Like I've said. Initially I wasn't sure why the parser wasn't working and assumed that updating the cfb crate could help.
Regarding ovba-cli I already assumed that much, it was just another straw that could have potentially solved my issue. Given that the issue was due to something in ovba, it wouldn't have helped much either way.
Thanks, I will have a look at calamine, maybe one day I want to extract from the xls OLE streams, but that's a task for another raining month. For this project I only wanted to be able to read the macro code: https://github.com/JanMarvin/ovbars

Sweet, thanks, that's helpful!
From what I understand, the parser reports an error when presented with an (unexpected) optional PROJECTCOMPATVERSION record, that's subsequently converted to a panic! via the extendr-api crate. This aligns with my expectations. Thanks for verifying.
Updating the cfb crate to a later version without exhibiting a change in observable behavior is reassuring as well.
While looking into this I realized that the final PROJECTCONSTANTS record is also optional (and has been per v9.1), but isn't honored by the implementation. That's a bug I'd also like to address here.
I'm putting together test cases to verify that either record is accepted, when present, without failing when either isn't.

That said, I'd like to address the path separator issue separately. I don't fully understand it or know how best to address it quite yet.
The cfb crate uses the Path type in its public interface. This type is designed to work with the local file system and file system APIs. CFB's make this tricky: They are essentially virtual file systems wrapped into files, allowing them to travel across systems with different local file system semantics. I'll have to look into this some more.
I'll get back to you once the testing is in place.

That didn't turn out as expected...
I'd implemented (and pushed) tests to the feature/parse_opt_records branch and changed the base branch for this PR, hoping that it would be moved to the tip of this branch. Apparently, that did not happen.
I'm not intimately familiar with Git or GitHub, but could you try to rebase and (force) push your PR?
Just to make sure we aren't talking past each other: This branch is all about addressing the issues revolving around acknowledging the optional PROJECTCOMPATVERSION and PROJECTCONSTANTS records in the PROJECTINFORMATION.

Sure thing, will push once I’m home

Sorry, this should fix the formatting warning. I had it adjusted, to look similar to the other code parts, but apparently that wasn't a good idea 😬

Thanks, I was about to ask whether you had a rustfmt.toml lingering somewhere that caused the formatting test failure. The last push fixes things up.
I'll publish a new release as soon as I can. Will be looking into the path separator issue, too.
The effort you put into this is much appreciated. Thanks a lot for that!


Added tests for fetching services defined in ConfigProvider

ping @Xerkus @weierophinney

Thanks, @webimpress; this looks like a really nice solution.


File-based service discovery doesn't accept relative paths to files

I tried configure the file-based service discovery with file pattern relatative to the main Prometheus configuration catalog (usually /etc/prometheus).
I've found that only full path is accepted - not relative path.
In my opinion issue is related to the line https://github.com/prometheus/prometheus/blob/master/discovery/file/file.go#L85 what currently is
	paths = append(paths, files...)
In opposite rule files can be provided with a relative path because the line https://github.com/prometheus/prometheus/blob/master/config/config.go#L63 contains the additional command
	resolveFilepaths(filepath.Dir(filename), cfg)
Confirmed with Prometheus 1.5.2.

#2247 covers this.


Message interface for inter-transport use.

Right now the Message interface  can't be used without casting to a transport-specific interface. Its only operation (GetVersion) is useless without additional knowledge about the transport-specific implementation, from which you could just as well get the version directly.
Propose an enhanced Message interface that can be used independently of the transport that created it, in particular as a go-between when forwarding messages from one transport to another.
In particular this solves the following problems that can't be solved with the existing API:

forward structured event encodings without decoding ; provided both transports are happy with the encoding format and don't need to validate or inspect the content. The resulting performance benefits are the main (only?) reason for using structured over binary bindings.
negotiate reliable qualities of service between different transports with similar QoS levesl (at-most-once, at-least-once, exactly-once) Although each transport can use its own mechanisms to ensure the QoS there is a need to hand-over responsibility between the transports, else messages can be lost if the process doing the forwarding crashes.

Code and examples are nearly complete - coming soon as  PR.

Code and examples in PR #164. The idea is not to push this PR but to use it to focus discussion, we will need to agree on how/if to integrate the various ideas into the main transport package.

When I was writing this originally I had never thought to pass a transport.Message to any other transport because the impl of the Message object is specific to a transport, and created after passing an Event through a Codec. The interface allows a codec to talk to a transport without being aware for the most part.

When I was writing this originally I had never thought to pass a transport.Message to any other transport because the impl of the Message object is specific to a transport, and created after passing an Event through a Codec. The interface allows a codec to talk to a transport without being aware for the most part.

Yep, this proposal is to augment the interface so they become useful between transports. The codec still makes sense as a separate entity for users that want to go Event -> http.Request or amqp.Message -> Event and use their own http or amqp client/server code rather than the Transport/Sender/Receiver we provide.

Fixed by #164


NVM, Node, NPM not available after closing Terminal session

Operating system and version: macOS Sierra v10.12.2

nvm debug output:

-bash: nvm: command not found

nvm ls output:

-bash: nvm: command not found

How did you install nvm? (e.g. install script in readme, homebrew):
Manual Install

What steps did you perform?
I followed Manual Install with Git.

What happened?
NVM, Node, NPM is installed during the first try. But, when I exit Terminal, NVM, Node, NPM is not there. I've tried node -v, npm -v and it says -bash: nvm: command not found

What did you expect to happen?

Is there anything in any of your profile files (.bashrc, .bash_profile, .zshrc, etc) that modifies the PATH?
This is the part I don't understand, how do I add export NVM_DIR="$HOME/.nvm" [ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm to my .bashrc, .bash_profile, .zshrc ?

There is no .bashrc, bash_profile or .zshrc in my $HOME folder.

If you are having installation issues, or getting "N/A", what does curl -v nodejs.org/dist/ print out?

The first step is to manually create an empty ~/.bash_profile file, and then, add these lines to it:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh" # This loads nvm

After doing that, and reopening your terminal, what happens?

Sorry for being a novice, but how to I add these lines? That's the problem I'm having now? Do I just copy these entirely and just paste to terminal? Or do I have to do something before or after?

You do need to know how to edit files in the terminal, I'm afraid. You can learn vi, or emacs, or nano - or you can use a graphical editor that allows you to edit dotfiles (files whose names start with a "."). http://vim-adventures.com may be useful.
However, while I'm happy to help, teaching terminal basics is out of scope of this repo, and is something you'll need to learn before being able to use node, nvm, as well as most developer tools - feel free to keep asking questions here and I'll keep trying to help :-) I'll close this for now.


stop reporting DownloaderStatus and VerifyImageStatus as fake disks to the controller

@zed-rishabh @adarsh-zededa with the contenttree and volume reporting we definitely don't need this hack any more. And avoids the size of the metrics message growing very large when we have lots of blobs to report.

I don't know much of the history of "this hack", but coming at it clean, looks good to me.


Address DelayedInitSpecs flakiness

The integration test DelayedInitSpecs fails more often than it passes, making our integration testing fairly unreliable.

Resolved as of 0.9.6


how to build a alone tesseract5 cli use for windows xp

Current Behavior
how to build a alone tesseract5 cli use  for windows xp
Can get me a tutorial
Expected Behavior
No response
Suggested Fix
No response
tesseract -v
No response
Operating System
No response
Other Operating System
No response
uname -a
No response
Compiler
No response
CPU
No response
Virtualization / Containers
No response
Other Information
No response

Did you try the latest installer? Maybe it still works with Windows XP (although that is no longer a supported Windows version).

@stweil  https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w32-setup-v5.0.0-alpha.20190623.exe
i install it in windows xp but run  tesseract.exe f.png a -c dotproduct=sse get

if i try tesseract.exe f.png a a.txt will be nothing

That version works as expected even on Windows XP. Tesseract does not always find text, especially for images with a single word. Did you also try the latest Tesseract version?

@stweil which version

The latest version is 5.3.1. See https://github.com/UB-Mannheim/tesseract/wiki/.

for w32 only have 5.3.0

@stweil  I try https://digi.bib.uni-mannheim.de/tesseract/tesseract-ocr-w32-setup-<IP_ADDRESS>21222.exe ,but get
GetTickCount64 error with KERNEL32.dll

@stweil can help ?

I am afraid no. Windows XP only works with old versions of Tesseract.

I tried all builds from version 3 to 5, none of them works in my WinXP SP3.
version 3.05 displays error of NtCreateFile API failure, which is typical for non-XP exe:

And the newest 5.03 will displays a info "Ctrl+C to exit".
Only version 3.02 build of sourceforge from original tesseract-OCR works for me.

If you're having trouble installing tesseract-ocr-w64-setup-<IP_ADDRESS>40606 on Windows 11, there's an essential step after downloading: you must add the Tesseract path to the system environment variables. Normally, you'd locate the executable, typically at "C:\Program Files\Tesseract-OCR\tesseract.exe", and add its directory to the PATH. On Windows 11, you need to add "C:\Program Files\Tesseract-OCR" to your system environment variables. After this, open a new Command Prompt window and run tesseract --help to confirm everything is set up correctly. Hope this helps!

That's not necessary. If you open the Tesseract command line, the PATH is set automatically.
I try to avoid setting the global PATH environment variable because it slows down each program execution. And there is also a length limit for it.

I close this issue because there is nothing we can do.


Only upload screenshots with deliver

As per this thread: https://github.com/fastlane/fastlane/issues/14126, I tried sending the empty string in "deliver" to just upload the screenshots. But it did not work. Here is the lane:
desc "Uploads screenshots only to appstore"
lane :uploadScreenshots do
    deliver(
        ipa: "",
        overwrite_screenshots: true,
        skip_binary_upload: true,
        skip_metadata: true,
        skip_screenshots: false,
        submit_for_review: false)
end

Error:
[19:11:02]: --- Step: default_platform ---
[19:11:02]: ------------------------------
[19:11:02]: Driving the lane 'ios uploadScreenshots' 🚀
[19:11:02]: Error setting value '' for option 'ipa'
[19:11:02]: You passed invalid parameters to 'deliver'.

Have you tried setting the path to a valid ipa file? It should probably just not do anything with it.

hmm, I also thought that a .ipa must exist at some point when I want to upload screenshots but yesterday, I found this when my CI server created a new VM, took the clone of my code, created a new app on the appstore via "produce" and took the screenshots via snapshot, to just upload them without a build. Since there was no .ipa file yet as gym/build didn't execute till then, it failed.

Yeah, I mean setting any ipad and test out if the code then works and just does not do anything with said ipa file. Very possible this is currently just not taken into account, but could be if we know it works.

Ok, I'll try this with a valid build path as well.


Counter: Give style to button

I would like to work on this issue. Please assign it to me.

Can you specify which button style should I change?

Does this look good? Or Should I add animation?
The button of "color flipper" is the effect after hover.

Hey @khushi935 thank you for showing your interest in this project. Yes you can do this & I guess it will be great if you align this button center. And Also change the button style of counter project.

@Vishal-raj-1  I have a small doubt, I need to align text(ex calculator) as well as visit button in center?

@khushi935 YES !! You have to align both of them !!

https://user-images.githubusercontent.com/62712155/107210236-dda95e80-6a29-11eb-8198-4edf12399c98.mp4
Does this look good?

@khushi935 This looks awesome !! Can you do one more thing ? Increase height of the project images. and give some fixed heights so that there would not be any extra white spaces.
Thanks

okay, I will do it.

Does this look good?

@khushi935 Perfect send the PR for the same !!

Does this issue is resolved?


docker-compose please?

Hi,
has anyone written a docker-compose.yml file yet?

sorry this should do it
docker-compose.yml
version: '3'
networks:
logging:
services:
logspout:
image: gliderlabs/logspout:latest
networks:
- logging
volumes:
- /etc/hostname:/etc/host_hostname:ro
- /var/run/docker.sock:/var/run/docker.sock
command:
syslog://<IP_ADDRESS>:514
restart: always
################

If someone needs it, here's a working version of docker-compose.yml for Docker Swarm:
version: '3.3'
services:
  logspout:
    image: gliderlabs/logspout
    command: syslog+tls://xxx.papertrailapp.com:xxx
    volumes:
     - /var/run/docker.sock:/var/run/docker.sock
    deploy:
      mode: global

You need to replace syslog+tls://xxx.papertrailapp.com:xxx with the right command


New Validation: Assignment of wrong type, according to JSDoc

Produce a builder marker when a variable gets assigned a value of a type that is not the same as the type defined in the JSDoc

To support this issue, I need the following PR https://github.com/marijnh/tern/pull/525 to know tha type comes from comment.

See https://github.com/angelozerr/tern-lint/wiki/Validation-JSDoc for configuration of JSDoc


Documented google integration requires domain wide privileges

Describe the bug
The documented google integration seems too permissive, in that it guides creation of an API key with domain-wide full access.
Expected behavior
Documentation clearly outlines minimal permissions required to function, and instructs user through granting only the minimal access required for dispatch (ideally per-plugin)
Additional context
I'm in the process of integrating dispatch into my organizations google workspace, but am receiving push back as the documented permissions are understood to be root level rw across the entire account, including ability to read others emails.  I'd anticipate this to be problematic at other orgs as well.
I'll be trying to find the minimal set of permissions to get the google plugins up and working through trial and error, but a clearly written guide would be very helpful.  Any info about required permissions that could be provided here will be helpful as well!

We recognize it's more permissive than it should, but we haven't had time to prioritize the changes required to reduce it. More context here.
Can you elaborate more on what you think it's missing in the documentation? All you need should be documented here, but let me know if you disagree and we can work together to improve it.

Thanks for the context, that helps!
With regard to the documentation, I think it'd be helpful to include additional detail about the permissions required at a per-plugin level, walking through how to set up the minimal access required, and including rationale for cases where domain-wide delegation is a requirement.
FWIW I was successful in connecting the drive/docs plugins with a normally privileged service account/key, and "sharing" edit access on the root incident storage folder with the IAM email address associated to the google service account (client_email entry in the json file).  Maybe this could be a documented configuration for the drive/docs setup (and maybe it'd work for other plugins too, have not yet tested that)

Yeah, I agree your suggestion would be a good documentation improvement. Feel free to submit a PR for documentation or other improvements!


intall lua-websocket is failed !!!

link is faile!!!!
I do not know why? has other software?
install command is "luarocks install --server=https://luarocks.org/dev lluv-websocket UV_INCDIR=G:\save\other\libuv-devinclude UV_DIR=G:\save\other\libuv-devlibuv-dev"

(Link to the master)[https://github.com/moteus/lua-lluv/archive/master.zip] is valid. May be your version on LuaRocks does not support TLS version required by githib.
Do you link against static library? If yes you have to add some additional library in the dependencies. Unfortunately i do not know how to handle this different using rockspec file and It is not possible handle all combinations via multiple rockspec files. I think you can find needed libraries in google. I can help if you will heve any problems

1.Support lua5.3? I build lua-lluv-websocket project, and need lluv .so use luarocks command
"luarocks install lluv" build this ,and out put problems(second png).
2.How to build lua-lluv-websocket , you with other compilation methods, not user luarocks?
3.My project support lua5.3 , and want to use websocket lib, you have better recommendations.

Both lluv and lluv-websocket support Lua 5.3
I use LuaRocks tou build them on Win/Linux/OSx systems
Integrate libuv to some existed project may be difficult
About building lluv library - rockspeck file does not support static version of libuv library becase it does not link with windows system libraries. You need to update rockspec file to fix it, or link with dynamic libuv library

Can you recommend the version of lluv you used to me?
How to build dynamic libuv library in Win?
I found lua-rocks build lluv with static version of libuv library.
And update lua-rocks version is luarocks-3.2.1 build lluv is failed!!

build lluv-websocket by luarocks ,be dependent on　lluv?

I build lluv-websocket on linux is success !!

On Windows I use my lua-windows-environment distro which includes some prebuild external libraries (not Lua modules) and sligthly patched LuaRocks.
  All rights reserved.

lluv.obj : warning LNK4197: export 'luaopen_lluv' specified multiple times; using first specification
   Creating library lluv.lib and object lluv.exp
lluv 0.1.10-1 is now installed in C:\luaenv\x64\5.1\systree (license: MIT/X11)

C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC>luarocks install lluv-websocket --from=https://luarocks.org/dev
Installing https://luarocks.org/dev/lluv-websocket-scm-0.rockspec

Cloning into 'lua-lluv-websocket'...
remote: Enumerating objects: 65, done.
remote: Counting objects: 100% (65/65), done.
remote: Compressing objects: 100% (63/63), done.
remote: Total 65 (delta 7), reused 26 (delta 2), pack-reused 0
Receiving objects: 100% (65/65), 46.84 KiB | 138.00 KiB/s, done.
Resolving deltas: 100% (7/7), done.
lluv-websocket scm-0 is now installed in C:\luaenv\x64\5.1\systree (license: MIT/X11)

Which version of libuv do you use？1.3XX version?

I build lluv.so success(lua5.3) ,but "lluv.websocket" not found in the lluv?
error : ./lua/testwebsocket.lua:4: module 'lluv.websocket' not found:
no field package.preload['lluv.websocket']
no file './lua/lluv/websocket.lua'
no file '/usr/local/share/lua/5.3/lluv/websocket.lua'
no file '/usr/local/share/lua/5.3/lluv/websocket/init.lua'
no file '/usr/local/lib/lua/5.3/lluv/websocket.lua'
no file '/usr/local/lib/lua/5.3/lluv/websocket/init.lua'
no file './lluv/websocket.lua'
no file './lluv/websocket/init.lualluv/websocket.lua'
no file '/data/project/LinuxLua/AgentServer/build/lua/5.3/include/lluv/websocket.lua'
no file '/usr/local/lib/lua/5.3/lluv/websocket.so'
no file '/usr/local/lib/lua/5.3/loadall.so'
no file './lluv/websocket.solluv/websocket.so'
no file '/data/project/LinuxLua/AgentServer/build/lua/5.3/lib/lluv/websocket.so'
no module 'lluv.websocket' in file '/data/project/LinuxLua/AgentServer/build/lua/5.3/lib/lluv.so'
stack[0] -> line -1 : unknown[[C] : line -1]
stack[1] -> line -1 : require()[[C] : line -1]
stack[2] -> line 4 : unknown[./lua/testwebsocket.lua : line 0]
stack[3] -> line -1 : require()[[C] : line -1]
stack[4] -> line 145 : unknown[lua/ManagerTest.lua : line 0]

luarocks build lluv-websock with this "lluv-scm-0.rockspec" default  x64? I build x86 version on Win．

luarocks build lluv-websock with this "lluv-scm-0.rockspec" default  x64? I build x86 version on Win．

CrtSetReportHook
GetMessageA
TranslateMessage
DispatchMessageA
MapVirtualKeyW
GetUserProfileDirectoryW
GetSystemMetrilluv
Link failed all win system function!!

Some of them are needed only for debug versions. Did you try link against to release version?

I build lluv.so success(lua5.3) ,but "lluv.websocket" not found in the lluv?

You install library to the project directoory but trying to find them in the system path.

luarocks build lluv-websock with this "lluv-scm-0.rockspec" default x64? I build x86 version on Win．

websocket library is pure Lua library so it is not x86/x64
lluv library - is the C library and can be compiled in any way you want. Luarocks supports both arhitectures. But I have to install 2 versions of them to do this.

"I build lluv.so success(lua5.3) ,but "lluv.websocket" not found in the lluv?"
This problem in Luarocks version 2.4.3 , No problem with later version．
My Win Project just only debug , I'm not going to get the release．
And Luarocks build by myself with Win x86. I build success mongo.dll(mongo.so) with this!!
I do not how to build lluv-websocket.
Why don't you tell me which version you use．which luarocks version and lluv version?

Once again. On windows I use my lua-windows-environment distro. Which contains LuaRocks 3.0.4. On linux I also use this version.

I build lluv.so success(lua5.3) ,but "lluv.websocket" not found in the lluv?

This does not make sence to me.
lluv-websocket is just set of Lua files. They do not need to be built - just be copied to your LUA_DIR.
From your post
lluv scm-0 is now installed in **/data/lua_project/luarocks** (license: MIT/X11)
lluv-websocket scm-0 is now installed in **/data/lua_project/luarocks** (license: MIT/X11)

But when you try to load them you looking in other directories (file names websocket.lu).
no file './lua/lluv/websocket.lua'
no file '/usr/local/share/lua/5.3/lluv/websocket.lua'
no file '/usr/local/share/lua/5.3/lluv/websocket/init.lua'
no file '/usr/local/lib/lua/5.3/lluv/websocket.lua'
no file '/usr/local/lib/lua/5.3/lluv/websocket/init.lua'
no file './lluv/websocket.lua'
no file './lluv/websocket/init.lualluv/websocket.lua'
no file '**/data/project/LinuxLua/AgentServer/build**/lua/5.3/include/lluv/websocket.lua'

This is not LuaRocks problem but your Lua environment configuration.
LuaRocks can install module in any place and it does not responsible for Lua configuration.

First Linux llbuv is OK, It's just a problem with the use
And I found problem with Win.
Libuv has been upgraded to the latest version v1.8, and the required lib files have changed: WS2 ﹣ u 32.lib is OK, but two LIBS need to be added. -->"userenv.lib", "msvcrtd.lib"
"libraries = {"libuv", "Iphlpapi", "Psapi", "userenv", "msvcrtd"},"
I try this llib is ok with C++ project . Now I want to upload luarocks file, But I do not how to do this.
I hope you can upload you luarocks file.  Thank you so much!!

websocket.extensions and lluv.ssl is not found.
If it's not found, is there any problem? Because it's not found under Linux, the same as win.


Improve Python environment activation

Currently for Python virtual environments we use ms-python.python's API to grab the currently selected Python interpreter and then we run:
$python_bin -c "import sys; print(sys.executable)"
This gives us the absolute path of the interpreter. We then call dvc with path/to/interpreter/python -m dvc with some args. We also prepend the directory that the python interpreter is in to the system $PATH variable before running the command. This works for the basic case of DVC being installed within the virtual environment and is the suggested approach from an ongoing github discussion.
Our current approach also works for global installs of DVC as long as the required dependencies are installed within the virtual environment. E.g for the demo project a global install of DVC does not work unless dvclive and ruamel.yaml are added to the virtual environment. I don't think there is a way around this for the brew installation. I looked into what can be done for a global install via pip but I can recreate the same behaviour via the terminal after activating the virtual environment.
I've been looking into how other extensions do this again and have found the following issues/discussions:

https://github.com/microsoft/vscode-python/issues/11039
https://github.com/microsoft/vscode-python/discussions/18978
https://github.com/guildai/guildai-vscode/issues/1
https://github.com/brettcannon/python-launcher/discussions/168

This seems to be an area that the vscode-python team is actively working on. We should keep an eye on those discussion tickets and see where they go. If they could expose the activation command in their API that would be ideal. Maybe we can get the correct PATH details from EnvironmentVariableCollection in the extension context.

Hi! I was directed here by @shcheklein. I am running into a problem with the DVC extension and virtual envs.
I have a codebase, and set up a dvc.yaml pipeline, which works. I tried dvc repro command (with the activated conda env) and everything succeeded. Then, I wanted to run an experiment and used dvc exp run, which also succeeded. Unfortunately, that is not the case when using the VS Code extension.
Then, I am getting an error ModuleNotFoundError: No module named 'src', which is weird as the codebase did not change between repro and exp run.
A potential culprit might be the pythonpath setting. In VS Code, I have an .env file with PYTHONPATH=src and in settings.json I have the following: "python.envFile": "${workspaceFolder}/.env". I added is as a suggestion from one of the users of DVC discord chat, but unfortunately it did not solve the problem.
To sum up, I am using macOS and working with a conda environment.
Any ideas about what might be the source of the issue and how to solve it?

Another related  issue that pops up while playing around with the extension is that once I try using the experiments tab (it does not work as mentioned above) to run an experiment, I can't run them anymore from the terminal using dvc exp run. I need to run export PYTHONPATH=$PWD in order to be able to run the experiments from the terminal again. This hints that the extension is unsuccessfully trying to set the path.

@erykml

I can't run them anymore from the terminal using dvc exp run

Does it depend on whether extension is active or not?
Could you also please share the project layout and how do you run your script usually? I'm curios why do we need to do PYTHONPATH at all tbh. There are ways to do something like python train.py w/o providing PYTHONPATH.

@shcheklein
After I tried running the dvc exp run command in the integrated terminal of VS Code after the failed run from the extension, then I am getting the error. I have killed the extensions terminal window by then. Does this answer your question?
The project's structure is as follows:
# the structure of the project

📦vscode_exp_tracking_with_dvc
 ┣ 📂.dvc
 ┣ 📂data
 ┃ ┣ 📂processed
 ┃ ┣ 📂raw
 ┃ ┣ 📜.gitignore
 ┃ ┗ 📜raw.dvc
 ┣ 📂models
 ┣ 📂src
 ┃ ┣ 📂stages
 ┃ ┃ ┣ 📜data_split.py
 ┃ ┃ ┣ 📜eval.py
 ┃ ┃ ┗ 📜train.py
 ┃ ┗ 📂utils
 ┃ ┃ ┣ 📜__init__.py
 ┃ ┃ ┗ 📜params.py
 ┣ 📜.dvcignore
 ┣ 📜.gitignore
 ┣ 📜README.md
 ┣ 📜dvc.lock
 ┣ 📜dvc.yaml
 ┣ 📜metrics.json
 ┣ 📜params.yaml
 ┗ 📜requirements.txt

And normally I run a script as python src/stages/train.py.

Got it.

This hints that the extension is unsuccessfully trying to set the path.

This is interesting. We'll check.

And normally I run a script as python src/stages/train.py.

thanks, it makes sense. Why do you need to use PYTHONPATH? I think you can import modules / scripts properly in the train.py so that PYTHONPATH is not required?

Currently, I import the modules/scripts as follows:
import pandas as pd
from lightgbm import LGBMClassifier
from sklearn.ensemble import RandomForestClassifier

from src.utils.params import load_params

@SoyGema could you please also add a summary of the road block that you hit with the env vars / PYTHONPATH?

CONTEXT : Reproducing the experiments for workshop , with dvc.yaml and params.yaml .
When executing dvc exp run train , the CLI is throwing the error:
Running stage 'train':
> python  ExperimentsDVC/src/models/train_model.py
Traceback (most recent call last):
  File "/path/ExperimentsDVC/src/models/train_model.py", line 2, in <module>
    from src.data.load_dataset import read_data
ModuleNotFoundError: No module named 'src'
ERROR: failed to reproduce 'train': failed to run: python  ExperimentsDVC/src/models/train_model.py, exited with 1

That leads to set PYTHONPATH in CLI  with  export PYTHONPATH = 'path/ExperimentsDVC'
With this, dvc experiments is able to take the absolute path and train the experiment , showing metrics in table.
Then, pressing left button and executing Modify and Run  UI feature , and changing some params , the following message is shown.
>>>> DVC Terminal >>>>

Running: dvc exp run -S params.yaml:train.n_estimators=150 -S params.yaml:train.max_depth=25 -S params.yaml:train.min_samples_split=10 -S params.yaml:load.data_path_train=ExperimentsDVC/data/raw/jan_train.csv -S params.yaml:load.data_path_test=ExperimentsDVC/data/raw/jan_test.csv -S params.yaml:featureselection.data=ExperimentsDVC/data/raw/jan_train.csv -S params.yaml:featureselection.features=["id","sat_id","x_sim","y_sim","z_sim","Vx_sim","Vy_sim","Vz_sim","Day","Hour","Minute"] -S params.yaml:featureselection.labels=["x","y","z","Vx","Vy","Vz"] -S params.yaml:splitdata.size=0.33 -S params.yaml:splitdata.state=42 -S params.yaml:standardscaler.data=null -S params.yaml:train.min_impurity_decrease=1

Running stage 'train':
> python  ExperimentsDVC/src/models/train_model.py
Traceback (most recent call last):
  File "/path/VSCode-DVC-Workshop/ExperimentsDVC/src/models/train_model.py", line 2, in <module>
    from src.data.load_dataset import read_data
ModuleNotFoundError: No module named 'src'
ERROR: failed to reproduce 'train': failed to run: python  ExperimentsDVC/src/models/train_model.py, exited with 1

Which seems similar to the previous error , which might lead to "assume" or "hypothesize" that the Workspace UI path config doesn´t relate to defined PYTHONPATH in CLI
CONFIG
Virtual env with requirements . Setup Workspace with DVC:Setup The Workspace  > Auto . Bottom Left DVC Circle UI complete and bottom right Python seems correct 3.9.13('env': venv)

Looks like PYTHONPATH is now exposed by the Python Extension: https://github.com/microsoft/vscode-python/blob/main/src/client/api.ts#L80

This is however marked as deprecated so we need to wait for it to get moved into stable before we go ahead and use it: https://github.com/microsoft/vscode-python/blob/main/src/client/api.ts#L30


cardano-wallet-js

Name: cardano-wallet-js
Description: a javascript/typescript SDK for Cardano Wallet with a several functionalities. You can use it as a client for the official cardano-wallet and also to create Native Tokens and NFTs.
Link:  https://www.npmjs.com/package/cardano-wallet-js
Source: https://github.com/tango-crypto/cardano-wallet-js
Tags:

opensource
metadata
nativetoken
nft
wallet

Hello, if this should replace #213 then please close #213.

The right one is #213


What is the path I need to use to import Plekan into my project?

I am having trouble importing plekan into my project.
After npm install plekan --save,

I tried to import it three different ways:
import plekan from 'plekan'
import plekan from './plekan.js'
import {editor} from './plekan.'

but I get this error, or the equivalent every time:
Module not found: Error: Can't resolve 'plekan'

Hi @retrogradeME, I'm sorry to say that we forgot to remove the docs line from .npmignore. Reinstalling plekan version 1.2.2 will solve the problem.


Tidy up where files come from

@coretl @GDYendell
So I'm finding it confusing that the files that are crucial to getting a fully working IOC instance are coming from a variety of places.

ibek itself
scripts and templates from epics-base
scripts and templates from ioc-XXX
YAML and helm from beamline repos
more helm from the helm library
things from ibek-support
CI scripts in all of the repos involved

When you are in a container and you find you have the wrong version of something or you want to reset something it may not be easy or clear where it comes from. I say this as the primary author of this framework! Goodness knows what a neophyte containers engineer will make of it all.
Some ideas for things that can be embedded in ibek itself are listed below. These would then always be a consistent set. (adding to this as I think of things)

global configure/RELEASE initial state epics-base
Makefile Jinja template epics-base
boilerplate helm chart for beamline/domain repos blxxy
boilerplate ioc folder structure epics-base
these could become functions of ibek

start.sh, liveness.sh stop.sh ioc-xxx
script for preparing the runtime target by copying module output files only epics-base
many of things that we do in CI (because this repeats knowledge about URLS and file locations etc.) all over the place

Now you might say this is hiding many things inside a python module. At present the scripts can be inspected and possibly customised. I think good customisation hooks would mitigate this.
But the benefit is you should get a consistent, working set of things versioned along with ibek itself.
It would give us clear(ish) separation of responsibility as follows:

epics-base is just a compilation of epics in a container: it will need to set a bunch of env vars to say where things are (/epics/epics-base /epics/support etc.) but that is all and hopefully only ibek needs to consume those env vars
ibek manifests all the rest of the things that enable epics-containers framework, EXCEPT:-

beamline repos supply a list of IOC.YAML only , uses ibek to publish them to registry in CI
ioc-XXX repos list a sequence of support modules to install uses ibek/ibek-support to compile them (mostly done already)
ibek-support provides

means to compile the support modules above
means to instantiate the IOC.YAML from beamline repos

helm chart library contains the common functions in the helm charts because its helm (but you could even embed that in ibek if you wanted to go the whole way)

Maybe on reflection we should keep ibek only for inside container things. So that eliminates the CI bits - my last bullet.
Perhaps we get ec to do some of the CI work though.
I'm also pretty sure we will find a few areas of overlap of configuration inside and outside the containers.

But another possibility is we go all in on ibek and accept outside of container into it - roll in ec, helm and CI things.

placeholder

ALSO: we currently mount the ioc-adaravis repo at /workspaces/ioc-adaravis in the devcontainer which is the default for devcontainers.
This is annoying for the scirpts and Dockerfile because the location of the source and in particular the ibek-support sub module varies from one GIOC to the next.
I propose that we always mount to:

this_repo_root:/ioc-src
this_repo_root/ioc:/epics/ioc

WIth the changes in the above comment epics/ioc would then be a compilable thing that you can fix/run and have your changes propgated back to the repo automnatically.

@coretl I propose that we change the ibek 'startup' command namespace to 'runtime'

runtime generate
runtime launch
runtime check-live
runtime stop

sounds fine

PROGRESS pm ibek change:

[ ] Remove all IOC things from epics-base
[x] ibek ioc make-source-template
[ ] create a new /epics/source/configure/RELEASE
[x] ibek ioc extract_runtime_assets
[ ] ibek runtime launch
[ ] ibek runtime check-live
[ ] ibek runtime stop
[x] fix mount locations to simplify runtime / devcontainer compatibility

I've addressed all of the above in #115.
Note that start.sh, stop.sh and check-live.sh are still scripts. But they now live as templates inside ibek at
https://github.com/epics-containers/ibek/tree/dev/src/ibek/templates/ioc
Looking at them I'm not sure of the benefit of converting to python. But this is an easy thing to add later if we think it makes sense.

I'm pretty happy with ec as well now.
Final tidy up is:

CI framework for process.run based commands in ibek
as above for ec (all the commands in ec in other words)

EC CI is done and its a reasonable framework for testing process.run functions via YAML!
TODO - add the same framework for testing the ibek process.run based commands.

broken the last remaining issue in here out into #137


Understanding PgPool-II architecture

Understanding PgPool-II architecture is the first thing to hack PgPool-II for AGE

Hello everyone!
I am looking forward to working on this project!

Hello, Moeed here. I'll be working on this task.

Hi, I am the new Operations Manager Jayden from Vancouver Team.

Hello, I'm Zahid and I'll be working on this project.

Hi, will be working on this project.

Hi Guys
Excited to work on this project.

Hi! I'm excited to work on this project!

Hi! I am Abdullah. Excited for this project !!

Hi! I am Waleed, excited to work on this project

PgPool-II architecture provides a robust and flexible architecture for scaling PostgreSQL deployments, improving performance, and ensuring high availability. Administrators can optimize database performance, enhance fault tolerance, and streamline management of PostgreSQL clusters.

@eyab  project was completed, issue can be closed.


Error in the code

The signal "chunk_rendered" in ThreadedTilemapGaeaRenderer doesn't return chunk_position

Love the addon btw :3

Thanks for letting me know!


Questions about tools

Hi.
Does the CORScanner tool support all the features of the "CORStest" tool?
What's the difference?

Thanks for the question.
This tool covers most of the CORS misconfiguration types in the CORStest tool, except for some types that are difficult to be exploited.  You can find them in the misconfiguration types section of the README.md.
The tool is originally developed for my large-scale measurement research(scanned 97,199,966 subdomains). This tool is faster and supports more customized features, e.g., file output, self-defined headers.

@chenjj  Very Thanks :+1:


Cordova requirements check fails

Looks like I need to manually set some env variables. I think we should add this to the README instructions. Here is the error:
$ cordova requirements
Requirements check results for android:
Java JDK: installed 1.8.0
Android SDK: not installed
Failed to find 'ANDROID_HOME' environment variable. Try setting setting it manually.
Failed to find 'android' command in your 'PATH'. Try update your 'PATH' to include path to valid SDK directory.
Android target: not installed
android: Command failed with exit code ENOENT
Gradle: not installed
Could not find gradle wrapper within Android SDK. Could not find Android SDK directory.
Might need to install Android SDK or set up 'ANDROID_HOME' env variable.
Error: Some of requirements check failed
┌──────────────────────────────────────────┐
│ Update available: 7.0.1 (current: 7.0.0) │
│ Run npm install -g cordova to update.    │
└──────────────────────────────────────────┘

Hi @rhagarty I had added a line in README.MD (just after adding the platform) (Check if you satisfy requirements for building the platform here) . However, I have added that line in the Pre-requisites section as well.