Rclone , Not running by crontab

When ever I am scheduling a rclone copy by crontab , I am getting  this error
Failed to create file system for "GDrive:/TestingMain": Couldn't read info about Drive: Get https://www.googleapis.com/drive/v2/about?alt=json: dial tcp: lookup www.googleapis.com on <IP_ADDRESS>:53: no such host
Scenario:

Crontask under root
Crontask to trigger rclone to copy a file to the following path in google drive "GDrive:/TestingMain"

Which OS are you using?
It looks like that DNS resolution isn't working under cron.
Does it happen every time, or does it work sometimes?
Do you have any name resolver environment variables set normally, eg RES_OPTIONS - these won't be set under cron.

Did you get anywhere looking at this?

Hi Nick,
I am using Fedora.
May be DNS Resolution is not working under cron.
It is happening when I run through cron only.. Everytime not running
through cron.
I did not set any name resolver environment variables.
Any solution ??
On Sat, Jan 2, 2016 at 6:22 PM, Nick Craig-Wood<EMAIL_ADDRESS>wrote:

Did you get anywhere looking at this?
—
Reply to this email directly or view it on GitHub
https://github.com/ncw/rclone/issues/260#issuecomment-168388712.

--
Thanks and Regards,
Sumanth Nagendla,
Contact no:<PHONE_NUMBER>6.

Did you manage to make this work?
You might get more help posting to a fedora forum - I don't think it is an rclone problem specifically.

I think this isn't an rclone bug so I'm going to close this ticket.
I'm sorry I haven't been able to help you work out what is going on - you might get more help from a general Fedora / linux forum.
Please re-open if you do discover it is an rclone bug.
Thanks
Nick

sorry, i have the same problem. I think that running inside cron, rclone is not able to read any (default) conf file, so it should be enough to use  --config /root/.rclone.conf ? can it be ?

Sumanthsas3: try with setting env var GODEBUG=netdns=go

@mejje good suggestion.  The next release of rclone v1.30 will be compiled like that by default, so it might be worth retrying with that.


Slice final states

in the t2v.py
Slice final states
l_f_source = lasagne.layers.SliceLayer(l_fgru_source, -1, 1)
why didn't use only_return_final =True?
l_fgru_source = lasagne.layers.GRULayer(only_return_final=True)

Ah, I was not aware of this feature. I guess you could do that too.


actions, publish: Push the main core image

After a PR is merge the kiagnose core image has to be pushed so users
can consume it. This change push the core kiagnose image on PR merges at
quay.io/kiagnose/kiagnose:main.
Notes for reviewers:
This cannot be tested on the PR since secrets are not passed to jobs running at PRs.
Tested with act

This also needs to be limited/filtered per changes in the cmd and kiagnose folders only. If possible, please excluding tests files will also be nice (but not a must).

This also needs to be limited/filtered per changes in the cmd and kiagnose folders only. If possible, excluding tests files will also be nice (but not a must).

Not really needed, the time it takes does not affect PRs CI and the image sha is going to be the same so pushing will not overwrite anything.

Not really needed, the time it takes does not affect PRs CI and the image sha is going to be the same so pushing will not overwrite anything, is better to just delegate container hash comparation on pushing.

Are you sure the build artifact is not affected? Remember that there is no cache between the runs, unlike when run locally.
Anyway, we can take it in and examine as it goes.


iptables: Limit smokescreen port 4750, add camo port.

Limit incoming connections to port 4750 to only the smokescreen host,
and also allow access to the Camo server on that host, on port 9292.

Merged, thanks @alexmv, and also posted a comment for future work :)


dealing with multi-class label

Hello,
I would like to train a sequence tagging model on potentially sensitive data. Right now we have collected some annotations on a text corpus, but each sentence just has one observation from the crowd. So my idea is to write some pattern based labeling function and integrate it with the crowd annotation to come up with a denoised version of the training set. I went through the tutorials but found out they are only for binary classification problems.
In this post (https://github.com/HazyResearch/snorkel/issues/599), I notice there is an upcoming paper on entity recognition, which would be very relevant to what I would like to have for the labeling function. So I write this post to check with you guys if there is something somewhere that can help me to come up with LFs which can annotate different kinds of entities in a sentence.
I am also wondering if there are some pointers available to show how to integrate LFs with crowd labels. Thanks!

Hi @chentc ,
Overall, sounds like a great plan to combine crowd labels + other pattern-based LFs!
Snorkel does in fact have full support for categorical classification- see this tutorial for example!  We also have a tutorial for a crowdsourcing here!
As for more stuff on NER tagging in particular, we will be adding some stuff from recent research (see @jason-fries ) at some point soon, will keep you posted
Thanks,
Alex

Closing for now--will be accessible via the "Q&A" link in README--but feel free to re-open!


✨ Need a webrtc frame processing plug-in.

What feature or enhancement are you suggesting?
Cooperate with react-native-vision-camera to make videos.
What Platforms whould this feature/enhancement affect?
iOS, Android
Alternatives/Workarounds
not yet.
Additional information

[X] I agree to follow this project's Code of Conduct
[X] I searched for similar feature requests in this repository and found none.

Sure, let me know when you build one and I can add it to the "Community Plugins" section in the docs :)

hello tell me how  can we app webrtc in  react-native-camera-vision thanks

Read the Frame Processors docs.

thanks

Read the Frame Processors docs.

Hey, I didn't find any solution, regarding WebRTC implementation, could you share the example if possible, it may help to lots of developers.

@salmanrajz did u found any solution

@salmanrajz did u found any solution

Not yet

@salmanrajz I am trying to integrate with WebRTC, here is the way I am trying to:

Create custom capture as global static class
Add it's into WebRTC videoCapturer
Inside frame processor, access to capture delete with new Frame

if let stream = webrtcModule.stream(forReactTag: localStreamTag){
      if let videoTrack = stream.videoTracks.first{
        if let videoCapturerController = videoTrack.captureController as? VideoCaptureController{
          videoCapturerController.capturer.stopCapture{
            let visionCapturer = VisionCameraCustomCapturer(delegate: videoTrack.source)
            let newVideoCapturer = RTCCameraVideoCapturer(delegate: visionCapturer.delegate!)
            let newVideoCapturerController = VideoCaptureController(capturer: newVideoCapturer, andConstraints: nil)
            videoTrack.captureController = newVideoCapturerController
            newVideoCapturerController?.startCapture()
            PipInstance.shared().visionCapturer = visionCapturer
          }
        }
      }
    }

@objc(VisionFrameProcessorPlugin)
public class VisionFrameProcessorPlugin: NSObject, FrameProcessorPluginBase {
  @objc public static func callback(_ frame: Frame!, withArgs _: [Any]!) -> Any! {
    guard let imageBuffer = CMSampleBufferGetImageBuffer(frame.buffer) else {
      return nil
    }
    let filter = YUCIHighPassSkinSmoothing()
    filter.inputImage = CIImage(cvImageBuffer: imageBuffer)
    filter.inputAmount = 0.8
    if let buffer = filter.outputImage?.pixelBuffer{
      if let newCmSampleBuffer = buffer.toCMSampleBuffer1(){
        print("newCMSamplBuffer: \(newCmSampleBuffer)")
        if #available(iOS 15.0, *) {
          if let visionCapturer = PipInstance.shared().visionCapturer{
            print("visionCapturer: \(visionCapturer)")
            let rtcPixelBuffer = RTCCVPixelBuffer(pixelBuffer: buffer)
            let frameTime = CMSampleBufferGetOutputPresentationTimeStamp(frame.buffer);
            let frameTimeMillisecs = Int64(CMTimeGetSeconds(frameTime) * 1000);
            var rotation = RTCVideoRotation(rawValue: 0)
            switch(frame.orientation){
            case .left:
              rotation = RTCVideoRotation(rawValue: 90)
            case .down:
              rotation = RTCVideoRotation(rawValue: 180)
            case .right:
              rotation = RTCVideoRotation(rawValue: 270)
            default:
              rotation = RTCVideoRotation(rawValue: 0)
            }
            let newVideoFrame = RTCVideoFrame(buffer: rtcPixelBuffer, rotation: rotation!, timeStampNs: frameTimeMillisecs)
            print("newVideoFrame: \(newVideoFrame)")
            visionCapturer.delegate?.capturer(visionCapturer, didCapture: newVideoFrame)
          }
        }
        return Frame(buffer: newCmSampleBuffer, orientation: frame.orientation)
      }
    }else if let buffer = filter.outputImage?.toCVPixelBuffer(){
      if let newCmSampleBuffer = buffer.toCMSampleBuffer1(){
        if #available(iOS 15.0, *) {
          if let visionCapturer = PipInstance.shared().visionCapturer{
            print("visionCapturer: \(visionCapturer.delegate)")
            let rtcPixelBuffer = RTCCVPixelBuffer(pixelBuffer: buffer)
            let frameTime = CMSampleBufferGetOutputPresentationTimeStamp(frame.buffer);
            let frameTimeMillisecs = Int64(CMTimeGetSeconds(frameTime) * 1000);
            var rotation = RTCVideoRotation(rawValue: 0)
            switch(frame.orientation){
            case .left:
              rotation = RTCVideoRotation(rawValue: 90)
            case .down:
              rotation = RTCVideoRotation(rawValue: 180)
            case .right:
              rotation = RTCVideoRotation(rawValue: 270)
            default:
              rotation = RTCVideoRotation(rawValue: 0)
            }
            let newVideoFrame = RTCVideoFrame(buffer: rtcPixelBuffer, rotation: rotation!, timeStampNs: frameTimeMillisecs)
            visionCapturer.delegate?.capturer(visionCapturer, didCapture: newVideoFrame)
          }
        }
        return Frame(buffer: newCmSampleBuffer, orientation: frame.orientation)
      }
    }
    return frame
  }
}

I think it's will work, could you make it work with me?


S. aureus MLST

Hi there,
I'm a Staph researcher (Cassat Lab at Vanderbilt) - excited about your database! I was planning on culling the list to ST8 only for my own research, but that involves me MLST-ing all 121k!
Alright if I contribute?
Cheers,
Andrew

Sorry for slow reply! You 100% welcome to contribute,  v glad that is your reaction! But let me check with the staph folk, who nay have done it already.  We have a whole bunch of completed analyses queued up for release

Sounds good - ran a test pipeline on 8000 and everything is working smoothly. Planning on getting the rest completed this week, but if you already have the data, no worries!


Update tdr-generated-graphql to 0.0.276

Updates uk.gov.nationalarchives:tdr-generated-graphql from 0.0.274 to 0.0.276.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "uk.gov.nationalarchives", artifactId = "tdr-generated-graphql" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "uk.gov.nationalarchives", artifactId = "tdr-generated-graphql" }
}]

labels: library-update, early-semver-major, semver-spec-patch, commit-count:1

Superseded by #2316.


VB sample added

Contributes to dotnet/dotnet-api-docs#990

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: arjanvanhuizen sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Some changes

VM Create Added
VM Pause Added
VM can be created with CDROM(iso image for boot).

Thanks for the PR - could you please use gofmt to fix the CI?
The command "gofmtresult=$(gofmt -s -l .); if [[ -n $gofmtresult ]]; then echo -e "Please run \"gofmt -s -w .\" before committing for the below:

@rdobson OK, I think it works now


Allows reporter to share the same id for different resources

This is something i found while looking on the storage part of inventory
A reporter might send different types of resources - if they
are using an incremental id or if 2 different resources
share an id, the reported data for the resource will only be stored once.
With this commit, it ensures that we store the reporter once
for each resource, thus each one having it's own reporting data
and console/api hrefs
PR Template:
Describe your changes

...

Ticket reference (if applicable)
Fixes #
Checklist

[ ] Are the agreed upon acceptance criteria fulfilled?

[ ] Was the 4-eye-principle applied? (async PR review, pairing, ensembling)

[ ] Do your changes have passing automated tests and sufficient observability?

[ ] Are the work steps you introduced repeatable by others, either through automation or documentation?

[ ] If automation is possible but not done due to other constraints, a ticket to the tech debt sprint is added
[ ] An SOP (Standard Operating Procedure) was created

[ ] The Changes were automatically built, tested, and  - if needed, behind a feature flag - deployed to our production environment. (Please check this when the new deployment is done and you could verify it.)

[ ] Are the agreed upon coding/architectural practices applied?

[ ] Are security needs fullfilled? (e.g. no internal URL)

[ ] Is the corresponding Ticket in the right state? (should be on "review" now, put to done when this change made it to production)

[ ] For changes to the public API / code dependencies: Was the whole team (or a sufficient amount of ppl) able to review?

No longer makes sense with the new db schema


Bump black version and relax constraint

Summary
See Qiskit/qiskit-terra#7615
Details and comments

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

5 of 5 (100.0%)  changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.09%) to 83.898%

Totals

Change from base Build<PHONE_NUMBER>:
0.09%

Covered Lines:
2850

Relevant Lines:
3397

💛  - Coveralls


Any chance to get a Dockerized version of Kin server and client?

It could be nice to provide a docker-compose.yml file ready-to-go for simple installation.

Not planned in the short-term (next couple of weeks).
Though Kin has been fairly stable infra/arch-wise in the last couple of months, a few changes are coming to support push sync with providers (Kin Client <-- WebSockets --> Kin Server <-- Webhooks -- Providers) and smarter notifications between the desktop clients and mobile apps (dedup notifications for the same events ... )
That said, if anyone is interested and wants to train his Docker skills on a real world example, I will be more than willing to help out / steer them in the right directions / answer questions on Kin.

@zowesiouff I would be down for this one – already built "a few" images in the past. A huge relief would to provision the configuration variables from the environment variables. Best case would be to have absolutely no persisted file for storing the configuration (server > secrets.js, server > config.js and client > config.js ==> ENVs).
So instead of mapping NODE_ENV to different configurations, providing them in an .env file, a preconfigured Dockerfile or even a docker-compose.yml.
Additionally we could automate the SSL certificate creation for development environments in the respective Dockerfiles (and even use Letsencrypt for production use cases).

Creating the images is way above my Docker league. Although usefully using them would be a great step for me to get started with docker.
I'm interested in the results of this.

I would be down for this one – already built "a few" images in the past. A huge relief would to provision the configuration variables from the environment variables. Best case would be to have absolutely no persisted file for storing the configuration (server > secrets.js, server > config.js and client > config.js ==> ENVs).
So instead of mapping NODE_ENV to different configurations, providing them in an .env file, a preconfigured Dockerfile or even a docker-compose.yml.

@beevelop This could certainly be done, though I'm not completely sure about the security risks of passing secrets through environment variables? Can't we use volumes to minimise the amount of changes? (at least to get a dockerized version out as early as possible?)
Though offering a way to more easily deploy Kin is definitely great, I would ideally like to decouple it as much as possible from the "core" repositories. (There is already PM2 stuff in the server that I would like to get rid off)

In regards to security, passing them in using a volume would be just as secure as passing them in using an environment variable. Both are still readily available if someone gains access to either the container or the host machine. One approach I've used in the past on client work has been to use Hashicorp Vault, and use wrapped tokens; this is where you pass in a single-use, short-lived token that only allows you to unwrap the "real" authentication token for Vault, which then lets you extract secrets from Vault and keep them in-memory.
In any case, having the configuration be handled with environment variables would be very nice indeed. In fact, if it were the case, I'm sure an image would already exist for Kin using them.
Docker does actually have support for "secrets", but I've not used this functionality yet, or how seen how it can help make anything more secure than volumes / env vars.

Both are still readily available if someone gains access to either the container or the host machine

Well in both cases, you're (almost) dead anyway :) a good advantage of files is that you directly inherit a lot of stuff to restrict access to them (but I'm not sure those carry over well to docker volumes tbh). What I find bad in env variables is that any deps / child process can (and will) dump env in logs even without intending to "steal" your secrets, you end up with a leak.
Anyway, I will dig a bit more, but I'm really reluctant to go the env way right now. Sadly Docker Secrets means we're highly coupling Kin to Docker :/

Both are still readily available if someone gains access to either the container or the host machine

Well in both cases, you're (almost) dead anyway :) a good advantage of files is that you directly inherit a lot of stuff to restrict access to them (but I'm not sure those carry over well to docker volumes tbh). What I find bad in env variables is that any deps / child process can (and will) dump env in logs even without intending to "steal" your secrets, you end up with a leak.
Anyway, I will dig a bit more, but I'm really reluctant to go the env way right now. Sadly Docker Secrets means we're highly coupling Kin to Docker :/

@zowesiouff Even though using environment variables is considered good practices in container land, I definitely get your point. What do you think about propagating the necessary settings from an  .env-file (e.g. via the dotenv package)?
IMHO this would combine the best of both worlds and should simplify the configuration management drastically.

@zowesiouff @welcoMattic @robinvalk @SeerUK I spent some time to get an initial dockerized version of Kin running. Have a look at https://github.com/beevelop/docker-kin
Instead of managing the certificates on your own, the current implementation automatically generates self-signed certificates on launch. The real certificate management is delegated to Traefik. Traefik has built-in support for Let's Encrypt and automates the whole certificate pain; all this by maintaining the same security measures as all traffic between Traefik and the Kin containers is still secured (self-signed certs).
@zowesiouff to get the images stable and potentially production-ready it would be awesome to get both repositories equipped with proper tags / releases. The config files (server and client) are coupled to the current master versions. As a compromise for the ENV discussion, we might add a default 'env' environment to config.js, etc. which gets its values populated from the respective process.env.
Feedback is greatly appreciated.

I've just tested your docker-kin-web and docker-kin-server, nice work @beevelop! I will test the meta repo using compose once I get a bit more time to dig into Traefik (never used it before).

Feedback is greatly appreciated.

Any reason for going with ubuntu? vs a more "lightweight" base (Alpine 😍  )?
Any reason for having Redis run in the same container as pm2?
You should update NODE_ENV to prod in the server image

Even though using environment variables is considered good practices in container land, I definitely get your point. What do you think about propagating the necessary settings from an .env-file (e.g. via the dotenv package)?

The config files (server and client) are coupled to the current master versions. As a compromise for the ENV discussion, we might add a default 'env' environment to config.js, etc. which gets its values populated from the respective process.env.

As I said, I'm not fond of putting secrets in ENV variables. But for config like API_HOSTNAME, STATIC_HOSTNAME etc ... it would definitely be something I would merge in Kin, considering they are already public information, making their leaking a non-issue.
For secrets, sadly I haven't found any interesting package that offers a nice agnostic API with pluggable backends yet (Docker Secrets, Kubernetes Secrets, fs, ... ). For now, our best bet is probably to go with ENVs but I would ideally like to have a plan for getting rid of it before going further.

to get the images stable and potentially production-ready it would be awesome to get both repositories equipped with proper tags / releases.

I have no experience with publishing images to Docker Hub. What's involved in getting something out? Is it easy to automate the publication (In TravisCI?)? Would publishing kin to npm make it any easier? Anyhow, it's definitely a good reason to start a proper versioning scheme. I'll have a tag/release out.

Ubuntu instead of Alpine: Personal preference / my whole Docker image hierarchy is based on Ubuntu Xenial. Might be pretty easy to port both images to Alpine; so I do agree to either switch completely or provide a respective tag (like :alpine-x.x).
Redis included: Convenience / simplification for now. On the roadmap for the production-ready version.
NODE_ENV=dev: On the roadmap too.
Using ENVs for configuration: Using dotenv should work fine and reflects most of your security concerns. Proof me wrong, but there is no significant difference in storing them inside static *.js files. Both can be properly chmodded to prevent unauthorised access. It would just help to uncouple the configuration from the required data structure.
Docker Hub Automation: Docker Hub can be integrated with GitHub as smooth as Travis. There are so called "Automated Builds" on Docker Hub which get triggered via Webhook. You can define rules to e.g. reflect the git branches and git tags as image tags. By default you just have to provide a Dockerfile in the repositories' root folder.

Proof me wrong, but there is no significant difference in storing them inside static *.js files.

requoting myself:

What I find bad in env variables is that any deps / child process can (and will) dump env in logs even without intending to "steal" your secrets, you end up with a leak.

To give you an example of what I'm thinking about: any library used by Kin can leak the envs without "intending" to leak Kin's secrets: util.inspect(process.env) or console.log(process.env) which are very common patterns in "debug" modes of many libraries. Whereas dumping/accessing the secrets stored in a file require intent.
Anyway - as I said - I couldn't find a proper solution that would be able to talk agnostically to any "secrets" backend :/

Well, I get your point, but there is no need to keep the sensitive configurations stored in the process.env variable after initialization. We could sanitize process.env properly to prevent unwanted log / debug leakage. Or: Just not dump them into process.env in the first place (dotenv is an inflexible example in this case).
Let's consider e.g. .ini, .yml or even .json files as an alternative (might be even more flexibel than .env files). Just to clarify: I'm not proposing to use ENVs (anymore), but to decouple the code from the configuration. This should simplify managing the configuration now and in the future (in case of upgrades, breaking changes,etc).

Well, I get your point, but there is no need to keep the sensitive configurations stored in the process.env variable after initialization. We could sanitize process.env properly to prevent unwanted log / debug leakage.

Yes unsetting the various secrets definitely would limit an awful lot of leakages :)

Or: Just not dump them into process.env in the first place (dotenv is an inflexible example in this case).

I'm not sure I understand "inflexible example"? Do you mean that dotenv shouldn't be used in this case - because we would need to extend it by unsetting parts of process.env after it ran?

Let's consider e.g. .ini, .yml or even .json files as an alternative (might be even more flexibel than .env files). Just to clarify: I'm not proposing to use ENVs (anymore), but to decouple the code from the configuration. This should simplify managing the configuration now and in the future (in case of upgrades, breaking changes,etc).

Technically it's partially "decoupled" [0] and it's even more flexible than a simple "data" format because it can execute arbitrary code, we can see config.js as just a "config example" file checked in (which is the way kin.today actually runs right now: config.js is overwritten by files coming from our own config management)
That said, I'm not sure I understand how using an other file format would make it easier for the Docker use case? You still need a way to add that config file either in the Docker image (by rebuilding), or via Docker Volumes?
[0] there is definitely an argument to be made to remove rp, rollbar_config and log4js_options ;)

Just to clarify, this is the plan I've in mind after re-reading this entire issue:

scrap rp, rollbar_config (I'll probably completely scrap Rollbar from the OSS version) etc ... from the config files as they are cruft
in config.js/secrets.js files: load values from process.env to make them easily configurable when running the base Docker images, and unset all keys used from process.env

This has the added advantage that the API between configuration/secrets and the rest of Kin's codebase is the same and extremely flexible (it's a JS modules exporting a bunch of constants :) ).
Or am I missing something?
Thanks a lot for your effort on this @beevelop, it's greatly appreciated :)


Vite overwrite the Access-Control-Allow-Origin from the CORS module

Package version
3.0.0-8
Describe the bug
Hello,
I'm using the CORS module this the setting origin to true. In the same time, I'm fetching my server, from another domain, with the withCredentials to true. The browser use a preflight request but the server return * instead of the domain that fetch the server. This cause an issue because it's forbidden.
similar to o https://github.com/nuxt/nuxt/issues/23962
Reproduction repo
No response

Should be resolved with https://github.com/adonisjs/vite/pull/14 and so version 3.0.0-11 !
But beware, we have a breaking change, see https://github.com/adonisjs/vite/releases/tag/v3.0.0-11


[DO NOT MERGE] Create brandon_interface.md

Moving to /docs as the github issue is getting too large
Do not merge until I am finished writing this

@all-contributors please add brandonskerritt for design, maintenance, code, ideas

@all-contributors please add brandonskerritt for design, maintenance, code, ideas, data


Network azure service

Network azure service

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 179780aa-4f46-5ea8-80ec-b3e003ffbb7b
Version Independent ID: fd5ec122-d4d8-5ec6-d07b-60e43e7b9362
Content: Azure networking services overview
Content Source: articles/networking/fundamentals/networking-overview.md
Service: virtual-network
GitHub Login: @mbender-ms
Microsoft Alias: mbender

@bhashas
Thanks for your feedback! We will investigate and update as appropriate.

@bhashas I am sorry, can you please provide more context on why you have raised this Github issue?

@bhashas we are closing this issue as of now. Once you are available to work with us, please tag me and let us know your issue, we shall reopen this and we can work on this issue


pdf2json Warning: TT: undefined funciton: 32

On my system:
Linux lje-DL 4.15.0-43-generic #46-Ubuntu SMP Thu Dec 6 14:45:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
when the pdf2html.js is run, the 1st line shows a warning:
evansl@lje-DL:~/prog_dev/pdf/npm/pdf-parser/pdf-parser/example/node$ node pdf2json.js|head -1
Warning: TT: undefined function: 32
I tried to use node inspect ...; however, that was mostly useless because I kept having
to press s or n and I could not make sense of what was happening.
Please provide some guidance about how to debug the problem of a solution
to the problem.
TIA.
-regards,
Larry

Explaination can be found here : https://github.com/mozilla/pdf.js/issues/3768#issuecomment-36468349
But still the "how to resolve this" is not necessarily clear (make PDF-A ?)

Hello! Just had this same problem. Making it a PDF-A helps, but there's also a probable more accurate way if you can use Acrobat. In the Edit menu go to Preflight and in PDF fixups choose Fix potential font problems. Worked perfectly in my broken PDF.

this is an awesome library, I used in my project, but this is only for the English language, It will be great If we provide some language option in it, like some of the PDF's are the combination of multiple languages at that time it creates problem while using. other than that this is awesome.


sidebar slide animation

The sidebar appears and disappears when toggled. But it doesn't have the nifly slide animation like on the main halfmoon site.
Is this something I need to roll myself or could it be a configuration error on my part?

Did you include halfmoon.js in your project? Right now, there is a small issue where the .with-transitions class is attached to the page wrapper using JavaScript. This class actually provides the transition you are looking for.
This was done to avoid a really annoying bug on Chrome that runs transitions on page load. I am looking for ways to change this in a future update.

Yeah I have the JavaScript file via CDN. All the functionality of it seems to work fine.
Now that I know that animation is supposed to work I can dig in and figure out why.


Fix minor typo

Fixes a minor typo in an invariant message.

@solon thanks! appreciate the fix

No problem!


WARNING: Failed to delete temp mount directory

Hey Jim
Fantastic script. Great job!
When I ran the script from differens locations, for example directly from the SOFS I got following Warning after the Shrink job has runned. The job running successfully, but the warning keep coming and feeling up my Temp folder on the server

Have checked access to the location with my used account
What do I wrong :-)
Thanks in advance
Gert Hansen - DK

So I try a number of different ways to delete this directory,  it doesn't affect then working of the script.  What kind of VM are you running it from?  OS/CPU/RAM

Thanks Jim for your feedback and deeply sorry I newer responded... Lets keep it closed until I get it tested.

Hi,
I'm having the exact same problem.
OS Windows Server2019, 6vCPUs, 32GB RAM
Best regards
Jonas

It's everyone.
I have only tested about 10 vhdx-files, but with the same error every time.
Den tors 23 mars 2023 kl 16:11 skrev Jim Moyle @.***>:

Is it all of the temp directories, or just some?
—
Reply to this email directly, view it on GitHub
https://github.com/FSLogix/Invoke-FslShrinkDisk/issues/37#issuecomment-1481372526,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWUQ3OMZMNEO5OL2JPBNW5TW5RR2HANCNFSM4ZIX3WZA
.
You are receiving this because you commented.Message ID:
@.***>


Fixed Log Permissions

Took care of #1554 just to make life easier.

No, I have to take a look and see if it can be allowed on public nadeko.


Change Direction of Drawer,when app in RTL Arabic mode.

When the System Language is English (Default/Base for my APP), the drawer enters screen from left.
I want to change it to enter the screen from Right when in RTL Arabic mode.

Well, I found the solution for my problem. I used the below Code
if let drawerController = navigationController?.parentViewController as? KYDrawerController {

           if NSLocale.preferredLanguages()[0].containsString("ar"){
               drawerController.drawerDirection = .Right
           } else {
               drawerController.drawerDirection = .Left
           }

       }


convert the real images and generated images back to original format to display???

Hi, here when displaying the images , donot we need to  convert back to the original images value range before storing them to disk.

@eriche2016 I believe that happens in vutils.save_image
https://github.com/pytorch/vision/blob/master/torchvision/utils.py#L81

@LukasMosser , hi, i notice that here the real image tensor  Input  has been normalized to Out by mean = [0.5, 0.5, 0.5] and
std = [0.5, 0.5, 0.5], should not the transformed value be converted back by using Out * 0.5 + 0.5 before storing back to the disk.

Hi, i think we should converted back by inserting the line
real_cpu = real_cpu.mul(0.5).add(0.5)

above here
and
fake.data = fake.data.mul(0.5).add(0.5)

above here,  otherwise, the generated images will be not good looking with some sharp weird edges.


Issue #SC-1591 feat: cassandra migration query fix

Codecov Report

Merging #770 into release-2.7.0 will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff                @@
##             release-2.7.0     #770   +/-   ##
================================================
  Coverage            38.61%   38.61%
  Complexity             979      979
================================================
  Files                  149      149
  Lines                10078    10078
  Branches              1118     1118
================================================
  Hits                  3892     3892
  Misses                5777     5777
  Partials               409      409

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41210a9...25cd010. Read the comment docs.


few changes to consider

repo reorganization, so that it can be downloaded as a zip and extracted to addons or pinned as a git submodule
dock size increase bug workaround

I looked at these changes and I can't accept the first commit. The current structure is required for the plugin to work when downloaded from AssetLib.
The second one seems to have many unrelated changes, as the base fix is simply enabling the horizontal scroll. I think it's acceptable for now; in 4.0 I could use the new HFlowContainer instead.
So if you remove the first commit and change the second one so that it only removes scroll_horizontal_enabled = false and updates the README, I could merge this.

oh, had no idea about AssetLib - then nvm :)


Auth0, React and Single-Spa [You forgot to wrap your component in ]

Describe the problem
I am creating an app using Single-Spa, React and Auth0. The @auth0/auth0-react documentation specifies to wrap the app in the following way:
  <Auth0Provider
    domain="YOUR_AUTH0_DOMAIN"
    clientId="YOUR_AUTH0_CLIENT_ID"
    redirectUri={window.location.origin}
  >
    <App />
  </Auth0Provider>

However with Single-Spa the components are mounted and un-mounted based on the url path, as specified by the root-config’s index.ejs file (there is no root component). With Single-Spa I do not have an App to wrap, as one normally would with React, though my individual components are still React-based.
  <template id="single-spa-layout">
    <single-spa-router mode="history" base="/">
        <application name="@example/navbar-app"></application>

          <route path="home">

              <application name="@example/app1"></application>
              <application name="@example/app2"></application>

              <application name="@example/lookup"></application>

          </route>

    </single-spa-router>
  </template>

I tried to wrap the Navbar, which displays on every page, but am getting the error:
You forgot to wrap your component in Auth0Provider
const {
  isLoading,
  isAuthenticated,
  error,
  user,
  loginWithRedirect,
  logout,
} = useAuth0();

return (

    <Auth0Provider
      domain={auth_config.domain}
      client_id={auth_config.clientId}
      redirectUri="https://example.com"
    >
      <Navbar bg="dark" variant="dark">
        <Container>
          <Navbar.Brand>Navbar</Navbar.Brand>

              { isAuthenticated ?
                [ Home,
                  Redirect,
                  Logout
                ]
                : Login
              }

        </Container>
      </Navbar>
    </Auth0Provider>

);

What was the expected behavior?

The Ability to choose what to wrap, if no root component exists, or an alternative.

Hi @natasha-blong - thanks for raising this

The Ability to choose what to wrap, if no root component exists, or an alternative.

You are able to wrap whatever component you like.
However, just like any other React Context, every component that uses the context must be wrapped in the Provider. I can't see how your application is organised and which component is throwing that error, but the Auth0Provider component must be an ancestor of that component - this is required to share React Context with that component, and is a limitation of React, not this SDK.

I imported a component that contained the useAuth0() functions into the rootComponent, where the imported component was wrapped in Auth0Provider. This worked great. Hopefully if anyone using Auth0, React and Single-Spa in the future, this can help them!


Publish new hex.pm package

Could you please publish a new hex package with the latest changes?
Thanks.

Ping

Hey, would it be possible to release a new version of the package with the commits I pushed some months ago?


A better deployment

Combine and minify all javascript
Combine all the static resources, and serve them separately

Will fix with Girder.


Remove index.yaml and add README.md

GH action fails when there is an empty index.yaml file.  It is better to
have none at all so that it can create a valid index.yaml file for us.

What this PR does / why we need it:
Which issue this PR fixes(if applicable):
fixes #
Special notes for reviewers:

Release note:

NONE

@lingxiankong @jichenjc

/ok-to-test
/lgtm
thanks for the update

/approve

/lgtm

Hmm there is no OWNERS file in this branch which is probably why the CI is ignoring your command.


Updated and more efficient Schema Version detection

Updated schema version detection to use RegEx

Thanks for the PR. I've updated the logic a little bit... is that ok?

Perfect - I was rushing this morning and hadn't had enough coffee before I commited :-)

Why did you close?

Really need that coffee!!!

Ok it's merged, you can get your coffee :) thanks for the PR (with unit tests!!)!


Error: Invalid option(s): -f net6.0    'net6.0' is not a valid value for -f. The possible values are:       net5.0          - Target net5.0       netcoreapp3.1   - Target netcoreapp3.1  For more information, run 'dotnet new webapp --help'.

Problem encountered on https://dotnet.microsoft.com/en-us/learn/aspnet/hello-world-tutorial/create
Operating System: windows
Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

That error message indicates you don't have .NET 6 SDK installed. Have you downloaded and installed .NET 6 SDK as shown on the install step?

 ~/workplace/Project
❯ dotnet new install Amazon.Lambda.Templates
Error: Invalid option(s):
  Amazon.Lambda.Templates
For usage information, run:
   dotnet new -h
- ~/workplace/Project   main
❯ dotnet --version
6.0.406


changing azure vm last modified source

due to unknown issue on Azure we are getting incorrect last_modified date, which update with the script run:
vm test-prov-bl5o - 2017-07-24 11:39:39
vm test-prov-mt7c - 2017-07-24 11:39:57
vm test-prov-nmzp - 2017-07-24 11:40:19
vm test-prov-npsw - 2017-07-24 11:40:37
vm test-prov-qxny - 2017-07-24 11:40:54
and our cleanup jobs are failing - some VMs stay on provider side for 15+hours until they stop updating  LastModified of their OSDisk (couldn't find any clues on that)
So the idea is to get last activity log of our VM and get it's timestamp. Unfortunately looking for logs older than 15h is a problem - too long and it fails to fetch results. For sake of housekeeping activities we could take only 12h old logs.

Lint report for commit 6d6a41d0a12b82b83886db1dd8e1ad216f5de969:
wrapanapi/azure.py:

:red_circle: Line 12:1: F401 'tzlocal' imported but unused

Please, rectify these issues :smirk: .
No commit flaws detected.
CFME QE Bot

@jteehan @anewmanRH @mshriver  please add your comments as this solution also has it weak sides (it can't pick up too old VMs which I assume shouldn't exist anyway)

found the issue -we need to update azure module on our server to 4.2. now we have  1.0.1. after that  GetAzureRmLog would fetch pretty fast and all required rows we ask

@lkhomenk can you mark RFR and ping me when the azure module is updated/tested and I'll review?

tested with the latest 4.2 - runs as expected. all other methods were not harmed too much. pushed another PRs to fix broken method which were not working at all


CommonMark compliant HTML block inside  tag is removed by JSDoc Intellisense

VSCode Version: 1.52.1
OS Version: Darwin x64 19.6.0

Steps to Reproduce:

Open the example code in VSCode.
Hover over Foo. The Tooltip reads "".
Hover over Bar. The Tooltip reads "A basic interface with bar."

The doc comment for Foo is compliant under CommonMark spec 0.29 4.6 HTML Blocks kind 6

Start condition: line begins the string < followed by p, followed by the string >
End condition: line is followed by a blank line

Example Code
/**
 *
A basic interface with foo.

 */
interface Foo {
  foo: number;
}

let f: Foo; // Tooltip reads ""

/**
 * A basic interface with bar.
 */
interface Bar {
  bar: number;
}

let b: Bar; // Tooltip reads "A basic interface with bar."

Screenshots

Does this issue occur when all extensions are disabled?: Yes

A similar issue discusses removal of code inside
 and  tags in https://github.com/microsoft/vscode/issues/110945, but that appears under CommonMark spec 0.29 4.5 Fenced code blocks

A similar issue discusses removal of code inside
 and  tags in https://github.com/microsoft/vscode/issues/110945, but that appears under CommonMark spec 0.29 4.5 Fenced code blocks

Looking at the code comments, the makrdownRenderer seems to comply with CommonMark 0.29 spec.
https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/browser/markdownRenderer.ts#L288

Looking at the code comments, the makrdownRenderer seems to comply with CommonMark 0.29 spec.
https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/browser/markdownRenderer.ts#L288

Some findings which could be helpful:

The markdownRenderer uses marked from vs/base/common/marked/marked: https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/browser/markdownRenderer.ts#L11
The renderer is markdown parser from https://github.com/markedjs/marked https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/common/marked/marked.js#L1-L5

The Marked Demo returns data as per CommonMark spec 0.29 4.6 HTML Blocks kind 6 for provided example.

Screenshot

Some findings which could be helpful:

The markdownRenderer uses marked from vs/base/common/marked/marked: https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/browser/markdownRenderer.ts#L11
The renderer is markdown parser from https://github.com/markedjs/marked https://github.com/microsoft/vscode/blob/6260e655bc606e204ad51ce5b1cb34216cd36dd3/src/vs/base/common/marked/marked.js#L1-L5

The Marked Demo returns data as per CommonMark spec 0.29 4.6 HTML Blocks kind 6 for provided example.

Screenshot

Duplicate of #110945 I believe, just with a different html tag

Duplicate of #110945 I believe, just with a different html tag


Limit recursion to avoid parsing attacks

Currently, when parsing map/array, we benefit from naive recursion.
This allows malicious users to generate messages that will overflow the call stack.

Fixed in https://github.com/codx-dev/msgpacker/releases/tag/v0.4.0


Sort tables

For https://github.com/open-cluster-management/backlog/issues/12610
With the update to ui-components, this was pretty straight-forward.

/lgtm


Make XML Security Level customizable

Currently, validator has a strict XML security policy and only allows loading XML Schema from local file resources.
Some users want a less strict security policy in their runtime context. Therefore, I consider this a useful feature.

This is also a prerequisite for issue  #15

xml security (loading) ist configurable via API now. See [API documentation] (https://github.com/itplr-kosit/validator/blob/master/docs/api.md) for details


App doesn't compatible with Django 1.5

Error message: TemplateSyntaxError: 'adminmedia' is not a valid tag library: Template library adminmedia not found.
Django 1.5 deprecation log:
The template tags library adminmedia and the template tag {% admin_media_prefix %} will be removed in favor of the generic static files handling. (This is faster than the usual deprecation path; see the Django 1.4 release notes.)
https://docs.djangoproject.com/en/dev/internals/deprecation/#id2

This should now work in 0.8.1 thanks to @zlorf's work to make this compatible.


Use same DB connection pool for data and logs

We used to create two connection pools to the database (one for data and one for writing availability update logs). This leads to a lot of connections, and on the new Render deployment we are sometimes running out of connections and dropping logs. This re-uses the same pool for both, which should be safe (we still keep separate references, so the availability log can be changed to point somewhere else) and solves the issue by using half as many connections.
Another option here is to add an environment variable for the maximum number of DB connections in the pool(s). That might still be a good idea, and could be useful in addition to the change I made here, but skipping it seemed simpler for now.
Fixes https://sentry.io/organizations/usdr/issues/3155794613.
@astonm do you mind giving me a gut-check on whether this seems reasonable? The biggest concern I can think of here is that this might reduce throughput slightly, but I think it should be low impact overall.

Re: Render, the DB maxes out at 97 connections (we can pay double for 197 connections), and this is happening when we autoscale to 5+ instances of the API server (another approach would be to make the max 4). In contrast, AWS RDS has no connection limit; it just lets us go until Postgres can’t handle it, and we never go anywhere near that. 🤷

97 seems like plenty, even for 5 instances of the API, although maybe I'm just not correctly thinking about how many connections we need at a time.

I’m pretty confident we don’t have leaky connections — we only allocate new pools at startup (the only allocations are top-level code in modules that are not dynamically imported, so they only get created on startup, and we don’t  tear them down until we shut down a server (which is correct). Even if connections were leaking at that point, the server gets killed and so the connections are very definitely dead.
The issue is really that the default pool size in Knex is 10, so if we have 5 instances, that is 20 connections (2 pools — regular DB access + availability log) per instance, or 100 connections total. Our autoscaling is set with 6 as the max, so we could (and I think have) ramp into a configuation where we attempt to create 120 connections. I believe we actually go over this during deploys, since new and old instances will both be running for a few seconds/minutes while traffic is shifted.
Since our usage is spiky (every few minutes we suddenly send ourselves thousands of requests), we tend to scale up and immediately create every connection allowable in the DB connection pool — that is, every instance will usually hit its max number of connections (20, right now).
Here’s Render’s scaling metrics for the API service:

ON HOLD while we see what we learn from #781.

Quick update: logs make it look like availability logging connections definitely don’t typically scale up as much as the data connections. I realized we can do better than the logs with “gauge” metric in #786, though.
Jul 29 02:52:31 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:52:31 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 5
Jul 29 02:52:48 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:53:00 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:53:02 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:53:32 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 8
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 9
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 10
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 6
Jul 29 02:53:33 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 7
Jul 29 02:54:20 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 8
Jul 29 02:54:51 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 7
Jul 29 02:55:16 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 6
Jul 29 02:55:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 5
Jul 29 02:55:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:55:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:55:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:55:20 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:55:29 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:55:35 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 02:55:43 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:55:43 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:55:43 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:55:44 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 02:55:44 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 02:55:54 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 3
Jul 29 02:55:54 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 4
Jul 29 02:55:54 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:56:24 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 4
Jul 29 02:56:25 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:56:25 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:56:34 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 3
Jul 29 02:56:34 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 4
Jul 29 02:56:34 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:56:34 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 3
Jul 29 02:56:34 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 02:56:38 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:56:38 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 7
Jul 29 02:56:40 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 8
Jul 29 02:56:41 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 5
Jul 29 02:56:42 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 9
Jul 29 02:56:43 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 6
Jul 29 02:57:14 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 5
Jul 29 02:57:17 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 8
Jul 29 02:57:17 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 02:57:18 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 3
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:57:19 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 3
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 4
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 3
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 02:58:26 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:58:27 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 5
Jul 29 02:58:58 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:58:58 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 02:58:59 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:58:59 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:58:59 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 02:58:59 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 02:58:59 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 02:59:38 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 3
Jul 29 02:59:38 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 4
Jul 29 02:59:38 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 5
Jul 29 02:59:51 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 6
Jul 29 02:59:51 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 3
Jul 29 02:59:51 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 4
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Data DB pool size: 2
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2
Jul 29 03:00:21 PM  [univaf] info: api-server-phc8-srv-cb7jc9qrrk00cmvca6kg-6bdd65f5bc-tlqw9 Availability DB pool size: 2

Some updates here: I added logging last week, and we now have some nice metrics in DataDog. You can view the Database Connection Pool dashboard at: https://app.datadoghq.com/dashboard/t4g-ibt-thh/database-connection-pool-sizes and you can use the “platform” and “instance” controls at the top to narrow things down.
It tracks both pool size:

And how long it takes to acquire a connection from the pools/how many acquisitions are waiting on the pools:

The availability log pool tends to be used a little less than the data pool (but when an instance maxes out the data pool for more than a moment, it will also tend to max out the availability poo).
Pending acquires tend to back up every so often on the data pool, but hardly ever on the availability pool. This matches expectations that the availability log writes are pretty quick and low overhead.
Time spent waiting for connections from the pool doesn’t tell us much interesting, it’s largely the same info as pending acquires with a little more noise that doesn’t seem usefully attributable to anything.
Surprisingly, times when we hit the limit don’t really stand out on these charts.

We ride close to that edge regularly
It’s not obviously attributable to old & new instances online at the same time during deploys.
It happens most when the two pools scale up together, which is not always the most common situation (see first bullet). My working theory is that this is one of two situations:

We have a big spike of lightweight updates without a lot of slot- or capacity-level detail (we’re processing the updates as quick as possible, approaching the same speed as availability log writes). I’m not sure about this, though, since it would imply that we should see problems whenever we run the CDC loader (it never has slots/capacity and it reliably produces thousands of updates), but we don’t see the connection limit being hit regularly at times similar to when that loader runs.

We have a spike of extremely heavy updates and those updates have actual new data. In this case, the availability log writes will take a long time, and overall CPU use should also go up (for validating and deep comparing big bodies), which should also make us scale instances as well. This also happens at unpredictable times.
We could probably add some metrics to see this better (how many updates and how big are they?), but I’m not sure how actionable it would be, even if it might help clarify the cause.

Pretty graphs! In terms of sorting out which of your two theories (if not both) is the main cause, maybe we should set up some simulated loads?
Although, zooming out, it may not really be important to know "why" so long as we know we're consistently under the limit. Are you still planning to eventually merge this PR?

zooming out, it may not really be important to know "why" so long as we know we're consistently under the limit.

Yeah, for this case, I think that’s probably true. I’d like to get around to adding some more metrics to measure this better, but I don’t there’s any reason to block a basic fix for the problem “we allow for more connections than we can have.”

Are you still planning to eventually merge this PR?

I just put up #818, based on your recommendation to just bring the limits for the logging connection down. It’s a little more complex, but not by a whole lot.

Closing since this has mainly been addressed by #818.


[New WTFBin]: AGMServive.exe LSASS read

g1ng3rr00t
AGMService.exe / Adobe Genuine Monitor Service
AGMService.exe opens and reads from the LSASS process
I was unable to find much information about this issue specifically (although I did not contact Adobe at all) aside from this forum post from 2019 https://community.adobe.com/t5/acrobat-discussions/credential-theft-adobe-genuine-software-service-agmservice-exe/td-p/10745517, and it being excluded in Olaf Hartong's sysmon modular configuration repository (https://github.com/olafhartong/sysmon-modular/blob/master/sysmonconfig-excludes-only.xml).

Added in e8acedb.


attributes function parameter for linky only works for first link

Note: for support questions, please use one of these channels: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#question. This repository's issues are reserved for feature requests and bug reports.
Do you want to request a feature or report a bug?
Bug.
What is the current behavior?
The attributes function is only called back for the first link.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem via https://plnkr.co or similar (template: http://plnkr.co/edit/tpl:yBpEi4).
As you can see in this plunk, the setAttributes function is only called back once, even though there are two links in the text.
http://plnkr.co/edit/d5zWDOcF46wZEQjdLUYf?p=preview
What is the expected behavior?
There should be 2 callbacks, one for each link.
What is the motivation / use case for changing the behavior?
So that each link can have different attributes depending on whether or not it's an external or internal link.
Which versions of Angular, and which browser / OS are affected by this issue? Did this work in previous versions of Angular? Please also test with the latest stable and snapshot (https://code.angularjs.org/snapshot/) versions.
All browsers and the latest version of Angular (1.5.5 I believe).
Other information (e.g. stacktraces, related issues, suggestions how to fix)
It appears that the attributes function is overwritten by its return value in the first call of addLink which causes it not to be called back in further calls (since it's now an object).

Oh wow, that's a bad one.


Surpress FPs for cryptomator libraries. Fixes #4177

Fixes Issue #4177
Description of Change
Due to CVE-2022-25366, dependency-check reports via its maven plugin every Cryptomator library as vulnerable, even though only the end user application is affected. This PR supresses these FPs by supressing any maven artifact of org.cryptomator/* for cpe:/a:cryptomator:cryptomator.
Have test cases been added to cover the new functionality?
no, only tested locally.

This suppression is definitely too broad as it also suppresses the desktop app itself

I adjusted the regex to exclude org.cryptomator.cryptomator from being supressed.

The desktop app is not a maven package though.

Correct, the vulnerability does not exist in the Maven module, only in the macOS .app based on it.
The CPE is inaccurate, we don't know who added it, as it was not present in the original CVE.
Sadly, the CPE nomenclature only knows vendors and products. Since the product name of the final app is Cryptomator (duh!) no distinction between Maven and binary artifacts is possible.
We're in the process of requesting official CPE names where we can distinguish the apps for several platforms, but for now the problem persists as organization name and product name are identical and basically every artifact published under the orga name is false positive.
Btw the Cryptomator Maven artifact is not published to any Maven repo, it is merely an intermediate build step towards the platform-specific apps.

I guess the root cause lies here:
https://github.com/jeremylong/DependencyCheck/blob/main/core/src/main/java/org/owasp/dependencycheck/analyzer/JarAnalyzer.java#L642-L645
The maven groupId is used as product name with confidence that can not be beaten by the artifactId.

Btw the Cryptomator Maven artifact is not published to any Maven repo, it is merely an intermediate build step towards the platform-specific apps.

If Cryptomator is not published and used by others - shouldn't this just be a suppression rule you use locally as opposed to putting this in the suppression rules of the core engine?

we publish Maven artifacts with the groupid org.cryptomator
we publish Cryptomator (a desktop application not a Maven artifact) via GitHub Releases

The latter had a vulnerability in its macOS package. The former do not even carry "cryptomator" in their artifact names, but the groupId causes FP.

Another option instead of a suppression rule would be to add a hint rule to remove the offending group id that is causing the FP. See https://github.com/jeremylong/DependencyCheck/blob/963cd91d96887af45e4dfb495c296d793d12e85e/core/src/main/resources/dependencycheck-base-hint.xml#L194-L207
Something like:
 <hint>
     <given>
         <evidence type="product" source="pom" name="groupid" value="org.cryptomator" confidence="HIGHEST"/>
     </given>
     <remove>
         <evidence type="product" source="pom" name="groupid" value="org.cryptomator" confidence="HIGHEST"/>
     </remove>
 </hint>

Wouldn't that cause false negatives, if there eventually was a vulnerability in a Maven artifact org.cryptomator:somelib?

Whether it would create a false-negative would be up to how the CPE was created for the library... I was just offering a different solution.  I'm okay with the suppression for the maven library the way it is.


Fix a broken markdown link

A link in paper/14-research-sources had flipped markdown braces.

https://www.microcovid.org/paper/14-research-sources#ventilation
vs
https://deploy-preview-615--microcov.netlify.app/paper/14-research-sources#ventilation

Thanks for catching this!


Add maximum size to gets() in HTTPServer

In here : https://github.com/manastech/crystal/blob/master/src/http/request.cr#L33
If I would send an endless stream of data (without \n) I can fill up the servers memory in no-time.
We should use gets("\n", 4096) or something like that.
For example:
https://github.com/ruby/ruby/blob/c8b3f1b470e343e7408ab5883f046b1056d94ccc/lib/webrick/httprequest.rb#L529
def read_line(io, size=4096)
  _read_data(io, :gets, LF, size)
end

So, it seems that I totally misunderstood how the IO.gets(sep, limit) thing works, after reading the documentations here : http://ruby-doc.org/core-2.2.2/IO.html#method-i-gets It seems the "limit" is not a maximum byte size to read, as this is the case and there is no way to specify a maximum read size for "gets" maybe we should use a read_nonblock(4096) and separate the body from the headers.
What do you think ?

Yes, we should do something about this. We'll also need an overload for gets that accepts a limit

@asterite Looking at a possible way to implement this, I found a Bug in ruby IO.gets :)
https://bugs.ruby-lang.org/issues/11400
I'll the new implementation after the bug will be fixed, then I'll try to PR a gets() with a limit

Just to hammer in the issue, I did some tests, right now I can crash any HTTPServer based appliance in less then 20 sec
Starting the application:

Running my "endless stream" application against it:

And then, crashed

OK, now there's a way to do gets with a limit after https://github.com/manastech/crystal/commit/66cc451b0a0444024646f0ff43e702f862b51686
Now we need some answers:

What limit to use for the status line?
What limit to use for each of the headers?
How many headers as a maximum to support? (otherwise you could end up building a huge headers list and fill the server's memory)
What to do when the limit is reached? Raise something like HTTP::BadRequest?

We'll also need to support body streaming in the HTTP server, right now it consumes the whole body, so if you stream an infinite body it will also consume the server's memory (I didn't test this, but it's a guess).

@asterite I know the Ruby handles gets("\n", 4096) like so:
Super pseudo code:
while read_data_from_users_socket(user_socket)
  does data == "\n" ? if so finish read and return data
  does data.size > 4096 ? if so finish read and return data
end

this meas that now the gets method can read until one of the two requirements are meet

reached a read size bigger then 4096 bytes
reached an "\n" indicator

I know that usually you wont hit the 4096 limit while reading a header (this is also the hard-coded limit for webrick)

Also, I have a small web server application in Ruby which I developed, I use gets("\r\n\r\n", 4096)
This way I can get the whole Header part of the request then start analyzing it, also, I can then do a loop of the body like while read_body < header.content_length; keep reading

Is this bug still present ( seems like it is) ? I'd like to work upon this , can anyone point me to the right file or give any other suggestions please ?

@asterite @gamebusterz
Is this still relevant ? if so it should be easy to fix no ?

@bararchy I'm wondering the same. Do you have an idea about which part of the code should I be looking into for this ?

@bararchy Yes, still relevant, and should be easy to fix because we have gets with limit :-)
I'll probably tackle this before the next release.

@gamebusterz => https://github.com/crystal-lang/crystal/blob/master/src/http/request.cr#L90
I guess there or something, the other thing is that it seems to read a line-by-line until line = \n
This is an issue too, or , test each line the header size as string to see we are not going over the maximum header size limit (can default to 4096 and be configurable)


Release v2.0.0

The main changes are

relaxing a required field to non-required (pc:schemas)
removing a field (pc:encoding)

Related issues

Closes #13

because [...] it is allowed to have fields w/ a pc: prefix that aren't in the extension.

That is not correct. Unknown pc: prefixed fields will be reported as invalid. @gadomski

Alrighty, then should be v2.0.0. Thanks @m-mohr I'll update.

Generally, the schema could be updated to a better version that we use in some other extensions. I'm not sure it's in the template extension yet, but if you go 2.0.0 preferrably also improve the schema ;-)

Generally, the schema could be updated to a better version that we use in some other extensions. I'm not sure it's in the template extension yet, but if you go 2.0.0 preferrably also improve the schema ;-)

Ok, I'll hold off on merging and see if I can get some time to update the schema 👍🏼

Here's a recent example of a "better" schema ;-)
https://github.com/stac-extensions/eo/blob/9dfcf270dbd210e74ee1e06a15c50beedc2e3b5d/json-schema/schema.json

The CI tests fail, could you fix those first? :-)
Or any specific issues I should look at?

The CI tests fail, could you fix those first? :-)

Yeah, I'm flailing around now — please ignore, I'll ping again when I get things green 🙇🏼 .

@m-mohr did my best to update the schema using the "better" version, so ready for re-review (finally).

In the meantime I created an even better version 😅 (sorry!)
https://github.com/stac-extensions/eo/blob/main/json-schema/schema.json
I still need to generalize it for the template repository though.
It uses if/else instead of oneOf which makes the error reporting in JSON Schema more smooth and actually correctly validates fields everywhere. Before it sometimes happened that parts were not validated.

@m-mohr ok I've updated with The New Way, and added some more tests (for assets and item assets, respectively).

Should the count in pc:statistics have a minimum value of 0 or 1?

It could be a zero point file, that's allowed by most formats, so 0 seems right.

@gadomski Yes, but there was NONE defined. I'm not talking about pc:count, but about pc:statistics.count!

Whoops. I'll open an issue, thanks!


Framework7App routes accumulates props

Hello.
I have an array of routers
const router = [
    {
        path: '/live/',
        component: Live
    },
    {
        path: '/live/sport/:sid',
        component: LiveSport
    },
    {
        path: '/live/event/:gid',
        component: LiveEvent
    }
];

I go through the pages in this order:
1/ Live
2/ LiveSport
3/ LiveEvent
And in the selector mapStateToProps LiveEventBets component make console.log
const mapStateToProps = function (state, props) {
    console.log(props);
    return {

    };
};

In it, I see the attribute of the sid that was passed to another page
{className: "", sid: "2589716", gid: "2589716"}
This is a very critical bug, because the props values will determine whether the component will be updated or not
Thanks

Sorry for the delayed response. I haven't had much time to try reproducing this. If you'd care to put together a fiddle or some example showing the bug reproduced, that would greatly accelerate the process. Otherwise, I'll do my best to find some time in the near future to reproduce the issue myself.

@bencompton ,
https://jsfiddle.net/w8vn0nwy/

@valentin93, thanks for putting together a fiddle. I updated the fiddle to work as it appears to be intended to work: https://jsfiddle.net/w8vn0nwy/1/.
I had to change the routes prop to wrap the route object in an array:
routes={[
  {
    path: '/test/',
    component: Test
  }
}]

I also changed the Framework7App component to have a pushState={true} prop. When running the fiddle, it now loads the Test page and shows "123" on the screen.
It sounded like from your issue description before that you were having trouble with it passing the wrong route parameter values into the component props, so this doesn't look like the same issue. Maybe you posted the wrong fiddle or something?

@bencompton  , I apologize. I sent you the wrong link. I will post the actual

@bencompton
Look please
https://jsfiddle.net/4yvdjzL0/
https://www.screencast.com/t/hQ1DySapD
On the video, everything is perfectly visible.

Hey, thanks for the fiddle and the screencast! This is an issue in the F7 Vue router code. For more detail, see this issue I created in the F7 Vue repo.
It may take a bit for this to get fixed in F7 Vue and for me to pull it into F7 React (F7 React is currently a bit behind F7 Vue), so a way to work around this issue in the meantime is to ensure that none of your route URLs have parameters in the same position. For example, instead of this:
'test/:testParams'
'test2/:test2Params'

Do this:
'test/:testParams'
'test/test2/:test2Params'

That's pretty annoying, I know.

I'm still looking into why the second render occurs...

@bencompton , The problem did not dare? Can I help you?

Sorry for the delay in getting back to you. I know what the problem is, but I haven't quite figured out what the best solution would be.
The re-rendering issue is occurring because every time a new page gets added, it updates the state of the Pages component, which forces it to re-render all of its children (every page that has been added so far). I found that this does not happen when using react-redux with pages wrapped in containers, because react-redux has componentShouldUpdate logic that only re-renders a page when the props that are mapped to state actually change.
I suppose something similar could be done where the framework7-react Page component has a shouldComponentUpdate function that only re-renders the page if it is the active page and not hidden in the DOM. It would need to ensure that a page gets re-rendered when going back to it so that any updates that were missed while it was hidden get applied.
Interestingly, this problem doesn't occur in F7 Vue. My guess is that Vue has logic to prevent re-renders on components with display: hidden, but I haven't gotten very far trying to prove that conclusively.
It might take some time to fix this properly. In the meantime, if you're using Redux with react-redux, this problem won't affect you. If not, you could also add your own shouldComponentUpdate function to your page components that checks whether or not your props / state have actually changed (assuming they're immutable).

@bencompton ,
You can not use a shouldComponentUpdate because all properties are overwritten. And as a result the shouldComponentUpdate does not help

@bencompton ,

It might take some time to fix this properly. In the meantime, if you're using Redux with react-redux, this problem won't affect you

I'm using redux, but the problem is that

@valentin93 - In an app I am currently working on, I am using f7-react + redux + react-redux. When I navigate from page to page, I can see that my mapStateToProps functions for hidden pages do get called when navigating to new pages. However, the render functions in my pages themselves never get called because the resulting props from my mapStateToProps functions have not changed for the hidden pages. The mapStateToProps functions are mostly calling memoized selectors, so the performance penalty of calling mapStateToProps unnecessarily for hidden pages should not be so bad.
Sounds like you are seeing different behavior than that in your app? Perhaps you have a case where your pages share a piece of state, and an update to that piece of state affects multiple pages, even hidden ones?

If this information is useful, then the re-erendor is accumulated in arithmetic order. More and more.
If you walk from page to page there and back.

I don't think I quite follow what you mean. Maybe you could provide an example?


Enhance null value support for minmax index

Enhancement
Currently, we not enable the minmax index when the pack has null values. Considering we now support nullable datatype in minmax index(ref #4787)  , we can support minmax index for the packs with null values, to enhance the query performance.
Besides, we also want support  'is null' , 'is not null' predicates with our minmax index.

/assign


Verification Falied

Please check
Name: Deepanshu Bansal
Email Id<EMAIL_ADDRESS>

@Hemu21 can you look into it

@Deepanshu7573Bansal  can you try once now. It is working fine

Same problem is occur

@Hemu21

@Deepanshu7573Bansal  It is working for you mail please enter your details carefully. If it doesn't work once try page refreshing.


Fix reads of metadata file in WAL

/cc @benbjohnson
@bobuss, I think this will fix #3730

:+1:

cherry picked into 0.9.3

@pauldix Ran into the same issue with 0.9.3rc1 last night. Would this be available in the latest nightly?

awesome ! I 'm going to test it ASAP


Loading of plugin server components

@gr2m:
Here's an initial concept for https://github.com/gr2m/milestones/issues/89
A few notes:

I couldn't see where to get all passed options from within plugins/index.js, so needed to extend options within config/parse-options.js. Please let me know if theres a more preferred way.
I wasn't too keen to use the package.json as a means of loading the plugins... thought it was best we use the config in the rc file, as it may be less confusing for users? Open to thoughts
I haven't created any form of tests yet, so only worth merging in if we're happy it doesn't break general operation of hoodie

Based on this, it's likely not ready for a PR, but is more a means of grouping the changes and discussing.

Change coming to this soon - I see why using the RC file may be less ideal... It's more environment specific than app specific, so we need in 'package.json'.

Change coming to this soon - I see why using the RC file may be less ideal... It's more environment specific than app specific, so we need in 'package.json'.

yes exactly. package.json could be app-specific defaults, as it gets checked in to source code. All the rc configuration like the .hoodierc file is environment specific, so you can run hoodie with a PouchDB backend locally and use a CouchDB in production, for example

@gr2m: Got distracted yesterday, but keen to get this closer today!
In regards to package.json structure - old hoodies structure looks like this:
  "hoodie": {     "plugins": [       "hoodie-plugin-appconfig",       "hoodie-plugin-email",       "hoodie-plugin-users",       "hoodie-plugin-agtract"     ]   },
I'm proposing we stick to to the "hoodie": {"plugins": []} structure, but discard the "hoodie-plugin-" portion. So something like this:
  "hoodie": {     "plugins": [       "store",       "account",       "agtract": {         "app": "specific"       }     ]   },
There's also nothing stopping us still parsing the "hoodie-plugin-" situation, as we'd just string.replace on the name prior to doing anything with it.
Assuming that solves "app"-side config, we'll obviously want to have environment config for each plugin. We could either leave the config/parse-options.js as I have it (as it simply merges all content from the RC file with the config), or maybe we should be looking to create a config/plugin-options.js that accepts the plugin name, and loads from RC/cli flags.
Hopefully this is feeling a little more on track?

it is possible that not all modules which will be used as hoodie plugins will follow the hoodie-plugin-<name> naming convention, but most of them will. I’m good with the "plugins' key, make things more clear. I’m not yet sure if an array or an object is better though. Probably an array were we can simply list the plugin names, and pass in an object if we want to set custom properties.
So for example, a Hoodie app’s default configuration could look like this:
{
    "hoodie": {
        "public": "app",
        "account": {
            "sessionTimeout": 1209600
        },
        "plugins": [
            "foo",
            "bar",
            {
                "name": "baz",
                "package": "hoodie-boozie",
                "options": {
                    "bazOption": "value"
                }
            }
        ]
    }
}

The "bazOption" can also be set using rc as hoodiePluginBazOption
Makes sense?

Need to assume that options.plugins will be either an array or object. See https://github.com/hoodiehq/hoodie/pull/486#issuecomment-219410988
If array, it will be converted to an object before bootstrapping plugins.

Need to assume that options.plugins will be either an array or object. See #486 (comment)

I’d say that in package.json, "hoodie": { "plugins": [] } is always an array. The values can either be plugin names (strings) or objects. A string is a shortcut for {"name": "plugin name here"}.
We would parse these options internally to an object before we merge the config from rc into it.
If that sounds all to confusing, then maybe we should have hoodie.plugins.* be an object everywhere, including in the package.json, and do something like
{
  "hoodie":  {
    "plugins": {
      "pommes": { "secret": 123 },
      "otherplugin": true
    }
  }
}

I think eventually we could even support both formats in package.json, where the array could be used as shortcut. Would that work?

@gr2m I think I've boiled this down to something more appropriate. Keen on your input!
Plugins can only be enabled in package.json. The formats accepted in this file are as follows:
Standard array:
{
  hoodie: {
    plugins: ["pluginone", "plugintwo"]
  }
}

Array of objects
{
  hoodie: {
    plugins: [{ name: "pluginone" }, { name: "pluginone" }]
  }
}

Objects
{
  hoodie: {
    plugins: { "pluginone": { name: "pluginone" }, "plugintwo":  {name: "pluginone" } }
  }
}

This allows for simple plugin loading, all the way through to defining plugin options as you can see here: https://github.com/agtract/hoodie/blob/10a4e5e84f254c83e510e1b76979b4617089dff3/server/config/parse-options.js#L36-L41
One thing I've changed is how argv/rc options affect plugin loading. This was what was complicating things before, and as you mentioned, an environment shouldn't be specifying a change to which plugins are loaded. I would suggest if a user wants to be able to do this, we either create an "enabled" option which can be defined in argv/rc, or the user takes advantage of environment variables inside their plugin, detecting what to do. The latter makes more sense to me at present.
So essentially argv/rc options are only used to define a plugins options - which means --plugins.pluginname.secretsalt 1234 maps to a plugins' options and will look similar to { name: "pluginone", options: { secretsalt: 1234 } }
I feel we still need tests for this PR, however maybe this is something we sort pre-stable?
Note: Once plugins are passed in either array or object format, they are then converted to an object (a result of running through defaultsDeep, where the default is an object), and then mapped back to an array for easy parsing. It felt somewhat dirty doing it this way, however it seems to be the cleanest as it assumes that package.json could be in any format, and plugin loading is always from an array of plugins.

sounds all good to me! I’d say let’s see how it works with your plugin? That will help us to see it in an actual real context.
What do you have in mind for the routes option here: https://github.com/agtract/hoodie/blob/10a4e5e84f254c83e510e1b76979b4617089dff3/server/config/parse-options.js#L39?
In my thinking, plugins will be similar like the core modules, they will simply be hapi plugins to extend the server, e.g. like the hodie-account-server plugin. So you get the server instance and can do whatever you want with it, including defining routes

can you rebase your PR on latest master?

@gr2m client loading, including minifying is work. Could be more efficient? I think we need to detect prod or dev environment as to when we minify, as it does slow startup down

client loading, including minifying is work. Could be more efficient? I think we need to detect prod or dev environment as to when we minify, as it does slow startup down

Yes I agree, but maybe lets do this in a follow up issue?

@gr2m latest commit allows plugins to get access to hoodie config through server.settings.app

@gr2m here's a working plugin example: https://github.com/agtract/hoodie-plugin-bundle-client
Uses an optional config option minify to test the ability to pass options, and also shows the need for server.settings.app

Can you add me to https://github.com/agtract/hoodie/ for simpler collaboration on this PR?

Added a few changes: https://github.com/agtract/hoodie/compare/plugins...hoodiehq:agtract-plugins

Things looking good to me, let’s add a few more tests with options in package.json, and overrides with rc, then let’s ship it :)

@gr2m sorry for the delays here - had a crazy past week!
I've merged in your code, all looks good. I'll run some tests against my setup, as I have a few plugins already :)
I'll see if I can get some basic tests in there today too.

As a note for anyone that may stumble across this:
I didn't want to create (yet) another repo/npm module for code that was specific only to my hoodie app. I still wanted this logic in a plugin to take advantage of what we've built above. So I decided to store this plugin logic in "./src/plugins/agtract", then link this directory using npm link. Example of what you'd see in package.json:
  "scripts": {
    "postinstall": "npm link ./src/plugins/agtract/;",
    "postupdate": "npm link ./src/plugins/agtract/;",
  },

Which means require('hoodie-agtract-plugin') would still work providing that directory had a package.json file.
The other reason for doing this was that I've split my server and client logic into two as well - so I have agtract-server and agtract-client which are dependencies for hoodie-plugin-agtract - all the later does is construct the hoodie plugin, the require the other two. Much like Hoodie's account & store plugins.

Hey @jameswestnz, we just shipped app-specific plugins \o/ #637. Check the README here: https://github.com/hoodiehq/hoodie/tree/v25.1.0#extending-hoodie-for-your-app
The implementation is really neat, and I think we are nearly done with 3rd party plugins now. All we have to discuss / figure out is how to configure the plugins that we want to load from npm packages.
Loading plugins for the server would be added to https://github.com/hoodiehq/hoodie/blob/master/server/plugins/index.js. Loading client plugins would be added to https://github.com/hoodiehq/hoodie/blob/5ec889b2b8f0ce0c3337eced8d0fc871f84d8eda/server/plugins/client/bundle.js#L31-L33
Also see the discussion at https://github.com/hoodiehq/discussion/issues/107 to move the configuration out of the package.json to be more flexible, e.g. to have configuration based on different environments, and to pass functions as values etc.
Getting close :)

Hi, we’re sorry we didn’t get this over the finish line. This is now being tackled again in https://github.com/hoodiehq/hoodie/pull/711 — closing for now, but thanks all for the contributions here <3


fix: Decrease white space above top buttons, increase space above sav…

…e/transfer, decrease space below save/transfer
Fixes #270
Changes:

Updated the layout with minor changes should fix the layout issues in small devices

@mariobehling @yashk2000 @iamareebjamal Review

Thank you. Please add a screenshot.

@mariobehling Updated with screenshot

Changes are not very visible: Please decrease white space above top buttons, increase space above save/transfer, decrease space below save/transfer.
Make it in a way that an (invisible) circle would not overlap with the transfer button, please and leaves a bit space below it.

I've removed the space above buttons competely, the 8dp margin is removed. Also added a 16dp padding above save/transfer and decresed the padding below the buttons from 32dp to 8dp. Here is the difference
Before

After

Space between red underline and circle should be same as the space between the circle and the buttons below. Could you make it into the vertical centre, please?

@mariobehling definitely

@mariobehling here

Thank you


Reset all the default css from the app

Create a folder called "index.css" in "src" folder.
in "index.css" ; Use "*" to reset margin, padding, outline and border to be "0" and also box-sizing to be "border-box".
The code should looks like this:
*{
border:0;
/rest of the code here.../
}

The CSS file in src folder has been created, and all box values seted to "0".


Upcoming events aren’t sorted chronologically

TAG is in November; the MA is in July. So I’d expect to see the MA listed before the TAG on the events page. And yet:

The later event is shown before the earlier one because currently, upcoming events are sorted reverse chronologically by start date:
https://github.com/IATI/preview-website/blob/fdf0238eea1c49e453876189dafed7aa59374c4c/iati/events/models.py#L33
While that makes sense for past events (because for past events, it’s common to start from ‘now’ and work backwards), it’s a bit jarring for upcoming events (where it’s common to start from ‘now’ and work forwards).

Fixed! #194.


Write a setup script to configure a new instance of Piecewise

A setup script should be written to replace much of the configuration of a new Piecewise instance. The script would:

assume a compatible *nix OS
Do an OS and prerequisites check
Install prerequisites if needed, prompting user
Take values from one or more configuration files and a "skeleton example" to configure a new, ready to deploy instance of the Piecewise server.
Configuration files should include values for the following, but there are likely others to be included:

User account to be used for Google APIs
Map tile provider (mapbox, OSM)
Mapbox account to be used for Mapbox tiles
Provide four coordinates for bounding box
Lat/lon coordinates for map center
Initial zoom level
Shapefile containing the areas raw data will be aggregated by
Topojson file containing corresponding areas for map front-end
File containing the form elements to define the extra_data fields and data types
Text file containing all of the application's text content

This issue was essentially an early nod toward #98, which makes it a duplicate. Closing.


E2E tests for new dcos-diagnostics API

High-level description
This PR changes test to test both legacy and current diagnostics API.

Use only one diagnostic client instead of creating one for every function
Parametrize tests to have the same source code for legacy and new API
Remove unecessary .gz suffix from expected files. For more context, see: DCOS_OSS-4531
Update dcos-test-util to version that supports new diagnostics API https://github.com/dcos/dcos-test-utils/pull/97

Corresponding DC/OS tickets (required)

DCOS_OSS-5426 Add E2E test for new API for bundle creation
DCOS_OSS-5435 Implement new API in dcos-test-utils

Related tickets (optional)

DCOS_OSS-5449
DCOS_OSS-5467

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain.
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

@mesosphere-mergebot request-review @drewvanstone @br-lewis @makkes

@mesosphere-mergebot request-review @orsenthil @jgehrcke

@mesosphere-mergebot request-review @drewvanstone @br-lewis @makkes @bamarni @rgo3

:wave: @drewvanstone @br-lewis @makkes @bamarni @rgo3  Can you review, please?

:wave: @drewvanstone @br-lewis @makkes @bamarni @rgo3  Can you review, please?


Remove duplicate DEBUG output

allow_multimon and new_cursors are printed twice.
I left the lines which make most sense with their surroundings, but feel free to switch to the other option.

Good catch!

Backport to v0.9 and v0.10.


F821 Weird failure

Hi! Love what you are building with ruff.
I'm trying it out now and found a weird behavior that i dont know if its a bug or something im missing.
def api_event_json(api_key, model_name, user_id=None, item_id=None):
    dict = {
        "headers": {
            "x-api-key": api_key,
        },
        "queryStringParameters": {"limit": "5"},
        "pathParameters": {"model": model_name},
    }

    if user_id is not None:
        dict["queryStringParameters"]["user_id"] = user_id
    if item_id is not None:
        dict["queryStringParameters"]["item_id"] = item_id

    return dict

Why would this code throw F821?
F821 Undefined name `queryStringParameters`
F821 Undefined name `queryStringParameters`

For the lines where we access the dict.
I can walk around this error by rearranging the code a bit:
def api_event_json(api_key, model_name, user_id=None, item_id=None):

    query_params = {"limit": 5}
    if user_id is not None:
        query_params["user_id"] = user_id
    if item_id is not None:
        query_params["item_id"] = item_id

    dict = {
        "headers": {
            "x-api-key": api_key,
        },
        "queryStringParameters": query_params,
        "pathParameters": {"model": model_name},
    }

    return dict

And is alright. Wondering if its a bug.

Definitely a bug! It’s treating it like a type annotation with a forward reference (eg, x: dict[“foo”] = {}, which would be referencing a variable named “foo”). But it should only be enforcing that behavior for annotations. Will fix today.

Another way to fix this is to use a name other than dict. Part of the problem is that the code is shadowing the built-in dict.

This'll go out in the next release, later today!


Add User Settings in Backend: Change Backend Colour for User

Please describe the feature you would like to see implemented.
To make it possible as visual warning for e.g. you are on the live environment, it is possible to change the backend background colour of your user to a different colour.

We work on it on the Boost Days, can't assign to me.


Add Core Configured Fonts

Is your feature request related to a problem? Please describe.
I added some fonts via the Foundry Core settings and was very surprised to see that they weren't included in the list of fonts in Health Estimate. The ones that were available were all the nonsense fonts from Polyglot, which isn't especially useful for Health Estimate.
Describe the solution you'd like
Ideally, the additional fonts from Core settings should appear in the dropdown list. Even better would be if the Polyglot fonts disappeared.
Is this specific to system or is it system agnostic?
This is system-independent.
Additional context
None that I can think of.

Thank you!