'terragrunt (run-all) show -json planfile' very slow

Describe the bug
We're using the terragrunt show command to inspect the planfile in order to automatically determine the drift and whether critical resources are being created.
However, we notice that the terragrunt version of the 'show' command is massively slower than the native terraform show.
For example some runs on the same (single) module:

terragrunt run-all show --terragrunt-non-interactive  -json planfile  32.12s user 16.94s system 115% cpu 42.445 total
terragrunt show --terragrunt-non-interactive  -json planfile  19.20s user 8.06s system 128% cpu 21.255 total
terraform show -json planfile  0.56s user 0.11s system 123% cpu 0.543 total

Note: we are using the run-all even in a single module as it allows us to use just one setting for TERRAGRUNT_SOURCE

The difference between the run-all show and regular show is explainable (although still quite a bit) as it needs to identify dependencies. Even though you would hope that when using the --exclude-external-dependencies this would be less than the roughly 10 seconds we're seeing now.
However, the terraform command is simply blindingly fast in comparison with the terragrunt counterpart, with a much bigger difference than I would expect.
To Reproduce
Steps to reproduce the behavior, code snippets and examples which can be used to reproduce the issue.
> terragrunt run-all plan  --terragrunt-ignore-external-dependencie
> time terragrunt run-all show --terragrunt-non-interactive  -json planfile

> terragrunt plan
> time terragrunt show --terragrunt-non-interactive  -json planfile

# cd into the right .terragrunt-cache directory
> time terraform show -json planfile

Expected behavior
Way smaller differences.
Versions

Terragrunt version: v0.45.15
Terraform version: v.1.4.5
MacBook Pro M2 Pro

Additional context
We are running the terragrunt show for each module one by one in a (sometimes large) project consisting of many modules, as when we are running the terragrunt run-all show we get lots of outputs but no option to find out to which module a particular output belongs. There might be ways, but we just didn't find them yet.

Hi,
first of all I wanted to clarify used CLI arguments, Terragrunt supports --terragrunt-ignore-external-dependencies CLI arg
Arguments like --exclude-external-dependencies or --terragrunt-ignore-external-dependencie aren't supported
References:
https://terragrunt.gruntwork.io/docs/reference/cli-options/#cli-options
Second, will be helpful to have a test repository where performance can be verified, from description is not clear how the repository is organized

Hi,
Apologies for the typos but the arguments are correct in reality.
Below the results (in seconds) per command:

terragrunt run-all show --terragrunt-non-interactive --terragrunt-ignore-external-dependencies -json planfile => 36,91s
terragrunt show -json planfile  => 28.73s
terraform show -json planfile => 0.96s

Having a test repo will be difficult, I am able to make the repos we work on public although I can confirm I see the same behaviour across multiple repositories I'm working on. Of course there are dependencies to other modules (which have dependencies to other modules) but nothing out of the ordinary I would expect? Anyways, I have added the output of terragrunt graph-dependencies at the end.
Are you saying that you see more or less the same response times with terragrunt show as you see with terraform show? The difference is that vast in my environment that I'm really wondering what's going on.
If that's the case it would definitely be useful to sort of reproduce it in a different environment.
digraph {
	"project-dir/terragrunt/dlzs/data-platform/global/ad-groups" ;
	"project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/ad-groups" ;
	"project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/ad-groups" -> "project-dir/terragrunt/dlzs/data-platform/global/ad-groups";
	"project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/az-devops" ;
	"project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/az-devops" -> "project-dir/terragrunt/dlzs/data-platform/global/ad-groups";
	"project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/az-devops" -> "project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/ad-groups";
	"project-dir/terragrunt/dlzs/data-platform/sbx/base" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/base" -> "project-dir/terragrunt/dlzs/data-platform/sbx/central-dns";
	"project-dir/terragrunt/dlzs/data-platform/sbx/base" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/dlzs/data-platform/sbx/central-dns" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/central-dns" -> "project-dir/terragrunt/dlzs/data-platform/sbx/networking";
	"project-dir/terragrunt/dlzs/data-platform/sbx/central-dns" -> "project-dir/terragrunt/smz/run/networking";
	"project-dir/terragrunt/dlzs/data-platform/sbx/networking" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/networking" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/dlzs/data-platform/sbx/networking" -> "project-dir/terragrunt/smz/run/networking";
	"project-dir/terragrunt/dlzs/data-platform/sbx/runtimes" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/runtimes" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/dlzs/data-platform/sbx/runtimes" -> "project-dir/terragrunt/smz/run/core";
	"project-dir/terragrunt/dlzs/data-platform/sbx/runtimes" -> "project-dir/terragrunt/dlzs/data-platform/sbx/central-dns";
	"project-dir/terragrunt/dlzs/data-platform/sbx/runtimes" -> "project-dir/terragrunt/dlzs/data-platform/sbx/base";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/base" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/base" -> "project-dir/terragrunt/dlzs/data-platform/sbx/central-dns";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/base" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" ;
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/az-devops";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/global/substacks/edh-dataprocessing-env/ad-groups";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/sbx/central-dns";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/sbx/networking";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/base";
	"project-dir/terragrunt/dlzs/data-platform/sbx/substacks/edh-dataprocessing-env/dataproc" -> "project-dir/terragrunt/dlzs/data-platform/sbx/runtimes";
	"project-dir/terragrunt/smz/run/core" ;
	"project-dir/terragrunt/smz/run/core" -> "project-dir/terragrunt/smz/run/monitor";
	"project-dir/terragrunt/smz/run/core" -> "project-dir/terragrunt/smz/run/networking";
	"project-dir/terragrunt/smz/run/monitor" ;
	"project-dir/terragrunt/smz/run/networking" ;
	"project-dir/terragrunt/smz/run/networking" -> "project-dir/terragrunt/smz/run/monitor";
}

I did a test with going to the module at the head of of the dependency graph and this makes a huge difference.
Times were ranging between 0.4 seconds (native terraform) to a bit less than 2 seconds (terragrunt run-all). And the regular terragrunt show somewhere in between.
So clearly it is the dependencies that is slowing it down.
My assumption was, that with the planfile already present on disk, there would be no need for dependency parsing but apparently that is not the case.
Not sure this is an inevitability of how terragrunt works, or something that could be altered over time.

Hello,
during run-all Terragrunt builds dependencies tree and read outputs of identified dependencies each time so it is expected to be slower
Passing plan file helps in Terraform execution and doesn't speed up dependencies identification / outputs reading

I figured something like that, but:

The regular terragrunt show is also doing something in this regard, otherwise why would it take almost 30s in our regular projects. That is definitely not what I would expect.
I would have hoped that using --terragrunt-ignore-external-dependencies while being in a module directory (with no further modules in the tree) would have reduced that significantly. But alas.

I have hacked around a little and are using the native terraform show now (when possible), which reduces showing and analyzing the planfiles from more than hour in a large project, to a few minutes. Not nice, but the overhead is simply too large to ignore.

We ran into the same issue, in our CI setup we have a special variable called TG_DISABLE_DEPENDENCIES which we use to disable dependencies when running terragrunt/terraform commands that clearly don't need to have their dependencies checked, such is a case here as well.
For each of our project folders containing a terragrunt.hcl file with a source block we add the following block
include "dependencies" {
  path = tobool(get_env("TG_DISABLE_DEPENDENCIES", "false")) ? find_in_parent_folders("generators/noop.hcl") : "./dependencies.hcl"
}

noop.hcl is an empty file while dependencies.hcl would contain your standard dependency declaration
i.e
dependency "vpc" {
  config_path = "../vpc"
}
dependency "rds" {
  config_path = "../rds"
}
inputs = {
  vpc_id = dependency.vpc.outputs.vpc_id
  db_url = dependency.rds.outputs.db_url
}

This cuts the plan time for our envs from 50-100% when using terragrunt show


Update Access.php

The parameters are wrong and causes phpstan to give errors for any code that invokes these two methods. Switching to the correct type (array) solves this.

A new inspection was created.

The inspection completed: 6 new issues

Great! Thanks @tristanr-cogapp !

Hmm actually it looks like something like array|string was better, since that's what this method accepts. Will that work fine with PHPStan @tristanr-cogapp ?


Redirection of page

Dear Team,
I am not sure if this is the right place but I need some help in the redirection of page.
Example:
I have two SharePoint communication site abc.com and xyz.com.
abc.com is for staff
xyz.com is for students
There is one more modern page 123.com
I need if the staff go to 123.com it redirects to abc.com and if student go to 123.com it should redirect to xyz.com.
Please help me in this query

How do you know they are students or staff? You could do that using a webpart that do that using window.location property

The email domain name is @mbru.ac.ae for staff and for student it is @students.mbru.ac.ae. I am not a developer so I need some assistance to achieve this.

Also I would like to let you know that am using sharepoint online

First of all you should know how to develop a custom webpart for modern pages using SPFx, look at this tutorial: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/get-started/build-a-hello-world-web-part
Then in the render method instead of render something in the HTML put the logic for compare the domain of the user using the context variable and redirect the user using "window.location.href"

I have an idea on how to create a webpart but limited to programming skills.


Get 400 error when the url paramater has { Symbol  whith use zuul proxy

I want to agent a web url with spring cloud zuul filter ,to auto Authentication . all is ok,but when the url has { ,such a left brackets . I got a 400 error. I debug for this error, It does not pass through my zuul filter.
whats the problem ?

thanks. I will close.


Add setup.py stub to support editable installs

How would this feature be useful?
Editable installs are very useful for development, but without the presence of a setup.py file pip refuses to install the project in editable mode.
Describe the solution you'd like
A simple stub for setup.py is sufficient to support editable installs since setup.cfg is used for all project description. See below for an example of what that sub would look like.
from setuptools import setup

setup()

Describe alternatives you've considered
Manually installing nox after every change...

You don't need a setup.py shim to do editable installs anymore!
Mine works as is:


Updating always resets mvc.sitemap

Every time I update via NuGet, it resets my mvc.sitemap file, which contains all my XML that lists and organizes my mvcSiteMapNodes.  I'd like the updater to leave that file alone unless the new version requires changes that aren't backwards compatible.
Most recently, this problem happened when I updated from <IP_ADDRESS> to <IP_ADDRESS>.

It seems this problem has been fixed for a long time now.  Closing this old ticket.


PDF viewer doesn't render in production builds

Configuration:

Web browser and its version: Firefox 66.0.4
Operating system and its version: Pop!_OS 18.10 (similar to Ubuntu 18.10)
PDF.js version: 2.0.943 using prebuilt pdfjs-dist
Is a browser extension: No

Steps to reproduce the problem:

Follow the instructions in this tutorial (should take less than 20 minutes)
Run the application on dev server
Build and run the production build of the application

What is the expected behavior? (add screenshot)
The PDF viewer renders as expected when running on the dev server. Ignore the purple dot, that's from a custom annotation layer.

What went wrong? (add screenshot)
The PDF viewer doesn't render at all for the production build. I'd add a screen shot, but it's an empty iframe.

Follow the instructions in this tutorial (should take less than 20 minutes)

That example is not an official one, but rather provided by a third-party, and as such it's unfortunately not possible for the PDF.js project to provide support for it.
Furthermore, please see https://github.com/mozilla/pdf.js/blob/master/.github/CONTRIBUTING.md (emphasis mine):

If you are developing a custom solution, first check the examples at https://github.com/mozilla/pdf.js#learning and search existing issues. If this does not help, please prepare a short well-documented example that demonstrates the problem and make it accessible online on your website, JS Bin, GitHub, etc. before opening a new issue or contacting us on the IRC channel -- keep in mind that just code snippets won't help us troubleshoot the problem.

@Snuffleupagus
If I recreate a codesandbox or jsfiddle with the tutorial code, would I still not be able to receive guidance?

If I recreate a codesandbox or jsfiddle with the tutorial code,

Considering that the PDF.js project only uses "regular" JavaScript, and that the example you're attempting to use seems to be based on React, the likelihood that a PDF.js contributor will be able to offer assistance cannot be guaranteed. (The people you're interacting with here are volunteers, and not paid support staff.)
Also, please keep in mind that the "PDFTron" project seems to be mostly interested in selling you there own PDF/document viewing solutions. Hence it may not be in their interest to provide a good PDF.js example, and it's also unclear which version of PDF.js the example was based on.

[...] would I still not be able to receive guidance?

As long as the example is small and self-contained a contributor may be able to offer help, but it cannot be guaranteed since it's quite probable that the issue lies in the React code.

The people you're interacting with here are volunteers, and not paid support staff.

I am aware and I thank you for taking the time to help me!

As long as the example is small and self-contained a contributor may be able to offer help, but it cannot be guaranteed since it's quite probable that the issue lies in the React code.

Here is a minimal implementation  of the PDFTron tutorial. In order to build a production version, you may need to download the contents and run npm react-scripts build" from the project root:
https://codesandbox.io/s/oo4j6xl1lq?fontsize=14
I understand the folks at PDF.js won't necessarily be able to help me, but I appreciate any and all assistance with this.

In order to build a production version, you may need to download the contents and run npm react-scripts build from the project root:

If it works in one React configuration, but not another one, that would point rather clearly to a problem with React rather than in the PDF.js library (since otherwise it should be consistently broken). In cases such as these, I'd assume that you'd probably get more useful insight from people actually familiar with React.
(Also, an example that requires both downloading of code and setting up React locally is probably not a great reduced test-case, but maybe that's just me :-)

Ah, ok. I'll keep hunting for a solution. Closing this for now!

@kwayebopp Excuse me, How to solve this problem?


Atom highlight support

Description
This PR allows users to provide numerically tagged smiles strings that can be used to highlight atoms in the molecule.
Questions

[ ] Should we highlight the bonds between these atoms?  or maybe have another arg to turn on bond highlighting?

Status

[ ] Ready to go

Thanks for this PR @jthorton - this will be a nice addition!
I wonder if the scatter function should take an options object as an arg / kwarg similar to:
@dataclass()
class MoleculeDrawOptions:

    highlight_indexed_atoms: bool = True
    highlight_indexed_bonds: bool = True

which then gets passed to smiles_to_svg and controls how the 2D structure will be rendered? That way we can build up these options gradually without having too many args.


INT B-21480 Full PPM Counseling Type

Agility ticket
B-21480 https://www13.v1host.com/USTRANSCOM38/assetdetail.v1?number=B-21480
Summary
All full PPM moves (meaning orders with only PPM shipments on them) will not be routed to the TOO and will only be routed to the service counselor.
Is there anything you would like reviewers to give additional scrutiny?
this article explains more about the approach used.
Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Agility acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

[ ] Has the branch been pulled in and checked out?
[ ] Have the BL acceptance criteria been met for this change?
[ ] Was the CircleCI build successful?
[ ] Has the code been reviewed from a standards and best practices point of view?

Setup to Run the Code

Instructions for starting storybook
Instructions for starting the MilMove application
Instructions for running tests

How to test

Login as a customer or Service Counselor.
Create a PPM that is the only move on the orders, or any other moves on those orders are also PPM moves (with the following details)
Create the move with an origin location of Mayport, FL 32228 (NS Mayport FL) and a destination zip of 62225 (Scott Air Force Base) -Mayport, FL is the only gbloc that currently routes to the TOO for Prime Counseling (we are verifying a PPM does not route to Prime Counseling)
Verify the move is created, shows on the counselor queue, and is put in the status "Needs Service Counseling"
Login as TOO
Verify the TOO does not have the move in their queue
search for the move
Verify the TOO does not have the MTO Service Items with the option to add "Counseling"

Frontend

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, Edge).
[ ] There are no new console errors in the browser devtools.
[ ] There are no new console errors in the test output.
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.
[ ] This change meets the standards for Section 508 compliance.

Backend

[ ] Code follows the guidelines for Logging.
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys.
[ ] Have been communicated to #g-database.
[ ] Secure migrations have been tested following the instructions in our docs.

Screenshots

Warnings

:warning:
This PR does not include changes to unit tests, even though it affects app code.

  Generated by :no_entry_sign: dangerJS against 74ab369a1ad725b62299847bc2df02a149a60869

@elizabeth-perkins looks like one of the test relating to ppm is failing not sure if its due to your changes though.

Test coverage on client failed.
Last run: Fri 01 Nov 2024 07:56:27 PM UTC
Refer to this report to see coverage details for the client app.
Documentation for test coverage can be found at this link.

Warnings

:warning:
This PR does not include changes to storybook, even though it affects component code.

  Generated by :no_entry_sign: dangerJS against a720f251e935ef776d42a9d61f2788509f5f1abe

Accidentally made that last commit directly on this INT branch instead of on Main and pulling it in.  Fixing now.

Failed test is a client_test labeled as flaky.  Merging PR.


Removing OS Specification and Using PSGallery to Install Pester in AppVeyor File

This is for PowerShell/DscResources#147.

This change is

PSSA errors failing the tests have been submitted under #24.


x/tools/internal/lsp: TestLSP/GOPATH/CompletionSnippets flake on darwin-amd64-10_14 builder

From the darwin-amd64-10_14 builder in https://build.golang.org/log/d8674baf25fbfd7a1da3118451ccb1b6efcedec2:
--- FAIL: TestLSP (240.36s)
    --- FAIL: TestLSP/GOPATH (189.27s)
        --- FAIL: TestLSP/GOPATH/CompletionSnippets (10.14s)
            --- FAIL: TestLSP/GOPATH/CompletionSnippets/deep_23_9_placeholders (10.00s)
                completion_test.go:42: /var/folders/9w/4l2_g3kx01x199n37fbmv3s80000gn/T/workdir-host-darwin-10_14/tmp/TestLSP_GOPATH313029317/lsp/src/golang.org/x/tools/internal/lsp/deep/deep.go:23:9-10: couldn't find completion matching "a.b"
FAIL
FAIL	golang.org/x/tools/internal/lsp	240.631s

It looks like this continues to happen. @muirrn: Any reason that you know of for that test to flake?

My first thought would be completion running out of budgeted time and not performing the deep search, thus not finding the a.b candidate. That test sets the budget to 5 seconds, so it would have to be a major hiccup. We could try just turning up the budget or tweaking tests to have unlimited budget.

I guess we do see that single test taking 10.00s in the output, so probably just the occasional slow run. I'll look at disabling the time limit for tests.

I believe the flakiness should be fixed as of the above CL.


Possible improvements

At @marc-barry's demand, here a couple things that could make usage simpler.
dedicated command for initial setup & migrations
In the readme you give a command to setup the db: https://github.com/digitalocean/docker-shipit-engine#database-management
Would be great if this could be done in a single command handled by the entry point, e.g. docker run shipit setup, and possibly docker run shipit upgrade to run post upgrade.
pubsubstub might cause issues in this setup
Something that's a bit finicky with Shipit is the SSE connection handling. Long story short client will keep a long lived connection per stack page. These aren't very expensive, but if your Puma config is a bit too strict in term of threads, you can easily hold all thread.
The ideal setup is to have dedicated containers for handling the streaming endpoints, see: https://github.com/Shopify/shipit-engine/issues/496#issuecomment-351429330 for more details. But that makes for a more complicated setup as you need a LB etc.
Let me know if you need more details on that.
Database
Maybe I misread, but it seems you are focusing on postgres, I guess it's fine, but I suppose MySQL could be supported as well (and is a much better tested solution with Shipit).
I'll update this issue if something else comes up.

@casperisfine added dedicated commands via Entrypoint for either setting up or upgrading (running migrations) a Shipit database - https://github.com/digitalocean/docker-shipit-engine/pull/4

Thanks @casperisfine. We'll work on some pull requests to address the comments in this issue. We are currently running ShipIt with https://www.digitalocean.com/products/managed-databases/ which supports PostgreSQL and hence our emphasizes on that adapter.


Add executePlugin tests

The e2e test (snapshot) caught a additional property runner that slipped through when refactoring executePlugin.
This should have been caught by one of the executePlugin tests.
Test the following scenarios:

failed plugins - here we need a better error message

Related to #215

closed in favour of #356


Use public API for OSSF Scorecard

https://github.com/ossf/scorecard/pull/1682

Can you explain a little bit more? What needs to be done exactly? I'm willing to do this but need some more context for me to understand the need better. Thx


xRemoteFile re-downloads file every time

I'm using push mode and need to download a large file from HTTP server. xRemoteFile works ok, but it always downloads the file (even if it didn't change), which takes long time to finish. This is because xRemoteFile uses local cache and compares LastWriteTime, but the cache is cleared every time I run Start-DscConfiguration.
MSFT_xRemoteFile.psm1:
"File" {
            Write-Debug "DestinationPath: '$DestinationPath' is existing file on the machine"

            $file = Get-Item $DestinationPath
            # Getting cache. It's cleared every time user runs Start-DscConfiguration
            $cache = Get-Cache -DestinationPath $DestinationPath -Uri $Uri

            if ($cache -ne $null -and ($cache.LastWriteTime -eq $file.LastWriteTimeUtc))
            {
                Write-Debug "Cache reflects current state. No need for downloading file."
                $fileExists = $true
            }
            else
            {
                Write-Debug "Cache is empty or it doesn't reflect current state. File will be downloaded."
            }
        }

If we're not able to have a persistent cache between Start-DscConfigurations (I'm not sure what are the limitations here), could we possibly add $MatchSource parameter (similar to File resource) - and when it's $false Test-TargetResource would return $true if the file just exists in the destination?
In my use case I'm certain the file will not change on the server as I point to a specific version of the software.
Please let me know your thoughts - I will try to provide a pull request if we're able to agree on approach.
Code to reproduce the issue:
Configuration Test {

    Import-DSCResource -Module xPSDesiredStateConfiguration

    Node localhost {
        xRemoteFile SolrZip {
            Uri = 'http://apache.mirror.anlx.net/lucene/solr/5.3.1/solr-5.3.1.zip'
            DestinationPath = 'c:\solrtest\solr-5.3.1.zip'
        }
    }
}

Test -OutputPath 'Test'

Start-DscConfiguration -ComputerName localhost -Path 'Test' -Wait -Verbose

Hi @mgr32
Cache is being cleared every time Start-DscConfiguration is being run manually, but not when the consistency engine kicks in. So in a usual scenario where you rely on DSC engine to maintain state of your machines, file will not be redownloaded every time.
If that doesn't satisfy your scenario, could you please give more details about it?
Thanks!

I'm using DSC for deploying an application in push mode (from CI server) - like described in this blog entry but using PSCI (example) instead of TFS Release Management. I need to have control when exactly the application is deployed and want to do it synchronously, plus I want to be able to deploy without any infrastructure changes (no need to setup pull server), thus I'm using push mode.

In that case a workaround you could use is to change path where cache for this resource is being stored, line 14:
# Path where cache will be stored. It's cleared whenever LCM gets new configuration.
14 $script:cacheLocation = "$env:ProgramData\Microsoft\Windows\PowerShell\Configuration\BuiltinProvCache\MSFT_xRemoteFile"

If you change it to some custom folder (needs to be outside of the BuiltinProvCache directory), it will not be cleared.

Thanks for the workaround. However, as this doesn't seem to be acceptable final solution, I understand we're not currently able to fix it on the dev branch.

I think we should implement the MatchSource property in a similar way to how File resource implements it, however until it's done, the above workaround is the only way to get this scenario working.

Thank you, if that's the case I will try to provide a pull request when time allows :)

Thanks! That would be much appreciated.

I'm having this exact same issue. Some sort of MatchSource would be perfect.
@KarolKaczmarek, the README.md file under xRemoteFile
"Ensure: Ensures that the local file is Present or Absent on the local system", however the Ensure property doesn't exist.

@Zuldan It actually have Ensure property, however it's read only:
https://github.com/PowerShell/xPSDesiredStateConfiguration/blob/dev/DSCResources/MSFT_xRemoteFile/MSFT_xRemoteFile.schema.mof#L10
It's main purpose is to tell you whether file in path specified by DestinationPath is present or not. I agree the name doesn't reflect it's meaning very well, but it's a common pattern among multiple resources to call it ensure, so we left it for consistency.
I've updated it's description in readme.md to match what's in the schema.mof file.