Update "Bug Reports" section of CONTRUBUTING.md

We need to make sure that this section is clear enough for bug reports, because it would be nice to link to it when bug report is not good (for example it is missing a way to reproduce the issue).
@faceleg Could you handle it maybe?

@sheerun If @faceleg  does not have time I can take this over and lay out proper procedures for reproducing steps for the bug report.

@pertrai1 please be my guest!

@pertrai1 Could you for now create a Wiki page so everyone can contribute?

@sheerun  I have started the wiki page with the outline of a bug report and will fill in the details of the outline.

@sheerun - I am going to go ahead and close this issue now that the wiki page has been created. We can continue to evolve the wiki page as I have more content that I need to add to each section of the bug report.


doc update, the bundle should be registered last

Maybe it should be specified that the bundle should be registered at the end of your appkernel file since we override some services aliases provided in symfony framework bundle.
Actually I spend one hour figuring out why the bundle wouldn't work because of that mistake ^^

It doesn't need to be registered at the end, just after the framework bundle. Standard practice is to register most bundles after the core SF bundles. I don't think many people will hit this issue.


chore: Bump version to 1.3.2

What this PR does / why we need it:
Bump version once more and include appropriate changelogs this time.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:

None

/lgtm


Allow passing custom REST configuration settings to extension shoot clients

How to categorize this PR?

/area performance
/kind enhancement
What this PR does / why we need it:
This PR allows overwriting certain REST configuration settings for shoot clients in extensions.
These are the options which could be overwritten
type RESTOptions struct {
	// QPS indicates the maximum QPS to the master from this client.
	QPS *float32
	// Maximum burst for throttle.
	Burst *int
	// The maximum length of time to wait before giving up on a server request. A value of zero means no timeout.
	Timeout *time.Duration
}

In healthcheck controller these options could be applied by using DefaultAddArgs.HealthCheckConfig.ShootRESTOptions structure.
Which issue(s) this PR fixes:
Fixes #6045
Special notes for your reviewer:
There is currently no test yet. The entire util and healthcheck packages have a lack of tests.
/cc @ialidzhikov @acumino
This is the second version of PR https://github.com/gardener/gardener/pull/6113 considering now, that extensions reuse data types in their data types.
Release note:

Allow passing custom REST configuration settings (QPS, Burst, Timeout) to extension shoot clients.

/retest

There is a release note for the breaking change now.
Regarding the future work on #6045. @timuthy could you share your observations which extensions are affected by the issue?

There is a release note for the breaking change now.
Regarding the future work on #6045. @timuthy could you share your observations which extensions are affected by the issue?

In my case the node check of the Azure extension ref was affected. However, this is a very generic check and thus I think it's not related to a particular extension.
The problem was more the API discovery calls that were executed when the health check was performed. In addition, there was an unavailable apiservice so that discovery was even performed twice which intensified the client-side throttling.


M m leapset fix

If I understand #928 correctly then this should give the same result without the need for the transformation.

I like this! :white_check_mark:

One question that @devinivy has asked in https://github.com/balderdashy/waterline/pull/928#issuecomment-93990300: is it legitimate to have undefined pas? Or should waterline throw an error there?

I can't think of a situation where you would get a record back that didn't have a primary key. The only way this should happen would be if you were using columnName in the map which this PR should fix.
Anyone have any ideas how/why it would happen?

Looks good to me.  :+1:


Relies on BCR for maliput_malidrive binaries.

🎉 New feature
Closes #8
Requisites

[ ] We need maliput_malidrive 0.2.1 release: https://github.com/bazelbuild/bazel-central-registry/pull/1423

Summary

Relies on maliput_malidrive in BCR -> 0.2.1 version

Test it
cargo run --example create_rn

Checklist

[x] Signed all commits for DCO
[ ] Added tests
[ ] Added example and/or tutorial
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] Consider updating Python bindings (if it affects the public API)

:exclamation:  CI will fail until https://github.com/bazelbuild/bazel-central-registry/pull/1423 is merged

@francocipollone can you please take a look at https://github.com/bazelbuild/bazel-central-registry/pull/1423, two presubmit checks failed. I think we are missing a few formalities there.

Changes here LGTM btw.

@francocipollone can you please take a look at bazelbuild/bazel-central-registry#1423, two presubmit checks failed. I think we are missing a few formalities there.

https://buildkite.com/bazel/bcr-presubmit/builds/3456#018d6ba5-7ba7-46ea-a71e-b8e89d46e6ff
This is the log and I was expecting a BCR maintainer to inform us if anything is missing.
Pinging @stonier as BCR savvy here :smiley: . Do you have extra info in case we are missing something here?

Pinging @stonier as BCR savvy here 😃 . Do you have extra info in case we are missing something here?

I guess the text in that log is pertinent, thanks for finding that.

It explains why it's waiting on a BCR maintainer to process the PR
We should work out how to put the bazel version into that yaml (must be new)

Created PR to update presubmit file with bazel version: https://github.com/maliput/maliput_malidrive/pull/275
Pushed a fix to bazel content registry: https://github.com/bazelbuild/bazel-central-registry/pull/1423


Support for HTML abbr tags

Discussed in https://github.com/scalar/scalar/discussions/1378

Originally posted by cpbr-dev April  4, 2024
Would love to see support for HTML abbreviation elements, specifically support for the title attribute.
I'm currently reworking API documentation for better accessibility and multiple acronyms are being used there which could benefit from this.

#1860 👀


Flake: waiting for catalog pod to be available (for sync) - IDLE

This is flake is occasionally consistent throughout workflow runs, but difficult to reproduce locally. Link to a GHA workflow run that contains this: https://github.com/operator-framework/operator-lifecycle-manager/pull/2367/checks?check_run_id=3760863789.
There are a couple of test instances where this issue has propagated:

User defined service account/with permission
User defined service account/with no permission
Operator Group/ static provider
Install Plan/with CRD schema change/existing version is present in new CRD (deprecated field)

Creating catalog source scoped-catsrc-785ll in namespace scoped-ns-xrfbn...
Catalog source scoped-catsrc-785ll created
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
waiting for catalog pod scoped-catsrc-785ll to be available (for sync) - IDLE
...
• Failure [300.404 seconds]
User defined service account
/home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/user_defined_sa_test.go:25
  with permission [It]
  /home/runner/work/operator-lifecycle-manager/operator-lifecycle-manager/test/e2e/user_defined_sa_test.go:82

  	Error Trace:	user_defined_sa_test.go:110
  	            				runner.go:113
  	            				runner.go:64
  	            				it_node.go:26
  	            				spec.go:215
  	            				spec.go:138
  	            				spec_runner.go:200
  	            				spec_runner.go:170
  	            				spec_runner.go:66
  	            				suite.go:79
  	            				ginkgo_dsl.go:238
  	            				ginkgo_dsl.go:221
  	            				e2e_test.go:57
  	Error:      	Received unexpected error:
  	            	timed out waiting for the condition
  	Test:       	User defined service account with permission

cc @dinhxuanvu


Code block styling

I am using the default dark theme in Subway Tooter.
When I see text marked-up as a code block it is impossible to read, because their text is black on a dark grey background.

what server you use?
to clarify, ST does not directly read markdown, just read HTML tags that
from server.
customized mastodon servers translate markdown to HTML, but it's HTML
tagging rule is not same in all servers, all fork.
Without knowing which server and which fork you're talking about, we can't
investigate the problem.
2023年8月27日(日) 10:28 ryliejamesthomas @.***>:

I am using the default dark theme in Subway Tooter.
When I see text marked-up as a code block it is impossible to read,
because their text is black on a dark grey background.
—
Reply to this email directly, view it on GitHub
https://github.com/tateisu/SubwayTooter/issues/254, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACRQ6BQOW36Q4SWDJC65LTXXKPFLANCNFSM6AAAAAA376SSPE
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

My server is Mastodon, but I believe the code blocks I've see have come from Akkoma. I'll share examples when I find them again.

mastodon is changing  sanitize HTML tags in post from remote server by
mastodon version.
2023年8月27日(日) 16:30 ryliejamesthomas @.***>:

My server is Mastodon, but I believe the code blocks I've see have come
from Akkoma. I'll share examples when I find them again.
—
Reply to this email directly, view it on GitHub
https://github.com/tateisu/SubwayTooter/issues/254#issuecomment-1694594363,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACRQ6DGFBD3VOQAJCETRCDXXLZRHANCNFSM6AAAAAA376SSPE
.
You are receiving this because you commented.Message ID:
@.***>

This is still very much an issue. My Mastodon version is always kept up to date (server is run by masto.host).
I've noticed if I change to a theme with white background it is readable. The background colour applied to code blocks is effected by the post background colour.
I've noticed that inline code is readable, because the font colour is the same as my theme's colour, whereas the code block font colour is always black no matter what settings I change.
I can post screenshots if it will help.


Does Prometheus have a phantom data directory?

The system I use have 3 servers, I see all of them are working and active through Nodetool, still I get that I have a dead node. and the information on the metrics is way off.
I Can't get rid of a phantom data, that keeps creeping on the system.
After deleting the docker volumes, reinstalled docker, reinstalled the grafana tool. Some phantom data keeps coming back, cleared the browser cache and killed the collector exporters on the servers.
Is it possible to have a way to clear all history data the tool has accumulated in the past?
./kill-all.sh only removes the current docker nodes running, and it is not clearing the information.

@eyalgutkind can you please try
sudo docker rmi aprom agarf after running kill-all.sh?
Thanks

In my case I had to remove the hash number for the images
docker rmi e180d8c8c83c9e937cd53cd35441aff8afdcc675e8381841fccdaf1375a5cd11
By name it didn't work.
Also, what is the decision criterion for the name of the instance Prometheus decide to fetch data by?
On the network metrics, the information is taken by the host name

However, on the load metric it doesn't capture the hostname:

The network metrics are reported directly from collectd interface plugin, while load metric is coming from scylla.
Both are going to change on 1.4
@amnonh can you validate this point for 1.4?

@eyalgutkind could it be that the Scylla server report old data (not the monitoring present old data)?
Go to scylla server 9103 port, and look for out of date metrics

prometheus does remember values and metric names between sessions
On Wed, Oct 5, 2016 at 9:29 AM, Tzach Livyatan<EMAIL_ADDRESS>wrote:

@eyalgutkind https://github.com/eyalgutkind could it be that the Scylla
server report old data (not the monitoring present old data)?
Go to scylla server 9103 port, and look for out of date metrics
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/scylladb/scylla-grafana-monitoring/issues/71#issuecomment-251592810,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBRv8vRCv0ejctzCTM4O3H-fyFxItYcks5qw0OsgaJpZM4KNW9L
.

@eyalgutkind Scylla exports metrics (e.g. load) using $(hostname) as host name. You can override it by passing --collectd-hostname xxx on Scylla's command line.
Alternatively, you can tell prometheus to ignore the hostname tag which comes with collectd metrics, and use the ip as configured for given source in prometheus.yml. To do that, change honor_labels to false in prometheus.yml. Downside of this change is that you can't have one exporter for a collectd instance which gathers metrics from multiple nodes.

@eyalgutkind About the dead node counter, the query for it looks like this count(up)-count(collectd_processes_ps_code{processes="scylla"}>0). It counts how many exporters are up and subtracts from that how many scylla processes there are. For counting processes it relies on some collectd plugin (tzach: which?). Maybe you don't have that plugin included on your Scylla nodes?

@tgrabiec you are right on the money.
The problem was with the processes plugin.

About the dead node counter, the query for it looks like this count(up)-count(collectd_processes_ps_code{processes="scylla"}>0)

This is less than ideal, see #65 on the same subject

Can we close this issue?

Closing since this issue no longer exist.


Benchmark-2 Nightly Issues

Recurring issues are seen on the nightly benchmark-2 server.  Benchmarks runs that previously worked without any breaking changes from either DHC or Benchmark suddenly fail.  Reinstalling the docker engine entirely fixes it for some weeks, but then the issue recurs.  This ticket is meant to collect information seen on the failed server for more analysis and hopefully a faster way to reproduce.
Symptoms:

Previously working benchmarks fail because docker ports hang (This happens after weeks of installing/uninstalling docker images)
Disk usage is seen at 100% on root.  Doing du -a /var | sort -n -r reveals that even after removing containers/images and pruning, docker volume and overlay data remains

Below is commonly seen in /var/log/syslog when problems start happening.  Is this normal when restarting docker containers for DH and redpanda?
Jun 26 06:34:25 localhost systemd[1]: docker-0b1920a0b7f6cfc40eb104af9d012ad699191ef08e0ae497b2043da49716104e.scope: Consumed 9.173s CPU time.
Jun 26 06:34:25 localhost containerd[2252]: time="2023-06-26T06:34:25.226960963Z" level=info msg="shim disconnected" id=0b1920a0b7f6cfc40eb104af9d012ad699191ef08e0ae497b2043da49716104e
Jun 26 06:34:25 localhost containerd[2252]: time="2023-06-26T06:34:25.226998157Z" level=warning msg="cleaning up after shim disconnected" id=0b1920a0b7f6cfc40eb104af9d012ad699191ef08e0ae497b2043da49716104e namespace=moby
Jun 26 06:34:25 localhost containerd[2252]: time="2023-06-26T06:34:25.227004249Z" level=info msg="cleaning up dead shim"
Jun 26 06:34:25 localhost dockerd[2366]: time="2023-06-26T06:34:25.226977856Z" level=info msg="ignoring event" container=0b1920a0b7f6cfc40eb104af9d012ad699191ef08e0ae497b2043da49716104e module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
Jun 26 06:34:25 localhost containerd[2252]: time="2023-06-26T06:34:25.251683723Z" level=warning msg="cleanup warnings time=\"2023-06-26T06:34:25Z\" level=info msg=\"starting signal loop\" namespace=moby pid=1325686 runtime=io.containerd.runc.v2\n"
Jun 26 06:34:25 localhost systemd-networkd[957]: veth22fa434: Lost carrier
Jun 26 06:34:25 localhost kernel: [1534557.347839] br-6b192c0f0bb8: port 1(veth22fa434) entered disabled state
Jun 26 06:34:25 localhost kernel: [1534557.347872] vethf4caab0: renamed from eth0
Jun 26 06:34:25 localhost networkd-dispatcher[1340]: WARNING:Unknown index 45866 seen, reloading interface list
Jun 26 06:34:25 localhost systemd-networkd[957]: veth22fa434: Link DOWN
Jun 26 06:34:25 localhost kernel: [1534557.450674] br-6b192c0f0bb8: port 1(veth22fa434) entered disabled state
Jun 26 06:34:25 localhost kernel: [1534557.452113] device veth22fa434 left promiscuous mode
Jun 26 06:34:25 localhost kernel: [1534557.452127] br-6b192c0f0bb8: port 1(veth22fa434) entered disabled state
Jun 26 06:34:25 localhost networkd-dispatcher[1340]: ERROR:Unknown interface index 45866 seen even after reload
Jun 26 06:34:25 localhost networkd-dispatcher[1340]: WARNING:Unknown index 45866 seen, reloading interface list
Jun 26 06:34:25 localhost networkd-dispatcher[1340]: ERROR:Unknown interface index 45866 seen even after reload
Jun 26 06:34:25 localhost systemd[1]: run-docker-netns-2394c40def07.mount: Deactivated successfully.
Jun 26 06:34:25 localhost systemd[1]: var-lib-docker-overlay2-636a8a15a28dd50ac433ef45faf51edba97943c0dac3fb1766f4288333cb251f-merged.mount: Deactivated successfully.

The following error is recent and not seen the previous times this issue occurred.
file '/var/log/auth.log'[8] write error - see https://www.rsyslog.com/solving-rsyslog-write-errors/ for help OS error: No space left on device [v8.2112.0 try https://www.rsyslog.com/e/2027 ]

7/21/2023 - Yet another issue in syslogs (This one corresponds with a timeout on Redpanda listTopics())
Jul 21 09:50:35 localhost systemd-udevd[2947697]: veth552822d: Could not set offload features, ignoring: No such device
Jul 21 09:50:35 localhost systemd-udevd[2947697]: Using default interface naming scheme 'v249'.
Jul 21 09:50:35 localhost networkd-dispatcher[1320]: ERROR:Unknown interface index 14099 seen even after reload
Jul 21 09:50:35 localhost networkd-dispatcher[1320]: WARNING:Unknown index 14099 seen, reloading interface list
Jul 21 09:50:35 localhost networkd-dispatcher[1320]: ERROR:Unknown interface index 14099 seen even after reload
Jul 21 09:50:35 localhost systemd[1]: run-docker-netns-1f4de79b8c83.mount: Deactivated successfully.
Jul 21 09:50:35 localhost systemd[1]: var-lib-docker-overlay2-b734934969c00717e8106b919c5d2c05eabd8798bfc9b73765228520e28f628f-merged.mount: Deactivated successfully.

PR https://github.com/deephaven/benchmark/pull/141

The issue has not happened for quite a while. Closing.


Added the Spock Web Development library

haskellPackages are deprecated in favour of haskell-ng.packages. Please see  https://github.com/NixOS/nixpkgs/pull/7465


main: refactor libre_init and re_global handling

Fixes #403
re_global is now created by libre_init (first re_thread_init) and deleted with libre_close and leaves a clean state (no leaking re_global pointer). This breaks tmr_debug  (shows nothing) if called after libre_close.
These changes are needed on application side:
/* check for open timers */
tmr_debug();

libre_close();

/* check for memory leaks */
mem_debug();

Also fd_setsize is affected if called before libre_init, new order:
libre_init();
fd_setsize(-1);

I think leaving libre in a clean state after libre_close is worth it. I added sanity checks and warnings (like fd_setsize: re not ready) for applications.

mtx_destroy calls are missing in thread_destructor, re_thread_close and re_alloc on the fail path. Also, you might want to re freeing code to a separate function and call it from thread_destructor and re_thread_close.
Also, in re_thread_enter, re->thread_enter need to be set inside the lock. Otherwise one thread may set it and block, the other thread would reset it in re_thread_leave and unblock the first one, and then the first thread fails re_thread_check check at some point.

In the latest version, in re_alloc, if mtx_init fails, mem_deref calls re_destructor that will call mtx_destroy on a non-initialized mutex. I think, you could call re_destructor manually in thread_destructor and re_thread_close instead of relying on mem_deref.

Sorry to nitpick, but mtx_alloc has the same problem - it will call mtx_destroy if mtx_init fails.
One solution might be a new mem_set_destructor method to set the destructor function. You would call mem_alloc with NULL destructor first, then mtx_init, and then mem_set_destructor after it succeeds.

Thanks for pointing this out, I added a new helper function as you mentioned:
https://github.com/baresip/re/pull/406

This version (with #406) seems to be working well for me. Thanks.


Circle onPress not working

Summary
Can't execute an action by touching on a circle
Reproducible sample code
<Circle
                      radius={50}
                      center={{
                        latitude: parseFloat(latitude),
                        longitude: parseFloat(longitude),
                      }}
                      key={`mapShape${index}`}
                      strokeColor="green"
                      fillColor="green"
                      strokeWidth={1}
                      onPress={event => {
                        if (shape.submissionID && shape.id) {
                          mapShapeOnPressHandle(
                            shape.submissionID,
                            shape.id,
                            event.nativeEvent.coordinate,
                          );
                        } else {
                          showNotFoundMessage();
                        }
                      }}
                      tappable={true}
                    />

Steps to reproduce
Touch a circle
Expected result
Execute an action
Actual result
Not working
React Native Maps Version
1.18.0
What platforms are you seeing the problem on?
Android, iOS (Apple Maps), iOS (Google Maps)
React Native Version
0.74.3
What version of Expo are you using?
Not using Expo
Device(s)
Ulefone
Additional information
No response

Same problem here. Did you find any solution or workaround?


Improving the UI of Home Page

Currently, the UI of website is not attractive. Different elements are overlapping on each other. Header is of extra height which made it overlapping with the hero section and also forms are not aligned perfectly.
I would like to add attractive UI to home page so that user will attract towards the website and explore more about the website.

i can make better ui @GarimaSingh0109 @Ishika-Singhal

pl assign me the task

Please assign me this I will make the home page attractive.


Presenter::intercept can have multiple handlers

In order to achieve consistency with effects and domains, the
presenter intercept method can now accept an array of handlers.

looks great 👍

Thanks guys!

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@b136cee). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master     #479   +/-   ##
=========================================
  Coverage          ?   97.65%
=========================================
  Files             ?       29
  Lines             ?      681
  Branches          ?      143
=========================================
  Hits              ?      665
  Misses            ?       13
  Partials          ?        3

Impacted Files
Coverage Δ

packages/microcosm-dom/src/utilities.js
100% <100%> (ø)

packages/microcosm-dom/src/presenter.js
100% <100%> (ø)

packages/microcosm-dom/src/intercept.js
100% <100%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b136cee...9f77ad4. Read the comment docs.


Support generic handlers

I switched from MediatR to Mediator recently, for sourcegen instead of reflections. Alas, it doesn't seem to support generic handlers, and I have a specific use case for them:
public sealed record Command<T>(...) : IRequest<ActionResult> where T : BaseModel, IBlockableContent;

public class Handler<T>(ApplicationDbContext context) : BaseHandler, IRequestHandler<Command<T>, ActionResult>
    where T : BaseModel, IBlockableContent
{
    public async ValueTask<ActionResult> Handle(Command<T> request, CancellationToken cancellationToken)
    {
        // snip
        var res = await context.Set<T>() // here's where the generic is important
            .Where(i => i.Id == itemId)
            .ExecuteUpdateAsync(i => i.SetProperty(p => p.ContentBlock, cb), cancellationToken);
        // snip
    }
}

[HttpPost("story")]
public async Task<ActionResult> BlockStory(BlockContent.Command<Story> data)
    => await mediator.Send(data);

[HttpPost("chapter")]
public async Task<ActionResult> BlockChapter(BlockContent.Command<Chapter> data)
    => await mediator.Send(data);

[HttpPost("blogpost")]
public async Task<ActionResult> BlockBlogpost(BlockContent.Command<Blogpost> data)
    => await mediator.Send(data);

In MediatR one could register it by manually registering handler:
services.AddSingleton<IRequestHandler<Command<Story>, ActionResult>, Handler<Story>>()

But it doesn't seem to work in Mediator
I'm still getting the error about handler not being found error.

True, doesn't support generic messages yet. Will require a refactoring of some things so not sure that will possible. I double checked, and it is documented in the "differences from MediatR" section in the readme: https://github.com/martinothamar/Mediator?tab=readme-ov-file#6-differences-from-mediatr
There is no support for manually registering message handlers, might be possible to change though, by checking the service collection before registering, though there may be more issues down the road..
The fundamental issue is that the sourcegen'd implementation is based around knowing concrete types, underlying handling functions need to namethem (it's not simply message.GetType())

Closing this and keeping track of #76

Hello, are there any updates on this topic yet? 😄
I was attempting to create generic entity notifications, such as EntityCreatedEvent<TEntity> and EntityUpdatedEvent<TEntity>, and use them within the CRUD functions of my RepositoryBase class. However, I encountered errors in the generated code. I did some research and eventually found my way here.
Is this feature still being considered?

I found a temporary solution to my intent like this:
public interface IDomainEventPublisher
{
    public ValueTask PublishEntitiesCreatedEventAsync(IEnumerable<Entity> entities, CancellationToken cancellationToken = default);
}

public class MediatorDomainEventPublisher : IDomainEventPublisher
{
    private readonly IMediator _mediator;

    public MediatorDomainEventPublisher(IMediator mediator)
    {
        _mediator = mediator;
    }

    public async ValueTask PublishEntitiesCreatedEventAsync(IEnumerable<Entity> entities, CancellationToken cancellationToken = default)
    {
        Type? entityType = entities.GetType().GetGenericArguments()[0];

        ValueTask publishTask = entityType switch
        {
            Type t when t == typeof(ProductionTask) => _mediator.Publish(new ProductionTasksCreatedEvent(entities.Cast<ProductionTask>()), cancellationToken),

            _ => ValueTask.CompletedTask
        };

        await publishTask;
    }
}

services.AddScoped<IDomainEventPublisher, MediatorDomainEventPublisher>();

I call publish methods inside my RepositoryBase:
public abstract class EfRepositoryBase<TEntity> : IRepository<TEntity> where TEntity : Entity
{
    protected EfDbContext Context { get; }
    protected IDomainEventPublisher DomainEventPublisher { get; }

    public EfRepositoryBase(EfDbContext context, IDomainEventPublisher domainEventPublisher)
    {
        Context = context;
        DomainEventPublisher = domainEventPublisher;
    }

    public async ValueTask CreateRangeAsync(IEnumerable<TEntity> entities, CancellationToken cancellationToken = default)
    {
        await Context.AddRangeAsync(entities, cancellationToken).ConfigureAwait(false);

        await DomainEventPublisher.PublishEntitiesCreatedEventAsync(entities, cancellationToken).ConfigureAwait(false);
    }
}

I need to modify this publisher's switch statements for each new Event I create though.


Errors with angular 6 using angular-cli

I'm submitting a ...  (check one with "x")
[x ] bug report => search github for a similar issue or PR before submitting
[ ] feature request
[ ] support request

Current behavior

Expected behavior

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

Angular version: 6

Browser: [all | Chrome XX | Firefox XX | IE XX | Safari XX | Mobile Chrome XX | Android X.X Web Browser | iOS XX Safari | iOS XX UIWebView | iOS XX WKWebView ]

Language: TypeScript 2.7.2

Node (for AoT issues): node --version =8.10

you have to use rxjs-compat until we add<EMAIL_ADDRESS>support

added in 4.0.0-alpha.0, here is a working example: https://stackblitz.com/edit/angular-2ulhgf

Nice I am going to give it a try today.


Add operators to futures

Signed-off-by: MrLotU<EMAIL_ADDRESS>

@tanner0101 @calebkleveter Tests are working! Can I have a final review? :)

@tanner0101 The reason I used // MARK: - <things>

🎉


Instantly teleport without delay or cooldown - problem

Plugin Version:
1.8.8
Describe The Bug:
Allowing players to instantly teleport without delay or cooldown?
After using command /rtp (with cooldown 100) - try to enter portal and your plugin write message: wait 90sec more... (this is ok)
After that, no longer portal working... Need to reconnect. (not ok)
How To Reproduce:
Turn RTP cooldown to 100
write /rtp
write /rtp
Try go thrrough the gate... bum
Expected Behavior:
Portal allowing players to instantly teleport without delay or cooldown?  - Not working.
Crash on cooldown and can't working, need to rejoin the world.
Screenshots/Error Log:
No errors in console.

This has been added, just pending release ;) Great recommendation

This has been fixed, pending release ;)

fixed in 3.6.12


[Subscript Refactor 1/x] Remove Literal Expression Initializers from Subscript Enum

Current dependencies on/for this PR:

main

PR #17   👈

This comment was auto-generated by Graphite.


align_embeddings()

What's the role of the function " align embeddings()  "?
Thank you
@HarryVolek

Answered in #40


CONFIG_LWIP_DHCP_RESTORE_LAST_IP fails to compile: dhcp_network_changed not defined (IDFGH-13710)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.
v5.4-dev-3023-g3c99557eee
Operating System used.
Windows
How did you build your project?
Command line with Make
If you are using Windows, please specify command line type.
PowerShell
What is the expected behavior?
Previously, CONFIG_LWIP_DHCP_RESTORE_LAST_IP compiled just fine.
What is the actual behavior?
As of tonight, CONFIG_LWIP_DHCP_RESTORE_LAST_IP fails to compile. This is because it invokes a macro defined as:
#ifdef CONFIG_LWIP_DHCP_RESTORE_LAST_IP
/*
 * Make the post-init hook check if we could restore the previously bound address
 * - if yes reset the state to bound and mark result as ERR_OK (which skips discovery state)
 * - if no, return false to continue normally to the discovery state
 */
#define LWIP_HOOK_DHCP_POST_INIT(netif, result) \
    (dhcp_ip_addr_restore(netif) ? ( dhcp_set_state(dhcp, DHCP_STATE_BOUND), \
                                     dhcp_network_changed(netif), \
                                     (result) = ERR_OK , \
        true ) : \
        false)
#else
#define LWIP_HOOK_DHCP_PRE_DISCOVERY(netif, result) (false)
#endif /* CONFIG_LWIP_DHCP_RESTORE_LAST_IP */

However, dhcp_network_changed() no longer exists.
Steps to reproduce.

Enable CONFIG_LWIP_DHCP_RESTORE_LAST_IP

Build or installation Logs.
In file included from C:/Users/Sean/esp/esp-idf/components/lwip/lwip/src/include/lwip/opt.h:51,
                 from C:/Users/Sean/esp/esp-idf/components/lwip/lwip/src/core/ipv4/dhcp.c:66:
C:/Users/Sean/esp/esp-idf/components/lwip/lwip/src/core/ipv4/dhcp.c: In function 'dhcp_start':
C:/Users/Sean/esp/esp-idf/components/lwip/port/include/lwipopts.h:365:38: error: implicit declaration of function 'dhcp_network_changed'; did you mean 'dhcp_network_changed_link_up'? [-Wimplicit-function-declaration]
  365 |                                      dhcp_network_changed(netif), \
      |                                      ^~~~~~~~~~~~~~~~~~~~
C:/Users/Sean/esp/esp-idf/components/lwip/lwip/src/core/ipv4/dhcp.c:945:7: note: in expansion of macro 'LWIP_HOOK_DHCP_POST_INIT'
  945 |   if (LWIP_HOOK_DHCP_POST_INIT(netif, result)) {
      |       ^~~~~~~~~~~~~~~~~~~~~~~~

### More Information.

This started failing this evening.

It looks like network_changed() was split into network_changed_link_down() and network_changed_link_up(), and in fact network_changed() was simply renamed to network_changed_link_up().
So this should be as simple as renaming the function.

As a workaround while this is getting fixed, you can add this to your CMakeLists.txt:
idf_build_set_property(COMPILE_DEFINITIONS "dhcp_network_changed=dhcp_network_changed_link_up" APPEND)

Still an issue here on 5.0.7 from what I can see


Suggestion: Use View Binding instead of synthetic binding and findViewById

(Our bot will not be happy with this issue)
Now that we're using Gradle 3.6.1, we could enable View Binding on our samples. Two reasons why I would recommend this change:

Kotlin synthetic binding is not recommended;
ViewBinding removes the need for findViewById, which would make our app implementation simpler, since it's meant for demonstration purposes.

@rosariopfernandes oh that's really cool!  I had no idea this was available.  I never liked Kotlin synthetic binding but I was trying to be one of the Kotlin Cool Kids 😎

Hahah, I see. Does that mean this is a go?

Do we need to use any alpha/beta tooling or is this all powered by AGP
3.6.1 stable?  If it's stable I'm good to go with this but I don't want our
devs to have to use AS Canary or anything like that.
On Thu, Mar 26, 2020, 11:19 AM Rosário Pereira Fernandes <
<EMAIL_ADDRESS>wrote:

Hahah, I see. Does that mean this is a go?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/firebase/quickstart-android/issues/1088#issuecomment-604490502,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACATB2R23CEKG437N77NFALRJNW6JANCNFSM4LUJVK7A
.

ViewBinding is included in 3.6.1 stable. No need to use any alpha/beta tools.

Ok then let's do it!
On Thu, Mar 26, 2020 at 11:30 AM Rosário Pereira Fernandes <
<EMAIL_ADDRESS>wrote:

ViewBinding is included in 3.6.1 stable. No need to use any alpha/beta
tools.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/firebase/quickstart-android/issues/1088#issuecomment-604496814,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACATB2S2OFL4IXU26FNW323RJNYIDANCNFSM4LUJVK7A
.

I have noticed that using ViewBinding, the camera in Kotlin MLKit sample app is finally working

@samtstern As I was implementing this, I noticed that some of our layouts only contain a single view (e.g. analytics/fragment_main.xml). I think for these cases it's simpler to use findViewById() instead of View Binding, since the number of lines will remain the same and we'll actually have less binding classes being generated by the plugin during build.
Unless we want to use View Binding in all views for consistency. WDYT?

@rosariopfernandes I don't have a strong opinion here, I think either choice you could make for a single-view layout is a reasonable one!  Whatever you think looks best as you implement is fine with me.


[Bug] Going offline/online after authenticating causes crash

[REQUIRED] Please fill in the following fields:

Pre-built SDK from the website or open-source from this repo: open source
Firebase C++ SDK version: 8.1.0
Problematic Firebase Component: database
Other Firebase Components in use: auth
Platform you are using the C++ SDK on: mac, linux
Platform you are targeting: desktop

[REQUIRED] Please describe the issue here:
An assert failure occurs if going offline/online too quickly after being authenticated. Persistent connections do not cancel their "fetch token" async work, and if two run at the same time, the app may crash.

ASSERT: [...]/firebase-cpp-sdk/database/src/desktop/connection/persistent_connection.cc(578): connection_state_ == kDisconnected

This is problematic for my actual app, as I want to be able to reconnect as soon as the network becomes reachable. If the network becomes unreachable/reachable in rapid succession (very common), it will crash because of this race, which is 100% reproducible.
I noticed that in the ObjC SDK, a guard was added to make sure the "fetch token" routine was still valid when handling its result by using a simple counter. I'd imagine this would have a similarly simple fix.
Steps to reproduce:
Have you been able to reproduce this issue with just the Firebase C++ quickstarts ? yes
What's the issue repro rate?  100%
Relevant Code:
Add the following to desktop/testapp/src/common_main.cc after authenticating (in quickstart-cpp):
database->GoOffline();
database->GoOnline();

If some time has elapsed after authenticating (e.g., the database is connected), you can still reproduce it if you call GoOffline()  / GoOnline() more than once.
database->GoOffline();
database->GoOnline();
database->GoOffline();
database->GoOnline();

Here's a forked version of quickstart-cpp with the modifications in-place: https://github.com/mikepulaski/quickstart-cpp/tree/auth-token-race

@mikehaney24 Thanks for reporting this.  I think this is a legit issue.
The assert occurred here.
https://github.com/firebase/firebase-cpp-sdk/blob/3c2190901976a218cd4dc91b0e6aae5afc8d9291/database/src/desktop/connection/persistent_connection.cc#L578
When GoOffline() is called, it would change the stated here.
https://github.com/firebase/firebase-cpp-sdk/blob/3c2190901976a218cd4dc91b0e6aae5afc8d9291/database/src/desktop/connection/persistent_connection.cc#L490
Though I cannot find "guard" in ObjC SDK. This seems to the same logic in C++.
https://github.com/firebase/firebase-ios-sdk/blob/4d518509ed2988e2be4edf61c770aa4c2495061d/FirebaseDatabase/Sources/Core/FPersistentConnection.m#L515
Or Android SDK
https://github.com/firebase/firebase-android-sdk/blob/cd6d1bcb8e3a6c070cb6cc76acdbdc65871cac24/firebase-database/src/main/java/com/google/firebase/database/connection/PersistentConnectionImpl.java#L771
Could you point me to the part of the code about what you meant by the "guard"?
Thank you!
Shawn

Hey Shawn,

[…]This seems to the same logic in C++.
https://github.com/firebase/firebase-ios-sdk/blob/4d518509ed2988e2be4edf61c770aa4c2495061d/FirebaseDatabase/Sources/Core/FPersistentConnection.m#L515 https://github.com/firebase/firebase-ios-sdk/blob/4d518509ed2988e2be4edf61c770aa4c2495061d/FirebaseDatabase/Sources/Core/FPersistentConnection.m#L515
It’s easy to miss, but the ObjC version has the following statement, which (I believe) protects against this sort of thing:

if (thisFetchTokenAttempt == self.currentFetchTokenAttempt) { /*…/* }
The ObjC block captures “thisFetchTokenAttempt” and compares it to one stored as an ivar. That was the “guard” I was talking about which I think would work pretty much the same with a lambda :)
Thanks for looking into this!

Could you point me to the part of the code about what you meant by the "guard"?
Thank you!
Shawn
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/firebase/firebase-cpp-sdk/issues/594#issuecomment-916596129, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAALQOXPXBW6Y66N3M22VKDUBFZHXANCNFSM5B3PIYYA.

Hi @mikepulaski,
I was able to reproduce this in our integration test as well, but calling GoOffline and Online a few times in the FirebaseDatabaseTest::SetUp method.
Code:
void FirebaseDatabaseTest::SetUp() {
  FirebaseTest::SetUp();
  InitializeDatabase();
  if(database_ != nullptr) {
    database_->GoOffline();
    database_->GoOnline();
    database_->GoOffline();
    database_->GoOnline();
    database_->GoOffline();
    database_->GoOnline();
  }
}

Output:
[ RUN      ] FirebaseDatabaseTest.TestSignIn
DEBUG: Initializing Firebase Database.
DEBUG: Attempting to initialize Firebase Database.
DEBUG: WaitForCompletion InitializeDatabase
DEBUG: Successfully initialized Firebase Database.
DEBUG: Shutdown the Database library.
ASSERT: firebase-cpp-sdk/database/src/desktop/connection/persistent_connection.cc(578): connection_state_ == kDisconnected

I'm going to mark this as a bug, thanks for the report!


web-ui: fix boolean as string ENV variables parsing

Small hotfix to correcly evaluate boolean like ENV variable passed as strings.
Eg. OAUTH_ENABLED="true" npm start
OAUTH_ENABLED will be parsed as a boolean:
const env = require('./env');

console.log(env.OAUTH_ENABLED === true); // true

Codecov Report

Merging #280 into master will increase coverage by 0.27%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #280      +/-   ##
==========================================
+ Coverage    90.6%   90.88%   +0.27%
==========================================
  Files          35       36       +1
  Lines         330      340      +10
  Branches       14       14
==========================================
+ Hits          299      309      +10
  Misses         28       28
  Partials        3        3

Flag
Coverage Δ

#webui
90.88% <100%> (+0.27%)
:arrow_up:

Impacted Files
Coverage Δ

web-ui/src/server/util.js
100% <100%> (ø)

web-ui/src/server/env.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e10b9aa...a1e387d. Read the comment docs.

👍


required=True support [typer.Option]

First Check

[X] I added a very descriptive title to this issue.
[X] I used the GitHub search to find a similar issue and didn't find it.
[X] I searched the Typer documentation, with the integrated search.
[X] I already searched in Google "How to X in Typer" and didn't find any information.
[X] I already read and followed all the tutorial in the docs and didn't find an answer.
[X] I already checked if it is not related to Typer but to Click.

Commit to Help

[X] I commit to help with one of those options 👆

Example Code
import typer
app = typer.Typer()

# currently required options look like this:
@app.command()
def main(
    username: str = typer.Option(..., help="A Username")
):

Description
Please make typer.Option required arguments conform to a more readable syntax than ellipsis.
From the function param:
username: str = typer.Option(..., help="A Username")
unless you have working knowledge of this library you wouldn't be able to know if:

the default value for username is ellipsis
would be confused why the default value doesnt form to the type str
would probably not know without inspecing typer.Option's signature that the first argument is a default value
would not know that an ellipsis as a default would invoke additional option logic

Please consider moving this feature to a kwarg in typer.Option.
Wanted Solution
typer.Option supports a new kwarg: required: bool
Wanted Code
import typer
app = typer.Typer()

# This is a more expressive and readable
@app.command()
def main(
    username: str = typer.Option("", help="A Username", required=True)
):

Alternatives
N/A
Operating System
Linux
Operating System Details
No response
Typer Version
0.4.0
Python Version
3.9+
Additional Context
No response

IMO, the current interface is good.   Adding the required parameter, would negate the first parameter (default value), which would be weird (the default value is ignored if required=True).
If anything, it should be made clearer in the instructions that to make an Option or Argument required, ... should be the default value.

check this out https://typer.tiangolo.com/tutorial/options/required/

I just came across the same problem. I wanted to add a required Option with Choices (Enum). And i didn't want to set a default value. Hope it makes some sense.
Great package btw ;P


No progress bar to know how long the process will take

We will use the tqdm lib for every process. This will allow the user to estimate better the progress of the long operational process.

Will not be done for now.
We need to minimize the terminal prompt to be clean for production used by MLOps SaaS solutions.


Stop Bot doesn't work when used with pm2

PM2 obviously restarts the process when it is killed. I'm not sure if there is a elegant way for that and I don't know if it is inteded to support PM2?

Don't you have a possibility of making PM2 restart the bot only on failure as it can be with systemd?

Thats is something I could check. But to be honest I don't use pm2 anymore because it doesn't work good with the update process either.

Then i close it. I find that using a process manager can mess also with update with ghost processes that prevent the bot from starting. I guess it should be made clear that this bot has to be launched outside from a process handler or without restart handling.


Allow PHP 7.0

Allow using the library on PHP 7.0, too

Coverage remained the same at 95.789% when pulling 115248aadf80b9fdec40cde31d90e55baaa06779 on liip-forks:allow-php-7-0 into d886a69c356140968ac3df4bc779f84f7d3b9f00 on spiral:master.

I was trying to avoid that, but I'm fine doing it since we can always revert nullable return types later.
Please, use @inheritdoc for send method as well as you did for receiveSync so the code is consistent.

Coverage remained the same at 95.789% when pulling 68ba1aed7da4813b286f2984c63cbfa592fb0312 on liip-forks:allow-php-7-0 into d886a69c356140968ac3df4bc779f84f7d3b9f00 on spiral:master.

@wolfy-j I updated the code and applied some doc-block cleanups :)

I've added 7.0 to travis as well.

Thanks! :)


[Feature Request] Support different randomness settings to train an ensemble of models with TorchOpt

Motivation
We recently used TorchOpt as a functional optimizer API  mentioned in functorch parallel training example to achieve batchable optimization training small neural networks on one GPU with functorch.vmap.
With TorchOpt, we can mimic the jax implementation to use vmap on the init function:
JAX:
def init_fn(input_shape, seed):
    rng = jr.PRNGKey(seed)                                     # jr = jax.random
    dummy_input = jnp.ones((1, *input_shape))
    params = classifier_fns.init(rng, dummy_input)['params']   # do shape inference
    optimizer_def = optim.Adam(learning_rate=1e-3)
    optimizer = optimizer_def.create(params)
    return optimizer
parallel_init_fn = jax.vmap(init_fn, in_axes=(None, 0))
model_states = parallel_init_fn((2,), seeds)

TorchOpt + functorch:
def init_fn(model_idx):
    _, weights = functorch.make_functional(MLPClassifier().to(DEVICE))
    opt_state = torchopt.adam(lr=0.2).init(weights)
    return weights, opt_state
parallel_init_fn = functorch.vmap(init_fn, randomness='same') # only 'same' works
batched_weights, opt_state = parallel_init_fn(torch.ones(num_models, 1))

instead of combine_state_for_ensemble
def init_fn(num_models):
    models = [MLPClassifier().to(DEVICE) for _ in range(num_models)]
    _, params, _ = combine_state_for_ensemble(models)
    return params
batched_weights = init_fn(num_models=2)

However, any other randomness setting in functorch.vmap(init_fn) threw a bug (i.e. if  randomness='different').
Traceback (most recent call last):
  File "parallel_train_torchopt.py", line 196, in <module>
    functorch_original.test_parallel_train_step_fn(num_models=2)
  File "parallel_train_torchopt.py", line 136, in test_parallel_train_step_fn
    weights, opt_state = parallel_init_fn(torch.ones(num_models, 1))
  File "/home/benjamin/miniconda3/envs/torchopt/lib/python3.8/site-packages/functorch/_src/vmap.py", line 365, in wrapped
    batched_outputs = func(*batched_inputs, **kwargs)
  File "parallel_train_torchopt.py", line 109, in init_fn
    _, weights = make_functional(MLPClassifier().to(DEVICE))
  File "parallel_train_torchopt.py", line 49, in __init__
    self.fc1 = nn.Linear(2, self.hidden_dim)
  File "/home/benjamin/miniconda3/envs/torchopt/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 101, in __init__
    self.reset_parameters()
  File "/home/benjamin/miniconda3/envs/torchopt/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 107, in reset_parameters
    init.kaiming_uniform_(self.weight, a=math.sqrt(5))
  File "/home/benjamin/miniconda3/envs/torchopt/lib/python3.8/site-packages/torch/nn/init.py", line 412, in kaiming_uniform_
    return tensor.uniform_(-bound, bound)
RuntimeError: vmap: Cannot ask for different inplace randomness on an unbatched tensor. This will appear like same randomness. If this is necessary for your usage, please file an issue with functorch.

functorch.vmap(init_fn, randomness='same') gives identical inits for each net in the ensemble, which is not desirable if we want to train ensembles averaging across random seeds, therefore functorch.vmap(init_fn) supporting different randomness settings is a needed feature in this kind of usage.
cc @waterhorse1 @JieRen98 @XuehaiPan
Solution
https://github.com/metaopt/TorchOpt/pull/32 can be runned with functorch.vmap(init_fn, randomness='different').
Resource

TorchOpt + functorch implementation
TorchOpt + functorch implementation colab
functorch parallel training example
JAX + FLAX parallel training example

Checklist

[x] I have checked that there is no similar issue in the repo. There is a one #909  to improve combine_state_for_ensemble for initialization of an ensemble of models and related issue #782 to ask for implemention for this usage but my request is more on giving an specific usage that requires this feature.

Thanks for the thorough writeup @Benjamin-eecs! To check a couple things, is this a current bottleneck for your examples? We had been under the assumption that the training would be much more expensive than the training but that may not be true (or it may be fair that we're losing out on performance by not doing this)
We're also currently looking at different ways that JAX libraries build neural nets and this is a great axis I hadn't thought of before. It looks like you might be using Flax or Haiku and I was wondering if you had tried this with Equinox at all?
cc @zou3519 This seems to be the same thing that the federated learning people were asking for. I forget if we got clear answer for them

Hi there @samdow , thanks for your quick and detailed feedback.

is this a current bottleneck for your examples?

I think I can call it bottleneck in some way, we can definitely initialize the ensemble of models and optimizers using for-loop. But our TorchOpt example mainly wants to show that we can support functorch.vmap for both initialization and training for ensemble of models. Also, in our specific usage where we want to repeat the same training process with different seeds or hyperparamters using functorch.vmap, we think it would be better if user can write it in a functional way to code the init_fn as a function of list of seeds or list of hyperparameters. But for now, we can only initialize a same set of models.

It looks like you might be using Flax or Haiku and I was wondering if you had tried this with Equinox at all?

I am not sure I fully understood, the Jax code snippet I showed in the writeup just to present that out TorchOpt example change the functorch example into Jax-style with extra optimizer such as adam other than sgd.

Also, in our specific usage where we want to repeat the same training process with different seeds or hyperparamters using functorch.vmap, we think it would be better if user can write it in a functional way to code the init_fn as a function of list of seeds or list of hyperparameters.

To be clear, if this is the end goal, it will probably always be easier to write this as a for loop. Most of the hyperparameters are scalar values and right now we can't vmap over lists or tensors of scalar values (1D tensors that we vmap over are going to be treated as scalar tensors instead of scalars).
As an example, if we had an ensemble of models like the ones in the TorchOpt PR but where the hidden dimension was being changed:
MLP(nn.Module)
    def __init__(self, hidden_dim=32, n_classes=2):
        ...
        self.fc1 = nn.Linear(2, self.hidden_dim)
        self.fc2 = nn.Linear(self.hidden_dim, self.n_classes)
  ...

we would never be able to vmap over different values for hidden_dim

the Jax code snippet I showed in the writeup just to present that our TorchOpt example

I see! Thanks for that clarification. I saw classifier_fns.init in the code snippet and assumed 😄

Any update on this issue? I want to train ensembles in parallel, and I am facing the same problem.


Implement history with reverse search

The plan is to implement history & reverse search like in a regular console.

History proposal:
General:

Controlled-component-like history API via a new prop

API:

A single prop history
history prop is of type boolean | HistoryConfig, where HistoryConfig is

interface HistoryConfig {
  history: Array<string>,
  onCommand: (string) => void,
}

Spec:

history prop contains ordered FIFO queue of previous commands
onCommand callback is triggered on every command submission, passes the new command as an argument


name of the course

Course name when "Turning a Project into A+" should be clear, can be without underscores, it's actually shown to users:

https://grader.cs.hut.fi/static/studio2_k2021dev/projects/s2_course_config.json
https://version.aalto.fi/gitlab/oseppala/studio2/-/blob/k2021dev/projects/create_s2_course_config.py#L37

https://version.aalto.fi/gitlab/oseppala/studio2/-/merge_requests/5/diffs it's in @oseppala 's hands now 😉


Table 1: pull data

This issue is complete when you have a redcap report with all variables for Table 1.

The following data has been pulled:

maternal age (using maternal birthdate and date of infant birth)
gestational age
breastfeeding
BMI/BMI groups to sort mothers by
https://github.com/lemaslab/BEACH-reports/blob/michele/code/get/get_RedCapData_36804.Rmd


[Migration] Need to change IDN_OAUTH2_TOKEN_BINDING table

Description:
With this https://github.com/wso2/product-is/issues/10481, we changed IDN_OAUTH2_TOKEN_BINDING table structure. This addresses the DB changes. https://github.com/wso2/carbon-identity-framework/pull/3303/files
We need to

Drop PRIMARY KEY (TOKEN_ID) column
Add a new unique key as UNIQUE (TOKEN_ID,TOKEN_BINDING_TYPE,TOKEN_BINDING_VALUE)

Migration is currently failing in this step in a Postgres env.


 When we start this function with state:
mKeepAnimating == false
mInTransition == true
force == false
mTransitionGoalPosition == 1.0
mTransitionLastPosition == 0.0

Most of the time we have at least one step between evaluate() call but sometimes there's a possibility when deltaPos > 1 but mTransitionLastTime == 0. In that situation mCurrentState == mEndState so we never set newState = true when at the end we have mNeedsFireTransitionCompleted |= newState :/
Some logs:
02-12 15:29:15.509 D: onMeasure(), mNeedsFireTransitionCompleted == false
02-12 15:29:15.533 D: evaluate() false || true && (false || 1.0 != 0.0<true>)
02-12 15:29:15.534 D: position = 1.0605366, mTransitionLastPosition = 0.0, deltaPos = 1.0605366, currentTime =<PHONE_NUMBER>35810, mTransitionLastTime = 0.0, mTransitionDuration = 0.6
02-12 15:29:15.534 D: dir = 1.0 (1.0 > 0.0 && 1.0605366 >= 1.0 <true> || 1.0 <= 0.0 && 1.0605366 <= 1.0 <false>), <position = this.mTransitionGoalPosition>
02-12 15:29:15.536 D: position = 1.0 [>= 1.0], mCurrentState =<PHONE_NUMBER>
02-12 15:29:15.537 D: mCurrentState =<PHONE_NUMBER>, mEndState =<PHONE_NUMBER>

I have the same issue...I can reproduce it in beta3 & beta4

I created an official issue. You can starred it there: https://issuetracker.google.com/issues/149423772


Messy static folder

There is no hierarchy in the static folder.
Organize it like:
static
|    img
|    js
|   css
|   fonts

Done


[chore] Upgrade kube version to v1.18.3

Change Overview

Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[x] :rainbow: Refactoring (no functional changes, no api changes)
[ ] :hamster: Trivial/Minor
[ ] :bug: Bugfix
[ ] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test

Issues

#XXX

Test Plan

[ ] :muscle: Manual
[ ] :zap: Unit test
[ ] :green_heart: E2E

Still have around 30+ files to go through. Looks good so far.


GitHub Action SF CLI Running Tests - Fatal Error

Hi,
I'm running a regular SF command which starts the deploy command with dry-run. The very same command runs in my local machine (tried in both Mac & Windows) successfully but fails in GitHub CI/CD action, which is a Linux machine.
SF CLI Command:
sf project deploy start --manifest manifest/package.xml --dry-run --wait 60 --test-level RunSpecifiedTests --tests Test1 --tests Test2 --tests Test3 --json --ignore-warnings
Steps To Reproduce

Download Salesforce CLI
Execute sf project deploy start command

Expected result
Successfully running all the tests and reporting the results.
Actual result
Returns status 1 without any explanation except FATAL ERROR on UI. There are 20 test classes specified in that command but all the test classes have 0 test coverage.
Failed Run (from GitHub action)

Success Run (from local machine - Mac)

System Information
{
  "architecture": "linux-x64",
  "cliVersion"<EMAIL_ADDRESS>  "nodeVersion": "node-v20.10.0",
  "osVersion": "Linux 6.2.0-1018-azure",
  "rootPath": "/home/runner/work/ePaaS-1.0/ePaaS-1.0/sf",
  "shell": "bash",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.0.4 (core)",
    "@oclif/plugin-commands 3.1.0 (core)",
    "@oclif/plugin-help 6.0.9 (core)",
    "@oclif/plugin-not-found 3.0.7 (core)",
    "@oclif/plugin-plugins 4.1.14 (core)",
    "@oclif/plugin-search 1.0.10 (core)",
    "@oclif/plugin-update 4.1.7 (core)",
    "@oclif/plugin-version 2.0.11 (core)",
    "@oclif/plugin-warn-if-update-available 3.0.8 (core)",
    "@oclif/plugin-which 3.0.14 (core)",
    "@salesforce/cli 2.23.20 (core)",
    "apex 3.0.14 (core)",
    "auth 3.1.0 (core)",
    "data 3.0.13 (core)",
    "deploy-retrieve 2.2.12 (core)",
    "info 3.0.13 (core)",
    "limits 3.1.0 (core)",
    "marketplace 1.0.15 (core)",
    "org 3.2.0 (core)",
    "packaging 2.0.1 (core)",
    "schema 3.1.0 (core)",
    "settings 2.0.14 (core)",
    "sobject 1.0.7 (core)",
    "source 3.0.10 (core)",
    "telemetry 3.1.8 (core)",
    "templates 56.0.8 (core)",
    "trust 3.2.5 (core)",
    "user 3.2.0 (core)"
  ]
}

Does anyone have any ideas? Thanks in advance.

You're not providing enough information (see above).  Have you checked the deploy status in the org UI?  To me, it looks like the CLI is behaving as expected and reporting an error that happened during deployment.

Hi @shetzel,
I've updated the ticket. This is the deploy status in the org UI.
Again, when I execute the very same command on my locals (Mac & Win) it works as expected. All the tests run and pass and we receive success, 0 as status code.

You can try getting debug output during the CI run by appending the --dev-debug flag and inspecting the output.  Another way to get that output is by running the same command through the doctor command so it generates the files for you.  E.g., sf doctor --command "sf project deploy start --manifest manifest/package.xml --dry-run --wait 60 --test-level RunSpecifiedTests --tests Test1 --tests Test2 --tests Test3 --json --ignore-warnings"
Another option is to set the SF_MDAPI_TEMP_DIR env var in your CI script to some known location and check the contents of the directory and manifest to ensure it looks correct.
Is it the same org being deployed to?  If everything else is the same as with the successful run then I would debug the CI script.

Thanks for the advice. To be honest, it's hard to extract the output of the doctor command since there are files saved into the temp folder of runner. I'll continue later today and share my update but is there an easier way to get the result from the doctor command?
By the way, yes, that is the same org with the same user and same package file.

The doctor command accepts an --output-dir flag to specify a location.  Alternatively, you can set these env vars in the CI script: DEBUG=* and SF_LOG_LEVEL=trace and the debug output will be part of the runner logs.
This doesn't seem like a CLI bug.  We have lots of CI tests running via GitHub actions using Ubuntu and Windows that run similar deploy and test commands.  It's much more likely that there is something wrong with the CI script.

closed because stale/no response

Hey,
Sorry for the late response. I wasn't able to find the root cause of the issue but I just found a workaround by changing my CICD script. I was executing the command as command + params (flags) but now I execute the whole thing as command.
I changed this
  let args = [...deploymentAttributes, "--json", "--ignore-warnings"];

  await exec.exec("sf project deploy start", args, {
    listeners: { stderr: errorHandler, stdout: successHandler },
    ignoreReturnCode: true
  });

To this
  let cmd = "sf project deploy start " + deploymentAttributes.join(" ") + " --json --ignore-warnings";

  await exec.exec(cmd, [], {
    listeners: { stderr: errorHandler, stdout: successHandler },
    ignoreReturnCode: true
  });

Thank you both for your time and support.


Pre-filter Data - System Fuel Flow Data

As an ECMPS user, I want the dropdown options for System Fuel Flow data fields to be limited so I can only make valid selections
Acceptance Criteria:
Given I am an ECMPS user making updates (Create/Edit) to the System Fuel Flow data
When I open a 'Create/Edit' modal
Then the options for the 'Units of Measure Code' should be limited to SCFH, GALHR, BBLHR, M3HR, LBHR, HSCF
And options for 'Max Fuel Rate Source Code' should be URV, UMX
Notes:

The 'Units of Measure Code' options should be limited to: SCFH, GALHR, BBLHR, M3HR, LBHR, HSCF
Tech team created the view camdecmpsmd.vw_systemfuel_master_data_relationships
Reference the master list in https://app.zenhub.com/workspaces/emissioners-scrum-board-5f36cd263511ad001a777f8e/issues/us-epa-camd/easey-ui/2357

https://app.zenhub.com/files/287570343/30225c4e-97d9-4255-b4d1-fdf4b868224a/download

Test Case: TC589

Test Passed


Undefined values of variables destructed in generator declaration

TypeScript Version:  3.4.1  (works in 3.2.4)

Search Terms:
destruct  generator
Code
export function* saveLike({ entity, id, xhr }: LikeToggleAction) {

    if (!(yield call(checkIfUserIsLoggedIn, ACTION_TO_ADD_REPLY))) {
        return;
    }

    // at this point entity, id, xhr are undefined

    // ... other code
}

Expected behavior:
Values of variables destructed from function declaration should not be undefined, i.e. should have values that were passed to function call
Actual behavior:
After first resume of generator, destructed values are lost (i.e. undefined).
Problem is in transpiled code. Destructed code is inside case 0, i.e. subsequent calls to generator (case 1, case 2, ...) do not see the values.
function saveLike(_a) {
    var _b, key, current, newState, fakeResponse, error_1;
    return tslib__WEBPACK_IMPORTED_MODULE_0__["__generator"](this, function (_c) {
        switch (_c.label) {
            case 0:
                 var entity = _a.entity, id = _a.id, xhr = _a.xhr;
                 return [4 /*yield*/,  ...];
            case 1:
               // at this point entity, id, xhr are undefined
               // ...

This issue forces redux-saga users to stay with v3.3.

same problem

@RyanCavanaugh Just hit us too. Repro:
function* doStuff(obj, { becomesUndefined }) {
    const styles = { [obj.id]: obj };

    yield 1;
    alert(becomesUndefined);
}

const q = doStuff("", { becomesUndefined: 5 });
q.next();
q.next();

https://www.typescriptlang.org/play/#src=function* doStuff(obj%2C { becomesUndefined }) {
    const styles %3D { [obj.id]%3A obj }%3B

    yield 1%3B
    alert(becomesUndefined)%3B
}

const q %3D doStuff(""%2C { becomesUndefined%3A 5 })%3B
q.next()%3B
q.next()%3B

This was fixed by #30726.


Props beggining with '_' character are not displayed in inspector

Reproduction:
Create a component property as
_options: [Array, Object]

then open props list from that component inside of component inspector.
Result
The property is not listed
Expected result:
Property should be listed.

This has been fixed along with the proper placement of props and vars/return data. Thanks for pointing these and other bugs out! Really appreciate it :)


Fixed issues with tests, code cleanup and CI integration

Hi there,
This is how I envision the tests being executed in a fashion that will allow for controlled start and stop of the server. As I wrote before the deal is to put a call to Spark.disableAutostart() before the first call to Spark.get() or any other DSL methods and after one is done with the setup to initiate the server manually. This way we can have the examples as they are and depending on which environment they are started from the actual outcome is as it should be either controlled (tests) or automated (from main method - just like Sinatra).
The rest is just some code cleanup done as per Sonar's pointers.

Dropping this one.


Enable Windows Build

Enable Windows build by adding windows-latest into build matrix ( .github/workflows/executables.yml). Resolve missing cl.exe command issue.

Fix by using a new graalvm action. See https://github.com/filip26/ld-cli/actions/runs/2843316769


WIP: Proper GDNative export on Android

Discovered that there is problem with storing libs under full paths, looking into it.

@reduz @karroffel So it seems Android expects libraries to be flattened (all located in the same dir) and their names to start with lib. How do you suggest to resolve that? I first thought doing something like path.replace_first("res://", "lib").replace("/", "_"), but the problem is that for dependencies we might need to keep the original name, because linker only stores the name. So we can just put the library with its original name into the root lib folder, but what do we do with name clashes? The same question probably arises on other platforms, too.

For now just made export use only the library file name exported and check that the name starts with lib. This works for our game and I think it can be merged and we can make further adjustments afterwards if needed.


Update README.md

Changes:

Edit banners
Change Title

@Rayiumir thank you very much for the contribution!


Add analytics and refactor system design

Coverage decreased (-2.2%) to 79.866% when pulling ba997bca64ad698b3bfa7389d46285779dc0a80d on analytics into d653793012a6a175f95149e15396d47d2422531f on next.


[$1000] Dev:  iOS - App crashes when clicking 'fix all errors'

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:
. Go to settings.
2. Go to your workspace.
3. Navigate to 'Travel' and connect bank account manually.
4. Add Routing number (074000078) and Account number (1234), then try to continue without accepting the terms.
5. Click on the "Fix errors" link.
Expected Result:
App does not crash
Actual Result:
App crashes
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[x] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/ac169a3d-0cad-4d09-9cc4-092b708d113d
Expensify/Expensify Issue URL:
Issue reported by: Sibtain Ali
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1687522491184139
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01b709d4daea7a4b8f
Upwork Job ID:<PHONE_NUMBER>422703616
Last Price Increase: 2023-07-06

Proposal
Please re-state the problem that we are trying to solve in this issue.
When the first invalid form input is a checkbox, an exception is thrown in the app.
What is the root cause of that problem?
On iOS, Pressable (and View) have a focus method, but calling it throws an exception. Checkbox forwards its reference to Pressable.
Form calls focus on the first invalid form input when the "Fix the errors" button is pressed.
What changes do you think we should make in order to solve the problem?
On Native, our Checkbox class does not support any kind of focus, so there's no need to expose a focus method.
Instead, it's enough to expose measureLayout, which is called by Form when determining where to scroll to.
As the Checkbox component is a Functional Component, exposing methods is done with the useImperativeHandle API.
function Checkbox(props) {
    const pressableRef = useRef();

    useImperativeHandle(
        props.forwardedRef,
        () => ({
            /**
             * Like measure(), but measures the view relative to an ancestor
             *
             * This method is used by Form when scrolling to the input
             *
             * @param {Object} relativeToNativeComponentRef - reference to an ancestor
             * @param {function(x: number, y: number, width: number, height: number): void} onSuccess - callback called on success
             * @param {function(): void} onFail - callback called on failure
             */
            measureLayout(relativeToNativeComponentRef, onSuccess, onFail) {
                if (!pressableRef.current) {
                    return;
                }

                pressableRef.current.measureLayout(relativeToNativeComponentRef, onSuccess, onFail);
            }
        }),
    );

    // ...

    return (
        <Pressable
            ref={pressableRef}
            // ...
        >
            // ...
        </Pressable>
    );
}

@allroundexperts could you please test this on the latest app version (<IP_ADDRESS>)? I was not able to reproduce using the steps you shared:
https://github.com/Expensify/App/assets/59587260/f2e7f398-b874-400d-b336-118c72855b98

@kadiealexander While the exception is thrown on all builds, the full-screen drawer with the exception description (the "crash" experience) is visible only on the developer (non-release) build.

Ah sorry for missing that! I was able to repro this in my dev environment:

@cubuspl42, thanks for the proposal!
The approach looks good to me, but the scrolling isn't working. You can test this by adding marginBottom: 800 to these styles.
When checkbox is the first element with the error, screen isn't scrolled to checkbox when pressing "fix all errors"
https://github.com/Expensify/App/assets/9059945/ecb55be8-5059-450c-bf09-59e3cf43a4a9

@eVoloshchak It seems to work for me... This is my branch. Could you try to test it there?
https://github.com/Expensify/App/assets/2590174/7f8005f8-5491-4356-9591-3282115cb053

@eVoloshchak could you please take a look here?

t seems to work for me... This is my branch. Could you try to test it there?

@cubuspl42, huh, weird, doesn't work for me when using your branch too
Which device/OS version you're using? Is it a physical device or an emulator?

@eVoloshchak Emulator, iPhone 14, iOS 16.2. How about you?
I just check on exactly the same page you were testing (as I chose another one with a checkbox before) and it didn't change a thing... Still works.

Huh, weird, this didn't work for me in simulator, but works fine on a real device
@cubuspl42's proposal looks good to me
🎀👀🎀 C+ reviewed!
cc: @kadiealexander

@amyevans could you please review this, and also investigate why @eVoloshchak's message didn't auto-assign an engineer?

Just to note, this isn't a crash, it's only a warning. I see we previously chose not to fix it in https://github.com/Expensify/App/issues/12508 and I think that's still a valid and perhaps better choice given this is dev-only on a single platform.
The proposal sounds like a workaround to me, so I don't love it. Wouldn't it be better to not call .focus() in the first place?

also investigate why @eVoloshchak's message didn't auto-assign an engineer?

Logs are here, it's not clear why it didn't work though, we don't log the payload so it's hard to know. cc @flodnv if you have any thoughts/guesses

@amyevans

Just to note, this isn't a crash, it's only a warning.

This is an unhandled exception, both on dev and on production. Unhandled exceptions are bugs, at least I haven't been involved in a project with a different perspective on this matter.

I think that's still a valid and perhaps better choice given this is dev-only on a single platform.

I cannot see how ignoring an unhandled exception is better than writing the code in a way that doesn't trigger the erroneous path.

The proposal sounds like a workaround to me, so I don't love it.

Could you elaborate on why you think that the proposal is a workaround? I cannot see a reason for Checkbox to expose methods it actually doesn't support.

Wouldn't it be better to not call .focus() in the first place?

Well, then the functionality which depends on it wouldn't work.
We're calling the focus method on focusInput if it exists, following the reasoning that if it exists, it implements this contract (which is, probably, intuitive). I find this assumption reasonable.
https://github.com/Expensify/App/blob/032006b241f14711d96630151624d2535824bfa1/src/components/Form.js#L361-L363
This is a polymorphic call. formInput can be one of many Form-compatible input components. I cannot see anything wrong with this logic, I don't think there's a valid reason for changing it.
Checkbox doesn't support focusing, so from my perspective exposing a focus method is misleading from the beginning. The fact that it throws an exception is just the final argument to fix this.

@amyevans I missed the previous GH where we chose not to fix this, thanks for linking that! If the logic here that this is iOS related (not due to our code) is still true then I think it's best that we continue to do nothing, and close this.

@amyevans I missed the previous GH where we chose not to fix this, thanks for linking that! If the logic here that this is iOS related (not due to our code) is still true then I think it's best that we continue to do nothing, and close this.

As I understand it, that's not the logic. The logic is that...

"We calling focus on Pressable which we shouldn't"
"there is no direct way to prevent that from Form"

And I agree with both points.

@amyevans

The proposal sounds like a workaround to me, so I don't love it.

What do you think about an alternative way of controlling method forwarding more precisely?
We could implement a small utility...
function useForwardingHandle(ref, methodTargets) {
    const obj = Object.entries(methodTargets).reduce(
        (acc, [methodName, targetRef]) => ({
            ...acc,
            [methodName]: (...args) => {
                if (!targetRef.current) {
                    return;
                }

                return targetRef.current[methodName](...args);
            }
        }),
    );

    useImperativeHandle(ref, () => obj);
}

...used like...
function Checkbox(props) {
    const pressableRef = useRef();

    useForwardingHandle(
        props.forwardedRef,
        {
            measureLayout: pressableRef,
        },
    );

    // ...
}

Maybe this would express the intention more clearly? I think such utility could become handy at times in other places, too.

Speedy fix! Thanks Flo :)

Okay, given this does not affect a customer's experience of the product (no change in UI, they are able to advance through the form as expected, etc), let's stay consistent with the outcome of the previous GH and close this as well.
Definitely appreciate your engagement though @cubuspl42! We have hundreds of other bugs open with higher impact on customers that are seeking help, so hope to see you around in the future


Cr2met v2 - Chilean precipitation and temperature data

cr2met v2 is a daily gridded dataset of precipitation and temperature covering Chile from 1979-2020.

Hi all,
This recipe is currently failing. Each input is a download link for a zipped file. Each zipped file contains a single netcdf file. ex: FILE
In pangeo-forge/staged-recipes#90, it's mentioned that fsspec can handle zipped files.
Is there a way I could help out in adding zipped file input functionality in pangeo-forge-recipes?
Thanks!
-Raphael

AFAICT, we don't need any special functionality. It should already work. You need to have your file_pattern function return URLs that look like this - zip://datapub/model_data/pism1.0_paleo06_6255/snapshots_-10000.000.nc::https://hs.pangaea.de/model/PISM/Albrecht-etal_2019/parameter-ensemble/Part2_pism_paleo_ensemble_v2.zip, or more generally - zip://path/within/zipfile/to/datafile.nc::https://url/to/zipfile/on/the/internet.
Fsspec should be able to open them directly.
You could try this now with this recipe.

😆 Ryan beat me to it, but just because I'd already typed this up, I believe your format function would just need to look something like this:
def make_filename(time, variable):
    http_base = "https://www.cr2.cl/download/cr2met_v2-0"
    if variable == "pr":
        fname = f"zip://<NETCDF_FILENAME>.nc::{http_base}_pr_day_1979_2020/?wpdmdl=28866&ind=XEBr5cgJaEiPZ1OFhMQLzsGEQ2nIIGyaUXHZlZqrWmCl8TFr4qSxI2eBWBXizHHeDZtpy7gsohOq0wPs20kBmsAZNDWjlaaT4SVwXpop6zGrOAOfHBGIo2U59eNpOjT7AuxDSkAuTBTvDIrDXFvDzg"  # noqa: E501
    elif variable == "t2m":
        fname = f"zip://<NETCDF_FILENAME>.nc::{http_base}_t2m_day_1979_2020/?wpdmdl=28864&ind=l5tlDuy3dq_CWqbJ-K3jvoxzm77YdYE7Nph_YyQ0A6j_scgZ-kaugoW2ox85O5hyrpnL0_OOwxFWr7LoODrNgB_F4Jg-7qaXpu_lWox8b9H6w6d7DrY_YJyRRzuU7SVMyCLKCJk-cxCZkcSalzRSPw"  # noqa: E501
    elif variable == "tmax":
        fname = f"zip://<NETCDF_FILENAME>.nc::{http_base}_tmax_day_1979_2020/?wpdmdl=28862&ind=FH-qiDSW-IDlWVbL97QIBIH9pJNC2zf1377t4DNb7arzaTLv8shTVbXZQf9RUmZtYRSpuadjVcWU9MppWLSDWDOvqoESBnsOzcB31o-14ETxSUppjjXDeqfawstEbah8fcIsg7Sj22RSpFvsVbwOzQ"  # noqa: E501
    elif variable == "tmin":
        fname = f"zip://<NETCDF_FILENAME>.nc::{http_base}_tmin_day-_1979_2020/?wpdmdl=28859&ind=TY0Apx4oPcU_XU_P4Tez5FMHTXgdcgQbyukVXiBT-0Sm9JsVwkTR7bS72tdh96ffXB2viQq8-sYBORa3OucO7dtGbckdXr5Dh-2O6ISVCW4NsKOBwSRv3h-wGW0aaSwJKpPTY6UXP9VdNk-y2_V8GQ"  # noqa: E501
    return fname

Ah thanks so much for the quick feedback! I'll update the recipe and test it out.

I'm glad the user-agent stuff has come up.
I think we should actually be setting user-agent to something Pangeo Forge specific to help data providers know when we are crawling their data. (Best practices.)
If a provider is blocking certain user agents, we should probably not work around that by spoofing the user agent field. Instead, we should work with the data provider to understand their policy and (perhaps) get an exception granted for our user agent.

@cam-gerlach, see above for prior discussion of zip file path formatting.


Caddy plugin does not build

I'm getting build errors with the current (d3992886c4dc) version of the caddy plugin. I trying to build with xcaddy:
$ xcaddy build --with github.com/darkweak/souin/plugins/caddy

Errors:
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:384:11: ck.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:386:11: ck.Hash undefined (type configurationtypes.Key has no field or method Hash)
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:470:18: config_key.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:472:18: config_key.Hash undefined (type configurationtypes.Key has no field or method Hash)
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\httpcache.go:192:108: dc.Key.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\httpcache.go:192:133: dc.Key.Hash undefined (type configurationtypes.Key has no field or method Hash)

The previous commit (ddc269daafa99c551913344137e9590b721e7dd5) does build successfully.
$ xcaddy build --with github.com/darkweak/souin/plugins/caddy@ddc269daafa99c551913344137e9590b721e7dd5

complete build log from xcaddy
$ xcaddy build --with github.com/darkweak/souin/plugins/caddy
2024/05/01 11:39:02 [INFO] absolute output file path: R:\tmp\caddy.exe
2024/05/01 11:39:02 [INFO] Temporary folder: r:\temp\buildenv_2024-05-01-1139.3123807730
2024/05/01 11:39:02 [INFO] Writing main module: r:\temp\buildenv_2024-05-01-1139.3123807730\main.go
package main

import (
        caddycmd "github.com/caddyserver/caddy/v2/cmd"

        // plug in Caddy modules here
        _ "github.com/caddyserver/caddy/v2/modules/standard"
        _ "github.com/darkweak/souin/plugins/caddy"
)

func main() {
        caddycmd.Main()
}
2024/05/01 11:39:02 [INFO] Initializing Go module
2024/05/01 11:39:02 [INFO] exec (timeout=0s): C:\go\bin\go.exe mod init caddy
go: creating new go.mod: module caddy
go: to add module requirements and sums:
        go mod tidy
2024/05/01 11:39:02 [INFO] Pinning versions
2024/05/01 11:39:02 [INFO] exec (timeout=0s): C:\go\bin\go.exe get -d -v -tags nobadger github.com/caddyserver/caddy/v2
go: added github.com/beorn7/perks v1.0.1
go: added github.com/caddyserver/caddy/v2 v2.7.6
go: added github.com/caddyserver/certmagic v0.20.0
go: added github.com/cespare/xxhash/v2 v2.2.0
go: added github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572
go: added github.com/golang/protobuf v1.5.3
go: added github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1
go: added github.com/google/uuid v1.3.1
go: added github.com/klauspost/cpuid/v2 v2.2.5
go: added github.com/libdns/libdns v0.2.1
go: added github.com/matttproud/golang_protobuf_extensions v1.0.4
go: added github.com/mholt/acmez v1.2.0
go: added github.com/miekg/dns v1.1.55
go: added github.com/onsi/ginkgo/v2 v2.9.5
go: added github.com/prometheus/client_golang v1.15.1
go: added github.com/prometheus/client_model v0.4.0
go: added github.com/prometheus/common v0.42.0
go: added github.com/prometheus/procfs v0.9.0
go: added github.com/quic-go/qpack v0.4.0
go: added github.com/quic-go/qtls-go1-20 v0.4.1
go: added github.com/quic-go/quic-go v0.40.0
go: added github.com/zeebo/blake3 v0.2.3
go: added go.uber.org/mock v0.3.0
go: added go.uber.org/multierr v1.11.0
go: added go.uber.org/zap v1.25.0
go: added golang.org/x/crypto v0.14.0
go: added golang.org/x/exp v0.0.0-20230310171629-522b1b587ee0
go: added golang.org/x/mod v0.11.0
go: added golang.org/x/net v0.17.0
go: added golang.org/x/sys v0.14.0
go: added golang.org/x/term v0.13.0
go: added golang.org/x/text v0.13.0
go: added golang.org/x/tools v0.10.0
go: added google.golang.org/protobuf v1.31.0
2024/05/01 11:39:03 [INFO] exec (timeout=0s): C:\go\bin\go.exe get -d -v -tags nobadger github.com/darkweak/souin/plugins/caddy github.com/caddyserver/caddy/v2
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.0.0-20230207041349-798e818bf904
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v2.12.1
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v1.16.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.10.1
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v1.26.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.16.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.0.0-20231127185646-65229373498e
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.14.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.19.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.17.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.15.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.14.0
go: accepting indirect upgrade from<EMAIL_ADDRESS>to v0.16.0
go: added github.com/RoaringBitmap/roaring v1.2.3
go: added github.com/antlabs/stl v0.0.1
go: added github.com/antlabs/timer v0.0.11
go: added github.com/armon/go-metrics v0.4.1
go: added github.com/bits-and-blooms/bitset v1.5.0
go: added github.com/buraksezer/consistent v0.10.0
go: added github.com/buraksezer/olric v0.5.4
go: added github.com/bwmarrin/snowflake v0.3.0
go: added github.com/coreos/go-semver v0.3.1
go: added github.com/coreos/go-systemd/v22 v22.5.0
go: added github.com/darkweak/go-esi v0.0.5
go: added github.com/darkweak/souin v1.6.47
go: added github.com/darkweak/souin/plugins/caddy v0.0.0-20240423070913-d3992886c4dc
go: added github.com/dgraph-io/badger/v3 v3.2103.5
go: upgraded github.com/dgraph-io/ristretto v0.1.0 => v0.1.1
go: added github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
go: added github.com/go-redis/redis/v8 v8.11.5
go: added github.com/gofrs/flock v0.8.1
go: added github.com/gogo/protobuf v1.3.2
go: added github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
go: added github.com/google/btree v1.1.2
go: added github.com/google/flatbuffers v23.1.21+incompatible
go: upgraded github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 => v0.0.0-20230207041349-798e818bf904
go: added github.com/hashicorp/errwrap v1.1.0
go: added github.com/hashicorp/go-immutable-radix v1.3.1
go: added github.com/hashicorp/go-msgpack v0.5.5
go: added github.com/hashicorp/go-multierror v1.1.1
go: added github.com/hashicorp/go-sockaddr v1.0.2
go: added github.com/hashicorp/golang-lru v0.5.5-0.20200511160909-eb529947af53
go: added github.com/hashicorp/logutils v1.0.0
go: added github.com/hashicorp/memberlist v0.5.0
go: upgraded github.com/imdario/mergo v0.3.12 => v0.3.13
go: added github.com/mschoch/smat v0.2.0
go: added github.com/nutsdb/nutsdb v0.14.3
go: upgraded github.com/onsi/ginkgo/v2 v2.9.5 => v2.12.1
go: added github.com/pquerna/cachecontrol v0.2.0
go: upgraded github.com/prometheus/client_golang v1.15.1 => v1.16.0
go: upgraded github.com/prometheus/procfs v0.9.0 => v0.10.1
go: added github.com/redis/rueidis v1.0.31
go: added github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529
go: added github.com/tidwall/btree v1.6.0
go: added github.com/tidwall/match v1.1.1
go: added github.com/tidwall/redcon v1.6.2
go: added github.com/vmihailenco/msgpack/v5 v5.3.5
go: added github.com/vmihailenco/tagparser/v2 v2.0.0
go: added github.com/xujiajun/mmap-go v1.0.1
go: added github.com/xujiajun/utils v0.0.0-20220904132955-5f7c5b914235
go: added go.etcd.io/etcd/api/v3 v3.5.12
go: added go.etcd.io/etcd/client/pkg/v3 v3.5.12
go: added go.etcd.io/etcd/client/v3 v3.5.12
go: added go.opencensus.io v0.24.0
go: upgraded go.uber.org/zap v1.25.0 => v1.26.0
go: upgraded golang.org/x/crypto v0.14.0 => v0.16.0
go: upgraded golang.org/x/exp v0.0.0-20230310171629-522b1b587ee0 => v0.0.0-20231127185646-65229373498e
go: upgraded golang.org/x/mod v0.11.0 => v0.14.0
go: upgraded golang.org/x/net v0.17.0 => v0.19.0
go: upgraded golang.org/x/sync v0.4.0 => v0.5.0
go: upgraded golang.org/x/sys v0.14.0 => v0.17.0
go: upgraded golang.org/x/term v0.13.0 => v0.15.0
go: upgraded golang.org/x/text v0.13.0 => v0.14.0
go: upgraded golang.org/x/tools v0.10.0 => v0.16.0
go: added google.golang.org/genproto v0.0.0-20231012201019-e917dd12ba7a
2024/05/01 11:39:05 [INFO] exec (timeout=0s): C:\go\bin\go.exe get -d -v -tags nobadger
go: downloading github.com/alecthomas/chroma/v2 v2.9.1
go: downloading github.com/yuin/goldmark v1.5.6
2024/05/01 11:39:07 [INFO] Build environment ready
2024/05/01 11:39:07 [INFO] Building Caddy
2024/05/01 11:39:08 [INFO] exec (timeout=0s): C:\go\bin\go.exe mod tidy -e
go: downloading gopkg.in/go-playground/assert.v1 v1.2.1
go: downloading github.com/hashicorp/go-uuid v1.0.1
go: downloading github.com/go-stack/stack v1.8.0
2024/05/01 11:39:09 [INFO] exec (timeout=0s): C:\go\bin\go.exe build -o R:\tmp\caddy.exe -tags nobadger
# github.com/darkweak/souin/plugins/caddy
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:384:11: ck.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:386:11: ck.Hash undefined (type configurationtypes.Key has no field or method Hash)
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:470:18: config_key.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\configuration.go:472:18: config_key.Hash undefined (type configurationtypes.Key has no field or method Hash)
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\httpcache.go:192:108: dc.Key.DisableScheme undefined (type configurationtypes.Key has no field or method DisableScheme)
C:\gomodcache\github.com\darkweak\souin\plugins\caddy@v0.0.0-20240423070913-d3992886c4dc\httpcache.go:192:133: dc.Key.Hash undefined (type configurationtypes.Key has no field or method Hash)
2024/05/01 11:39:19 [INFO] Cleaning up temporary folder: r:\temp\buildenv_2024-05-01-1139.3123807730
2024/05/01 11:39:19 [FATAL] exit status 1

I have the same problem.  I reported it to @darkweak

Hello, sorry for that. You can build it using xcaddy build --with github.com/darkweak/souin/plugins/caddy@ d3992886c4dc227746a071e637e2b01b09b0e2a9 --with github.com/darkweak/souin@ d3992886c4dc227746a071e637e2b01b09b0e2a9.

Still does not build, see the build job: https://github.com/darkweak/souin/actions/runs/8796276967/job/24138843072

The build job runs the following command:
xcaddy build --with github.com/darkweak/souin/plugins/caddy@$(git rev-parse --short "$GITHUB_SHA") so it doesn't refer to the latest commit of the Souin repository and refers to the core v1.6.47.
With the command below you are able to build the caddy module
xcaddy build --with github.com/darkweak/souin/plugins/caddy@d3992886c4dc227746a071e637e2b01b09b0e2a9 --with github.com/darkweak/souin@d3992886c4dc227746a071e637e2b01b09b0e2a9

It should work now @ueffel

It is not building again.
15:05:04 [INFO] exec (timeout=0s): /usr/local/go/bin/go build -o /usr/bin/caddy -ldflags -w -s -trimpath -tags nobadger
183.8 # github.com/darkweak/souin/plugins/caddy
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:367:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:464:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:515:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:530:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:548:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:561:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/configuration.go:580:50: unknown field Found in struct literal of type configurationtypes.CacheProvider
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/dispatch.go:11:41: s.Configuration.DefaultCache.Badger.Found undefined (type configurationtypes.CacheProvider has no field or method Found)
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/dispatch.go:31:40: s.Configuration.DefaultCache.Badger.Uuid undefined (type configurationtypes.CacheProvider has no field or method Uuid)
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/dispatch.go:39:39: s.Configuration.DefaultCache.Etcd.Found undefined (type configurationtypes.CacheProvider has no field or method Found)
183.8 /go/pkg/mod/github.com/darkweak/souin/plugins/caddy@v0.0.0-20240717102502-b0a36db1b550/dispatch.go:39:39: too many errors

the Dockerfile used:
FROM caddy:2-builder-alpine AS builder

RUN xcaddy build --with github.com/darkweak/souin/plugins/caddy --with github.com/darkweak/storages/go-redis/caddy

FROM caddy:2-alpine

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

xcaddy build --with github.com/darkweak/souin/plugins/caddy@b0a36db1b550b66f768a3793701b69358200fed3 --with github.com/darkweak/souin@b0a36db1b550b66f768a3793701b69358200fed3  --with github.com/darkweak/storages/go-redis/caddy

should build without any issues

Would this be the normal way to build?
Is there a better way?
Should we reopen this issue till the build instructions succeed?


Create folder structure for pages and components

Create folder structure for pages and components

Should we add copyright in each file?

Should we add copyright in each file?
Nah


Changing --out setting in project properties should mark project as needing a clean/rebuild

It appears that changing the 'Combine JavaScript output into file' option in project properties does not cause the project system to believe that the project needs a full rebuild. This led to confusion with this situation:
Project:

a.ts
stuff/b.ts

Build, now a.js exists at the root of your project. Now turn on 'Combine JavaScript output into file' and choose a.js as the name of the target file. Build again and the project will think it's up to date, a.js will not contain the results of compiling b.ts. If you manually do a clean/(re)build then a.js will have the right stuff in it.

We just need to add the project file tot he list of MSBuild task inputs, to ensure that a change in project properties would result in a rebuild

Closing due to lack of activity - presumed fixed or irrelevant


--use-ingularity breaks shadow usage in non-singularity rules

Snakemake version
5.7.4
Describe the bug
In a workflow with some rules using singularity and some not, the ones without singularity container will not switch into a shadow directory if the workflow is executed with --use-singularity.
Logs
running the minimal example with snakemake all produces the correct
test.out, test_a.out, test-b.out and test_c.out
running the minimal example with snakemake --use-singularity -k all fails in rule_b and rule_c and produces:
test_a.out, junk_b.out, junk_c.out
Minimal example
Consider the following example
rule all:
    input: "test.out"

rule singularity_ok:
    input:
        "test_a.out",
        "test_b.out",
        "test_c.out"
    output:
        touch("test.out")
    shell:
        """
        test ! -f junk_a.out
        test ! -f junk_b.out
        test ! -f junk_c.out
        """

rule a:
    output:
        "test_a.out"
    singularity:
        "docker://bash"
    shadow:
        "minimal"
    shell:
        'echo 1 > junk_a.out; echo "test" > {output}'

rule b:
    output:
        "test_b.out"
    shadow:
        "minimal"
    shell:
        'echo 1 > junk_b.out; echo "test" > {output}'

rule c:
    output:
        "test_c.out"
    shadow:
        "minimal"
    run:
        with open(output[0], 'w') as fp:
            print("test", file=fp)
        with open('junk_c.out', 'w') as fp:
            print("junk", file=fp)

Additional context
I played a bit around, seems to be related to code in executor.py:
# Change workdir if shadow defined and not using singularity.
# Otherwise, we do the change from inside the container.
passed_shadow_dir = None
if use_singularity and job_rule.singularity_img:
    passed_shadow_dir = shadow_dir
    shadow_dir = None

This is a duplicate of issue 1280 in the bitbucket repo.
Pay

Thanks for reporting! Should be fixed by PR #89.

Thank you for the prompt reaction. And for the great software!

Thanks!