MPIPool crashes when not in loadbalance mode due to MPI buffering failure

Hi, thanks for this great tool.
We've been running emcee using MPI and have occasionally found that the program crashes with the following error:
  File ".../emcee/ptsampler.py", line 259, in sample
    results = list(self.pool.map(fn, p.reshape((-1, self.dim))))
  File ".../emcee/mpi_pool.py", line 185, in map
    result = self.comm.recv(source=worker, tag=i)
  File "Comm.pyx", line 816, in mpi4py.MPI.Comm.recv (src/mpi4py.MPI.c:66815)
  File "pickled.pxi", line 248, in mpi4py.MPI.PyMPI_recv (src/mpi4py.MPI.c:28013)
  File "pickled.pxi", line 109, in mpi4py.MPI._p_Pickle.load (src/mpi4py.MPI.c:26549)
EOFError

This would occur when running PTSampler with a large number of temperatures (50) and walkers (400; so 20,000 calls to the likelihood/prior functions per step). After googling for the error message, we found a discussion on the mpi4py mailing list in which another emcee user had encountered exactly this same problem:
https://groups.google.com/forum/#!msg/mpi4py/OJG5eZ2f-Pg/EnhN06Ozg2oJ
If you read the discussion, it turns out that the problem is that when not using loadbalancing, the default strategy is to send out ALL of the tasks to the workers asynchronously and then wait for them to respond. This relies on MPI's own buffering to handle the build up of any messages. When the number of tasks is much greater than the number of workers, MPI is more likely to drop messages and cause this approach to fail. This was consistent with our experience: when running our job on 40 cores we did not get the error, whereas when running them on 12 cores we got it every time.
Using the back-of-the-envelope implementation given by Lisandro Dalcin in the thread above fixed the problem (though, as we discovered, it introduced a substantial performance penalty--since it does not store the PTLikePrior object locally on each of the workers, each MCMC step involved pickling our functions, with args, 20,000 times!). After we understood the issue, we switched emcee to loadbalance mode, which also fixed the problem.
It seems that there are three possible ways to address this:

Remove the non-loadbalance mode from MPIPool entirely.
Leave the non-loadbalanced code unchanged in MPIPool, while documenting this issue, and potentially making the loadbalanced code the default.
Modify the non-loadbalanced code to do the following:

Send (synchronously or asynchronously) only as many tasks as there are workers.
Do a waitall until all workers are finished.
Repeat above until all tasks are completed.

Option 3, revising the non-loadbalanced mode, would appear to be very inefficient relative to the loadbalanced mode, since it makes each round of execution wait for the slowest worker. I'm not sure I understand the advantage of the nonloadbalanced mode. If there is one, perhaps we go with option 2, allowing this mode to be "use at your own risk"? If there are no substantial advantages, I would suggest just eliminating this approach entirely and making the loadbalanced version the default (option 1).

The next version of emcee won't include an MPIPool (partly because of this unresolved issue). We're recommending that users switch to https://github.com/adrn/schwimmbad instead.


vimPlugins: Update

Motivation for this change
Various upstream updates
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Assured whether relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build vimPlugins.vim-pathogen vimPlugins.vim-codefmt vimPlugins.vim-snippets


[dApp Connector] Implement dApp Connector

Is your feature request related to a problem? Please describe.

hi @hoailinh1210 , @djhahe already had the demo and somethings to kick start before. Please collab with him to make this done. Cheers!


Display "best of" poidh on the homepage

Update the homepage so that the section below "select a network" is autofilled with completed bounties + the accepted claim for those bounties.
header for this section should read "or browse past bounties"
Details should include:

bounty title
bounty amount paid in ETH/DEGEN
the winning poidh NFT card
a link to "see the bounty" page

Page should be filled with 25 bounties, randomly selected across all chains. Every time the page reloads a new 25 bounties should be picked.

Very rough mockup


404 links at https://ocm.software/*

I found the following 404 links:
https://ocm.software/docs/guides/structuring-software-with-ocm/

logical units called [components](https://github.com/open-component-model/ocm-spec/blob/main/doc/specification/elements/README.md#components)
development process new [component versions](https://github.com/open-component-model/ocm-spec/blob/main/doc/specification/elements/README.md#component-versions)
The [Flux OCM controllers](https://ocm.software/docs/guides/controller-reference/overview.md) offer this functionality too

https://ocm.software/docs/controller-reference/api-reference/ocm-controller/#delivery.ocm.software/v1alpha1.FluxDeployerSpec

helmReleaseTemplate
github.com/fluxcd/helm-controller/api/v2beta1.HelmReleaseSpec	(Optional)

fixed with https://github.com/open-component-model/ocm-website/pull/308
Final published html pages are now checked - https://github.com/open-component-model/ocm-website/actions/runs/11480219055/job/31948140998


Non-determenistic last test case for Robot Name

  def test_generate_all_robots
    all_names_count = 26 * 26 * 1000
    time_limit = Time.now + 60 # seconds
    seen_names = Hash.new(0)
    robots = []
    while seen_names.size < all_names_count && Time.now < time_limit
      robot = Robot.new
      seen_names[robot.name] += 1
      robots << robot
    end
    assert_equal all_names_count, robots.size, timeout_message
  end

I believe that the while loop here leads to non-deterministic behavior. The amount of robots generated is almost always different than the expected number because the loop uses the length of the seen_names keys and fails to take into consideration the total number of times a name was created

The seen names will never include duplicates, so this should be deterministic. The number of times the same name is created should be immaterial (and discarded).
Is this causing an intermittent failure for your solution?

The test does seem strange though, because there should be only unique robot names (I will read the description, I have not looked at it for years.)

This test looks correct to me. The number of names will be different if the code fails to generate all names within 60 seconds.

That is what I am seeing too @SleeplessByte .  I am going to close it.
@sanevillain can comment still though.  Let us know what the doubts are, and a test to maybe prove the issue, if you can.

Okay. This is super weird. Yesterday the problem for me with this last test case was that on every run a different number of robots was created and that number was unpredictable. What I mean by that is that the number of unique names that the while loop depends on can't be exceeded since they are used as hash keys in the condition and the number of times any given name could've been added is saved but not used. So all in all I was creating more robots in the given time than the total number of unique names.
However, today this last test case passes without a problem. I haven't changed my solution, but for some reason, it's passing every time I run it. I uploaded my solution again, but it seems to be failing on the website now. Will appreciate it if you can tell me if it's something in my code that leads to this weird behaviour
class Robot
  @@NAMES = []

  attr_reader :name

  def initialize
    self.class.generate_names if @@NAMES.empty?
    reset
  end

  def reset
    @name = @@NAMES.pop
  end

  class << self
    def forget
      @@NAMES = []
    end

    def generate_names
      ('A'..'Z').each do |a|
        ('A'..'Z').each do |b|
          (0...10).each do |c|
            (0...10).each do |d|
              (0...10).each do |e|
                @@NAMES << "#{a}#{b}#{c}#{d}#{e}"
              end
            end
          end
        end
      end

      @@NAMES.shuffle!
    end
  end
e

This solution looks like it would work (albeit the reset name being not optimal in mu opinion), without anything that's particularly causing different runs to have different results.

I really don't understand how this issue is already closed when every correct solution that passes the test locally times out once uploaded to the platform. (Including the example solution to the problem)

I closed it because the title of the problem states that it is non-deterministic last test case.  I believed that you indicated this is not the case.
Would you want to change the title of the issue to match the problem that you are seeing, and perhaps reopen and comment?

If this is the message that we get every time, then we have a different issue, I think.
Can you make the adjustments that you think are needed for this issue, if it is no longer a "non-deterministic last test" and a timing problem perhaps on the server.  Then we can move this issue to the test runner repository and address it there.


Detect use of 3.0 and use api instead of compile when adding runtime dep

Otherwise the plugin causes a warning in the user's build.

We'll probably just require 3.0 for the next version at this point.

51e715f202df176990b46d22ee854f2fcd2ba29f


improvements top the export dialog box

in the export dialog box the funcionality in the download button need to be transferred to the ok button (i.e. one of the 2 buttons has to go)
also: clicking the button should close the box (at the moment true for the [ok] and the [cancel], but not for the [download]

done


[Bug]: When view is enabled the breadcrumbs link is still to edit page...

What happened?
I think that if we have a view page, the breadcrumbs should link to the view page (which is the default for clicking on the record itself), not the edit link that it is currently linking to.
Is there any way to overwrite it?
How to reproduce the bug
just make a nested rescource with the --view flag
Package Version
3.x
PHP Version
8.3
Laravel Version
10
Which operating systems does with happen with?
No response
Notes
No response

Currently the it opens the same page you're viewing: so if you're on an edit page, it should open also on edit page, if you're on a view page, it should open in a view page also.

Hello, I'm closing this in favor of the newly released production version. If you still encounter any issues with the plugin, please feel free to open a new issue.


Make the name generator more random

The random shuffle algorithm used by the name generator was buggy.
It swapped N pairs of randomly selected names, where N is the total
number of names.  What it should have been doing was swapping each
name with a randomly chosen name.  This makes it much more likely
that each name will be moved from its original position in the
slice of possible names.  There should be less alarmed-agravains
from now on.
Signed-off-by: Mark Ryan<EMAIL_ADDRESS>

Coverage decreased (-0.01%) to 68.212% when pulling e2a455c2fcdd00324ddd494ff4d1bff4b989005d on markdryan:fix-shuffle into 580e0afd42ae1bb67f2bd00d597dd7064bf80c6f on intel:master.


Validation stop working if extendSchema is called.

As title, if I use extendSchema after registering mercurius-validation, it no longer works.
${mercuriusValidation.graphQLTypeDefs}

type Query {
  foo(page: Pagination!): Int!
}

input Pagination {
  skip: Int @constraint(minimum: 0)
  take: Int @constraint(minimum: 1)
}

await fastify.register(mercurius, {
    schema,
    resolvers,
    path: "/graphql",
    graphiql: false
  });
await fastify.register(mercuriusValidation);

// From another plugin
fastify.graphql.extendSchema(schema);

Great spot! Would you like to send a PR to address this?

I have notice that it is able to refresh onGatewayReplaceSchema.
https://github.com/mercurius-js/validation/blob/6f8584016d601b7d1b3940ad0b144bb194dbb1c4/index.js#L19-L21
However, there are not any hooks on extendSchema. Unless, this plugin expose a function to force refresh, it needs mercurius to add a new hook.

I think adding an onExtendSchema system hook to mercurius could be pretty useful! Would you be interested in adding this as well?
@mcollina what do you think?

I concur with your assessment!

I'm attempting to work on this, but I ran into a problem where extendSchema is defined outside of the fastifyGraphql method, making it impossible for me to access to source or context parameters that I need to create the request to send to the hook handler. Am I missing something?


In the editor, the water is drawn over gizmos.

See this:

The transformation gizmos are fine, but icons, grid lines and others are rendered under the water. On stock Godot it's an annoyance at worst, however some addons I use render the information on the icon layer, meaning that important stuff is hidden under the water and I have to hide the water mesh just to see it.
//edit: Camera frustum preview is affected as well:

as well as the 3d path editor stuff:

So even with stock Godot the problem is bad.

Hey there! Thanks for reporting!
I always thought of this as a general Godot with custom shaders problem.
Anyway, I found a solution for now:

If you turn down the render priority of the gdshader on the water surface it works:

I'm hesitant to push this to the repo just yet, because I don't know the side effects of this yet.

I see. BTW. Setting render priority to -1 does the trick just fine, you don't need to set it that low.


Cannot run the tests

Hi,
I want to use your library but I can't run the unit tests.
First upon initilization the library couldn't find the julia bin folder. I solved this for now by simply setting _juliaDir in the Julia.cs file like this:
private static string _JuliaDir= @"C:\Users\rreij\AppData\Local\Programs\Julia-1.8.0\bin";
But there has to be a better way. Should I set an environment variable in windows?
Next the initialization halted at the init_mods() method when running the line:
   JULIAdotNET = Julia.Eval("JULIAdotNET");
When I open a Juia REPL in visual code and enter using JULIAdotNET  I get the error at the end of this message.
I'm using Windows 11 and Julia 1.8.0.
Because I don't have .net 5 I also changed the project to .net 6.
Here's the error code when running using JULIAdotNET :
[ Info: Precompiling JULIAdotNET [6b20c3d1-d1a1-4518-91d2-6d987d94bff0] TESt!! ERROR: LoadError: ccall method definition: argument 2 type doesn't correspond to a C type Stacktrace:   [1] top-level scope     @ C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\julia\SharpForJulia\NetCore.jl:74   [2] include(mod::Module, _path::String)     @ Base .\Base.jl:419   [3] include(x::String)     @ JULIAdotNET.Sharp C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\julia\Sharp.jl:1   [4] top-level scope     @ C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\julia\Sharp.jl:21   [5] include(mod::Module, _path::String)     @ Base .\Base.jl:419   [6] include(x::String)     @ JULIAdotNET C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\JULIAdotNET.jl:1   [7] top-level scope     @ C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\JULIAdotNET.jl:5   [8] include     @ .\Base.jl:419 [inlined]   [9] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)     @ Base .\loading.jl:1554  [10] top-level scope     @ stdin:1 in expression starting at C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\julia\SharpForJulia\NetCore.jl:1 in expression starting at C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\julia\Sharp.jl:1 in expression starting at C:\Users\rreij\.julia\packages\JULIAdotNET\EITFN\src\JULIAdotNET.jl:1 in expression starting at stdin:1 ERROR: Failed to precompile JULIAdotNET [6b20c3d1-d1a1-4518-91d2-6d987d94bff0] to C:\Users\rreij\.julia\compiled\v1.8\JULIAdotNET\jl_9C01.tmp.
I am currently fairly busy with college but I will do my best to patch them. Thanks for the log, I will check this out when I have time.
I am currently trying to build an experimental julia runtime based on .NET.
This project would provide full native interopability with .NET and julia and would deprecate JuliadotNet. I will continue to maintain JuliaDotNet for now but keep in mind I plan to deprecate this once I have a working julia runtime .NET prototype.

Thank for the quick reply.
I'm really looking forward to the new project.

if i can actually run code like this https://github.com/weymouth/WaterLily.jl
i would probably use it..   and see about doing the viusalization using a game engine like monogame or mabye Silk.. . his other version is in protobuld.. which is easy to port C# but doesn't not include any of his performance stuff...    even if its a many months time, im just curious how likely that is..  i haven't evaluated the code yet, ran pretty fast before but on net 6 might be blazing and with his newest code.

if i can actually run code like this https://github.com/weymouth/WaterLily.jl
i would probably use it.. and see about doing the viusalization using a game engine like monogame or mabye Silk.. . his other version is in protobuld.. which is easy to port C# but doesn't not include any of his performance stuff... even if its a many months time, im just curious how likely that is.. i haven't evaluated the code yet, ran pretty fast before but on net 6 might be blazing and with his newest code.

If you only intend to plot 2D stuff, try ScottPlot

Quick update on my julia runtime in .net project.
Made some pretty good progress! Very far from a working (able to actually run julia code) though. It is quite a massive project that will take a while. I have load out a plan in the readme of the project. Take a look if your curious and/or interested in helping out! This will take a team effort to complete but I think will be worth it. I call it Julia++ as it would be more a fork of Julia since its not official at all.

thanks,  Scott Plot looks ok,  but im needing to display a visual flow,
like an advected  density field to show a fluid  represented as an 2d
image,  at real time interactive rates,  many of his Weymouth simulations
are real time interactive.   so i need it to be   deployable on mobile and
pc and that one Scott Plot looks to high level and  wpf based..
https://github.com/weymouth/WaterLily.jl/blob/master/examples/julia.gif
i hope not TLDR but I can't overstate how important i thnk this project is.
So, Weymouth is a leading researcher solving problems for the US Navy (
you won't find on open source)... hes movng to julia because he has fortran
MPI code, tensor flow on cloud in C and python scripts, to implement for
actually practical and cheap Machine learning to help solve nonlinear  ODEs
faster in real time , approximately linearizing very chaotic and complex
phenomena such as the wakes of boats, visible to satellites, in almost real
time as opposed to weeks,   This is for opimizing swimming robots , foils
and understanding fishes ,   to swim wiht least effort while generating
turbulence in 3d which has never been accurately modelled.
AS for  and me, im merely making a 2d game with surfers and fishes and
breaking waves so that kids can play games, encourage sports,  but learn
how thngs really move not like Mario Bros.
so i  i aim to run the physics at up to  1000 fps if i can load all the
cores and hopefully SIMD lanes, on  background threads, then I  draw the
last good frame of visible data using the producer/consumer pattern, so,
when i get a refresh call usually at 60hz...at which pont i draw the last
updated physics image without needing a lock, since its building a the next
frame in another bufer at the same time .
his old java code is pretty fast and interactive, easier to port to C#,
but the new julia one is much more advanced.. its using quite alot of Julia
math and standard  libraries.. for performance.  with ..net i can alos use
generalize shaders at least for upscaling wth monogame so ill stick with
mongame for now.
Silk .net is making huge strides in generalized scientific visualization
and gpgpu will works hopeful wth one code source , and easy to deploy...
with 500 people on it i think the progress will outpace others.   Since
microsoft sells cloud services and use ARM.. I think they dont care so much
about locking oto  windows , certain hardware, or anything anymore, so we
benefit from this new model.
*But i can't  understate the importance of this or a similar project...*i
think of this project because of how much effort Microsoft is putting in
Net6 and now 7 has net intrinsics.   possible google are openng sources too
but they dont have a sort of IL as far as  I know..   What this will do is
justify the 20 year old unfullfilled  promse of JIT compiling as the
intrinsics on a particular device are not known A priori ( to use a trendy
AI term from Kant)   to deployment.   SIM vectorization is now hideous
code.. must be done for each type of processor, hopefully generalized and
easier in Net 7.  I don't know who to talk to at microsoft but tooling is
another way they make money but JetBrains, a direct competitor  has
sponsored silk and its all under MIT licences..  Ther is very little
squabbling and its alll transparent and visible in the vautls f anyone has
worries about MSFT being a sort of evil /greedy / force or foottrap, (
anymore)  ..
i wont use hs code until mabye next year because its too slow and too exact
for my needs so i invent first order chimera methods for now.  but id love
to just script it wiht Rosylin and buld it with and deploy it wht Net7 wiht
fast math libraries (either Julia or MathNet)  that use JIT and simd.
On Mon, Sep 26, 2022 at 4:16 AM BambOoxX @.***> wrote:

if i can actually run code like this
https://github.com/weymouth/WaterLily.jl
i would probably use it.. and see about doing the viusalization using a
game engine like monogame or mabye Silk.. . his other version is in
protobuld.. which is easy to port C# but doesn't not include any of his
performance stuff... even if its a many months time, im just curious how
likely that is.. i haven't evaluated the code yet, ran pretty fast before
but on net 6 might be blazing and with his newest code.
If you only intend to plot 2D stuff, try ScottPlot
https://github.com/ScottPlot/ScottPlot
—
Reply to this email directly, view it on GitHub
https://github.com/HyperSphereStudio/JULIAdotNET/issues/16#issuecomment-1257811170,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD74XGKADMPHDDUG2KJJEETWAFZYHANCNFSM6AAAAAAQIR5TVQ
.
You are receiving this because you commented.Message ID:
@.***>

I have been working on this project this past winter break. Check out the new readme and feel free to ask if you have any questions. A lot of my energy has been going into developing my Spinor Prog Lang. (Which is actually using a lot of the ideas from this project). But I will be happy to answer them! This newest version does not have a lot of the features of the old version (V3) but I don't think anyone used them anyway. It was a nightmare it maintain.


can't  support set deployment.spec.template.spec.containers[0].env.valueFrom.fieldRef.fieldPath from metadata.labels in kuboard-ui

What happened:
1、can't  support set deployment.spec.template.spec.containers[0].env.valueFrom.fieldRef.fieldPath from metadata.labels in kuboard-ui
What you expected to happen:
1、support set deployment.spec.template.spec.containers[0].env.valueFrom.fieldRef.fieldPath from metadata.labels in kuboard-ui

I think this is caused by Kubernetes version compatibility. Make sure the kuboard version compatible with kubernetes:v1.19.8


Initial Template example for Harness CI Image Factory

A Terraform template to build a custom Harness pipeline.  The pipeline will pull all listed Harness CI images into a list which is then used to build and push those images to the chosen container registry.

@goodrum-harness just noticed something, you put a provider definition in the module.
This is a bad pattern as deleting the module from your terraform code produces errors as the provider definition is gone. I think we should remove the provider definition from the terraform and force the user to configure it themselves in the root of their terraform project. I have bitten myself in the behind in a previous life when I made my first module and included the provider definition within it.
We would delete provider.tf along with all of the vars associated with it: harness_platform_account harness_platform_key
I have this change ready to commit/push, just let me know if you agree.


Build error when make examples , patch with glib-2.0 positon move will solve.

In case of some glib-2.0 linker error ,
make examples
can't link with glib2.0,  in this case -lglib-2.0  to after includes and move to last will solve the issues.

This just re-orders the arguments. When did you encounter a problem with the current order?


fix(sec): upgrade github.com/dapr/dapr to 1.11.2

What happened？
There are 1 security vulnerabilities found in github.com/dapr/dapr v0.0.0

CVE-2023-37918

What did I do？
Upgrade github.com/dapr/dapr from v0.0.0 to 1.11.2 for vulnerability fix
What did you expect to happen？
Ideally, no insecure libs should be used.
How can we automate the detection of these types of issues?
By using the GitHub Actions configurations provided by murphysec, we can conduct automatic code security checks in our CI pipeline.
The specification of the pull request
PR Specification from OSCS

@ChengDaqi2023 Please sign off the commits for DCO checks to pass https://docs.dapr.io/contributing/contributing-overview/#developer-certificate-of-origin-signing-your-work

This PR is completely unnecessary due to the replace directive in the go.mod that already exists.

@mukundansundar I recommend closing this PR.


start playing with panel

Goal:

expose the interactive datashader map of NYC taxi data as a Panel (https://panel.holoviz.org/) application

Where I am now:

panel serve 2-dataframes-at-scale.ipynb --show runs a panel application showing (just to make iteration faster for now) a subset of the data using a LocalCluster

Where I'm stuck:

if I load the application in multiple tabs and play with it in both, I get ValueError: Inputs contain futures that were created by another client. and all but one of the tabs are broken

In the real deployment I'd hope to use the same cluster, but this is happening even with each browser tab independently launching a separate dask LocalCluster.
I guess something in Datashader or Panel or the interaction isn't thread-safe maybe? I don't really know anything about how Panel or Datashader work.
Next steps if I get past this:

switch to a Coiled cluster w/ bigger data
learn how to use pn.state.cache to cache some shared state (e.g. a reference to the cluster, and how to find the already-loaded dataset in the cluster)
deploy it somewhere

More details on my error:

```
panel serve 2-dataframes-at-scale.ipynb --show
2022-12-10 09:41:19,417 Starting Bokeh server version 2.4.3 (running on Tornado 6.2)
2022-12-10 09:41:19,417 User authentication hooks NOT provided (default user enabled)
2022-12-10 09:41:19,418 Bokeh app running at: http://localhost:5006/2-dataframes-at-scale
2022-12-10 09:41:19,418 Starting Bokeh server with process id: 97777
hello!!!
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:108: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"            \n",
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:111: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"\n",
WARNING:param.RGBPlot00990: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
2022-12-10 09:41:48,648 Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
^C
Interrupted, shutting down
(dask-tutorial-panel) (main *) /Users/davidchudzicki/dask-tutorial $
panel serve 2-dataframes-at-scale.ipynb --show
2022-12-10 09:42:04,607 Starting Bokeh server version 2.4.3 (running on Tornado 6.2)
2022-12-10 09:42:04,608 User authentication hooks NOT provided (default user enabled)
2022-12-10 09:42:04,609 Bokeh app running at: http://localhost:5006/2-dataframes-at-scale
2022-12-10 09:42:04,609 Starting Bokeh server with process id: 98437
/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 51027 instead
warnings.warn(
2022-12-10 09:42:05,329 State start
2022-12-10 09:42:05,332   Scheduler at:     tcp://<IP_ADDRESS>:51028
2022-12-10 09:42:05,332   dashboard at:           <IP_ADDRESS>:51027
2022-12-10 09:42:05,349         Start Nanny at: 'tcp://<IP_ADDRESS>:51031'
2022-12-10 09:42:05,428         Start Nanny at: 'tcp://<IP_ADDRESS>:51032'
2022-12-10 09:42:05,436         Start Nanny at: 'tcp://<IP_ADDRESS>:51033'
2022-12-10 09:42:05,443         Start Nanny at: 'tcp://<IP_ADDRESS>:51034'
2022-12-10 09:42:05,450         Start Nanny at: 'tcp://<IP_ADDRESS>:51035'
2022-12-10 09:42:06,573 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51050', name: 4, status: init, memory: 0, processing: 0>
2022-12-10 09:42:06,584 Starting worker compute stream, tcp://<IP_ADDRESS>:51050
2022-12-10 09:42:06,584 Starting established connection to tcp://<IP_ADDRESS>:51053
2022-12-10 09:42:06,585 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51049', name: 3, status: init, memory: 0, processing: 0>
2022-12-10 09:42:06,586 Starting worker compute stream, tcp://<IP_ADDRESS>:51049
2022-12-10 09:42:06,586 Starting established connection to tcp://<IP_ADDRESS>:51054
2022-12-10 09:42:06,600 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51055', name: 1, status: init, memory: 0, processing: 0>
2022-12-10 09:42:06,600 Starting worker compute stream, tcp://<IP_ADDRESS>:51055
2022-12-10 09:42:06,601 Starting established connection to tcp://<IP_ADDRESS>:51061
2022-12-10 09:42:06,602 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51057', name: 0, status: init, memory: 0, processing: 0>
2022-12-10 09:42:06,602 Starting worker compute stream, tcp://<IP_ADDRESS>:51057
2022-12-10 09:42:06,602 Starting established connection to tcp://<IP_ADDRESS>:51062
2022-12-10 09:42:06,604 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51056', name: 2, status: init, memory: 0, processing: 0>
2022-12-10 09:42:06,604 Starting worker compute stream, tcp://<IP_ADDRESS>:51056
2022-12-10 09:42:06,604 Starting established connection to tcp://<IP_ADDRESS>:51063
2022-12-10 09:42:06,645 Receive client connection: Client-d1abf140-7898-11ed-8085-1236337d691c
2022-12-10 09:42:06,646 Starting established connection to tcp://<IP_ADDRESS>:51064
hello!!!
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:110: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Dashboard: <a href="http://<IP_ADDRESS>:50957/status\" target="_blank">http://<IP_ADDRESS>:50957/status\n",
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:113: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Total threads: 10\n",
WARNING:param.RGBPlot00990: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
2022-12-10 09:42:14,944 Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 51081 instead
warnings.warn(
2022-12-10 09:42:23,451 State start
2022-12-10 09:42:23,454   Scheduler at:     tcp://<IP_ADDRESS>:51082
2022-12-10 09:42:23,454   dashboard at:           <IP_ADDRESS>:51081
2022-12-10 09:42:23,473         Start Nanny at: 'tcp://<IP_ADDRESS>:51085'
2022-12-10 09:42:23,473         Start Nanny at: 'tcp://<IP_ADDRESS>:51086'
2022-12-10 09:42:23,475         Start Nanny at: 'tcp://<IP_ADDRESS>:51087'
2022-12-10 09:42:23,491         Start Nanny at: 'tcp://<IP_ADDRESS>:51088'
2022-12-10 09:42:23,509         Start Nanny at: 'tcp://<IP_ADDRESS>:51089'
2022-12-10 09:42:24,989 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51102', name: 1, status: init, memory: 0, processing: 0>
2022-12-10 09:42:24,991 Starting worker compute stream, tcp://<IP_ADDRESS>:51102
2022-12-10 09:42:24,991 Starting established connection to tcp://<IP_ADDRESS>:51115
2022-12-10 09:42:24,992 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51103', name: 3, status: init, memory: 0, processing: 0>
2022-12-10 09:42:24,992 Starting worker compute stream, tcp://<IP_ADDRESS>:51103
2022-12-10 09:42:24,992 Starting established connection to tcp://<IP_ADDRESS>:51112
2022-12-10 09:42:24,993 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51101', name: 0, status: init, memory: 0, processing: 0>
2022-12-10 09:42:24,993 Starting worker compute stream, tcp://<IP_ADDRESS>:51101
2022-12-10 09:42:24,993 Starting established connection to tcp://<IP_ADDRESS>:51111
2022-12-10 09:42:24,993 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51100', name: 2, status: init, memory: 0, processing: 0>
2022-12-10 09:42:24,994 Starting worker compute stream, tcp://<IP_ADDRESS>:51100
2022-12-10 09:42:24,994 Starting established connection to tcp://<IP_ADDRESS>:51114
2022-12-10 09:42:24,994 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51104', name: 4, status: init, memory: 0, processing: 0>
2022-12-10 09:42:24,994 Starting worker compute stream, tcp://<IP_ADDRESS>:51104
2022-12-10 09:42:24,994 Starting established connection to tcp://<IP_ADDRESS>:51113
2022-12-10 09:42:25,014 Receive client connection: Client-dc9eac1e-7898-11ed-8085-1236337d691c
2022-12-10 09:42:25,014 Starting established connection to tcp://<IP_ADDRESS>:51116
hello!!!
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:110: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Dashboard: <a href="http://<IP_ADDRESS>:50957/status\" target="_blank">http://<IP_ADDRESS>:50957/status\n",
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:113: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Total threads: 10\n",
WARNING:param.RGBPlot01484: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
2022-12-10 09:42:32,029 Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/distributed/node.py:182: UserWarning: Port 8787 is already in use.
Perhaps you already have a cluster running?
Hosting the HTTP server on port 51129 instead
warnings.warn(
2022-12-10 09:42:32,212 State start
2022-12-10 09:42:32,214   Scheduler at:     tcp://<IP_ADDRESS>:51130
2022-12-10 09:42:32,214   dashboard at:           <IP_ADDRESS>:51129
2022-12-10 09:42:32,303         Start Nanny at: 'tcp://<IP_ADDRESS>:51134'
2022-12-10 09:42:32,304         Start Nanny at: 'tcp://<IP_ADDRESS>:51135'
2022-12-10 09:42:32,305         Start Nanny at: 'tcp://<IP_ADDRESS>:51133'
2022-12-10 09:42:32,328         Start Nanny at: 'tcp://<IP_ADDRESS>:51136'
2022-12-10 09:42:32,359         Start Nanny at: 'tcp://<IP_ADDRESS>:51137'
2022-12-10 09:42:33,477 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51149', name: 2, status: init, memory: 0, processing: 0>
2022-12-10 09:42:33,477 Starting worker compute stream, tcp://<IP_ADDRESS>:51149
2022-12-10 09:42:33,477 Starting established connection to tcp://<IP_ADDRESS>:51153
2022-12-10 09:42:33,478 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51148', name: 3, status: init, memory: 0, processing: 0>
2022-12-10 09:42:33,478 Starting worker compute stream, tcp://<IP_ADDRESS>:51148
2022-12-10 09:42:33,478 Starting established connection to tcp://<IP_ADDRESS>:51151
2022-12-10 09:42:33,490 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51154', name: 0, status: init, memory: 0, processing: 0>
2022-12-10 09:42:33,491 Starting worker compute stream, tcp://<IP_ADDRESS>:51154
2022-12-10 09:42:33,491 Starting established connection to tcp://<IP_ADDRESS>:51156
2022-12-10 09:42:33,528 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51157', name: 4, status: init, memory: 0, processing: 0>
2022-12-10 09:42:33,528 Starting worker compute stream, tcp://<IP_ADDRESS>:51157
2022-12-10 09:42:33,528 Starting established connection to tcp://<IP_ADDRESS>:51161
2022-12-10 09:42:33,529 Register worker <WorkerState 'tcp://<IP_ADDRESS>:51158', name: 1, status: init, memory: 0, processing: 0>
2022-12-10 09:42:33,529 Starting worker compute stream, tcp://<IP_ADDRESS>:51158
2022-12-10 09:42:33,529 Starting established connection to tcp://<IP_ADDRESS>:51162
2022-12-10 09:42:33,567 Receive client connection: Client-e1b7d7de-7898-11ed-8085-1236337d691c
2022-12-10 09:42:33,567 Starting established connection to tcp://<IP_ADDRESS>:51163
hello!!!
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:110: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Dashboard: <a href="http://<IP_ADDRESS>:50957/status\" target="_blank">http://<IP_ADDRESS>:50957/status\n",
/Users/davidchudzicki/dask-tutorial/2-dataframes-at-scale.ipynb:113: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
"                        Total threads: 10\n",
WARNING:param.RGBPlot01866: Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
2022-12-10 09:42:39,572 Due to internal constraints, when aspect and width/height is set, the bokeh backend uses those values as frame_width/frame_height instead. This ensures the aspect is respected, but means that the plot might be slightly larger than anticipated. Set the frame_width/frame_height explicitly to suppress this warning.
WARNING:param.dynamic_operation: Callable raised "ValueError('Inputs contain futures that were created by another client.')".
Invoked as dynamic_operation(height=526, scale=1.0, width=700, x_range=(-74.1102442857143, -73.79091428571431), y_range=(40.623141095057036, 40.900907095057036))
2022-12-10 09:42:42,227 Callable raised "ValueError('Inputs contain futures that were created by another client.')".
Invoked as dynamic_operation(height=526, scale=1.0, width=700, x_range=(-74.1102442857143, -73.79091428571431), y_range=(40.623141095057036, 40.900907095057036))
WARNING:param.dynamic_operation: Callable raised "ValueError('Inputs contain futures that were created by another client.')".
Invoked as dynamic_operation(height=526, scale=1.0, width=700, x_range=(-74.1102442857143, -73.79091428571431), y_range=(40.623141095057036, 40.900907095057036))
2022-12-10 09:42:42,227 Callable raised "ValueError('Inputs contain futures that were created by another client.')".
Invoked as dynamic_operation(height=526, scale=1.0, width=700, x_range=(-74.1102442857143, -73.79091428571431), y_range=(40.623141095057036, 40.900907095057036))
Traceback (most recent call last):
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/plotting/util.py", line 280, in get_plot_frame
return map_obj[key]
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 1217, in getitem
val = self._execute_callback(*tuple_key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 984, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 582, in call
ret = self.callable(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/util/init.py", line 1012, in dynamic_operation
key, obj = resolve(key, kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/util/init.py", line 1001, in resolve
return key, map_obj[key]
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 1217, in getitem
val = self._execute_callback(*tuple_key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 984, in _execute_callback
retval = self.callback(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/spaces.py", line 582, in call
ret = self.callable(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/util/init.py", line 1013, in dynamic_operation
return apply(obj, *key, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/util/init.py", line 1005, in apply
processed = self._process(element, key, kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/util/init.py", line 987, in _process
return self.p.operation.process_element(element, key, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/operation.py", line 194, in process_element
return self._apply(element, key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/operation.py", line 141, in _apply
ret = self._process(element, key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/operation/datashader.py", line 1593, in _process
agg = rasterize._process(self, element, key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/operation/datashader.py", line 1572, in _process
element = element.map(op, predicate)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/data/init.py", line 204, in pipelined_fn
result = method_fn(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/data/init.py", line 1216, in map
return super(Dataset, self).map(*args, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/dimension.py", line 700, in map
return map_fn(self) if applies else self
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/operation.py", line 214, in call
return self._apply(element)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/core/operation.py", line 141, in _apply
ret = self._process(element, key)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/holoviews/operation/datashader.py", line 522, in _process
agg = getattr(cvs, glyph)(dfdata, x.name, y.name, agg_fn, **agg_kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/datashader/core.py", line 217, in points
return bypixel(source, self, glyph, agg)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/datashader/core.py", line 1253, in bypixel
return bypixel.pipeline(source, schema, canvas, glyph, agg, antialias=antialias)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/datashader/utils.py", line 109, in call
return lk[typ](head, *rest, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/datashader/data_libraries/dask.py", line 29, in dask_pipeline
return da.compute(dsk, scheduler=scheduler)[0]
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/dask/base.py", line 600, in compute
results = schedule(dsk, keys, **kwargs)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/distributed/client.py", line 3102, in get
futures = self._graph_to_futures(
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/distributed/client.py", line 3010, in _graph_to_futures
dsk = dsk.dask_distributed_pack(self, keyset, annotations)
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/dask/highlevelgraph.py", line 1078, in dask_distributed_pack
"state": layer.dask_distributed_pack(
File "/opt/miniconda3/envs/dask-tutorial-panel/lib/python3.10/site-packages/dask/highlevelgraph.py", line 391, in dask_distributed_pack
raise ValueError(
ValueError: Inputs contain futures that were created by another client.

I think by default Panel doesn't even use threads to parallelize work from different users.
Somehow the state for different users must be mixing anyway.

I'd like to better understand how you're setting up the application.
Background: if one client is used to persist a dataset and then if another client tries to touch that dataset Dask will say "Hrm, that seems maybe scary, you shouldn't do that"
client = Client(...)
df = df.persist()

client2 = Client(set_as_default=True)
df.sum().compute()  # this errs

What's common is to record the dataset in the scheduler and replicate it to different clients
client = Client(...)
df = df.persist()
client.datasets["df"] = df

client2 = Client(set_as_default=True)
df = client2.datasets["df"]  # new dataset, new futures, all tied to client2

df.sum().compute()

This seems maybe a little pedantic, but it's useful for things like garbage collection and tracking the lifecycle of data.
Now, how you got two clients I don't know.  Maybe this is Panel running the same code making different clients but pointing to the same cluster?  I don't know panel well enough nor what you're doing with it well enough to diagnose currently.

Now, how you got two clients I don't know. Maybe this is Panel running the same code making different clients but pointing to the same cluster?

Seems like it should be different clusters. At least my logs above show several LocalClusters starting up.
My problem maybe isn't very dasky. I think I'm confusing panel very badly with my stuff in the global namespace.


Upgrade to Rust 1.50.0

Let's update everything to "latest and greatest" Rust v1.50.0.
Let's do this after the CosmWasm 0.14.0-alpha1 release today, so that alpha2 and upcoming use 1.50.0.

Since our stack is not only compiled by us but also by contract developers, we always need to define a range of versions that are supported. CI should always check that the lowest supported version works. We bumped th min Rust version to 1.49.0 in order to get the implementation of std::convert::From<std::vec::Vec<u8>> for [u8; 20].
So the qustion is: do we only check 1.50.0 locally and in rust-optimizer? Or do we bum the min Rust version for all contract devs to 1.50.0? I'm fine either way.

Already checked 1.50.0 runs locally
I would bump everything to 1.50.0 (or, in any case, to 1.49.0) for simplicity.

Fine, then we use 1.50.0 as the minimum version for the 0.14 release series.


ci: build examples and deply in a subfolder

Addition of building of examples site and publishing under /examples/.

I'm getting some interesting errors:
+ yarn build

yarn run v1.22.17
$ run-s build:**
$ tsc -p tsconfig.json
$ tsc -p tsconfig.dev.json
src/lib/discovery/enrtree.spec.ts:20:14 - error TS2571: Object is of type 'unknown'.
20       expect(e.toString()).includes(
                ~

src/lib/discovery/enrtree.spec.ts:30:14 - error TS2571: Object is of type 'unknown'.
30       expect(e.toString()).includes('Unable to verify ENRTree root signature');
                ~

src/lib/discovery/enrtree.spec.ts:38:14 - error TS2571: Object is of type 'unknown'.
38       expect(e.toString()).includes('Could not parse ENRTree root entry');
                ~

src/lib/discovery/enrtree.spec.ts:54:14 - error TS2571: Object is of type 'unknown'.
54       expect(e.toString()).includes(
                ~
...

https://ci.status.im/blue/organizations/jenkins/website%2Fjs-waku.wakuconnect.dev/detail/js-waku.wakuconnect.dev/26/pipeline/

I'm getting some interesting errors:
+ yarn build

yarn run v1.22.17
$ run-s build:**
$ tsc -p tsconfig.json
$ tsc -p tsconfig.dev.json
src/lib/discovery/enrtree.spec.ts:20:14 - error TS2571: Object is of type 'unknown'.
20       expect(e.toString()).includes(
                ~

src/lib/discovery/enrtree.spec.ts:30:14 - error TS2571: Object is of type 'unknown'.
30       expect(e.toString()).includes('Unable to verify ENRTree root signature');
                ~

src/lib/discovery/enrtree.spec.ts:38:14 - error TS2571: Object is of type 'unknown'.
38       expect(e.toString()).includes('Could not parse ENRTree root entry');
                ~

src/lib/discovery/enrtree.spec.ts:54:14 - error TS2571: Object is of type 'unknown'.
54       expect(e.toString()).includes(
                ~
...

ci.status.im/blue/organizations/jenkins/website%2Fjs-waku.wakuconnect.dev/detail/js-waku.wakuconnect.dev/26/pipeline

Please use npm. The errors are related to the  latest TypeScript version. We do not use it in the repo (yet).
By using yarn it installed the latest version of TypeScript, with npm, it should use the correct version.

The errors are related to the latest TypeScript version. We do not use it in the repo (yet).
By using yarn it installed the latest version of TypeScript, with npm, it should use the correct version.

What? Why would it install the wrong version if we explicitly specify the verison in package.json?
https://github.com/status-im/js-waku/blob/671c757fa21c06de37c7ad6e2d4aa877f49a688b/examples/eth-pm/package.json#L23
We can use npm but this is some weird shit.

The errors are related to the latest TypeScript version. We do not use it in the repo (yet).
By using yarn it installed the latest version of TypeScript, with npm, it should use the correct version.

What? Why would it install the wrong version if we explicitly specify the verison in package.json?
https://github.com/status-im/js-waku/blob/671c757fa21c06de37c7ad6e2d4aa877f49a688b/examples/eth-pm/package.json#L23
We can use npm but this is some weird shit.

Good point. I'll investigate more.

Weird failure for eth-pm:

+ npm ci
npm ERR! code ENOTEMPTY
npm ERR! syscall rename
npm ERR! path /home/jenkins/workspace/website/js-waku.wakuconnect.dev/node_modules/sane
npm ERR! dest /home/jenkins/workspace/website/js-waku.wakuconnect.dev/node_modules/.sane-dx8KQgp8
npm ERR! errno -39
npm ERR! ENOTEMPTY: directory not empty, rename '/home/jenkins/workspace/website/js-waku.wakuconnect.dev/node_modules/sane' -> '/home/jenkins/workspace/website/js-waku.wakuconnect.dev/node_modules/.sane-dx8KQgp8'
npm ERR! A complete log of this run can be found in:
npm ERR!     /home/jenkins/.npm/_logs/2022-01-25T11_19_04_344Z-debug.log
script returned exit code 217

Wat?

 ENOTEMPTY: directory not empty, rename '/home/jenkins/workspace/website/js-waku.wakuconnect.dev/examples/eth-pm-wallet-encryption/node_modules/acorn-globals' -> '/home/jenkins/workspace/website/js-waku.wakuconnect.dev/examples/eth-pm-wallet-encryption/node_modules/.acorn-globals-h3wtysQn'

Which appear mostly random.

Using npm install causes the same issue. And it makes no sense because we call cleanWs() after every build:
 > a ci-slave-linux -a 'ls /home/jenkins/workspace/website/js-waku.wakuconnect.dev'
linux-03.he-eu-hel1.slave.ci | FAILED | rc=2 >>
ls: cannot access '/home/jenkins/workspace/website/js-waku.wakuconnect.dev': No such file or directorynon-zero return code
linux-04.he-eu-hel1.slave.ci | FAILED | rc=2 >>
ls: cannot access '/home/jenkins/workspace/website/js-waku.wakuconnect.dev': No such file or directorynon-zero return code
linux-01.he-eu-hel1.slave.ci | FAILED | rc=2 >>
ls: cannot access '/home/jenkins/workspace/website/js-waku.wakuconnect.dev': No such file or directorynon-zero return code
linux-02.he-eu-hel1.slave.ci | FAILED | rc=2 >>
ls: cannot access '/home/jenkins/workspace/website/js-waku.wakuconnect.dev': No such file or directorynon-zero return code

So we always start with a fresh clone.

Do you want me to convert to yarn see if it helps?

@D4nte should be done, check it out.

@jakubgs awesome! I did not realize file: would make a difference (it did not locally)! well spotted.


Pantabox API change for current power

Describe the bug
With the latest firmware v1.8.6 the API response for current power was changed from kW to W.
So now PR #7986 needs to be reverted

Steps to reproduce
See #7984
Configuration details
.

Log details
.

What type of operating system are you running?
Linux
Version
0.117.3

Closed in https://github.com/evcc-io/evcc/commit/6699cca0253cff62a8d42c71c0dde67e9fd77217


request for a simpler way to simply share a cookie

The premise here is perfect and on point.  That is, as an organization, we want to move forward with our Microsoft based technology stack, and .NET Core looks great.  But... we have a very large investment in our current Microsoft based technology stack, one that is working very well, mind you.  It happens to be a web forms based website that has over 10 years of development investment in it.  So, rewriting that in one fell swoop is not doable.  Bringing up a new .NET Core site along side of the existing web forms and migrating over makes perfect sense.
Authentication is key here.  So we spent a week trying to follow this technique, including having to add OWIN and "Katana-ize" a web forms website.  OK.  Then add nuget pkg after nuget pkg.  Then tweak the code here and fight Visual Studio (2017, targeting .NET Framework 4.8) to find the right references and usings.  41 DLLs were added to our Bin directory.  It more or less works, but this feels awfully fragile and way too prone to issues.
41 DLLs.  All to share one encrypted cookie to be recognized as the "authentication" cookie.  Do you see how over complicated this is?
It's hard to fathom that Microsoft couldn't manage one method() to add to .NET core, a CookieAuthenticationOption, a DataProtectionProvider option, something... that simply sets up a "FormsAuthCompatible" cookie.  That's it.  Then all the organizations that have vested years of Web Forms (and even MVC) forms auth are off and running in that "side by side" development.  I'm not asking for .NET to "support" Web Forms.  Just a simple mechanism to share the well established authentication, encryption cookie.   41 DLLs does not seem like a viable solution.  I'd like to believe that for the MS team, however, there is something that could be done.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 99307104-c0d7-3e82-f0b2-af61dcfe072a
Version Independent ID: 315c02cb-54c4-e386-da85-6e4311ca10c2
Content: Share authentication cookies among ASP.NET apps
Content Source: aspnetcore/security/cookie-sharing.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Thank you for contacting us. Due to a lack of activity on this discussion issue we're closing it in an effort to keep our backlog manageable. If you believe there is a concern which hasn't been addressed, please file a new issue.

I'll be honest in saying that we are really disappointed.  Having a simple way to share the authentication cookie between web forms and .NET Core (aka .NET now) would have made a significant difference in moving forward.  If we have to rewrite everything, including authentication, it is definitely a barrier to entry.


Introduce objects and subjects

Add the type system including objects.
Define subjects in the various protocol stages and rework the spec
accordingly.
Signed-off-by: Andrea Frittoli<EMAIL_ADDRESS>

This was discussed on today's CDEvents Vocabulary meeting. Outcome:
Proposed event structure, the CDEvents context part or data payload: "meta": { "version" : "draft", "event_id" : "A234-1234-1234", "source" : "/staging/tekton/", "type" : "dev.cdevents.taskrun.started", (dev.cdevent.. "timestamp" : "2018-04-05T17:31:00Z" }, "subject": { "id": "/namespace/taskrun-123", "type": "taskrun", "content": { "URL": ... "pipelinerun": ... } }, "extensions": [{ "name": <ext_name>, "content": ...} ], "links": [{ ... }]
The part in italics should not be added through this PR, but I just added it here as a reference since we discussed what further properties could be added on top level to the event structure later.
The reason for having so many sub objects in the structure is to simplify the SDK coding parts, and to have a clear json path extraction structure, like this: .data.
.<properties/content>.<content_properties>
Examples of such json paths would then be: event.data.subject.id event.data.subject.content.URL
cc: @afrittoli , @m-linner-ericsson , @erkist

Updated accordingly

@afrittoli I don't have any more comments currently


Tracking issues for updates for FRAME/Substrate for PBA5 Singapore

I am using this issue to track some changes to polkadot-sdk that would require updating content for the next PBA-round.

https://github.com/paritytech/polkadot-sdk/issues/2415
https://github.com/paritytech/polkadot-sdk/pull/3107

I think there is a chance in the Substrate module to really drive home some points that are set up in Game Theory and also in Blockchain. cc @jonasW3F @coax1d
In Game Theory they talk about the default effect. In blockchain we talk about upgrading networks by forking and how the default is to not change the rules at all. If you want to change anything, you can opt in to the change by updating your node.
In Substrate of course you present forkless runtime upgrades which is one of the killer features of Substrate. I think the point to drive home is that Substrate's forkless upgrade feature simply changes the default.
When using Substrate, the default is to follow the onchain governance whereas in the old world, the default is to keep the old rules. Even in Substrate you can of course still have consensus level forks. And if governance decides something you dislike, you still have the ability to "fork off" and keep the old rules. And you would do so by writing some new code and updating your node. Same as always.
Forkless runtimes just change the default
WDYT?


Allow upload file with same name

Hello, I want to upload file with same name. Please help me.  Thanks

download the new version and try the allowDiplicates option

Thanks you very much!


[Enhance] Pipeline function process points and masks simultaneously

Fix issue#440. When filtering/sampling points in pipeline, perform the same actions on masks too.

Maybe one potential question about ObjectSample in transforms_3d.py. If I understand correctly, this function sub-samples GTs so that we can train on a sub-class of original all classes. As it performs sampling, we also need to consider seg_mask here (currently these is only gt_bbox processing in this function). However, when I look into dbsampler and it seems impossible to fix this issue in a few lines of code. I tend to leave this function into future PRs.


Add Malay language

@conghc94 please address the conflict thanks.

@getneil Done, I just fixed the conflicts


Call to a member function toWebFormat()

Hey,
Hope you are doing good...
I have integrate the SendGrid PHP API Library and whenever I am going to call the Sendgrid from the Model then I got the following error,
Fatal error:  Call to a member function toWebFormat() on a non-object in
Can you please help me out?
Thanks,

It sounds like the $email variable you might be passing to $sendgrid->send($email); is not of the type SendGrid\Email, however it's a little difficult to diagnose your issue without a code snippet.

Hello @helloarpitgshah,
@Dreyer is correct, please provide us with a code snippet so we can further assist.
Thanks!

We have resolved the issues, Thanks for your response.

@helloarpitgshah awesome, thanks for the follow up!


removed second controller from home state.

Who put that in there?

This was definitely breaking stuff. :shipit:


fix(QueryManager): don't return undefined when the last result is empty

This change is

Reviewed 2 of 2 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


REDMD.md

Description

[ ] I have added a folder(Named it as my GitHub user id)
[ ] I have added a Readme.md file inside the folder
[ ] I understand the GitHub Forking Concept fully

j


Replace register_runtime_udf by register_runtime_extension_functions

On notebooks/omnisci-demo.ipynb, one uses register_runtime_udf but this function is deprecated:
 Using register_runtime_udf is deprecated, use register_runtime_extension_functions instead.

It is not easy to register_runtime_extension_functions explicitly because of the signature is not passed as a string (as is in the register_runtime_udf case) but as a dictionary of codes that contains the signature type information.
We can move the advance section from the notebook to keep it simple.

register_runtime_udf was removed from the notebook


2 auth modes problem

Hi,
It is a schema for a chat user. I want that the list of users is public (without private conversations).  Sorry to tell that but I think that multiple auth documentation is bad explained.
I do not understand why this does not work :
schema {
   query: Query
}

type Query {
   listUsers(): [User]
   @aws_api_key
}

type User
@model(subscriptions: { level: public })
@key(name: "userByUsername", fields: ["username"], queryField: "userByUsername")
@aws_cognito_user_pools
@aws_api_key {
  id: ID!
  username: String!
  online: Boolean
  conversations: [Conversation]
  @connection(name: "UserConversations")
  @aws_cognito_user_pools
  createdAt: String
  updatedAt: String
}

This schema is rejected. And how to use that with Angular ?

Hi @rudyhadoux
Can you tell what is the error you are getting when you said schema is rejected?
Also can you tell how did you setup your API so that I can reproduce this?

The schema failed after compilation.
I have an Api key and a Cognito user pool. And storage for other objects.

Hi @rudyhadoux
You can change your Type Query to either remove arguments from the listUsers like below:
type Query {
   listUsers: [User]
   @aws_api_key
}

or you can put some arguments in your query:
type Query {
   listUsers(id : ID!): [User]
   @aws_api_key
}

Let me know if this solve for issue.

I don't want to find a user(id) but only the public list of users.
I give you my whole schema and my Amplify configuration, so you can reproduce it and see bugs.
This schema below works and it is well compiled.
type User
@model(subscriptions: { level: public })
@key(name: "userByUsername", fields: ["username"], queryField: "userByUsername")
@aws_cognito_user_pools
@aws_api_key {
id: ID!
username: String!
online: Boolean
conversations: [Conversation]
@connection(name: "UserConversations")
@aws_cognito_user_pools
createdAt: String
updatedAt: String
}
type Conversation
@model(subscriptions: { level: public })
@auth(
rules: [
{
allow: owner
ownerField: "members"
operations: [create, update, delete, read]
}
]
) {
id: ID!
members: [String!]!
user: User @connection(name: "UserConversations")
messages: [Message] @connection(name: "ConversationMessages")
createdAt: String
updatedAt: String
}
type Message
@model(subscriptions: { level: public })
@auth(
rules: [
{
allow: owner
ownerField: "members"
operations: [create, update, delete, read]
}
]
) {
id: ID
members: [String!]!
conversation: Conversation! @connection(name: "ConversationMessages")
content: String
bucket: String
region: String
key: String
path: String
vu: Boolean
createdAt: String
updatedAt: String
}
BUT if I add this, to set listUsers( ) public
type Query {
listUsers(): [User]
@aws_api_key
}
I have this error message.
✖ An error occurred when pushing the resources to the cloud
Syntax Error: Expected Name, found )
An error occurred during the push operation: Syntax Error: Expected Name, found )


Please support edition 2021

Last week I tried to compile rhai into our script, but one of its denpendecies named smartstring requires edition 2021. 😓
rhai seems to be a good choice for integrating some simple DSL into scripts, so I hope you guys could upgrade the docker image to support edition 2021 in the new future.

What's your currently docker image used?
The latest capsule docker image is thewawar/ckb-capsule:2021-12-25 and it is built from the Dockerfile in current project. You can see the rustc version is nightly-2021-12-25, since edition 2021 support from stable rust 1.56.0, I think it's already supported in that docker image. You may change the config item here to use that docker image.


Add license

[x] Add CC0 license. I used gitignore's exact version.
[x] Add license and contribution info to readme
[ ] Add hpp-skel badge

cc @springmeyer @mapsam

@GretaCB here are the hpp-skel badges to use! Also included an hpp-skel README image with the dancing skel if you want to use it.

![readme image](https://mapbox.s3.amazonaws.com/cpp-assets/hpp-skel-readme_blue.png)

And the badge for others to use

[![badge](https://mapbox.s3.amazonaws.com/cpp-assets/hpp-skel-badge_blue.svg)](https://github.com/mapbox/hpp-skel)


Android build failing with error: Unresolved reference: uri

react-native: "^0.69.3",
react-native-file-access: "^2.4.4",
Platform: Android
Bug
> Task :react-native-file-access:compileDebugKotlin FAILED

error Failed to install the app. Make sure you have the Android development environment set up: https://reactnative.dev/docs/environment-setup.
Error: Command failed: ./gradlew app:installDebug -PreactNativeDevServerPort=8081
e: /home/shivam/ziki-tunes/music-app/node_modules/react-native-file-access/android/src/main/java/com/alpha0010/fs/FileAccessModule.kt: (10, 17): Unresolved reference: documentfile
e: /home/shivam/ziki-tunes/music-app/node_modules/react-native-file-access/android/src/main/java/com/alpha0010/fs/FileAccessModule.kt: (484, 19): Unresolved reference: DocumentFile
e: /home/shivam/ziki-tunes/music-app/node_modules/react-native-file-access/android/src/main/java/com/alpha0010/fs/FileAccessModule.kt: (494, 23): Unresolved reference: DocumentFile
e: /home/shivam/ziki-tunes/music-app/node_modules/react-native-file-access/android/src/main/java/com/alpha0010/fs/FileAccessModule.kt: (497, 75): Unresolved reference: uri

To reproduce
Install this library with latest react native version
Details
Not sure what else to put

Thanks for the report; known regression in v2.4.4.  I should have a fix out soon.  In the mean time, you can rollback v2.4.3.

Resolved in v2.4.5.


Automatic installation via Composer

When installing via composer, all lang folders should be automatically copied to app/lang so updates of the repo via composer also reflect any changes.

Auto update is not as good ides as it might look at first.
By design Laravel language files stores not only Laravel but user resources as well. Thus auto update for such files will lead to overwriting user strings.

Hello,
I started a new project to manage languages in a project

Is this project already on GitHub?

not yet

IMHO it should migrate into tools like transifex.com or similar
It supports arrays in array( ... => ...) notation but not [] and nested arrays which requires some pre/post handling however it gives all the goodies of tracking and translation reuse

Maybe it is possible to make a laravel package for the translations:
http://laravel.com/docs/master/packages#translations
Not sure if that would work but worth a try.

Arn't packages for additional translations? Laravel-lang is replacing core translations.

https://github.com/overtrue/laravel-lang :smile:

@overtrue exactly. That's what I am talking about. Awesome!

@kw-pr Thanks!


Set up Google Cloud Platform Postgres as a source in Airbyte

How can I connect to a GCP Postgres database as a Airbyte source?

What set up steps do I need to take?
What values do I use for the new Airbyte source configuration?  (and where do I get them?)

I specifically need this information for these

Airbyte Deployment locations

Local Airbyte deployment
GCP Compute Engine
On Kubernetes

Methods of authorizing to GCP Postgres

Cloud SQL Auth proxy
Self-managed SSL/TLS certificates
Authorized networks

It would be great if https://github.com/airbytehq/airbyte/blob/master/docs/integrations/sources/postgres.md could be updated with this information
Stretch
For the tutorial to be most helpful for the widest audience, it would probably be best if the tutorial showed this information for all of the other places Airbyte can be deployed to

Airbyte Cloud
On AWS (EC2)
On AWS ECS
On Azure(VM)
On Plural
On Oracle Cloud Infrastructure VM
On DigitalOcean (Droplet)

I ended up not being able to figure out how to do this locally.
⚠ THIS DOES NOT WORK ⚠
This was as far as I got but I couldn't figure out either the right host name or what I was doing wrong in the addition to docker-compose.yaml

In your airbyte repository, open your docker-compose.yaml file and add this at the bottom of the services top level section  proxy:
    container_name: cloud-sql-proxy
    image: gcr.io/cloudsql-docker/gce-proxy:1.30.0
    volumes:
      - ..\{{FULL_PATH_TO_SERVICE_ACCOUNT_JSON_FILE}}:/config
    ports:
      - <IP_ADDRESS>:5436:5436
    command: "/cloud_sql_proxy -instances={{INSTANCE_CONNECTION_NAME}}=tcp:5436 -credential_file=/config/{{SERVICE_ACCOUNT_JSON_FILE_NAME}}"

Follow the instructions in the Start Airbyte section above
Once Airbyte loads, in the lefthand menu click Sources
Click + add new source

Choose Postgres from the dropdown menu
Host: :question:
Port: 5436
Fill in other fields
Replication Method: Standard
SSH Tunnel Method: No Tunnel
Click Set up source

⚠ This also did not work ⚠
Run the Cloud SQL Auth proxy in a Docker container (per https://cloud.google.com/sql/docs/postgres/connect-admin-proxy#connecting-docker) as I couldn't figure out

Which Airbyte Docker container needed to connect to the Proxy Docker container
How to connect them

I did get the combo of GCP Compute Engine and Cloud SQL Auth Proxy working and have submitted a PR to update https://docs.airbyte.com/deploying-airbyte/on-gcp-compute-engine for that use case: https://github.com/airbytehq/airbyte/pull/12086


Fix windows dev env

Signed-off-by: Lauri Nevala<EMAIL_ADDRESS>

I second this PR - I had made very similar changes on my fork because I am starting to hack on this project. :-)

Thanks @ocdi 🙏
Btw, do you have any ideas what could be Windows equivalent for this line?
test -f out/main.js || make init

So test if out/main.js file exists and if not, then run make init ?

Btw, do you have any ideas what could be Windows equivalent for this line?
test -f out/main.js || make init

So test if out/main.js file exists and if not, then run make init ?

Maybe this helps? http://www.humbug.in/2012/makefile-check-if-a-file-exists-using-wildcard-function/


ng build --deploy-url "https://cdn.balabla.com" fail with unexpected token

Bug Report or Feature Request (mark with an x)
- [ ] bug report -> please search issues before submitting
- [ ] feature request

Versions.

@angular/cli: 1.4.3
node: 7.8.0
os: darwin x64
@angular/animations: 4.1.3
@angular/common: 4.1.3
@angular/compiler: 4.1.3
@angular/compiler-cli: 4.1.3
@angular/core: 4.1.3
@angular/forms: 4.1.3
@angular/http: 4.1.3
@angular/platform-browser: 4.1.3
@angular/platform-browser-dynamic: 4.1.3
@angular/platform-server: 4.1.3
@angular/router: 4.1.3
@angular/cli: 1.4.3
typescript: 2.5.2
Repro steps.

ng build --prod --deploy-url="https://cdn.blablabala.com"
The log given by the failure.

Uncaught SyntaxError: Unexpected token <  inline.082294f5457a019a4cd7.bundle.js:1
Uncaught ReferenceError: webpackJsonp is not defined
at polyfills.ac0bdd9514ff9092ada5.bundle.js:1   polyfills.ac0bdd9514ff9092ada5.bundle.js:1
Uncaught SyntaxError: Unexpected token <  vendor.2e243af5ebaa1a18c269.bundle.js:1
Uncaught SyntaxError: Unexpected token <  main.1f0af20d1fd500348a02.bundle.js:1
Desired functionality.

Mention any other details that might be useful.

@filipesilva we are experiencing the same thing.  Its interesting, just doing a build again most times resolves the issue.
polyfills.674b5ddae76fd2b73f01.bundle.js:1 Uncaught ReferenceError: webpackJsonp is not defined
    at polyfills.674b5ddae76fd2b73f01.bundle.js:1
(anonymous) @ polyfills.674b5ddae76fd2b73f01.bundle.js:1
main.b92620fddd4bc7634de0.bundle.js:1 Uncaught ReferenceError: webpackJsonp is not defined
    at main.b92620fddd4bc7634de0.bundle.js:1```

Thanks for reporting this issue. This issue is now obsolete due to changes in the recent releases. Please update to the most recent Angular CLI version.
If the problem persists after upgrading, please open a new issue, provide a simple repository reproducing the problem, and describe the difference between the expected and current behavior.


No logs to the console (develop branch)

Logging is only going to the log file(s).

Remove this line ?

I thought the current behaviour is that logging goes to both the log file, configured ^there, and the console. Apparently, the logrotate winston transport causes the console logging to vanish.

So it looks like winston-logrotate is doing evil things by clobbering the getters for stdout, stderr entirely.

Aww :cry:

Having changed that, things work, but it seems that this library also doesn't pass formatter so we can't have our logs looking the same. At this point it would probably be best to use the daily rotation logs provided by winston or roll a winston-logrotate thing ourselves.

Fixed by #93


Source/target pair text files; Icelandic parsing support; fixes

This PR adds text data generators that read source and target sentences in pairs, separated by tabs (\t), from a single source file.
It also adds scaffolding to support parsing of Icelandic text, either from characters or from subword tokens. (More coming later on that front.)
A few smaller fixes are included, such as replacing magic constants to do with reserved token ids with more meaningful identifiers, and getting rid of annoying  strings in decoder output trace messages.

My pleasure! :-)

I just found one small problem (source_vocab is used instead of target_vocab in wmt.tabbed_generator). I'll correct it in my next PR and release. It'd be great though to add an option to download the Icelandic parsing set instead of assuming it's there in a file. Is the set public?


arabic pdf

i am trying to generate pdf file in arabic :
var doc = new jsPDF();
doc.text(الحالة الدراسية", 35, 157", {align: "right", lang: 'ar'});
doc.output('datauri');

it gives :

i tried to add fonts but also didn't work :
doc.addFont("../views/home/Amiri.ttf" ,"Amiri" , "normal");
doc.setFont("Amiri");

change font.ttf to Base64 encode solved the problem
giftofspeed.com

I hope the developer of the program to solve the problem.

hello you said fixed by changed to Bas64 , can you link a working full example ?  with imports and where did you put the Base64 file?


feat: Add specific type to query key function

Status

READY
Description
The generated query key function would generate a function with an inferred return type (string | ParamType)[]. We can be more specific by defining [string, ParamType] instead. This is a small change, but more strict types improves the usability and cleanliness of code.
See: https://github.com/anymaniax/orval/issues/500
Todos

[ ] Tests
[x] Documentation
[ ] Changelog Entry (unreleased)

Steps to Test or Reproduce
Outline the steps to test or reproduce the PR here.

Update a sample

The query key function now has a strict return type of the following possible forms:
[ ] [string] when only the route is returned
[ ] [string, ParamsType] when the params are not optional
[ ] [string, ParamsType | undefined] when the params are optional

@anymaniax I see that I misunderstood your second comment. I also see that you fixed the issue, so I will close this


static_cast for develop branch.

Hi,
Here is the static_cast patch for develop branch.
Cheers!
Colin

Hi,
I have reviewed and merged your pull request into the "develop" branch. Note that I have just removed the line adding the build/ directory into the .gitignore file. This is related to your personal configuration (building inside the source folder) and I think this should not appear in the .gitignore file.
Again, thanks a lot for this pull request.


fix: release of v2

I guess this is applicable now, right?

Ignore my comment! I didn't notice the tag 🙈


License lacking Dependencies and batch error

Dear verhas,
The first issue I found is the lack of version 1.0.8 in the central repository. Which also seems not exist in the project.
The second, which I am having struggles with, is the error to find/load the main class. Either with package "com.javax0.license3j.License3j" or simply "License3j". I am using java 8 and the Windows .bat  file.
Moreover, I used the branch jdk8 and the license3j-1.0.7 tag with no success. Hope you can help, thank you.

1.0.8 was not released yet. That is the reason why you can not find it in the central repo. It is consistent with your finding that it is not in the project. When you look at the documentation and see that it talks about the version 1.0.8 you are looking at the documentation of the development version preparing for the next release.
The version in the branch jdk8 has the main class in the package com.javax0.license3j. The windows BAT file defines this class on the line 161.


Bug "Managed Debugging Assistant 'RaceOnRCWCleanup'"

This bug appear when closing the program while refreshing the hardware info

This exception appears to be thrown when the refresh thread is running. Possible fix would be to stop the thread before allowing the program to close.


added projects to test drools bundles installed into OSGi environments.

DROOLS-80. For while only tests for bundle installation was added.

I am closing this old PR. The OSGi related integration was moved to https://github.com/jboss-integration/fuse-bxms-integ


Make SourceKind a required parameter

Summary
This PR refactors the check_path function to make SourceKind a required parameter i.e., remove the Option<...>. The main blocker for this change was that add_noqa wasn't supported for Jupyter Notebooks but it's still fine to pass in the SourceKind without it. This is ok because the PySourceType is checked before and we don't proceed unless it's either a Python or stub file.
This is also required for the PEP 701 f-string parser changes because we want the source code to be available while parsing to extract the leading and trailing text for debug expressions. Here, SourceKind will provide that but it was an Option<...> before.

Current dependencies on/for this PR:

main

PR #7013   👈

This comment was auto-generated by Graphite.

(Closing and re-opening the PR to trigger CI)


Undeploy does not remove lustrefilesystems crd

Running nnf-deploy undeploy does not remove the lustrefilesystems crd, which seems to stay around for some amount of time before eventually disappearing.

Also, attempting to delete it manually does not work.

After reporting the issue, Nate asked me for the
[corbett8@hetchy12:Flux]$ kubectl get pv -A
NAME                                  CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                                                STORAGECLASS    REASON   AGE
kern-nnf-dm-system-readwritemany-pv   1          RWX            Retain           Released   nnf-dm-system/kern-nnf-dm-system-readwritemany-pvc   nnf-lustre-fs            57d
[corbett8@hetchy12:Flux]$ kubectl get pv -A -oyaml
apiVersion: v1
items:
- apiVersion: v1
  kind: PersistentVolume
  metadata:
    creationTimestamp: "2022-12-08T18:12:12Z"
    finalizers:
    - kubernetes.io/pv-protection
    name: kern-nnf-dm-system-readwritemany-pv
    resourceVersion: "38615543"
    uid: 4ae2b0b9-adcc-4993-868d-c67a4dfc9909
  spec:
    accessModes:
    - ReadWriteMany
    capacity:
      storage: "1"
    claimRef:
      apiVersion: v1
      kind: PersistentVolumeClaim
      name: kern-nnf-dm-system-readwritemany-pvc
      namespace: nnf-dm-system
      resourceVersion: "35486532"
      uid: 43818781-1923-4b53-9566-8f02368ccc8c
    csi:
      driver: lustre-csi.hpe.com
      fsType: lustre
      volumeHandle: 7@kfi:/lslide
    persistentVolumeReclaimPolicy: Retain
    storageClassName: nnf-lustre-fs
    volumeMode: Filesystem
  status:
    phase: Released
kind: List
metadata:
  resourceVersion: ""

(38T4) [corbett8@hetchy12:nnf-deploy]$ kubectl get pvc -A
No resources found

Furthermore, in the current state of hetchy there is now no lustrefilesystems CRD even though we deployed cleanly:
[corbett8@hetchy12:Flux]$ kubectl get lustrefilesystems
error: the server doesn't have a resource type "lustrefilesystems"

Please see the newly added (undeploy instructions)[https://github.com/NearNodeFlash/nnf-deploy#undeploy]. If you continue to have issues please re-open the ticket


Support React as a template engine

Yes, you've read well. And no, I'm not drunk. Yet 😄
React has proven it's not just made for HTML generation. You can find multiple projects using it for different purposes, like Ink or React Native. So why not code generation?
Reasons why we should consider it

It’s markup in JS not JS (or logic) in markup. We’re always creating a lot of basic filters because we’re on the markup language by default. Same problem appears when using Angular or Vue (not sure latest versions, I think they changed something wrt this). If we choose something like React, we could easily import any npm package or do basic stuff like typeof myvar === 'object' without having to create a filter.
Composability and reusability. If templates (and its parts) are React components, they can be reused in other templates too. Think, for instance, we have a POJO template and we want to use the POJO template in many Java code generators. Of course, the POJO template can use our future feature for model generation but it can actually improve it with some other stuff that may not be possible or available. Like generating other files that operate on the POJO, for instance. I’ve been thinking a lot how cool would it be to use templates inside templates. Same way you use Node.js packages in your Node.js packages.
We can benefit from tooling for rendering static React markup. In the end, it’s no different from what Gatsby and Next.js are doing. We just sometimes output HTML and sometimes Markdown, code, etc. which are all strings of text and perfectly doable with React.
We can benefit from React Fast Refresh. While developing a template, you would instantly have the generated result after saving. Way faster than what we have right now and, in the case of developing HTML templates, preserving the state and refreshing without reloading the page.
We get a more precise error context than we do now with Nunjucks. If you're a template developer, you know how difficult could it be to guess where the error is exactly located. With React, you get precise information in the CLI and in the browser (if your template is for the browser). See the example below.
And last but not least, React is a stable project with a huge community while Nunjucks is pretty much stuck in adding new features. I think it’s due to the fact that they want to be compatible with Python’s Jinja2.

Example of how I imagine it
Please note this is a draft. Things ~may~ will for sure change. Read it as a way to understand what I'm thinking about.
The following is a translation of the index.js file you can find here: https://github.com/asyncapi/nodejs-template/blob/master/template/src/api/index.js
import { convertToFilename } from '@asyncapi/generator/sdk'
import { File, Line } from '@asyncapi/generator/react-sdk'
import { capitalize, camelCase } from 'lodash'

export default function IndexFile({ asyncapi, params }) {
  const protocol = asyncapi.server(params.server).protocol()
  const configKey = protocol === 'ws' ? 'ws' : `broker.${protocol}`

  return (
    <File>
      const Hermes = require('hermesjs');
      const app = new Hermes();
      const {cyan, gray} = require('colors/safe');
      const buffer2string = require('./middlewares/buffer2string');
      const string2json = require('./middlewares/string2json');
      const json2string = require('./middlewares/json2string');
      const logger = require('./middlewares/logger');
      const errorLogger = require('./middlewares/error-logger');
      const config = require('../lib/config');
      const { capitalize(protocol) }Adapter = require('hermesjs-{protocol}');
      {
        Object.entries(asyncapi.channels()).map(([channelName, channel], index) => (
            <Line key={index}>
              const { camelCase(channelName) } = require('./routes/{ convertToFilename(channelName) }.js');
            </Line>
          )
        )
      }

      app.addAdapter({ capitalize(protocol) }Adapter, config.{configKey});

      app.use(buffer2string);
      app.use(string2json);
      app.use(logger);

      // Channels
      {
        Object.entries(asyncapi.channels()).map(([channelName, channel], index) => (
          <Line key={index}>
            { channel.hasPublish() && `app.use(${ camelCase(channelName) });` }
            { channel.hasSubscribe() && `app.useOutbound(${ camelCase(channelName) });` }
          </Line>
        ))
      }
      app.use(errorLogger);
      app.useOutbound(logger);
      app.useOutbound(json2string);

      app
        .listen()
        .then((adapters) => {
          console.log(cyan.underline(`$\{config.app.name\} $\{config.app.version\}`), gray('is ready!'), '\n');
          adapters.forEach(adapter => {
            console.log('🔗 ', adapter.name(), gray('is connected!'));
          });
        })
        .catch(console.error);
    </File>
  )
}

Examples of error information
On the markup

On the JS code

And of course the Stack trace too

I'm template developer, and with Nunjucks i could write template on the Java with slight influence of Nunjucks filters. I like this feature, because it's my primary language and if template user would like to have a look on the code, it will be understandable for the user.
Regarding "Fast Fresh" will be work with hooks? Especially with hooks which change file names.

Interesting... I'm literally sold just because of "We get a more precise error context than we do now with Nunjucks" and then keeping everything in javascript 😄 Are you thinking complete replacement or just another thing to support? 🤔

@Tenischev I understand the friction. The problem is that, as templates get more and more complex, Nunjucks sucks in readability. Actually, all the React files would start with something like:
import { convertToFilename } from '@asyncapi/generator/sdk'
import { File, Line } from '@asyncapi/generator/react-sdk'
import { capitalize, camelCase } from 'lodash'

export default function MyFile({ asyncapi, params }) {
  return (
    <File>
     ...

So when reading, even if you don't understand much about it, you can pretty much ignore it as it's boilerplate code that's very similar everywhere. In any case, don't be afraid of React, it's JS with HTML inside 😄 You'll be fine with it in 20 minutes or less.
@jonaslagoni Initially, I'd do it as a separate engine we support. Depending on how we this working we might decide to keep both or just go for one. Both approaches have pros and cons. Definitely not something we have to decide now 😄

Another thing I didn't mention: since we now would have a top-level <File> tag, we can put meta-information about the file there, like its name, permissions, etc. Something you'd do with hooks now could be easily done here like this:
// File: $$schema$$.java

...
import { generateCamelCaseSchemaName } from '@asyncapi/generator/sdk'

export default function SchemaFile({ schema }) {
  return (
    <File name={generateCamelCaseSchemaName(schema)}>
      ... (your Java code here)
    </File>
  )
}

This approach cuts down filters and hooks usage by a lot.

I vote for having it as an experimental additional template engine which we can slowly worm up and see where it can go, or just get rid of it after all if we are not happy

I vote to move this feature along. I am starting to get really frustrated with Nunjucks it takes so long time to find the bugs the generator are reporting with --debug flag or without... It takes longer and longer to create more and more complicated templates.
Example I am currently getting the following error:
Something went wrong:
Template render error: (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\<EMAIL_ADDRESS>[Line 66, Column 5]
  unknown block tag: endif
    at Object._prettifyError (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\lib.js:36:11)
    at C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:561:19
    at eval (eval at _compile (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:20:11)
    at C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:611:9
    at eval (eval at _compile (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:60:12)
    at Template.getExported (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:599:16)
    at eval (eval at _compile (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:59:6)
    at createTemplate (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:313:9)
    at handle (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:325:11)
    at C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:337:9
    at next (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\lib.js:326:7)
    at Object.asyncIter (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\lib.js:332:3)
    at Environment.getTemplate (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:319:9)
    at eval (eval at _compile (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:57:5)
    at C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:613:9
    at Template.root [as rootRenderFunc] (eval at _compile (C:\Users\Lagoni\AppData\Roaming\npm\node_modules\@asyncapi\generator\node_modules\nunjucks\src\environment.js:631:18), <anonymous>:41:1)

Where the file $$channel$$.cs.njk only contains 33 lines of code so the error is neck deep in one of the macros, which might go as deep 3 atm..

This is now fully supported check out the docs https://github.com/asyncapi/generator/blob/master/docs/authoring.md