Support for plotting distribution with different number of samples

Hello,
I haven't found a way to produce boxplots where boxes describe populations with different sample sizes. As far as I understood, the problem is that numpy arrays do not support non-rectangular data.
E.g. with standard matplotlib it is possible to plot:
data = [ (2,3,4), (1,2), (1,2,3,4,5,6,7) ]
plt.boxplot(data)

As far as I know, there is no way to do the same with proplot.
My only workaround has been to define a function that converts a list of lists in a rectangular array with size (NxM), where M are the number of boxes, and N is the maximum length across the list elements. For each column, the residual elements are filled with np.nan. This seems to work because proplot filters the bad values when computing the statistics. This is a draft of the function:
def convert_to_proplot(list_of_lists):
    maxval = max([len(elem) for elem in list_of_lists])
    out = np.empty((maxval, len(list_of_lists)))
    out[:] = np.nan
    for i,elem in enumerate(list_of_lists):
        out[:len(elem),i] = elem
    return out

Would it be possible to have proplot handle non-rectangular data?
Alternatively, would it be possible to have within the proplot package a function similar to the one above to quickly convert non-rectangular data to rectangular data?
Thank you in advance.
Enrico

Hi Enrico, this should already be supported in both proplot/matplotlib. It's kind of weird to wrap your head around, but basically numpy supports defining "ragged" nested arrays where each object is itself a 1D array. In your example, the idea would be to define a 3-element array, each of which contains a tuple from your list. Works as follows:
data = np.array([(2, 3, 4), (1, 2), (1, 2, 3, 4, 5, 6, 7)], dtype=object)
fig, ax = pplt.subplots()
ax.boxplot(data)

Also simply passing your nested list to ax.boxplot() should work without any warnings, but may raise a warning (may try to suppress this warning in future versions):
/Users/ldavis/mambaforge/lib/python3.10/site-packages/numpy/core/shape_base.py:65:
Creating an ndarray from ragged nested sequences (which is a list-or-tuple
of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated.
If you meant to do this, you must specify 'dtype=object' when creating the ndarray.

Thanks for the answer. I tried the code you provided:
data = np.array([(2, 3, 4), (1, 2), (1, 2, 3, 4, 5, 6, 7)], dtype=object)
fig, ax = pplt.subplots()
ax.boxplot(data)

but it yields the error:
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Is it a problem only on my side?

Hmm I think you might be using an older version? What does print(pplt.__version__) show? You can install the "dev" version (includes newest/unreleased changes) using pip install git+https://github.com/proplot-dev/proplot.git.

my version was 0.9.7. Now I installed the dev version and it shows that it's version 0.9.5.post358 (is it expected that the version number is lower in a dev install?). When I run the same code, now I get an exception from function_base.py of numpy:
_4559 def _lerp(a, b, t, out=None):
4560     """
4561     Compute the linear interpolation weighted by gamma on each point of
4562     two same shape array.
(...)
4571         Output array.
4572     """
-> 4573     diff_b_a = subtract(b, a)
4574     # asanyarray is a stop-gap until gh-13105
4575     lerp_interpolation = asanyarray(add(a, diff_b_a * t, out=out))
TypeError: unsupported operand type(s) for -: 'tuple' and 'tuple'_

Hello, I am re-opening the issue because I tested your code:
data = np.array([(2, 3, 4), (1, 2), (1, 2, 3, 4, 5, 6, 7)], dtype=object)
fig, ax = pplt.subplots()
ax.boxplot(data)

and I get an error that seems related to the masked_invalid function that proplot calls to mask invalid values.
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[471], line 3
      1 data = np.array([(2, 3, 4), (1, 2), (1, 2, 3, 4, 5, 6, 7)], dtype=object)
      2 fig, ax = pplt.subplots()
----> 3 ax.boxplot(data)

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/proplot/internals/process.py:284, in _preprocess_args.<locals>.decorator.<locals>._redirect_or_standardize(self, *args, **kwargs)
    281             ureg.setup_matplotlib(True)
    283 # Call main function
--> 284 return func(self, *args, **kwargs)

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/proplot/axes/plot.py:3618, in PlotAxes.boxplot(self, *args, **kwargs)
   3614 """
   3615 %(plot.boxplot)s
   3616 """
   3617 kwargs = _parse_vert(default_vert=True, **kwargs)
-> 3618 return self._apply_boxplot(*args, **kwargs)

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/proplot/axes/plot.py:3551, in PlotAxes._apply_boxplot(self, x, y, mean, means, vert, fill, filled, marker, markersize, **kwargs)
   3549 if means:
   3550     kw['showmeans'] = kw['meanline'] = True
-> 3551 y = process._dist_clean(y)
   3552 artists = self._plot_native('boxplot', y, vert=vert, **kw)
   3553 artists = artists or {}  # necessary?

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/proplot/internals/process.py:298, in _dist_clean(distribution)
    296 if distribution.ndim == 1:
    297     distribution = distribution[:, None]
--> 298 distribution, units = _to_masked_array(distribution)  # no copy needed
    299 distribution = tuple(
    300     distribution[..., i].compressed() for i in range(distribution.shape[-1])
    301 )
    302 if units is not None:

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/proplot/internals/process.py:143, in _to_masked_array(data, copy)
    141 if ndarray is not Quantity and isinstance(data, Quantity):
    142     data, units = data.magnitude, data.units
--> 143 data = ma.masked_invalid(data, copy=copy)
    144 if np.issubdtype(data.dtype, int):
    145     data = data.astype(float)

File ~/.conda/envs/polvcopd2/lib/python3.9/site-packages/numpy/ma/core.py:2360, in masked_invalid(a, copy)
   2333 """
   2334 Mask an array where invalid values occur (NaNs or infs).
   2335
   (...)
   2357
   2358 """
   2359 a = np.array(a, copy=False, subok=True)
-> 2360 res = masked_where(~(np.isfinite(a)), a, copy=copy)
   2361 # masked_invalid previously never returned nomask as a mask and doing so
   2362 # threw off matplotlib (gh-22842).  So use shrink=False:
   2363 if res._mask is nomask:

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

While this wouldn't be a proplot issue per se, as far as I understood the problems comes from masked_invalid using the numpy's isfinite function, which expects a homogeneous array. I found some threads where people had this problem with np.isfinite(), and the suggestions were essentially to cast the array to float (see for example here). But in ragged arrays this can't be done, and I've read somewhere that masking ragged arrays is not well supported because they are tricky.
Would there be a workaround to this issue? Alternatively, what is a version of numpy that works as expected?
I am on the latest numpy (1.26) and proplot (0.9.7) versions.
Thank you very much.


Filter out candidates with the same name but different instructions, …

…file IDs, and function names

Why are these changes needed?
Filter out candidates with the same name but different instructions, file IDs, and function names.

Related issue number
Refer https://github.com/microsoft/autogen/issues/892#

Checks

[x] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally.
[x] I've added tests (if relevant) corresponding to the changes introduced in this PR.
[ ] I've made sure all auto checks have passed.

Codecov Report
Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (8ea6377) 26.60% compared to head (e794e6b) 17.21%.

Files
Patch %
Lines

autogen/agentchat/contrib/gpt_assistant_agent.py
0.00%
28 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #925      +/-   ##
==========================================
- Coverage   26.60%   17.21%   -9.39%
==========================================
  Files          28       28
  Lines        3733     3758      +25
  Branches      847      855       +8
==========================================
- Hits          993      647     -346
- Misses       2667     3033     +366
- Partials       73       78       +5

Flag
Coverage Δ

unittests
17.21% <3.44%> (-9.34%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@gagb @sonichi PTAL

@gagb @sonichi PTAL

See this PR:
https://github.com/IANTHEREAL/autogen/pull/2
Otherwise looks good


App crashes when opening the app

ITS-PRO time category: https://its-pro.ucsd.edu/browse/MA-34
Mu UCSD app crashes everytime I open it, especially when the Finals/Class Schedule Cards start loading. This may be due to the fact that I enrolled in summer session 1 classes and then dropped them.

Verified, closing


Support for Solidity 0.8.2 and 0.8.3

Changes

[x] Bumped development dependency versions: semver and eslint plugin-related ones.
[x] Added support for Solidity 0.8.2 and Solidity 0.8.3 releases on compile level.
[x] Added support for Solidity 0.8.2 and Solidity 0.8.3 releases on AST level:

Introduced documentation for Statement descendants.
Adopted changes from #22 as they are part of the Solidity 0.8.2 and Solidity 0.8.3 releases.
The legacy AST reading is adusted just to not crash. It is only in maintenance mode until further notifications.

[x] Tweaked AST-to-source writing components to respect the changes.
[x] Tweaked ASTNodeFactory to respect the changes.
[x] Tweaked related test suites and samples.

Notes

This PR causes backward incompatibility.

Related links

Solidity 0.8.1 release notes
Solidity 0.8.1 release blog post
Solidity 0.8.2 release notes
Solidity 0.8.2 release blog post
Solidity 0.8.3 release notes
Solidity 0.8.3 release blog post

Regards.

Codecov Report

Merging #25 (d1894cb) into master (0bafe5c) will increase coverage by 0.48%.
The diff coverage is 96.77%.

@@            Coverage Diff             @@
##           master      #25      +/-   ##
==========================================
+ Coverage   93.50%   93.99%   +0.48%
==========================================
  Files         199      207       +8
  Lines        3802     3878      +76
  Branches      499      498       -1
==========================================
+ Hits         3555     3645      +90
+ Misses        159      148      -11
+ Partials       88       85       -3

Impacted Files
Coverage Δ

...implementation/declaration/variable_declaration.ts
93.10% <ø> (ø)

src/ast/modern/throw_processor.ts
40.00% <40.00%> (ø)

src/ast/ast_node_factory.ts
69.39% <66.66%> (+4.62%)
:arrow_up:

src/ast/implementation/statement/block.ts
100.00% <100.00%> (ø)

...ast/implementation/statement/do_while_statement.ts
100.00% <100.00%> (ø)

src/ast/implementation/statement/emit_statement.ts
100.00% <100.00%> (ø)

...t/implementation/statement/expression_statement.ts
100.00% <100.00%> (ø)

src/ast/implementation/statement/for_statement.ts
100.00% <100.00%> (ø)

src/ast/implementation/statement/if_statement.ts
100.00% <100.00%> (ø)

...rc/ast/implementation/statement/inline_assembly.ts
100.00% <100.00%> (ø)

... and 57 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0bafe5c...d1894cb. Read the comment docs.


a question about updating centroids

hi ~ I have a question about updating centroids in your code as follows:
        # computation of the means
        if sp.issparse(X):
            centers = _k_means._centers_sparse(X, labels, n_clusters,
                                               distances)
        else:
            centers = _k_means._centers_dense(X, labels, n_clusters, distances)

        # l2-normalize centers (this is the main contibution here)
        centers = normalize(centers)

When using cosine similarity in clustering, if you just normalize the centers calculated  with _k_means._centers_XXX, which were designed to update centers when using eu distance, won't the result centers have different directions from what they should be?
Hope I've describe my question clearly and looking forward to your reply~ Thanks~

This was meant to only be used with euclidean distance (I suppose the option to use a different metric is one of the pitfalls of only patching on a change to sklearn's existing kmeans utility).
I'm not sure that normalizing will change directions (as we are just scaling by positive scalars), can you provide an example? Its possible that the directions of the cluster centers obtained using cosine-similarity vs. euclidean will have the opposite directions (for the same initialization) due to the relationship below.
Cosine similarity has a 1-1 relationship to the euclidean dist when the the vectors are unit-normalized (https://en.wikipedia.org/wiki/Cosine_similarity#Properties):

Hopefully that helps!

Thanks a lot for answering me, and sorry I didn't describe my question properly.
When we calculate the center of A, B and C in fig 1 with euclidean distance, the result will be D. So after L2 normalization, the new center will be point B. However, when we calculate center of A and B first, the result will be E and after calculating center of E and C, the result will be somewhere like F.  What's more, when calculating B and C first, the result will be different from F.
On the other hand, when A, B and C are points in fig 2, the center will be O, and how can we normalize it(although it can hardly happen)?
So what do you think about this issue?
Thanks again for answering~

Regarding Figure 1
I believe that your geometry is basically correct so perhaps I can clarify how the algorithm is setup to reduce confusion.
A key part of k-means and its variants is picking the parameter k, the number of clusters a priori.  The algorithm repeats in two phases from there:

assign each data point to a cluster center
recompute cluster centers based on data-points with the same cluster assignment

In your first calculation that inspects the center of A, B, C (resulting initially in roughly D which normalizes to roughly B) there is an implicit assumption that k=1 or that we are only evaluating one set of points that are all assigned to the same cluster (e.g., say the first iteration of k-means with random cluster assignments, k > 1 but we're just examining one of the assignment sets).  In the case where k = 1, we're done the cluster center becomes B, which seems like a reasonable point on the circle to me.  In the other case, we still obtain B as a cluster center, however in the next iteration of the algorithm, its unlikely that C will be assigned to this cluster center because it is likely closer to a different center due to its location.
In your second calculation that inspects the center of A and B  (resulting in Eafter normalization), the key point is that normalization will not happen before averaging in C.  It is true that if you normalize before averaging in C, the result will be different (this is because a/b + c/d ~= (a + b) / (c+d). So either in this scenario k > 1 and C is not included in this cluster assignment, or if C is included then it will only ever be averaged in before normalization as in your first calculation.  Either way, normalize(normalize(A + B) + C) and normalize(normalize(B + C) + A) will not happen in the algorithm.
Regarding Figure 2
Yes, it seems this can happen in the following cases:

k = 1 and all points are organized as you depict; however, if you only care about finding the center of all points, you may want to handle this computation manually and handle this edge case.

k >1 and an initialization of cluster assignments gets unlucky (unlikely but possible); assuming the first pass of the algorithm succeeds, it doesn't seem to me that this case can happen when k > 1 in subsequent iterations.  k-means is often initialized by random cluster assignments which could introduce this edge case, with low probability (requires both at least one subset of the data to have 0-mean and for the random cluster assignment to choose that subset);  increase k will lower this probability. Since k-means is often run multiple times and the best result is chosen, one could handle an exception cause by normalizing by zero, if it did come up.  I think by default we use k-means++ to initialize this algorithm which tried to spread out the cluster centers (in euclidean space), which should also reduce the likelihood of this case from happening.


Add Color Helper CSS Classes

I imagine helper classes for text colors could be very useful.
What if you don't want a heading not to be in primary color or with class "subheader" in light-gray but in your secondary or medium-gray?
For Example: Create for each color:
.text-color-#{$color-name} {
  color: {$color-value} !Important;
}

so you can do the following:

Some heading in secondary color

A very medium day and text

Since 6.1.1, you can override the $foundation-colors map to add / modify any color you want.
Then you can easily iterates through this map to create your own classes like this:
@each $name, $color in $foundation-colors {
  .text-color-#{$name} { color:$color !important; }
}

However, I highly recommend you to avoid using the !important flag when you can.
You could instead use the same design Zurb is using for many components in Foundation (buttons, callouts, badges...etc.) and avoid using !important:
@each $name, $color in $foundation-colors {
  h1.#{$name} { color:$color; }
}

Then you can use colored h1 titles using the same syntax than with other Foundation components:

I'm a green title

I'm a yellow title

I'm a red title

We try to keep our suite of utility classes slim, so text color classes aren't of interest to us right now.


Error handling in DB check loop

Do we really need this set -e call? I'm not sure on what systems the bash script will continue to the line 10, but for me, as per bash documentation, the script exits if first test (L10) exists with code 1.
https://github.com/erroneousboat/docker-django/blob/4b9ee750776933c690e18074ce02981b88057290/webapp/config/start.sh#L9-L15
Instead, I think we can keep using set -e and simplify the script with something like:
set -e
until python3 ./database_check.py; do
    sleep 5; echo "*** Waiting for postgres container ..."
done
echo "*** Postgres container available..."
set +e

Thank you for your suggestion, and I've implemented it.


KEDA documentation around number of replicas supported for metrics server is contradictory

Under https://keda.sh/docs/2.14/operate/cluster/#high-availability, there is a table like so:

For the "Metrics Server" deployment, the "Support Replicas" column has the value "1", which reads as only one replica is supported. However, the "Note" column mentions that "You can run multiple replicas of our metrics sever", which indicates that multiple replicas are supported.
It should be clarified exactly how many replicas are supported and perhaps the language for the "Support Replicas" column should be updated.

"You can run multiple replicas of our metrics sever", which indicates that multiple replicas are supported.

It is not because you can, that it is supported :)
Metric server only has 1 replica that will really do anything, if you add more they are just in standby mode until the other one goes down. But @zroubalik / @JorTurFer can verify my thoughts

"You can run multiple replicas of our metrics sever", which indicates that multiple replicas are supported.

It is not because you can, that it is supported :)
Metric server only has 1 replica that will really do anything, if you add more they are just in standby mode until the other one goes down. But @zroubalik / @JorTurFer can verify my thoughts

Correct, unless you add the cli flag mentioned in the screenshot.

I see, so it's essentially the same situation as the operator, where only one replica does work and the other is a standby? In that case, why would the "Support Replicas" value for the operator be 2?

As for the operator, having more replicas improves the HA but they won't balance the load. I mean, if you have 2 replicas for each component, only one of each will work (operator runs with leader election and metrics server is called reusing the socket by the control plane) but in case of disruption where you lose a node and the new pods can't be scheduled, the second replica (already scheduled, up and running) can take the control and keep the system running

Thanks all! I actually tried deploying two metrics-server pods without setting enable-aggregator-routing=true. I observed that both metrics-server pods have a log line like the following:
I0712 19:42:37.026404       1 trace.go:205] Trace[364897128]: "List" url:/apis/external.metrics.k8s.io/v1beta1/namespaces/devx-keda-poc/s0-prometheus-devx-test,user-agent:kube-controller-manager/v1.22.17 (linux/amd64) kubernetes/a7736ea/system:serviceaccount:kube-system:horizontal-pod-autoscaler,audit-id:7fd1c8f2-46ca-4ef1-b67d-d598a47b00e6,client:<IP_ADDRESS>,accept:application/vnd.kubernetes.protobuf, */*,protocol:HTTP/2.0 (12-Jul-2024 19:42:36.173) (total time: 852ms):
Trace[364897128]: ---"Listing from storage done" 852ms (19:42:37.026)
Trace[364897128]: [852.824475ms] [852.824475ms] END

This makes me think that both metrics-server pods are doing useful work even without enable-aggregator-routing=true. Additionally, I did a kubectl get leases in the keda namespace, and there's only a lease for the keda-operator, not the keda-operator-metrics-server. Does the metrics-server actually do leader election as well?
% kg leases -n keda
NAME               HOLDER                                                                AGE
operator.keda.sh   keda-operator-68c4f4fb76-c2nft_70ad8b8f-c463-4371-9190-08aebf193774   22d

We're running keda 2.8 on K8s 1.22.

Metrics server doesn't have leader election, all the instances are functional. It's the control plane who calls to a single instance based on the cli flag. I don't have information about how different providers configure this, so maybe it's enabled in your clusters

Yup!


Mutable dict used as default arg?

https://github.com/mjumbewu/django-rest-framework-csv/blob/2ff49cff4b81827f3f450fd7d56827c9671c5140/rest_framework_csv/renderers.py#L197

It's true -- technically bad form. However, that dict does not get modified anywhere (though it could). I would accept a pull request to fix!


Performance issues with TP and PP settings

When I use 4 A10s and use PCIe to connect, I can only set TP=4 and PP=1 when I accelerate inference. If I use tp=2 and pp=2, the inference speed decreases significantly.This is an anti-intuitive phenomenon, generally speaking, when using the PCIe bus, the use of PP can reduce the communication consumption, and the model segmentation based on the Decoder structure is very neat, and its computational complexity should be consistent. It was observed that only the first GPU occupancy could reach 95%, and the subsequent GPU occupancy was very low. Ask if there may be some parameter setting issues, or if there are other possibilities

Could you provide the commit you are using?

Could you provide the commit you are using?

I use the release version tags: [v0.9.0] with commit ID: [250d9c2],  Is the new version updated with features related to parallel computing?

Yes, there used to be a known performance issue of PP + IFB (v0.9.0 has this issue). The issue has been fixed since the commit a96cccafcf6365c128f004f779160951f8c0801c in the main branch. Can you try to use the latest main branch and see whether the problem can be solved?

Thank you, I will try testing on the latest main branch and provide you with the test results later


Birth Death processes...?

Assume I simulate individual organisms.  In that case I don't want the effect of them to be calculated. I now do that by
function f(i,j,d2,d)
	d[i]+= u[i]>0.0 ? exp(-d2*dc)*u[j]*(u[j]/u[i])^ac : 0.0
	return d
end

in the function called by CellListMap.map_pairwise! which speeds it up somewhat. But can I also keep a list of organisms which are dead and make CellListMap.map_pairwise! just skip the call to the function altogether? When I have a birth of an organism, I take one dead organism and revive it with the new genetic makeup (traits) by setting its mass>0.
So the short question is, can I somehow keep a list of which organisms are alive without having to redo the entire SArray of positions for every death/birth event?

Just had a thought, could one do it as a view of a SArray?
xpositions = rand(SVector{2,Float64},10)

ids=[2,5,6,7]
system = PeriodicSystem(
           xpositions = view(xpositions,ids),
           unitcell=[1.0,1.0],
           cutoff = 0.1,
           output = 0.0,
           output_name = :organism
       );

(late now, will test tomorrow if you haven't commented yet :-) )

That won't really bring you any benefit, because the coordinates are internally copied anyway to make them local to each computation.
The simplest approach is to simply put the live individuals at the beginning of the array and resize it. Partition the array in live/death is a relatively cheap operation (actually there is an internal function in CellListMap that can almost do that, here is a variation of it:
julia> function keep_alive!(alive, x::AbstractVector)
           iswap = 1
           @inbounds for i in eachindex(alive,x)
               if alive[i]
                   if iswap != i
                       x[iswap], x[i] = x[i], x[iswap]
                       alive[iswap], alive[i] = alive[i], alive[iswap]
                   end
                   iswap += 1
               end
           end
           return resize!(x,iswap - 1), resize!(alive,iswap - 1)
       end
keep_alive! (generic function with 1 method)

julia> x = rand(SVector{2,Float64}, 100); alive = rand(Bool, 100);

julia> keep_alive!(alive, x)
(SVector{2, Float64}[[0.29119093606094626, 0.14799284331418305], [0.4972730380281174, 0.09269265678798855], [0.9159907295858599, 0.023425334184397073], [0.6070998442833461, 0.5991916701778053], [0.3160812299339907, 0.3164821161774142], [0.7439114803685659, 0.11559108939285456], [0.6670382934847978, 0.9839740944475307], [0.657358664500978, 0.32044636378412406], [0.6082878186504177, 0.50068291214486], [0.08662133450538845, 0.4684000312786505]  …  [0.3151220417919539, 0.4605952237829716], [0.2875938461476413, 0.5834502287377114], [0.030323370160795338, 0.6514632971028673], [0.11688254221953154, 0.6127507999882679], [0.44709649959706943, 0.41760496956657467], [0.7726898156341978, 0.6743612022838364], [0.189673060868509, 0.2515943452830405], [0.6597804541954473, 0.028298360650823362], [0.002315181581732384, 0.30165618551226736], [0.9019060110665619, 0.9487908898003208]], Bool[1, 1, 1, 1, 1, 1, 1, 1, 1, 1  …  1, 1, 1, 1, 1, 1, 1, 1, 1, 1])

Then, for example, let us say that you initialized the system with the original coordinates:
julia> x = rand(SVector{2,Float64}, 100); alive = rand(Bool, 100);

julia> sys = PeriodicSystem(xpositions=x,unitcell=[1,1],cutoff=0.05,output=0.0)
PeriodicSystem1{output} of dimension 2, composed of:
    Box{OrthorhombicCell, 2}
      unit cell matrix = [ 1.0, 0.0; 0.0, 1.0 ]
      cutoff = 0.05
      number of computing cells on each dimension = [22, 22]
      computing cell sizes = [0.05, 0.05] (lcell: 1)
      Total number of cells = 484
    CellList{2, Float64}
      100 real particles.
      87 cells with real particles.
      123 particles in computing box, including images.
    Parallelization auxiliary data set for:
      Number of batches for cell list construction: 1
      Number of batches for function mapping: 1
    Type of output variable (output): Float64

Note that the system has 100 real particles.
If you now "kill" the individuals:
julia> keep_alive!(alive, x);

The next time map_pairwise! is called the system will be updated for the new set of coordinates:
julia> PeriodicSystems.map_pairwise!((x,y,i,j,d2,out) -> out += d2, sys)
0.00416479370378711

julia> sys
PeriodicSystem1{output} of dimension 2, composed of:
    Box{OrthorhombicCell, 2}
      unit cell matrix = [ 1.0, 0.0; 0.0, 1.0 ]
      cutoff = 0.05
      number of computing cells on each dimension = [22, 22]
      computing cell sizes = [0.05, 0.05] (lcell: 1)
      Total number of cells = 484
    CellList{2, Float64}
      45 real particles.
      41 cells with real particles.
      55 particles in computing box, including images.
    Parallelization auxiliary data set for:
      Number of batches for cell list construction: 1
      Number of batches for function mapping: 1
    Type of output variable (output): Float64

(note now that there are 45 real particles, the ones that were alive).
Note that that partition and resizing is very, very cheap, and won't affect the overall performance of almost any simulation:
julia> @btime keep_alive!($alive, $x);
  55.408 ns (0 allocations: 0 bytes)

julia> @btime PeriodicSystems.map_pairwise!((x,y,i,j,d2,out) -> out += d2, $sys)
  7.318 μs (19 allocations: 1.45 KiB)
0.00416479370378711

Great explanation,thank you very much. I think I can adapt this to birth events too. Enjoying the package, very efficient!


Pinot Long Term Data Store

We have certain use-cases wherein we would like to move older data to cheap object stores like the S3 and store only the most recent data in Pinot. One such use-case is storing distributed trace data - Our query patterns show that more than 90% of the queries lie in the last 24 hours. Having said that, we have a retention period of 30 days. So in this case, we would like to keep only the last 24 hours worth of data in Pinot, and move the rest to a cheap store like the S3.
From what I concluded from our initial discussions, this would involve work on two fronts:

Moving older data from Pinot to the S3 (this already happens, mentioning it for completeness).
Pinot-Presto connector, so that it can query from both Pinot and the S3 based on the time span of the query.

Here is a rough sketch of the design we came up with in our discussion the other day (this is relevant to the distributed tracing example I gave above):

I have created this issue to get the discussion started.
Thanks!

@kishoreg please let me know your thoughts, thanks!

Useful feature.
Another way to do this may be to evict the segments from pinot-servers, but load them on-demand when a query comes in. Mark something in the IDEALSTATE to indicate that the segment is not there, so that the broker is aware (this may or may not be needed).

Yes @mcvsubbu let us evaluate the pros and cons of each of these approaches. I mentioned this one specifically as work has already started on the connector side I think.

Yes @mcvsubbu let us evaluate the pros and cons of each of these approaches. I mentioned this one specifically as work has already started on the connector side I think. If this is done on-demand though, will it not add to the response times as the entire segment might have to be loaded from S3 to Pinot?

True, but something worth considering

Also if lower granularity for older data (>24h) is an option, maybe introducing roll-up can compact older data which lead to lower cost of having pinot servers. In that case, even for older data, query latency will be sub-seconds.

@sajjad-moradi Our trace data has very high cardinality dimensions(trace_id, span_id, etc,.), so compaction won't give us much benefits here(roll-up ratio would be very poor). It would have helped if we are only storing aggregates.

This makes sense and something worth doing. We need two things

enhance tiered storage concept to move segments older than X to S3.
either enhance presto Pinot connector to directly load data from S3 or enhance Pinot to lazy load segments from S3 on demand. Later option is probably better? Note that the latency will be higher when we lazy load a segment.

Thoughts?

Whether we do it from connector or lazy load in Pinot the latency will be the same. Lazy load might be better if there is ssd one so let of no PoC  as l cache on Pinot servers.

@kishoreg just a thought - Can we do something that doesn't require us to download the complete segment from the S3? Some way of looking "inside" a segment and download only a part of it.

possible but it will be a huge undertaking. we can probably get there in small steps.

I would vote for lazy load in pinot as a first step. I will be glad to review the design and PRs  if you can contribute.

@mcvsubbu Will come up with a design proposal for this soon.

@mcvsubbu @kishoreg I am thinking along the lines of what @mcvsubbu proposed earlier. Introduce a new state like "EVICTED" in the segment FSM. A segment can move from ONLINE -> EVICTED (this transition can take place when a segment's retention period expires). The invariant is that an evicted segment is always present in the deep store. While querying, the broker gets all of the segments to be queried, sees that some of them have been evicted, triggers a download to a sub-set of servers, and then proceeds with its regular business. The segments remain cached in these servers, and can be replaced with some eviction policy if need be. What are your thoughts on this? (I through it will be a good idea to vet this approach before I get on with the design).

If you are changing the state machine, it is best to draw a picture and float it. We can then look at each state transition. This has to be done in a compatible manner in Helix. @kishoreg  can comment more on whether that is easy to do.
Can we even do this without modifying the state machine?
Say, the query lands on the server and the server finds that it does not have the segment. It can then hold the query until downloading the segment from deep store (a lazy download), and return the response to the broker like before. The only thing that needs to be done here is to have a larger time out to allow for the segment to be downloaded.
The table config can set some criteria for segment availability in servers (say, upto 5d recent, or upto 5d after push in refresh use cases). It can also set some retention time (say. 15m).
Such code complexity will also need to be evaluated against memory map on a fast local store (cloud costs for this may be high).  You can memory-map 3TB worth segments off an SSD onto (say) 64G of memory, and the OS will pretty much keep these segments in disk all the time .. until a query needs it. And then it will be paged out in a timely manner.  We have done this successfully in LinkedIn for several years now.
Have you considered this option, or will that still be more expensive than actually lazy-loading the segment from deep store?

Thanks @mcvsubbu for the review and the pointers. As for the FSM, this is the new state I proposed:

The reason I introduced a new state is because it looked like a cleaner way for the broker to determine that a segment has been purged and has been moved to the deep-store. Certainly, the broker can also determine this by first determining that the segment is absent, and then looking at its S3 location in the segment config. Just that the code will be a bit less clean in this case. With the new state, I was thinking of defining an invariant that a segment moves to this state iff it was successfully uploaded AND its URL was successfully updated in its metadata (so the broker can be sure that the segment was actually uploaded just by looking at the new state, can be helpful in case when the deep-store was bypassed during commit for some reason). But on second thoughts it appears this adds unnecessary complexity.
We haven't really done a cost comparison of lazy-loading vs. mmap on local Pinot servers. But I'll throw some numbers here. Our ingestion rate is increasing quite rapidly and we're looking at around 4-5T/day of data in the next few months (these are conservative numbers). With a retention period of 30 days (again min.), we'll have to store 150T worth of segments on SSDs at any time. Storage costs can be prohibitive with this much data. Not to mention all of this to serve a tiny amount of queries (< 10%) We'll try to do a proper cost analysis of this today.

I was just proposing what worked for us in terms of cost. Your mileage may vary, of course.
In terms of state machine, we need to transition out of "Moved" state to ONLINE and OFFLINE states as well (what if a moved segment is deleted)
Lastly, let us ask ourselves why the broker needs to know that a segment has been moved.

+1 for tiered storage. We (@elonazoulay ) had very similar ideas on the Presto-Pinot part (let it scan segments directly and offload real-time serving part).

@suddendust this is a pretty big undertaking. @noahprince22  had attempted this before. https://github.com/apache/pinot/issues/6187

From your initial diagram, which path are we trying to solve? Presto -> Pinot -> s3 or Presto -> pinot-connector -> s3

@mcvsubbu @kishoreg here's the updated FSM. I had not marked any transitions out-of MOVED earlier.

We are trying to go with Presto -> Pinot -> S3 (no changes to the connector atm).
I haven't really given a thought to how this design would behave when there are too many segments to load. With 2T/day and 500M segments, we have around 4000 segments per day. With a retention of 30 days, we are looking at 4000 * 30 = 120,000 segments. If someone makes a query that literally queries data for last 30 days, we might have to load all of them (yikes!). May be this can be controlled with a max segment config, as has been done in this PR?
I guess we can avoid much of the complexity by not touching the FSM? Essentially offloading all the download business to the servers itself - They determine they don't have the segment and trigger a download, an respond some time later asynchronously.

Lastly, let us ask ourselves why the broker needs to know that a segment has been moved.

@mcvsubbu I gave it some thought and looks like it doesn't. It is the responsibility of the servers to furnish the segment - Either local or from the deep store. The broker just needs to query them and wait, as it is doing right now.
This implementation does seem to come with a fair bit of warning - Too many segments to download, chatty cluster due to too many state transitions, long download times, etc.
I am sure I am missing some minute but important details here, so would appreciate some expert guidance. Thanks :)

OK, so we have some conclusions here, let me know if this sounds right.
Happy path:

The broker does not need to know whether a segment is in memory or not. So, maybe we don't need to change the Segment State machine, after all (let me know if you think otherwise)
When the server gets a request, it needs to handle the case that the segment is not in its own hard disk. So, we need to have the server initiate the load and wait for the segment to load before executing the query.

We will need to change the table config to indicate a few things:

When will segments be "unload"ed? (num hours after create/update, maybe one criteria.  Total number of loaded segments may be another criteria -- we will unload older ones as the newer ones come in. Maybe there are other ways to specify this as well)
Maybe we should limit the number of queries that use deep store segments within a certain time frame. Limiting the overall number of loaded segments  may also help, since the other queries may indicate partial results.

Not-so-happy-path:

Server may fail the download or take too long that the query has already timed out.
Servers may be asked to download too many segments, causing the server to thrash. We can think of a few solutions here. Maybe limit the number of old segments each server will contain. Any more than that, either we return partial or the server ousts one of the segments and downloads another, increasing query latency.

@kishoreg , does something like this seem a reasonable path to lead to a usable feature?

I haven't really given a thought to how this design would behave when there are too many segments to load. With 2T/day and 500M segments, we have around 4000 segments per day. With a retention of 30 days, we are looking at 4000 * 30 = 120,000 segments. If someone makes a query that literally queries data for last 30 days, we might have to load all of them (yikes!). May be this can be controlled with a max segment config, as has been done in this PR?

Possibly two things to consider here:

I am wondering if data in S3 needs to have the same granularity as the data in Pinot or can we aggregate the data to a higher-level dimension while aging it out to S3? For example, if data in the latest segment has a granularity of 1 second, then data in a segment 10 days old may have a granularity of 10 seconds (thereby reducing the data size by factor of 10), and data 30 days old may have a granularity of 1 hour (thereby reducing the data size by a factor of ~4000).
Also, would adding a segment cache between Pinot and S3 help with latency? Usually I would expect some sort of a locality of reference when we pull in data from S3

I haven't really given a thought to how this design would behave when there are too many segments to load. With 2T/day and 500M segments, we have around 4000 segments per day. With a retention of 30 days, we are looking at 4000 * 30 = 120,000 segments. If someone makes a query that literally queries data for last 30 days, we might have to load all of them (yikes!). May be this can be controlled with a max segment config, as has been done in this PR?

Possibly two things to consider here:

I am wondering if data in S3 needs to have the same granularity as the data in Pinot or can we aggregate the data to a higher-level dimension while aging it out to S3? For example, if data in the latest segment has a granularity of 1 second, then data in a segment 10 days old may have a granularity of 10 seconds (thereby reducing the data size by factor of 10), and data 30 days old may have a granularity of 1 hour (thereby reducing the data size by a factor of ~4000).
This is a good point but I consider it more of optimization and should not be designed for this. There are cases where users would not want to compress the granularity. Moreover, one of the main reasoning here is s3 is cheaper and users want to keep the data for long.

Also, would adding a segment cache between Pinot and S3 help with latency? Usually I would expect some sort of a locality of reference when we pull in data from S3
This is definitely possible and something to consider once we have the ability to load a segment when we receive the query

@kishoreg , does something like this seem a reasonable path to lead to a usable feature?

I don't know. This needs more thought. We took a similar approach last time when Noah Prince wanted to solve it but we abandoned it after hitting a lot of obstacles.  should we start a proposal doc this time?

I'll start with the proposal doc for this today.

@kishoreg @mcvsubbu Here is the design proposal on the feature. One section is still in todo (evaluating the performance implications of this feature).

@mcvsubbu @kishoreg any thoughts of this?

Been a very busy week, I will take a look hopefully this week

@mcvsubbu

The broker does not need to know whether a segment is in memory or not. So, maybe we don't need to change the Segment State machine, after all (let me know if you think otherwise)

Broker timeout may be one of the factors for broker to know about the status of the segments, since it takes time for servers to download and load stale segments into memory.

@suddendust changing the state machine is not what we agreed on. It is a LOT easier to keep the state machine as it is, and pull the segment from deepstore when needed.

@mcvsubbu We're not changing the SM. I put it there just because it was discussed here. I'll state this more clearly in the doc or remove the section altogether.

Good. always nice to have the rejected alternatives. Just add them as appendix. thanks. I will go through it again.

I will also review this in detail. The best way to implement this would be to either implement a new implementation of existing interfaces like segment fetcher or enhance/add interfaces. That way this can be implemented as a plugin in the first version and once it's tested in production, we can make the plugin available by default.
Let's try to see if we can achieve this without touching pinot-core.

@mcvsubbu thanks for the thorough review of the proposal. There are some gaps in it that I am working on right now. Will resolve them soon as ask you for a re-review. Thanks again :)

Please also add a link o https://docs.google.com/document/d/1Z4FLg3ezHpqvc6zhy0jR6Wi2OL8wLO_lRC6aLkskFgs/edit#heading=h.sbtdm9byxqnc in the design document, and comment on how your solution may work with or complement tiered storage, if we build that.

Thanks @mcvsubbu, have done a thorough update of the proposal. I have a question though that I couldn't figure out a solution to. When servers download/evict segments, what would be the best way to update the segment idealstate? Similarly for rebalancing from broker for even distribution of frozen segments. Certainly servers shouldn't be doing this and the controller can't get to know about them.

@suddendust our current design is to NOT change idealstate. Please conform that this is indeed the case. This is so precisely for the reasons you mention.
One way to let the controller know is via some API, but we will need to manage multiple replicas doing this same thing, etc. We can get to this later. Let us do one version in which the idealstate is not touched.

The segments should be in ONLINE state in idealstate.  A server may find it not in local store, and download it before serving the query.

This makes the design quite simpler as I don't have to deal with the vagaries of state transitions. I'll update the doc with this (my thought process was that an unmapped segment should be moved to OFFLINE as an invariant. I was doing this solely to maintain adhere to this).

@npawar Any updates on this? Thanks!


Remove Global State From Client And Add Benchmarks

This is two parts.

Remove the global connection state from the client. It was only required for event dispatch to echo and update functions. I've managed to register a small function as the RPC handler for each, which in turn dispatches to an OvsdbClient scoped echo and update function instead.

Add a set of tests that benchmark the update method of the client

Updates: #80

Coverage decreased (-0.3%) to 76.674% when pulling 0e10d7ff72e973c2b71872d43dddfe998d3efad1 on dave-tucker:client into 177adf8aee1fac2e7bf25c6651f0bcef0484e61d on socketplane:main.

Running the benchmarks locally show the following:
$ go test -run=XXX -bench=. -cpuprofile=bench-update.out
goos: linux
goarch: amd64
pkg: github.com/socketplane/libovsdb
cpu: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
BenchmarkUpdate1-8         24507             56607 ns/op
BenchmarkUpdate2-8         17331             62400 ns/op
BenchmarkUpdate3-8         15979             78947 ns/op
BenchmarkUpdate5-8         11407            102405 ns/op
BenchmarkUpdate8-8          7744            148025 ns/op
PASS
ok      github.com/socketplane/libovsdb 10.177s

Pull Request Test Coverage Report for Build 771754867

26 of 31 (83.87%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.3%) to 76.674%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

client.go
26
31
83.87%

Files with Coverage Reduction
New Missed Lines
%

client.go
1
78.9%

Totals

Change from base Build 771641894:
-0.3%

Covered Lines:
664

Relevant Lines:
866

💛  - Coveralls

LGTM


2.0.0-beta.2 doesn't work with<EMAIL_ADDRESS>

Bug report
Describe the bug
the tag 2.0.0-beta.2 doesn't work using<EMAIL_ADDRESS>The panel admin doesn't build.
Expected behavior
I would be able to use the admin panel with the last release candidate.
Screenshots

System

Node.js version: 20.15.0
NPM version: 10.8.1
Strapi version: 5.0.0-rc.1
Plugin version: 2.0.0-beta.2
Database: sqlite
Operating system: macOs

Additional context

Hi @jordanernult,
Thanks for reporting this bug!
I’ve resolved the issue in PR https://github.com/pluginpal/strapi-plugin-config-sync/pull/130.
Will release a new beta version shortly.

Hi @boazpoolman,
Great job, I was missing the addition of config-sync in renderAdmin function, in order to do the PR.

This was released with version v2.0.0-beta.3.
Please report a new issue if you experience any other problems.


Buffer overflow in the native windows PDB.exe binary

Describe the bug
There is a buffer overflow in the native windows PDB.exe binary due to an unchecked call to strncpy in pdb/cpp/util.cpp
To Reproduce
Steps to reproduce the behavior:

Run the pdb.exe binary in the debugger of your choice with the following arguments: fakefilename.pdb AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA- 0
Observe the STATUS_STACK_BUFFER_OVERRUN (0xC0000409) exception thrown by the stack cookie check.

Expected behavior
An error to be thrown if the string to be copied is larger than the stack buffer size (256).
Environment (please complete the following information):

OS: Windows 10
Java Version: 11.02
Ghidra Version: 9.0.2

Additional context
I was not going to submit this originally because I don't believe it to be actually exploitable, but I figured that somebody else should decide on that.
I don't think this is exploitable through loading a malicious pdb into Ghidra, as the GUID and signature are stored as the GUID and int types respectively before being passed to pdb.exe, effectively limiting their values, lengths, and formats.

Maybe not exploitable, but that's not the first buffer overflow we've found in this code (and likely not the last).  This code needs to be scrubbed, if not replaced.


Need bsField name when spreading fragments

Migrating from 0.2.7 from 0.2.8 creates errors when sprearding fragments:
  80 ┆ edges @bsRecord {
  81 ┆   node @bsRecord {
  82 ┆     ...Document
  83 ┆   }
  84 ┆ }

  You must use @bsField(name: "fieldName") to use fragment spreads

This is really strange because I've seen a test for this:
module ExternalFragmentQuery = [%graphql {|
  fragment Fragment on VariousScalars @bsRecord {
    string
    int
  }
   {
    variousScalars {
      ...Fragment
    }
  }
|}];

Not sure what is the difference between my code and the test.

This sounds like a regression in 0.2.8 unfortunately :( I'll add tests for fragments in interfaces.

Temp dirty runtime hotfix for anyone struggling with this...
let bsFieldFix =
  Js.String.replaceByRe(
    Js_re.fromStringWithFlags("@bsField(.*)", ~flags="g"),
    "",
  );

module GQL = ...actual graphql query...

module GQLFix = {
    include GQL;
    let query = bsFieldFix(GQL.query);
    let ppx_printed_query = bsFieldFix(GQL.ppx_printed_query);
  };

  module Query = ReasonApollo.CreateQuery(GQLFix);

:(


Add config file

What? and Why?

What does this pull request change/fix? Why was it necessary?

Move all service config to app config file. Should make config easier to maintain.
Checklist

[x] CHANGELOG.md updated? (if required)

Codecov Report

Merging #45 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #45   +/-   ##
=======================================
  Coverage   84.59%   84.59%
=======================================
  Files           6        6
  Lines         331      331
=======================================
  Hits          280      280
  Misses         51       51

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f72260...9e87fff. Read the comment docs.

 Continue to create config.ini files for SDX services


two notebooks to illustrate some features of resampling

This adds two very simple Jupyter notebooks that demonstrate some facilities of the library.

This adds two very simple Jupyter notebooks that demonstrate some facilities of the library.

@HDembinski How do you feel about checking in code-only notebooks? That way the diff is minimized if / when someone makes changes. Or do you think it's best to do a clean run so that some useful output is already there?

Taking a quick glance at them, if we're to use them for communicating features it might be helpful to break them up into multiple cells and include markdown descriptions between so that they read more like prose

Taking a quick glance at them, if we're to use them for communicating features it might be helpful to break them up into multiple cells and include markdown descriptions between so that they read more like prose

Feel free to improve them as you see fit. I didn't spend a lot of time writing them, consider these as stubs.

This adds two very simple Jupyter notebooks that demonstrate some facilities of the library.

@HDembinski How do you feel about checking in code-only notebooks? That way the diff is minimized if / when someone makes changes. Or do you think it's best to do a clean run so that some useful output is already there?

I am not sure what you mean, so I apologize in advance if I am not answering on piont. The notebooks should serve as little tutorials on specific topics, they are supposed to be educational and to advertise the library. So they should not only contain code, but also prose. Diffing them is difficult, as is diffing documentation in general.

We need to add some unit tests for the notebook, to make sure they continue to work as we make changes.
I use this script in iminuit to test my notebooks. These not tests for correctness of the output (which is guaranteed by the actual unit tests), but tests that change in the interface did not accidentally break the notebooks.
https://github.com/scikit-hep/iminuit/blob/develop/tutorial/test_notebooks.py

I am not sure what you mean, so I apologize in advance if I am not answering on piont. The notebooks should serve as little tutorials on specific topics, they are supposed to be educational and to show-case the library. So they should not only contain code, but also prose. Diffing them is difficult, as is diffing documentation in general.

I suppose instead of "code-only" I more meant "no output" (so restarting a clearing cells before committing). That way we're only version controlling code and markdown content (and not metadata associated with a notebook run).

The idea is to have the notebooks in the repo so that people can look at them and the cell output.

The idea is to have the notebooks in the repo so that people can look at them and the cell output.

Yeah, having them be viewable from the GitHub UI seems more important

@dsaxton Can this be merged now? We can always improve the notebooks later.

@dsaxton Can this be merged now? We can always improve the notebooks later.

My preference would probably be to include notebooks that are more or less complete tutorials (like the old one that was recently deleted: https://github.com/dsaxton/resample/blob/caa95e7b399b87752004dcffda3cd0e2a4e91c99/examples/resample.ipynb) rather short "in progress" ones. How about including the notebook from the scikit-hep presentation (or a variation thereof), which seemed pretty comprehensive?

My preference would probably be to include notebooks that are more or less complete tutorials

I think having something is better than having nothing. We can always improve these resources.

(like the old one that was recently deleted: https://github.com/dsaxton/resample/blob/caa95e7b399b87752004dcffda3cd0e2a4e91c99/examples/resample.ipynb)

Good point, we should restore that as a notebook.

How about including the notebook from the scikit-hep presentation (or a variation thereof), which seemed pretty comprehensive?

I was about to ask you about that, I think it makes sense to add. The talk is a bit long and detailed for a tutorial, but like I said, having something is better than having nothing at this point. We can replace these resources if we have better ones.

I guess we can merge them, although we should try to update soon with more helpful tutorials (FYI I renamed the folder to notesbooks, fixed a typo in the text, and did a clean rerun)


JWM complains about missing libpng12.so.0 lib

Hi,
JWM has been cruising along fine until this morning.  First indication of a problem was when windows that should not have borders and title based on .jwmrc settings did.  I then tried doing jwm -v in a terminal and the following comes up:
jwm: error while loading shared libraries: libpng12.so.0: cannot open shared object file: No such file or directory
There were a few packages updated last night, which I list below.
I uninstalled JWM, did make clean, ./configure..., make then make install as root and the problem still persists.    Looks like the git pull I did messed something up.  It's no longer complaining about libpng 12.  Sorry for the noise...again


Create SubChapter of Embedding Deno

Related to denoland/manual#15
It is important to have an easy-to-use explanation of all of deno's crates and what features they provide.
It makes it easier to build custom operators and pick and choose features to create the perfect Embedded Runtime.

Should check out the linked pull request to bring over into the new docs repo

@NinekoTheCat It would be helpful if you could reopen the PR in this repository 🙏


Incorrect documentation for ExnihiloCreatio

In documentation if you go Mods -> Ex Nihilo Creatio -> Sieving -> Removal
there is mods.exnihilocreation.Sieve.removeAll(); - >
exnihilocreation while should be exnihilocreatio(without last n)

Just found out same mistake in Crucible heat sources(adding and removing recipes),crooking removing line include same mistake

Dang it


Support linting Stalark packages

Background & motivation
https://github.com/kurtosis-tech/eth-network-package/pull/54
https://github.com/kurtosis-tech/eth2-package/pull/109
We don't have liniting
Desired behaviour
We want something like
kurtosis lint .
kurtosis lint /path/to/file.star
How important is this to you?
Painful; the lack of this feature makes using Kurtosis frictionful.

@mieubrisse Agreed on the above!

[ ] convert asser to verify PR
[ ] release the docker version In Progress
[ ] support pooling + release the KT version


Error when logging in through the TrueCraft launcher

Operating system: Debian Stretch 64-bit (testing)
I successfully built TrueCraft, then when logging in through the launcher, I get this:
System.NotImplementedException: The requested feature is not implemented.
  at System.Net.WebRequest.CreateHttp (System.String requestUriString) [0x00000] in <filename unknown>:0
  at TrueCraft.Launcher.Views.LoginView.LogInButton_Clicked (System.Object sender, System.EventArgs e) [0x00000] in <filename unknown>:0
  at Xwt.Button.OnClicked (System.EventArgs e) [0x00000] in <filename unknown>:0
  at Xwt.Button+WidgetBackendHost.OnClicked () [0x00000] in <filename unknown>:0
  at Xwt.GtkBackend.ButtonBackend.<HandleWidgetClicked>m__0 () [0x00000] in <filename unknown>:0
  at Xwt.Toolkit.Invoke (System.Action a) [0x00000] in <filename unknown>:0

Attempting to launch the client directly also returns an error:
Unhandled Exception:
System.IndexOutOfRangeException: Array index is out of range.
  at TrueCraft.Client.Program.Main_Thread (System.String[] args) [0x00000] in <filename unknown>:0
  at TrueCraft.Client.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0
[ERROR] FATAL UNHANDLED EXCEPTION: System.IndexOutOfRangeException: Array index is out of range.
  at TrueCraft.Client.Program.Main_Thread (System.String[] args) [0x00000] in <filename unknown>:0
  at TrueCraft.Client.Program.Main (System.String[] args) [0x00000] in <filename unknown>:0

When it comes to the second one (just Client) as parameters you have to provide
TrueCraft.Client host[:port] username

Failing to provide username throws this exception.

How do you play singleplayer through the client?

You need a newer version of Mono.


Key IDs should not be calculated from sha256(cjson(encoding(pubkey)))

From the spec:

The KEYID of a key is the hexdigest of the SHA-256 hash of the canonical JSON form of the key.

For future flexibility, I think key IDs should be calculated as sha256(pub_key_bytes). Right now the spec relies on the usage of CJSON and also a permanent fixed encoding of the public key.

I'm adding another note to this where we have had problems between PEM encoding RSA keys with one lib using PKCS#1 and one using SPKI giving us different strings / bytes for the same prime / exponent in the public key.
For a point of comparison, does anyone know exactly what GPG does for calculating the fingerprints of RSA keys?

I just ran into an issue similar to this here. So I thought it'd be worth bumping.

For a point of comparison, does anyone know exactly what GPG does for calculating the fingerprints of RSA keys?

This is specified in rfc4880 and we have implemented for in-toto here. The process is basically the hash over:

the byte \x99 (still a mystery to me)
length of the public key packet
hash of all the MPI's of the pubkey packet

The 0x99 (0b_10011001) octet is used in place of the packet's actual tag octet (rfc2440 actually used the wording: "A V4 fingerprint is the 160-bit SHA-1 hash of the one-octet Packet Tag").
They probably chose that particular octet as one that wouldn't conflict with any existing packet tags (v4  packet tags can't have bit 6 be 0 iirc), is still a valid packet tag with leading 1 bit that wouldn't break old programs (impossible to mistake for an MPI too), and it's also maybe a possibility the the bit pattern has some intetesting mathmatical property being a palindrome.
Last one is just a nerdy guess of course, haha.

There's a draft TAP for keyid flexibility https://github.com/theupdateframework/taps/blob/master/tap12.md

I'm closing this as it does not seem to be a python-tuf issue: we do not choose keyids, securesystemslib does. Please open a securesystemslib issue if this is still valid
I think I agree with tap12: advantages of complex keyid generation algorithms do not seem at all clear to me when we could just require that keyids must be unique in that metadata only.


New RdTest mode for very large variants

This PR introduces a new RdTest genotyping mode intended to be applied to very large variants.
RdTest currently genotypes events larger than 1MB by only looking at depth information from the bins covering the middle 1MB of the event. In some cases, especially for very large events, the middle 1MB can fall into regions of the genome for which we do not collect coverage (eg centromeres) which can cause the depth to appear to be zero and make every sample genotype as a homozygous deletion. In other case it may cause incorrect results (eg if the middle of the event is covered by segmental duplications or a nested CNV). This behavior is currently controlled by the -s, --sizefilter parameter.
This change adds a new genotyping method that avoids this problem. In the new method, RdTest loads depth from m windows of size n with centers that are evenly spaced across the event. Based on testing I have set the number of windows m to 500 and the window size n to 2000 -- this results in 1MB of depth data being loaded as before. For each window the median depth is calculated. The resulting medians are then used as inputs to the regular RdTest algorithms. Events over a given size threshold (set to 2.5MB by default) can be genotyped according to this method. This behavior can be controlled by the -x, --verylargeeventsize, --verylargevariantpoints, and --verylargevariantwindows parameters.
Note that I have tried to be conservative about changing RdTest in other scenarios. This means that the old "middle megabase" method is still used for events between 1MB and 2.5MB in size.
I've tested this method on a number of events and data from the gnomad and SSC cohorts. I've also run the single sample pipeline through with this version (I think this mode was triggered a few times).
One note is that loading depth data from these windows is slower than loading a contiguous 1MB chunk of data. We should assess how long it takes in a full run on a large test cohort batch to make sure that the performance is still acceptable (I don't have my exact timing data available anymore but I believe that genotyping a large event took a few times longer in a gnomad batch than using the previous method).
This PR also includes a few other changes to RdTest, including renaming the sample include and exclude lists, refactoring some chunks of code into methods for readability, and adding automatic failure on warnings and printing of tracebacks in case of errors.

@cwhelan looks great! Have you done a test run with inputs/build/ref_panel_1kg/test/GATKSVPipelineBatch/GATKSVPipelineBatch.FromSampleEvidence.json? I'd want to see that that runs ok and look at the cleaned vcf.

I've completed a run with inputs/build/ref_panel_1kg/test/GATKSVPipelineBatch/GATKSVPipelineBatch.FromSampleEvidence.json
The clean vcf is here: gs://broad-methods-cromwell-exec-bucket-instance-8/GATKSVPipelineBatch/ba3adf13-b406-4577-84b4-9e03644b7ff0/call-MakeCohortVcf/MakeCohortVcf/7c7a73d5-15dc-44b9-8545-0fb104d57672/call-CleanVcf/CleanVcf/a780c30e-6d96-4c49-828d-b77cc3475ab0/call-ConcatCleanedVcfs/ref_panel_1kg.cleaned.vcf.gz
So far by scouring the logs I've found a few sites where the new genotyping mode was invoked:
chr18:68537528-80263001 in GenotypeDepthPart2 / RdTestGenotypeOver5kb
chr14:80522636-106847714 in TrainRDGenotyping
chr6:153431038-160785696 in CPX genotyping
There may be more, I'll keep looking through the logs.


Toward 0.3.0

add networked-share to getNetworkOwner
make remote template instantiations monkey-patch the root element
support falsy selector to mean this.el
(networked-share) automate default networkId and owner
add my quick-n-dirty ball-dropping example (plaid-polo)
add useShare option, to use networked-share for remote entities
add ball-room and ball-room-4 examples

Gave it a brief read through. Lgtm.


Add report/unreport to post & comment, discussion needed

Describe your changes
Made xplat functions addReport, removeReport, reportedBy (to check if given user has reported the content).
Reports are stored in a new Firebase collection.
I didn't make a new type for reports but can change that if it's a big deal.
@JakeSteinebronn @tylerhm @abrugal  For auto-hiding content after 3 reports I'm gonna add a boolean "hiddenByReports". Do want to filter out that content when fetching or client-side? I think it should be a front-end check.
Issue ticket number and link
Finishes some sub-tasks for #105
Next up: auto-hide content, employee actions.

for auto-hiding, we can have the report transaction also increment a counter on every post like numReports (and when a report is resolved, it decrements this count)
Then, front-end can just (JUST) check if that number is gequal to some const, like 3 or 5 or whatever.
What do you think

That would work, the report list is currently a field on every post & comment so front-end can check the size of the list.
It saves us from storing an extra bool on every post/comment, but is probably a bit more annoying to use. I think I'd rather keep an updated bool

Well you wouldn't ever need the bool, you could have your checkShouldBeHidden just do the count / length check. I also think we don't need to maintain the actual array of the reports, just the reports themselves

I moved the redundant functions to User, but didn't opt to support reporting a User itself, it'll keep reporting simpler for employees because they won't see a reported user then have to dig through all their content to find why they're reported.
added checkShouldBeHidden() to Post and Comment
also simplified the code for the functions by checking if they have data & getting data at the top of the functions

We can get to this later, but user reporting should be implemented if their avatar, displayname, or bio is inappropriate

oh true I'll add that next then

Do want to filter out that content when fetching or client-side?

Front-end check sounds good to me :)


WIP: Installing build requirements via pep517 mechanics

Install build requirements and build using pep517
PEP517 requires projects to list their build dependencies and build
system (setuptools for us) in pyproject.toml. Build tools like pip can
then know how to build the package, installing their build dependencies
automatically (in an isolated environment such that they are not
installed in your current environment) and then running the build system
to build a wheel or sdist.
Most people use pip to build wheels for projects that use this
mechanism. But pip hasn't implemented the sdist part of it yet, which
would leave us still manually installing dependencies just to make
sdists.
So whilst the Python packaging authority makes up its mind where the
general-purpose PEP517 build commands belong, we use the reference
implementation package pep517 (which IMHO should just be given a
better name and become the blessed tool for PEP517 builds).

CI is failing because of setuptools_conda being broken, see #21, once that's merged I'll rebase.

CI is failing because of setuptools_conda being broken, see #21, once that's merged I'll rebase.

Wow, well spotted.


Please consider excluding junit from the list of runtime dependencies

https://repo1.maven.org/maven2/com/pinterest/ktlint/0.36.0/ktlint-0.36.0.pom
Is junit:junit:4.12 really required as a runtime dependency of ktlint?
The list of dependencies is as follows:
\--- com.pinterest:ktlint:0.36.0
     +--- com.pinterest.ktlint:ktlint-core:0.36.0
     |    +--- org.jetbrains.kotlin:kotlin-compiler-embeddable:1.3.60
     |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60
     |    |    |    +--- org.jetbrains.kotlin:kotlin-stdlib-common:1.3.60
     |    |    |    \--- org.jetbrains:annotations:13.0
     |    |    +--- org.jetbrains.kotlin:kotlin-script-runtime:1.3.60
     |    |    +--- org.jetbrains.kotlin:kotlin-reflect:1.3.60
     |    |    |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     |    |    +--- org.jetbrains.kotlin:kotlin-daemon-embeddable:1.3.60
     |    |    \--- org.jetbrains.intellij.deps:trove4j:1.0.20181211
     |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     |    \--- org.ec4j.core:ec4j-core:0.2.0
     +--- com.pinterest.ktlint:ktlint-reporter-checkstyle:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-reporter-json:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-reporter-html:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-reporter-plain:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-ruleset-experimental:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    +--- com.pinterest.ktlint:ktlint-test:0.36.0
     |    |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    |    +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     |    |    +--- org.assertj:assertj-core:3.12.2
     |    |    \--- junit:junit:4.12
     |    |         \--- org.hamcrest:hamcrest-core:1.3
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-ruleset-standard:0.36.0
     |    +--- com.pinterest.ktlint:ktlint-core:0.36.0 (*)
     |    +--- com.pinterest.ktlint:ktlint-test:0.36.0 (*)
     |    \--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- com.pinterest.ktlint:ktlint-test:0.36.0 (*)
     +--- org.jetbrains.kotlin:kotlin-stdlib:1.3.60 (*)
     +--- org.jetbrains.kotlin:kotlin-compiler-embeddable:1.3.60 (*)
     +--- com.github.shyiko.klob:klob:0.2.1
     \--- info.picocli:picocli:3.9.6

Thank you, that is the bug in project Gradle configuration. Should not be the case for previous versions of ktlint as 0.36.0 was first published using Gradle.

The first broken version is 0.34.0
com.pinterest.ktlint:ktlint-test:0.32.0
\--- org.assertj:assertj-core:3.9.0

com.pinterest.ktlint:ktlint-test:0.33.0
\--- org.assertj:assertj-core:3.9.0

com.pinterest.ktlint:ktlint-test:0.34.0
+--- org.assertj:assertj-core:3.12.2
\--- junit:junit:4.12
     \--- org.hamcrest:hamcrest-core:1.3

com.pinterest.ktlint:ktlint-test:0.35.0
+--- org.assertj:assertj-core:3.12.2
\--- junit:junit:4.12
     \--- org.hamcrest:hamcrest-core:1.3

I've removed not-needed dependency on :ktlint-test (uses junit) from experimental ruleset via #679, though removing it from ktlint cli would require more work as printAST subcommand uses "dump" rule from :ktlint-test


New linter suggestion : Bandit (a Python code security analyzer)

cf recent announcement on https://wiki.openstack.org/wiki/Security/Projects/Bandit
(seen on reddit at https://www.reddit.com/r/Python/comments/2ndtva/introducing_bandit_a_python_code_security_analyzer/ )
It should be a fairly easy to integrate linter (simple executable, probably not dependant on a python version).
I guess it would provide some traction to that interesting project.
If someone wants to work on it before I have the time to do it myself, please proceed !

Closing with merges of #518


Implement SchemaError#value and SchemaError#key

This PR implements SchemaError#key & SchemaError#value. It makes SchemaErrors interface more in line with MissingKeyError, UnknownKeysError & ConstraintError which have (most) of their instance variables accessible to the public.

thanks!


Get semantic information from build script

Hello,
I'm looking for a way to get the semantic information (or compilation information) from a build script. For example, given a part of build.gradle script:
dependencies {
    compile gradleApi()
}

for symbol dependencies, it will be the method call dependencies() of interface org.gradle.api.Project in execution. For symbol gradleApi(), it will call gradleApi() in org.gradle.api.artifacts.dsl.DependencyHandler. Is there any API or way to get this semantic information programmatically for a script? I just tried to get some information from Project instance in a custom gradle plugin or directly use tooling API, but nothing found.
Thanks in advance!

Expected Behavior

Current Behavior

Context

Please take a look at https://groovy-lang.org/closures.html#_delegation_strategy
The gradle build script is simply a Groovy file which complies with Groovy Closure delegation strategy.


Why only send the position of root joint of each frame into the model as input?

Hi,
I was kind of curious that why only send the position of root joint of each frame into the model as input?
I mean after parsing the .bvh file, we could get the positions of every joint at every frame. But we just use the root joint's position as input, is it because other joints' position info is redundant?
Mant thanks!

As we only generate the rotation and global position, we find it's sufficient to use rotation as the network's input. However, we do use the position information in rec_loss and ee_loss to get more accurate result.

As we only generate the rotation and global position, we find it's sufficient to use rotation as the network's input. However, we do use the position information in rec_loss and ee_loss to get more accurate result.

Oh, it makes sense! To reconstruct the bvh file, we just need the position info of the root joint. I should have known that.


Update riak_cs_multibag tag to 1.5.4p1

There are several unreleased changes in release/1.5 branches of riak_cs and riak_cs_multibag.
This PR updates deps for riak_cs_multibag to tag/1.5.4p1 to avoid one step of manual checkout under deps.

You've been also assigned to update the version of all other dependencies in develop. :imp:

@borshop merge


CVXPY can't recover MOSEK solution of a power cone problem

Describe the bug
When I try to solve a power cone problem with MOSEK, MOSEK solve the problem but CVXPY can't recover the solution. However, when I solve the problem with CLARABEL, there is no problem.
To Reproduce
import numpy as np
import cvxpy as cp
import scipy.stats as st

T, N = 200, 10

rs = np.random.RandomState(123)
mean = np.zeros(N) + 1/1000
cov = rs.rand(N,N) * 1.5 - 0.5
cov = cov @ cov.T/1000 + np.diag(rs.rand(N) * 0.7 + 0.3)/1000

Y = st.multivariate_normal.rvs(mean=mean,
                               cov=cov,
                               size=T,
                               random_state=rs)

w = cp.Variable((N, 1))
t = cp.Variable((1, 1))
z = cp.Variable((1, 1))
omega = cp.Variable((T, 1))
psi = cp.Variable((T, 1))
nu = cp.Variable((T, 1))
epsilon = cp.Variable((T, 1))
k = cp.Variable((1,1))
b = np.ones((1, N))/N

X = Y @ w

h=0.2
ones = np.ones((T,1))
constraints = [
    cp.constraints.power.PowCone3D(z * (1+h)/(2*h) * ones, psi * (1+h)/h, epsilon, 1/(1+h)),
    cp.constraints.power.PowCone3D(omega/(1-h), nu/h, -z/(2*h) * ones, (1-h)),
    -X - t + epsilon + omega <= 0,
    w >= 0,
    z >= 0,
    ]

obj = t + z + cp.sum(psi + nu)

constraints += [cp.sum(w) == k,
                k >= 0,
                b @ cp.log(w) >= 1,
                ]
objective = cp.Minimize(obj)

prob = cp.Problem(objective, constraints)

prob.solve(
    solver=cp.MOSEK, #The error happens when I use cp.MOSEK
    # solver=cp.CLARABEL,
    verbose=True,
    )

print(obj.value)

Expected behavior
I expect that solving the problem with MOSEK give me the same values than with CLARABEL
Output
MOSEK output:

CLARABEL output:

Version

OS: macOS 13.2.1 (22D68)
CVXPY Version: 1.3.0

Additional context
No other context.

Fixed via #2043.


Feature/10 switch node server to quarkus server

There is still a lot of work to do setting this project up. But I wanted to get it in soon so that we don't have too many conflicts

So what is missing to get this PR merged?

So what is missing to get this PR merged?

I think mostly, can you run this branch successfully?  It is "done" as a preliminary workflow idea.


added blog examples to Node and Csharp

This is for Blog Entry: "Custom Channel Data"

@matthewshim-ms,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


getTickerApi() giving error : AttributeError: 'str' object has no attribute 'pop'

While running the following code, it is giving error :
bitbnsObj = bitbns(key, secretKey)

bitbnsObj
Out[23]: <bitbnspy.bitbnspy.bitbns at 0x224b0b8b2b0>

bitbnsObj.getTickerApi( 'MATIC' )
Traceback (most recent call last):

  File "<ipython-input-24-17c6c2c9e69c>", line 1, in <module>
    bitbnsObj.getTickerApi( 'MATIC' )

  File "C:\anaconda\lib\site-packages\bitbnspy\bitbnspy.py", line 99, in getTickerApi
    req[key].pop('yes_price', None)

AttributeError: 'str' object has no attribute 'pop'

HI @boral , could you please describe your environment ?

@sanket-buyhatke I am using Python 3.8.5

While using the getTickerApi function, how to get the price of the token as per both INR and USD?

Hi @boral, Are you still facing this issue? I have tried running this with python 3.8.5, and did not face this issue.

No, it is working fine now.
On Jun 7, 2021, 2:20 PM +0530, its-mohnish @.***>, wrote:

@.***, Are you still facing this issue? I have tried running this with python 3.8.5, and did not face this issue.
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

great, will be closing this issue, feel free to re-open if you face the issue again.


Swap UI updates

PR-Codex overview
This PR focuses on updating the UI elements related to wallet selection and swap transactions.
Detailed summary

Updated background colors and styles for various UI components related to wallet selection and swap transactions
Added a new function formatSeconds for formatting time durations
Added a new 4xl size to iconSize
Refactored the TokenSymbol component to accept a color prop
Updated styles and animations for the Overlay and Drawer components
Added a new DrawerOverlay component and useDrawer hook
Introduced useSwitchActiveWalletChain hook in Details.tsx

The following files were skipped due to too many changes: packages/thirdweb/src/react/web/ui/ConnectWallet/Details.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/ConnectWallet.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/swap/BuyTokenInput.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/swap/PayWithCrypto.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/TokenSelector.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/swap/ConfirmationScreen.tsx, packages/thirdweb/src/react/web/ui/ConnectWallet/screens/Buy/SwapScreen.tsx

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

/release-pr

/release-pr

/release-pr

/release-pr

/release-pr


⚠ UdeS website (prod) has degraded performance

In 49dfdd9, UdeS website (prod) (https://www.usherbrooke.ca/) experienced degraded performance:

HTTP code: 200
Response time: 1715 ms

Resolved: UdeS website (prod) performance has improved in 5b50b76.


Merge master branch to prTesting1_202007151929 branch

Docs Build status updates of commit 006eb55:
:clock10: Pending: waiting for processors (7 builds ahead of you)
:warning: Docs Build is busy, currently there are 7 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit 006eb55:
:warning: Validation status: warnings

File
Status
Preview URL
Details

split/updateTestCon1.md
:warning:Warning

Details

split/updateTestCon1.md

Line 0: [Warning-h1-not-first] Markdown content is not allowed before H1.
Line 1: [Suggestion-h1-missing] H1 is required. Use a single hash (#) followed by a space to create your top-level heading. NOTE: This Suggestion will become a Warning around July 2020.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching in the Docs contributor and Admin GuideSee the frequently asked questionsPost your question in the Docs support channel


[RFE] [SDE-1767] Ability to configure Volume for Expansion

Once a volume is created, users must be able to increase the size of the existing volume. When the user "edits" the volumes, the option to allow users to specify the size (GBs) to be expanded. We should display the current size allocated, "Size to Expand" along with the available storage capacity for users to expand the volume size.

[x] https://github.com/ManageIQ/manageiq-ui-classic/pull/7788
[x] https://github.com/ManageIQ/manageiq-ui-classic/pull/7779

This has been implemented and backported to Morphy


Is there a class to do a datatable search?

Hello
I need to do a table search.
For example, my table has multiple columns.
I need to find rows with a salary greater than 30000 and less than 50000.
Is there a way to do this using datatable2?
Thank you

Any progress for this ?

There's no plan to include this feature as a part of package. data_table_2 was never intended to be a rich grid control, hence the assumption is that one can create own grid by adding additional widgets (e.g. filter) and managing the state.
Here's an example:
https://maxim-saplin.github.io/flutter_web_spa_sample/canvaskit/#/
https://github.com/maxim-saplin/flutter_web_spa_sample

Hello
There are no issues now. You've already closed the ticket.
Thank You
Dmitri Kozlov    On Thursday, November 21, 2024 at 02:12:24 AM EST, Maxim Saplin @.***> wrote:
Reopened #311.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>


Comprehensive Refactoring and Documentation of Clustering

Description
This pull request encompasses a series of significant updates aimed at refactoring the project's structure, enhancing code readability, and improving documentation. The changes are detailed across several commits, focusing on modularizing the codebase, implementing thorough testing, and providing comprehensive documentation for each module and test suite.
Key Changes
Commit #1d5f553: Regrouping Files and Splitting Modules

Restructured the project by moving clustering_core/, visualizing/, tests_core/, and embeddings_core/ into the src/ directory for better organization.
Divided KmeansND.h into five distinct modules for clarity and adherence to the S.O.L.I.D. principles:

kMeansLogic.hpp for the k-means algorithm logic.
readData.hpp for file reading functionalities.
structPoint.hpp for the Point structure implementation.
writeData.hpp for result saving functionalities.
clusterTools.hpp for additional methods like returnClustersSize.

Simplified complex functions into more readable modules, such as splitting readData and saveData into specific functions for different file formats.
Introduced a .save() function in the KMeansND class to save clustering results and centroids efficiently, considering the _with_coordinates flag to reduce memory usage in higher dimensions.

Commit #23f1af4: Implement Documentation for Modules

Added documentation files in the documentation/ directory for the newly created modules.
Strived for intuitive and understandable code with self-explanatory names to minimize over-commenting.

Commit #6aa74d4: Tests and Fixes

Implemented tests for all modules, including sample data and output file verification.
Fixed constructors in Struct Point and implemented operator overloading for enhanced usability.
Added a .norm() method to struct Point for vector norm calculation.
Improved writing modules to include proper headers and enhanced reading modules to accurately handle clustered points.
Provided instructions for installing Google Test locally, with plans to switch to built-in testing frameworks.

####Commit #c62065c: Fixing ClusteringND + Tests and Documentation Updates

Addressed issues in the ClusteringND logic and its tests, ensuring all constructors of the Point structure are thoroughly tested.
Fixed modules related to centroid recalculation and point assignment to centroids.
Updated documentation to reflect these changes and added a changeLog.md to summarize commit history.

Impact
These changes significantly improve the project's maintainability, scalability, and readability. By modularizing the codebase, we've laid a solid foundation for future enhancements and made the project more accessible to new contributors. The comprehensive testing suite ensures reliability and stability, while the detailed documentation facilitates easier onboarding and reference.
P.S. All tests are passed

Mark you?


Break apart 'core' monolith

The file core.js in the build system is large and does way too much stuff. I should break it apart into two or three files with single responsibilities.

I factored out argument parsing, but core.js is still nearly 300 lines long, so that could be better.

Good enough.


React component with date formatting broken after 1.5.0

Hey,
Date formatting seems to be broken in React component after 1.5.0.
No fork necessary, see console for error: https://jsfiddle.net/nosir/gLLsrxxf/
The error occurs in DateFormatter.js#L9.
Called in the react component here, Cleave.react.js#L147
Introduced in the PR #473.
We've downgraded to 1.4.10, which works as expected.

Yes, we encountered the same thing.
Basically, DateFormatter signature changed(added minDate and maxDate), but not all the consumers(method callers) have been adapted.

@nosir Can we go ahead and merge #478 ?

Hello @nosir, is there a reason why this was not merged yet?

Got

With
Options : {date: true, datePattern: ['Y', 'm', 'd']}

Any news on this? Can you release this? ❤

Sorry guys, work stuff.
Thanks for the email @tcp
Released in 1.5.1 and npm published


Handle type_member redefinition gracefully

Motivation
Fixes #1769: this splits type_member handling into two places: first into a DSL pass to handle repeated name usage, and second in the namer by properly looking up the names of existing type_member values. This PR also fixes a bug with other constant redefinitions where it wouldn't show an error in the fast path, as well.
Test plan

See included automated tests.

https://github.com/sorbet/sorbet/pull/1779#discussion_r325461760
Did you forget to push the second file? I see only one file in this commit.

…I did forget. (That's what I get for trying to finish a PR while also making dinner.)


KubeEdge Counter Demo is not working properly

What happened:
KubeEdge Counter Demo is not working properly
What you expected to happen:
use [kubectl get pods -o wide] can get ke-edge1's IP and use [docker logs -f counter-container-id] can get count value.
Environment:

KubeEdge version(e.g. cloudcore/edgecore --version): v1.4.0
CloudSide Environment:
Hardware configuration (e.g. lscpu):Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz
OS (e.g. cat /etc/os-release):centos8
Kernel (e.g. uname -a):Linux ke-cloud 4.18.0-193.el8.x86_64 #1 SMP Fri May 8 10:59:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Go version (e.g. go version): go version go1.14.4 linux/amd64
Others:

EdgeSide Environment:

edgecore version (e.g. edgecore --version):v1.4.0
Hardware configuration (e.g. lscpu):Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz
OS (e.g. cat /etc/os-release):centos8
Kernel (e.g. uname -a):Linux ke-edge1 4.18.0-193.el8.x86_64 #1 SMP Fri May 8 10:59:10 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Go version (e.g. go version): go version go1.14.4 linux/amd64
Others:

请问怎么解决的，我遇到一样的问题了

edit /etc/kubernetes/manifests/kube-apiserver.yaml
--insecuret-port=8080
--insecure-bind-address=<IP_ADDRESS>
restart

the quickesty way, i think； file: examples/kubeedge-counter-demo/web-controller-app/utils/kubeclient.go should be edit like this
// replace this with the K8s Master IP
var KubeMaster = "https://<IP_ADDRESS>:6443"
var Kubeconfig = "/static/admin.conf"

and copy admin.conf to your container；fileexamples/kubeedge-counter-demo/web-controller-app/Dockerfile add line
COPY admin.conf /static

the better way is get permission by clusterrole && clusterrolebinding && serviceaccount，i think this demo has a better way to update CR，if i will solve those problems，i will share it

edit /etc/kubernetes/manifests/kube-apiserver.yaml
--insecuret-port=8080
--insecure-bind-address=<IP_ADDRESS>
restart

I got the issue that the port is in use by the Kube-api. any suggestions? thanks
2020/12/11 13:22:23 Get kubeConfig successfully
2020/12/11 13:22:23 Get crdClient successfully
2020/12/11 13:22:24.001 [I]  http server Running on http://:8089
2020/12/11 13:22:24.001 [C]  ListenAndServe:  listen tcp :8089: bind: address already in use

edit /etc/kubernetes/manifests/kube-apiserver.yaml
--insecuret-port=8080
--insecure-bind-address=<IP_ADDRESS>
restart

I got the issue that the port is in use by the Kube-api. any suggestions? thanks
2020/12/11 13:22:23 Get kubeConfig successfully
2020/12/11 13:22:23 Get crdClient successfully
2020/12/11 13:22:24.001 [I] http server Running on http://:8089
2020/12/11 13:22:24.001 [C] ListenAndServe: listen tcp :8089: bind: address already in use

maybe you should check deployment，hostNetwork: true i delete it and didn`t get error

Please try the latest example to confirm this issue exists or not, thanks :)

Please try the latest example to confirm this issue exists or not, thanks :)

Please try the latest example to confirm this issue exists or not, it works at my env, thanks :)
If it still exists, please reopen it!
/close

Please try the latest example to confirm this issue exists or not, it works at my env, thanks :)
If it still exists, please reopen it!
/close


Add first end to end test for new ssa module

Problem
With the SSA generation section of the code complete, we want to have an end to end test added as soon as possible. This will allow us to further parallelilze the work needed since one can add optimizations into the ACIR module and ACIR gen more parts of the SSA IR in parallel.
Proposed solution

[ ] Add an experimental flag to enable the new SSA module. This is similar to what we did when we moved from the interpreter to the current SSA code.

[ ] Minimal program

As a first milestone, lets get the following program running from end to end:
fn main(x : pub Field) -> pub Field {
  x + 1
}

Alternatives considered
No response
Additional context
The experimental flag will allow us to make circuit size comparisons easier too.
Submission Checklist

[X] Once I hit submit, I will assign this issue to the Project Board with the appropriate tags.

We won't be able to run the full test suite until all features have been enabled, but we can keep a branch open that simply enables the new SSA and comments out the tests.
I'm usually against long standing branches, though because this branch will only be used to keep track of what features are supported, it is fine

Various tests have been added at this point


How to send Response/Request?

Hey!
I've been looking through the documentation but I can't find any example of how to actually send a Response. How do I get headers+body as a u8 slice to write to a socket?
Thanks!

This crate is meant to just be the types, without a client or server. The hope is that other libraries could depend on some HTTP things without having to declare which client or server implementation a user wants to use. You could use https://hyper.rs as a possible client or server.

Ah I see. I looked into Hyper but I need more low level control for my application. I'm using reqwest for incoming request which works great and was hoping to find something similar for generating responses.


DBZ-6551 Modularising the project

https://issues.redhat.com/browse/DBZ-6551
Modularising the project in order to provide better structure, split the complexity of single pom and prepare for builder and documentation generation in the future.

@jcechace I definitely like the approach. One question though, is api module conseptually really an API?

@jpechane  define API... it's also going to be used for generating for example model builders.
Strimzi's equivalent module is also named api. https://github.com/strimzi/strimzi-kafka-operator/tree/main/api/src/main/java/io/strimzi/api/kafka