[#8162] cache applications api on browser/server side

Resolve #8162
Feature
By default, the application list would be cached by Ehcache for 120 seconds(configured in ehcache.xml) and be cached by browser  for 60 seconds (configured by web.applicationList.cacheTime in pinpoint-web-root.properties). If the request having no "If-Non-Match" header(when it's the first time to request or after expired browser cache time), ehcache would be evicted and retrieve data from hbase. The cache can also be clear by requesting {web_host}/applications.pinpoint?clearCache manually.
The first time to request for a user, there would be no cache in use.
You may let the web browser cache for a longer time by setting -Dweb.applicationList.cacheTime to a bigger value (in seconds). The following request would get "304 Not Modified response header" and no response body. The browser would use previously cached data.

In the above picture:

no cache on the browser & server sides.
cache on the browser side.
cache on the server side only and no application list data to transfer.

Verification
# request for the first time or with expired cache
$ curl 'http://localhost:8080/applications.pinpoint' -D-

# request with eTag returned by the previous response eTag header value
$ curl 'http://localhost:8080/applications.pinpoint' -D- -H 'If-None-Match: "applicationNameListVer1"'

# clear server cached list
$ curl 'http://localhost:8080/applications.pinpoint?clearCache' -D-

How about changing the applicationList's reload button click action?

Bypass browser cache and force update applicationlist for data freshness

eg)

Add magic key to /applications.list
Add clearCache parameter to applications.list

/applications.list?clearCache&magic=1234

@emeroad  great idea.

@emeroad implemented.

@yjqg6666
When I click the reload button, it passes the parameters which I guess is for bypassing the cache and update the data but I noticed the browser keeps fetching the data from the cache even when clicking the button in short period.

Is this an expected behavior?

@binDongKim It's not and a new commit is appended to fix it.

@binDongKim Done and thanks for the guide and review.

@yjqg6666
Things look good to go in frong-end side. Thank you!


Add CodeGen for Swift (URLSession) #140

PR Description
Add CodeGen for Swift (URLSession) #140
Related Issues

Related Issue #140
Closes #140

Checklist

[x] I have gone through the contributing guide
[x] I have run the tests (flutter test) and all tests are passing

Added/updated tests?

[x] Yes

@CodeHariK Can you kindly update the test cases to include example Request Models POST 4 to POST 9 which have been recently added.


Add more readline bindings

These changes add more readline keybindings to zf. They are definitely all about opinion, so if adding these keybindings does not align with this project, let me know and I can close this PR. Thanks!

I am definitely okay with removing the ctrl-k binding. I think adding in the binding with an environment variable would be a nice addition in the future.

I will get this merged soon, and I have added a task for an environment variable in #7. Thanks again!


feat(vite-plugin-nitro): provide req and res to renderer

PR adds full req and res object from h3 to the analog event handler for  app rendering
closes #802
PR Checklist
Please check if your PR fulfills the following requirements:

[ ] The commit message follows our guidelines: https://github.com/analogjs/analog/blob/main/CONTRIBUTING.md#-commit-message-guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

Which package are you modifying?

[ ] vite-plugin-angular
[ x] vite-plugin-nitro
[ ] astro-angular
[ ] create-analog
[ ] router
[ ] platform
[ ] content
[ ] nx-plugin
[ ] trpc

What is the current behavior?

Current behavior is that the full, raw request and response are not provided to Analog, which limits the ability to use cookies from the incoming request.
Closes #802
What is the new behavior?
Now both the runtime renderer event handler and the nitro dev server have access to the request and response, which can then be provided to the application.
Does this PR introduce a breaking change?

[ ] Yes
[x ] No

Other information
[optional] What gif best describes this PR or how it makes you feel?

Looks great @Jefftopia! One small cleanup item on the interceptor naming

Thanks! @allcontributors add @Jefftopia for code


Improve Visibility of Image Content

Codecov Report

Merging #238 (97fcf04) into master (c5f60a4) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #238   +/-   ##
=========================================
  Coverage     58.80%   58.80%
  Complexity     1042     1042
=========================================
  Files           161      161
  Lines          3945     3945
  Branches        539      539
=========================================
  Hits           2320     2320
  Misses         1447     1447
  Partials        178      178

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c5f60a4...97fcf04. Read the comment docs.


Rename project to Corrosion

[ ] Change crate, module, and library names
[x] Change the GitHub project name
[x] Update README
[x] Change function names

Hey, @AndrewGaspar. I wonder, what the project was named back then, and why this issue is still opened?

It used to be called "cmake-cargo".  Haven't gotten around to cleaning all of them up yet.

Ah, "cmake-cargo".  I think I could recall hearing that name long time ago.

There are a bunch of CMake variable names that are still CMAKE_CARGO_*

~IIRC, custom variable names are not allowed to start with CMAKE_. Which is probably the reason why they actually start with underscore-prefixed _CMAKE_CARGO_*.~ Both (underscored and not) variants are reserved per cmake-variables(7) documentation. If you don't mind, I'll give it a shot.


Support Private Gateway and Static Route

Cloudstack now allows regular users to create Private Gateways without root access.
However, this update hasn't been integrated into the Terraform provider. Would like to request that this be supported to reduce our manual efforts.
Additionally, support for Static Routes configurations, especially with user-created gateways, could enhance functionality.

@btzq  thanks for rasing the issue
We can consider this for future release


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/65207

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/67904


🚀 Feature: Rewrite standalone installation instructions

🔖 Feature description
Rewrite standalone installation instructions
Rewrite the standalone installation. Make clear that this is a non-production install that a user might want to do for one of several reasons: demo, proof of concept, development environment, etc. Organize under Getting Started in the doc TOC.
Audience: Developer, Admin
Type: Task
The existing documentation might contain helpful source material that you can pull into this doc change. See this catalog of the existing (at the time of the CNCF tech doc analysis) Backstage technical documentation pages:
https://github.com/cncf/techdocs/blob/main/assessments/0008-backstage/backstage-doc-survey.csv
🎤 Context
This issue is a recommended change resulting from an analysis of the Backstage documentation commissioned by CNCF. The analysis and supporting documents are here:
https://github.com/cncf/techdocs/tree/main/assessments/0008-backstage
✌ Possible Implementation
Related instructions for standalone installation in the current doc:

https://backstage.io/docs/getting-started/
https://backstage.io/docs/getting-started/create-an-app

(These instructions are redundant, BTW, and should be consolidated in addition to other changes.)
Suggested changes:

Clarify the user roles and use cases that the install is for.
Rewrite as a step-by-step process.
Rename to something like "Standalone installation".
Add "What's next" section.

👀 Have you spent some time to check if this feature request has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
None

Happy to pick this up


Hi, why can't I find this script in System Setting?

I can't not find this script in system setting by searching the name of the script.
I mean, from here:

It still exists on the website at least: https://store.kde.org/p/1312691


Nested logprob rewrites proof of concept

This PR is a proof of concept for how we could handle composite RVs that require nested aeppl rewrites. The goal is to figure out if this approach is too limited / cumbersome and, if so, what could work better.
Roughly, most of our rewrites come in two parts:

Check if an initially non-measurable variable was assigned to a value variable
If so, try to convert this variable into an temporary measurable variable for which _logprob can be safely dispatched

In step 2, we often need inputs that are themselves measurable variables, but which don't have value variables assigned to them:
https://github.com/aesara-devs/aeppl/blob/05d0c683aa568e59bab8fcce4618f89c57355122/aeppl/mixture.py#L283-L287
If these inputs are RandomVariables things are fine, as those are measurable by default. Otherwise, we might still be dealing with things that could be measurable... but we will never know because they don't (and shouldn't) have value variables and their corresponding rewrites won't be triggered due to step 1.
The illustrating example in this PR is a scalar mixture that has a clipped variable as one of it's components:
    x = at.clip(at.random.normal(loc=1), 0.5, 1.5); x.name = "x"
    y = at.random.beta(1, 2, size=None, name="y")

    comps = at.stack(x, y); comps.name = "comps"
    idxs = at.random.bernoulli(0.4, size=None, name="idxs")
    mix = comps[idxs]; mix.name = "mix"

    mix_vv = mix.clone()
    idxs_vv = idxs.clone()
    logp = joint_logprob({idxs: idxs_vv, mix: mix_vv})

I have also added a test example with nested scalar mixtures
Suggestion
The suggestion is that any rewrite that depends on other inputs being measurable, can assign a temporary value variable UPSTREAM_VALUE to those inputs. This value variable is automatically discarded by the PreserveRVMappings when such variables are converted to measurable ones by their own rewrites.
Limitations
This would work for some current and future rewrites like #26, but not for everything.
For example nested rewrites that depend on the direct manipulation / creation of value variables such as inc_subtensor and scans would not work. This would also not work when the _logprob dispatched function depends on having RandomVariables as inputs, as happens with non-scalar mixtures:
https://github.com/aesara-devs/aeppl/blob/05d0c683aa568e59bab8fcce4618f89c57355122/aeppl/mixture.py#L354
Also depending on the order with which rewrites are called this may not always work, because the attribution of the UPSTREAM_VALUE is not a visible graph change, and I guess the Equilibrium optimization will stop if after passing through all nodes, none is changed

Roughly, most of our rewrites can be divided in two parts:

Check if an initially non-measurable variable was assigned to a value variable
If so, try to convert this variable into an temporary measurable variable for which _logprob can be safely dispatched

It's better to think of this process as first producing a "measurable" intermediate representation (IR) of the original model graph.  The fact that the value variable checking/manipulation is also done at that point is mostly a matter of convenience.
This distinction is important, because there may be use/need for the identification of measurable terms independent of value variables, to which it seems you're ultimately alluding.
It's possible that the entire scenario you're describing could be handled by simply loosening the overly strict value variable requirements instead of adding a new placeholder element and associated logic.
In other words, why work around/within existing restrictions when they might be unnecessary in the first place?  If we don't consider these kinds of things first and foremost, we'll likely end up building something that becomes more and more difficult to develop.

Also depending on the order with which rewrites are called this may not always work, because the attribution of the UPSTREAM_VALUE is not a visible graph change, and I guess the Equilibrium optimization will stop if after passing through all nodes, none is changed

#78 is an answer (of sorts) to this.  The idea is to put value variables in the graph, then we don't need a Feature and the rewrites we're using make a lot more sense at a higher level.

It's possible that the entire scenario you're describing could be handled by simply loosening the overly strict value variable requirements instead of adding a new placeholder element and associated logic.
In other words, why work around/within existing restrictions when they might be unnecessary in the first place?  If we don't consider these kinds of things first and foremost, we'll likely end up building something that becomes more and more difficult to develop.

That was my other alternative, just eagerly convert / flag all nodes that can be made measurable as such, but it seemed like this would be a bit insane/expensive the more I thought about it.

That was my other alternative, just eagerly convert / flag all nodes that can be made measurable as such, but it seemed like this would be a bit insane/expensive the more I thought about it.

What made you think that?

In my mind it meant we would end up replacing almost all nodes in a graph by equivalent "measurable" ones, specially as we increase the "coverage" for the basic Aesara Ops.

In my mind it meant we would end up replacing almost all nodes in a graph by equivalent "measurable" ones, specially as we increase the "coverage" for the basic Aesara Ops.

Yes, how else would we do these kinds of things?  In other words, if a mixture needs to consist of Joined MeasurableVariables, how are we going to avoid determining whether or not the mixture components are MeasurableVariables in the first place?  Since the latter requires that the IR rewrites be performed first, there's really no avoiding a broad application of them throughout the sub-graphs within the "valued" mixture node.
The value variables are used to determine the subgraphs to which we apply IR rewrites in the first place (i.e. they determine the conditional log-probabilities), and those subgraphs will be the outputs of the FunctionGraph being operated upon, so we already have a means of applying rewrites to only the relevant sub-graphs.
One could make a distinction between taking a bottom-up and top-down approach, but I don't think there's a meaningful difference between the two in this scenario.  We already know that the "bottom"/outputs of the FunctionGraphs need to be measurable—otherwise they wouldn't be there—and, in all "composite" node cases (e.g. Mixture and Scan) I can imagine right now, we still need the rewrites to make specific sub-graphs measurable.
Even so, we always have control over how the rewrites traverse the graphs, so there's nothing preventing us from applying them more intelligently at a high level and avoiding unnecessary applications.

The value variables are used to determine the subgraphs to which we apply IR rewrites in the first place (i.e. they determine the conditional log-probabilities), and those subgraphs will be the outputs of the FunctionGraph being operated upon, so we already have a means of applying rewrites to only the relevant sub-graphs.

I don't quite see how do we distinguish between the following two cases (and if we have to):
x = at.random.normal()
y = at.clip(x, 0, 1)
z = at.random.normal(y)

x_vv = x.clone()
y_vv = y.clone()
z_vv = z.clone()

# not measuring clip node
joint_logprob({x: x_vv, z: z_vv})

# measuring the clip node
joint_logprob({y: y_vv, z: z_vv})

In both cases y is part of the FunctionGraph but only in the second do we need the rewrite to apply.
Currently we distinguish between the two based on whether it has a value variable or not, but if I understand you we would now always rewrite a clip so that it would work in nested expressions like the mixture example above.

Alternatively, there is some information already that we don't want to convert it in the first case because the measurement input from which the logprob derives, x, is already valued / accounted for. But this seems like a similar type of local information passing just in the other direction.

Currently we distinguish between the two based on whether it has a value variable or not, but if I understand you correctly, we would now always rewrite a clip so that it would work in nested expressions like the mixture example above.

Yes, we're currently performing (IR) normalizing/canonicalizing rewrites to the entire graph, even if there are some "upstream" terms that may be irrelevant.  This is the case in the former example, where the measurability of the y term isn't relevant and the rewrites for clip aren't needed.
That's done because it's (generally) cheap and it provides a simplicity to our rewrites and the way in which they're applied.  It's basically a "greedy" approach.
As I mentioned above, we can always control the graph traversal and application of rewrites so that certain rewrites are only applied upstream from a valued variable when they're needed; however, complications arise in this scenario due to an added dependency between the traversal logic and the rewrite rules themselves.
For instance, starting at the "bottom"-most/valued nodes, in the former case, we would start at the expression graph for z and immediately determine that it's measurable and that no further traversal + rewrites are necessary.  That's the easy case.
In the latter case, we would start at y and determine that it's only measurable if x is measurable, so we need to walk up the inputs of y and potentially rewrite those so that they're measurable.  In this example, no rewrites are necessary, but the general logic still requires that the rewrites be applied by same logic that determines whether or not something is measurable.  This is the dependency that conditional rewrite applications introduce.
Taking the above literally and creating a routine based simply on walk and manually applications of rewrites would result in a very convoluted and error-prone interface/development context.  If we want to take a more efficient approach, we need to carefully consider how best to handle this dependency without foregoing a lot of the existing graph and rewrite tooling.
There may be a way to do this using "preconditioning" passes that truncate graphs via their naive "base" measurability.  For example, we could walk up graphs and determine the relevant subgraphs using isinstance(..., MeasurableVariable) (i.e. the atomic/"base" measurable nodes):
I = at.random.bernoulli(0.5)
C = at.random.normal(0, 1)
A = at.clip(C, -1, 1)
B = at.random.normal(0, 1)
R = at.ifelse(I, A, B)
X = at.random.normal(R)
Y = at.clip(X, 0, 1)

# Traversal should stop at `Y -> X`
joint_logprob({Y: y_vv})

# Traversal should stop at `X`
joint_logprob({X: x_vv})

# Traversal should stop at `R -> [I, A -> C, B]` (i.e. the immediately
# measurable `I`, `C`, `B`)
joint_logprob({R: r_vv})

We could also make it possible to determine the non-measurable Ops for which we have rewrites and use that to better trim the subgraphs.


(<IP_ADDRESS>.) Dynamic Device Type?

(This is not an issue!)
Dear All,
I am on an early stage of CHIP development.
I am wondering if our products possible to fit the CHIP standards.
Here are our simplified products/plans:

Products: one control-center; many on/off switches.
The control-center can control/communicate all the switches.
Users can add/remove switches into the control-center any time.
(The control-center and switches use wires to communicate)
============
Here raise the question:

Only the control-center has the WIFI ability.
Is this scenario is able to make the control-center CHIP-able?
Such as:
A) We are thinking to commission the control-center into the nest-hub.
B) Switches connect with control-center by our own wire/flow.
C) The control-center would know all the switches’ status, and can report it to nest-hub.
D) And, the switches are able to remove/add into the control-center in the future. And the control-center can "update/change" this info to the nest-hub.

Is this kind of control-center being able to fit with CHIP? What type should be for it? (<IP_ADDRESS>.) Dynamic Device Type?
Thank you any info....
BR,
Vic

is matter bridge what I want ?


latestTo() has undefined html and text when email contains attachments

When a message is retrieved with latestTo that contains attachments the text and html properties are undefined. Removing the attachments from the messages results in the properties being set and retrievable.

I use search(from, 'from') and get the same

I found library
https://nodemailer.com/extras/mailparser/
for example
let emails = await mailhog.search(from, 'from');
let parsedEmail = await simpleParser(emails.items[0].Raw.Data);

//parsedEmail.html
//parsedEmail.attachments[0].filename

Unfortunately I could not reproduce this yet. A reproducible test case or a pull request would be welcome!


Add grasp to pypi so it's installable using pip/pipsi

It would be great to be able to install grasp_server as a pypi package to facilitate installing it and including in service scripts for macOS.

better late then never :) it's on pypi now
https://github.com/karlicoss/grasp?tab=readme-ov-file#setup


[easy] Fix logs to use the normalized instruction counter

Checked the e2e, using the next_instruction_counter() does not change anything for either SyscallReadPreimage nor SyscallWriteHint (the only instructions which were setting the memory access to be self.instruction_counter + 1 before the fixme of https://github.com/o1-labs/proof-systems/pull/2311)


Complete QCxMS workflow

This PR creates an end-to-end workflow to predict semi-empirical quantum chemistry mass spectra. An SDF file of two small molecules: H2O and Methane, is included for testing purposes.

Link to the workflow: https://usegalaxy.eu/u/fec10a00c58544209329288deb71ea45/w/end-to-end-spectra-predictions-multi-atoms-dataset
Link to history test: https://usegalaxy.eu/u/fec10a00c58544209329288deb71ea45/h/end-to-end-spectra-predictions-multi-atoms-dataset

closes https://github.com/RECETOX/workflow-testing/issues/33

The workflow is up to date with the new release of the tools QCxMS neutral run and QCxMS get results.


[NTTY] Add new object type NTTY

Hi @wurzka,
thank you for the review!
I applied all the suggestions, it makes sense for me.


📝 Adds alexas contact information docs

Proposed changes
Added docs on how to retrieve alexa skill contact information (name, email, mobile number) via the Alexa Contact Permissions API.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[x] My code follows the code style of this project
[x] My change requires a change to the documentation
[x] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[x] All new and existing tests passed

Awesome! Thanks


Consider adding refactored convenience methods

We previously had something like:
impl Verfer {
  new_with_code_and_raw(...) {
  }

  // ...

  new_with_qb2(...) {
  }
}

Which was removed to consolidate an API closer to the Python variant. During this process it was noted we had this pattern:
impl Verfer {
  new_with_code_and_raw(...) {
    // delegate to Trait
    // custom logic
  }

  // ...

  new_with_qb2(...) {
    // delegate to Trait
    // custom logic
  }
}

The refactor left us with:
impl Verfer {
  new(...) {
    // delegate to Trait
    // custom logic
  }
}

and allows simplication of the implementation of the convenience methods (should we want them back - which I think we do for Rust coders) like this:
impl Verfer {
  new(...) {
    // delegate to Trait
    // custom logic
  }

  new_with_code_and_raw(...) {
    Self::new(...)
  }

  // ...
}

This is better structure, and provides a better experience for Rust developers.

Notes about the difference in the API:

the new API is consistent to program and easy to test against but hard to review, as the editor will label params but github won't.
the existing implementation won't make it through the uniffi interface we've been playing with

i spiked this out


ColorScheme.SetAttribute uses a static var as a lock

Current code:
		bool preparingScheme = false;

		Attribute SetAttribute (Attribute attribute, [CallerMemberName] string callerMemberName = null)
		{
			if (!Application._initialized && !preparingScheme)
				return attribute;

			if (preparingScheme)
				return attribute;

			preparingScheme = true;

			switch (caller) {
			case "TopLevel":
				switch (callerMemberName) {
				case "Normal":
					HotNormal = Application.Driver.MakeAttribute (HotNormal.Foreground, attribute.Background);
					break;
				case "Focus":
					HotFocus = Application.Driver.MakeAttribute (HotFocus.Foreground, attribute.Background);
					break;
....
			}
			preparingScheme = false;
			return attribute;
		}

Shouldn't preparingCode be a lock?
While I'm here, the logic in the switch statement has always confused me. I do not understand the intent! It seems arbitrary to be making style decisions in code like this. Can anyone explain clearly?

I think i'm figuring out. New docs:
/// <summary>
		///
		/// </summary>
		/// <remarks>
		/// <para>
		/// If <see cref="Application"/> is not initialized yet and <see cref="_preparingScheme"/> is set to <see langword="false"/>
		/// then the attribute is returned as is (because it's not possible to modify a Scheme wihtout a valid Driver).
		/// </para>
		/// <para>
		/// If <see cref="Application"/> has been initialized and <see cref="_preparingScheme"/> is set to <see langword="true"/> it means
		/// <see cref="SetAttribute(Attribute, string)"/> is being called recursively and the attribute is returned as is.
		/// </para>
		/// <para>
		/// Otherwise, the attribute returned is created by Driver.MakeAttribute, using the
		/// foreground and background colors of the attribute.
		/// </para>
		/// </remarks>
		/// <param name="attribute">THe attribute to set.</param>
		/// <param name="callerMemberName">The name of the property (e.g. <see cref="Normal"/> or <see cref="Focus"/>
		/// that called the function. This is used to determine how to make the attribute.</param>
		/// <returns>The attribute that was set.</returns>
		Attribute SetAttribute (Attribute attribute, [CallerMemberName] string callerMemberName = null)

And... I just set a breakpoint here:

And I ran all unit tests and UICatalog and it was never hit.
Is this all defunct code!??!??!

This was made to create automatic attributes, but since they are been creating on the Init method this wasn't used anymore. I don't know who create the preparingScheme flag but for sure was to avoid running the bellow code. I think this private method can be deleted and set the attributes directly with the value.

After diving in deeper, I agree!


[Feature Request] [SQL Syntax] Collection range specifiers

Explanation
Currently when selecting a collection one can choose a specific or multiple indices:
SELECT path[1,3] from shortestPath(#8:32, #8:10)

However, what if we want to select all elements except the 1st? or all elements except the last?
Example
Syntax:
[<start>, <end>]

start: index of first element. use negative numbers to select an element from the end of the collection.
end: index of the last element. Use negative numbers to select an element from the end of the collection. if omitted, all elements until the end are selected

note: start may be bigger than end, which will return "reverse elements".
Select all elements from index 2 until 4

SELECT path[2,4] from shortestPath(#8:32, #8:10)

Select all elements from index 1 until the end

SELECT path[1,] from shortestPath(#8:32, #8:10)

Select all elements from 1 item before the last, up until the 2nd item:

SELECT path[-1,1] from shortestPath(#8:32, #8:10)

Select all elements from 2 items before the last, up until 1 item before the last:

SELECT path[-2,-1] from shortestPath(#8:32, #8:10)

Use Case
I'm using shortestPath which return an array of elements, but:

The last element is a known element and I don't need it. Since I'm returning a lot of values, I want to omit it.
The first element is the target, so I want more information about it
the elements in between are less important, so I only want a few fields about them.

The result of my query once implemented:
SELECT path[0]:{*} as target, path[1,-1]:{@RID, name} as between FROM (select shortestPath(#8:32, #8:10))
Which will return:
| target | between |
---------------------
| {#8:32, name, location, etc} | [{#8:24, name}, {#8:5, name}] |

Hi @matanshukry
the [a..b] and [a...b] syntax is already supported, see https://orientdb.com/docs/3.0.x/sql/SQL-Syntax.html#square-bracket-filters
But as of now it does not use negative indexes to select from the end (it's a good idea btw)
I'll keep this issue open and and mark it as an enhancement. I'll see if I can improve it further
Thanks
Luigi


compiling to Android works, however the app crashes at startup

Use instructions at
https://www.reddit.com/r/Electrum/comments/akxh5o/how_to_compile_an_electrum_testnet_apk/
After installing the docker, also copy from
Git clone https://github.com/spesmilo/electrum.git
cp -r  /home/stefan/Downloads//electrum/electrum/gui gui/
then
docker run -it --rm     --name electrum-android-builder-cont    -v $PWD:/home/user/wspace/electrum     -v ~/.keystore:/home/user/.keystore    --workdir /home/user/wspace/electrum     electrum-android-builder-img     ./contrib/make_apk
compiles
to an apk (see attachment).
However it crashes on my androids ;(
Electrum-<IP_ADDRESS>-debug.zip

Are you trying this with electrum or cb-client-wallet? Any error logs you can share?

Compiled using instructions of https://www.reddit.com/r/Electrum/comments/akxh5o/how_to_compile_an_electrum_testnet_apk/  (they are more up-to-date)
compiled cb-client-wallet
Error, (selected from logcat):
05-09 15:58:00.348  3925  3975 E WindowManager: RemoteException occurs on reporting focusChanged, w=Window{4d379b4 u0 org.electrum.electrum/org.kivy.android.PythonActivity}
05-09 15:58:00.348  3925  3975 E WindowManager: android.os.DeadObjectException
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.os.BinderProxy.transactNative(Native Method)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.os.BinderProxy.transact(Binder.java:1143)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.view.IWindow$Stub$Proxy.windowFocusChanged(IWindow.java:500)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at com.android.server.wm.WindowState.reportFocusChangedSerialized(WindowState.java:3879)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at com.android.server.wm.WindowManagerService$H.handleMessage(WindowManagerService.java:5426)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.os.Handler.dispatchMessage(Handler.java:106)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.os.Looper.loop(Looper.java:214)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at android.os.HandlerThread.run(HandlerThread.java:65)
05-09 15:58:00.348  3925  3975 E WindowManager: 	at com.android.server.ServiceThread.run(ServiceThread.java:44)
05-09 15:58:00.351  5258  5258 D PageOverlayMover: updateActivityLifecycleState : 1
05-09 15:58:00.351  5258  5258 D ViewRootImpl@318cc9a[LauncherActivity]: setWindowStopped(false) old=false
05-09 15:58:00.351  5258  5258 V Launcher: Launcher.onResume()
Now I think of it, maybe we need to set testnet:
https://www.reddit.com/r/Electrum/comments/akxh5o/how_to_compile_an_electrum_testnet_apk/

Not sure sorry, we currently don't support an Android version. You might want to check with upstream electrum.


Question: the ClientInterface::eval signature and PhpRedis

Maybe it's my mistake, but I've thought that the eval method signature that provided by the Predis should be compatible with the PhpRedis one. Since #762 it's not and in particular I have Phan errors about this.
Should the method signature be fixed in the ClientInterface annotation, or is it OK and I'm doing something wrong?

There is no API compatibility between Predis and PhpRedis and likely will never be.


error when saving TFR: unsupported type <class 'datetime.datetime'>

I am computing power on epochs and want to save the output. Since I updated MNE to
0.20.dev0, I get the following error:
  File "/home/sh254795/anaconda3/lib/python3.7/site-packages/mne/externals/h5io/_h5io.py", line 225, in _triage_write
    raise TypeError(err_str)

TypeError: unsupported type <class 'datetime.datetime'> (in <class 'list'>[0][1]["info"]["meas_date"])

Here is an example that reproduces the error:
import mne
from mne.datasets import sample

print(__doc__)

###############################################################################
# Set parameters
# --------------
data_path = sample.data_path()
raw_fname = data_path + '/MEG/sample/sample_audvis_raw.fif'
tmin, tmax, event_id = -0.3, 0.6, 1

# Setup for reading the raw data
raw = mne.io.read_raw_fif(raw_fname)
events = mne.find_events(raw, stim_channel='STI 014')

include = []
raw.info['bads'] += ['MEG 2443', 'EEG 053']  # bads + 2 more

# picks MEG gradiometers
picks = mne.pick_types(raw.info, meg='grad', eeg=False, eog=True,
                       stim=False, include=include, exclude='bads')

# Load condition 1
event_id = 1
epochs = mne.Epochs(raw, events, event_id, tmin, tmax, picks=picks,
                    baseline=(None, 0), preload=True,
                    reject=dict(grad=4000e-13, eog=150e-6))

# Take only one channel
ch_name = 'MEG 1332'
epochs.pick_channels([ch_name])

decim = 5
freqs = np.arange(8, 40, 2)  # define frequencies of interest
sfreq = raw.info['sfreq']  # sampling in Hz

power, itc = mne.time_frequency.tfr_morlet(epochs, freqs=freqs, return_itc=True, n_cycles=4., decim=decim)

power.save('TMP', overwrite=True)

I can replicate. let me have a look

in the mean time you can do:
del power.info['meas_date']
before saving.

Fixed in #7284


develop

Changes Unknown when pulling e5e94fc6ed02d773023659f25031f557c6d55f79 on develop into ** on master**.

Changes Unknown when pulling af47888b283d38ee3b1d631655ce2045324c1d38 on develop into ** on master**.

Changes Unknown when pulling 1b7cb080ee0c6d32742417c1ca10a3583d1c0502 on develop into ** on master**.


instance.child() should set the level property in pino

Fixes #55.
@davidmarkclements please review.
Benchmarks stays the same.

Coverage remained the same at 89.243% when pulling 49763a862deac4a49a5d40d2e43fce889873f0fa on child-level-constructor into e7d91d4a647c1e0e821aaaffbca1676baf785c02 on master.

Coverage increased (+2.8%) to 92.032% when pulling edbd4339a5be27507a346252a3715082d54698b6 on child-level-constructor into e7d91d4a647c1e0e821aaaffbca1676baf785c02 on master.

good to go man


refactor MolStandardize::PipelineOptions::allowEmptyMolecules

What does this implement/fix? Explain your changes.
This PR implements an alternative cleanup of the MolStandardize::Pipeline workflow in relation to the handling of molecule with no atoms, keeping the allowEmptyMolecules option, but moving the conditional logic into RDKitValidation.
Any other comments?

@greglandrum if you liked #7694 better, we can reopen it and decline this one. Both solutions would be fine for me.

I didn't remove allowEmptyMolecules from PipelineOptions because with the implemented changes it can be now used to control the behavior of RDKitValidation (that was previously hard-coded to reject molecules with no atoms).
Making RDKitValidation configurable reduces the functional overalap with NoAtomsValidation and is a less drastic solution compared to removing allowEmptyMolecules from PipelineOptions altogether.

Ok, I must have missed something by reviewing just using the github UI. Let me pull the code and actually look at it.

Ok, thanks. If you think PipelineOptions::allowEmptyMolecules would never be used, we can just remove it and leave RDKitValidation as it is (i.e., reopen #7694).

Ok, thanks. If you think PipelineOptions::allowEmptyMolecules would never be used, we can just remove it and leave RDKitValidation as it is (i.e., reopen #7694).

Nope, I think it's good as is.

Thank you, @greglandrum !


Reworking some old functions to better support returned values after using NS-PHP package

What's this PR do?
This PR updates the dosomething_helpers_get_current_user() function so that it returns the user as NorthstarUser object thanks to the newly added NS-PHP package. It also adds an additional parameter to the function so that the user information can be returned as an array. This is used for outputting user information into the Drupal.settings.dsUser object for use by JavaScript on the front-end.
How should this be reviewed?
👓 👀  all looks good?
Relevant tickets
Refs https://github.com/DoSomething/TeamRocket/issues/11
Checklist

[ ] Tested on staging.
[ ] Pinged a PM if this is a larger PR that would benefit from some additional testing love.

@DFurnes @angaither

@DFurnes found a small bug while testing. See latest commit for fix and also did a little extra cleanup!


dotnet issues onPrem since update to AL .5.0.254558

Describe the bug
Working code with AL extensions 4.0. does no longer work with AL extension 5.0.
Code in question. dotnet assembly definition and usage.
To Reproduce
Steps and to reproduce the behavior:

use mentioned AL extensions . AL:Go - Version 4.0
change your setting to use the probingpath
    "al.assemblyProbingPaths": [      "./.netpackages",      "c:/windows/assembly/"  ]
add a dotnet object with the following content:

dotnet
{
    assembly("System")
    {
        Version = '<IP_ADDRESS>';
        Culture = 'neutral';
        PublicKeyToken = 'b77a5c561934e089';

        // will not work
        type("System.Collections.Queue"; "Queue") { }
        type("System.Convert"; "SystemConvert") { }

        // will work
        type("System.Security.Cryptography.X509Certificates.StoreName"; "StoreName") { }

    }
}

Expected behavior
The System is able to find the assemblies and use them.
5. Versions:

AL Language: 5.0.254558

Hi! There are a few things at play here:

System.Collections.Queue is not in the System assembly https://docs.microsoft.com/en-us/dotnet/api/system.collections.queue?view=netframework-4.8
It used to work in older versions of the VSIX due to a but in the .NET Interop system implementation when the compiler was running on the .NET Framework
The bug might still be there if you "al.useLegacyRuntime":true and restart your enlistment. If that works, keep in mind that the legacy runtime is about 2-3 times slower than the new one.

Considering that the assembly is not part of System, at least not according to the public docs, I would recommend fixing the code.

That's interesting! so you moved away from the old .Net framework? Isn't that worth to mention somewhere?! :D
Apart from that: it was just an example on the things we have problems with. the list is a bit longer... i will check those if they are also not (any)longer part of the assembly we used to use.
I have tested with "al.useLegacyRuntime":true - this "fixes" all the errors we got! Time to do some homework (literally). Thanks for the quick answer!
Will this introduce some compatibility problems with using al 5.0 against newer BCs (v.16) v5 apps vs older BC (v3, v4 ?) or is it just for VS Code? So when the App gets published - v15 was happy to compile and run it - will v16 now fail as it will not be able to handle the assemblies? do I need to set BC v16 also in "legacy" mode?

booomer  - I was to fast - it did not help :/
all those do not work:
         type("System.Collections.Queue"; "Queue") { }         type("System.Convert"; "SystemConvert") { }         type("System.DateTime"; "SystemDateTime") { }         type("System.Environment"; "Environment") { }         type("System.Text.UTF8Encoding"; "UTF8Encoding") { }         type("System.AppDomain"; "AppDomain") { }         type("System.Net.ICredentials"; "ICredentials") { }         type("System.Reflection.Assembly"; "Assembly") { }         type("System.Object"; "Object") { }         type("System.Collections.Generic.IList1"; "IList_Of_T") { }
type("System.Collections.Generic.ICollection1"; "ICollection_Of_T") { }         type("System.Nullable1"; "Nullable_Of_T") { }
type("System.Collections.Generic.List1"; "List_of_T") { }         type("System.Collections.Generic.Dictionary2"; "Dictionary_Of_T_U") { }
type("System.Collections.Generic.IEnumerable1"; "IEnumerable_Of_T") { }         type("System.Collections.Generic.KeyValuePair2"; "KeyValuePair_Of_T_U") { }
`

Moving them to the assembly mscorlib helped! (while legacy is enabled...)
turning it off seems also to work but no I get "ambiguous" references. Wasn't that allowed in the past to have the same reference and name...so this is no longer ok it seems ...

What do you mean by ambiguous reference?

like this guy here:
type("System.Environment"; "Environment") { }  that is in our extensions dotnet.al but it is also part of SystemApplications dotnet.al ...

Hi Kristof! Coming back after a long vacation: can you provide a self-contained project that we can use to reproduce this problem? Maybe there's something else on top of the transition from .NET Framework to .NET Core.

Hi Alex,
i will try and let you know.

In case anyone else like me ends up here and is super confused at what the resolution is, you have to know what parts are for what assemblies.
For example, if you are using OnPrem and DotNet to generate files, and your older code has a batch of types like:
assembly(System)
{
    type(System.IO.File; FTPFileStream) { }
    type(System.IO.FileStream; RealFileStream) { }
    type(System.Text.Encoding; dotEncoding) { }
}

Then those, despite the name, don't go under the assembly(System).  You would instead reference them like assembly(mscorlib).
This can be a little challenging for a novice to sort out, since Microsoft's site on System.IO says the assembly is "System.Runtime" (which does not work here) but almost every 3rd party reference site gives more clarity as to the proper 'assembly' name.
Your dotnet wrapper can have multiple assembly nodes, so a wrapper like this is perfectly valid:
dotnet
{
    assembly(System)
    {
        type(System.Net.FtpWebRequest; FTPRequest) { }
        type(System.Net.FtpWebResponse; FTPResponse) { }
        type(System.Net.NetworkCredential; Credentials) { }
        type(System.Net.FtpStatusCode; StatusCode) { }
        type(System.Text.RegularExpressions.Regex; dotRegex) { }
    }
    assembly(mscorlib)
    {
        type(System.IO.Stream; ResponseStream) { }
        type(System.IO.StreamWriter; dotStreamWriter) { }
        type(System.IO.File; FTPFileStream) { }
        type(System.IO.FileStream; RealFileStream) { }
        type(System.Text.Encoding; dotEncoding) { }
        type(System.Collections.ArrayList; FilesList) { }
    }
}


Docker compose quick start?

Hi, i cant figure out how to make this work with docker compose.
My matrix is runing on separate compose file
I created new folder:
mx-puppet-steam/
./docker-compose.yml
version: "3.4"
services:
  mx-puppet-steam:
    hostname: mx-puppet-steam
    image: icewind1991/mx-puppet-steam
    restart: always
    volumes:
      - ./config.yaml:/data/config.yaml

    networks:
      - internal
networks:
  internal:

./config.yaml
bridge:
  # Port to host the bridge on which your homeserver will connect to
  port: 6000
  # Address for the bridge to bind to; if running as a Docker container, you
  # probably want <IP_ADDRESS> here
  bindAddress: <IP_ADDRESS>
  # Name of your homeserver
  domain: matrix.example2.com
  # URL where the bridge can connect to your homeserver
  homeserverUrl: https://matrix.example2.com

provisioning:
  # Which users are allowed to issue commands to @_steampuppet_bot
  # whitelist:
  #  - "@*:matrix.example.com"
  # A blacklist can be used instead:
  # blacklist:
  #   - "@username:matrix\\.myhomeserver\\.com"

  # Shared secret for the provisioning API for use by integration managers.
  # If this is not set, the provisioning API will not be enabled.
  #sharedSecret: random string
  # Path prefix for the provisioning API. /v1 will be appended to the prefix automatically.
  apiPrefix: /_matrix/provision

presence:
  # Bridge Steam online/offline status
  enabled: true

  # How often to send status to the homeserver in milliseconds
  interval: 5000

Then i run: docker compose up -d
It runs, but logs say it cant connect to my matrix. Ok, so now i have to register the bridge with my matrix homeserver.yaml.
But how? Where can i get the registration file? How to expose it from docker container? What lines do i need to add to my homeserver.yaml ???
EDIT:
I got it to run and kinda connect to my matrix.
Had to do the following:
Instead of creating ./config.yaml - > ./data/config.yaml  and then bind the folder data. Also removed the network thing and added ports.
./docker-compose.yml
version: "3.4"
services:
  mx-puppet-steam:
    hostname: mx-puppet-steam
    image: icewind1991/mx-puppet-steam
    restart: always
    volumes:
      - ./data/config.yaml:/data/config.yaml
      - ./data:/data
    ports:
      - "6000:6000"

Then in the data folder the registration file was generated.
./data/steam-registration.yaml
Then i copied the registration file to the same folder as my matrix homeserver.yaml (dendrite.yaml) and added the path to the:
homeserver.yaml (dendrite.yaml)
# Configuration for the Appservice API.
app_service_api:
  # Disable the validation of TLS certificates of appservices. This is
  # not recommended in production since it may allow appservice traffic
  # to be sent to an insecure endpoint.
  disable_tls_validation: false

  # Appservice configuration files to load into this homeserver.
  config_files:
    - ./steam-registration.yaml

After reloading both composes, the mx container shows me this:

Sep-13 19:07:59.619 [Store] info: Starting DB Init
Sep-13 19:07:59.621 [Store] info: Filename present in config, using sqlite
Sep-13 19:07:59.621 [SQLite3] info: Opening database.db
Sep-13 19:07:59.626 [Store] info: Database schema version is 16, latest version is 16
Sep-13 19:07:59.627 [PuppetBridge] info: Starting application service....
Sep-13 19:07:59.630 [bot-sdk-MatrixHttpClient] info: (REQ-1) POST https://matrix.example2.com/_matrix/client/r0/register
Sep-13 19:07:59.662 [PuppetBridge] info: Application service started!
Sep-13 19:07:59.662 [PuppetBridge] info: Setting bridge user data...
Sep-13 19:07:59.662 [bot-sdk-MatrixHttpClient] info: (REQ-2) GET https://matrix.example2.com/_matrix/client/r0/profile/%40_steampuppet_bot%3Aexample2.com
Sep-13 19:07:59.688 [PuppetBridge] info: Activating users...
Sep-13 19:07:59.689 [PresenceHandler] info: Starting presence handler with new interval 5000ms
matrix@matrix:~/mx-puppet-steam$

Now how do i control it from my element app?
I created private space, private room, and then i sent a message:
@_steampuppet_bot:matrix.matrix.example2.com

but nothing happened.
What am i missing?
EDIT
So i invited the bot to my private room but it shows offline (the bot is offline) and when i mention the bot nothing happens
Thanks in andvance!

I think it's related to this.
https://github.com/icewind1991/mx-puppet-steam/issues/44


refactor(types): fix query generic

This PR fixes the type so that the result is correctly returned when using a type generic.

I just did this on github.com quickly, so let me know if there's any failing tests with this change! :)

lgtm - but is a breaking change so bump to 0.6 is required at release.


Fix mongo session persistence for failover

Following a large number of tests failing to pass in the past week investigations demonstrated that the cause was a defect in session persistence and revival when using express-session and connect-mongo. It is not clear yet why this problem only appears to show up only now, replacement of connect-mongo by connect-mongodb-session yielded the same result suggesting the source cause might reside in express-session.
Due to time constrains on the release of IDEAFast we have disabled persistence on the current develop branch and will be tracking the issue here. The objective is to revert the change and re-enable session persistence.

The patching move was done in #602 which is the PR that aims to be reverted when a solution has been found.


Patch 2

Added game port to template as per the enshrouded wiki.
https://enshrouded.zendesk.com/hc/en-us/articles/16055441447709-Dedicated-Server-Configuration

Hopefully this is correct. I've tried to replicate the technique used on another template in the repo :)

That's outdated. They deprecated that ports a few months ago

Aah, my mistake! Apologies.


How to align text content

Hi, how to align the text cell to left or right.

hi @mikelode, how  to do this?

hi @mikelode, how to do this?

with something similar like this:
#myGrid .jexcel > tbody > tr > td{
text-align: left;
padding-top: 0px;
padding-bottom: 0px;
}


Make the sender name appear in the bold weight font, fix overflow issues

This makes the message sender name appear in the correct bold weight font, and fixes overflow issues in the message sender header as well as the sidebar footer when usernames are super long.
Fixed Issues
N/A, but relevant discussion here: https://expensify.slack.com/archives/C03U7DCU4/p1608314618489300
Tests

Open up a chat and start sending messages, make sure the sender name appears in bold
Sign up with a super long email address and make sure it is truncated properly in the Left Hand Navigation footer, and make sure the sender line with Sender Name + Timestamp breaks to two lines on smaller viewports (mobile).

Screenshots

Reassigning PullerBear since Bondy is OOO this week.

Verified on Desktop, Web, iOS, Android, mobile Web 👍


No .c line in traceback during module init

Since the introduction of the disabling of .c lines in tracebacks, such line numbers are unconditionally not shown during the module init code:
      __Pyx_AddTraceback("init sage.structure.coerce_maps", 0, __pyx_lineno, __pyx_filename);

Since I find these .c line numbers really handy for debugging, I consider it a serious regression.

I guess the reason for that disabling should be fixed anyway by #2199.

I should also add that C lines are particularly useful during module init, because the init logic is quite complicated and the Cython traceback is not very useful there.

@jdemeyer we're currently building bleeding-edge cython with flags to make sure that it doesn't segfault. At what point can we use a stable version of cython again? currently we have to do this: https://github.com/tardis-sn/tardis/blob/master/.travis.yml


sig-network: clustered failues over several lanes

What happened

Flakefinder report from Fri Sep 20th 2024 showed clustered test failures for sig-network lanes 1 2 3
The first two clustered failures were occurring on batch runs, associated PRs are:
1

https://github.com/kubevirt/kubevirt/pull/12355
https://github.com/kubevirt/kubevirt/pull/12835

2

https://github.com/kubevirt/kubevirt/pull/12694
https://github.com/kubevirt/kubevirt/pull/12857
https://github.com/kubevirt/kubevirt/pull/12862

Clustered failure 3 occurred on PR https://prow.ci.kubevirt.io/pr-history/?org=kubevirt&repo=kubevirt&pr=12599
EDIT Addition 1
Flakefinder report from Sun Sep 22nd 2024 showed another clustered failure:
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12858/pull-kubevirt-e2e-k8s-1.30-sig-network/1837845954506526720
EDIT Addition 2
Flakefinder report from Sat Sep 21st 2024 showed two other clustered failures:

https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12664/pull-kubevirt-e2e-k8s-1.29-sig-network/1837097975784935424
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12664/pull-kubevirt-e2e-k8s-1.31-sig-network/1837097983359848448

EDIT Addition 3
FlakeFinder report from Thu Sep 19th 2024
showed another clustered failure: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.29-sig-network/1836565906797367296

/kind flake
/priority critical-urgent

/sig network

Additional context
Add any other context about the problem here.
Flake Action Plan
As the assignee, thoroughly review the issue and put the resulting report as comment into this issue.
Then decide on one of the following actions:
The flake is a bug

Add label commenting /triage accepted
Create a PR to fix the bug
Reference this issue in the PR
Keep this issue open until the testcase does not fail anymore

The flake is non-critical or an issue that is hard to fix

Quarantine the test by creating a pull request assigning [QUARANTINE] tag to test name and Quarantine decorator to the test
Reference this issue on the PR

There was an infrastructure issue
An infra issue is anything "below" the testcase

Add label commenting /triage infra-issue
Close the issue, adding a comment with details about the infrastructure issue.

After the flake has been fixed, document the learning from it inside the fix PR

Looks like something bad happened on the 20th, but I cannot see it actively happening now.
The sig-network team is a bit overloaded at the moment, we will try to look into this later this week.
If this starts occurring more often, please let me know.

Added
Addition 1
Flakefinder report from Sun Sep 22nd 2024 showed another clustered failure:
prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12858/pull-kubevirt-e2e-k8s-1.30-sig-network/1837845954506526720

@dhiller , is there some correlation with changes done to the infrastructure? E.g. kubevirtci bump?

Added
Addition 2
Flakefinder report from Sat Sep 21st 2024](https://storage.googleapis.com/kubevirt-prow/reports/flakefinder/kubevirt/kubevirt/flakefinder-2024-09-21-024h.html) showed two other clustered failures:

https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12664/pull-kubevirt-e2e-k8s-1.29-sig-network/1837097975784935424
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12664/pull-kubevirt-e2e-k8s-1.31-sig-network/1837097983359848448

@dhiller , is there some correlation with changes done to the infrastructure? E.g. kubevirtci bump?

I'll check

Hey @dhiller I am going to take a look

Added
Addition 3
FlakeFinder report from Thu Sep 19th 2024
showed another clustered failure: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.29-sig-network/1836565906797367296

/cc

/cc @qinqon

@dhiller it's not related but looks like alpine images do not have lspci
{"component":"portforward","kind":"VirtualMachineInstance","level":"info","msg":"[{1 lspci\r\n-ash: lspci: not found\r\ntestvmi-9mdzl:~#  [lspci\r\n-ash: lspci: not found\r\ntestvmi-9mdzl:~#  \r\n-ash: lspci: not found\r\ntestvmi-9mdzl:~ # ]}

That may stop reporter from dumping stuff ?

I see some port forward errors too
APIVersion:\"kubevirt.io/v1\", ResourceVersion:\"31182\", FieldPath:\"\"}): type: 'Normal' reason: 'Started' VirtualMachineInstance started.","pos":"watcher.go:150","timestamp":"2024-09-19T01:22:16.501195Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:16.916976Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:16.917038Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:21.917332Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:21.917392Z"}
{"component":"portforward","level":"info","msg":"Sent: \"fedora\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:41.925377Z"}
{"component":"portforward","level":"info","msg":"Sent: \"fedora\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:43.917883Z"}
{"component":"portforward","level":"info","msg":"Sent: \"sudo su\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:45.918043Z"}
{"component":"portforward","level":"info","msg":"Sent: \"stty cols 500 rows 500\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:45.999965Z"}
{"component":"portforward","level":"info","msg":"Sent: \"echo $?\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.009567Z"}
{"component":"portforward","level":"info","msg":"Sent: \"dmesg -n 1\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.013670Z"}
{"component":"portforward","level":"info","msg":"Sent: \"echo $?\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.021014Z"}
{"component":"portforward","level":"info","msg":"read closing down: EOF\n","pos":"expect.go:1081","timestamp":"2024-09-19T01:22:46.025422Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.050706Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.050760Z"}
{"component":"portforward","level":"info","msg":"read closing down: EOF\n","pos":"expect.go:1081","timestamp":"2024-09-19T01:22:46.054432Z"}
{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.077467Z"}
{"component":"portforward","level":"info","msg":"Sent: \"nc  -klp 1500 -e echo -e \\\"Hello World!\\\"&\\n\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.080702Z"}
{"component":"portforward","level":"info","msg":"Sent: \"echo $?\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:22:46.089613Z"}
{"component":"portforward","level":"info","msg":"read closing down: EOF\n","pos":"expect.go:1081","timestamp":"2024-09-19T01:22:46.095868Z"}
could not find healty compute container for pod virt-launcher-testvmi-8z88p-wxs6d
failed to execute command [/bin/bash -c pgrep -f "monitor.*uid 98633dfa-1ec9-48ce-a639-f2d7a85d11d1"] on virt-handler-nl54h, stdout: , stderr: , error: command terminated with exit code 1{"component":"portforward","level":"info","msg":"Sent: \"\\n\"\n","pos":"expect.go:1158","timestamp":"2024-09-19T01:24:29.103106Z"}

First failing test from https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.29-sig-network/1836565906797367296
vm: testvmi-9mdz
artifacts at: /logs/artifacts/k8s-reporter/4/1_*
[FAILED] [173.435 seconds]
01:07:54: [sig-network] [rfe_id:253][crit:medium][vendor:cnv-qe@redhat.com][level:component]Expose Expose service on a VM Expose ClusterIP service [label:masquerade_binding_connectivity]Should expose a Cluster IP service on a VMI and connect to it [It] [test_id:1531] over default IPv4 IP family [sig-network, Expose]
01:07:54: tests/network/expose.go:136

01:07:54:   STEP: Exposing the service via virtctl command @ 09/19/24 01:05:24.915
01:07:54:   STEP: Getting and validating the cluster IP given for the service @ 09/19/24 01:05:24.928
01:07:54:   STEP: Iterating over the ClusterIPs and run hello-world job @ 09/19/24 01:05:24.93
01:07:54:   STEP: Starting a job which tries to reach the VMI via the 1 ClusterIP @ 09/19/24 01:05:24.93
01:07:54:   STEP: Waiting for the job to report a successful connection attempt @ 09/19/24 01:05:24.934
01:07:54:   [FAILED] in [It] - tests/network/expose.go:134 @ 09/19/24 01:07:24.938
01:07:54:   On failure, artifacts will be collected in /logs/artifacts/k8s-reporter/4/1_*
01:07:54:   STEP: Collecting Logs due to failure @ 09/19/24 01:07:24.938

01:07:54:   [FAILED] Unexpected error:
01:07:54:       <*errors.errorString | 0xc00627ef00>:
01:07:54:       failed running job against service with ClusterIP <IP_ADDRESS>
01:07:54:       {
01:07:54:           s: "failed running job against service with ClusterIP <IP_ADDRESS>",
01:07:54:       }
01:07:54:   occurred
01:07:54:   In [It] at: tests/network/expose.go:134 @ 09/19/24 01:07:24.938

Adding wrapping the Job error here https://github.com/kubevirt/kubevirt/pull/12899.
@dhiller do you know if it would be possible to dump the Jobs too ? looks like we are not dumping them

Clustered failure from Monday: https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12667/pull-kubevirt-e2e-k8s-1.29-sig-network/1838121657995104256

@EdDev @dhiller
Those tests passed on the presubmit run for the merge PR: https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12677/pull-kubevirt-e2e-k8s-1.30-sig-network/1836411846844747776
I haven't went through each one by one but on this run everything passed, sig network lane also ran twice on that PR before it got merged so its unlikely to be infra related
If you think otherwise are there any areas you recommend i start checking or suspect might be the root cause ?

Merged PR with error wrapping at e2e tests:

https://github.com/kubevirt/kubevirt/pull/12934

Following https://github.com/kubevirt/kubevirt/issues/12898#issuecomment-2370242323 and #12934, we have the full error message in the following job:
https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12934/pull-kubevirt-e2e-k8s-1.29-sig-network/1839027779501297664
Error message:
{ Failure tests/network/expose.go:701
Unexpected error:
    <*fmt.wrapError | 0xc0067fe520>:
    failed running job against service with ClusterIP <IP_ADDRESS>: job netcatgrvmf timeout reached, status: {Conditions:[] StartTime:2024-09-25 20:32:53 +0000 UTC CompletionTime:<nil> Active:1 Succeeded:0 Failed:0 Terminating:0xc004677ad4 CompletedIndexes: FailedIndexes:<nil> UncountedTerminatedPods:&UncountedTerminatedPods{Succeeded:[],Failed:[],} Ready:0xc004677ad8}, err: context deadline exceeded
    {
        msg: "failed running job against service with ClusterIP <IP_ADDRESS>: job netcatgrvmf timeout reached, status: {Conditions:[] StartTime:2024-09-25 20:32:53 +0000 UTC CompletionTime:<nil> Active:1 Succeeded:0 Failed:0 Terminating:0xc004677ad4 CompletedIndexes: FailedIndexes:<nil> UncountedTerminatedPods:&UncountedTerminatedPods{Succeeded:[],Failed:[],} Ready:0xc004677ad8}, err: context deadline exceeded",
        err: <*errors.errorString | 0xc006900ed0>{
            s: "job netcatgrvmf timeout reached, status: {Conditions:[] StartTime:2024-09-25 20:32:53 +0000 UTC CompletionTime:<nil> Active:1 Succeeded:0 Failed:0 Terminating:0xc004677ad4 CompletedIndexes: FailedIndexes:<nil> UncountedTerminatedPods:&UncountedTerminatedPods{Succeeded:[],Failed:[],} Ready:0xc004677ad8}, err: context deadline exceeded",
        },
    }
occurred

Indicating timeout reached for the Job to report success, although we can see the Job actually executed and succeeded to connect to the guest UDP server.
Tests involve executing a Job that is timed out with this error seem to suffer from what looks like a general slowness in the env/KAPI, causing various tests to fail due to timeouts and "deadline exceeded" errors.

@ormergi can you put a PR to also dump the Jobs at k8s reporter ?

According to SearchCI [1], first occurrence of the these failures seem on sig-net periodic labe on 19 September [2].
I went through most failing jobs mentioned so far, it looks like all tests fails due to the same root cause which is general slowness in the cluster, causing pods/jobs to not become ready or switch to desired state on time, causing tests to fail due to timeouts / deadline exceeded.
It looks like something bad happens to the tests cluster causing general slowness.
Can someone please check what was the nodes state during the failing jobs?
As mentioned above, before this commit there was a batch merge of few PRs, one of them is KCI bump https://github.com/kubevirt/kubevirt/pull/12677
The KCI bump is pretty large, notable changes were VM node OS update and configuring etcd to avoid writing to disk.
In practice, the VM OS upgrade was merged few days before first failure - 16 September https://github.com/kubevirt/kubevirt/pull/12840
Could these be related some how?
Note about tests of  virtctl-expose-service suite (start with Expose Expose service...), almost half of the failing tests in each job are expose tests, which are going to be removed soon by https://github.com/kubevirt/kubevirt/pull/12855
The job ran against commit 549216c7810e279a08f2260d9043cb894c88d247.
[1] https://search.ci.kubevirt.io/?search=failed+running+job+against+service+with+ClusterIP&maxAge=336h&context=1&type=bug%2Bissue%2Bjunit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job
[2] https://prow.ci.kubevirt.io/view/gs/kubevirt-prow/logs/periodic-kubevirt-e2e-k8s-1.29-sig-network/1836734599254773760

I think in general sig-net is more prone to flakiness when there is a general slowness in the cluster, because it has tests that spins up multiple VMs, Jobs, pods, on top of that the lane take few hours to finish, which increases the chance for it to be affected by some slowness  or resource intensive neighbor.
It also runs multiple components along with Kubevirt (CNAO, Istio).
There is on going effort to optimize the lane and drop redundant tests.
Was there some job density changes on CI nodes? (i.e.: does it run more jobs then it used to?)
Does it make sense to reduce the amount of parallel jobs that run next to sig-net jobs?

Adding sig-network failures from today's SIG CI meeting:

https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12928/pull-kubevirt-e2e-k8s-1.30-ipv6-sig-network/1838884829605662720
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12934/pull-kubevirt-e2e-k8s-1.29-sig-network/1839027779501297664

@ormergi AFAIK there's no density changes in the lanes, however @brianmcarey was looking at node problems and didn't spot anything IIRC

Adding sig-network failures from last week:

https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.31-sig-network/1840658127209369600
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12972/pull-kubevirt-e2e-k8s-1.29-sig-network/1841092059746799616

I think in general sig-net is more prone to flakiness when there is a general slowness in the cluster, because it has tests that spins up multiple VMs, Jobs, pods, on top of that the lane take few hours to finish, which increases the chance for it to be affected by some slowness or resource intensive neighbor. It also runs multiple components along with Kubevirt (CNAO, Istio). There is on going effort to optimize the lane and drop redundant tests.

@EdDev @ormergi Do you want to try to decrease the number of parallel nodes (which will decrease how many vms/pods you create in point of time)?
@brianmcarey Did you try monitor the nodes during a run to see if we can spot any spikes? It does sound weird that we would be limited here...

In general searching for timeout or deadline exceeded in SeachCI shows we have these errors in most sig lanes.
Was there some job density changes on CI nodes? (i.e.: does it run more jobs then it used to?) Does it make sense to reduce the amount of parallel jobs that run next to sig-net jobs?

@brianmcarey Can you answer this one?

I think in general sig-net is more prone to flakiness when there is a general slowness in the cluster, because it has tests that spins up multiple VMs, Jobs, pods, on top of that the lane take few hours to finish, which increases the chance for it to be affected by some slowness or resource intensive neighbor. It also runs multiple components along with Kubevirt (CNAO, Istio). There is on going effort to optimize the lane and drop redundant tests.

@EdDev @ormergi Do you want to try to decrease the number of parallel nodes (which will decrease how many vms/pods you create in point of time)?
@brianmcarey Did you try monitor the nodes during a run to see if we can spot any spikes? It does sound weird that we would be limited here...

In general searching for timeout or deadline exceeded in SeachCI shows we have these errors in most sig lanes.
Was there some job density changes on CI nodes? (i.e.: does it run more jobs then it used to?) Does it make sense to reduce the amount of parallel jobs that run next to sig-net jobs?

@brianmcarey Can you answer this one?

There was no change to the number of jobs that we are running on the clusters - the number of parallel tests that each job can run at one time was increased from 3 to 6 though - https://github.com/kubevirt/kubevirt/pull/11524
I am going to test with reducing this to see if it improves stability - there will be a lane runtime cost to it.

Clustered failures from 2024-10-08 flakefinder report:

https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12854/pull-kubevirt-e2e-k8s-1.29-sig-network/1843284952272080896
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12992/pull-kubevirt-e2e-k8s-1.30-ipv6-sig-network/1843264358751145984
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/kubevirt_kubevirt/12897/pull-kubevirt-e2e-k8s-1.30-sig-network/1843270052313305088
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.30-sig-network/1843422498088030208
https://prow.ci.kubevirt.io/view/gcs/kubevirt-prow/pr-logs/pull/batch/pull-kubevirt-e2e-k8s-1.30-sig-network/1843724510897901568

There was no change to the number of jobs that we are running on the clusters - the number of parallel tests that each job can run at one time was increased from 3 to 6 though - #11524

From my perspective this is a equivalent, the load on the node is doubled in all resources including disk IOP/s.
Eventhough etcd runs on memory there could be some slowness on KAPI by the load.

I am going to test with reducing this to see if it improves stability - there will be a lane runtime cost to it.
Awesome, let me know if you need some help / reviews.

In case parallel jobs are going to increase or other dramatic change, please consider announcing it so each sig would have some heads up and chance to monitor it.
In addition if parallel tests is going to increase I suggest to revisit the etcd-on-memory configuration and check how much it consume and adjust the its ramfs mount accordingly.

We reduced the number of parallel tests that run at once to 3 and will monitor it for the next two weeks - if the clustered failures don't appear this ticket can be closed.
https://github.com/kubevirt/kubevirt/pull/13032

@brianmcarey note that in the past week and a half (since 16 Oct.) there were holidays and naturally less burden on CI systems.
Please consider starching monitoring time to additional week so we could have better indication on CI state following the changes.


Convert DreamValue.cs to file-scoped namespace

Same reasoning and method as #1922, it gets rid of a warning and an indent.

those should all be pre existing issues


Support request/worker queues

Current Status
If one request take longer to process (e.g., database operation, clustering) all upcoming requests are pending, too.

Wanted
All request or at least longer processes should be added to a queue and not block upcoming requests. A common library for that task is Celery: Distributed Task Queue.
Questions

Is there a response while the request is being processed? If yes, what will be returned and how would the processed response be returned to the client?
Applications have to be aware that some results might come later. We have to check if promises will still work?

The basic integration of Celery into Flask is described in the Flask documentation.
This tutorial uses a combination of Celery and Flask. The progress bar is updated by polling the backend periodically. The author extended this tutorial by using the Flask Application Factory Pattern

Please see the implementation in the new repository caleydo_processing_queue.

see also Caleydo/caleydo_web_container#82

not part of the core


Disable jekyll-feed (_site/feed.xml) during development

Hey all, thanks for the amazing library! Just had a quick question/idea for a potential quality-of-life (at least, during development) improvement. Happy to help build a solution here however I can! Fastpages should be fast 😄

Is your feature request related to a problem? Please describe.
When developing the site via make server, we have a watchmedo command that rebuilds the site entirely. Re-building the website seems to happen almost instantly, but re-building the RSS feed takes ~4-6 seconds (edit: I'm guessing this increases with the number of posts? After creating a few dummy posts it's now at ~12-14 seconds). This latency can be frustrating when you just want to update HTML/CSS/etc.
Describe the solution you'd like
Ideally, if I remove the jekyll-feed plugin from _config.yml, we should no longer build feed.xml. I can see why that may be tricky, since you'd have to then figure out what to do with the RSS link in the footer.
Alternatively, maybe it's possible to have a no-feed-during-dev setting in _config.yml. If set to true, we don't build feed.xml. If we push to a PR on master, however, we can have a GitHub action to spot-check that this is intended? I imagine some may be frustrated with how slow it is to build feed.xml during development, but still want to build it when pushing to master.
Describe alternatives you've considered

Commenting out the jekyll-feed plugin from _config.yml, as seen in https://github.com/fastai/fastpages/issues/446.
Setting post_limit: 0 and excerpt_only: true.
Commenting out the jekyll-feed gem dependency from the Gemfile, adding RUN bundle update to the Dockerfile, then re-building the Docker images.

Additional context

This seems to be an issue on the base minima theme, as well: https://github.com/jekyll/minima/issues/375, https://github.com/jekyll/minima/issues/553
Other complains about jekyll-feed: https://github.com/github/pages-gem/issues/627

Thanks for raising this.  I'm still searching for a way to fix this if anyone has any ideas on how to fix this please let me know

@dataframing does this work?

Setting post_limit: 0 and excerpt_only: true.

I'm thinking I could override post_limit during development mode.  Do you want to try that and let me know if it works?

@hamelsmu Thanks for looking into it! I'm still trying to solve this one, as well.

Setting post_limit: 0 and excerpt_only: true.

I'm thinking I could override post_limit during development mode. Do you want to try that and let me know if it works?

Unfortunately, I've tried that and it doesn't work 😢 It would work as a solution, if/when we fix the underlying feed generation. Does it work on your end?
I've raised an issue with the upstream minima repo here. I'm hoping we can get a bit more insight into this, since it's kinda painful to have this bottleneck during local development 😕

Ran into the same issue and looks like apart from removing the jekyll-feed gem, monkey patching is the only thing that works. Can confirm neither commenting out - jekyll-feed under plugins: nor setting excerpt_only: true works for me.
My solution is mostly similar to what was suggested in the minima issue, only a little bit different
# frozen_string_literal: true

module JekyllFeed
  class Generator < Jekyll::Generator
    safe true
    priority :lowest

    alias_method :old_generate, :generate

    def generate(site)
      if site.config["feed"]["disable"]
        Jekyll.logger.info "Jekyll Feed disabled"
      else
        old_generate(site)
      end
    end
  end
end

this way I can disable feed by putting
feed:
  disable: true

in _config.yml. This allows me to enable it again just by leaving that out of the config
Another thing I did is create an extra config file for dev, _config_dev.yml and put the feed-disabling config there instead of the main _config.yml, then add --config _config.yml,_config_dev.yml to jekyll-serve. This way _config.yml doesn't need modifying for production.

Ran into the same issue and looks like apart from removing the jekyll-feed gem, monkey patching is the only thing that works. Can confirm neither commenting out - jekyll-feed under plugins: nor setting excerpt_only: true works for me.
My solution is mostly similar to what was suggested in the minima issue, only a little bit different
# frozen_string_literal: true

module JekyllFeed
  class Generator < Jekyll::Generator
    safe true
    priority :lowest

    alias_method :old_generate, :generate

    def generate(site)
      if site.config["feed"]["disable"]
        Jekyll.logger.info "Jekyll Feed disabled"
      else
        old_generate(site)
      end
    end
  end
end

this way I can disable feed by putting
feed:
  disable: true

in _config.yml. This allows me to enable it again just by leaving that out of the config
Another thing I did is create an extra config file for dev, _config_dev.yml and put the feed-disabling config there instead of the main _config.yml, then add --config _config.yml,_config_dev.yml to jekyll-serve. This way _config.yml doesn't need modifying for production.

I followed this solution and it seems to disable Jekyll feed. However, I observe that the build time is unchanged (still very high). Not sure if I did something wrong...

I actually noticed that too. Maybe something else was causing the long build time, but I haven't looked into it.

NB: I am writing this comment on all open issues
Thank you for posting this issue, and I'm really sorry I haven't gotten around to fixing this.  All of my time has been focused on nbdev.   In the new version of nbdev, we knew that we wanted to include blogging rather than it being a separate project.  We realized that Quarto, which the new version of nbdev is built on, is much better than fastpages.    Rather than keep this project going, we are recommending that people migrate to Quarto as we think that will be more powerful for users and is a much better tool for blogging.
I've written a detailed migration guide with some tools that will hopefully make the process of switching much easier.  Overall I think it will be worth it for users.
My apologies again for not getting to this issue sooner.  Will close this issue with this PR https://github.com/fastai/fastpages/pull/657, where we are letting people know that we are deprecating this project.  Thanks again for your support and for using fastpages!  🙇🏽


T channels

This is just for viewing the docs It will be deleted in a few minutes.

Hi, thanks for contributing to Strawberry 🍓!
We noticed that this PR is missing a RELEASE.md file. We use that to automatically do releases here on GitHub and, most importantly, to PyPI!
So as soon as this PR is merged, a release will be made 🚀.
Here's an example of RELEASE.md:
Release type: patch

Description of the changes, ideally with some examples, if adding a new feature.

Release type can be one of patch, minor or major. We use semver, so make sure to pick the appropriate type. If in doubt feel free to ask :)
Here's the tweet text:
🆕 Release (next) is out! Thanks to Nir for the PR 👏

Get it here 👉 https://github.com/strawberry-graphql/strawberry/releases/tag/(next)

I had a messy main branch.. fixing that..


TODO Beryllium: add in the list of fields if the field is Indexed or not

add in the list of fields if the field is Indexed or not
in sobject description
and potentially also in the custom object tab

I don't want to do the isIndex in the custom object tab, it requires to retrieve this information in the FieldDefinition ...
makes the code too complex.


Added the test cases for the context menu features and message access request in to-do list.

Summary
This PR consists the test cases for the following scenarios,

The feature to create Gitlab issue from context menu.
The feature to attach message to Gitlab issue.
The format message access request notification in the to-do list.

@DHaussermann @saturninoabril Can you please have a look at this?

This looks good @arush-vashishtha. The script looks accurate to me.
I will approve for merge once I confirm if we need any further validation of the Zephyr test data.


Added Comment node to Visual Shaders

I know this feature is long-awaited and desired.

Shouldn't the description use more spacious, multiline TextEdit?

Shouldn't the description use more spacious, multiline TextEdit?

Yeah, thank you, fixed this to be like:

A suggestion, I don't know if this is even possible with godot's GraphNodes or if this has already been discussed, but I think it'd be more intuitive if instead of having to right click on it and use set title or set comment, if the title and comment were just a TextEdit and a multiline TextEdit that you could just left click on to edit.

A suggestion, I don't know if this is even possible with godot's GraphNodes or if this has already been discussed, but I think it'd be more intuitive if instead of having to right click on it and use set title or set comment, if the title and comment were just a LineEdit and a TextEdit that you could just left click on to edit.

That would be nice indeed if it's not too complex to implement with the current system. Maybe it could just be a GraphNode with a text field like the Expression node?
A potential drawback is that the text field would always be editable, while it might be useful to have separate edit and read-only mode, to prevent accidental edition / focus grab when clicking around? There could maybe be a checkbox to turn the TextEdit to read-only and avoid accidental edition? Just throwing some ideas.
I guess with this approach adding an actual custom title to the comment box might not be well supported, but a side question is whether that part of the feature is really useful? Code comments rarely start with a title either, though the use case here might be a bit different.
I'm not against the current implementation though, just throwing some ideas. Like always we should find the right tradeoff between ease of implementation/maintainability and user experience (so we shouldn't need to add e.g. 500 lines of code to support being able to use a text edit directly in the GraphNode).

I have another suggestion: what if we make the comment not a node but a semitransparent colored area instead. That would allow us to place nodes on top of it and visually group them together.

I have another suggestion: what if we make the comment not a node but a semitransparent colored area instead. That would allow us to place nodes on top of it and visually group them together.

Where would the comment go? On top like this?

I have another suggestion: what if we make the comment not a node but a semitransparent colored area instead. That would allow us to place nodes on top of it and visually group them together.

Oh I thought it was supposed to work like that. Does the current proposed implementation just like add a text box in node space?

I have another suggestion: what if we make the comment not a node but a semitransparent colored area instead. That would allow us to place nodes on top of it and visually group them together.

That's exactly what a comment node is though. It's a GraphNode with the comment flag turned on, and it is displayed behind other nodes and is used for visual grouping. It's an existing functionality of GraphEdit.

I have another suggestion: what if we make the comment not a node but a semitransparent colored area instead. That would allow us to place nodes on top of it and visually group them together.

That's exactly what a comment node is though. It's a GraphNode with the comment flag turned on, and it is displayed behind other nodes and is used for visual grouping. It's an existing functionality of GraphEdit.

So It was supposed to work like that, nice :D

Yes, but looking at the code now, maybe this PR doesn't use existing functionality, but reimplements it instead. So I'm not sure anymore.

@pycbouh

All is good then, this should indeed already act exactly like @Ansraer wanted it to.

All is good then, this should indeed already act exactly like @Ansraer wanted it to.

Yeah I just tested it locally, it works exactly like that.

Thanks!


Checkpoint Download

Hi
Can you share another link for downloading checkpoint file nyu.pth and kitti.pth (eg. gdrive link).
I am  not able to download from Baidu servers, is there any other way to access these?

Hi, I have shared the gdrive links in ReadMe.


Installed package is erroneously removed at the end of container build

With this vagga.yaml:
containers:
  build:
    setup:
    - !Ubuntu trusty
    - !UbuntuUniverse

    - !BuildDeps [zlib1g-dev]

    - !Install [libpq-dev]

The libpq-dev package is removed and cannot be found in the container:
$ vagga _run build ls /usr/lib/libpq.a
... snip ...
Setting up libpq-dev (9.3.9-0ubuntu0.14.04) ...
... snip ...
Removing libpq-dev (9.3.9-0ubuntu0.14.04) ...
... snip ...
/bin/ls: cannot access /usr/lib/libpq.a: No such file or directory

If I remove the line with zlib1g-dev, or change it to !Install instead of !BuildDeps then everything works fine:
$ vagga _run build ls /usr/lib/libpq.a
... snip ...
/usr/lib/libpq.a

Yeah, it seems that libpq-dev (indirectly) depends on zlib1g-dev.
I'm not sure what I can do with apt-get. Presumably I should want to mark zlib1g-dev as dependendency and run apt-get autoremove instead of removing it with apt-get remove directly. But I failed to be able to do that in the past (don't remember what exact problem was).
So if you know what commands should be run on ubuntu, please help. (I'm mostly used to pacman which both don't delete packages which have others depend on them and has simple and clean --asdeps option)

I'm not an expert in apt-get and don't know offhand, but I'm sure there's a way to handle this.
A workaround: add an additional step during the container build just after removing all the !BuildDeps packages, where it just does an "apt-get install" again of all the !Installed packages. In most cases they will all already be installed in the system so nothing will happen. But if it hits this issue then the relevant package(s) will be re-installed

@bitc, I have to think about it a bit more. The problem is that someone may !Install a package then remove it. Or do other crazy things, like building same package from source and replacing all or some of the files. I'm trying to choose right trade-off between declarative (i.e. you just declare what to build) and imperative approach (most of the things run just step by step, because you can't reproduce build reliably otherwise). So things should not work too magical.
Anyway I think there is some combination of commands that work. Probably I'll investigate this in detail when will migrate vagga to post-1.0 rust, which is the most valuable task for now.

Well, I figured it out. The missing knowledge for me was apt-mark command :)


Unable to install <IP_ADDRESS> nuget package

Hey, not sure if this is the right place to post this but this was the last thing I could think of. Basically I'm unable to update to the latest version with the following error message:
Install-Package DiscordSharp -Pre
Attempting to gather dependencies information for package 'DiscordSharp.0.5.3-alpha' with respect to project 'DiscordBot', targeting '.NETFramework,Version=v4.5.2'
Attempting to resolve dependencies for package 'DiscordSharp.0.5.3-alpha' with DependencyBehavior 'Lowest'
Resolving actions to install package 'DiscordSharp.0.5.3-alpha'
Resolved actions to install package 'DiscordSharp.0.5.3-alpha'
Removed package 'DiscordSharp.0.5.1-alpha' from 'packages.config'
Successfully uninstalled 'DiscordSharp.0.5.1-alpha' from DiscordBot
Adding package 'DiscordSharp.0.5.3-alpha' to folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Added package 'DiscordSharp.0.5.3-alpha' to folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Install failed. Rolling back...
Package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' does not exist in project 'DiscordBot'
Package 'DiscordSharp.0.5.1-alpha' already exists in folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Added package 'DiscordSharp.0.5.1-alpha' to 'packages.config'
Removing package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' from folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Removed package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' from folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Install-Package : Failed to add reference to 'opus'.
At line:1 char:1
+ Install-Package DiscordSharp -Pre
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand
 package 'DiscordSharp.0.5.3-alpha' to folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Install failed. Rolling back...
Package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' does not exist in project 'DiscordBot'
Package 'DiscordSharp.0.5.1-alpha' already exists in folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Added package 'DiscordSharp.0.5.1-alpha' to 'packages.config'
Removing package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' from folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Removed package 'DiscordSharp.0.5.3-alpha : NAudio [1.7.3, ), Newtonsoft.Json [8.0.2, ), WebSocket4Net [0.14.1, ), WebSocketSharp [1.0.3-rc9, )' from folder 'A:\docs\PROGRAMMING\C#\Projects\DiscordBot\packages'
Install-Package : Failed to add reference to 'opus'.
At line:1 char:1
+ Install-Package DiscordSharp -Pre
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Install-Package], Exception
    + FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PackageManagement.PowerShellCmdlets.InstallPackageCommand

I'm using VS2015 and both the IDE and nuget are updated to the latest version. This issue occurs only when I'm trying to install this particular package (I thought at first that the dependencies might cause it but their install went fine on its own). Trying it out on a different computers produced the same error message.
I've tried clearing the nuget package cache, I've deleted all relevant packages manually, I've tried adding your package into a completely new project. Nothing worked.
Updating to the previous version works fine.
So my question basically is - could this be a package issue? Are you perhaps able/willing to test this out in your IDE?
Found a similar issue here and a package update solved the issue.

Looks like it is a package issue. This morning, I bundled opus in the lib folder with the DiscordSharp DLL. Looks like I need to add it as native. I'm currently out right now but I will fix this asap. Thank you!

For further elaboration: opus is a native win32 DLL as opposed to a .Net CLR DLL.

Temporary fix: Added opus.dll as content. You will probably have to manually copy it from the packages/DiscordSharp{version}/content/ folder though :( but hey NuGet is fixed! :D


UWP rework

Here is the first draft of the UWP rework discussed in discord and relating to issue #1182
What I have done:

Replaced the UWP messaging system that I added last time with a function callback system.
The developer is made to implement all UWP support now (for window and application lifecycle only).
The new system supports more of raylib's core.

What I have tested:

Keyboard input
Mouse input
Window lifecycle

What I am still to test:

[x] Gamepad input
[x] Keyboard character recording
[x] Whether or not UWP will support GIF recording.

What I am still to add/do:

[x] Move UWP specifics to a separate header.
[x] Add keyboard character recording.
[x] I still need to work out if locking the mouse cursor position is truly possible or if the developer will have to do it themselves. This would be easy as they can set their own functions for this anyway.
[x] Move BaseApp into the main App. This is now less of a boilerplate and more of an example
[x] Add full set of keyboard keys to example
[x] Try to fix the gamepad hack
[x] Implement UWP lifecycle (Resume and Suspend) properly.
[x] Ensure code matches raylib standards.

@Rover656 this is a big batch of changes... looking at the code it probably makes sense to move all UWP specific functionality to a self-contained uwp_events.h external library, used only on core.c.
Also, where is done the events polling? is it done by the App?

Hi ray, I agree with the moving of the UWP functionality to another header. Also, the event polling is only done by the App now, as most if not all of the functionality requires c++ access.

I believe that this is now ready for reviewing.

I'm going to look into UWP touch support, however I'm unsure how compatible it will be with raylib's setup as I've never used the touch APIs.

@Rover656 I looks very nice! Congratulations! I like it! 😄

@Rover656 Great improvement! Thank you very much!


mindex not working as on tin

i don't have time to debug your code myself, but after running a sample script from your examples i've got to conclude that it's simply not working as designed.
var Mindex = require('mindex')
var index = Mindex(['age'])
index.insertRecord({
id: 1,
name: 'John',
city: 'Denver',
age: 35
})
index.insertRecord({
id: 2,
name: 'Betty',
city: 'Denver',
age: 37
})
console.log(index.get('Denver'))
// returns [2]
// and unfortunately not [1,2] as it should.

You are indexing by Age...
var index = Mindex(['age'])
And then querying by City...
console.log(index.get('Denver'))
Try var index = Mindex(['city'])

great. thanks for the reply!
i could also index by both, i guess:
var mindex = Mindex(['city', 'age'])

Just to be clear, indexing by both does not mean you can query by both anyway you want...
If you index like this...
var mindex = Mindex(['city', 'age'])
Then you must query like this....
console.log(index.get('Denver', 25))

i see.
much impressed by the precision and promptness of your replies. much thanks!


Add a bio for @tiruv

This pull request adds a bio for  @tiruv as requested in #28.
I would love some :eyes: on this when you get a chance @githubstudent.

This looks very good :sparkles:
emojis are :+1:

Good job!


Remove qobj (QasmQobj and PulseQobj) support in backend.run

Summary
Fixes #38
Removed backend.run(qobj) from:
ibm_account.ipynb: updated text in notebook (not sure if more should be done here)
ibmqbackend.py: removed params from run methods
program_backend.py: removed params from run method
test_ibmq_backend.py: removed test_run_qobj test case
Details and comments

Could you please check the failing tests?
https://github.com/kt474/qiskit-ibm/runs/3563897960?check_suite_focus=true
Also why does the tutorial file have so many changes? You just need to edit the source and make necessary changes manually.

Will double check the push tests - as for the tutorial file I only changed one line so i'm not sure why it's showing so many differences in the git diff (maybe because I tried doing it in my ide). I'll try to revert the file

waiting on qiskit_rng PR for cqc test

Instead of waiting for qiskit_rng PR, which requires a release (or we'll need to switch to installing from master), perhaps we should just add a "skip if qiskit_rng version is <= 0.2.2" to test_cqc_extractor.

@kt474 I see more occurrences of QasmQobj and PulseQobj after merging PR #63. Could you please review again and fix?

I don't believe that PR (#63) has any occurences of backend.run(qobj) - I think only circuits are run or am I missing something?

@kt474 I see more occurrences of QasmQobj and PulseQobj after merging PR #63. Could you please review again and fix?

I don't believe that PR (#63) has any occurences of backend.run(qobj) - I think only circuits are run or am I missing something?

We need to remove QasmQobj and PulseQobj support in general, not just from backend.run (sorry if the issue description was not clear). I think job.qobj() https://github.com/Qiskit-Partners/qiskit-ibm/blob/c1d49055d5b7dc6377833baca3a524fec7cd3029/qiskit_ibm/job/ibm_job.py#L250 can be removed as well since it's deprecated.
I can also see QasmQobj and PulseQobj in other files like ibm_circuit_job.py, ibm_composite_job.py, sub_job.py and few more. @jyu00 we can remove all these right?

Ah ok I assumed removing qobj completely would be handled in a different issue but I could remove it in this PR too

Removing the qobj method requires a couple changes to test cases. Should the tests here in test_serialization just be removed because we are no longer retrieving the qobj?

Removing the qobj method requires a couple changes to test cases. Should the tests here in test_serialization just be removed because we are no longer retrieving the qobj?

@jyu00 Could you help answer this and also help review this PR?

Removing the qobj method requires a couple changes to test cases. Should the tests here in test_serialization just be removed because we are no longer retrieving the qobj?

We still need these tests because they are to ensure we can send and receive Qobj to/from the server properly, as Qobj is still the transportation format we use.

so then we shouldn't remove the qobj method? Those test cases depend on it

so then we shouldn't remove the qobj method? Those test cases depend on it

I think we should remove qobj method and test serializing circuits, backend_options and header instead.

Can you also investigate whether functions qobj_utils.py are still needed?

I think qobj_utils.py are still needed for now as the dict_to_qobj method is used a few times in the IBMCircuitJob methods - including _get_qobj
_get_qobj is then used for returning circuits and backend_options

I think qobj_utils.py are still needed for now as the dict_to_qobj method is used a few times in the IBMCircuitJob methods - including _get_qobj

What about the other 2 functions? _serialize_noise_model and update_qobj_config ?