api: adds CallTo to avoid allocations

Before that change each function call creates a new slice, which copies the result from wasm function. That causes a allocation on each call, since golang-compiler are not able to optimize such case.
That patch introduce a new function, CallTo, which makes possible to re-use an already existent slice. In some cases that makes possible to call multiple functions without any allocation.
Currently, such optimization only affects compiler/engine.go, and was not fully ported to interpreter/interpreter.go.

Thanks, I understand the motivation, but we have to be really careful adding a new API.
One quick question is I wonder if this is really the bottleneck of your application. I would appreciate it if you could give us the benchmark and profiling result. Thanks!

Currently, I don't have any benchmarks to give. But in the simple benchmarks I've ran, the performance difference between the original implementation and my suggested improvements is hardly noticeable. But, I am worried that the additional allocation would increase the GC work, which may impact the performance.
Maybe CallTo method isn't a good idea. The "With_Something_" methods is already used by Wazero, so maybe we can create one "WithOutputBuffer([]byte)" method that would enable all results to be transferred to that buffer.
Also, the "input" have issues. My patch doesn't try to address that. Currently, the input is params ...uint64 that is one "implicit slice". Calling as Call(ctx, input) that x will get allocated too. However, if I do something like: paramBuf := []int{input} and then Call(ctx, paramBuf...) it will not allocate. Maybe we could have one function that uses param []uint64 and re-use the buffer.

If reducing these allocations is considered relevant, one other way to achieve it is to replace these lines:
https://github.com/tetratelabs/wazero/blob/84ea9efc143f20f5e2dc5feaf37401952c63ad75/internal/engine/compiler/engine.go#L703-L706
With (reusing the params slice for the result):
	if resultCount := tp.ResultNumInUint64; resultCount > 0 {
		results = append(params[:0], ce.stack[:resultCount]...)
	}

Callers that don't care, are probably using varargs, the slice gets created for them; this avoids them some (50%?) allocations "for free."
Callers that do care, can take care to keep a []uint64 to avoid these allocations entirely.
OTOH, this causes issues for callers that reuse the argument slice expecting it not to be modified (which is a reasonable expectation).
I couldn't really measure any appreciable (CPU) difference in benchmarks. But allocs/op do drop 60% on some benchmarks (with these changes).

I'm not sure if results = append(params[:0], ce.stack[:resultCount]...) would be great. That mitigates the allocation, but not remove them. Also, variadic also causes allocations in some cases, and that might force to be one of them.
I'm using the following code:
        wasmfn := make([]api.Function, 100)
        for i := 0; i < 100; i++ {
		wasm,_, _ := initWebAssemblyModule(data, 256, _guestZIG)
		// ...
		wasmfn[i] = wasm.ExportedFunction("work")
	}

        res := make([][]uint64, 100)
	for i := range res {
		res[i] = make([]uint64, 1)
	}
	in := make([]uint64, 1)
        in[0] = 515249275

	for i := 0; i < runtime.NumCPU(); i++ {
		go func() {
			for j := range jobs {
				if err := wasmfn[j].CallTo(nil, res[j], in...); err != nil {
					return
				}
				// ...

I have 100 runtimes/modules and I'm calling them with the same input and storing each result to later process. In that case, res is re-used for each call to the same module, and I can them loop each result and do what I need to do. Also, I'm always calling with the same input (from  in[0] = 515249275). The  wasmfn[j].CallTo(nil, res[j], in...)  is 0 allocs/op, instead of 2 allocs/op.
So, doing some benchmarks, such as:
    for i := 0; i < b.N; i++ {
		wait.Add(len(wasmfn))
		for i := range wasmfn {
			i := i
			jobs <- i
		}
		wait.Wait()
    }

That is calling 100 wasm-function per benhmark-loop.
Without any optimization that was 201 allocs/op:
	res, err :=wasmfn[j].Call(nil, 515249275)
	if err != nil {
		b.Error(err)
		return
	}

One allocation due to 515249275 instead of re-using the input (in...); Another because each Call instead of  CallTo.
Replacing the Call to CallTo and also 515249275 to in... results on 1 alloc/op. Previously, it was 201 allocs/op, due to wazero allocations.
I guess GoCompiler can optimize in..., so it drops o 1 allocs/op. Then, using CallTo drops to 0. But, I'm almost sure that modifying Call to results = append(params[:0], ce.stack[:resultCount]...) will prevent such optimization. 🧐

The performance still almost the same, but I guess the performance hit will be caused by GC. So, CallTo might put less pressure on GC.

I think the issue is when the called function have more returns than input argument, which seems like to cause allocations. I didn't test it. That might be more noticeable when calling functions with zero arguments and one return value.

If optimizing, it would seem less a cognitive punch to use something like GoFunc/GoFunction so at least the style of invocation is known. Ack it doesn't have an error return but even if it had similarity it would feel more familiar even if also weird.
// GoFunction is an optimized form of GoModuleFunction which doesn't require
// the Module parameter. See GoFunc for an example.
//
// For example, this function does not need to use the importing module's
// memory or exported functions.
type GoFunction interface {
	Call(ctx context.Context, stack []uint64)
}

No that's not an issue, not really. You can easily pass in a slice that has length 0 but capacity 10; append can reuse that capacity.

You are right. I assumed I would call Call(ctx), without params. Since the type is ...uint64 and not []uint64, I initially overlooked the possibility of passing in a 'empty' slice, but with capacity.  😓

I understand the carefulness of adding new API, but callEngine.Call is indeed responsible for a significant number of allocations.

Maybe one way to take a first step is to add an interface in the experimental package that adds the CallTo method?
The Go program using wazero can then type assert api.Module on that interface to access the CallTo method to enable the optimization.
This lets us experiment without committing to an exported interface. After getting experience with it we can consider graduating the API in a 1.x version of wazero so the type assert isn't needed anymore.
If you think this can work feel free to go ahead and send a PR, I'll be happy to review 👍

@ncruces thanks for the help on this. A couple questions (and we are likely to cut a release tues fyi)

it should return the appended-to stack, and for ergonomics, even if the return is empty. Otherwise simple usages of the API eventually “leak” the slice (it gets GCed) and have to create a new one.

IIUC the godoc in Notes for this would be something like "this returns the input slice if it was large enough to hold any results. Otherwise, this returns a newly allocated slice"

If I we can coalesce around an acceptable API, I don't mind doing some/all of the legwork.

Since api.Function is already a interface and we don't support custom implementations of it, we can add a method and argue it is a compatible change.
Here's a stab, please wrap manually at 80 columns when polished. :)
	// CallWithStack is an optimized variation of Call that saves memory allocations when the
	// stack slice is large enough to hold the maximum length of functions parameters or results.
	//
	// The caller adds parameters in order to the stack, and reads any results in order from the
	// result stack, except in error case. This means the caller must be aware of the count of
	// parameters and results.
	//
	// # Example
	// Add one
	// # Notes
	//
	//   - When the stack parameter is large enough to hold the max count of parameters or results, it is returned. Otherwise, a new slice is allocated and returned. This means the caller should always prefer using the result slice for subsequent calls.
where parameters are added in order
	//   - This is similar to GoFunc, except for using calling functions instead of implementing them. Moreover, this is used regardless of whether the callee is a host or wasm defined function.
	CallWithStack(ctx context.Context, stack []uint64) ([]uint64, error)

PS ideally Call can be reverse implemented with CallWithStack, though it isn't required to do this if it doesn't work out.

wires crossed on @achille-roussel's comment. I think that approach is compatible with my guidance too. the internal types implementing will be the same, and the signature discussion is the same.
Having CallWithStack in an interface experimental.StackFunction first only changes where it is first. Either way works for me: tentatively adding as experimental (expecting some sort of alternate signature to be more ideal than what we thought of) or adding directly (if we expect we've analyzed this enough).

one slight advantage of making this experimental is we can sneak it in without a minor bump. Main cost is that consumers using it can expect their code to break when we move it out which might not be very long from now. OTOH those who are asking about it have dealt with plenty of breaks so far ;) EOF

last point is that https://github.com/tetratelabs/wazero/issues/1364 is likely to land before our next release Tuesday which would make next version 1.1.0 not 1.0.2, that could negate the point I just made.

Not requiring a minor version bump was in the back of my mind too.
If we can afford it I'd prefer if we made the API experimental at least for one minor version release (so targeting 1.2 since 1.1 is around the corner) to give us the time to learn from using the API before we commit to the 1.x compatibility promise.
Looking for your feedback @ncruces on feasibility, let us know what you think!

Let's keep Tuesday a patch (means this change -> experimental). thanks for thinking this through!

I'll respond to the rest later, but first of all: no urgency.
You guys do the release, and let's take time with API design.
Experimental is fine.
My post with numbers was just to show that yes, this can be a significant source of memory pressure, even if it's just a bunch of [1]uint64 (my case), so people may want to optimize it, and we should give them a way too, even if it isn't the default.

OK, some further notes that might inform the way forward.
The benchmark I'm running is github.com/maragudk/sqlite-benchmark adapted to use my driver rather than mattn's. Nothing special about this, but that's pretty much the point: it's not designed to stress this, rather it tries to simulate a realistic workload. I'm running my benchmark with -benchtime=1000x to get (more) consistent numbers, but there's non-determinism (goroutines and connection pools).  Onto numbers.
These changes do very little for CPU time, because the benchmark spends most of its time on syscalls. I'll investigate, but I suspect I/O:
Showing nodes accounting for 12.92s, 92.95% of 13.90s total
Dropped 168 nodes (cum <= 0.07s)
Showing top 10 nodes out of 212
      flat  flat%   sum%        cum   cum%
     8.08s 58.13% 58.13%      8.08s 58.13%  syscall.syscall
     2.54s 18.27% 76.40%      2.54s 18.27%  syscall.syscall6
     0.78s  5.61% 82.01%      0.78s  5.61%  runtime.madvise
     0.42s  3.02% 85.04%      0.42s  3.02%  runtime.pthread_cond_wait
     0.28s  2.01% 87.05%      0.28s  2.01%  runtime._ExternalCode
     0.27s  1.94% 88.99%      0.27s  1.94%  syscall.syscall6X
     0.26s  1.87% 90.86%      0.26s  1.87%  runtime.pthread_cond_signal
     0.11s  0.79% 91.65%      0.11s  0.79%  runtime.memclrNoHeapPointers
     0.09s  0.65% 92.30%      0.09s  0.65%  runtime.kevent
     0.09s  0.65% 92.95%      0.09s  0.65%  runtime.usleep

There are two sides to a api.Function.Call: the input params ...uint64 and the output []uint64.
The both input/output seem to always escape to heap, I assume because api.Function is an interface.
We can try to optimize neither, either or both. My SQLite wrapper makes it "easy" to test because all calls go through a single wrapper function.
Neither:
Showing nodes accounting for 1474.46MB, 87.07% of 1693.40MB total
Dropped 149 nodes (cum <= 8.47MB)
Showing top 10 nodes out of 99
      flat  flat%   sum%        cum   cum%
  455.23MB 26.88% 26.88%   455.23MB 26.88%  github.com/tetratelabs/wazero/internal/wasm.NewMemoryInstance (inline)
  289.08MB 17.07% 43.95%   289.08MB 17.07%  github.com/tetratelabs/wazero/internal/engine/compiler.(*moduleEngine).newCallEngine
  162.43MB  9.59% 53.55%   162.43MB  9.59%  github.com/tetratelabs/wazero/internal/engine/compiler.(*engine).NewModuleEngine
     120MB  7.09% 60.63%   169.50MB 10.01%  github.com/ncruces/go-sqlite3.(*Stmt).ColumnRawText
  114.16MB  6.74% 67.37%  1505.74MB 88.92%  github.com/maragudk/sqlite-benchmark.(*DB).ReadPost
  104.50MB  6.17% 73.54%   166.44MB  9.83%  github.com/tetratelabs/wazero/internal/engine/compiler.(*callEngine).Call
   73.12MB  4.32% 77.86%   242.63MB 14.33%  github.com/ncruces/go-sqlite3.(*Stmt).ColumnText (inline)
      72MB  4.25% 82.11%   101.50MB  5.99%  github.com/ncruces/go-sqlite3.(*Stmt).ColumnType
      52MB  3.07% 85.19%       52MB  3.07%  github.com/ncruces/go-sqlite3/driver.maybeTime
   31.93MB  1.89% 87.07%    31.93MB  1.89%  github.com/tetratelabs/wazero/internal/wasm.(*MemoryInstance).Grow

So the interesting thing about this is that varargs slices escape on the caller, not the callee. I have a single function that performs all api.Function.Calls but that is itself varargs so the allocations don't pop up there.
The more popular (for this benchmark) callers of that function are ColumnRawText and ColumnType, so they show up quite high on the list.
 This can be achieved now with no API change. My case was easy, for others it may be harder.
Next, if I use the above trick (reuse the input slice for the output slice), we get:
(pprof) top
Showing nodes accounting for 1264.24MB, 90.87% of 1391.21MB total
Dropped 145 nodes (cum <= 6.96MB)
Showing top 10 nodes out of 102
      flat  flat%   sum%        cum   cum%
  494.90MB 35.57% 35.57%   494.90MB 35.57%  github.com/tetratelabs/wazero/internal/wasm.NewMemoryInstance (inline)
  297.60MB 21.39% 56.96%   297.60MB 21.39%  github.com/tetratelabs/wazero/internal/engine/compiler.(*moduleEngine).newCallEngine
  159.67MB 11.48% 68.44%   159.67MB 11.48%  github.com/tetratelabs/wazero/internal/engine/compiler.(*engine).NewModuleEngine
  111.63MB  8.02% 76.47%  1214.02MB 87.26%  github.com/maragudk/sqlite-benchmark.(*DB).ReadPost
   64.62MB  4.65% 81.11%    64.62MB  4.65%  github.com/ncruces/go-sqlite3.(*Stmt).ColumnText (inline)
   48.50MB  3.49% 84.60%    48.50MB  3.49%  github.com/ncruces/go-sqlite3/driver.maybeTime
   27.79MB  2.00% 86.59%    27.79MB  2.00%  github.com/tetratelabs/wazero/internal/wasm.(*MemoryInstance).Grow
   24.89MB  1.79% 88.38%    24.89MB  1.79%  github.com/tetratelabs/wazero/internal/asm/amd64.(*nodePool).allocNode (inline)
   18.72MB  1.35% 89.73%    18.72MB  1.35%  github.com/tetratelabs/wazero/internal/wasm.(*ModuleInstance).applyData
   15.91MB  1.14% 90.87%    62.01MB  4.46%  github.com/tetratelabs/wazero/internal/engine/compiler.compileWasmFunction

Now callEngine.Call drops out as well. Total heap allocations went from 1693.40MB to 1391.21MB which is a significant change.
Finally, there's the possibility of optimizing the return value but not the parameters. This means above trick, but calling with varargs and no cacheing. Not a huge fan of this (because changing an input slice is surprising) but maybe it's worth considering since it likely won't break anyone and is a "free" optimization for users.
 The reduction is not as significant as optimizing params, because functions tend to have more params than returns (this C API as a single return value for 99% of functions, only free returns void). Still somewhat significant.

@ncruces thanks for the excellent analysis!
Here are some summary takeaways, important for use cases that call into the same module many times.
These changes might not do a lot for wasm that only call in via entry points, e.g. WASI's _start or http-wasm's handle_request. However, FFI interactions which call back into the same module a lot, like sqlite or maybe proxy-wasm or @jerbob92's PDF utilities may end up benefiting.
I'll rule out changing Function from an interface to a struct as I think that ship has sailed. It would be hard to do something like this without adding circular deps as well.
Changes with no API impact?
Users can: copy in parameters to a reused slice to prevent varargs from escaping.
For example:
-	r, err := fn.Call(m.ctx, params...)
+	m.arg = append(m.arg[:0], params...)
+	r, err := fn.Call(m.ctx, m.arg...)

wazero can: re-use the params slice (from varargs) for return values (internal to callEngine)
https://github.com/tetratelabs/wazero/pull/1332#issuecomment-1502124325
How to we go this route?
If we went this route, this is mostly documentation revision. For example, in a new heading on api.Function, describe slice nuance, including the trick to avoid allocations on input and also to beware that the input slice will be re-used for results (usually only 1 value).
Should we go this route?
I agree that it is unlikely anyone will expect to re-use values inside a varargs slice, except it could already be optimized. For example, if someone was passing constant input parameters via a pre-allocated slice, this would corrupt their calls. For example, var fnFooParams = []uint64{1, 2, 3}. I'm not saying this is normal, but it would make an uberbug I think.
Personally, I feel this is super tempting, but I also don't think a new API CallWithStack method similar to our GoFunc is a bad thing. It would be more a discussion if we didn't already have a GoFunc. OTOH the trick to prevent escapes (return the stack to the caller), is going to need to be documented. However, documenting that is no more tricky than documenting how to prevent varargs from escaping (e.g. re-using a slice and copy-ing into that). Finally, the nice thing about adding a new method is CallWithStack is already advanced, so it contains the advanced stuff in on godoc.
What do you think?
I'm giving a perspective not porting anything like sqlite, pdfium, pdfx etc. Mainly, I'm thinking about api shape in general and maintenance both of code and docs, as well potential Uber-bombs. Take my opinion as just an opinion.

My opinion matches this sentence exactly: “Personally, I feel this is super tempting” … but deep down, I know it's wrong.
As long as it's possible, I'm relatively agnostic on the mechanism, and can totally get behind https://github.com/tetratelabs/wazero/pull/1332#issuecomment-1507793809.
The only thing I'd add, is that you may want codify the notion that some wazero APIs are not meant for external implementation and are fair game to extend in this way. Given the way wazero is structured, I think it makes sense.
One way to do that is in documentation. The other way is to add an unexpected method to those interfaces (breaking compatibility right now for all of them, but not carrying the problem forward).

A different PR that helps us solidify the notion that most types are not for external implementation would be a fantastic add imho. I don't know state of the art here.. I suspect we aren't 100pct covered with docs, and some might be surprised by cast errors meanwhile.

obviated by #1407


Fix state attribute in GeoJSONs

The state attribute on some GeoJSON files was still agreed instead of agreement_process.
It also adds the missing rsv1-baden-wuerttemberg to the meta.json
See #75 for an idea how to avoid this in future data imports

Looks nice, thank you! I'll merge it.


Debugging SharpGL

Hi folks,
I've been using SharpGL to build a raytracing engine and have got a couple of odd behaviours that I'd like to get a better handle on.

Every frame SharpGL throws this warning...
"OpenGL Error: "Invalid operation.", when calling function SharpGL.ReadBuffer"
Which I think is related to me changing the framebuffer. I always call gl.BindFramebufferEXT(OpenGL.GL_FRAMEBUFFER_EXT, 0); at the end of my GLSL code, but this error seems to be related to using framebuffer rendering.

When I resize the window or change my shader fragment I reinitialise all of the framebuffers and textures at the new size (sometimes the same size). One out of ten times this doesn't seem to work and my application just produces a black screen. There are no gl.geterror()'s when I poll for them during initialisation or rendering. What's the best way to get more information about what's going on here? I've tried hooking up AMD's PerfStudio 3, but that doesn't want to attach to the OpenGL instance in the app for some reason. Has anyone had any luck with this previously?

Anyway, appreciate any feedback you could give me on either of the above issues (perhaps they're related?)
Ta,
Dom

Ok, so I've got to the bottom of the rendering issue I was getting, although not through debugging it to my satisfaction. :) Instead I noticed quite a few issues with different threads rendering OpenGL on this site and others.
The app I'm building has two main windows. The second window is triggered from the first window and OpenGlControl's exist in both windows. When the second window is initialised I now remove the OpenGlControl from the first window...
MainCanvas.Children.Remove(openGlCtrl);
...and put it back after I've closed the second window...
MainCanvas.Children.Add(openGlCtrl);
This solved all the black screen issues I was getting.
I'm still seeing the "OpenGL Error: "Invalid operation.", when calling function SharpGL.ReadBuffer" each frame, but there's no big deal with that one.
I'd still be interested in working out how to attach a debugger to my app. I've tried AMD's PerfStudio, glDebugger, GLIntercept. No luck with any of them...


Lack of definitions in 5.0, 5.1. 6.0 jars

Hi,
I don't know, maybe I'm missing something, but 4.0 jar weighs ~850kb and has all the definitions(for example Functional class), while 5.0 or 5.1 weigh only ~50kb and barely have any code at all.
in my build.gradle.kts i specify dependency as follows
dependencies {
implementation("org.jetbrains.kotlinx:kotlin-deeplearning-api:0.5.0")
//    implementation("org.jetbrains.kotlinx:kotlin-deeplearning-api:0.4.0") // with 4.0 everythings fine
}
Even if i download your jars from jar-download.com then they also seem corrupted(5.0, 5.1, 6.0 very small without definitions but 4.0 is ok)
How come nobody else have this problem? I'd be gratefull for any support in this matter :)

sorry, i didn't read carefouly enough and missed info about setting dependency to kotlin-deeplearning-tensorflow. Now everything is fine. Feel free to remove this thread.

Nice, that you gound a solution


Enhancement: Metadata for files uploaded via native protocol

What did you do?
Upload files via Storj native protocol and access them via the gateway.
What did you expect to see?
Files should have metadata.
What did you see instead?
Files have no metadata.
Files uploaded via uplink cli or filezilla won't have metadata when accessed via the gateway, which have compatibility issue with S3 client (e.g. Nextcloud will always report "no ETag" error).
I understand that Stroj native protocol does not upload metadata, but can this be solved by enhancement of the s3 gateway?
Can such metadata be generated once accessed via the gateway, or some command that scan the bucket and create missing metadata?
Related issue: #https://github.com/storj/storj/issues/5424

I understand that Stroj native protocol does not upload metadata, but can this be solved by enhancement of the s3 gateway?

Objects uploaded through the gateway will have the necessary metadata. If you need S3-related metadata, it's best to use the gateway to upload them. Gateway-ST (this repository) has similar self-hosting capabilities to Uplink CLI.


An error occurred (AccessDenied) when calling the AssumeRole operation: User

After running any of these commands:
`
aws iam get-role --role-name "AWSServiceRoleForElasticLoadBalancing" || aws iam create-service-linked-role --aws-service-name "elasticloadbalancing.amazonaws.com"
aws iam get-role --role-name "AWSServiceRoleForECS" || aws iam create-service-linked-role --aws-service-name "ecs.amazonaws.com"
`
I get this error message:
An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::350514568196:assumed-role/ecsworkshop-admin/i-03a0645e089123f6f is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::350514568196:role/ecsworkshop-admin
Which steps I need to do in order to fix this issue?

This was as a result of the change in the way IAM  role trust policy behavior
https://aws.amazon.com/blogs/security/announcing-an-update-to-iam-role-trust-policy-behavior/
This has been addressed by #71


[$250] Android - Notifications - Message notification leads to LHN instead of chat

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 9.0.74-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught on HybridApp, is this reproducible on New Expensify Standalone?: N
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/5328739&group_by=cases:section_id&group_order=asc&group_id=296775
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Component Notifications
Issue reported by: Applause - Internal Team
Action Performed:
Prerequisite: Two accounts has to be opened in different environments or devices

Open the Expensify app
Switch to another account on a different environment
User B, send some messages to user A
User A, with the app backgrounded, wait for the messages notifications to be received
User A, open the received notifications

Expected Result:
The user should land on the unread chat when interacting with messages notifications
Actual Result:
After interacting with messages notifications, the user lands on LHN instead of landing on the unread chat
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Standalone
[x] Android: HybridApp
[ ] Android: mWeb Chrome
[ ] iOS: Standalone
[ ] iOS: HybridApp
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

https://github.com/user-attachments/assets/004ad3d1-88c3-43c6-89b2-c046872d5d5b
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~021867273920217509041
Upwork Job ID:<PHONE_NUMBER>217509041
Last Price Increase: 2024-12-26

Issue OwnerCurrent Issue Owner: @hoangzinh

Now notification leads to the correct conversation, but before landing on it, inbox is displayed for a second. This issue was only reproducible on hybrid app for me.
https://github.com/user-attachments/assets/2470b285-ea5d-4725-b2db-44b25f9bdd51

Issue not reproducible during KI retests. (First week)


Grammar error on export type *

Fixes #36966
The first commit added the missing calls to markSymbolOfAliasDeclarationIfTypeOnly so as to recognize export type * forms as type-only while simultaneously disallowing it with a grammar error. Mostly I was curious how close we were to supporting export type *, and it appears that it’s only a couple extra lines. Afterwards, I reverted the extra calls because it created two errors where one would suffice.

@typescript-bot cherry-pick this to release-3.8

Did I mess that up by merging it?
@typescript-bot cherry-pick this into release-3.8


Added tests of PublicResolver.

Can you remove the 'exact gas' tests? Other than that, this looks great.


VS Code: this document has been excluded

Cannot find a way to include a dotfile written in TOML. VS Code always prints "this document has been excluded". Adding a config file with include "**/.file" does not work. File associations neither. Anything I have missed?

I have the same message for Python Pipfile with the VSCode extension. Is there any extension/VSCode setting im missing to add Pipefile as a valid toml file?

+1 same issue

I have this issue on Even Better TOML 0.19.2. Try as I might, settings.json
    "files.associations": {
        "**/*.wara": "toml"
    },

has no effect and the angry squiggle always says this document has been excluded.

Has anyone tried this?


Create rules for the trailing slash matching configuration

According to the migration guide:

As of Spring Framework 6.0, the trailing slash matching configuration option has been deprecated and its default value set to false. This means that previously, the following controller would match both "GET /some/greeting" and "GET /some/greeting/" [...]

Investigate and create rules if necessary.
In the rule description and message, try to be as verbose and clear as possible. Code examples are always welcome and useful. Feel free to use markdown syntax for these.

Need to investigate the three proposed solutions. The last one seems to be straightforward to implement and show a snippet.


Drag And drop widget

In this test, we have added a test to drag and drop form widget to canvas.

Drag and Drop Widget test added


Bug in the PPOTrainer.log_stats code

https://github.com/lvwerra/trl/blob/main/trl/trainer/ppo_trainer.py#L1144
How can torch.distributed.all_reduce work if it's inside of a self.accelerator.is_main_process if block?
You should be using self.accelerate.gather_for_metrics anyways btw.
Have you folks tried/tested the distributed code recently at all?

nvm my bad ahah.
Still, gather_for_metrics would be interesting.

Yes, we might switch to that at some point. Since there doesn't seem to be a problem I am closing the issue for now.


Nav/Athena behavior when disconnected

Describe the bug
Tried to use navigation on the return leg of my road trip Sunday and had several weird issues:

The UI showed connect offline, although I seemed to be in good LTE coverage
I chose my preprogrammed home location from the C3 UI as my nav destination
Navigation never really started, but the Nav UI pane did come up
I tried to cancel navigation, but the Nav menu disappeared in favor of the Nav/Prime ad screen
Rotating among UI views, the Nav UI pane was still accessible despite this
If I selected the usual full-screen onroad view, the Nav pane would reappear on its own within about a minute

I drove in this state for about 90 minutes. To get connectivity back and make the nav pane stop popping up, I had to reboot the device at my next stop.
 The instances I made note of were right at the end of the route, about 6:36PM and 6:41PM (might be off by a timezone). It popped back up after about a minute each time. I actually had the UI process die and restart on me right at the end too, but that could be the usual watchdog thing.
What hardware does this issue affect?
comma three
Provide a route where the issue occurs
3cfdec54aa035f3f|2021-12-19--17-16-31
openpilot version
master @ 16dd7ce1fe2a2b0537fefd5ad131d1494d0d09ad
Additional info
There's several issues to unpack here.
The C3 UI showed I had good LTE coverage. I don't know if I had working Internet connectivity; I wasn't in a position to SSH in and check. Athena certainly thought I was offline. Device reboot was definitely needed to get it back.
Earlier that day, several drive-cycles (but not power-cycles) prior, I did spend time in an area that legitimately had no cell service. I drove into the area, stopped and parked for an hour, and started a new drive out of that area. I don't know when the Athena no-connectivity issue started, but it's possible that it never recovered from earlier that day.
Even if I had no Internet connectivity, the UI shouldn't have treated me as if I weren't a Prime/Nav subscriber.
It shouldn't be possible to lose the Nav settings/destination menu while still having the Nav onroad pane in rotation.
The Nav pane shouldn't have kept popping up unsolicited. That was weird. Maybe a nexus with #23281?

This happens to me with my comma two. Unsure if it's the same issue, but after a day (usually when I get back into the car of a morning), it'll say the device is offline. Requires a reboot to get connection back.
LTE reports full signal during this time.

I have noticed this same behavior several times this week while on a cross-country drive. If I start a drive cycle in an area with no cell coverage, the device requires a hard power cycle to restore connect services once cell coverage is restored. The C3 indicates cell coverage correctly, but connect remains offline across multiple drive cycles until I perform a hard reset.


Web GUI Improvements for Raven

While playing around with Raven I noticed a few things in the GUI that could be better.
I basically only turn wakeword on when I am testing bacause I haven't gotten to a point where it doesn't randomly activates and annoys me. Whenever I turn it back on the wakeword id field says default, the list of wakewords is empty and I have to hit refresh to even see my non default named wakeword. If I select that, the wakeword id field still displays default. I am not sure if what is in that field even matters once I select one in the list, but I always replaced that with the name in he list.
What would make it better would be loading the settings from before i deactivated it from the profile file, as far as I know they stay in there, even if I deactivate it. Doing a list reload when I open the settings for Raven might also be nice, the first time it happened I wondered if my wakeword got deleted.
My guess is that the wakeword id field is used to select the current wakeword as well as which folder the recorded samples end up in. If that is the case, then I am not sure what the list below is for. It might be easier to use if just the list is used to select the current wakeword and where the recordings are saved and there would be a button or something to explicitly create a new entry in the list. Another way to solve the list/textfield confusion would be if the textfield updates when I select something in the list.
The last thing I wanted to write about is the fact that the option to save recordings for future use has no option at all inside the GUI, the only way I knew about that is because I read about it it being possible somewhere in the forum. It might be a good idea to include that in the GUI also, a simple checkbox to activate it and some kind of text that the folder name can be changed in the profile file directly would do it, there is next to no reason I see for changing the name of the folder, only thing that might be interesting would be if I could change where it saves them so I could direct them to some place on the network but since that is not an option as far as I know, folder name seems to be something that doesn't need to be on the GUI because it rarely matters.

I did my first test with Raven some days ago and was also kind of confused of its WebUI section. I would like to see something like @Daenara mentioned.

New UI is coming in 2.5.9 :+1:

New UI is coming in 2.5.9 :+1:

Thank you, @synesthesiam. I am very excited about the new design 😄

Thank you, @synesthesiam. I am very excited about the new design 😄

I saw the new ui and it certainly looks better, but I found a few problems already.

Design wise it looks good, if you ignore the checkbox which should not hang around in the center of the screen. But there are 3 problems I noticed in the functionality of it. First off, my already existing wakeword (computer) is missing from the list, after the upgrade the list was empty. So I decided to add a new wakeword (test) which showed up, but since there is a way to listen to the recordings, I would have loved to have the play button and the re-record button directly after I record a wakeword, thought I am not sure it is possible. Lastly, I decided to delete my test wakeword via console, and ended up with it still being in the gui afterwards.
That combined with the fact that my old wakeword did not show up tells me that the folder is not scanned to display the list.
I fixed the first problem by saving a wakeword with the same name as my old one without recording anything. It then showed up and I could listen to my existing recordings. I got rid of the test wakeword the same way, after saving it again I could delete it from the gui.

I saw the new ui and it certainly looks better, but I found a few problems already.

Design wise it looks good, if you ignore the checkbox which should not hang around in the center of the screen. But there are 3 problems I noticed in the functionality of it. First off, my already existing wakeword (computer) is missing from the list, after the upgrade the list was empty. So I decided to add a new wakeword (test) which showed up, but since there is a way to listen to the recordings, I would have loved to have the play button and the re-record button directly after I record a wakeword, thought I am not sure it is possible. Lastly, I decided to delete my test wakeword via console, and ended up with it still being in the gui afterwards.
That combined with the fact that my old wakeword did not show up tells me that the folder is not scanned to display the list.
I fixed the first problem by saving a wakeword with the same name as my old one without recording anything. It then showed up and I could listen to my existing recordings. I got rid of the test wakeword the same way, after saving it again I could delete it from the gui.


"sorry my bot code is having an issue" when removing user data

I'm trying to reset the user session by sending the activity like so:
switch (activity.GetActivityType())
{
case ActivityTypes.Message:
await Conversation.SendAsync(activity, () => chain);
break;
case ActivityTypes.ConversationUpdate:
break;
case ActivityTypes.ContactRelationUpdate:
break;
case ActivityTypes.Typing:
break;
case ActivityTypes.DeleteUserData:
activity.GetStateClient().BotState.DeleteStateForUser(activity.ChannelId, activity.From.Id);
await Conversation.SendAsync(activity, () => chain);
break;
}
This does reset the user session but the chat bot gives a "sorry my bot code is having an issue" after.
However, when i attach my debugger, no error message is given, and the chatbot does not post the message.

Can you enable "break on thrown exceptions" and disable "just my code" and obtain a call stack for the thrown exception?

Closing this because of inactivity. Feel free to follow up or open another issue if you couldn't resolve the problem.


Check for walls during movement animations

fixes https://github.com/microsoft/pxt-arcade/issues/1677
Here's my change so that sprites animated with movement animations will respect walls. The issue was that any movements that are more than a pixel at time don't get checked for wall collisions, so I'm doing it myself before moving the sprite. It'll calculate a line (using the line logic from the Sprite Editor) and check along every point if we'll be colliding with a wall.
I'm not sure how much movement animations are used and if we're concerned about performance. I can try to optimize the check by simplifying the line calculation or by only checking every 16 positions - let me know what you think!
Here are two gifs to show the performance difference with the change.  It's kinda hard to tell from the gifs, but the change makes it clunkier to move around.
With change, 30 moving sprites - it's harder to move around without dragging

Without change, 30 moving sprites - pretty smooth

I think this might be too big of a perf degradation.
Not sure the bresenham ray-tracing is necessary. Maybe instead we should just add another internal moveSprite method that always runs collisions in the physics engine? Otherwise we'll need to replicate the tilemap collision logic here.

Hmm, could we handle with https://github.com/microsoft/pxt-common-packages/blob/master/libs/game/physics.ts#L576 by adding a flag to say 'force check collisions for non-ghost sprites`

Hmm, could we handle with https://github.com/microsoft/pxt-common-packages/blob/master/libs/game/physics.ts#L576 by adding a flag to say force check collisions for non-ghost sprites that overrides the check against movement being within one step?

hmmmmm I just tried this. The issue with only checking in moveSprite is that it'll only account for the current position, and not everything in between. So if we have a line movement that would take the sprite beyond the wall, it would allow that. I could have a return value, and stop animations if we can't move? I'm not sure if I can stop animations in a certain direction
(gif loops at a bad spot, sorry about that D:)

@riknoll oo I'll look into how the tilemap does collisions

@riknoll I made the change to use the tilemap logic :D it's super fast now! and the walls block properly

Overall looks like a good refactor. @jwunderl can you double check this? Shouldn't affect perf too much?

yay thanks for taking a look! I'll do a perf test too


Semantics and need for local system datetime

The oddest of the DateTime classes is undoubtebly LocalDateTime. This is because its value is tied to the system timezone, which may change at any time.
This leads to tricky situations:

a LocalDateTime can become non-existent after initialization
a LocalDateTime can suddenly require disambiguation after initialization
the meaning of the disambiguation can change (i.e. later is chosen, but the system timezone changes so that later is now a more/less further in time)
converting from other aware types preserves the same moment in time, but can then abruptly change:

# always preserves the same moment in time, no matter now much you convert
original = UTCDateTime.now()
original.as_utc().as_offset().as_zoned("Asia/Toykyo").as_offset().as_zoned("Europe/Paris") == original

# however, as soon as you use `as_local()`, a race condition can occur in which the local datetime is shifted
intermediate = d.as_local()
os.environ['TZ'] = ... # set the system timezone to something different
intermediate.as_utc() == original  # false!

The current mititagations in place:

a warning in the docs about the effect of changing the timezone
all LocalDateTime methods check for existence first and raise exceptions.

Potential solutions:

Put LocalDateTime in a separate category from the other aware types. Instead of as_local, make the method better reflect that you're entering a new reality.
Name the class FloatingLocalDateTime to reflect how it 'floats' on the system timezone
Have a way to (explicitly) automatically handle non-existent times
an as_offset_local method to avoid race conditions when converting to local datetime at a particular moment.
Make is necessary to call to_offset(dismabiguate=...) before comparison operations

One solution i'm gravitating towards: making LocalDateTime more like NaiveDateTime:

this is similar to the standard library, so it is recognizable for Python devs
a big case for using LocalDateTime is when the system timezone changes. It should be rock-solid reliable for this case.
disambiguation can't happen on instantiation, but explicitly every time it needs to be compared with 'aware' timezones to_offset(disambiguate=...) method. It's cumbersome, but the only correct way.
LocalDateTime.now() would actually make limited sense—as soon as it's determined, the moment in time could change. Perhaps it should take a different name...

Examples
Problem with the current situation
os.environ['TZ'] = 'America/New_York'
# this time exists just fine in NYC
backup_scheduled = LocalDateTime(2024, 3, 31, hour=2, minute=30)
os.environ['TZ'] = 'Europe/Paris'
# but it gets skipped in Paris DST transition!
should_run_backup = UTCDateTime.now() > backup_scheduled

The last line is unreliable, as the backup_scheduled could be non-existent depending on the system timezone. This is bad because:

it's pretty unexpected that this exception is raised during comparison
it doesn't allow the user to handle this more gracefully

What would the solved problem look like
os.environ['TZ'] = 'America/New_York'
backup_scheduled = LocalDateTime(2024, 3, 31, hour=2, minute=30)
os.environ['TZ'] = 'Europe/Paris'
# We are forced to call the `with_local_offset` method which offers various methods to deal with ambiguity/non-existence
should_run_backup = UTCDateTime.now() > backup_scheduled.with_local_offset(disambiguate=..., nonexistent=...)

Side comment: Have you considered a different name for LocalDateTime? Because LocalDateTime in other popular libraries (Joda-Time, java.time, NodaTime) means a different thing. In those libraries, LocalDateTime is what is currently called NiaveDateTime in this library. There is too much potential for confusion.
Possible alternatives:

SystemDateTime
LocalizedDateTime

I'm coming around to this as well. Ideally I'd like consistent as_X and XDateTime.

SystemDateTime/as_system()
LocalSystemDateTime/as_local_system()
LocalOSDateTime/as_local_os()
OSLocalDateTime/as_os_local()

I'm less of a fan of Localized — the essential word here is "system"

Following up the discussion in #59, and after reading the notes in this ticket, I'm still confused about the intended behavior of LocalDateTime/SystemDateTime when the OS timezone changes.
My expectation is that the timezone of the OS gets bound to the instance of the LocalDateTime/SystemDateTime at creation, and if the OS changes its timezone, it should not affect the instance of the LocalDateTime/SystemDateTime that I created earlier.
So I don't follow how the problems mentioned above can happen: "a LocalDateTime can become non-existent after initialization", and "a LocalDateTime can suddenly require disambiguation after initialization". My expectation is that the instance of LocalDateTime/SystemDateTime always remains valid.

My expectation is that the timezone of the OS gets bound to the instance of the LocalDateTime/SystemDateTime at creation, and if the OS changes its timezone, it should not affect the instance of the LocalDateTime/SystemDateTime that I created earlier.

I've come around to this as well, and in the 0.4 release candidate this has been changed. See the 0.4-dev docs here. Have a look at the API documentation of the LocalDateTime class.

Sorry for being dense, but I'm still not understanding the intended behavior of LocalDateTime.
On the one hand, the document says: "Instances have the fixed offset of the system timezone at the time of initialization." That says to me that it's the same as an OffsetDateTime, with a fixed UTC offset that existed at the time of creation.
On the other hand, the doc also says: "Unlike OffsetDateTime, it knows about the system timezone and its DST transitions." That says to me that it's a ZonedDateTime with a tzinfo that defined the OS timezone at the time of creation.
I'm confused why the LocalDateTime/SystemDateTime cannot be just a ZonedDateTime, with its tzinfo set to be the OS tzinfo at the time of creation.

Sorry for being dense, but I'm still not understanding the intended behavior of SystemDateTime.

Not at all. SystemDateTime is obviously the most unusual of the classes. If it isn't clear to you, it probably isn't clear to the general public.

I'm confused why the SystemDateTime cannot be just a ZonedDateTime,

Assuming we keep ZonedDateTime for IANA timezones (which is a very common use case consistent across languages), this isn't possible. An operating system isn't in any way guaranteed to have a IANA-compatible timezone configured.

with its tzinfo set to be the OS tzinfo at the time of creation.

Note there there is no system tzinfo or ZoneInfo. There is only the ability to use datetime.astimezone(None) to convert to/from the local system timezone.

That says to me that it's the same as an OffsetDateTime, with a fixed UTC offset that existed at the time of creation.

The underlying data that SystemDateTime stores may be the same as OffsetDateTime, but its behavior is different. You can shift a SystemDateTime (i.e. it supports + and -) and it will account for the system DST. You cannot do this with OffsetDateTime.

I've been pondering why we have a disconnect on this topic, and I think it comes to this: I get the impression that you are restricting theZonedDateTime class to support only IANA timezones (more precisely, it only supports subclasses of the tzinfo that handles only IANA timezones). Whereas, I tend to think of ZonedDateTime as capable of supporting any valid subclass of tzinfo. (This is probably the design philosophy that leads me to ask if ZonedDateTime should accept a tzinfo instead of a str, as wrote in #65).
From my understanding of Linux/Unix/POSIX machines, the system could be configured with an IANA timezone, but the system could also be configured with an arbitrary timezone rule such as the POSIX-style DST transition rules using the TZ environment variable. I am not familiar with how Python datetime supports such arbitrary timezone specifications. (To be honest, I have never used those datetime functions with tz=None. I have only used them with an explicit tzinfo from pytz, dateutil, or zoneinfo.) After reading the documentation for datetime.astimezone(tz=None) and the docs for the tzlocal package for example, it looks like Python datetime creates a special subclass of tzinfo to represent the "system timezone" on POSIX systems.
This is where I would say that the LocalDateTime/SystemDateTime should be just the ZonedDateTime class with whatever tzinfo class is needed to represent the "system" timezone. Because the behavioral difference between the "system timezone" and a valid "IANA timezone" is not in the DateTime class. The difference is fully defined by the tzinfo subclass.
It seems to me that primary reason that whenver needs two separate subclasses (LocalDateTime/SystemDateTime versus ZonedDateTime) is because the library has tightly coupled the concept of "time zones" (represented by tzinfo in Python) with the concept of the "date and time" system (represented by the Gregorian calendar and UTC time system DateTime). I think there is value in keeping these concepts as decoupled as possible. For one thing, it allows end-users to use a different timezone library (as I alluded to in #65), or even a completely different timezone classification that is not IANA.

I get the impression that you are restricting the ZonedDateTime class to support only IANA timezones

Yes. My approach is: "Different classes for different semantics":

UTC and fixed offsets are unambiguous, linear through time, and integrate with ISO8601 an RFC3339.
IANA zoned timezones are sometimes ambiguous. They are interoperable only thanks to tz IDs, which have near universal recognition.
local time may also be ambiguous. It may be configured in various ways (IANA tz, POSIX string, or something else—you never know), and change at any time.

Mixing semantics in one class is one of the problems with the standard library I'm aiming to avoid. The problem with a ZonedDateTime with arbitrary tzinfo is that it's semantics because overly broad. The limiting of functionality to clarify semantics is part of any well-designed library.

Back on the topic of SystemDateTime in particular: I'm on the fence whether to scrap the class or not. Here are my considerations at the moment:
The use cases
For Python programs, it is totally reasonable to want to interact with local system time. Here are the use cases I imagine:

converting to the local system timezone
converting from the local system timezone (including ambiguity)
performing arithmetic in local system timezone (i.e. when is 5 hours later)
convert a unix timestamp into the local system timezone
find out whether a time is ambiguous in the local system timezone

The alternatives
[option A] An immutable SystemDateTime class as currently implemented in 0.4-dev branch

converting to the local system timezone:

>>> dt.as_local()
LocalDateTime(2020-01-01 13:45:00+01:00)

converting from the local system timezone (including ambiguity)

>>> LocalDateTime(2020, 1, 1)
LocalDateTime(2020-01-01 00:00:00+01:00)
>>> local_dt = LocalDateTime(2023, 10, 29, 2, 15, disambiguate="earlier")
LocalDateTime(2023-10-29 02:15:00+02:00)

performing arithmetic in local system timezone (i.e. when is 5 hours later)

>>> local_dt + hours(5)
LocalDateTime(2023-10-29 06:15:00+01:00)

convert a unix timestamp into the local system timezone

>>> LocalDateTime.from_timestamp(ts)
LocalDateTime(2023-10-29 06:15:00+01:00)

find out whether a time is ambiguous in the local system timezone

>>> local_dt.is_ambiguous()
True

[option B] Using OffsetDateTime to express system local datetimes

converting to the local system timezone:

>>> dt.as_local_offset()
OffsetDateTime(2020-01-01 13:45:00+01:00)

converting from the local system timezone (including ambiguity)

>>> NaiveDateTime(2020, 1, 1).assume_local_offset()
OffsetDateTime(2020-01-01 00:00:00+01:00)
>>> local_dt = NaiveDateTime(2023, 10, 29, 2, 15).assume_local_offset(disambiguate="earlier")
OffsetDateTime(2023-10-29 02:15:00+02:00)

performing arithmetic in local system timezone (i.e. when is 5 hours later)

>>> (local_dt.as_utc() + hours(5)).as_local_offset()
OffsetDateTime(2023-10-29 06:15:00+01:00)

convert a unix timestamp into the local system timezone

>>> UTCDateTime.from_timestamp(ts).as_local_offset()
OffsetDateTime(2023-10-29 06:15:00+01:00)

find out whether a time is ambiguous in the local system timezone

>>> try:
...     local_dt.naive().assume_local_offset(disambiguate="raise")
... except Ambiguous:
...     ambiguous = True
... else:
...     ambiguous = False

Other considerations
why drop the class:

The semantics of LocalDateTime are not obvious at first glance
There aren't many other libraries that have an equivalent. This is a signal that it may not be needed.

why keep the class:

the class conveniently encapsulates interactions with the system timezone

I get the impression that you are restricting the ZonedDateTime class to support only IANA timezones

Yes. My approach is: "Different classes for different semantics":
* UTC and fixed offsets are unambiguous, linear through time, and integrate with ISO8601 an RFC3339.

* IANA zoned timezones are sometimes ambiguous. They are interoperable only thanks to tz IDs, which have near universal recognition.

* local time may also be ambiguous. It may be configured in various ways (IANA tz, POSIX string, or something else—you never know), and change at any time.

* highly customized datetimes. Advanced users may want to create arbitrary variable-offset zones. Because these are specific to the user, they are not interoperable with other systems.

My argument is that those 4 things are exactly the same, in the sense that they are all DateTime that use the Gregorian calendar system and UTC time (more pedantically, POSIX time, because I don't  think whenever deals with leap seconds). The semantic differences in those things are their timezones, or more accurately, how they choose to encode their UTC offsets (fixed, IANA, POSIX-style, or something custom).

Let me present my thoughts from a different angle.
I think you have correctly identified the usability problem in the datetime.datetime library, which conflates "naive datetime" with the "aware datetime" into a single datetime class. Your NaiveDateTime and AwareDateTime separation is the right call.
Maybe I'm suggesting that the pendulum may have swung too far with the creation of numerous subclasses to model the different use cases of AwareDateTime. I tend to favor composition and aggregation instead of inheritance for most problems. I consider the various subclasses to be just an AwareDateTime with different TimeZone instances. Some of those subclasses are useful for ergonomic reasons for the programmer. To me, that's a different argument from saying that they capture different semantics.

Absolutely agree. They are the same in that they represent a moment in POSIX timeline, and their "only" difference is how their UTC offset is defined.
I'd argue though, that it's exactly this difference that profoundly affects the behavior of the class! Addition/subtraction, parsing, formatting, and disambiguation are all but defined by their handling of the offset.
😉 Taking a cheeky straw-man metaphor here: imagine if a geometry library had one class Shape to describe squares, circles, triangles and the like. After all, they all have in common that they're bounded shapes in 2 dimensions. However, isn't it more valuable to tailor the classes to the differences instead of the commonalities? Having a radius attribute on Circle, but not on Square?

Maybe I'm suggesting that the pendulum may have swung too far with the creation of numerous subclasses to model the different use cases of AwareDateTime. I tend to favor composition and aggregation instead of inheritance for most problems. I consider the various subclasses to be just an AwareDateTime with different TimeZone instances.

I think we're getting to the core here. Would you consider that different TimeZone subclasses would be warranted?

Some of those subclasses are useful for ergonomic reasons for the programmer. To me, that's a different argument from saying that they capture different semantics.

Important not to confuse the two, I'll reflect on this.

Hi, my bits:
I think this is extremely simple.
I think LocalDateTime should not exist:

It is not always possible to know the local date time
You can not pretend to have LocalDateTime to be dynamic. It should be up to the users to update this whenever is needed.
whenever could implement some methods (there are plenty) to extract the system local datetime like tzlocal does. But this should return one of the other possible datetime clases (naive if no local datetime could be extracted, Iana or offset).
but, at the end, the local date time is something that can be provided (or not) and it's up to the user to account for that.

Just my thoughts
Keep it simple

Also I don't see a use case for UTCDateTime that can't be done with Zoned, Offset or Naive.

Iana has a UTC timezone... so Zoned is in fact UTCDateTime
Offset as well as it's +00:00
and last, you can use Naive and interpret it as UTC.

I understand why you create a UTCDateTime class, but I don't think is necessary
For example:
utcdate = UTCDateTime(2022, 1, 1, 12)  # it's utc
offsetdate = OffsetDateTime(2022, 1, 1, 12, offset=hours(0))  # it's utc as well
zoneddate = ZonedDateTime(2022, 1, 1, 12, tz='UTC')  # it's utc as well

print(f'UTCDateTime is {utcdate}')
print(f'OffsetDateTime is {offsetdate}')
print(f'ZonedDateTime is {zoneddate}')

print(utcdate == offsetdate == zoneddate)

This outputs:
UTCDateTime is 2022-01-01T12:00:00Z
OffsetDateTime is 2022-01-01T12:00:00+00:00
ZonedDateTime is 2022-01-01T12:00:00+00:00[UTC]
True

So the difference here is:

The representation of the datetime ('Z' vs '+00:00' vs '+00:00[UTC]'):
This is easy: a user should be able to get the representation he prefers from the other classes if possible.

The methods available in each class (from text representations, offset methods, to/from tz, etc.)
This is also easy. I can't think of any method present in UTCDateTime that is not possible to implement in the other classes.

Either way, let the user decide if he wants UTC in Zoned, Offset or Naive datetimes...
Hope this adds up to the issue.

I think LocalDateTime should not exist:

It is not always possible to know the local date time

Is this the case? Doesn't every OS have a system time? The python standard library (both time and datetime) don't mention any caveats. A system local time seems to be available at all times.

You can not pretend to have LocalDateTime to be dynamic. It should be up to the users to update this whenever is needed.

Depends what exactly you mean with dynamic. I've already amended it to no longer 'automatically' adjust to changes in the system timezone. The user is in charge when to update this. See the API docs for upcoming release, what do you think?

whenever could implement some methods (there are plenty) to extract the system local datetime like tzlocal does. But this should return one of the other possible datetime classes (naive if no local datetime could be extracted, Iana or offset).

The reason I chose for a fixed-offset is that this is what Python's astimezone(None) returns. Fixed offset seems like the most that can be relied on from an OS in general. tzlocal can always be used to try to get the IANA name.

but, at the end, the local date time is something that can be provided (or not) and it's up to the user to account for that.

True that explicitly providing the timezone can be useful. In calendar applications, for example, you can have the user explicitly set it.
However, for smaller things like CLI tools, you can totally imagine wanting to just print a timestamp in whatever timezone the user expects, right?
print(f"Your download is expected to be ready at {LocalDateTime.now() + hours(1)")

Keep it simple

I can definitely relate to this feeling. What would you think of removing LocalDateTime from the README example code, and describing it on a special 'advanced features' page (along with other niche features).

Also I don't see a use case for UTCDateTime that can't be done with Zoned, Offset or Naive.

Correct, but: the value of a class isn't just in what it supports, but also in what it doesn't.
Example: imagine encountering this in a codebase:
class ChatMessage:
    sent: UTCDateTime
    text: str

This tells us that there is no local time stored with the message. This knowledge is invaluable for all sorts of features you might want to build: how to display such a time, how to store it in a database, etc.

Well then LocalDateTime should disappear and just have something like the following.

This is definitely something I've considered. Have a look at the second part of this comment: https://github.com/ariebovenberg/whenever/issues/30#issuecomment-1962890545.
The reason LocalDateTime may be useful it that it makes certain actions clearer:
(legend: A = using LocalDateTime class, B = using OffsetDateTime instead)

converting to the local system timezone:

# A
dt.as_local()
# B
dt.as_local_offset()

converting from the local system timezone (including ambiguity)

# A
LocalDateTime(2023, 10, 29, 2, 15, disambiguate="earlier")
# B
NaiveDateTime(2023, 10, 29, 2, 15).assume_local_offset(disambiguate="earlier")

performing arithmetic in local system timezone (i.e. when is 5 hours later)

# A
local_dt + hours(5)
# B
# note this isn't exactly the same for periods containing days, months, etc.
(local_dt.as_utc() + hours(5)).as_local_offset()

convert a unix timestamp into the local system timezone

# A
LocalDateTime.from_timestamp(ts)
# B
UTCDateTime.from_timestamp(ts).as_local_offset()

Also I don't see a use case for UTCDateTime that can't be done with Zoned, Offset or Naive.

Correct, but: the value of a class isn't just in what it supports, but also in what it doesn't.
Example: imagine encountering this in a codebase:
class ChatMessage:
    sent: UTCDateTime
    text: str

This tells us that there is no local time stored with the message. This knowledge is invaluable for all sorts of features you might want to build: how to display such a time, how to store it in a database, etc.

Well my system can definitely be set at utc but anyway, if you look it this way we should have 1000 classes to describe exactly what contents you should expect at a glance.
I thing there’s a balance that for me goes into “practicality beats purity” 99% of the time.
A docstring will suffice for this usecase.
The other 3 classes (zoned, offset and naive) are really useful and I think everything fits inside.
Anyway, just my thoughts. Keep the great work!

Well then LocalDateTime should disappear and just have something like the following.

This is definitely something I've considered. Have a look at the second part of this comment: #30 (comment).
The reason LocalDateTime may be useful it that it makes certain actions clearer:
(legend: A = using LocalDateTime class, B = using OffsetDateTime instead)

converting to the local system timezone:

# A
dt.as_local()
# B
dt.as_local_offset()

converting from the local system timezone (including ambiguity)

# A
LocalDateTime(2023, 10, 29, 2, 15, disambiguate="earlier")
# B
NaiveDateTime(2023, 10, 29, 2, 15).assume_local_offset(disambiguate="earlier")

performing arithmetic in local system timezone (i.e. when is 5 hours later)

# A
local_dt + hours(5)
# B
# note this isn't exactly the same for periods containing days, months, etc.
(local_dt.as_utc() + hours(5)).as_local_offset()

convert a unix timestamp into the local system timezone

# A
LocalDateTime.from_timestamp(ts)
# B
UTCDateTime.from_timestamp(ts).as_local_offset()

I work with local times all the time in my project O365 (using tzlocal extensively) and I really don’t see what a specific LocalDateTime class will help me with. I kind of don’t needed. In O365 I just store the local timezone and work with it. It’s a regular datetime…
I still think you don’t need a specific class for this. Again this is maybe a practicality vs purity kind of thing.

IANA zoned timezones are sometimes ambiguous.

I don’t see how a datetime with IANA timezone could be ambiguous. Can you provide an example?
I think is the least ambiguous one
thanks

I don’t see how a datetime with IANA timezone could be ambiguous. Can you provide an example?

Constructing a IANA zoned datetime sometimes requires disambiguation. For example, the below datetime is ambiguous:
ZonedDateTime(2023, 10, 29, 2, 15, tz="Europe/Amsterdam")

Necessary to do:
ZonedDateTime(2023, 10, 29, 2, 15, tz="Europe/Amsterdam", disambiguate="later")

I don’t see how a datetime with IANA timezone could be ambiguous. Can you provide an example?

Constructing a IANA zoned datetime sometimes requires disambiguation. For example, the below datetime is ambiguous:
# at 3am the clock is moved 1 hour back (DST). 2:15am occurs twice
ZonedDateTime(2023, 10, 29, 2, 15, tz="Europe/Amsterdam")

Necessary to do:
ZonedDateTime(2023, 10, 29, 2, 15, tz="Europe/Amsterdam", disambiguate="later")

Ok, True. I get it. Does any timezone format (iso, or whatever) encodes if it’s the earlier 2:15 or the later one?

Ok, True. I get it. Does any timezone format (iso, or whatever) encodes if it’s the earlier 2:15 or the later one?

AFAIK there are two approaches to encode this:

what python datetime does:  a 'fold' parameter. fold=0 is the first, and fold=1 is the second
what everybody else seems to do: store the UTC offset.

You can see this in the canonical ZonedDateTime format: 2023-10-29 02:15:00+01:00[Europe/Amsterdam] vs. 2023-10-29 02:15:00+02:00[Europe/Amsterdam]

ZonedDateTime should always expose the offset along with the timezone name.

Indeed, it does 😄
This reminds me of why UTCDateTime, OffsetDateTime, and ZonedDateTime all exist:

UTCDateTime
OffsetDateTime
ZonedDateTime

represents a moment in time
X
X
X

represents a local time with UTC offset

X
X

Knows how UTC offset changes over time

X

An improved LocalSystemDateTime is now part of release 0.4


chore: removed network ID parameter in Settings page URL

Motivation
Going through the /settings page, I found out that the parameter network in the URL is not used, nor necessary for the sub-components.
Changes
Removed the network parameter from the /settings page.
Before:
https://oisy.com/settings/?network=ETH
After:
https://oisy.com/settings/
Tests
Manual tests throughout the Settings page on local development env.

I did not double check but, I would bet the parameter is required otherwise when you go back the network is lost.
e.g. ICP -> Settings -> Back -> ETH 🫠

I did not double check but, I would bet the parameter is required otherwise when you go back the network is lost. e.g. ICP -> Settings -> Back -> ETH 🫠

Indeed, it goes back to the "default" network, that right now is ETH. I thought this could be acceptable for now, since we will not have a "network" to go back to, for the  single-page view, no?

I think this should remains in place until it's effectively decided to implement a single-page view and how - i.e. not before.

Then closing this PR, to revisit in future.


Getting tp, tn, fp, and fn for each class in the case of Multi-class classification

Hi,
I have a multi-class classification and I need to evaluate the classifier based on micro-averaging measures, e.g. micro-precision, micro-recall and micro-f1. For this purpose, I need to get tp, tn, fp, fn of each class of the problem. Unfortunately, it is not possible to get such information from MulticlassClassificationEvaluator class. Can you please add such capabilities.
Thanks in advance.

Hi Nasrin,
Currently KeLP does not provide this capability but I'm planning to add it soon. Perhaps, I will add it within this week. I will write here as soon as it will be available.
Regards,
Simone

Hi Nasrin,
I included the capabilities you asked in the development branch of kelp (version 2.2.2-SNAPSHOT). Please check it and let us know if everything is properly working.
Thanks,
Simone


project.search method does not work

I tried using this wrapper's search method with no luck. I tried
project.search('Destroy death star')
after setting up the client and the project variables correctly.

@tverbrugge-athn what error were you seeing? I'm currently getting a 400, Your client has issued a malformed or illegal request.  That's all we know.

The issue appears to be here:
https://github.com/dashofcode/tracker_api/blob/master/lib/tracker_api/endpoints/search.rb#L13
Unsure what the intent of the options[:body][:query] hash is, instead fixing it this way:
options[:params] = { query: query }

@mattyohe I think the intent was not to overwrite an existing body. Also, need to make sure not to overwrite existing options[:params].
Want to send a pull request?


MBL-1394 Remove RXJava usage from new AddOnsViewModel

📲 What
Refactor the AddOnsViewModel  to remove all RxJava usages.
🤔 Why
AddOnsViewModel was originally written with a mix of RxJava and Coroutines, and we no longer want it to contain RxJava.
🛠 How
👀 See
No user facing changes.
📋 QA
Open a late pledges project and confirm the AddOnsVM is working the same as before.
Story 📖
https://kickstarter.atlassian.net/browse/MBL-1394

Codecov Report
Attention: Patch coverage is 76.92308% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 67.77%. Comparing base (13639aa) to head (4ff1ed9).

Files
Patch %
Lines

...kstarter/viewmodels/projectpage/AddOnsViewModel.kt
76.92%
5 Missing and 1 partial :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2044      +/-   ##
============================================
- Coverage     67.80%   67.77%   -0.04%
+ Complexity     2097     2094       -3
============================================
  Files           353      353
  Lines         21570    21559      -11
  Branches       3044     3044
============================================
- Hits          14626    14611      -15
- Misses         5304     5307       +3
- Partials       1640     1641       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add --log-level flag to agent command.

Description:
Add --log-level flag to agent command.
Configures log level with debug, info, warn, error, panic, or fatal.
Why:

We can configure the log level on starting the agent.

Could i try this issue? :)


Any rules on PR?

I would like to contribute back to this library (we made a stripe library for rust but we did use some parts from this library to complete some sections of our code during our rewrite) but would like to know if there is any specific formats, comments, etc., that is prefer before making a pr.

Sorry I missed this somehow.  Yes:

Minimize your changes to just the intended feature (no formatting/etc fixes).
Create one PR for each feature (or group of related features)

And generally follow best practices. See https://github.com/rapiditynetworks/stripe-rs/pull/6 for an example.


Call to undefined method setencryptions

Hi guys,
when index.php?r=user%2Fregistration%2Fregister
it says:
PHP User Error – yii\base\ErrorException
Call to undefined method setencryptions
whats mean?

Q

Yii2 2.0.9

Yii2-user <IP_ADDRESS>

PHP 5.6.8

Yii2-user does not use any methods named setencryptions. If problem still exists, then provide more information and reopen the issue.


resetting schedule to an old version can make submissions unavailable

create submission: sub1
publish a schedule containing only sub1: ScheduleA
create submission: sub2
publish a schedule containing both sub1 and sub2: ScheduleB
reset the current draft to ScheduleA

Sub2 is not part of the current draft and should therefore be visible in the list on the right. But the list is empty. Resetting the draft to ScheduleB restores the talk.

Schedule.unfreeze gets rid of all talks in the current working draft. It then adds all talks, which are present in ScheduleA. Since sub2 did not exist back then, it does not get added in. I think the most practical solution would be to also add any talks out of the old working draft, which can't be found in ScheduleA. All talks added this way should have their room set to null. This way they'll show up in the sidebar as expected.


Instead of individually aliasing types, maybe reexport http

Related to #217. Currently, we're aliasing types from the http crate. It might be potentially better to reexport the whole http crate? Particularly since we've been encouraging a use bulwark_wasm_sdk::*; pattern, I think this would just result in plugins getting the familiar patterns of http::Request, http::Response, etc, while being less brittle to any changes in the http crate.

I forgot Request and Response are generic types, so we need those to be http::Request<Bytes> so those two aliases should stick around. The rest can get reexported as http.


related_field not recognised in URL path

We are using RelationshipView from the DRF JSON:API package, and are experiencing limitations when using the schema tester on these type of endpoints.
We have the following path/view that handles the application of team members (POST/DELETE)
    re_path(
        r"^team/(?P<pk>\d+)/relationships/(?P<related_field>[-\w]+)/?$",
        views.TeamMembersRelationshipView.as_view(),
        name="team-members-relation",
    ),

When SchemaTester(schema_file_path=path/to/schemae/file.yaml) hits this path
        response = self.client.delete(
            reverse(
                "teams:team-members-relation",
                kwargs={
                    "pk": self.team_001.pk,
                    "related_field": "members",
                },
            ),
            payload,
            content_type="application/vnd.api+json",
        )
        self.assertResponse(response)

We receive an UndocumentedSchemaSectionError
Traceback (most recent call last):
  File "../.venv/lib/python3.9/site-packages/openapi_tester/schema_tester.py", line 102, in get_key_value
    return schema[key]
KeyError: '/api/team/{id}/relationships/{related_field}'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../.venv/lib/python3.9/site-packages/openapi_tester/schema_tester.py", line 403, in validate_response
    response_schema = self.get_response_schema_section(response)
  File "../.venv/lib/python3.9/site-packages/openapi_tester/schema_tester.py", line 144, in get_response_schema_section
    route_object = self.get_key_value(
  File "../.venv/lib/python3.9/site-packages/openapi_tester/schema_tester.py", line 104, in get_key_value
    raise UndocumentedSchemaSectionError(
openapi_tester.exceptions.UndocumentedSchemaSectionError: Error: Unsuccessfully tried to index the OpenAPI schema by `/api/team/{id}/relationships/{related_field}`.

Undocumented route /api/team/{id}/relationships/{related_field}.

Documented routes:
        • ...
        • /api/teams
        • /api/team/{id}
        • /api/team/{id}/relationships/members

Why isn't the (undocumented) path resolving /api/team/{id}/relationships/{related_field} to the (documented) desired path /api/team/{id}/relationships/members as expected?
Is there something additional that needs to be defined in order for the schema tester to recognise these related_field path keys?

This is the relevant code tasked with resolving paths: https://github.com/snok/drf-openapi-tester/blob/master/openapi_tester/loaders.py#L141
My guess is that the answer will be found there if you take a look. A PR with a fix is, as always, welcome 🙂 I could also take a look at this at some point, but would need reproducible code in some form or another

I could also take a look at this at some point, but would need reproducible code in some form or another

Thanks for your reply. I will try to push reproducible code as you have described here

I was able to reproduce this locally, however when I push to the PR it fails. Seems it cannot import rest_framework_json_api module?
Example:
FAILED tests/test_django_framework.py::TeamsAPITests::test_schema_using_assert_response - ModuleNotFoundError: No module named 'rest_framework_json_api'
Could you please have a look and assist me in how to get this operational. Thanks

I'll try to take a look at both issues tomorrow 👍

@sondrelg I fixed the failing test in relation to the missing rest_framework_json_api module. (I updated .github/workflows/testing.yml and .pre-commit-config.yaml)
The latest commit failing in the PR now reproduces the error.
I have updated the PR description with latest.

@sondrelg  Did you get the opportunity to look into this issue, from the reproducing PR?


Updating INIT usage in lut mapping for AP3 and fixing a bug in AP2 lu…

…t mapping

@lnsharma Once Yosys LUT mapping is corrected both simulation model for LUT4_CARRY and LUT initializations in VPR adder techmaps will have to be fixed as well:
https://github.com/lnsharma/symbiflow-arch-defs/blob/340b91710caf1b2df60c1ffea0d2f8253d20cf14/quicklogic/ap3/primitives/lut4/lut4_carry.sim.v#L29
https://github.com/lnsharma/symbiflow-arch-defs/blob/quicklogic-upstream-rebase/quicklogic/ap3/techmap/adder_map.v


pull request for e2e test at 02/05/2017 14:26:08

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Pass a JSON string and get an error mismatched or missing input parameter

System Info
LangChain: 0.0.230
Python: 3.10
Who can help?
No response
Information

[ ] The official example notebooks/scripts
[X] My own modified scripts

Related Components

[ ] LLMs/Chat Models
[ ] Embedding Models
[X] Prompts / Prompt Templates / Prompt Selectors
[ ] Output Parsers
[ ] Document Loaders
[ ] Vector Stores / Retrievers
[ ] Memory
[ ] Agents / Agent Executors
[ ] Tools / Toolkits
[ ] Chains
[ ] Callbacks/Tracing
[ ] Async

Reproduction
I have a dictionary in python

dict = {
                          "keyfeatures": [
                              {
                                  "title": "Search for Charitable Activities",
                                  "description": "The system must provide a search function that allows users to enter keywords and returns a list of charitable activities that match those keywords."
                              },
                              {
                                  "title": "Display Charitable Activities",
                                  "description": "The system must display the search results in a user-friendly format."
                              },
                              {
                                  "title": "Filter and Sort Charitable Activities",
                                  "description": "The system must provide options for users to filter and sort the search results."
                              },
                              {
                                  "title": "View Details of a Charitable Activity",
                                  "description": "The system must allow users to select a charitable activity from the search results and view more detailed information about it."
                              },
                              {
                                  "title": "Save or Bookmark Charitable Activities",
                                  "description": "The system must allow users to save or bookmark charitable activities that they're interested in."
                              }
                          ]
                      }

i convert the dictionary in a json string

json_string = json.dumps(dict)

and i obtain the following string

{"keyfeatures": [{"title": "Search for Charitable Activities", "description": "The system must provide a search function that allows users to enter keywords and returns a list of charitable activities that match those keywords."}, {"title": "Display Charitable Activities", "description": "The system must display the search results in a user-friendly format."}, {"title": "Filter and Sort Charitable Activities", "description": "The system must provide options for users to filter and sort the search results."}, {"title": "View Details of a Charitable Activity", "description": "The system must allow users to select a charitable activity from the search results and view more detailed information about it."}, {"title": "Save or Bookmark Charitable Activities", "description": "The system must allow users to save or bookmark charitable activities that they're interested in."}]}

if i pass that string to an AIMessagePromptTemplate

AIMessagePromptTemplate.from_template(msg)

i get the following error:

File "/home/andrea/PycharmProjects/ArchAI/venv/lib/python3.10/site-packages/langchain/prompts/chat.py", line 85, in from_template
    prompt = PromptTemplate.from_template(template, template_format=template_format)
  File "/home/andrea/PycharmProjects/ArchAI/venv/lib/python3.10/site-packages/langchain/prompts/prompt.py", line 145, in from_template
    return cls(
  File "/home/andrea/PycharmProjects/ArchAI/venv/lib/python3.10/site-packages/langchain/load/serializable.py", line 74, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for PromptTemplate
__root__
  Invalid prompt schema; check for mismatched or missing input parameters. '"title"' (type=value_error)

i have already added f-string formater with multiple curly-brackets but still fail
thanks in advance
Expected behavior
the PromptMessage must import a formatted JSON string correctly without interfere with internal template curly-brackets format

The answer is not totally correct
imagine that i have a prompt where i would represent as an example a prompt question and an answer that is a json
to pass that json string I need to put a JSON string on that prompt.
The question is:
How i would put a JSON string into a prompt?
Thanks in advance

I found a solution:
just use jinja as template format

AIMessagePromptTemplate.from_template(msg, template_format="jinja2")

and the prompt variables must use {{variable}} and not {variable}


Introduce Spring Security

Encrypt passwords when persisting to db
Refactors member endpoints to use JdbcUserDetailsManager
Add Basicauth

Pull request: https://github.com/molgergo01/BBG-Pictures-Repository-Backend/pull/26

Completed in: b4bba9618a00ce6eca6520caec0ace6bee3abfc1


Defining env variables in Subscription creates two ReplicaSets

Bug Report

What did you do?
Create a Subscription with environment variables defined via spec.config.env:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: my-infinispan
  namespace: namespace-for-testing
spec:
  channel: 2.2.x
  name: infinispan
  source: operatorhubio-catalog
  sourceNamespace: olm
  config:
    env:
      - name: example-env
        value: some-value

What did you expect to see?
I expect the operator's deployment to be created with a single ReplicaSet containing the configured environment variables.
What did you see instead? Under which circumstances?
The deployment was created with two ReplicaSet instances. The first ReplicaSet does not contain the defined environment variables, but the second does. Eventually only the second ReplicaSet remains in an active state and the operator pods have the environment variables as expected.
The problem is that while the first ReplicaSet is active, the Deployment's readiness probes can pass and the operator pods without the environment variables will start reconciling CR resources. This is problematic in our case, as we use env variables in the operator pods to define default annotation/labels that must be applied to CR resources. Therefore CRs reconciled when the first ReplicaSet is active will be missing these defaults.
Environment

operator-lifecycle-manager version:

v.0.20.0

Kubernetes version information:

v.1.21.1

Kubernetes cluster kind:

kind

We're hitting this same issue when installing the RHOAI operator. We add the following to our subscription:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: rhods-operator
  namespace: redhat-ods-operator
spec:
  channel: stable
  installPlanApproval: Automatic
  name: rhods-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace
  config:
    env:
    - name: DISABLE_DSC_CONFIG
      value: "True"

The above env variable is supposed to prevent the creation of a so-called default-dsci object. The result is, however, very unpredictable depending on timing.
The reason is that OLM creates the deployment without the DISABLE_DSC_CONFIG env variable first, the operator pod will be spawned without this env variable and this default-dsci object will be created. Only later the deployment gets patched and a new ReplicaSet gets triggered with the correct environment variable. At this point it is too late though since the object has already been created.


Negotiations in client-auth and server-auth

We would like to introduce negotiable parameters that are being sent in client-auth and server-auth:

[ ] Keep alive interval (integer in the range of 0..3600 seconds) in client-auth where 0 means no keep alive.
Task (required) (makes no sense between client and server)

@dbrgn Do you think having a keep alive would make sense? Personally, I think the clients should just send a ping themselves if they want to keep alive the WS connection. I know that the previous Java client I've used for the Web Client prototype was not able to send pings which was the original reason to have keep alives. Maybe we should just wait out the big beta test and then decide if we need it?

Right now we configured our SaltyRTC server (proxied behind Nginx) to set a keep-alive value of 5 minutes IIRC. But pings would probably make sense. I'd leave that to the client library implementation though. Or do you see any advantages in negotiating these parameters?
The Java library used supports PING/PONG messages: https://github.com/TooTallNate/Java-WebSocket/issues/213

Note that the Websocket is only open during connection build-up. So the only time we're waiting for more than a few seconds is when waiting for the initiator/responder to connect. Not sure if it's useful to do that for more than a few minutes. (Although some use cases might profit from it...)

The only advantage of negotiating this would be for clients that don't support sending ping messages on their own - for example the browser WS API offers no method to send pings. It's not well defined how the connection is kept alive here between client and server.
It wouldn't take a lot of effort to implement this in the server though. And the message change to 'client-auth' is negligible. (In fact, this isn't even a negotiation as the server simply accepts the request to keep alive.)

It's already implemented and active in the server. However, the message field is not. So, I might actually add this one as an optional feature for servers.


Add @bigtest/interactor testing examples

What is this?
First, I want to say this is such an awesome idea and the site site really well put together. I love how the exact source files are used in the examples. Well done, I love it! 👏 😃
This PR adds testing examples for @bigtest/interactors. Interactors are composable and immutable component page objects. I think they fit in really nicely with the other two examples.
Let me know if there's anything I missed!

@Robdel12 Thanks for your contribution! It's live: https://react-testing-examples.com/jest-interactor/


Animated GIF become JPEG

Hi,
When I want to use an animated GIF with the picker, the image I receive is a JPEG.
Do you know how to tell the picker this image is a GIF and aplly to it a GIF data64?
I'm on Android, my piece of code:
selectPhotoTapped() {
        const options = {
            quality: 1.0,
            maxWidth: 800,
            maxHeight: 800,
            title: 'Select a picture',
            cancelButtonTitle: 'Cancel',
            title: 'Select a picture',
            cancelButtonTitle: 'Cancel',
            takePhotoButtonTitle: 'Take picture',
            chooseFromLibraryButtonTitle: 'Phone'
            //noData: false, // we use response.data to display gif
            //allowsEditing: false // make sure we don't edit the gif
        };
        ImagePicker.showImagePicker(options, (response) => {
            if (response.didCancel) { }
            else if (response.error) { alert('ImagePicker Error: ', response.error); }
            else if (response.customButton) { alert('User tapped custom button: ', response.customButton); }
            else {
                var source;
                source = {uri: 'data:image/jpeg;base64,' + response.data, isStatic: true};
                //Or:
                //if (Platform.OS === 'android') { source = {uri: response.uri, isStatic: true}; }
                //else { source = {uri: response.uri.replace('file://', ''), isStatic: true}; }
                //this.setState({ avatarSource: source });
                this.setState({ photo: source });
                this.setState({ photoUpload: source.uri });
            }
        });
}

Thank you for your help!

#106

Thank you @yfuks but I already tried this solution: no auto resize, no quality, add noData: false, allowsEditing: false
I saw this in the Java code (ImagePickerModule, line 659), I don't know if it is related to my issue:
String filename = "image-" + UUID.randomUUID().toString() + ".jpg";

I tried this code to be sure that the module exports a GIF:
const re = /(?:\.([^.]+))?$/; type = re.exec(response.uri); alert(type[1]);
But it's always alert "jpg", expect for this image where I receive nothing: http://ressources.blogdumoderateur.com/2013/02/gif-anime.gif

const options = {
            // quality: 1.0, // we don't change the quality
            // maxWidth: 800, // we don't resize
            // maxHeight: 800, // we don't resize
            title: 'Select a picture',
            cancelButtonTitle: 'Cancel',
            title: 'Select a picture',
            cancelButtonTitle: 'Cancel',
            takePhotoButtonTitle: 'Take picture',
            chooseFromLibraryButtonTitle: 'Phone'
            //noData: false, // we use response.data to display gif
            //allowsEditing: false // make sure we don't edit the gif
        };

What is the filename of the file you get ?

Merci yfuks ;)
My problem is that the component always render a JPG file, you can try if you load an animated GIF and export the extension filename:
... var source; const re = /(?:\.([^.]+))?$/; type = re.exec(response.uri); alert(type[1]); ...
You will get JPEG/JPG.
If I change the data:image to gif, Imagick (PHP treatment) tell me that this GIF is not in a good format: "no decode delegate for this image format"
If I don't force resize, any GIF file will not have render in the app, just a blank. If I force resize I will have the render in the app but animation will be lost.
Thanks for your help!

If I change the data:image to gif, Imagick (PHP treatment) tell me that this GIF is not in a good format: "no decode delegate for this image format"

Are you sure that your Imagick support gif ? stackoverflow

If I don't force resize, any GIF file will not have render in the app, just a blank. If I force resize I will have the render in the app but animation will be lost.

If you force resize that basically transform your image into jpg. So don't do it if you want gif.
To display animated gif in your Android app : react native doc

Imagick supports animated GIF: http://stackoverflow.com/questions/13997518/php-imagick-create-gif-animation
Thanks for the link to display GIF in Android, I saw it before, but I can't upload animated GIF so I will see that at the right moment ;)

If I don't force resize on animated GIF, it will be uploaded as a JPG (the fille type is undefined regarding to type[1]).

And in response.type ?

response.type image/gif

You can't always get the file extension with the uri, the uri don't always have the file extension in it.
response.type is there for the purpose you want.

If I don't force resize on animated GIF, it will be uploaded as a JPG

And know i repeat my self but don't resize the image if you want a gif

response.type image/gif

so you have a gif you know how to display it you are SURE that your version of Imagick support gif and you still have issue ?

Thank you for response.type I will use it now.

And know i repeat my self but don't resize the image if you want a gif
That's the point, I don't force resize to be sure i'm getting GIF but the image I receive is a JPG.

I'm sure Imagick is working fine, I've tested it with a classic web upload.

Can you paste your upload code and your picker code ?

My bad, I sent full header to Imagick, not just the data64. Thanks for your time!

@yfuks in my case I am resizing the image deliberately, I just want a way that when the gallery of my photo opens, we filter out gifs ... so my gallery would only show image/jpeg or image/png ONLY. so that the user does not get a chance to select gifs


Add some labels as described at label-schema.org

Following on from Shaun's suggestion on this PR here's labelling applied to the Calico for containers image

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

Please hold off merging this PR until I've had chance to digest and discuss the proposed org.label-schema proposal.  There are a lot things going with labels at the moment and we need to make sure we are aligning them all appropriately.  Hopefully this will prove to be complimentary.

@lxpollitt any update on this?


add support for skos:example

This fixes #124

@droi, can you please deploy this to test alongside #119?

Deployed alongside #119 to test.

1+ Looks good, see e.g. https://test.skohub.io/acka47/hcrt/heads/master/w3id.org/kim/hcrt/portal.de.html. @dr0i this is ready to be deployed on production.

Deployed to production, closing.


Deleted job for merged PR also gets pulled.

The below snapshot is for a job against PR, that was already merged a day ago.

Also, clicking on Build button for PR-11, basically does nothing.
May be it should show an appropriate error message.
This is in logs
jenkins.js:31 Uncaught (in promise) TypeError: Cannot read property 'parameterDefinitions' of undefined
    at jenkins.js:31
    at _ (runtime.js:63)
    at Generator._invoke (runtime.js:282)
    at Generator.e.(anonymous function) [as next] (file:///C:/Users/userId/AppData/Local/Programs/deployer/resources/app.asar/build/static/js/2.61643de5.chunk.js:1:298559)


Clean up access modifiers and documentation

opt/ is a bit messy right now; inconsistent use of private, and some documentation is missing. Also consider splitting declaration and definition into .hpp and .hxx files like pinocchio does.

A lot of documentation is missing. Will clean up this week.

Added the missing documentation. However, some members are still public but should be protected/private. Also, the namespace scheme could use some work.


Faulty solution to: Tutorial: From dimensional model to stunning report in Power BI Desktop

I followed the tutorial provided called "Tutorial: From dimensional model to stunning report in Power BI Desktop".
Mostly I managed to get through it with good visuals and learning as I went. However I struggled with the slicer adding a (Blank) selection. And the data from year 2021 apparently not getting detected.
Thought I made the mistake, but the downloaded example of a completed version also contains both problems.
I solved the (Blank) simply by filtering the (Blank) from month.
Looking into the data, I see that there are blank data located within the Sales Table under ShipDateKey
And I guess the data is limited to 2017-2020, since tracking the date keys to the sales keys there were no correlation in sales for 2021.
Thank you for the tutorial :) It was educational.

Thanks for your comment, @Gamborg1987. This issue has been assigned to the doc author for review.

Thank you for your feedback and for getting involved. I'll forward your message to the tutorial owner and hopefully we'll be able to fix the tutorial (and sample) soon.


LightTable fails to find plugins behind a proxy

LightTable fails to find plugins behind a proxy.
My environment proxy settings are set and work for other applications.
I have also set some npm proxy settings.
npm config list
gives me the following output:
; cli configs
user-agent = "npm/3.8.3 node/v5.9.1 linux x64"

; userconfig /home/zorn/.npmrc
http-proxy = "http://<proxy_ip>:<proxy_port>/"
https-proxy = "http://<proxy_ip>:<proxy_port>/"
proxy = "http://<proxy_ip>:<proxy_port>/"
registry = "http://registry.npmjs.org/"
strict-ssl = false

The error in LightTable is the following:
TypeError: Request path contains unescaped characters.
    at new ClientRequest (_http_client.js:51:11)
    at TunnelingAgent.exports.request (http.js:31:10)
    at TunnelingAgent.createSocket (/opt/LightTable/resources/app/core/node_modules/request/tunnel.js:119:25)
    at TunnelingAgent.createSecureSocket [as createSocket] (/opt/LightTable/resources/app/core/node_modules/request/tunnel.js:186:41)
    at TunnelingAgent.addRequest (/opt/LightTable/resources/app/core/node_modules/request/tunnel.js:80:8)
    at new ClientRequest (_http_client.js:133:16)
    at Object.exports.request (http.js:31:10)
    at Object.exports.request (https.js:163:15)
    at Request.start (/opt/LightTable/resources/app/core/node_modules/request/main.js:559:30)
    at Request.end (/opt/LightTable/resources/app/core/node_modules/request/main.js:1075:28)

I think this is related to this known issue: #1984

@rundis: I encountered the same problem as you I believe, and found a workaround which I described in #1984. May be it'll work for you. Kind regards