Framerate drop and CPU at 100% after loading a 'big enough' tilemap and animations

Hello,
I notice a significant CPU load (around 100 pct for Electron v16.0) - memory around 190MB.
Using IndigoEngine v 0.10 with Scala 3.1.0
and a drop in frame rate (to 2 fps) when loading a tile map (84 x 48 tiles) and each tile is 32 x 32 px.
Magnification = 2
(see animation below)
The scene contains a couple of dozen animated bushes.
I wonder if this is stretching the limits? After reading the docs: IndigoEngine Performance
, I am convinced it could be done better with some modifications/tweaking.
Remark: I can open this tile map with the animations in Tiled (see screenshot below) without any problems on the same machine.
I was thinking, one of the reasons, is that maybe the parts of the tile map and animations that are off-screen are still being rendered and causing trouble?
Other suggestions / possible causes?

First of all, thank you for the screenshots, made my day! 😍
Yes, everything you've said is correct.
You're trying to render 4,032 tiles and the standard pipeline will struggle with that. I'm very happy to help you make it go faster though - and we can for sure! - great to have a solid use case.
Two things to look at:

As you correctly said, it is rendering tons of tiles that aren't on screen, here is some discussion (https://github.com/PurpleKingdomGames/indigo/issues/194), it is a good idea to only draw the things on the screen.
Having said that, the latest Indigo has some exciting new primitives that can easily render 4032 things, but there are some tradeoffs to make: https://indigoengine.io/docs/presentation/clones-and-mutants

Have a read and let me know what you think, do swing by Discord or Gitter (links on the site or repo readme) if you'd like to discuss - otherwise happy to talk about it here too. 😄

Thanks for checking with me and OK, my first idea would start where the tile map is being constructed from the json and the spritesheet into all the different objects to be rendered.

This happens at boot time and put in the GameContext - which I haven't mentioned yet, it maybe not the best option as this can take up to 30 seconds before anything can be shown at all. The whole tile map is converted into single group as I learned from one of the examples. At a later point I may have to investigate how to use the dynamic asset loader for this, but I am not sure it can work for this scenario.

Anyway, for now it ends up in the GameContext as a 'Group'
So in the IndigoGame class under:
setup(bootData: BootData, assetCollection: AssetCollection, dice: Dice): Outcome[Startup[GameContext]]

it will do:
(...)
val maybeTiledMapGroup = for {
      t <- maybeTiledMap
      g <- t.toGroup(MyGameAssets.terrianImage) // read image from tileset json
    } yield g

val maybeAdditionalAnimations: Option[Seq[Iterable[Animation]]] = for {
      t <- maybeTiledMap
      a <- t.parseAnimations()
    } yield a

val startup = maybeTiledMapGroup match {
      case None =>
        Startup.Failure("Could not generate TiledMap from data.")

      case Some(group) =>
        maybeAdditionalAnimations match {
          case None => Startup.Success(GameContext(group, ...))

          case Some(additionalAnimations) =>
            Startup.Success(GameContext(group, ...))
            .addAnimations(additionalAnimations.flatten.toList)
        }
    }
    Outcome(startup)

and then in the scene, indeed the whole thing is being simple (re-)rendered every frame in the Scene class:
override def present(context: FrameContext[GameContext], model: CaveSceneModel, viewModel: CaveSceneViewModel): Outcome[SceneUpdateFragment] = {
    Outcome(SceneUpdateFragment(context.startUpData.group))
  }

I see two ways to proceed and optimize:

adjust the composition of the group by making use of the new data structures you suggest - the toGroup function I use can be found here: toGroup
e.g. a suspicious part of the code I introduced because the engine was complaining about duplicate sprite keys could be already be optimized, not sure how though and if it is an issue:

val key = AnimationKey((i - firstgid).toString)
Sprite(BindingKey((i - firstgid).toString + System.currentTimeMillis().hashCode().toString), 0, 0, 1, key, Material.Bitmap(assetName))

adapt the group in coming from the startup data (to a new group for presentation in that frame) with only those elements that will in fact be visible in the viewport - the view culling.

Maybe there are other approaches as well?

Load speed
The trick to improving start up time as you've correctly identified is to boot your game minimally, then load your heavy tile assets and process them later. The pirate demo does this:
https://github.com/PurpleKingdomGames/indigo/tree/master/demos/pirate
View culling
Pre-processing the group as you are doing is a good idea. The question is what to pre-process it into?
One thing to produce is CloneBatch or CloneTiles instances. These will work great if your tiles basically never change - animated maybe - but basically static backgrounds. Very very fast to render, and you can produce the data in advance, and even (if not animated) mark them as static. You can still move around these sorts of primitives by using layer level cameras.
If however your tile maps are going to change, then another approach is to create a data structure - like a QuadTree possibly, but maybe just an array or arrays or a map of groups - allowing you to chop up your map into chunks, and store them in the tree. You can query the tree with a BoundingBox (in the case of a QuadTree) that represents the part of the screen your camera is looking at and just render that bit.

Hello,
I decided to start with the CloneTiles approach, the tile map is fixed .
Note before, I mentioned the tile map is sized 4000 tiles, this is correct but it could theoretically be up to 20000, as there are 5 layers of tiles in the tile map (in the upper layers many empty tiles though- they do not end up as Graphic or as CloneTileData).
Instead of creating graphics for each tile, based on:
https://github.com/PurpleKingdomGames/indigo/blob/9f72e94f2a6957d6b04bb574a1993b7cb8176a64/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala#L122
https://github.com/PurpleKingdomGames/indigo/blob/9f72e94f2a6957d6b04bb574a1993b7cb8176a64/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala#L132
I now create a single graphic as CloneBlank which I add to the scenegraph. The scenograph is the constructed from a group of groups. Each group containing a CloneTiles(...) with all the CloneTileData (instead of a list of Graphics).
Sample of the CloneTileData array in one of the CloneTiles
CloneTileData(416,0,0,1,1,2368,544,32,32), CloneTileData(448,0,0,1,1,2400,544,32,32), CloneTileData(416,32,0,1,1,2368,576,32,32), CloneTileData(448,32,0,1,1,2400,576,32,32), CloneTileData(352,64,0,1,1,2144,256,32,32), CloneTileData(384,64,0,1,1,2144,256,32,32), CloneTileData(352,96,0,1,1,2144,256,32,32), ....
I notice that now more (GPU) memory is being used as before 718 instead of about 200 MB.
However, when rendering, there are some weird artifacts on the screen now (see screenshot below, it got even worse when I wanted to resize the window). This time both CPU & memory also seems to get a hit.

After a while, my Mac starts freezing until this notice:
WebGL: CONTEXT_LOST_WEBGL: loseContext: context lost
What is going on here?

Just quickly as I can't look at this properly right now but:

CloneTileData(352,96,0,1,1,2144,256,32,32)

Is that x position correct? Is means your tile map is 67 tiles long? 2144 / 32 = 67
Also is there any chance I can look at the code? Invite me as a collaborator or something if it's a private? You can kick me out again afterwards. If so then I'll take a look tonight and see if I can work out what's going on. But difficult to see anything obvious from the description.

x-position
yes and no :-)
yes: the x position 2144 is for cropping from the tileset (a large spritesheet).
no: the resulting tilemap is smaller.
The first coordinates represent the position of the tile in the tilemap. (which is smaller).
class CloneTileData(x: Int, y: Int, rotation: Radians, scaleX: Double, scaleY: Double, cropX: Int, cropY: Int, cropWidth: Int, cropHeight: Int) from the javadoc
The spritesheet graphic from the the tileset loaded is a .png 2,432 x 1,792 px and is 512,91 kB.
The tile map should look like this screenshot from tiled:

code
You can find the modified toGroup function on the indigo fork in my GitHub, here: toCloneGroup
After doing toCloneGroup I added the clone blanks and the group of groups with clones to the GameContext/StartUpdata which is then being included in the present function of the Scene as follows:
Outcome(SceneUpdateFragment.empty.addCloneBlanks(context.startUpData.cloneBlanks).addLayer(context.startUpData.group)))

That's some excellent looking work! I'm going to have to take a bit of time to read and absorb it though - hope it results in a PR. 😄

Intermediate update from my side on the other two points:
view culling
I managed to implement view culling using an RTree and noticed improvement already, but it still bad framerate on my mac, only a couple of frames more per second. And when I resize the viewport or zoom out (more objects are included for rendering), the effect quickly goes away.
asset loading
I mentioned slow loading at boot, it wasn't the loading of the assets, in fact that's only the single spritesheet PNG. But it is the generation of the tilemap and positioning all the tiles and their layers. I moved it from the boot time of the game to a custom subsystem. I have to investigate why - even though I wrapped the expensive calculation in a Future (that should now be supported in scalajs) - is still blocking the main event loop (update / present is also not being executed during the calculation).

Info on the context lost error, I've never seen it before:
https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/isContextLost#usage_notes

This is quite hard to debug from a distance, so I'm guessing a bit here and might be barking up the wrong tree.
Looking at the toCloneGroup function (I may be reading it wrong):
You appear to be making one enormous CloneTiles object (and/or CloneBatch). Maybe I forgot to mention it in the docs, but if you are doing that, it's a bad idea.
The problem is that even if your data never changes, all that data has to be transferred to the graphics card on every frame in one massive lump. That is slow both in terms of GC pressure and GPU communication time. GPUs are super fast, sending data to them isn't.
What you need to do instead is batch up your clones into chunks of, say, 400 (made up number, try it and tweak the values). Here I'm doing it with CloneTiles in the confetti scene for 20-30k instances (dynamic data):
https://github.com/PurpleKingdomGames/indigo/blob/master/indigo/sandbox/src/main/scala/com/example/sandbox/scenes/ConfettiScene.scala#L102
And here with a static list of 150k instances that are animated by never move - closer to your use case:
https://github.com/PurpleKingdomGames/indigo/blob/master/indigo/perf/src/main/scala/com/example/perf/PerfView.scala#L30
Please note that at the extremes, performance is going to vary machine by machine, and even depending on whether your laptop is plugged into a battery or power source and so on. So you want to be doing as little work as possible at all times.
I like the R-Tree idea, I tend to favour quad trees but that's just personal taste. 😄
So in an ideal scenario:

Your map is divided into regions
You only render the regions your camera is overlapping
Each region is each a good number for your chunks, or further sub-divided into chunks.
Finally, mark clone types that never change as static, to reduce re-processing - see below - I think this is ok with animated tiles, it certainly is the this link:

https://github.com/PurpleKingdomGames/indigo/blob/master/indigo/perf/src/main/scala/com/example/perf/PerfView.scala#L52

I mentioned slow loading at boot, it wasn't the loading of the assets, in fact that's only the single spritesheet PNG. But it is the generation of the tilemap and positioning all the tiles and their layers. I moved it from the boot time of the game to a custom subsystem. I have to investigate why - even though I wrapped the expensive calculation in a Future (that should now be supported in scalajs) - is still blocking the main event loop (update / present is also not being executed during the calculation).

Futures are supported in Scala.js, however, JavaScript is single threaded so evaluating your future inline without a callback (which I guess you must unless your sub system is doing something really funky?) will still block. I suspect that's what is happening. Many moons ago we did have a go at building an expensive computation thing. It did work for the original authors use case but we/I could never quite get it looking right to include in Indigo:
https://github.com/PurpleKingdomGames/indigo/pull/71

It terms of working out with the weird artefacts - I really can't say without seeing the code. If I had to debug it though, I would make sure my map was working for a much smaller grid, 1x1 2x2 3x4 15x10, and check the rendering appeared to be doing what you expect. Usually weirdness at scale is reproducible with a few elements. An exception to that can be if your work relies a lot on floating point values where precision is noticeably worse at larger values.

Performance profiling:
If you want to know where your frame time is being spent, you should make friends with Chrome's developer tools (also available in the electron window) and the profiling tab in particular. It will tell you how long each frame takes to render, which functions are taking all the time, and where exactly your code is snarling up.

Groups also have a performance penalty. It's hard to tell from the code, but a group per layer should be totally fine. If you have groups of groups and lots of them, you should try an alternate strategy to flatten out your scene.
The performance reduction comes because during rendering each group sets a base transform matrix on the graphics card and does a context change twice, and the matrix for the group is calculated on the CPU side. Done tens of times per frame (guess - I haven't measured it) it's not a problem. Hundreds or thousands of times though would be bad in amongst everything else.

Hi @laurensdv - I was wondering if you'd managed to make any progress?
If not, I wondered if perhaps you could provide a minimal game that demonstrates the problem and maybe I can work on it with you?

OK, this clarifies a lot.
clones
I will produce the clone groups in batches as you recommend, and keep you posted on the progress.- I'll check the two examples. I consider this is still the main point for this github-issue.
rendering artifacts
I am starting to think the rendering artifacts and the drop of the GL context look like it was overloading the graphics card memory and /or communication with the CPU/mem. Working on an old(er) mac, 1536 MB VRAM with Intel HD Graphics 4000. But nonetheless, it should not happen- with this kind of graphics level.
culling
I am happy with the current state of the culling. Setting the zoom / magnification (to 2), makes the number of items to be rendered significantly (4 times) smaller.
loading
For the loading, something is indeed going on with the future being immediately resolved.
On the other hand, if I manage to put some notice 'Loading' in time on the screen, before it freezes, nobody should notice... It only takes about 10 seconds.
object TileMapLoader extends SubSystem {
  type EventType = GlobalEvent
  type SubSystemModel = TileMapHolder
  var loadedMap: Option[TileMapContext] = Option.empty

  (...) in update function:
   buildTileMap(...).onComplete {
          case Success(context) => TileMapLoader.loadedMap = Option(context)
          case _ => Option(TileMapContext()) //empty if failure
        }

   (...)

I'll look into putting a demonstration version of the game available to you and mark the critical parts.
You can try the game in its current state (with culling - no clonetiles yet) here:
Indigo Tilemap loading demo
In the demo version there is a clock ticking following the system clock in seconds, when it stops, this is when the future is being resolved (blocking).

clones
I ran the perf example on my machine, it doesn't go beyond 14 fps and also consumes extremely much (V)RAM.
As mentioned, it is an old Mac, but even so,
at the moment the culling approach by its own gives much better performance (on the same zoom/magnification level).
I implemented the clone tiles and clone batches in groups of first max. 512 and then 256 each with a static batch key binding.
See my updated implementation here:
indigo/shared/formats/TiledMap.scala

Performance is still worse with his clone approach. Further,  it doesn't seem to make any notable difference to me 256 vs 512 as batch key size.
I took a closer look on the artifacts issue, by running on a more powerful machine (you can try yourself on the link in previous post with the technical demo - it is now using the clone tile / batch approach),
and my impression is that rather than that the rendering is 'failing', all tiles seems to be rendered in the wrong size / scale - zoom/magnification should be 2 but it looks way much more for some reason.
Which may also mean, that a way too big map/frame (in number of pixels) is being rendered every frame, the clone approach doesn't use any culling at the moment...

I've tried running the demo on my phone and my dev machine. Dev machine it's all Teal, phone it was Teal with a slight pattern (same colour at the bottom of your screenshot).
Looking at the performance tooling, it's barely touching the sides. 4-5ms spent out of 33 available per frame, so something else is going on.
I'll see if I can replicate the problem, if you have some dummy assets I could use that illustrate the issue that would be helpful.

Hi, I made as small as possible Indigo project (based on the perf example project as a model),
that replicates the issue clearly, you'l see it. You see colored blocks but you should see the tile map as shown in the screenshot before in this thread.
you can find it in my GitHub fork of indigo
Note: it relies on the 'toCloneGroup' function, and the function that I mentioned before in this issue.

Good evening @laurensdv,
Thank you very much for the code example. I've made a lot of progress!
I can render your level in all it's glory at 60fps, VRAM is at 350Mb, frame processing takes < 3ms on my machine.

I think there is something up with the CloneTiles though and the way the cropping works (it's why you're just seeing massive lumps of colour), and I don't understand it yet. I'm out of time for today but give me a little longer to dig and I'll work it out.
I also opened a ticket to look into the way the static keys work. I think I've looked at it before but I still think it should work a different way.

@laurensdv I forgot the ask, the demo you provided shows the rendering faults, but how is the performance?
If it's poor, could you try running it in electron / Chrome, letting it run briefly, recording the performance for 5 seconds or so and saving the results for me to look at?

So, I've got quite a few changes going on in my fork of that repo - but most of them are just tests or code re-organisation.
I think the only part that is actually wrong, is that this:
      val tileMapGraphic: Graphic[Material.Bitmap] =
        Graphic(tiledMap.width * tiledMap.tilewidth, tiledMap.height * tiledMap.tileheight, Material.Bitmap(assetName))

Should be
    val tileMapGraphic: Graphic[Material.Bitmap] =
      Graphic(tiledMap.tilewidth, tiledMap.tileheight, Material.Bitmap(assetName))

And I'd like to admit right now that the current CloneTiles examples don't do that, and that I had indeed misunderstood something about how they work! Apologies.
Can you try that change and see if it fixes your rendering defects?

Ok actually there is also a bug in CloneTiles, doesn't affect you right now (I don't think), but I've raised it here:
https://github.com/PurpleKingdomGames/indigo/issues/252

#252 has been fixed on the main repo. In practical terms, this means that the size given to the graphic (from above):
val tileMapGraphic: Graphic[Material.Bitmap] =
      Graphic(tiledMap.tilewidth, tiledMap.tileheight, Material.Bitmap(assetName))

...here tiledMap.tilewidth, tiledMap.tileheight, no longer matters, as the crop settings are respected correctly. In the below, four instances of the same CloneTiles entity render different parts of the image correctly to show the four different crates.

This update hasn't been released yet but you can get it @laurensdv, by updating your fork.

clones
The scaling is OK now!
The performance on my old Mac could be better (now at 17 fps instead of 2) but on a faster machine it seem OK.
Best results in Chrome. I will do a profile later.
First, there are still parts not showed correctly (see screenshot).
The first level is small enough as it doesn't need more than 256 blocks / tiles in a batch.
So the static hash key in the TiledMap / toCloneGroup function doesn't do much - is my assumption.
But with a much larger tilemap, I notice missing areas (or areas rendered in the wrong order?). Is it due to the batching?
Now, I updated my fork. and branch 'tiled_animations' with a bigger example that shows it.

clones
UPDATE:
when removing the static batch keys and the splitting in groups (of 256) in the TiledMap / toCloneGroup function, everything is rendered correctly.

I suspect System.currentTimeMillis().hashCode().toString is not unique enough for a static key. A millisecond is a long time. Will try and verify later unless you beat me to it.

I have a feeling that a big lump of the slowness you're seeing may be because you're using Sprites. Sprites are interactive, and not lightweight animation entities, and a believe a long time is being spent on event collection. Need to dig further incase I'm barking up the wrong tree.
But if I'm right, then we might need to do something like this for you(/everyone!): https://github.com/PurpleKingdomGames/indigo/issues/125

So if I do this, static keys are working, it renders correctly at about 55 fps (up from 17 on my machine!).
Mostly what I've done it turned off the clone batches, but there are other tweaks like removing a layer of Group.
diff --git a/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala b/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala
index f4767baa..8fe3ed3b 100644
--- a/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala
+++ b/indigo/indigo/src/main/scala/indigo/shared/formats/TiledMap.scala
@@ -225,7 +225,7 @@ object TiledMap {

         val tileMapCloneBlanks: List[CloneBlank] = List(CloneBlank(CloneId("graphic"), tileMapGraphic))

-        val layers = tiledMap.layers.filter(_.`type` == "tilelayer").map { layer =>
+        val layers = tiledMap.layers.filter(_.`type` == "tilelayer").zipWithIndex.map { case (layer, layerIndex) =>
             val tilesInUse: Map[Int, String] =
                 layer.data.toSet.foldLeft(Map.empty[Int, String]) { (tiles, i) =>
                     tiles ++ Map(
@@ -274,16 +274,18 @@ object TiledMap {
                                     case _ => Nil
                                 }
                                 .getOrElse(Nil)
-                            }.toArray).withStaticBatchKey(BindingKey("batch_" +  System.currentTimeMillis().hashCode().toString))
-                }).toList
+                            }.toArray)
+                }).toList.zipWithIndex.map { case (cb, i) =>
+                  cb.withStaticBatchKey(BindingKey("batch_" + layerIndex.toString + i.toString))
+                }

-            val cloneTiles: List[SceneNode] = (
+            val cloneTiles: List[CloneTiles] = (
                 for {
                     layerDataChunk <- layer.data.zipWithIndex.grouped(256)
                 } yield {
                     CloneTiles(CloneId("graphic"),
-                    layerDataChunk.flatMap {
-                        case (tileIndex, positionIndex) =>
+                    layerDataChunk.zipWithIndex.flatMap {
+                        case ((tileIndex, positionIndex), i) =>
                             if (tileIndex == 0) Nil
                             else
                                 tilesInUse
@@ -296,14 +298,17 @@ object TiledMap {
                                         List(CloneTileData(pos.x, pos.y, Radians.zero, 1, 1, cropPos.x, cropPos.y, tlSize.x, tlSize.y))
                                     }
                                     case _ => Nil
-                                }.getOrElse(Nil)}.toArray).withStaticBatchKey(BindingKey("tiles_" +  System.currentTimeMillis().hashCode().toString))
-                }).toList
+                                }.getOrElse(Nil)
+                        }.toArray)
+                }).toList.zipWithIndex.map { case (ct, i) =>
+                  ct.withStaticBatchKey(BindingKey("tiles_" + layerIndex.toString + i.toString))
+                }

-            val clones = cloneBatches ++ cloneTiles
-            Group(clones)
-            }
+            // cloneBatches ++ cloneTiles
+            cloneTiles
+        }

-        (tileMapCloneBlanks ++ animationCloneBlanks, Group(layers))
+        (tileMapCloneBlanks ++ animationCloneBlanks, Group(layers.flatten))
         }

   private def toGroup(tiledMap: TiledMap, assetName: AssetName): Option[Group] =

I would like to suggest a couple of things to make your life better, just suggestions:

Install Scalafmt and turn on "Format on save" in your editor - after a brief period of adjustment I promise you'll never go back. Use of a formatter with consistent project wide settings makes it easier for everyone working on the code, because it all looks the same. It also reduces the diffs during code reviews. Suggest you format after you've committed your work though and commit the formatting as a separate change.
Break out the code into separate functions. The work I did in the original Tiled implementation is not a shining example of good code, but if you take a little time to break up the code it'll be easier to reason about and to modify. You could go further and break it up into a few well named files if that helps. This is a complicated thing you're doing, compartmentalise.
Once you've broken up the code, consider writing a few tests - even pretty simple happy path ones for some of the sub functions. They will provide you confidence around your solution, e.g. are all my static keys unique? There is a small TiledMapTests suite already you could extend.

Anyway, if after making the change above, I then bump the tiled batch size from 256 to 512, I get a comfortable 60 fps, frame processing takes about 10ms.

Moving on to the CloneBatchs.
If I turn off CloneTiles and turn on CloneBatchs instead (just return them from the function alone), I get this in the console:

This implies that the clone batch implementation isn't doing the right thing somehow, on the basis that we don't see that error in the CloneTiles version.

Hello, the diff works for the CloneTiles on my side as well.
And, I confirm that for the CloneBatch containing the animated tiles (Sprite) when including them again, even when turning off the static hash keys. In both cases there is an enormous performance hit and still they are not being shown.
For the formatting: I'll check to enable it also in my editor.
For the splitting code & testing: let's say this project is like getting (a bit) bigger then I anticipated as last year when I started,
then I moved on to work on the tile map generator (outside of Indigo), now since last month coming back to integrate it all.

Once you've broken up the code, consider writing a few tests - even pretty simple happy path ones for some of the sub functions. They will provide you confidence around your solution, e.g. are all my static keys unique? There is a small TiledMapTests suite already you could extend.

See also #67

Apologies I meant to say: I'm now assuming that my performance concerns due to Groups and Sprites are probably a red herring. WebGL errors are often the source of massive slowness so we should work out why that's happening first before wondering if it's anything else.

The use of a Group of Groups makes sense 'structurally' as I consider each TileMap layer in the Tilemap JSON, a group, which then is grouped again for the TileMap as a whole. It allows to process the Tilemap layers further (for example: for culling) before they are converted to a (single) Indigo Layer (with only a single group).
Using a nested group like this, shouldn't be such a performance hit and / or the fact that it is not performing that well (and with the sprites enabled) might be due something else underlying - that I understand from your comment.

Hello @laurensdv - trying to pull together the people interesting in improving Tiled support:
https://github.com/PurpleKingdomGames/indigo/discussions/270


Firestore QuerySnapshot.documentChanges(includeMetadataChanges: false) always returns DocumentChange with only metadata changes

Description
In Firestore's QuerySnapshot#documentChanges API at https://github.com/firebase/firebase-ios-sdk/blob/03ce722d0e511a9f606ea95077d416aaaea4bdb6/Firestore/Source/Public/FirebaseFirestore/FIRQuerySnapshot.h#L68, it allows us to specify whether to include document changes with only metadata changes. However, from my observation, no matter this is set to true or false, it always returns document change even if there is only metadata change. This make it impossible for us to know which document change actually comes from the server.
I've tested the same piece of code in both Android and iOS. In Android, this behaviour works as expected, when I do querySnapshot.getDocumentChanges(MetadataChanges.EXCLUDE), it will return 0 documentChange if there is only metadata changed.
Reproducing the issue
For both Android and iOS, I'm using the exact same setup, listening to same collection with the same query, with includeMetadataChanges set to true.
Here's my code in iOS that processes the received QuerySnapshot and its log output when I write a new document locally:

Here's my code in Android that processes the received QuerySnapshot and its log output when I write a new document locally:

Firebase SDK Version
10.23.1
Xcode Version
15.2
Installation Method
Swift Package Manager
Firebase Product(s)
Firestore
Targeted Platforms
iOS
Relevant Log Output
No response
If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

Replace this line with the contents of your Package.resolved.

If using CocoaPods, the project's Podfile.lock

Expand Podfile.lock snippet

Replace this line with the contents of your Podfile.lock!

It's difficult for me to determine what is happening in your screenshots because I can't see the full application or document changes. Can you provide a minimal reproducible example?
However, I tested the following and did not find any issues.

coll.addSnapshotListener() defaults to not raising a snapshot for metadata-only changes.
coll.addSnapshotListener(includeMetadataChanges: true) causes snapshots to be raised for metadata-only as well as document changes.
querySnap.documentChanges(includeMetadataChanges: true) returns an array of documents that have changed since the last snapshot, and includes documents where only the metadata has changed.
querySnap.documentChanges(includeMetadataChanges: false) returns an array of documents that have changed since the last snapshot, and excludes documents where only the metadata has changed.

Creating and sharing a minimal reproducible example can help us more easily diagnose the behavior you are experiencing.

Firestore has microsecond precision, so it seems expected that the backend truncates the nanosecond-precision value you provided 161365032 to a microsecond-precision value 161365000.  The SDK, storing nanoseconds, detects this difference and raises a snapshot.
What's not clear to me is how you are setting this nanosecond precision value in the SDK? I'm seeing that the SDK also truncates nano-second precision values to micro-seconds when I attempt to write a date value.

When I initially tested I was using Timestamp(), which was giving microsecond precision values, and therefore I was not experiencing this snapshot behavior. When I test with Date.now, I see the reported behavior.
To get a different behavior from Firestore, it might be worth generating a date or timestamp with microsecond behavior. Also, I would consider using FieldValue.serverTimestamp() to generate your timestamp from the server and not based on the client device's clock.
It's unlikely we will modify the current SDK behavior to change the snapshot behavior you are seeing. Such changes could have implications for other current and future users.
If you have a path forward, then I will close this issue.


chore(SetTheory/Cardinal/Basic): generalize bddAbove_range

In truth we could simply replace all instances of bddAbove_range with bddAbove_of_small, but I've kept the lemma anyways for discoverability.

Eh, fine. I'll un-deprecate it.

Thanks! :tada:
bors merge


Scatter not working v R2015a

Scatter plots are not working in version R2015a.

Can reproduce.
function scatter_test()
%SCATTER_TEST Demonstrates the need for a small patch.

    badMarkerTypes = {'+', '*', 'x', 's', 'd', 'p', 'h', '^', '<', '>'};
    testResults = cellfun(@try_to_convert_scatterplot, badMarkerTypes);
    fprintf(1, '%d out of %d marker types succeeded.\n', sum(testResults), numel(testResults));
end

function success = try_to_convert_scatterplot(markerType)
    testOutputFilename = 'scatter_test.svg';
    h = figure;
    cleanupFigure = onCleanup(@()close(h));
    scatter(1:10, 1:10, markerType);
    cleanupFile = onCleanup(@()close_and_clean_file(testOutputFilename));
    try
        plot2svg(testOutputFilename, h);
    catch exc
        warning('Marker type "%s" failed to convert. Message is "%s"', markerType, exc.message);
        success = 0;
        return;
    end

    success = 1;
end

function close_and_clean_file(filename)
    fclose all;
    delete(filename);
end

I have a fix for this, hang on while I pullrequestify it.

Amazing :-). Will check tomorrow.
Best regards,
Carl Wouters
Oil-Free Engineering - Calculations Competence Coach
On 14 Oct 2015 18:49, Ben Ellis<EMAIL_ADDRESS>wrote:
I have a fix for this, hang on while I pullrequestify it.

Reply to this email directly or view it on GitHubhttps://github.com/jschwizer99/plot2svg/issues/6#issuecomment-148113858.

Hey Ben,
Any luck on getting it through? :-)

I'm not much experienced in the etiquette, but I don't particularly want to repay Juerg's hard work making plot2svg by nagging him...
For the time being, the change is committed to the master branch of my fork ( https://github.com/ble/plot2svg ) and you can get it there.
You can see the diff of the commit here: https://github.com/ble/plot2svg/commit/b6987af8f2adf92a3d3cd7816a5fcfb6e473c982

Since Matlab 2015a it has a build-in svg engine. Just type

saveas(gcf,filename,'svg')

Hi,
I've been on this errand with the newer versions of Inkscape and Matlab as well. kromuchi solution unfortunately doesn't work even for slightly sophisticated plots such as mesh surface and 3dplots (the text is embedded on the plot).

Hello.
I have the same issue with R2016a:
Warning: Unhandled child type: scatter

Fixed in my fork: https://github.com/kupiqu/plot2svg

just wanted to let you know about the first release of fig2svg which fixes this issue:
https://github.com/kupiqu/fig2svg


bug

请问修复好bcfn的bug为什么精度没提升。
修复之前仅使用x_1相当于没有特征融合，更改之后x_1和x_2进行特征融合也没能提升性能！

Yes, we have also observed that the performance of PETdet did not show significant improvement after fixing the bug. I think this indicates that the key to enhancing performance with BCFN lies in the bilinear operation and the decoupling of features between the two stages. When designing BCFN, I assumed that multi-scale features would enhance performance. However, it suggests that since BCFN takes the feature maps from the FPN as input, which have already undergone fusion, the feature fusion operation in BCFN is indeed of limited significance.


Ubuntu: cannot load opencv library which is required by simplecv

Hello. I am running a parallels Ubuntu 12.04 on my Mavericks Mac Air.
So I have followed the commands here:
sudo apt-get install ipython python-opencv python-scipy python-numpy python-  pygame python-setuptools python-pip
sudo pip install https://github.com/sightmachine/SimpleCV/zipball/develop

However, when I go to run simplecv either by import SimpleCV or shell command simplecv I see that error message.
I notice when I import Opencv:
Only import cv2 works (but import cv2.cv as cv does not). import cv does not work.
Any help would be awesome!

Is it still unsolved?

Well??? Five years???


Simulation models incorrect rotation

When I run the simulation, the various furniture have incorrect rotation leading to collisions when the robot follows the nav graph.

Hi @rcywongaa
Does your traffic_editor and rmf_demos repository contain these fixes?
https://github.com/osrf/traffic_editor/pull/159
https://github.com/osrf/rmf_demos/pull/90

Thanks, that fixed it!


public_path error in ImageMeta.php

Good day! Can you tell me what the problem might be? I prepare the seo data and send it to the template.
		$title = mb_strimwidth($this->name, 0, 70, "");
        $description = mb_strimwidth($this->description, 0, 500, ".");
        $imageUrl = $this->hasMedia('products') ? $this->getFirstMediaUrl('products', 'products-image') : null;
        return new SEOData(
            title: $title,
            description: $description,
            image: $imageUrl,
        );

And, as expected, I successfully receive them on the page. When checking the $imageUrl variable before sending it to the template, I also get the correct path.
        <meta name="image" content="https://test.ts/media/images/products/9244/5911/conversion/products-image-prothermix-skat-9-ke.webp">
But laravel logs an error: We get this error when checking in the file ImageMeta.php this package.
ERROR: Path /home/testy/public_html/public/https://test.ts/media/images/products/9244/5911/conversion/products-image-prothermix-skat-9-ke.webp is not a file. {"exception":"[object] (Exception(code: 0): Path /home/testy/public_html/public/https://test.ts/media/images/products/9244/5911/conversion/products-image-prothermix-skat-9-ke.webp is not a file. at /home/testy/public_html/vendor/ralphjsmit/laravel-seo/src/Support/ImageMeta.php:18)
//vendor/ralphjsmit/laravel-seo/src/Support/ImageMeta.php
public function __construct(string $path)
    {
        $publicPath = public_path($path);

        if (! is_file($publicPath)) {
            report(new Exception("Path {$publicPath} is not a file."));

            return;
        }

        [$width, $height] = getimagesize($publicPath);

        $this->width = $width;
        $this->height = $height;
    }

Tell me why is the check not going through in the file ImageMeta.php?

Hello Ramis, thank you for your message! I think the reason was that the ImageMeta works fine if you pass a path, and then it tries to find the image size from that. If you pass a url, it would return and report it as invalid. In your, the report was not was not really necessary if $path is a valid URL, so I have just made a fix in 22bb096 that will return early if the path is a URL. That has been released in 1.6.4. Let me know if that fixes it!


Matte L1 Loss Maybe have some wrong ？

https://github.com/ZHKKKe/MODNet/blob/44667be551eff6427e774b60e3b251f90b9dac61/src/trainer.py#L162
calculate matte boundary (or detail) Loss value,  we should using pred_boundary_matte and gt_detail (Not gt_matte) , right?
https://github.com/ZHKKKe/MODNet/blob/44667be551eff6427e774b60e3b251f90b9dac61/src/trainer.py#L156
https://github.com/ZHKKKe/MODNet/blob/44667be551eff6427e774b60e3b251f90b9dac61/src/trainer.py#L161

matte_compositional_loss seem to has same problem
https://github.com/ZHKKKe/MODNet/blob/44667be551eff6427e774b60e3b251f90b9dac61/src/trainer.py#L163-L164
I think it should be
 matte_compositional_loss = F.l1_loss(image * pred_matte, image * gt_matte) \
          + 4.0 * F.l1_loss(image * pred_boundary_matte, image * gt_detail)

right?

Hi, thanks for your attentions.
Nope. In our code, gt_detail is the same as gt_matte. The foreground and the background regions in pred_boundary_matte are the same as trimap, i.e., the same as gt_matte. Therefore, the loss term  F.l1_loss(image * pred_boundary_matte, image * gt_detail) is only be calculated on the boundary regions.

Thanks for your reply!


Unable to generate OTP

After restoring my iPhone from an iCloud backup I can no longer select any of the three accounts I have in FreeOTP. I can see the accounts but nothing happens when click on them.

Any help to get me back into my accounts would be appreciated.

This is expected behavior of biometric locked tokens due to Apple keychain security, we added a warning when you add a new 'Locked' token to make this evident.

So there is no way to unlock?

Seems that there is not.
Unfortunately the warning was added after I created the tokens :(
Time to send my drivers license to Amazon (woohoo!)


Question regarding a base model for query models

I was wondering if it can be done that I can extend query models from a base model that does basic functions. For example I have this model:
export class FooModel {
    static async findById(id: number) {
        const result = await db
            .selectFrom('foo')
            .selectAll()
            .where('id', '=', id)
            .execute()
        return result[0] as Foo
    }

    static async delete(id: number) {
        await db.deleteFrom("foo").where("id", "=", id).executeTakeFirstOrThrow();
    }

    static async findAll() {
        const result = await db.selectFrom("foo").selectAll().execute();
        return result as Foo[];
    }

    static async update(id: number, name: string) {
        await db
            .updateTable("foo")
            .set({ name })
            .where("id", "=", id)
            .execute();
    }

    static async create(values: object) {
        let foo = FooSchema.parse(values)
        await db
            .insertInto("foo")
            .values(foo)
            .returningAll()
            .executeTakeFirstOrThrow();
    }
}

I'm wondering whether its possible to create a BaseModel and extend the FooModel like:
export class FooModel extends BaseModel {
    constructor(){
        super("foo")
  }
}

And the BaseModel can do something like
export class BaseModel {
    protected readonly tableName: string;

    constructor(tablename: string){
        this.tableName = tablename
    }
    static async findById(id: number) {
      const result = await db
        .selectFrom(this.tableName)
        .where('id', '=', id)
        .execute()
      return result[0]
    }

    static async delete(id: number) {
      await db.deleteFrom(this.tableName).where('id', '=', id).execute()
    }
  }

Issue of course in this implementation would be selectFrom is expecting a TableExpression and not a string. Any suggestions/better approaches?

Hey 👋
This might work if BaseModel has a TB generic (e.g. 'person') and BaseModel.tableName has a TB type, and all tables have a numeric 'id' column.

Hey thanks for the reply. I think I got what you are saying, please correct me if I am wrong. Given that all tables will have a numeric id column, the BaseModel Class would look like
interface Table {
  name: string;
  idColumn: number;
}

export class BaseModel<TB extends Table> {
  protected readonly tableName: string;

  constructor(table: TB) {
    this.tableName = table.name;
  }

  async findById(id: number) {
    const result = await db
      .selectFrom(this.tableName)
      .where('id', '=', id)
      .execute();
    return result[0];
  }

}

however once I hover over this.tableName I get
No overload matches this call.
  Overload 1 of 2, '(from: TableExpression<Models, "foo">[]): SelectQueryBuilder<From<Models, TableExpression<Models, "foo">>, any, {}>', gave the following error.
    Argument of type 'string' is not assignable to parameter of type 'TableExpression<Models, "foo">[]'.
  Overload 2 of 2, '(from: TableExpression<Models, "foo">): SelectQueryBuilder<From<Models, TableExpression<Models, "foo">>, any, {}>', gave the following error.
    Argument of type 'string' is not assignable to parameter of type 'TableExpression<Models, "foo">'

Sorry dont know why this closed

Reopening since it closed by mistake

Something like this:
export class PersonModel extends BaseModel<'person'> {
  constructor() {
    super('person')
  }
}

export class PetModel extends BaseModel<'pet'> {
  constructor() {
    super('pet')
  }
}

export class BaseModel<TB extends TablesWithID<Database>> {
  protected readonly tableName: string;

  constructor(table: TB) {
    this.tableName = table.name;
  }

  async findById(id: number) {
    // ...
  }
}

type TablesWithID<DB> = { [TB in keyof DB & string]: 'id' extends keyof DB[TB] ? TB : never }[keyof DB & string]

Thank you that worked!


[Bug] no oregen at all

Minecraft Version

1.16.5

Forge Version

36.2.2

Mod Version

2.5.1

Modpack & Version

Crashed on Gyrnolia (CurseForge) 0.7.5
(you have to update oretweaker in this version of the pack to get the error occuring)

Describe the bug

no specified ores generate at all. the world consists of stone and features by other generators like quark

How to reproduce

this error occurs to me when i update OreTweaker from 2.3.1 to 2.5.1 in my modpack "Crashed on Gyrnolia" (see above). this even happens if OreTweaker's config files are completely deleted or reinstalled.
furthermore all entries in every single file of the data folder gets multiplied. i have this paste which shows before and this paste which shows after a world generation. this continues each time a world generates

Screenshots / Images

you can probably imagine a world containing only minecraft:stone :P

The auto updater to the new v2 data structure does not always work in all cases. In rare cases like this you have to delete the entire config/oretweaker folder and restart the game. You can backup your old oretweaker.json by moving the config/oretweaker/backup folder outside the config/oretweaker before deleting it.
The new v2 data structure will then generate properly and you can tweak the files again. Not to much to fix here cause the auto updater was made to help convert from the v1 data structure to the new v2 data structure. Hard to say why it breaks in this case and hard to replicate so doubt it will be fixed.

i'm sorry but i have to insist on this..
like i said in the original:

this even happens if OreTweaker's config files are completely deleted or reinstalled.

just tried it again:
i. modpack on old version: i delete oretweaker config folder
ii. update oretweaker
iii. launch modpack, generate world
and its stone + vinteum from Mana and Artifice + Quark Features
the generated files in data look fine but they don't work!
example of iron_ore.json which autogenerated:
{
  "oreConfig": [
    {
      "ore": "minecraft:iron_ore",
      "filler": "minecraft:stone",
      "minY": 1,
      "maxY": 64,
      "maxVeinSize": 8,
      "spawnRate": 20.0,
      "biomeBlacklist": [],
      "biomeWhitelist": []
    }
  ]
}

Just tested it with your pack and found a way to fix it.

Delete the old OreTweaker-1.16.5-2.3.1.jar
Delete the config folder config/oretweaker
Install the new OreTweaker-1.16.5-2.5.1.jar
Run Minecraft and create a new world.

Works just fine.


[AutoRelease] t2-communication-2022-06-07-28754(Do not merge)

https://github.com/Azure/sdk-release-request/issues/2825
Live test success
https://dev.azure.com/azure-sdk/internal/_build?definitionId=2047
BuildTargetingString
azure-mgmt-communication
Skip.CreateApiReview
true

issue link:https://github.com/Azure/sdk-release-request/issues/2825


Improve multi-process memory management

The current allocator in the multi-process environment (--procs=P) is a hack. It claims a large piece of anonymous pages using mmap at pre-fork time. Memory allocations are handled by giving out pointers continuously from this region (HREdefaultRegion(HREglobal()) is automatically set in the process environment).
For lack of administration freeing or reallocating memory is not possible.
To limit fragmentation two sets of functions were designed:
RTmalloc(...) - for local (non-scaling) allocation by the OS
HREmalloc(region, ...) - allocating from a designate region (Warning: it may fall back on RTmalloc)
RTswitchAlloc (true/false) is used to switch the behavior of RTmalloc to HREmalloc(HREdefaultRegion(HREglobal()), ...) transparently. This enables the use of allocator oblivious data structures.
TODO:
1 - integrate a true allocator that works with inter-process memory.
a viable option seems the simple SSmalloc, it needs to be adapted to used the preallocated region though
2 - consolidate HREmalloc and RTmalloc (See #698)
with good allocator, HREmalloc with default region can be used as default

Also:
Traces left of grey:
GB* functions in pins.[ch]
Also rename functions such as Warning to e.g. Report. Warning can print all types of messages: info, error, debug, etc.
Create naming scheme for similar functions, c.f.: Warning, Debug, DEBUG, Abort, etc
Add multiple info levels -v and -vv and -vvv (the last one should also provide localized tourist information)
consolidate HRE* and RT* ?

@alaarman what is the status of this issue? Should we still implement this?

Either this, or drop support for multi-process (find a solution for MCRl2's non-thread-safe aterm library).


Cannot Unset Restrict Access to IPs after Setting It

When an API user is first created in the web interface, their allowed_ips value is omitted entirely, granting them access from any IP.  However, after setting a value for this field and then later clearing it, the value in mongodb is set to "allowed_ips" : [ "" ], which means they are granted access from no IP address. This is inconsistent with the default behaviour for not specifying a value for this field.

Ah, yeah, this is a bug with the allowed IPs and referrers. It's fixed in master and will be part of the v0.7 package release. Sorry for the issue!
I thought I was going to get the v0.7 packages released last week, but I'm aiming for the end of this week instead. If you don't want to wait for the fix, you can deploy master on top of your existing install, but waiting for the packages may be easier.
And just for reference, here's the details from when this issue originally got fixed.
https://github.com/18F/api.data.gov/issues/155
https://github.com/NREL/api-umbrella-web/commit/92f5ed7214310335eb488e301ccfc7ce54c34941

Thanks. I did a search before raising the bug but didn't notice that one since the title was referring to the referrer. Is there a way to build a custom package from the source instead? This is one thing I haven't been able to figure out how to do yet.

This has been released as part of the new v0.7 packages. Thank you very much for your contributions!

Thanks! Looking forward to giving the new version a try.


useCompiler return type missing ?

It seems useCompiler returns Promise<any> when used like so:
import { useCompiler } from "#vue-email";

export default defineEventHandler(async () => {
  const template = await useCompiler("TestEmail.vue");
  return template.html;
});

Am I missing something ?

we just merged a new project rewrite, please do check the docs and the updated logic https://vuemail.net/


Steps for installation?

Hi Vdaubry, I am running youtuble-dl from linux machine. I was able to clone using git. However, I am a little confused with the rest of the installation. Could you provide some insight? I am running this on Ubuntu Vivid. Thanks.

Hi marry, did you manage to run the bundle install command ?

Hi @vdaubry yes I was able to get bundle install. However it stalls when i start youtube-multiple-dl and nothing downloads. How did you set the path environments for this? I am running this on my Ubuntu 15.04 box

In order to start a download make sure that you add some URL to download :
youtube-multiple-dl -add https://www.youtube.com/watch?v=Y7RnH6bnGHc
You can check that you have some URL in your queue with the list option :
youtube-multiple-dl --list
When you have some URL in your queue you can can some workers, 3 workers will download 3 video simultaneously :
youtube-multiple-dl --start --worker_number 3

Thanks!


0.7.4-RC1 Changelog intro paragraph - fix copy/paste error

Signed-off-by: Stephen Curran<EMAIL_ADDRESS>

And again -- screwed up a text update.  Twice.  Sigh.
@ianco @shaangill025


nice job！how can I get the “only vocals”？

nice job！
how can I get the “only vocals”？
I try to change the model_hash.
but it doesn't work.
Thank you very muck indeed

The problem was resloved.
Last day,I used the model:UVR-MDX-NET-Inst_HQ_3.onnx.
Then,I download the new model:UVR_MDXNET_Main.onnx
I still change the model hash.It works.


Template literal arrays should not be marked.

The idea is that template literal objects are frozen arrays, so their properties and prototype could not be changed. Their properties are strings (no need marking) and their prototype is a built-in (marked elsewhere). Therfore marking these objects is unnecessary. This is compatible with a future improvement called realms, since the realm of the function and its own template literal is the same.
There are two ways to handle unmarking these objects:

The current choice: prototype refing is moved to the end of ecma_gc_mark. Slight binary increase and each array object has an extra check.
Other option: check these objects when root objects are marked (their refcount is always > 0). Slight binary increase and each root object has an extra check.

I am ok with both, let me know your opinion.
Make array object big endian compatible: I noticed that the current code is wrong on big endian system, since the ecma_property_t is one byte long, and mapped to the highest byte instead of the lowest in big endian system.

#4347 should land first

Please rebase.

Rebased.

I'm fine with the first approach!


Expand on Request/Response manipulation in Docs

The docs in question (referenced below) provide too little detail into request/response operations, I should say manipulation with regards to what was previously available in asp.net with Filters/Actions.
I think it's pretty safe to assume most people come to the page referenced below trying to find detailed info on request/response manipulation, in particular modifying the response, be it either modifying the views or API responses (none of which the Docs cover).
Also, the three code samples that are included cover the I/O operations with Streams and then expand on to cover Pipelines. Why is this even here? Streams and pipelines are well documented in the IO section of the docs and have little relevance to the middleware and request/response manipulation/rewriting. So, the example topic has been chosen poorly.
Unfortunately, there's this trend of declining quality of the docs all over the ASP.NET Docs, I could post more examples of this as I have them open at the moment. I still remember the good old days of MSDN docs that were just pure joy and brilliant, sadly that's not the case anymore.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 49b15fd3-39b5-3ad9-8c4e-b8160360dee9
Version Independent ID: c33981af-93ce-d4be-a265-5d4696666c7a
Content: Request and Response operations in ASP.NET Core
Content Source: aspnetcore/fundamentals/middleware/request-response.md
Product: aspnet-core
Technology: aspnetcore-fundamentals
GitHub Login: @jkotalik
Microsoft Alias: jukotali

Duplicate of #20936


Not able to connect with firebase functions

I have deployed code as per gatsby guide lines. I am calling /auth end point for firebase auth() function. It is working fine on local with proxy(developMiddleware) but not able to connect on production. Please help.

gatsby-config.js file code ---
developMiddleware: app => {
app.use(
"/auth",
proxy({
target: "http://localhost:5001",
pathRewrite: {
"/auth": /${FIREBASE_PROJECT_ID}/${FIREBASE_APP_ZONE || 'us-central1'}/auth,
},
})
)
}


v5.2.4

[x] I have followed (at least) the PR section of the contributing guide.

Missed the root package version, update in this PR #30204


fix connecting parameter

verify_cert to verify_certs

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?


[Test] Matched ordering in ptrauth-partial-apply.sil.

In f1d263b2ba24e4facc9975ecd2bc6bbead356d98, partial apply forwarder emission was tweaked by moving the ptrauth emission for dynamic function pointers prior to the bitcast to enable the emitted ptrauth info to be used in extracting values from the async function pointer.  Here, the ptrauth-partial-apply.sil test is updated to reflect that change.

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test and merge

@swift-ci please smoke test linux platform


Support deletion of un-generated files

Several architectural choices were made particularly with a mind towards being able to delete files which were previously generated but are not longer generated even outside watch mode. The work just needs to be finished out.

Fixed in #26


Fix a number of misspellings

Note that there were a couple of _tcsat that I changed to _tcscat (the correct function name).

Thank you so much! Lots of fixes! -John


⚠ Samsung has degraded performance

In 0d3ca88, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2637 ms

Resolved: Samsung performance has improved in 317abcf.


Add in a row height property to customize row height of cells

Thanks for the PR, @jhaddadnia!
Ah, this good ol' property again. Any reason we don't want take the approach outlined in https://github.com/venmo/Static/pull/60? height has been added and removed several times from the API now.

@eliperkins in this use case the tableView/Controller itself has been abstracted away from where the rows are being declared. So I'd like to be able to specify a height on a specific row/rows rather than subclassing the tableViewController and create a new cell class for every height requirement.
In regards to the row being agnostic of its presentation, sending a height seems analogous to sending in a title string. We don't send it the label or view itself, just a config string/float which is used to build the cell later.

I've been convinced 🙏
...but no promises that I won't make the same pr in a couple months 😉

Haha awesome.
If there's anything we can make more clear in the docs or examples, I'd love to get that in there to convince Future Us™ too!


Issue with sticky sessions

Hi All,
Basically we are running Java application using Tomcat Docker image which is deployed on kubernetes. Everything works fine when there is only 1 replica. When we try to run 2 replicas for instance, at first there was a problem where user was not able to log in the application, this was kind of fixed by using kubernetes nginx-ingress sticky sessions using cookie as affinity. It basically forced user to access the same tomcat POD all the time and user was able to log in. But this is not ideal as whenever new POD is deployed, user needs to log in again. Thats when we started to look around for some session management and came upon Redisson for Tomcat and it sounded exactly what we needed so I decided to try it out.
So first I added this to context.xml file
<Manager className="org.redisson.tomcat.RedissonSessionManager"
        configPath="${catalina.base}/redisson.conf"
        readMode="REDIS" updateMode="DEFAULT" broadcastSessionEvents="false"
        keyPrefix=""/>

and redisson.conf looks like this (redis single master node deployed on kubernetes as well)
---
singleServerConfig:
  idleConnectionTimeout: 10000
  connectTimeout: 10000
  timeout: 3000
  retryAttempts: 3
  retryInterval: 1500
  password: null
  subscriptionsPerConnection: 5
  clientName: null
  address: "redis://redis-master:6379"
  subscriptionConnectionMinimumIdleSize: 1
  subscriptionConnectionPoolSize: 50
  connectionMinimumIdleSize: 24
  connectionPoolSize: 64
  database: 0
  dnsMonitoringInterval: 5000
threads: 16
nettyThreads: 32
codec: !<org.redisson.codec.MarshallingCodec> {}
transportMode: "NIO"

Added also both redisson-all-3.17.3.jar and redisson-tomcat-9-3.17.3.jar files to lib folder.
Tomcat version is 9.0.20
When first accessing my application, in catalina log file I saw message
INFO [http-nio-8080-exec-10] org.redisson.tomcat.RedissonSessionManager.findSession Session 43903E22A6119519F53C530A5EA2C3B4 can't be found

and I was never able to log in my app, even using single POD. Although from redis-cli I saw that keys like "redisson:tomcat_session:5555BC7641D738082FD1087373A8816B" are being stored
Then I changed readMode="MEMORY" in redisson.conf file and I was able to log in my app, as for my surprise even after deleting PODs and/or scaling to 2 or 3 PODs, I was still logged in my app after refreshing the browser, awesome, exactly what I wanted.
The issue right now is that whenever there are 2 or more PODs (for example POD-1 and POD-2) and I try to log in my app, from the POD logs I can see that when first accessing the login screen, it goes through POD-1 and when trying to log in, request goes through POD-2, thus never logging user in. Then again when logging in on POD-2, request goes through POD-1. After several tries, once the request goes through the same POD, I am able to log in and use the application even if PODs are being recreated/deleted afterwards, I am still logged in.
So the question - is it somehow possible for the user on the first log in stay at the same tomcat POD even after login request is sent? Similarly as with nginx-ingress sticky sessions when requests are sent to the same POD.
Thanks in advance!

So looks like when setting updateMode="AFTER_REQUEST" everything is working just fine. User is able to login in whether its one POD or multiple PODs and when logged in, after redeploying PODs, user stays in as expected, which is awesome. Not entirely sure if using AFTER_REQUEST is fine though.

Actually looks like we are still having issues with this because looks like with redisson K8S sticky sessions are not working like expected. Whenever user is logging in, the tomcat serving the request can choose random backend thus breaking some functionality.
For example this is how we would like it to work with 3 replicas
graph TD
    B[CLIENT]
    B --> C[K8S INGRESS]
    C --> D[TOMCAT A REDISSON]
    C --> E[TOMCAT B REDISSON]
    C --> F[TOMCAT C REDISSON]
    D --> G[JAVA APP A]
    E --> H[JAVA APP B]
    F --> J[JAVA APP C]

So whenever TOMCAT A is first accessed, it automatically sends all the requests to JAVA APP A when logging in. This works as expected without Redisson.
With Redisson, when user is logging in, it can suddenly change the JAVA backend (or maybe its even different Tomcat isntance serving the request).
For example, using the same 3 replicas, user accesses login screen first on TOMCAT A and after logging in, the requests could start to serve on JAVA APP B (or TOMCAT B) so we are getting some functionality errors related with missing information and null pointer exceptions because user was not logged in through the same Tomcat.
graph TD
    B[CLIENT]
    B --> C[K8S INGRESS]
    C --> D[TOMCAT A REDISSON]
    C --> E[TOMCAT B REDISSON]
    C --> F[TOMCAT C REDISSON]
    D & E --> H[JAVA APP A]
    E & D--> G[JAVA APP B]
    F --> J[JAVA APP C]

I would really appreciate help on this.

Just to update on this again, turns out the issue was because of the wrong cookie name for nginx-ingress
Before I had nginx.ingress.kubernetes.io/session-cookie-name: JSESSIONID which most probably conflicted with tomcat security.
Changing this to nginx.ingress.kubernetes.io/session-cookie-name: route fixed it and now everything works as expected.
I believe this can be closed now.


No debug

Hi,
There is no debug trait implemented. This would be useful for debug purpose.

Close with commit https://github.com/blackbeam/rust-crc16/commit/803f0ebfbc0f9d8aa5b148b3f3462c4e9b929931


Support Zookeeper ACL

Please answer these questions before submitting pull request

Why submit this pull request?

[ ] Bug fix

[x] New feature provided

[ ] Improve performance

Related issues
#3256

New feature or improvement
Support Cluster-Zookeeper-Plugin config ACL.

/run ci

/run ci

What is ip schema? Just a literal random String? @kezhenxu94

What is ip schema? Just a literal random String? @kezhenxu94

ip schema designates which ip (or ip range) can access the nodes, but in my test, if I pre-create the nodes (say /skywalking/remote) and set the ACL to ip of my machine (in zookeeper side), and set the enableACL to true and schema to ip (in SkyWalking side), it failed to connect to zk, after changing the enableACL to false, it works, which I think is a little strange

We use Zookeeper ACL that has 2 steps,

set ACL for zNode.
authorization, like login.

IP schema, we need not authorization step. But I will do those steps in this pr. @kezhenxu94

I think whether we support digest only. Because digest is more widely used, and it is more clear.

How do you think? @wu-sheng @kezhenxu94

@dmsolr One mode is good for me. Make sure it is clear at document.


Division operator inside stencil performs integer floor division instead of floating point devision on cpu backends

For example the code in_field[0,0,0] = 1/2 run inside the computation context manager of a stencil will assign 0 to all elements of in_field instead of 0.5. This issue is encountered only for the cpu backends (including dace) but not for the numpy backend.

Great find! This should be consistent. As a user, which would you rather see?

It should be Python semantics.

Gridtools backends generate
...
    template<typename Evaluation>
    GT_FUNCTION static void apply(Evaluation eval, gridtools::stencil::core::interval<gridtools::stencil::core::level<0, 1, 2>, gridtools::stencil::core::level<1, -1, 2>>) {

        eval(fout()) = static_cast<double>((static_cast<std::int64_t>(1) / static_cast<std::int64_t>(2)));
    }
...

and the numpy backend generates
        fout[i:I, j:J, k:K] = np.float64((np.int64(1) / np.int64(2)))

These are consistent with the IR, but have different semantics at runtime. I think it should be consistent, and it would be nice to have it follow the Python semantic (as is the guiding principle in gt4py). We'd either need to modify the upcasting rules in gtir_upcaster.py or add backend passes on each C++ backend. At the same time to support integer division, we'd want to add ast.FloorDiv support (//).

We can either add a special case for division or leverage the utilities that numpy provides to look up proper casting rules. (or both.)
Basically, numpy provides for each ufunc (i.e. unary, binary op) a list of implementations identified by input and output types.
E.g. for python division this is
>>> np.divide.types
['ee->e', 'ff->f', 'dd->d', 'gg->g', 'FF->F', 'DD->D', 'GG->G', 'mq->m', 'md->m', 'mm->d', 'OO->O']

We can then test our arguments to the operation with can_cast and each of those implementations and pick the first one where the inputs can be cast and the output type is also supported in gt4py.
Documentation can be found here https://numpy.org/doc/stable/user/basics.ufuncs.html?highlight=index#type-casting-rules

That's a neat idea! I'm a bit confused: the numpy docs claim the character for integers is l, yet I don't see that in the list there. Maybe there are some implicit upcast rules too?
Regardless, this integer division rule seems to exist outside numpy even on the python interpreter level
>>> 1/2
0.5

I put up a PR that does this the simple way for now. @gronerl I'll set you as the reviewer. Feel free to go in and add commits, etc. if you'd like to try out this idea.

I put up an alternative solution in #778 .


Add App and User Services Initializers

Adds App and User Services initializers to allow consumers to have code called when these services are brought up or torn down.
#93  && #29

Thanks! Dagger made this a breeze!


Mount-NavTenant Error

When I try to mount new tenant database restored from bacpac as described in https://freddysblog.com/2019/11/04/using-sql-server-on-the-host/ I get error
The following SQL error was unexpected:
The transaction operation cannot be performed because there are pending requests working on this transaction.
at , : line 15
The following SQL error was unexpected:
The transaction operation cannot be performed because there are pending requests working on this transaction.
At C:\Program Files\WindowsPowerShell\Modules\navcontainerhelper\<IP_ADDRESS>\ContainerHandling\Invoke-ScriptInNavContainer.ps1:37 char:13

        Invoke-Command -Session $session -ScriptBlock $scriptbloc ...

        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (0:Int32) [Mount-NAVTenant], FaultException`1
FullyQualifiedErrorId : MicrosoftDynamicsNavServer$BC/MyData,Microsoft.Dynamics.Nav.Management.Cmdlets.MountNavTenant

Where do you have the .bacpac from?
Is there anything in the eventlog? (get-navcontainereventlog)

Please reopen and provide more info if the problem persists


Improve grafana value grammar

Deploy preview for calico ready!
Built with commit 748075034bed60f2a603ab0eed3169b7c5118606
https://deploy-preview-3700--calico.netlify.app


Updated Get-DistributionGroupMember.md-ADO-8426457

@chrisda Could you please guide how to resolve this error.
ADO: https://office.visualstudio.com/MAX/_workitems/edit/8426457/

@chrisda Thank you very much! Going forward, I will follow this guidelines for all cmdlet reference topics.


app group for quickstarts

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/fabric8-quickstarts-master-pullreq/585/
Test PASSed.


Error: spawn \angular-seed\node_modules.npminstall\protractor\3.3.0.bin\webdriver-manager.cmd ENOENT

IMPORTANT: This repository's issues are reserved for feature requests and bug reports. Do not submit support requests here.
Steps to reproduce and a minimal demo of the problem

try remove node_modules, retry npm install
try remove all the project files, retry git clone and npm install
Problem still

What steps should we try in your demo to see the problem?

after git clone
npm install

Current behavior
[17:34:17] Starting 'webdriver'...
Error: spawn E:\projects\LearnAngular\ng-demo\angular-seed\node_modules\.npminstall\protractor\3.3.0\.bin\webdriver-manager.cmd ENOENT
    at exports._errnoException (util.js:1012:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:182:32)
    at onErrorNT (internal/child_process.js:348:16)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickCallback (internal/process/next_tick.js:98:9)
    at Module.runMain (module.js:577:11)
    at run (bootstrap_node.js:352:7)
    at startup (bootstrap_node.js:144:9)
    at bootstrap_node.js:467:3
Error: post install error, please remove node_modules before retry!
Run "C:\WINDOWS\system32\cmd.exe /d /s /c gulp check.versions && gulp build.bundle.rxjs && npm prune && gulp webdriver && gulp print.banner" error, exit code 1
Error: post install error, please remove node_modules before retry!
Run "C:\WINDOWS\system32\cmd.exe /d /s /c gulp check.versions && gulp build.bundle.rxjs && npm prune && gulp webdriver && gulp print.banner" error, exit code 1
    at ChildProcess.proc.on.code (C:\Users\work-liwei\AppData\Roaming\npm\node_modules\cnpm\node_modules\runscript\index.js:67:21)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:852:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:215:5)

Expected/desired behavior
Other information
npm's version: 3.10.3
node version: v6.3.1
the latest angular-seed

Use npm run start can load the project and run...

Yes, you need to run npm start once the installation is completed.
The Duplicate identifier should have been caused by a modification of the seed since it is not reproducible in the Windows CI, neither or my parallels Windows VM.


I imported my mod features into this newer version 0.6.3, and my song won't work...

Describe your problem here.
It's been a while since I released my mod on [NON PROMOTIONAL SITE] and I wanted to keep working on a sequel to that mod, but I went on hiatus for a while before it could be finished. I only recently got back into finishing the mod and was hard at work finishing some features. When I open the game, the "new version" message appeared on the screen, and feeling hopeful, I installed the new version. I assumed that some features were similar to the version the mod was on, so I carefully copied some files over to the newer engine version and made sure it mirrored the files exactly where they were placed like before. I executed the exports perfectly and made sure everything was working.
Most songs played perfectly like before, but then the problem began with one of the first songs I've ever made. The Song titled "G.A.M.E.R" and files "g.a.m.e.r-easy -hard.JSONs" couldn't load even though it was working perfectly before I imported everything to the newer psych engine version. I have no idea why it's not working nor know how to fix this. The text below is what pops up when I attempt to load in the song:
**C:\HaxeToolkit\haxe\std/cpp/_std/sys/io/File.hx (line 30)
Song.hx (line 104)
FreeplayState.hx (line 371)
flixel/FlxState.hx (line 202)
flixel/FlxGame.hx (line 747)
flixel/FlxGame.hx (line 677)
flixel/FlxGame.hx (line 545)
openfl/events/EventDispatcher.hx (line 402)
openfl/display/DisplayObject.hx (line 1399)
Uncaught Error: [file_contents,assets/data/gam.e.r/game.r-hard.json]
Please report this error to the GitHub page: https://github.com/ShadowMario/FNF-PsychEngine
Crash Handler written by: sqirra-rng**
Yes, I just created a Github account to make this.
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No response

UPDATE: Turns out I had to change the name of the song multiple times. I managed to get it working withy no further issues.

UPDATE: Turns out I had to change the name of the song multiple times. I managed to get it working withy no further issues.


Problem with Method Side Effects

Methods are having very wide-ranging side-effects. For example, the following fails:
method read(&int x) -> (int r):
    return 0

public export method test() :
    //
    &int p = new 1
    //
    int v = read(p)
    int w = read(p)

This fails because, for the second invocation of read(p), the reference p is not considered to meet its type constraint.   In effect, Boogie thinks that p could point to anything after the first method call, including something which is not an integer.

Right, there are two issues here:

Type Invariants.  These should always be preserved.
Reference Values.  These should be preserved for anything which is not reachable.

Right, there are two issues here:

Type Invariants.  These should always be preserved.
Reference Values.  These should be preserved for anything which is not reachable.

For reasons unknown, this is now failing:
public type Sum is &int

method start(Sum _this) :
    skip

method create() -> Sum:
    return new 0

public export method test() :
    Sum sum = create()
    start(sum)

For reasons unknown, this is now failing:
public type Sum is &int

method start(Sum _this) :
    skip

method create() -> Sum:
    return new 0

public export method test() :
    Sum sum = create()
    start(sum)

Right, any indirect invocations currently don't work.  Though that does not explain the above.

Right, any indirect invocations currently don't work.  Though that does not explain the above.

Fixed indirect invocations.  Strange example which doesn't verify now is:
public type Event is &{ int timestamp }

public export method test(method listener(Event)):
    // Fire the listener
    listener(new {timestamp: 123})

(Observe that the direct invocation does work)

Fixed indirect invocations.  Strange example which doesn't verify now is:
public type Event is &{ int timestamp }

public export method test(method listener(Event)):
    // Fire the listener
    listener(new {timestamp: 123})

(Observe that the direct invocation does work)

Fixed above.  Back to this example:
method start(&int _this) :
    skip

method create() -> &int:
    return new 0

public export method test() :
    &int sum = create()
    start(sum)

This fails the postcondition on test() which basically says that existing HEAP locations should be unchanged.  In fact, it's probably getting confused into thinking about the result of create() could be an existing heap location.

Fixed above.  Back to this example:
method start(&int _this) :
    skip

method create() -> &int:
    return new 0

public export method test() :
    &int sum = create()
    start(sum)

This fails the postcondition on test() which basically says that existing HEAP locations should be unchanged.  In fact, it's probably getting confused into thinking about the result of create() could be an existing heap location.


Add support for powerworld PW040 monoblock inverter heat pumps.

https://www.powerworldhp.com/heat-pump/commerciale-heat-pump/r32-air-to-water-monoblock-inverter-heat-pump.html (z.B. PW040-DKZLRS-A)
https://michl.com/bilder/intern/downloads/Anleitung-SMP-Serie-08-2023.pdf

How can I run "YAML tests" standalone?

If you set up your python environment (recommend to use local environments) using pip -r requirements-dev.txt to install the dependencies, then you should be able to run pytest tests/test_device_config.py from the top level directory.


Move content to a Git submodule

No longer needed.
Don't know why I wanted to do this in the first place ~2 years ago


URL Cleanup

This commit updates URLs to prefer the https protocol. Redirects are not followed to avoid accidentally expanding intentionally shortened URLs (i.e. if using a URL shortener).
Fixed URLs
Fixed Success
These URLs were switched to an https URL with a 2xx status. While the status was successful, your review is still recommended.

http://maven.apache.org/xsd/maven-4.0.0.xsd with 5 occurrences migrated to:
https://maven.apache.org/xsd/maven-4.0.0.xsd (https result 200).
http://www.apache.org/licenses/LICENSE-2.0 with 2 occurrences migrated to:
https://www.apache.org/licenses/LICENSE-2.0 (https result 200).

Ignored
These URLs were intentionally ignored.

http://maven.apache.org/POM/4.0.0 with 10 occurrences
http://www.w3.org/2001/XMLSchema-instance with 5 occurrences

Closing in favour of https://github.com/spring-cloud-incubator/spring-cloud-square/pull/19.


Support allowClear in InputNumber component

[ ] I'm sure this does not appear in the issue list of the repository

Basic Info
What are the similar cases of this feature
<Input allowClear />
<DatePicker allowClear/>

....
What problem does this feature solve?
Is there any play to add allowClear to InputNumber?
Input, DatePicker and other input components support allowClear to quick set the value to null. I think it is OK to match with others for InputNumber

https://github.com/arco-design/arco-design/pull/549
The allowClear icon conflicts with the InputNumber chart，Can be supported, there may be other considerations.


WIP

For the issue #4120 , I added the field next_bankruptcy_reminder.

@swekshajha12 It will be helpful if you can change the title to something descriptive like [WIP] models: Add 'next_bankruptcy_reminder' field to UserProfile model. This helps reviewers while looking at the PRs to review. Also I think you would like to split this PR into two separate PRs one for updating the documentation and another for adding the field to the model since both are unrelated changes.  Thanks for working on this! :)

Sure , I'll do it! :+1:


fatal: Could not read from remote repository

This might be obvious, but I'm having a lot of trouble using an SSH key (a deploy key) using this action. I have the following workflow (fragment):
      - uses<EMAIL_ADDRESS>        with:
          ssh-private-key: ${{ secrets.DeployKey }}

      - name: Checkout the other repository
        run: |
          git clone git@github.com:Org/RepoName.git

However that gives me:
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

I know the deploy key works, and I also know it's correctly inserted as a secret. Any idea why this might be happening?

Ah, seems that this is orthogonal to this particular action. It can be solved by using actions/checkout to clone and ad-m/github-push-action to push, e.g.
      - uses<EMAIL_ADDRESS>        with:
          ssh-private-key: ${{ secrets.DeployKey }}

      - uses: actions/checkout@v2
        name: Checkout the other repo
        with:
          path: SomeRepo
          ref: master
          repository: 'org/SomeRepo'

      - name: Commit files
        run: |
          cd SomeRepo
          git config --local user.email<EMAIL_ADDRESS>          git config --local user.name "GitHub Action"
          git add -A
          git commit -m "Update"

      - name: Push changes
        uses: ad-m/github-push-action@master
        with:
          directory: SomeRepo
          github_token: ${{ secrets.GITHUB_TOKEN }}


HTTPS_URCE typos

Hi,
I think someone screwed up a search & replace somewhere.

Good catch a fix has been applied to main. https://github.com/Islandora-Devops/isle-buildkit/commit/ce8dd66b0023b2f80e3d5f50675849a6b3a83839


Feature/object get value deep

This PR adds a getValueDeep function to easily get a deep value from an object by its dotted path:
import getValueDeep from '@studiometa/js-toolkit/utils/object/getValueDeep';

const obj = { foo: { bar: 'baz', array: ['one', 'two'] } };

console.log(getValueDeep(obj, 'foo.bar')); // 'baz'
console.log(getValueDeep(obj, 'foo.array')); // ['one', 'two']
console.log(getValueDeep(obj, 'foo.array.0')); // 'one'

Hey @titouanmathis, maybe we should use optionnal chaining, a babel plugin exists?

Well, that's indeed more practical and future-proof than this proposal, that did not cross my mind. Thanks for the tip @lusimeon 👍
I am closing this PR for now, as it is already obsolete (😭).


Bank account - Workspace default currency becomes accessible after opening bank account RHP

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.51-3
Reproducible in staging?: y
Reproducible in production?: y
Issue reported by: Applause - Internal Team
Action Performed:

Go to staging.new.expensify.com.
Go to workspace settings > Free workspace > Bank account.
Add a bank account.
Refresh the page.
Go to workspace profile > Currency.
Note that Currency is not clickable.
Click Bank account.
Close the RHP.
Go to workspace profile > Currency.

Expected Result:
The workspace currency list will not be accessible because a US bank account is added.
Actual Result:
The workspace currency list becomes accessible after reopening bank account when US bank account is added.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence
https://github.com/Expensify/App/assets/93399543/319eb65f-8614-4897-9952-b9bd7efe0552
View all open jobs on GitHub

@MitchExpensify I haven't added the External label as I wasn't 100% sure about this issue. Please take a look and add the label if you agree it's a bug and can be handled by external contributors.

Same issue as https://github.com/Expensify/App/issues/36376

Closing because dupe


Microsoft Remote Desktop Beta

Short story...
Microsoft has a much improved version of the remote desktop app, called "Microsoft Remote Desktop Beta". To use the wonderful "Swap command and control" complex modifier you have to change the "bundle_identifiers" in the .json file.

Import the Swap command and control for remote desktop complex modifier.

Duplicate the .json (in the hidden location)

Change all occurrences of "com\.microsoft\.rdc\.mac" to "com\.microsoft\.rdc\.osx\.beta"

Change the description (e.g. add the word "beta") so you know it's targeting the newer version.

Long story...
I use Windows RDS from my Mac most of the day, and searched for a way to swap Cmd and Cntl keys. I only want to do this while in RDS. Attempts to modify the keyboard layout on Windows failed. I suspect it's because the remapping it done on Windows by intercepting USB. But (again I suspect) RDS is not sending USB keycodes. So none of the Windows tools will let me remap the Apple and Windows modifier keys when I'm using a Mac with RDS.
I think the Apple keyboard preferences used to let us modify keys for specific apps, but no longer. So it seems the only tool that can do the solve the problem above is Karabiner.
It took me too much time trying to find where the .json config files are located. Please make this more obvious in the documentation.
The new Microsoft Remote Desktop Beta includes the option to make the Cut, Copy, Paste options to the Cmd key (instead of the Windows key). I'd call that less than ideal. I keep ending up typing the Apple Cmd key by itself, which in RDS is the Windows key. That being up the start menu, argh! I often need to use the Select All command, which in RDS is Windows-a (or Cntl-a on the Mac while using RDS). Cmd-a in Windows 10 brings up the "Action Center." Cmd-f brings up the Feedback option. Way too difficult and my muscle memory is causing too much grief to try and change my touch typing skills. So simply providing Cut, Copy, Paste is basically no help at all.
I need to swap all Cmd and Cntl events. Thank you for the very useful tool!!!

Hi, this is a bit meta, I hope y'all don't mind...
I've been doing these things for a long time. I don't use your approach, because I try hard to NOT move keystrokes around, because that breaks muscle memory.
Let's say I want to hit ctrl-c to stop a script. Then I hit ctrl-c. You on the other hand must first consider if your script is local or remote, and hit cmd-c or ctrl-c.
My solution is to explicitly map all important shortcuts on Windows. For your RDP session that would be:
cmd-c to ctrl-c
cmd-a to ctrl-a
...and so on, for every global shortcut I use.
Now cmd-c ALWAYS copies, and ctrl-t does whatever the forground app defines.
This will obviously override the native Windows shortcuts. When you map "LEFT_cmd-a" to "ctrl-a", you can still press right_cmd-a to trigger "win-a" (action center).
Because [reasons] I also run AutoHotkey scripts on my Windows machines. It's a nice and powerful tool, but life is easier when you can have only one config (Karabiner in your case).
AHK scripts are quickly created and look like this:
LeftWin & c :: Send ^c
<#d::Run "cmd.exe"    ;short syntax for: leftWin+D opens dos box

cajhin, I'm not a Windows person. I'm a software engineer and have been on the Mac and Linux all my life. Windows is new, and I'm only using it for software development on Server 2016. And only with RDP from my Mac. The problem for me and my muscle memory is Mac and Windows basically swap the Cmd and Cntl keys. I'm on a Mac, using Windows (via Desktop Remote). My muscle memory is for all commands (e.g. copy, paste, undo, select all) use the Apple Command modifier key closest to the space bar (but that is the Cntl key on Windows). That's how the Mac works (and how I've been doing it since 1984).
To type the Windows "command" it's the Apple Cntl key, which is the furthest key from the space bar. I'm not just fighting my muscle memory, it's also an awkward key combination to type. I'm switching back and forth, quickly, on my Mac from a Mac application to a Windows application. Swapping the Cntl and Cmd key only while I'm in the Windows application solves the problem. It's a simple fix with Karabiner. It's just one change.
Here's what I need. I want the Windows key to be mapped to the Apple Command key, in all cases. I also want the Apple Cntl key to always been the Windows Cntl key. I don't know how to do this with AHK.
I tried AHK and couldn't make it work. It would swap the two keys, but then failed in my PowerShell and PowerShell ISE. After some searching, I found the problem is because my PowerShell is running as admin. I found out I have to run the AHK script as admin to make it work in my PowerShell and PS ISE.
But even that didn't work. Pressing the Apple Command key alone normally triggers the Windows key, which brings up the Start menu. After running my AHK script, pressing either of the two keys alone worked as I expected. Pressing the Cntl key on my Mac keyboard acts like the Windows key and brings up the start menu. So far so good.
But typing Cmd-A doesn't select all. Instead, it ends up typing the "a" into PowerShell ISE. Same problem with Cmd-Z, which should undo but types the "z". So I don't understand how to use AHK to map the two modifier keys. The entire AHK script I tried is:
LCtrl::LWin
LWin::LCtrl
Q: How would I use AHK to swap the Windows and Cntl key, and function properly as modifier keys? This is for Windows 10, and to work in all applications even if they're run as admin.

I think you've messed up something with the ctrl key.
I just tried it out: no Karabiner on the Mac, remote desktop into a Win10 box, replaced the AHK script with your two lines.
Result: it works. The CMD and STRG keys are switched (both stand-alone and as modifiers for other keys).
Cmd+a selects all in powershell. Ctrl+D shows the Windows desktop.
I never had issues with AHK and admin. I'm local admin on all Win boxes, but I start AHK normally (without admin credentials). The only place where it doesn't work, by design, is the Windows security panels (modal dialog over blurry desktop "do you want to allow this application to steal all your moneys bla?").
You say that Ctrl is the leftmost key? On a Mac, that should be 'Fn', no??
When I press Fn+A in PowerShell, I get an 'a'... I think your modifier keys are already switched somehow.
(BTW, I intentionally switch Ctrl and Fn because Apple made the wrong choice there, IMO. Ctrl belongs into the corner...)

Without Karabiner running, my two line AHK script fails.
LCtrl::LWin
LWin::LCtrl
Within PS ISE on my Mac via Remote Desktop, if I type...
Cmd key alone I get nothing (instead of the usual Windows key which brings up the startup menu)
Cntl key alone I get the Windows key press and it brings up the startup menu
Cntl-a I get the Action Centre
Cmd-a I get "a"
Cmd-a and then releasing keys and type Cmd-a again I get another "a"
Cmd-a a second time while continuing to hold the Cmd key I get Select All
Cmd-a and I get "a" and while continuing to hold the Cmd key I type o then I get the Open File dialog
In other words, to get Cmd to act like a modifier I first have to hold it down then type a letter. That puts types the letter (bad). But if I keep holding it down and type again, then that letter is interpreted as a modified key.
Karabiner is working better, but I find it has conflicts with SteerMouse on my Mac. It's also worrisome that Karabiner is always running, intercepting keys, when I only need this effect while in Windows Remote Desktop.
BTW - On my Mac keyboard, the left most key on the bottom row is Control. Then Option, and then next to the space bar it's Command. It's been this way on all my keyboards since I can remember. I always use the Apple desktop keyboard with numeric keypad. That one is a close match to an industry standard keyboard. They're laid out like this.


ci: grant cla-assistant ability to write comments on issues and PRs

The GITHUB_TOKEN is read-only by default in the public repository. This update grants the cla-assistant configured in the cla.yml permission to create a comment on the Pull Request it is checking. This way, the cla-assistant can notify the Contributor to sign the CLA and provide the link.

Thank you.


Correct relative CSS urls without period directory references

In app.html, if you require a module's CSS that includes a url reference, like for an image or a font, Aurelia does not determine the relative path if the link does not contain a period or double-period directory reference.  For example, in the module's CSS,
background-image: url('images/background.jpg');

will not be translated, causing the application to look for an images directory in the root, but
background-image: url('./images/background.jpg');

will correctly be translated, causing the application to look in the images directory relative to the module's js.  The problem can be "patched" in the fixupCSSUrls function of css-resource.js (https://github.com/aurelia/templating-resources/blob/master/src/css-resource.js#L16) by replacing
return 'url(\'' + relativeToFile(p1, address) + '\')';

with
return 'url(\'' + relativeToFile('./' + p1, address) + '\')';

However, I'm hesitant to perform a pull request here if the problem should be really be corrected in the relativeToFile function of https://github.com/aurelia/path/blob/master/src/index.js#L34 without breaking any of its other callers.

I think the ./ is required for relative urls.  Items without ./ could be an installed dependency eg bootstrap/css/bootstrap.css, the ./ helps make the distinction and keeps things explicit.
Closing- @EisenbergEffect please re-open if I have this wrong.


Downloading, Installing, Configuring.

Hello. I have a problem with the latest version.

I did update & upgrade
sudo apt update
sudo apt upgrade

I have installed apache 2
sudo apt install apache2
3.I installed ufw and did port 80 and 433 permission
sudo ufw allow ssh
sudo ufw allow 80
sudo ufw allow 443
sudo ufw enable

turned on ufw.
5.I have installed php 7.4
sudo apt install php7.4 php7.4-mysql php-common php7.4-cli php7.4-json php7.4-common php7.4-opcache libapache2-mod-php7.4

I have installed sqlite
sudo apt install sqlite3

I downloaded unam web panel
sudo wget
https://github.com/UnamSanctam/UnamWebPanel/releases/tag/uwp-v1.5.0

I unpacked the zip
unzip uwp-v1.5.0.zip

I changed the password in config
10 login

Waited 24 hours and nothing show.

In miner i have http: //ip/api/endpoint.php
What step i left?

Firstly, that's an old version (1.5.0), secondly are you sure that you have port 80 (http) open?

I just use VPS, there are all ports open TCP/UDP also in firewall

And if you visit http://ip/api/endpoint.php from the computer you're running the miner on then you see the endpoint page?

yeah, there is communication - >Error: Invalid data

vps is using http and miner using http protocol

First update your web panel and check.

ahh, i remember now why i chose 1.5. In 1.6 version i cant even login. There is communication Error, Something went wrong. When i will make all htdocs with chmod 777 login work. So what i need to increase permission?

one session i had valid, so i receive communication: DataTables warning: table id=miner-table - Ajax error. For more information about this error, please see http://datatables.net/tn/7

You need to make the db folder and unamwebpanel.db inside it writable. Did you do all the correct chown and chmod?

what chmod unamwebpanel.db should to have?

i didnt use chown anymore.

Usually you do sudo chown -R $USER:$USER htdocslocation and sudo chmod -R 755 htdocslocation and then you might also need additional permissions for the database.

database has nothing to login i think, so i didnt do anything with it yet. i have still Error problem with login after your commands

in htdocs folder i just did sudo chown -R $USER:$USER * then sudo chmod -R 755 * * - mean ALL in this folder

i see that not only me has this problem before someone warn also

The reason you get an error is because you have set up something wrong, check for an error.log file or for a log file inside the __UNAM_LIB\Logs folder.

i didnt make any additional steps than above and your suggestions. I described all step i made

so i dont understand more what is wrong?. I try to understand.  During one week i reinstalled it so many times, same problem still.

i dont have any logs or error,log in htdocs.  And are you sure that you have all the files? There shouldn't be any problem with setting up apache and then the web panel, I helped someone do it just yesterday. If you use Ubuntu then you can use this guide https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-ubuntu-20-04.

I followed this guide + installed php 7.4 then sqlite3. Same problem with login

Hi Unam, I tried do it on 2 vps, still problem with login. Error something went wrong. After resolving problem i would like to make installation script for everyone, that will configure everything itself, but first I need to resolve my problem . I don't know what is wrong

And you tried using sudo wget https://github.com/UnamSanctam/UnamWebPanel/releases/download/uwp-v1.6.0/UnamWebPanel.zip to download the files?

Yeah exactly. with sudo.  Maybe I should to download by sftp, but i don't see difference

You downloaded that exact file? I set up a test Ubuntu VM today for the web panel and it worked fine.

Exactly this file, then unzip UnamWebPanel command. In /var/www/html

I will try install LAMPP ( xampp on Linux) then install sqlite3 and sqlite3-php. It have preconfigured settings. Maybe it will work

Hello, Unam. XAMPP on Linux works, but i needed to put command chmod 777. I tried to download database but it's forbidden. If there is any more risk, if everything in htdocs has 777 permission?

Hello, Unam. XAMPP on Linux works, but i needed to put command chmod 777. I tried to download database but it's forbidden. If there is any more risk, if everything in htdocs has 777 permission?

It's generally not recommended, but if that server is only used to host the web server and nothing else then it's technically fine if you can't get it working otherwise. Though you should be able to lower the required permissions as long as the owner of the all the files is the correct user. The standard permissions used are usually 644 or 755 on the premise that the owner of the files is the user that is running Apache.