[WIP] TrueCraft FNA

This is less of a pull request and more of an experimental branch of TrueCraft that uses FNA over MonoGame for the TrueCraft.Client library. The game should function exactly the same as it did before.
All of the changes here are strictly changes to make the game XNA4-compliant. This includes API changes such as the SpriteBatch.Begin calls and runtime changes such as setting the GraphicsProfile to HiDef, as XNA4 would need for the 32-bit index buffers.
Here's a screenshot of this branch in action:

This branch's performance has not yet been compared to the original master branch.

Earlier PR for comparison/discussion: https://github.com/SirCmpwn/TrueCraft/pull/148

Initial thoughts:

This references FNA as a submodule, but you haven't added the submodule

The last time around I was concerned about making people build FNA in the first place, NuGet packages are still not published?

On Linux I had some issues with water textures loading with incorrect colors without using PngReader. Is that an issue here?

What is in your TODO list? This PR has [WIP] on the title, wondering what all remains to be done. I noticed, for example, that your screenshot has no hotbar.
Can you also summarize the reasons FNA is worth it over MonoGame?

For now this is just a sample case for FNA's accuracy and not meant to be used in production for any real reason. There are likely performance benefits with our renderer, but this hasn't been measured yet.
I've not made NuGet packages only because there isn't a way to package as source and not as binary, and I don't want to publish FNA as binaries even for NuGet.
I'm having a look at the textures with apitrace now.

For now this is just a sample case for FNA's accuracy and not meant to be used in production for any real reason. There are likely performance benefits with our renderer, but this hasn't been measured yet.

So you're not looking to get it merged? How would you like me to treat this PR?

I've not made NuGet packages only because there isn't a way to package as source and not as binary, and I don't want to publish FNA as binaries even for NuGet.

I'm curious about why you've made that choice. I notice you publish binaries for all of your dependencies, for example.

I'm having a look at the textures with apitrace now.

A simple visual inspection of water should do the trick :wink: if it's purple, it's borken.

I publish the native bins, but all the managed assemblies are what people usually need to debug - XNA inaccuracies are hard to catch without the ability to step through FNA.
A good example caught quickly with breakpoints in FNA: it turns out some of the rendering code depends on MonoGame's GL inaccuracy with regard to things like depth bias (glPolygonOffset); a possible reason why things like the HUD aren't showing up is because of rasterizer/blend/depth states varying between MonoGame and FNA.

Well, I have more to say about that, but it's not really on topic for this PR. Feel free to reach out externally if you want to hear my thoughts on that.
In my role as the TrueCraft maintainer, though, how do you want me to deal with this PR? Do you want it considered for merge? I'm still not sure what you want me to do with this PR.

For now, just leave it as-is and don't worry about merging it. This will only ever matter if it's possible to have both MonoGame and FNA rendering correctly with the same code, but at the same time other people may be interested in this if MonoGame does not work correctly for whatever reason (MG dependencies like SDL 1.2 and OpenTK, etc.).

Alright. Do you want my feedback on your changes?

Give me a few days to poke around the GL a little bit more and I'll put together a build for people to try out.

Alright, good luck!

Quick update on the HUD, might affect the MonoGame build depending on OpenTK's behavior per-target: ScaleFactor is not set unless the client size is changed after the window is made. FNA/XNA only call that function when the user resizes a resizable window, so the scale was never getting set. This fixes the HUD for FNA:
https://github.com/flibitijibibo/TrueCraft/commit/3eafa3a94a8a8e846da834a92c71f0cfd5feb939
That said, if TC ever gets a resolution option in-game, that callback may need to be something like GraphicsDevice.DeviceReset instead.

Nice find. For what it's worth, I doubt TrueCraft will have in-game options in the foreseeable future.

As an aside, I'd also be curious to see how you'd deploy TrueCraft with FNA (with the Linux installer and the OSX archive) to deal with the launcher.

I'll be honest, the Xwt dependency tree kind of scares me... I'm looking at the csproj files now and it somehow manages to have enough gunk in it to the point where it may as well just be System.Windows.Forms! It's definitely a tough call, but the good news is that the rest would be pretty easy to work with. Instead of calling mono you'd just call, for example, TrueCraft.Client.bin.x86_64. The bin suffix could be detected at runtime without much fuss, and you might even be able to optimize the binary size by reverting the recent change of statically linking the Mono runtime into the binary and have all the kick.bin.* files refer to that one libmonosgen lib.
But, packaging aside, I think I have the game rendering accurately now... modified this one depth bias value to be accurate to D3D:
https://github.com/flibitijibibo/TrueCraft/commit/8e7164977452d016500366b0b422c7893187765d
I also figured out why the water was purple for you - it actually had to do with both MonoGame's RenderTargetUsage accuracy as well as the final target draw's blending mode:
https://github.com/flibitijibibo/TrueCraft/commit/0e66c1421c302f221ce122db9d248499f15b0bea
The purple tint you were seeing was actually the recently-cleared backbuffer, which by default clears to that awful purple color by default in XNA. Of course, you draw to the entire screen and don't care about the previous contents anyway, so you can skip it with BlendState.Opaque. Even better, you don't even need to clear at all - on top of the previous contents of the backbuffer not being important, you also don't care about the RenderTarget's contents, since you clear to CornflowerBlue at the start of the frame. So you can actually skip a clear by making use of RenderTargetUsage.PreserveContents, giving us the behavior we want and making things a LOT faster than you'd expect!

I'll be honest, the Xwt dependency tree kind of scares me... I'm looking at the csproj files now and it somehow manages to have enough gunk in it to the point where it may as well just be System.Windows.Forms! It's definitely a tough call, but the good news is that the rest would be pretty easy to work with. Instead of calling mono you'd just call, for example, TrueCraft.Client.bin.x86_64. The bin suffix could be detected at runtime without much fuss, and you might even be able to optimize the binary size by reverting the recent change of statically linking the Mono runtime into the binary and have all the kick.bin.* files refer to that one libmonosgen lib.

Thanks for the insight. At some point it might make sense to move the launcher code into an in-game GUI, but that's quite an effort.

But, packaging aside, I think I have the game rendering accurately now... modified this one depth bias value to be accurate to D3D

Cool. This code was written pretty blindly to adjust the rendering of the outline around the block your mouse points to. Does that still work as expected?

I also figured out why the water was purple for you - it actually had to do with both MonoGame's RenderTargetUsage accuracy as well as the final target draw's blending mode

Nice, I'm glad this ends with better performance too.

The DepthBias fix works for me; my test case was the cracking sprite when destroying a block. The change just takes in the calculation we normally do to recreate the D3D-like bias and inverts it, so the result is '-3' as the client was originally expecting.
The only GUI lib I've ever shipped without feeling like I've done something horribly offensive is ImGui, but it doesn't really have anything for web rendering like the launcher wants at the moment:
https://github.com/ocornut/imgui
It does work with GPU rendering though, as I did with file dialogs for TowerFall Ascension:
https://github.com/flibitijibibo/XNAFileDialog

Oh man that interop is awful, I'd sooner just make my own GUI library with the Minecraft L+F

Yeah... life with C++! :(

An FNA build is downloadable from here:
http://www.flibitijibibo.com/TCFNA.tar.bz2
This only provides the native libs used by FNA and modifies the script to add the folders to the LD_LIBRARY_PATH, it does not provide the full MonoKickstart runtime due to the Xwt dependency. But, as long as you have Mono 4.x and Xwt's dependencies, this packages should work just fine.

I did a bit of an ugly thing to Paths.Base just now - it sidesteps SpecialFolder to get working paths on OSX when the time comes for that target.  Normally I just use SDL_GetPlatform() but without linking to FNA I had to improvise. It should be fine until someone tries running this on one of the BSDs or something, in which case we just have to use shell instead of checking for the OSX /home/ folder. I'll likely change this before that happens either way, but just FYI!

This is a thing: https://github.com/SirCmpwn/TrueCraft/blob/master/TrueCraft.Core/RuntimeInfo.cs

Oh ho, glad to see I don't have to write that then! Will update tomorrow morning.

This branch has been updated for the latest changes. Note that this may still crash due to problems with Xwt.Gtk3, as I had this crash too:
[ERROR] FATAL UNHANDLED EXCEPTION: System.Reflection.ReflectionTypeLoadException: The classes in the module cannot be loaded.
  at (wrapper managed-to-native) System.Reflection.Assembly:GetTypes (System.Reflection.Assembly,bool)
  at System.Reflection.Assembly.GetTypes () [0x00000] in /builddir/build/BUILD/mono-4.0.5/mcs/class/corlib/System.Reflection/Assembly.cs:360
  at TrueCraft.Core.Logic.BlockRepository.DiscoverBlockProviders () [0x00021] in /home/flibitijibibo/Programming/csProjects/TrueCraft/TrueCraft.Core/Logic/BlockRepository.cs:30
  at TrueCraft.Launcher.Singleplayer.Worlds.Load () [0x0002c] Vector smash protection is enabled.
in /home/flibitijibibo/Programming/csProjects/TrueCraft/TrueCraft.Launcher/Singleplayer/Worlds.cs:24
  at TrueCraft.Launcher.Views.SingleplayerView..ctor (TrueCraft.Launcher.LauncherWindow window) [0x00016] in /home/flibitijibibo/Programming/csProjects/TrueCraft/TrueCraft.Launcher/Views/SingleplayerView.cs:35
  at (wrapper remoting-invoke-with-check) TrueCraft.Launcher.Views.SingleplayerView:.ctor (TrueCraft.Launcher.LauncherWindow)
  at TrueCraft.Launcher.LauncherWindow..ctor () [0x00087] in /home/flibitijibibo/Programming/csProjects/TrueCraft/TrueCraft.Launcher/LauncherWindow.cs:40
  at (wrapper remoting-invoke-with-check) TrueCraft.Launcher.LauncherWindow:.ctor ()
  at TrueCraft.Launcher.Program.Main (System.String[] args) [0x00097] in /home/flibitijibibo/Programming/csProjects/TrueCraft/TrueCraft.Launcher/Program.cs:36

The fix is to go to Program.cs:Main() and change the Gtk3 line to just Gtk.

Digging this old thing up from the grave... the branch still doesn't do anything remarkable and appears to work as much as it doesn't appear to be explicitly breaking anything, but the engine doesn't appear to want to render any blocks (as in, no GL calls happen for FNA to send out). I did notice that initial world loading had this inside...
"Threadpool worker"  at System.Collections.Generic.Stack`1<TrueCraft.Profiling.Profiler/ActiveTimer>.Pop () <0x0007d>
  at TrueCraft.Profiling.Profiler.Done (long) <0x000ab>
  at TrueCraft.Core.Lighting.WorldLighting.LightVoxel (int,int,int,TrueCraft.Core.Lighting.WorldLighting/LightingOperation) <0x00d83>
  at TrueCraft.Core.Lighting.WorldLighting.LightBox (TrueCraft.Core.Lighting.WorldLighting/LightingOperation) <0x0055b>
  at TrueCraft.Core.Lighting.WorldLighting.TryLightNext () <0x00127>
  at TrueCraft.Core.Lighting.WorldLighting.InitialLighting (TrueCraft.API.World.IChunk,bool) <0x0028b>
  at TrueCraft.MultiplayerServer.HandleChunkGenerated (object,TrueCraft.API.World.ChunkLoadedEventArgs) <0x00157>
  at (wrapper delegate-invoke) System.EventHandler`1.invoke_void_object_TEventArgs (object,TEventArgs) <0x000b2>
  at (wrapper delegate-invoke) System.EventHandler`1.invoke_void_object_TEventArgs (object,TEventArgs) <0x000b2>
  at (wrapper delegate-invoke) System.EventHandler`1.invoke_void_object_TEventArgs (object,TEventArgs) <0x000b2>

// Like a jillion more of these

  at (wrapper delegate-invoke) System.EventHandler`1.invoke_void_object_TEventArgs (object,TEventArgs) <0x000b2>
  at (wrapper delegate-invoke) System.EventHandler`1.invoke_void_object_TEventArgs (object,TEventArgs) <0xffffffff>
  at TrueCraft.Core.World.World.OnChunkGenerated (TrueCraft.API.World.ChunkLoadedEventArgs) <0x0002f>
  at TrueCraft.Core.World.Region.GenerateChunk (TrueCraft.API.Coordinates2D) <0x0017f>
  at TrueCraft.Core.World.Region.GetChunk (TrueCraft.API.Coordinates2D,bool) <0x000a7>
  at TrueCraft.Core.World.World.GetChunk (TrueCraft.API.Coordinates2D,bool) <0x00107>
  at TrueCraft.RemoteClient/<UpdateChunks>c__AnonStorey1.<>m__0 () <0x000b8>
  at System.Threading.Tasks.Task.InnerInvoke () <0x00051>
  at System.Threading.Tasks.Task.Execute () <0x0004c>
  at System.Threading.Tasks.Task.ExecutionContextCallback (object) <0x00053>
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object) <0x00060>
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x00017>
  at System.Threading.Tasks.Task.ExecuteWithThreadLocal (System.Threading.Tasks.Task&) <0x00137>
  at System.Threading.Tasks.Task.ExecuteEntry (bool) <0x000bf>
  at System.Threading.Tasks.Task.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () <0x0000f>
  at System.Threading.ThreadPool.<UnsafeQueueCustomWorkItem>m__0 (object) <0x000ae>
  at (wrapper runtime-invoke) <Module>.runtime_invoke_void__this___object (object,intptr,intptr,intptr) <0xffffffff>

Not sure what that's about, but maybe it's getting stuck somewhere before block buffers can be generated. In any case, now that TC seems to be picking up again I thought I'd at least get the FNA version updated.

A rewrite to OpenTK straight-up is in the pipeline.

Whatever works. I may do a similar branch that replaces the non-GL/AL bits with SDL2# and wedges MiniTK into the middle for those bindings. Same idea, different libs.


MSL：open argument buffer option , do not support  set buffer index offset

we use argument buffer as vs or ps param input , is  the [[buffer]]  support has a index offset to start from , example ,use id between 16-31

Yes, you should be able to remap argument buffers' binding indices. You can set up an MSLResourceBinding using the descriptor set number corresponding to the argument buffer as the desc_set, and the special constant kArgumentBufferBinding as the binding, and the Metal binding number you want as the msl_buffer.

Wonderful! thanks!


Source file not found when relative path contain multiple double-dot

Describe the bug
When relative path to source file contains multiple double-dots @mikro-orm/reflection is unable to find files.
Stack trace
[Nest] 48471  - 01/14/2023, 3:39:52 PM   ERROR [ExceptionHandler] Source file '../../shared/db/dist/src/entities/asset.entity.ts' not found. Check your 'entitiesTs' option and verify you have 'compilerOptions.declaration' enabled in your 'tsconfig.json'. If you are using webpack, see https://bit.ly/35pPDNn
MetadataError: Source file '../../shared/db/dist/src/entities/asset.entity.ts' not found. Check your 'entitiesTs' option and verify you have 'compilerOptions.declaration' enabled in your 'tsconfig.json'. If you are using webpack, see https://bit.ly/35pPDNn
    at TsMorphMetadataProvider.getSourceFile (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:113:19)
    at TsMorphMetadataProvider.getExistingSourceFile (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:31:28)
    at TsMorphMetadataProvider.getExistingSourceFile (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:27:89)
    at async TsMorphMetadataProvider.readTypeFromSource (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:64:24)
    at async TsMorphMetadataProvider.initPropertyType (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:53:36)
    at async TsMorphMetadataProvider.initProperties (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:38:17)
    at async TsMorphMetadataProvider.loadEntityMetadata (/home/mohammad/workbench/odin/node_modules/@mikro-orm/reflection/TsMorphMetadataProvider.js:23:9)
    at async MetadataDiscovery.discoverEntity (/home/mohammad/workbench/odin/node_modules/@mikro-orm/core/metadata/MetadataDiscovery.js:225:9)
    at async MetadataDiscovery.discoverReferences (/home/mohammad/workbench/odin/node_modules/@mikro-orm/core/metadata/MetadataDiscovery.js:164:13)
    at async MetadataDiscovery.findEntities (/home/mohammad/workbench/odin/node_modules/@mikro-orm/core/metadata/MetadataDiscovery.js:91:9)

Expected behavior
Source file should be found regardless of how many steps its go up in directory
Additional context
The problem is regex of replace() in TsMorphMetadataProvider getSourceFile method.
Something like /^\.+(\/\.+)*/ could fix this issue.
Versions

Dependency
Version

node
18.12.1

typescript
4.9.4

mikro-orm
5.6.4

mikro-orm/reflection
5.6.4

@mikro-orm/postgresql
5.6.4

It could be solved with baseUrl.

How was the problematic file named? By double-dot, do you mean two consecutive dots?

I wanted to try out svelte-kit with mikro, but i'm running into this bug as well. svelte-kit's $lib alias maps to a relative URL apparently, leading to errors like Source file '../../../../src/lib/server/entities/index.ts' not found..
I would love to be able to follow svelte-kit's prescribed paradigms (idiosyncratic though they may be) while still using mikro-orm, so I hope this issue gets re-opened.

Provide a reproduction and we can reopen.

My apologies, I think my issue was actually related to sveltekit using vite, which apparently does not support metadata reflection. I think this requirement could have been made a little more clear in the docs, but if I disable the TsMorphMetadataProvider and specify types on my properties, I am able to use the ORM without issue. Sorry for the noise!

Maybe you could leverage this in v6: https://github.com/mikro-orm/mikro-orm/pull/4167
Could be used locally as well if you treat it as a build step.

Thanks @B4nan , so no chances we can have this working with v5.8.5 ?

I'll be happy to look into a complete reproduction, or accept a PR if you see a way to fix your problem.


Location Serialization saves BlockType?

    @Override
    public DataContainer toContainer() {
        final DataContainer container = DataContainer.createNew();
        container.set(Queries.CONTENT_VERSION, getContentVersion());
        if (getExtent() instanceof World) {
            container.set(Queries.WORLD_NAME, ((World) getExtent()).getName());
            container.set(Queries.WORLD_ID, getExtent().getUniqueId().toString());
        } else if (getExtent() instanceof Chunk) {
            container.set(Queries.CHUNK_X, ((Chunk) getExtent()).getPosition().getX())
                .set(Queries.CHUNK_Y, ((Chunk) getExtent()).getPosition().getY())
                .set(Queries.CHUNK_Z, ((Chunk) getExtent()).getPosition().getZ())
                .set(Queries.WORLD_NAME, ((Chunk) getExtent()).getWorld().getName())
                .set(Queries.WORLD_ID, ((Chunk) getExtent()).getWorld().getUniqueId().toString());
        }
        container.set(Queries.BLOCK_TYPE, this.getExtent().getBlockType(getBlockPosition()).getId())
            .set(Queries.POSITION_X, this.getX())
            .set(Queries.POSITION_Y, this.getY())
            .set(Queries.POSITION_Z, this.getZ());
        return container;
    }

The current Location classes serialization is interesting.
Location seems to have somewhat of a crisis of identity, It more closely represents a BlockLocation rather then a (floating) location inside a world or extent, personally I think it could be due for a rename.
Anyway, the current serialization above, only serializes a reference to an extend if it's a world or chunk, as extent has no id to speak of...
And at the end it falls back to serializing the Position XYZ of the block, as well as the BlockType.
I feel like, at minimum, the BlockType needs to be removed?
It makes sense to me to treat a Location like a Unit Extent except for when it comes to the serialization, In this case it just makes no sense, we already have ways of saving out extents as schematics, or saving worlds, and it doesn't suit the naming.
I see it as an access object / lens to an extent, and serializing it shouldn't serialize the contents of the extent, it should be serializing the lens itself.
If it were possible to get an ID from an extent, and serialize that as well as it's name I'd advocate that as well, but that will involve API changes.
e.g.
A Location Would be serialized as a GridLocation X,Y,Z with a serialized identifier/reference of it's extent, defined by the extent.
World extends Extent would output the world name and world ID for this reference.
Chunk extends Extent would output the reference to it's container (World usually) as well as a Chunk X and Chunk Z.
Arena extends Extent would output the registered name of the arena as it's extent id / reference. e.g. MyArenas:DeathArena2

https://github.com/SpongePowered/SpongeCommon/issues/2109#issuecomment-425076609
Linking that since that issue was closed in favor of this issue.

Fixed by https://github.com/SpongePowered/SpongeAPI/commit/b6e56252f976bd47fbc4a9e82a660a36a0e29cb8


[ETHOSN] Add support for Ethos-N 21.05 driver stack release

Updated default Ethos-N driver stack to 21.05

@leandron if you have time.

Thanks @Leo-arm, this is merged now.


add support to kafka IAM-tls

Pull Request type

[ ] Bugfix
[ X] Feature
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes (Please run ./gradlew generateLock saveLock to refresh dependencies)
[ ] WHOSUSING.md
[ ] Other (please describe):

Changes in this PR
Describe the new behavior from this PR, and why it's needed
Issue #
Add ability to interact with Kafka server that uses tls and client authentication, those configuration would be per request
Alternatives considered
Describe alternative implementation you have considered

Please fix build failures.

Build still fails due to lack of version -

If you want to use IAM client auth you need this lib so it is used only if you add the IAM authentication in the request which declare the class in this lib

Can you add this as a runtimeOnly dependency in that case?

If this is still needed, please open PR in https://github.com/Netflix/conductor-community.


results: add GH pages generation and deployment

This PR adds scripts to fetch and build an HTML representation of the results generated by FPGA tool perf CI.
It also adds a cron scheduled workflow to automatically build and deploy on GH pages.

This PR does not modify any of the parts involved in building the tests, merging.


Cannot use molfile_to_params_polymer_main.py

Thank you Ben for posting this tutorial.
I read your ppt and it helps my understanding of bcl based ncAA rotamer preparation.
As of now, I am trying to reproduce workshop_2022's tutorial 8 instead of this 'brownbp1/ncaa_tutorial' since files at that workshop are easier for me to follow step by step.
I watched youtube video as well.
I generated dipeptide and conformers according to this workshop's tutorial pdf file.
However, I see

when I tried to run

When I prepared rosetta ncAA rotamer by a classic method, I just ran
py_file="/people/kimd999/bin/rosetta_bin_linux_2021.16.61629_bundle/main/demos/public/using_ncaas_protein_peptide_interface_design/HowToMakeResidueTypeParamFiles/scripts/molfile_to_params_polymer.py"

mol_file="/people/kimd999/script/python/ncaa/prepare_rotlib/1_Add_new_ResidueTypeParam/3_molfile_to_params/1_Prepare_to_run/working_example/A50/output/A50_from_ornithine_from_rosetta_I_STO_appended.mol"

python2.7 $py_file --clobber --polymer --no-pdb --name A50 $mol_file

acccording to classic rosetta document. This old method worked well and I generated a full ncAA rotamer by this classic method eventually.
However, it appears that you used
../scripts/molfile_to_params_polymer/molfile_to_params_polymer_main.py
Can I use molfile_to_params_polymer_main.py as well?
Obviously, only molfile_to_params_polymer_main.py can deal partial_charge, -i
while old molfile_to_params_polymer.py can't deal with them.
I can't find molfile_to_params_polymer_main.py in this brownbp1/ncaa_tutorial, workshop_2022's tutorial 8 and rosetta_bin_linux_2021.16.61629_bundle/main

Thanks for the reminder. I did indeed not receive a notification. Sorry about that.

As of now, I am trying to reproduce workshop_2022's tutorial 8 instead of this 'brownbp1/ncaa_tutorial' since files at that workshop are easier for me to follow step by step.
The tutorial is now out of date because of updates from Oanh Vu and Rocco Moretti that were merged into the Rosetta master branch. A graduate student in the Meiler Lab is going through and updating the tutorials, but he is still early in the process. I strongly recommend you follow the steps in 'brownbp1/ncaa_tutorial'. I made it recently, and it is a far more generic and step-by-step introduction to NCAA param generation.

Spoiler - A massive chunk of the API from that workshop (basically tutorials 5-7) will be changing over the next few months because of code refactoring to integrate multiple separate small molecule drug design projects in Rosetta.

Spoiler - A massive chunk of the API from that workshop (basically tutorials 5-7) will be changing over the next few months because of code refactoring to integrate multiple separate small molecule drug design projects in Rosetta.

Yes, I like this kind of spoiler alert~
For now, I will focus on ncAA part (e.g. tutorial 8) only. I will definitely check out the contents of the rest tutorial later.

For now, I will focus on ncAA part (e.g. tutorial 8) only. I will definitely check out the contents of the rest tutorial later.

Sounds good. Just keep in mind that if the commands are not working, it may be worth looking for an analagous procedure in the brownbp1/ncaa_tutorial set.
Best of luck.

Thanks for the reminder. I did indeed not receive a notification. Sorry about that.

As of now, I am trying to reproduce workshop_2022's tutorial 8 instead of this 'brownbp1/ncaa_tutorial' since files at that workshop are easier for me to follow step by step.

The tutorial is now out of date because of updates from Oanh Vu and Rocco Moretti that were merged into the Rosetta master branch. A graduate student in the Meiler Lab is going through and updating the tutorials, but he is still early in the process. I strongly recommend you follow the steps in 'brownbp1/ncaa_tutorial'. I made it recently, and it is a far more generic and step-by-step introduction to NCAA param generation.

I spent few months to generate ncAA rotamers with the classic method.
But BCL is a game changer!
It appears that you used "/home/brownbp1/bensbin/ncaa_tutorial_rdg/from_github/bcl/rotamer_library/ncaa_base/glycine_neutral.sdf.gz"
but I don't have this sdf file.
So here is my understanding,
First of all, prepare dipeptide only.
generate initial 3D conformer
$BCL molecule:ConformerGenerator
-explicit_aromaticity
-add_h
-generate_3D
-max_iterations 2000
-top_models 1
-ensemble_filenames $sdf
-conformers_single_file ${tag}_3d.sdf
generate dipeptide
$BCL molecule:GenerateRosettaNCAAInstructions
-generate_3D
-chirality "L_AA"
-add_h
-neutralize
-explicit_aromaticity
-input_filenames ${tag}_3d.sdf
-output_prefix ${tag}_rosetta
-extra_properties ALPHA_AA
Second of all, minimize xxx_rosetta_0.pdb by DFT
(use gaussian, NWChem etc,,,,)
Third of all, generate rosetta instructions again (because NWChem lost RosettaNCAAInstructions)
generate initial 3D conformer
$BCL molecule:ConformerGenerator
-explicit_aromaticity
-add_h
-generate_3D
-max_iterations 2000
-top_models 1
-ensemble_filenames $sdf
-conformers_single_file ${tag}_3d.sdf
generate dipeptide
$BCL molecule:GenerateRosettaNCAAInstructions
-generate_3D
-chirality "L_AA"
-add_h
-neutralize
-explicit_aromaticity
-input_filenames ${tag}_3d.sdf
-output_prefix ${tag}_rosetta
-extra_properties ALPHA_AA
Fourth, generate conformers

$BCL molecule:ConformerGenerator
-explicit_aromaticity
-conformation_comparer SymmetryRMSD 0.25
-max_iterations 2000
-top_models 50
-cluster
-ensemble_filenames
-conformers_single_file ${codename}_Rotamer.base.sdf

$BCL molecule:ConformerGenerator
-explicit_aromaticity
-conformation_comparer SymmetryRMSD 0.125
-max_iterations 500
-top_models 20
-cluster
-skip_rotamer_dihedral_sampling
-skip_bond_angle_sampling
-skip_ring_sampling
-ensemble_filenames ${codename}_Rotamer.base.sdf
-conformers_single_file ${codename}_Rotamer.sdf
Final, post processing
align to backbone (cosmetic)
$BCL molecule:AlignToScaffold $dipeptide ${codename}_Rotamer.sdf ${codename}_Rotamer.ats.sdf -explicit_aromaticity ; mv ${codename}_Rotamer.ats.sdf ${codename}_Rotamer.sdf
Add the cap atoms instruction to the rotamer file
sed -e "/M  END/r ${tag}_rosetta.RosettaInstructions.txt" ${codename}_Rotamer.sdf | awk '!f && /M  END/ {f=1;next}1' > ${codename}_Rotamer_rosetta.sdf
Make the pdb files for the molecules and the rotamer lib
rm -f ${codename}_params.log
$PYTHON $ROSETTA --clobber --polymer --all-in-one-pdb --name ${codename} -i ${codename}_Rotamer_rosetta.sdf >> ${codename}_params.log
Add PDB rotamers with full path
echo "PDB_ROTAMERS" readlink -e ${codename}_rotamer.pdb >> ${codename}.params
In this way, I could sample many rotamers

and I saw this new ncAA in my FastDesign derived designs.

bcl/rotamer_library/ncaa_base/glycine_neutral.sdf.gz

Yes you do. It's just not that exact path. It is in your BCL installation, assuming you cloned from BCLCommons GitHub page.

I spent few months to generate ncAA rotamers with the classic method. But BCL is a game changer!

I am glad you like it! If it is useful to folks, I'll try to update it sooner rather than later to include more exotic NCAAs in the automated pipeline. I'm also looking at adding a quick semi-empirical package as an external library, along with an interface, to use for the geometry optimizations. So many projects, so little time.
Best of luck.

Thanks Ben, I could find glycine_neutral.sdf.gz and use it.
Anyway, I'm afraid that your https://github.com/brownbp1/ncaa_tutorial/blob/master/scripts/quick_parmgen.sh changes DFT optimized input (geometry wise).
Based on sdf files in
https://github.com/brownbp1/ncaa_tutorial/tree/master/inputs
I think that input sdf file for quick_parmgen.sh is AA core region itself only (e.g. without dipeptide).
molecule:GenerateRosettaNCAAInstructions will conveniently add dipeptide anyway.
Therefore, I think that my above understanding (e.g. run BCL molecule:GenerateRosettaNCAAInstructions twice) is not necessary.
I can't run quick_parmgen with Ace.sdf and NMe.sdf with
=crt=bcl::chemistry=> Molecule contains unsatisfied valences; ConfScore will be very unreliable and conformers will suffer greatly in accuracy
terminate called after throwing an instance of 'std::bad_alloc'

When I ran quick_parmgen with Fmoc-5F-TRP.sdf,
it generated lots of rotamers well.
However, the distance between F and C is changed.
For example, if you do
distance #3/?:1@C9 #3/?:1@F1
(please change model # of course)
at chimeraX with Fmoc-5F-TRP.sdf,
you can see that C-F distance is 0.825 A

However, if you do
distance #1/X:1@FT1 #1/X:1@CZ2
at chimeraX with the 1st molecule of Fmoc-5F-TRP_rotamer.sdf,
you can see that C-F distance is changed to 1.346 A

I think that you designed quick_parmgen.sh to use DFT optimized input geometry sdf file.
Then, let's fix quick_parmgen.sh so that DFT optimized input geometry is maintained?
or I didn't understand something correctly?
I think that you can quickly (within 2 minutes) verify my concern.
However, here I attach my files just in case.
Fmoc-5F-TRP.zip

Anyway, I'm afraid that your https://github.com/brownbp1/ncaa_tutorial/blob/master/scripts/quick_parmgen.sh changes DFT optimized input (geometry wise).

Yeah, it definitely does. The quick_parmgen script uses BCL::Conf to generate the 3D structures and build rotamers. The QM optimization is only going to be useful if you either (A) use parent rotamers, or (B) use MakeRotLib. Otherwise, you would need to generate a conformational ensemble and DFT-optimize the whole thing.  Again, the BCL thing is a quick protocol.
The quick paramgen script is just to high-throughput quick params for mono-substituted alpha-carbon non-canonical amino acids using a cheminformatics small molecule conformer ensemble generator to build rotamers (https://pubs.acs.org/doi/10.1021/acs.jcim.0c01140 and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4607025/).

Also, from the images you sent, it looks like you are comparing distances from a 2D structure without hydrogen atoms and a 3D structure, which seems odd. Is there a reason for this that I am not understanding?

I can't run quick_parmgen with Ace.sdf and NMe.sdf with

Yeah, these do not fall within the narrow scope of application of the molecule:GenerateRosettaNCAAInstructions application, which currently just works for mono-substituted alpha-carbon non-canonical amino acids. For these or other such NCAAs, you should follow the manual protocol in brownbp1/ncaa_tutorial.

Anyway, I'm afraid that your https://github.com/brownbp1/ncaa_tutorial/blob/master/scripts/quick_parmgen.sh changes DFT optimized input (geometry wise).

Yeah, it definitely does. The quick_parmgen script uses BCL::Conf to generate the 3D structures and build rotamers. The QM optimization is only going to be useful if you either (A) use parent rotamers, or (B) use MakeRotLib. Otherwise, you would need to generate a conformational ensemble and DFT-optimize the whole thing. Again, the BCL thing is a quick protocol.
The quick paramgen script is just to high-throughput quick params for mono-substituted alpha-carbon non-canonical amino acids using a cheminformatics small molecule conformer ensemble generator to build rotamers (https://pubs.acs.org/doi/10.1021/acs.jcim.0c01140 and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4607025/).

Yes, I think that you wrote somewhere that using a parent rotamer is useful (although quick) only for certain limited cases. I ran MakeRotLib for other ncAA rotamer generation once, but I had to spend too much time with many manual processes.
Therefore. I prefer to optimize quick_parmgen_before_NWChem derived rotamer pdb file with QM method.
Not all 762 rotamers of course (I plan to choose 100 random rotamers from this pdb file).
With NWChem, DFT 6-311G** based geometry optimization took 1 hr per each dipeptide (e.g. 1 ncAA core + N-term + C-term). Therefore, I think that this is manageable.

Also, from the images you sent, it looks like you are comparing distances from a 2D structure without hydrogen atoms and a 3D structure, which seems odd. Is there a reason for this that I am not understanding?

Yes, that may seem odd. However, since what I care is a distance between Carbon and fluorine, I think that this is fine.
When I use your Fmoc-5F-TRP.sdf, I think that add_h option in ConformerGenerator/GenerateRosettaNCAAInstructions adds hydrogens. Of course, for final rotamer preparation (e.g. DFT optimization after quick_parmgen), I will use hydrogen added one.

manual protocol in brownbp1/ncaa_tutorial

hm,, I don't know how to run manual protocol in brownbp1/ncaa_tutorial... even after I look at its files...

I can't run quick_parmgen with Ace.sdf and NMe.sdf with

Yeah, these do not fall within the narrow scope of application of the molecule:GenerateRosettaNCAAInstructions application, which currently just works for mono-substituted alpha-carbon non-canonical amino acids. For these or other such NCAAs, you should follow the manual protocol in brownbp1/ncaa_tutorial.

hm,, I don't know how to run manual protocol in brownbp1/ncaa_tutorial... even after I look at its files...

@brownbp1 I learned that quick_parmgen.sh clips full dipeptide.
Here is a detail.
I entered core ncAA part without full dipeptide as

(here 5-fluoro tryptophan is a core ncAA part as you can see).
Then, quick_parmgen.sh generates full dipeptide and keeps full dipeptide as

well.
However,
PYTHON=/usr/bin/python2.7

ROSETTA=/people/kimd999/bin/rosetta.source.release-331/main/source/scripts/python/public/molfile_to_params_polymer.py

$PYTHON $ROSETTA --clobber --polymer --all-in-one-pdb --name ${codename} -i ${codename}_Rotamer_rosetta.sdf >> ${codename}_params.log

step clips full dipeptide.
Do you use this kind of clipped full dipeptide as rotamer.pdb ?
When I use this kind of clipped dipeptide as final rotamer pdb that is specified
by PDB_ROTAMERS in .params file, I see weird hydrogen geometry as


[dreamkast-releasebot] Automatic Release

Automatic Release

Review app
https://dreamkast-dk-1017.dev.cloudnativedays.jp


[$500] Web - Workflows - After the page refreshes, the switch returns to the default value

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: 1.4.44-7
Reproducible in staging?: Y
Reproducible in production?: N
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers<EMAIL_ADDRESS>Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
Issue found when executing PR https://github.com/Expensify/App/pull/36345
Action Performed:

Go to URL https://staging.new.expensify.com/
Log in to account with the Collect policy (Admin or employee)
Navigate to Settings > Workspaces. You should see your Collect workspace, click on it
On the LHP, you should only see the menu items below, specific to paid policies
Click on Workflows
Click on Delay Submissions toggle
Refresh the page

Expected Result:
The switch for Delay Submissions should be on
Actual Result:
The switch for Delay Submissions returns to the default value
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

Screenshots/Videos
Add any screenshot/video evidence

https://github.com/Expensify/App/assets/78819774/76977da7-4d98-4b78-a431-e1adcafb6b16
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0113a3783dd1a2d250
Upwork Job ID:<PHONE_NUMBER>414375936
Last Price Increase: 2024-02-27

Can not reproduce in MacOS: Chrome / Safari, dev environment (enabled staging server)

Can't reproduce 🤔

I'm able to repro, but I don't think it needs to be a blocker. @lakchote FYI this is likely coming from your PR.

@dukenv0307 thanks for your proposal I've already implemented the fix here  and used the same solution.
@puneetlath @dukenv0307 should be eligible for a partial payment for their solution, thank you.

@mollfpr requires payment (Needs manual offer from BZ)

Not eligible for payment, since I'm not reviewing the PR.

@dukenv0307 sending partial offer. Lmk when you accept

@kevinksullivan should we close this?

@kevinksullivan should we close this?

friendly bump :)

The contributor has been paid (see here). Closing the issue.


Annotate types with validity state

We have a number of types that want to preserve invariants. We can avoid constructing invalid instances of these types by hiding the constructors and only exposing safe constructors that we can test enforce the invariants.
This works for most cases, but decoding from the binary format presents a couple of issues. Performing checks during decoding can be expensive and cause significant slow downs. In many cases the checks require external information not available during decoding anyway, so we have to construct unvalidated values in those cases.
So to get around this problem we want to encode the validity in the types. There are a couple of techniques for doing this.
One is to add an UnvalidatedFoo newtype-wrapper around a Foo. We can then construct safe Foos as normal, but not decode them. Instead we decode an UnvalidatedFoo, that we can only turn in to a Foo through the validation function validateFoo :: UnvalidatedFoo -> Foo.
The other is through a phantom type, so we change Foo to Foo (v :: Validity) for some kind data Validity = Valid | Unvalidated. This would allow us to write functions that are polymorphic over the validity state. We could also do data FooV (v :: Validity) = ... and type Foo = FooV Valid. So that all the existing references to Foo that we thread a FooV Valid to don't have to change at all, but are now safer.
Both of the depend on a splitting of the Bi class into ToCBOR and FromCBOR. We would want ToCBOR Foo and FromCBOR UnvalidatedFoo, in the first case, and ToCBOR (Foo Valid) and FromCBOR (Foo Unvalidated) in the second case.

This sounds like an interesting issue. I don't have enough domain knowledge to see why would you store some data in a type you want to have some properties, but to actually check these properties after already having data in this type. I'd be interested to chat about this to understand the issue at hand better.

Probably won't end up doing this for now

But don't we want to do this at all?


Add Stringable caster

see https://github.com/laravel/tinker/pull/121

Needs to be if-wrapped. Since it works with "all" versions of tinker, when you use <2.6.1 this class might not exist?

@emargareten Thanks for your contribution. I made a small change to this so we know sure that it keeps working for people not updating their tinker installments to 2.6.1. I will add this to the 2.2.2 release :)


[zeromq] Building package zeromq:x86-windows failed with: BUILD_FAILED

OS: Windows 10
When trying to install the zeromq package the build fails with the following console output:
C:\DM\vcpkg-2019.10>.\vcpkg install zeromq
The following packages will be built and installed:
zeromq[core]:x86-windows
Starting package 1/1: zeromq:x86-windows
Building package zeromq[core]:x86-windows...
-- Using cached C:/DM/vcpkg-2019.10/downloads/zeromq-libzmq-8d34332ff2301607df0fc9971a2fbe903c0feb7c.tar.gz
-- Using source at C:/DM/vcpkg-2019.10/buildtrees/zeromq/src/903c0feb7c-34456404d0
-- Configuring x86-windows
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:72 (message):
Command failed: ninja -v
Working Directory: C:/DM/vcpkg-2019.10/buildtrees/zeromq/x86-windows-rel/vcpkg-parallel-configure
Error code: 1
See logs for more information:
C:\DM\vcpkg-2019.10\buildtrees\zeromq\config-x86-windows-out.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_configure_cmake.cmake:296 (vcpkg_execute_required_process)
ports/zeromq/portfile.cmake:22 (vcpkg_configure_cmake)
scripts/ports.cmake:94 (include)
Error: Building package zeromq:x86-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: zeromq:x86-windows
Vcpkg version: 2019.09.12-nohash
config-x86-windows-out.log
I cloned the "vcpkg-master" file which gave me this issue. So I downloaded the newest version I could find "vcpkg-2019.10" and the build still failed.

Error:
CMake Error at CMakeLists.txt:2 (project):
  No CMAKE_C_COMPILER could be found.

Please provide your Visual Studio version.
Thanks.

I was originally using Visual Studio 2019 but that was giving me this same problem. So I reverted back to Visual Studio 2017 version 15.0.0. I only have this version installed on my machine. Both gave me the same issue.

I noticed a needs-repro tag was added. Is more clarification needed from my end?

@mawdis2 Yes, I still cannot repro it, and I have no idea where goes wrong, we need a discussion before you provide more information.

@JackBoosY  These are the exact steps I took to get to this point. I hope this helps:

Download and unzip "vcpkg-2019.06.zip" from here:(https://github.com/microsoft/vcpkg/releases/tag/2019.06 )

Run the command prompt and CD to vcpkg-2019.06

Build vcpkg using: .\bootstrap-vcpkg.bat (Build was successful)

Apply user-wide integration: .\vcpkg integrate install

Attempt to install zeromq: .\vcpkg install zeromq (Build Fails here)

The build fails with the following cmd output:

I then made sure that the packages were all up to date: .\vcpkg update (No packages needed updates)

config-x86-windows-out.log

@mawdis2 Could you use powershell instead of Visual Stdio command prompt and try again?
Thanks.

@JackBoosY I still get the same error using powershell:

@vicroms I don't know what's wrong here, it should recognize the compiler.

@mawdis2 Could you update vcpkg and try again?
Is there anything special about your environment?

We haven't been able to repro this; if more information comes up, or this issue appears again, please reopen.


Integration with iNaturalist

iNaturalist has a significant amount of presence data for many marine species within Acartia's geographic domain (SRKW range and the Salish Sea). For example these are killer whale locations across the NE Pacific:

Would it be possible to ingest these data within Acartia for resiliency and easier programatic access, including regional dashboards and analyses?
Would iNaturalist like to ingest Acartia data?
What collaborations are possible with iNaturalist projects like these?

https://www.inaturalist.org/projects/salish-sea-biodiversity
https://www.inaturalist.org/projects/marine-life-of-the-salish-sea (started 2015)
https://www.inaturalist.org/projects/the-great-salish-sea-bioblitz (July 2020)
https://www.inaturalist.org/projects/marine-biodiversity-of-the-pacific-northwest

[ ] Contact iNaturalist about ingesting and contributing SRKW/other Regional Data?

[ ] Archived marine mammal data?
[ ] Regular xfer of all marine species?

[ ] How close to real-time is possible?
[ ] Why did Eli have such a positive experience?

I don't believe iNaturalist wants to ingest Acartia data in bulk, but I can certainly imagine Acartia users wanting to post specific observations to iNaturalist for confirmation or posterity.
As far as pulling data from iNaturalist, that shouldn't be a problem. If you're interested in just a few species in the Pacific Northwest, we can probably pull that quite frequently. Data are also exported to GBIF every 1-2 weeks, which would be a good source for backfill data. Either way, observations are often licensed for use with attribution, sometimes for only non-commercial use.
For some sense of volume:

Observations of Whippomorpha in the Salish Sea: 3,678 all time, 769 in last 12 months
Observations of Chromista in same bounding box: 24,608 all time, 6,745 last 12 months
Observations of Actinopterygii in same bounding box: 32,363 all time, 7,624 last 12 months

Hopefully that gives some broad sense for what's possible. To dig in, it would be good to have specific ideas of motivations for the integration. Perhaps:

to backfill observation data, partially mitigating a start problem for Acartia
to backfill information about species, including photos
to get a sense for historical and seasonal trends
to have timely awareness of ecological events: the start of fish runs, e.g.

What are your goals?

Thanks for the volumetric estimates, @rainhead ! As of today, Acartia holds about 33k records, so it seems to me a backfill with iNaturalist data would be a significant increase (~tripling) but not unacceptable.
The long-term, over-arching goal is to provide open programmatic access to marine animal location data with clear governance and provenance, despite the exploding number of data sources (e.g. Facebook/WhatsApp/Discord/etc groups, plus more and more nature observation apps). Acartia partners have started with real time applications in mind (e.g. SRKW risk mitigation schemes) and visualizing/modeling movement of cetaceans.
But cooperatives like the Puget Sound Ecosystem Monitoring Program have troves of historic data that are in disparate data bases and disk drives... A best example that I know of is (Joe Evenson's WDFW aerial survey data](https://wdfw.wa.gov/species-habitats/at-risk/species-recovery/seabirds/surveys-winter-aerial) which I think also gets submitted to GBIF.
So yes, backfilling as a way to initiate the process of aggregating historic data is a short-term goal for iNaturalist (or just GBIF?) data exchange(s). Once we have enough historic data, folks like @liu-zoe have already started building tools for looking at spatio-temporal trends.
In fact, @liu-zoe , Peter's helped me realize that a bonus of the orca-salmon visualization could be to add adult Chinook data points to Acartia as you scrape them from the Albion and Bonneville web tables!
Peter, do you think it makes most sense to backfill from GBIF the points with licensing that allows incorporation into Acartia's CC-BY-SA records? I wonder how much your estimates from iNaturalist would increase if the same bounding box was used with GBIF marine data (or OBIS-SEAMAP?)...

I couldn't search GBIF for all Whippomorpha, but the occurrence data for Delphinidae in approximately that bounding box are contributed by these data sets:

iNaturalist Research-grade Observations 1,149
Galiano Island BC Canada Marine Zoology 1893–2021 431
Happywhale - Killer Whale in North Pacific Ocean 349
Sanctuary Soundscape Monitoring Project (SanctSound) Daily Aggregated Species Detections 294
Observation.org, Nature data from around the World 177
MMS Marine Mammal Survey, PNW 1989-1990 141
SWFSC Marine Mammal Survey, CCES 2018, Cruise 1651 83
SWFSC Marine Mammal Survey, CSCAPE 2005, Cruise 1627 65
NMNH Extant Specimen Records (USNM, US) 41
UWBM Mammalogy Collection (Arctos) 16

Any GBIF data should be OK to use in Acartia, with attribution as required. Attribution might require changes to the observation popover, and perhaps to other screens.
iNat, Happywhale, and observation.org data probably make sense to just merge with Acartia data. Other historical datasets, with observations just in particular time frames or regions, might be confusing to the user without changing the UI to orient around the datasets themselves. That doesn't seem worthwhile at this point, unless you think they tell some story that would be particularly valuable to Acartia's users.

@liu-zoe You've motivated a valuable new question for me: where should code to scrape & aggregate data for a local ecoregion live?
In the long run, it seems like it should live close to the data cooperative (e.g. in the acartia repository or salish-sea organization), rather than in an application like the orca-salmon dashboard you've pioneered. That way, not only could your initial application access the salmon time series, but so could any future application (like the game Val @veirs imagined where humans and algorithms try to predict where SRKWs will go tomorrow).
Perhaps when you migrate your repo to the salish-sea organization we could discuss such architectural strategies, along with how new ecological visualization apps should be deployed? @aalaydrus is currently working on how/when to migrate the Acartia deployment from Digital Ocean to Orcasound's AWS account. I wonder if a similar devops approach could work for your long-term deployment of your app and/or scraping scripts, and any integration code @rainhead might consider for iNaturalist, GBIF, or other overarching systems... or if there is a more distributed approach that would be smarter (i.e. the Acartia app deployed as a peer manages scraping from centralized sites and then distributed storage via OrbitDB).

@rainhead Thanks for the Delphinidae data size computations! It's both exciting and daunting to see totally novel data sources for a group I know well. (I didn't know that killer whale observations were coming in through Happy Whale! Or that the Burke had so many marine mammal specimens, including a Steller sea cow!)
My gut says it's worth pulling historic data from the GBIF source you listed in to Acartia and building exploratory, analytic, visualization, and story-telling tools around the aggregated local species occurrence, track, and/or count data sets, with ecological context coming in from other feeds (e.g. oceanographic conditions, stream flow gauges) -- at least historic, but eventually also real time and predictions. But a revision of the Acartia data scheme is almost certainly in order before doing so... (You've gotten me reading more about the Darwin Core standard (Wieczorek et al. 2012) and using the World Register of Marine Species (WoRMS) data for local classifications.)
An assumption motivating such an effort is that it would enable almanac-like perspective on news-worthy events in the Salish Sea ecosystem. While I'm still exploring the existing data summary and mapping tools within the iNaturalist or GBIF web sites...

...I'm also noticing that historical context feels valuable and meaningful when reporting on or interpreting a real time event. A good example is this After the Breach podcast from June 2023 in which they discuss both historic sightings of a pair of male Bigg's killer whales to contextualize a recent observation of unusual behaviors. I have similar questions about the T65A matriline that has been scouring Puget Sound this last week (most of the southernmost locations in the current view of Acartia data, first in south Puget Sound a week ago, yesterday into Hood Canal and there still this morning):

When were they previously sighted within Puget Sound or sub-regions of it?
How long did they stay in the region previously compared to this time?
What were their movement patterns the last time(s) they were here?
How do those patterns relate to known seal and sea lion haulout sites and/or historic occurrence data for their other favorite prey item: harbor porpoises?

What if the goal of a new open source analytic tool -- the Salish Sea almanac? -- was to monitor and model the present ecosystem, detect emerging or upcoming events (real time or seasonal/historic), and generate open data products like stories that are programmatically accessible to other applications? It could have a dashboard with a distillation of overall conditions and daily/seasonal events (a bit like the Old Farmer's Almanac or The Writer's Almanac), but also an API to access data, plots, media...
As a practical example, I was trying to amend my morning new ritual with iNaturalist's iOS mobile app last week. I got close to a way to monitor my local watershed (Ravenna Ravine/Creek) as a way to decide whether to take a walk there with the dog. I found a local project...

...saw that one observer had seen a bunch of species there that I don't yet know how to ID...

...noticed that there was a News tab for the local group...

...and was excited to see a new post just 19 hours old, but then was disappointed to see that it was just a site-wide post not relevant to my local/walkable environment and therefore not helpful in guiding my day's activities or providing insights and meaning about my home ecology.

What I was hoping for was that local expert to have shared some exciting emerging event they had witnessed, or an upcoming event they were excited to see soon.
What do you (all) think?

What I was hoping for was that local expert to have shared some exciting emerging event they had witnessed, or an upcoming event they were excited to see soon.

I don't engage with projects on the app—or much, in general. They are mostly ways of aggregating observations, e.g. to collect data as part of surveys run by organizations. Projects will very occasionally keep their own journal. The iOS app tab appears to be in error, perhaps vestigial, see https://github.com/inaturalist/INaturalistIOS/issues/654. The app is in the process of being replaced by https://github.com/inaturalist/iNaturalistReactNative, and receives minimal attention.
iNaturalist users can also post to a personal journal. Most people don't, but when experts do, they can be quite insightful. The posts are not particularly discoverable or integrated into the site, not even into site search. Knowledge is mostly shared between people and communities by way of comments on observations and identifications, and through private messages. This severely limits the spread of knowledge, and risks creating games of "telephone", as "tips and tricks" can't easily be traced back to subject authorities. I would love to create better opportunities for community members to learn about taxa from one another (read: better opportunities to create community), but the best ways to do so would be integrated into iNaturalist itself.

What if the goal of a new open source analytic tool -- the Salish Sea almanac? -- was to monitor and model the present ecosystem, detect emerging or upcoming events (real time or seasonal/historic), and generate open data products like stories that are programmatically accessible to other applications?

A cheap, flexible way to prototype something like this would be to set up some Jupyter notebooks on AWS and point them at the GBIF data already in S3. I've done this kind of thing only a little. Do you have experience with Jupyter, Pandas, Spark, or other relevant tools?

Going back to your questions:

When were they previously sighted within Puget Sound or sub-regions of it?
How long did they stay in the region previously compared to this time?
What were their movement patterns the last time(s) they were here?
How do those patterns relate to known seal and sea lion haulout sites and/or historic occurrence data for their other favorite prey item: harbor porpoises?

Where are the data to answer these questions right now? Is there anything besides Acartia with the necessary granularity of data? How would answers to these questions fit into a larger information landscape, and especially, who besides you has these questions? Perhaps meeting an existing audience where they are means answering fewer, more basic questions, with mostly static answers?
Birds are the only other organisms I can think of with similar requirements for study. They are tracked with radar, people observe them with at least day-to-day time granularity, they're charasmatic enough that serious investment goes into data tracking and sites and apps for birding. Cornell Ornithology provides a good starting place for inspiration.


codeclimate support

http://blog.codeclimate.com/blog/2015/07/07/build-your-own-codeclimate-engine/
https://github.com/codeclimate/spec/blob/master/SPEC.md

see https://github.com/tkqubo/codeclimate-tslint

will work towards extending abaplint.org instead


how to replace file

I use webpack for help me build a isomorphic javascript app.
I have a problem , I have two module :

'./api/client/api.js'
'./api/server/api.js'

because the server and client have different get data method, so I use

var api  = require('./api/server/api.js')

but when webpack use same file build the client js I want use './api/client/api.js' for replace, is there have any setting like

{
"./api/server/api.js":"./api/client/api.js"
}

to solve my problem?
thanks

Will alias solve your problem?
http://webpack.github.io/docs/configuration.html#resolve-alias

In coding phase, I am not sure we could use ../xxx/ path as alias name.
Sokra suggest using NormalModuleReplacementPlugin
see https://github.com/webpack/webpack/issues/1090

@pwang2  Yes, I think NormalModuleReplacementPlugin can solve my problem,thanks


Classes added to  elements not persisting to compiled th.

Will submit a PR shortly.

Were you using the latest version of the module when you tested this? As of version 0.8.7 any attributes you place on the th element should be preserved. In fact, I just tested adding a class to the th element and it is still there.


vetted_guardians input format unclear in GuardianUI

The correct input format of vetted_guardians in GuardianUI meta information setup is unclear. An example for users on how to correctly enter multiple gateways would be useful.
E.g. vetted_gateways: Eine Liste von Gateway-Identifikatoren, die von der Föderation geprüft wurden. Beispiel: ["gateway1", "gateway2"]

I feel like eventually we should have specialized input forms for all standardized meta fields. For now additional help texts sound useful though.


Create offline install USB stick

Create USB stick with everything listed in setup section
Windows

[x] Visual Studio 2017 Community Preview
[x] Visual Studio Code
[x] dotnet SDK

Mac

[x] Visual Studio for Mac
[x] Visual Studio Code
[x] dotnet SDK

Done


chore: bdd-spring-1572277693 to 0.0.1

chore: Promote bdd-spring-1572277693 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


CRI: Enrich with initial PodLabels

CRI: Enrich with initial PodLabels
CRI gives us the option to query the pod labels. This information is in the PodSandbox, but unfortunately this only gives us the initial pod labels. So the labels with which the pod was started. Updated labels are not possible with only reading PodSandbox
Since this information is only available through CRI, I introduced a switch (currently the flag is still todo), so the ContainerRuntimeEnrichment can be used with CRI only (instead of docker or containerd API).
This has the advantage that we are saving API calls, but would mean if one wants to see K8s Pod Labels with ig that non K8s containers will not be captured. This is an open question.
The other option is implemented in this branch: https://github.com/inspektor-gadget/inspektor-gadget/tree/burak/pod_label_2
Use with flag --use-cri for ig:
$ ig trace exec -o columns=k8s.pod,k8s.labels,runtime.containerName,comm --use-cri
K8S.POD                                                                K8S.LABELS                           RUNTIME.CONTAINERNAME                                                 COMM
test-pod                                                               foo=bar,abc=xyz                      test-pod                                                              date
gadget-9tvcn                                                           controller-revision-hash=74d5bc749,… gadget                                                                gadgettracerman
gadget-9tvcn                                                           pod-template-generation=1,k8s-app=g… gadget                                                                gadgettracerman
test-pod                                                               foo=bar,abc=xyz                      test-pod                                                              date
test-pod                                                               foo=bar,abc=xyz                      test-pod                                                              date
test-pod                                                               foo=bar,abc=xyz                      test-pod                                                              date
test-pod                                                               foo=bar,abc=xyz                      test-pod                                                              date

TODO

[ ] Use CRI only or additionally
[x] Introduce flag for ContainerRuntimeEnrichment, to use CRI
[ ] Introduce flag for specifying the path to the cri docker shim socket

This has the advantage that we are saving API calls, but would mean if one wants to see K8s Pod Labels with ig that non K8s containers will not be captured. This is an open question.

The labels on the pod or sandbox and the labels at the OCI level are not related. We only need the former. And the former only makes sense on Kubernetes, so I think it is fine.

This has the advantage that we are saving API calls, but would mean if one wants to see K8s Pod Labels with ig that non K8s containers will not be captured. This is an open question.

The labels on the pod or sandbox and the labels at the OCI level are not related. We only need the former. And the former only makes sense on Kubernetes, so I think it is fine.

Okay, so its okay to loose that capability when wanting K8s pod labels (foobar won't be shown):
$ ig trace exec -o columns=k8s.pod,k8s.labels,runtime.containerName,comm --use-cri
K8S.POD                     K8S.LABELS                           RUNTIME.CONTAINERNAME                                                 COMM
test-pod                    foo=bar,x=y                          k8s_test-pod_test-pod_default_0f42539f-a584-482c-a45d-0e9e41686fa7_0  date
                                                                 foobar                                                                date

Testing with Minikube and Docker. My pod has 2 labels:
$ kubectl get pod -n kube-system storage-provisioner -o yaml
apiVersion: v1
kind: Pod
metadata:
  labels:
    addonmanager.kubernetes.io/mode: Reconcile
    integration-test: storage-provisioner
...

But ig finds more labels:
$ sudo -E /opt/ig list-containers -o json --use-cri
...
    "k8s": {
      "namespace": "kube-system",
      "podName": "storage-provisioner",
      "podLabels": {
        "addonmanager.kubernetes.io/mode": "Reconcile",
        "integration-test": "storage-provisioner",
        "io.kubernetes.pod.name": "storage-provisioner",
        "io.kubernetes.pod.namespace": "kube-system",
        "io.kubernetes.pod.uid": "c6f5db9b-f991-4314-9fcf-55860bd32ec0"
      },
      "containerName": "storage-provisioner",
      "podUID": "c6f5db9b-f991-4314-9fcf-55860bd32ec0"
    },
...

It seems that cri-dockerd merges the labels from the Kubernetes pod and other labels:
https://github.com/Mirantis/cri-dockerd/blob/519e39ceaa7f9e00319149b9d74b243466fa3963/config/constants.go#L27-L30
	KubernetesPodNameLabel       = "io.kubernetes.pod.name"
	KubernetesPodNamespaceLabel  = "io.kubernetes.pod.namespace"
	KubernetesPodUIDLabel        = "io.kubernetes.pod.uid"
	KubernetesContainerNameLabel = "io.kubernetes.container.name"

Note that only the first three labels (name, namespace, uid) are visible. The 4th one (name) is added too but it is not visible from the CRI API because it is filtered out:
https://github.com/Mirantis/cri-dockerd/blob/master/core/helpers.go#L85C18-L85C46
I think Inspektor Gadget should not report those synthetic labels that are not actually defined on the pod resource.

I rebased on main and fixed the conflicts.

I see the CI has the following failures:
On Test ig w k8s (containerd):
=== NAME  TestEnrichmentPodLabelNewPod
actual event:
{
   "container" : {
      "k8s" : {
         "containerName" : "test-enrichment-pod-label-new-pod",
         "namespace" : "test-enrichment-pod-label-new-pod-2280675184725679420",
         "podLabels" : {
            "run" : "test-enrichment-pod-label-new-pod"
         },
         "podName" : "test-enrichment-pod-label-new-pod"
      },
      "runtime" : {
         "containerImageName" : "docker.io/library/busybox:latest",
         "runtimeName" : "containerd"
      }
   },
   "event" : "CREATED"
}

expected event:
{
   "container" : {
      "k8s" : {
         "containerName" : "test-enrichment-pod-label-new-pod",
         "namespace" : "test-enrichment-pod-label-new-pod-2280675184725679420",
         "podLabels" : {
            "run" : "test-enrichment-pod-label-new-pod"
         },
         "podName" : "test-enrichment-pod-label-new-pod"
      },
      "runtime" : {
         "containerImageName" : "docker.io/library/busybox:latest",
         "containerName" : "test-enrichment-pod-label-new-pod",
         "runtimeName" : "containerd"
      }
   },
   "event" : "CREATED"
}

I don't know why container.runtime.containerName is missing.
On Test ig w k8s (cri-o):

FAIL: TestTraceTcpconnect_latency: we need to add the podLabels in the expected event
FAIL: TestTraceNetwork: we need to add the podLabels in the expected event
FAIL: TestTraceOOMKill: podLabels is expected but it is not found

I rebased on main and fixed the conflicts.
I see the CI has the following failures:
On Test ig w k8s (containerd):
=== NAME TestEnrichmentPodLabelNewPod
[...]
I don't know why container.runtime.containerName is missing.

The container name is not there because of this:
https://github.com/inspektor-gadget/inspektor-gadget/blob/692306499f2bfdfae49bb865a77e738d0c6423b7/pkg/container-collection/options.go#L100-L103
I don't know why it's returning only if the k8s metadata is enriched. It still needs to enrich the runtime metadata in the below code.
On the other side, I still don't understand the need of this block. Is it because CRI-O might get nothing in this GetContainer call? If so, why it should get the PodSandbox but nothing in the GetContainer()? Do we have a deadlock with CRI-O that I'm not aware of?

After several tests, things seem more clear.
For new CRI-O containers:

The container-hook provides the PID, ContainerID and ContainerName when notifying the new CRI-O container. NOTE: This could extended to provide the runtime name so that we avoid trying all the runtimes.
The OCIConfig enricher enriches the container with RuntimeName, ContainerImageName, SandboxId and all the k8s information except the PodLabers: Namespace, PodName, ContainerName and PodUID.
When the ContainerRuntimeEnricher is called, the container doesn't exist yet for CRI-O. So, this GetContainer call will fail. However, the PodSandbox indeed exist. That's why using the PodSanboxID, this GetPodLabels call will succeed.

Said that, I got the answers to my questions:

On the other side, I still don't understand the need of this block. Is it because CRI-O might get nothing in this GetContainer call? If so, why it should get the PodSandbox but nothing in the GetContainer()? Do we have a deadlock with CRI-O that I'm not aware of?

As said in (3), this block is indeed useful. However, I suggest applying it only for CRI-O as for other runtimes the GetContainer call will succeed and it will enrich the container with the PodLabels and other missing fields, e.g., ContainerImageDigest:
	// For new CRI-O containers, the next GetContainer call will always fail
	// because the container doesn't exist yet. So, Let's enrich the container
	// at least with the PodLabels as the PodSandbox do exist at this point.
	if container.SandboxId != "" && (container.Runtime.RuntimeName == "" || container.Runtime.RuntimeName == types.RuntimeNameCrio) {

I don't know why it's returning only if the k8s metadata is enriched. It still needs to enrich the runtime metadata in the below code.

This block needs to be indeed removed or modified. At most, it can return for CRI-O (we know the next GetContainer will fail) and let it go through for other runtimes.

Hi @alban, I fixed the CI and managed the comments. The only open discussions are:

The flag. This should be addressed in this PR.
The dependency between the OCI Config and Runtime enrichers, see below. IMO, we could improve this in another PR.
For the initial containers, the number of gRPC calls toward the runtime increased from 2 to 5 for each container when using the CRI. It means for CRI-O or for any runtime if --use-cri is specified. IMO, we could improve this in another PR.

I summarized in this table how SandboxID and PodLabels are now enriched. Notice that the SandboxID is used only internally in IG to compute the PodLabels in a special case, while the PodLabels field is now being added to the gadgets events within the k8s struct:

Initial Containers
New Containers

CRI-O
Containerd
Docker
Podman
CRI-O
Containerd
Docker
Podman

SandboxID
X
X
X
N/A
OCI Config Annotations
OCI Config Annotations
X
N/A

k8s.PodLabels
CRI-Runtime GetContainers()
X
X
N/A
CRI-Runtime GetPodLabels(k8s.SandboxID)*
X
X
N/A

k8s.PodLabels (--use-cri)
CRI-Runtime GetContainers()
CRI-Runtime GetContainers()
CRI-Runtime GetContainers()
N/A
CRI-Runtime GetPodLabels(k8s.SandboxID)*
CRI-Runtime GetContainer()
CRI-Runtime GetContainer()
N/A

*: New CRI-O containers are a special case for two reasons:

The ContainerRuntime enricher requires that the container be already enriched with the SandboxID, which is enriched by the OCI Config enricher. Currently, based on the order the enrichers are added to the container collection, it's known that the OCI Config enricher will be always called before the ContainerRuntime enricher. However, a better mechanism should be found to avoid such a dependency.
We can't use the normal GetContainer call because, by the time the enricher is called, the container doesn't exist yet in CRI-O. So, we implemented a method to enrich the pod labels as the Sandbox already exists.

Thanks @blanquicet !
I rebased on main again and I will try to implement --runtime-protocol=internal|cri.

# ./ig trace exec --runtime-protocol=cri -o columns=runtime.containerName,pid,comm,ret,k8s.labels
RUNTIME.CONTAINERNAME PID    COMM RET K8S.LABELS
shell8                766439 cat  0   run=shell8

IMO, we are ready to go. I didn't test the new flag locally but the tests we have for it are covering most cases, so I trust them.

Thanks. I updated the PR, and I will merge it once the CI passes.

If you agree, I can create the issues to track the discussions we didn't resolve in this PR: https://github.com/inspektor-gadget/inspektor-gadget/pull/2398#issuecomment-1950260439

Yes, please.

Issues to track missing tasks:

https://github.com/inspektor-gadget/inspektor-gadget/issues/2523
https://github.com/inspektor-gadget/inspektor-gadget/issues/2524
https://github.com/inspektor-gadget/inspektor-gadget/issues/2525


DOI field can't be blank.

If a user adds a DOI to a study and then tries to clean it the system will not allow an empty string but 1 space is fine. We should allow to add empty strings.
cc @ackermag

What about DOI not referring to DOI?
On Mon, Nov 7, 2016 at 2:13 PM, Yoshiki Vázquez Baeza <
<EMAIL_ADDRESS>wrote:

Closed #1975 https://github.com/biocore/qiita/issues/1975 via 2ab49dc
https://github.com/biocore/qiita/commit/2ab49dc6f4cf5501424f4d250941f2e70e8c19cf
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/biocore/qiita/issues/1975#event-850653386, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AB69gKpmJegrY4RKLmAQpdSUcvqsq3Gkks5q76KNgaJpZM4Kemys
.

--
Gail Ackermann
Knight Lab
UCSD
<EMAIL_ADDRESS><EMAIL_ADDRESS>


Changed event type for greetings.yml

Codecov Report

Merging #8 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #8   +/-   ##
=======================================
  Coverage   79.79%   79.79%
=======================================
  Files          78       78
  Lines        9560     9560
  Branches     1218     1218
=======================================
  Hits         7628     7628
  Misses       1876     1876
  Partials       56       56

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 38331fb...5655b34. Read the comment docs.

The greeting thing seems to be stuck, might be worth it just removing it

Looks like the reason this was getting stuck is because the secret was not set, but im fine with deleting it


Truly excellent documentation

Hey there, apologies if this is not what feedback is meant for - but just wanted to say this was one of the best if not the best pages of documentation I've read, in terms of how much my level of understanding of the topic changed from start to finish.
I was asked about asynchronous programming in C# the other day and after reading just this article I feel more than ready to start implementing async and await and playing around with them. Starting with the simple and real life example, then progressing from traditional synchronous code to well optimised asynchronous code, was excellently done.
High praise for whichever individual or team put this together - thanks again!
Jason

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 17d27a0d-c072-8c9f-7e31-3de60bee9ae8
Version Independent ID: e0c09794-e712-19ff-cb9f-12648d60d4f6
Content: Asynchronous programming in C#
Content Source: docs/csharp/programming-guide/concepts/async/index.md
Product: dotnet-csharp
Technology: csharp-async
GitHub Login: @BillWagner
Microsoft Alias: wiwagn

Thank you for the kind words @JasonSanderEgan We appreciate it!  I'm glad this example resonated and helped understand async and await.
I'll close this as kudos.
Thanks again! 🎆


Add support for timestamps as DEFAULT constraints

Part of the fix for https://github.com/rqlite/rqlite/issues/1072

@otoolep Would you be interested in just taking over the repo? I can transfer it to your username.

@benbjohnson -- sure I plan to continue work on this, for the purposes of rqlite support.

@otoolep Sorry for the delay. It looks like you need to delete your repo first and then I can transfer it.

@benbjohnson -- I moved is repo out of the way (renamed to sql-2). Want to try again?

@otoolep I'm getting this error now:

"rqlite already has a repository in the benbjohnson/sql network and You don’t have the permission to create public repositories on rqlite"

It might be easier if I just delete my repo? :)
Otherwise you'll need to delete your sql-2 repo and temporarily add me as an owner/collaborator on the rqlite repo.


Integrationtest "test_key_rotation" starts to fail beginning from keycloak version 18

The step about this tests needs to be checked why the integrationtest fails.
Probably only the tests needs a little bit modified for this keycloak version ... or the feature about key rotation in keycloak is not working.
Investigation needs to be done to ensure that the feature about key rotation does not break with this keycloak version

Now it is clear what is the problem. With keycloak 18 the way how key rotation in keycloak need to be done hase changed.
For this reason the unittests in python no longer work in a way like they should.
I think the integration tests needs to be reworked in a way that they can run will multiple versions of keycloak.
... i think i will care the upcoming month about this test issue


BUG: some DICOM files crash GDCMIO (and Slicer, ITK-Snap, etc)

Particular DICOM files crash GDCMIO and related applications. To reproduce crash drag-and-drop a file onto Slicer window or ITK-Snap. Here is trivial test.
Particular files has Bits Allocated 16 and Bits Stored  8. Crash happens because here re-scale is applied without check of intercept and slope values ("are they different from 1 / 0?"). And here re-scale is applied only if intercept and slope are different from 1 / 0. gdcm::Rescaler returns UINT8 for the files, even if slope = 1 and intercept = 0. The bug is very old, at least 5 years.
Here is patch
Just close the issue if don't want fix, i don't rely on this code. Thanks.
FYI @malaterre

Can the linked images be added to a unit test?

They are public, i hope, AFAIK from old DCMTK demo CD, also available here
https://www.creatis.insa-lyon.fr/~jpr/PUBLIC/gdcm
in gdcmSampleData/Philips_Medical_Images/Integris_H_3_V_11 folder, but compressed (.Z)

Obviously no interest, closing

The automatic "timeout" for no interest is set to warn after 3 months, and close after 6. 13 days is a bit aggressive, in my opinion.


Insight 5.8.4

Add insight entry

built at https://snoopycrimecop.github.io/www.openmicroscopy.org/2024/05/13/omero-insight-5-8-4.html


Capture UPDATE/PATCH considerations when working with controllers

What this PR does / why we need it:
When reviewing, writing or debugging controller code the implications of the usage of UPDATE/PATCH calls should be clear.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #
Special notes for your reviewer:
Release note:
NONE

/cc @jean-edouard
/cc @davidvossel

I think it makes sense, and the Update Status section encompasses David's concerns, as the operator who owns the status can update the status.
/approve

/approve

/reopen
@rmohr I think this would be still valuable for our contributors

/lgtm

/lgtm

sgtm! Thanks.

/remove-lifecycle rotten


Some of the unit tests are too slow

Currently it takes ~40s to run the test suite. It used to be much faster, but we added a lot of unit tests since then and some of them are bottlenecks. We can use karma-time-stats-reporter plugin to discover which of the tests are slowing down the suite. Here's the list of the slowest ones:
4707ms Map #flyTo move to requested center and zoom, and call zoomend once
2877ms Map #flyTo flyTo start latlng == end latlng
2862ms TileLayer number of kittens loaded Loads 290, unloads 275 kittens on MAD-TRD flyTo()
2699ms GridLayer number of 256px tiles loaded in synchronous animated grid @800x600px Loads 290, unloads 275 tiles on MAD-TRD flyTo()
1329ms ImageOverlay #getCenter should open popup at the center
808ms  Popup L.Map#openPopup moves the map over a long distance to the popup if it is not in the view (keepInView)
683ms  Map.ScrollWheelZoom changes the option 'wheelPxPerZoomLevel'
612ms  Path #events can add a layer while being inside a moveend handler
358ms  Map.ScrollWheelZoom changes the option 'wheelDebounceTime'
314ms  LineUtil #polylineCenter computes center of a small line and test it on every zoom
309ms  Map.ScrollWheelZoom zooms out while firing 'wheel' event
307ms  Map.Keyboard plus/minus keys zoom in
306ms  Map.ScrollWheelZoom scrollWheelZoom: 'center'
306ms  Map.Keyboard does not move the map if disabled no move north
306ms  Map.Keyboard does not move the map if disabled no zoom in
306ms  Map.Keyboard arrow keys move the map west
305ms  Map.Keyboard plus/minus keys zoom out
305ms  Map.Keyboard arrow keys move the map east
305ms  Map.Keyboard arrow keys move the map north
303ms  Map.Keyboard arrow keys move the map south
297ms  Map.ScrollWheelZoom zooms in while firing 'wheel' event
266ms  Map #containerPointToLayerPoint return point relative to LayerPoint
263ms  Map.DoubleClickZoom doubleClickZoom: 'center'
260ms  Map.DoubleClickZoom zooms out while dblclick and holding shift
256ms  Map #layerPointToContainerPoint return point relative to ContainerPoint
252ms  TileLayer #setUrl fires only one load event
244ms  Map.DoubleClickZoom zooms in while dblclick
220ms  Control.Layers sortLayers keeps original order by default
193ms  Map.Drag touch events reset itself after touchend
191ms  Map.TouchZoom Dragging is possible after pinch zoom

It would be nice to fix some of these if they don't depend on such a long running length. Faster test suite means less friction when developing.

Isn't this something very hardware and OS specific? For me running the test suite takes 24 seconds under Chrome Headless 105.0.5195.127 (Windows 10) and other tests are displayed as slowest.
I just tried adding the karma-parallel plugin to run tests in parallel. However, it still takes 24 seconds and even though 4 headless browsers are spun up, it looks like they run sequentially...
@mourner you may check this out:
https://github.com/Leaflet/Leaflet/compare/main...edding4500:Leaflet:add-karma-parallel
Notice that karma-parallel had its last commit in February 2020.

Heya folks, I'm interested in having a look at these.
Reviewing the tests, the biggest savings I anticipate come from reducing the use of animations and setTimeouts within tests. I think there are some places those changes could be made safely.
I'm less certain if there are improvements that can be made to the GridLayer and TileLayer tests, but I'm hoping a few smaller savings elsewhere will still make a difference.
As a question of workflow, would y'all prefer a singular PR with all my attempted improvements or separate ones that are a little more focused.
I imagine separate would reduce the chance of some acceptable improvements getting blocked by others warranting discussion, but I don't want to overly burden yourselves with admin – especially during Hacktoberfest.

A good chunk of our tests appear bottlenecked by the speed that Chrome running in headless mode will deliver animation frames. I've noticed this looking into tests using Leaflet/prosthetic-hand, as they seem to run twice as fast in a non-headless browser.
The tests overall do run a lot quicker in a windowed instance of Chrome. I suspect this is due to GPU acceleration.
npm run test -- --browsers Chrome

Test
Headless time
Headful time
Difference

Total time
11.465s
7.186s
1.59x

Net time
10.898s
6.763s
1.61x

Slowest tests

GridLayer number of 256px tiles loaded in synchronous animated grid @800x600px Loads 290, unloads 275 tiles on MAD-TRD flyTo()
2700ms
1349ms
2.00x

TileLayer number of kittens loaded Loads 290, unloads 275 kittens on MAD-TRD flyTo()
2678ms
1344ms
1.99x

Popup L.Map#openPopup moves on setLatLng after initial autopan
531ms
515ms
(same)

Map #setMaxBounds avoid subpixel / floating point related wobble
303ms
302ms
(same)

Popup L.Map#openPopup moves the map over a short distance to the popup if it is not in the view (keepInView)
267ms
257ms
(same)

Map #containerPointToLayerPoint return point relative to LayerPoint
253ms
254ms
(same)

Canvas #events does fire mousedown on layer after dragging map
198ms
81ms
2.44x

Map.TouchZoom Dragging is possible after pinch zoom
196ms
89ms
2.20x

Map.Drag touch events reset itself after touchend
195ms
97ms
2.01x

Map #flyTo flyTo start latlng == end latlng
115ms
108ms
(same)

I'm not sure if we want to do anything with this information. I don't think it's practical to run a windowed version of Chrome every time we run the tests, as the window popping up is quite disruptive. Nevertheless I wanted to document the finding :)

I think this is all the savings I've found from all >100ms tests. The ones that remain I believe are bottlenecked by animation frames, as mentioned above.
Reductions so far:

Test
Before
After
Reduction

Popup L.Map#openPopup keepInView (sped up one test, introduced another test)
812ms
270ms
542ms

Map.ScrollWheelZoomSpec
1881ms
332ms
1549ms

Map.DoubleClickZoomSpec
812ms
47ms
765ms

Path #events can add a layer while being inside a moveend handler
611ms
5ms
606ms

Map #layerPointToContainerPoint return point relative to ContainerPoint
269ms
1ms
268ms

Map #containerPointToLayerPoint return point relative to LayerPoint
258ms
1ms
257ms

TileLayer #setUrl fires only one load event
251ms
13ms
238ms

Marker #setIcon pan map to focus marker with no iconSize
103ms
1ms
102ms

Marker #setIcon pan map to focus marker
102ms
1ms
101ms

Total
5099ms
671ms
4428ms

@rjackson you did really great work on this, thank you! 🙏
The next obvious step for this ticket would be to review all async tests for whether they need that much duration, and reduce it where possible.  Also, we need to make sure there are no hardcoded animation durations like this, pretty much making all auto-panning tests have 250ms duration minimum:
https://github.com/Leaflet/Leaflet/blob/900a36cf124f67be87e7ced2e5610d0eb38a31ef/src/map/Map.js#L346
E.g. perhaps we need to introduce a map option for default auto panning duration so that we could easily reduce it from 0.25s in tests.
Another thing to watch out for is that in many cases, we do a "big enough" setTimeout in tests instead of waiting for an actual end of an async action, which makes things not only slower but also less reliable.


Typo in type definitions (Jimp.intToRGBA)

Hi there!
I just ran into a problem. I can't the function intToRGBA because it has the wrong signature in TypeScript types definition file.
Thanks!

You should keep both definitions and get them to point to the same function in the actual javascript file

@Ojas-Gulati Sorry? Not sure I understand you.

This would be also fixed with #329


Add/modify e2e tests to use 1.22

Issue #, if available:
Description of changes:
Modifying the e2e tests to either add a new test case for 1.22 or replace the 1.21 test cases with 1.22. This will give us more visibility on EKS-A with 1.22, which is still behind a feature flag.
Testing (if applicable):
make e2e
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/lgtm
/hold

Created an issue to address the above point: https://github.com/aws/eks-anywhere/issues/1377
/unhold


NPM run dev error

Describe the bug
From the beginning of this project I have been getting the "No Preview Available" screen. I try to get the AI to find and fix the error, which is has on occasion, but lately it's gone downhill. I'm constantly getting the same NPM error over and over again. It can't find the error, or maybe it's hallucinating that it's finding the error, when it's really not. I'm not sure what the issue is. I've spent millions of credits trying to get it to identify and fix it. When it does detect a problem, and I click "fix problem" it just recreates a package.json file and then says "All good!" but then I get the same "No Preview Available" screen again. Additionally, often it'll also show me what I call "the gray screen of death" where the browser is just a blank gray/blackish screen (see screenshot).  Would deploying it to Netlify provide a different outcome? Maybe I'll try it just to see.

Link to the Bolt URL that caused the error
https://bolt.new/~/sb1-myw6qv
Steps to reproduce

Go to: YourSiteGuru Client Portal (forked)
It will try to load the preview, it will fail.
Click "fix problem" ... it will say it did.
It will say "No Preview Available"

Expected behavior
When I see the actual preview, I get so excited! But most of the time - it's just "No Preview Available" and I pull more hair out of my head.
Screen Recording / Screenshot
I've included 2 above. If you need more just let me know.
Platform
Browser name = Safari
Full version = 18.1
Major version = 18
navigator.appName = Netscape
navigator.userAgent = Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.1 Safari/605.1.15
performance.memory = undefined
Username = lifehiker
Chat ID = 40d02ac50ba9

Additional context
I've spent millions of credits now just trying to solve these issues, I think it's kind of crazy that just telling it "run the code" costs 100,000 credits - or so it seems to cost that much - and feels wildly expensive. I'm not sure how it's structured but it seems like every command to the ai is at least 100,000 credits, but not all commands are equal, so the cost shouldn't be either. With Cursor.ai I have not run out of credits and have provided it with way more commands and I haven't exceeded the cost of $20/mo, whereas I blew through Bolts allocation of $20 worth of credits and now am on the verge of running out at the $50/mo level, all without what seems to be an app that won't work (at least doesn't show any preview of it working).

Oh my gosh, I just tried "deploying" it to Netlify - AND IT'S WORKING!!!! I can't believe it's working! So my next question would be - why would it work on Netlify and not on internally, on the preview?

yeah, I really wish the preview would work - I hope it can be solved. But in the meantime, I'm so excited and thrilled it's working on the "live" deployment. I will be up early in the morning to keep working on it!!! Woohoo!!!

and of course, with my last try and update of the night, the preview shows up. Unreal. haha

But, back to the gray screen of death and "no preview available" is constantly happening. Sometimes, the preview will show up, but then it'll go to the gray screen of death and I'll have to refresh the page.

and the conversation that I was having will be gone, losing whatever progress I had been making with the ai

Appreciate the feedback and glad you are liking Bolt! For best compatibility use Chrome or Firefox as described here. It is likely that some of your issues may have stemmed from using Safari. Hope this helps!


add calibre 5.1.0

[ x ] Have you signed the Contributor License Agreement?
[ x ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ x ] Have you validated your manifest locally with winget validate <manifest>, where <manifest> is the name of the manifest you're submitting?
[ x ] Have you tested your manifest locally with winget install -m <manifest>?

Microsoft Reviewers: Open in CodeFlow

@wingetbot run


chore(installer): Moved automaticProvisionMsg under features

features:
  automaticProvisioning:
    serviceURL: ""
    message: ""

Related Issues
Closes #7752
How to test

the helm chart is generated with the correct substitutions, eg. bridge and shipyard env vars
https://github.com/keptn/keptn/runs/6999271761?check_suite_focus=true#step:3:2122
https://github.com/keptn/keptn/runs/6999271761?check_suite_focus=true#step:3:2772

BREAKING CHANGE:  values related to automaticProvisioning options are now both set under the features object.  bridge.automaticProvisioningMsg  --> features.automaticProvisioning.message
features.automaticProvisioningURL --> features.automaticProvisioning.serviceURL


FR: Open list of updates in a new tab

The list of plugin updates takes over the current tab and does not restore it when updates are completed.
It would be nice if the list of plugin updates opened in a new tab and closed when done.

Thank you!


Editor styling buttons reload the page

Describe the bug
Clicking on an editor style button causes the page to reload (the first time)
Steps to Reproduce
Steps to reproduce the behavior:

Add the component to a page and include styling buttons
When loading for the first time clicking on one of the buttons causes the page to reload
After reloading, clicking on the styling button works as expected - ie styles are applied
NOTE: This almost always only happens on the FIRST load but unfortunately it doesnt always happen, making it hard to debug.
If you can't reproduce then see the Testing info below

Fork this or create a new Codesandbox replicating your error
https://codesandbox.io/s/vue-issue-template-h0g28
Expected behavior
Styling buttons should never reload the page but should just apply the styling as expected
Testing

Login to mymemry.com with
<EMAIL_ADDRESS>demo123
Click on New Memry
In the editor highlight some text and click BOLD. Page will reload. Subsequently it will work fine.
To reproduce again, logout, close the browser tab, reload the site and login again and it will happen again. But only the first time.

Environment

Browser chrome
Version
"tiptap": "^1.25.0",
"tiptap-extensions": "^1.27.0",

I've followed the Testing path and took a quick look at the source code.
If I'm not mistaken, when you have a button element inside of a form element, it automatically submits the form after clicking the button.
There are multiple ways of fixing the issue but I think the best approach would be adding the type="button" attribute to each button that is being rendered as the Text Editor button inside the  .menubar div container

@kkolasinski  I took your advice and it worked!
Thanks so much!


NTP: serial GPS does not get initialized?

OPNsense 20.1.7-amd64
Not sure if I simply misunderstand the way things should work here ... I've recently connected a serial GPS receiver with UBX-M8030 chipset and consequently selected U-Blox as type on the page Services -> Network Time -> GPS, serial port cuau0 and baud rate 9600 (manufacturer's default for the connected receiver).
In GPS initialization -> Advanced I see as last line:
$PUBX,41,1,0007,0003,4800,0*13
I believe this should set the baud rate to 4800.
Then why does the log read:
ntpd[50477]: GPS_NMEA(0) serial /dev/gps0 open at 9600 bps
/dev/gps0 correctly links to /dev/cuau0:
# ls -l /dev/gps0
lrwxr-xr-x  1 root  wheel  10 May 29 05:00 /dev/gps0 -> /dev/cuau0

However, the baud rate remains at the initial 9600:
# stty < /dev/gps0
speed 9600 baud;

Meanwhile, cat /dev/gps0 shows correct messages like $GNGGA and $GNRMC.
If I change the baud rate in the serial port settings from 9600 to 4800, the log file and port settings do indeed reflect this change:
ntpd[39392]: GPS_NMEA(0) serial /dev/gps0 open at 4800 bps
# stty < /dev/gps0
speed 4800 baud;

But at the same time, the status of the receiver changes from active peer to unavailable:
Status | Server | Ref ID | Stratum | Type | When | Poll | Reach | Delay | Offset | Jitter
Unreach/Pending | <IP_ADDRESS> | .GPS. | 0 | l | - | 16 | 0 | 0.000 | +0.000 | 0.000

and cat /dev/gps0 no longer shows any output.
Now I wonder if the initialization sequence is not processed at all.

I've just found the closed bug #1712 from 3 years ago (not sure why it didn't show up before I've opened this one). It seems that the problem was never fixed: the initialization sequence is not sent.
The only difference is that my GPS has a real serial connection instead of USB.

This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository,
please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue,
just let us know, so we can reopen the issue and assign an owner to it.


IPv6 support

Hi, I was wondering if IPv6 is supported by sish? I currently run it on IPv4, however I want to add IPv6 support as well, how would I do that?

IPv6 is indeed supported by sish!
To use IPv6, set your listen addresses to a comma separated string of addresses. For example:
      --tcp-address=tcp://<IP_ADDRESS>,tcp6://[fdf6:a22a:2e47:fd87::1]
      --ssh-address=tcp://<IP_ADDRESS>:22,tcp6://[fdf6:a22a:2e47:fd87::1]:22
      --http-address=tcp://<IP_ADDRESS>:80,tcp6://[fdf6:a22a:2e47:fd87::1]:80
      --https-address=tcp://<IP_ADDRESS>:443,tcp6://[fdf6:a22a:2e47:fd87::1]:443

Thanks! I think this would be a good addition to the docs for any one in the future wanting IPv6 support :)