Quantized models on multi-GPU

I'm experimenting with the new implementation of CUDA acceleration for quantized models and wondering how to use sharded tensors in this context. I'm having a hard time adapting the ShardedVarBuilder to load like quantized_var_builder::VarBuilder::from_gguf.
Do you have any recommendations on the best approach in this case?
@LaurentMazare

@LaurentMazare I'm sorry to bother, but I just want to ask: is it possible to use the current implementation of quantized models in a multi-GPU setup (like the llama_multiprocess example)? If not, is there any plan to support this feature in the future?
I appreciate your work on advancing the CUDA kernels for quantization.

I'm not sure that having the same technique as what is used for llama-multiprocess would make sense here. The llama-multiprocess version is useful when some tensors have to be shared across different gpus, however I don't think there would be quantized models that would be large enough so that this would actually be useful?
If the goal is just to have multiple models that live on different gpus, then that part should be reasonably easy to do even with the current api by creating one device per gpu that you want to target, but maybe you're after something more complex than this?

I'm after sharding larger models that wouldn't fit on a single 24GB GPU and could instead be split across, for example, 4 of them. If I'm not mistaken, llama.cpp supports multi-GPU through pipeline parallelism but supported tensor splitting between GPUs before that.

If there is no need to shard one tensor one multiple gpus, I would recommend doing something a lot simpler than llama-multiprocess and instead put the different weights on different gpus. I guess it's likely what the pipeline processing of llama.cpp is doing.

Unfortunately, it is necessary to shard the tensors for both larger models (40b+ params) and to speed up larger batch sizes. My use case is an API serving multiple concurrent requests.
Is the solution you're suggesting of putting different weights (layers?) on different GPUs similar to transformers' device_map? I suppose it's slower than sharding, right?

I have a similar use case, where I need to shard a large model (gradient.ai llama 3 262K context) across multiple GPUs. Looks like Pytorch has "fully sharded data parallel" https://pytorch.org/blog/introducing-pytorch-fully-sharded-data-parallel-api/ .
Is there long term plans to add something similar to candle?


Add Per-Sensor Descriptor and Support Multiple IMUs

This modification introduces sensor descriptor to specify sensor config in a clearer way. Currently it contains sensor types, address, pins and extra configs like axis remapping.  It may extend to more parameters like magnetometer type in the future.
Descriptors are written in macros in the following format thanks to @yoyobuae :
#define IMU_DESC_LIST \
             IMU_DESC_ENTRY(IMU_MPU6050, 0x68, DEG_270, PIN_IMU_SCL, PIN_IMU_SDA) \
             IMU_DESC_ENTRY(IMU_BMI160,  0x69, DEG_270, PIN_IMU_SCL, PIN_IMU_SDA, AXIS_REMAP_DEFAULT)

They will be expanded to sensor building and detecting in the SensorManager. Sensor ID is assigned by the sequence of sensors in the list, and other parameters will be passed to building functions. This creates a direct connection between sensor id, sensor address, and sensor pins / configs in the same place.
The new descriptor also allows to add more than 2 sensors by appending more lines to the descriptor. To implement this, each sensor config contains the pin for that sensor. Internal pinmux inside ESPs are used to switch between different sensors, so more than 2 sensors with the same base address could be hooked up to different pins without extra hardware.
The code is highly experimental but attempted to be compatible with previous configs. When not specified, the descriptors will be generated from previous defines with matching address and ids. If a previous config uses two types of sensors but with incorrect address pin connection, the new code won't try the other address to avoid mismatch between config and assigned sensor id.
Axis remapping for BMI160 may not be fully compatible with previous configs. Now axis mapping is described by a number generated from defines. The sensor will use the number for no remapping by default to match the previous default behavior. Each BMI160 could have its own number for a different remapping in its descriptor. To migrate for this patch, previous changes in bmi160_defines.h for axis mapping will need to be rewritten in the new descriptors unless no remapping is used.
I have tested this in 2-sensor setups with combination of BMI160 and MPU6050s. An experimental setup driving 8 MPU6050s are also tested and works with only additional lines in the descriptor. INT pin for BNOs may need to be tested but should be working as the only difference is in passing pin numbers.

Firmware webflasher would need to be rebuilt for this... @ButterscotchV @loucass003...


Performance Improvements

Increasing up to 4 gunicorn worker processes for handling requests. Required for on-boarding RSI & MWSS.

Increasing workers timeout to 60 secs. Required for uploading large sample files.

May require additional tuning before on-boarding MBS.

 Performance: Performance test RAS/RM Release 16

Codecov Report

Merging #126 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #126   +/-   ##
=======================================
  Coverage   96.13%   96.13%
=======================================
  Files          32       32
  Lines        1138     1138
=======================================
  Hits         1094     1094
  Misses         44       44

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e199c7d...72c9e03. Read the comment docs.


⚠ Bofrost.de has degraded performance

In 4e9563b, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 829 ms

Resolved: Bofrost.de performance has improved in f6641b5.


Add option to filter by labels

Context
We are using labels to differentiate between things to be reviewed, and things that is WIP,
and I believe there are few other organisations that do the same way.
Though now, arguable we have PR draft, but I think this could be still useful for other organisations,
to filter out by labels.

Great idea! I'll file a separate issue about draft PRs. Both sound like valuable additions.


AMD modules should add AMD deps

We need to support dep ammendments from the deps API to complete support over global, cjs and AMD.
Alternatively we should disable AMD deps support in SystemJS.

Released in 0.13.0.


Move files to zivid data

We've agreed to stop duplicating sample data in each repository. Instead we will host all sample data on webpage/downloads/ZividSampleData.zip.

The KB instructions for ZividSampleData:
Some Zivid Samples require Sample Data (point clouds, file cameras, transformations, etc.).

Instructions
Download Sample Data from the following link:
https://www.zivid.com/software/ZividSampleData.zip

Extract the content to:
Windows: C:\Program Files\Zivid\data
Ubuntu: usr/share/Zivid/data

For Halcon Samples, you need to set the ZIVID_DATA system variable:
Windows: setx ZIVID_DATA "C:\Program Files\Zivid\data" /m
Ubuntu: export ZIVD_DATA=usr/share/Zivid/data

This should be a quick one to review. I would appreciate if we can do it before we start working on the 2.0. samples. Sorry if I am nagging too much. @nedrebo @knatten

Ubuntu: usr/share/Zivid/data -> Ubuntu: **/**usr/share/Zivid/data

Why "For Halcon Samples, you need to set the ZIVID_DATA system variable:"?

This looks like a good change to me, keeping copying of sample data out of CMake.
If I understand the changes correctly you can also delete Zivid3D.zdf, some yaml files, Settings, maybe more from repository.

yes, we will delete everything that is loaded by a program from the repository. Therefore, leaving just code and readme files.

Why "For Halcon Samples, you need to set the ZIVID_DATA system variable:"? When it is not needed for other languages?

Because in Halcon (HDevelop) we don't have access to Zivid SDK, only Genicam producer stuff, Therefore, we don't use Zivid::Environment::dataPath(). Am I wrong and it is possible to get the same thing from GeniCam producer?

Ubuntu: usr/share/Zivid/data -> Ubuntu: **/**usr/share/Zivid/data

Do I need to change this line as well? export ZIVD_DATA=usr/share/Zivid/data

Why "For Halcon Samples, you need to set the ZIVID_DATA system variable:"? When it is not needed for other languages?

Because in Halcon (HDevelop) we don't have access to Zivid SDK, only Genicam producer stuff, Therefore, we don't use Zivid::Environment::dataPath(). Am I wrong and it is possible to get the same thing from GeniCam producer?
See this (if my link works): https://github.com/zivid/halcon-samples/blob/4eff2985a8605d67336e0c9a8688d44c02033771/source/Applications/Advanced/ObjectMatching/SurfaceMatchingCreateModelFromFile.hdev#L34

Understand, no it is not exposed afaik.

Ubuntu: usr/share/Zivid/data -> Ubuntu: **/**usr/share/Zivid/data

Do I need to change this line as well? export ZIVD_DATA=usr/share/Zivid/data

Yes

Why "For Halcon Samples, you need to set the ZIVID_DATA system variable:"? When it is not needed for other languages?

Because in Halcon (HDevelop) we don't have access to Zivid SDK, only Genicam producer stuff, Therefore, we don't use Zivid::Environment::dataPath(). Am I wrong and it is possible to get the same thing from GeniCam producer?
See this (if my link works): https://github.com/zivid/halcon-samples/blob/4eff2985a8605d67336e0c9a8688d44c02033771/source/Applications/Advanced/ObjectMatching/SurfaceMatchingCreateModelFromFile.hdev#L34

Understand, no it is not exposed afaik.

Can it be (theoretically)? If so, I can make an issue for it?

Ubuntu: usr/share/Zivid/data -> Ubuntu: **/**usr/share/Zivid/data

Do I need to change this line as well? export ZIVD_DATA=usr/share/Zivid/data

Yes

Done

Opp / was an attempt to make text bold. It should be only /.

Ready for the next round.

lgtm

@nedrebo can you confirm? This is not just about this PR it is about all samples. Once this one is merged, I will make a PR for each other.


Importing latest version of chart.js in NodeJS application throws @kurkle module error.

Expected behavior
We are using the chart.js module in one test graphical client as given below:
https://github.com/umati/UA-for-Industrial-Joining-Technologies/tree/main/OPC_UA_Clients/NodeOPCUA_IJT_Client
The page should load successfully when launched.
Current behavior
While using chart.js 4.0.1, it works fine. But, while using a later version, it throws an error saying:
Failed to resolve module specifier "@kurkle/color". Relative references must start with either "/", "./", or "../".
It seems to be some usage of relative paths.
Reproducible sample
4.2.1
Optional extra steps/info to reproduce
Launch the project as per the ReadMe file given the project link.
In the browser when we click F12, we will see an error @kurkle module not found.
Possible solution
The following work-around fixed the issue temporarily but is there any better way to handle it?
In node_modules/chart.js/dist/chart.js, change import '@kurkle/color; to import<EMAIL_ADDRESS>In node_modules/chart.js/dist/chunks/helpers.segment.js, change import { Color } from '@kurkle/color; to import { Color } from<EMAIL_ADDRESS>Context
We are using the chart.js module for a graphical OPC UA client to showcase graphs. Maybe the way to use the latest version is a bit different and could not find any reference in the google search, so submitting a issue request.
chart.js version
v4.2.1
Browser name and version
Google Chrome Latest Version
Link to your project
https://github.com/umati/UA-for-Industrial-Joining-Technologies/tree/main/OPC_UA_Clients/NodeOPCUA_IJT_Client

The way you are importing chart.js directly form the node modules is not supported, you should use a bundler to bundle your code then it will build correctly.
Otherwise you can import chart.umd.js that contains all code packaged as 1 file already

@LeeLenaleee  Thank you very much, I will  try your suggestion.

@LeeLenaleee  We have tried with chart.umd.js and that works as expected. Thanks a lot for your support. This ticket can be closed now.


Add finesse to flying snake bite attack.

Fixes #2735
Add finesse property to flying snake bite attack.

Rather than setting this one weapon to finesse, perhaps we would be better served by implementing some of the suggestions from #2735? I'd propose that for any weapon of type 'Natural' with the attack ability modifier set to 'Default' that it should use the highest of Str/Dex automatically (min 0). Exceptions could be handled by setting an explicit ability, or by using attack.flat proposed in #2592.
We'd have to evaluate any disruption this might cause for pre-existing Actors using natural weapons but I'm hoping that (a) natural weapons are rare on non-NPCs, and (b) that it will produce the correct result in most cases.


Show available balance at gallery item at Basilisk

Thank you for your contribution to the KodaDot NFT gallery.
👇 _ Let's make a quick check before the contribution.
PR type

[x] Bugfix
[ ] Feature
[ ] Refactoring

What's new?

[x] PR closes #3387
[ ] Requires deployment <rubick/magick_issue>
[ ] <brief_description_of_what_I've_added>

Before submitting Pull Request, please make sure:

[x] My contribution builds clean without any errors or warnings
[x] I've merged recent default branch -- main and I've no conflicts
[x] I've tried to respect high code quality standards
[x] I've didn't break any original functionality
[x] I've posted a screenshot of demonstrated change in this PR

Optional

[ ] I've tested it at </rmrk/collection/26902bc2f7c20c546a-1FVG7>
[ ] I've tested PR on mobile and everything seems works
[ ] I found edge cases
[ ] I've written some unit tests 🧪

Had issue bounty label?

[x] Fill up your KSM address: Payout

Community participation

[x] Are you at KodaDot Discord?

Screenshot

[x] My fix has changed something on UI; a screenshot is best to understand changes for others.

wfm!
pay 50 usd

😍 Perfect, I’ve sent the payout
💵 $50 @ 53.11 USD/KSM ~ 0.941 $KSM
🧗 Caiv9TbPz68q5dC8EcHu5xKYPRnremimGzqmEejDFNpWWLG
🔗 0xb62bb64ce56f036e50e4f9f81b768b115eb37a5cdb03fea4e8f86f91f9b1ec3a
🪅 Let’s grab another issue and get rewarded!
🪄 github.com/kodadot/nft-gallery/issues


fail fast if trout isn't installed?

run    trout checkout --source-root=template/trout Gemfile git://github.com/thoughtbot/suspenders.git from "."
/Library/Ruby/Site/1.8/rubygems.rb:777:in `report_activate_error': Could not find RubyGem trout (>= 0) (Gem::LoadError)

Latest suspenders does not use trout. Closing.


Use MPS backend on Apple Silicon devices if it's available. (Updated)

This is slighly updated version of #108. Since #108 was accidentally merged and reverted and I can no longer update it with new changes, this Pull Request is newerly created.
For MacBook users who want to try, follow next steps. It needs enough memory, probably about 32GB.

Clone this repository jy0205/Pyramid-Flow and apply this Pull Request changes.
Install Python 3.10 by using Homebrew, for example, brew install<EMAIL_ADDRESS>Use virtualenv to install dependencies, for example, python3.10 -m venv .venv && .venv/bin/pip3 install -r requirements.txt
Install extra dependency, gradio for app.py, like .venv/bin/pip3 install gradio
Run .venv/bin/python3 app.py, then open http://<IP_ADDRESS>:7860/.

To generate video, try minimum settings first. It takes loooong time anyways on MacBook (about 10 minuts for 3 seconds video, for example, well, it's still remarkable, tho!)

Model resolution: 384p
Duration 2 or 3

Problems
For the inference, it works faster by using MPS backend on Apple Silicon devices but it's not enabled by default and requires some modification to the code, which only considering CUDA availability.
Solution
Use MPS backend if it's available.

Use compatible dtype.
Enable CPU offloading only if CUDA backend is used (disabled for MPS backend, which is unnecesary because of unified memory.)
Use the latest pytorch, which seems required to address VAE issue.

NOTE: This patch is not taking trainig account at all, only for inference. I tried to make it works as well as CUDA with this patch, but because of for example, dependencies update, which may not be preferred, therefore I don't expect that this Pull Request is mergable into main for now. However, anyways posting here because I think it’s worth to have it for those who want to try inferencing easily on such as thier MacBook Pro.

Hey I tried applying your patch locally but I keep getting this error when trying to generate
[ERROR] Error during image-to-video generation: module 'torch.mps' has no attribute 'is_available'
Can you please help resolve this? Running it on M1 MAc 32 GB
Torch version ->2.1.2
Torchvision version -> 0.16.2
torch.has_mps -> true

@rahulbudhrani01 You need to update torch and torchvision from nightly build, even the latest released version can not work properly especially VAE encoder.

Hello - I've been working on trying to get this to work on MPS backend too.  My system is M2 with 24GB ram.  I set python 3.10.13 with nightly pytorch (2.6.0).
I'm using the 384 model.
I also did the modifications in the code base throughout such as changing "CUDA" device references to "MPS", changing any float64 errors for bfloat16(rope) and adding an extra with torch.autocast("mps", dtype=torch.bfloat16): to the textencoder block in the pipeline. (otherwise, the prompt_embeds would be NaN).   So, I can generate a 1 frame video...
https://github.com/user-attachments/assets/210ed9d3-24f2-474e-b2bc-5004ecb632e9
BUT, If I try to generate more then 1 frame, I get this...
https://github.com/user-attachments/assets/f44e9eb8-8295-4aaa-8c7a-9726871d0bf0
I'm wondering if you might have an idea what is the problem.   This looks like when you try to create a pyramid flow video with a resolution that is not 640x384

@YAY-3M-TA3 See this pull request for more details but what you need to do is following inline.

changing any float64 errors for bfloat16(rope)
adding an extra with torch.autocast("mps", dtype=torch.bfloat16): to the textencoder block in the pipeline. (otherwise, the prompt_embeds would be NaN).

mps doesn't support bfloat I believe, also torch autocast may not working on mps, therefore probably just using float32 and disabling autocast would need.

If I try to generate more then 1 frame, I get this...
I set python 3.10.13 with nightly pytorch (2.6.0).

In my case, I've also seen this colorful output caused in step of vae encoding, and it was caued with old pytorch. If I use the latest nightly pytorch 2.6.0.dev then it worked. I haven't really traced which part of vae encoding causing the issue though.
So.. I recommend to ensure which version of pytorch is really used, also try float32 instead.

@YAY-3M-TA3 You're correct! I just chekd pyrotch changes and yeah, the nightly supports bfloat16 on Sonoma and later also autocast.
I am now testing bfloat16 now and it seems working to me on my test script not app.py with limited duration, and memory pressure gets low about 20 to 30GB total at max which is good!
If it seems okay, I'll try update app.py and this pull requests.

Ok - I tried your changes - I did get an error. (Did you get this at all?)
File "/Pyramid-Flow-MPS/video_vae/modeling_causal_conv.py", line 137, in forward
    x = self.conv(x)

  File "/miniforge3/envs/Pyramid-Flow-MPS/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/miniforge3/envs/Pyramid-Flow-MPS/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)
  File "/miniforge3/envs/Pyramid-Flow-MPS/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 725, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/miniforge3/envs/Pyramid-Flow-MPS/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 720, in _conv_forward

    return F.conv3d(
RuntimeError: Input type (float) and bias type (c10::BFloat16) should be the same

So, I made this change in video_vae/modeling_causal_conv.py to solve...(caching the x dtype, then casting, then casting back)
        x_dtype = x.dtype
        x = x.to(self.conv.weight.dtype)

        x = self.conv(x)

        x = x.to(x_dtype)

While this did fix the error, I still get that weird video for videos longer than 1 frame.

@YAY-3M-TA3 I didn't see such error while I was testing. What if you use https://github.com/niw/Pyramid-Flow/blob/add_simple_cli_command/generate.py which is a simple version of script I use for testing this implementation.
I don't think it would address your issue because it's mostly the same logic used in app.py (just simplified for repeating operations.) though.
Also I wonder which macOS/MacBook are you using? I'm on 15.0.1 (24A348) with M1 Max SoC, and I know MPS (or CoreML) sometimes has different behavior per macOS version and/or SoC.

@YAY-3M-TA3 I didn't see such error while I was testing. What if you use https://github.com/niw/Pyramid-Flow/blob/add_simple_cli_command/generate.py

Yeah, I also have my own test script which is based on generate.py to simply render a video with hardcoded values.  I get the same video issue.

Also I wonder which macOS/MacBook are you using?

  (Both with mflux   and comfyUI)
.
.

I'm on 15.0.1 (24A348) with M1 Max SoC, and it's very unlikely though I know MPS (or CoreML) sometimes has different behavior per macOS version and/or SoC.

Haha - I've been reluctant to upgrade my OS to 15 because I heard it was broken with torch... (I also follow this torch MPS thread... pytorch MPS issue)

Interesting..., I may want to try on macOS 14 and if I can repro (need to find someone nearby who has that machine, likely VM is not an option for GPU work). Sounds likely it's related.
I know that M2 SoC has some unexpected behavior with specific math graph on CoreML, but that should be unrelated. May need to trace VAE code as well as pytorch mps implementaion, I haven't really looked into them yet...

Also i noticed I am using slightly newer version of nightly build, but even if I downgraded to 2.6.0.dev20241011, I couldn't repto the problem.

torch                     2.6.0.dev20241015
torchvision               0.20.0.dev20241015

May need to trace VAE code as well as pytorch mps implementaion, I haven't really looked into them yet...

On your small test video, how many frames did you do? 8?  Are you able to create a video with only 2 frames? (I can at least use a 2 frame video as a confirmed positive case, which will make the VAE tracing faster...)

With help from @kagemiku, it's identified that macOS 14 caused the issue but macOS 15 seems okay.
Next step is understanding "why," but at least that is the reason I think.

May need to trace VAE code as well as pytorch mps implementaion, I haven't really looked into them yet...

Looking in modeling_causal.vae.py in def tiled_decode
down at line 517
dec = torch.cat(result_rows, dim=3)
For a 1 frame video, the decode tensor values look like this before image processing:
tensor([[[[[-0.2451, -0.3242, -0.3574,  ..., -0.9727, -0.9727, -0.8555],
           [-0.3359, -0.4043, -0.3984,  ..., -1.0000, -1.0156, -0.9492],  ...

However, on a 2 frame video, thedecode  tensor values look like this:
tensor([[[[[-8.6892e+24, -1.9796e+25, -3.0676e+25,  ...,  2.9014e+26,
             2.4179e+26,  4.4730e+25],
           [-1.8814e+25, -4.2010e+25, -6.3771e+25,  ...,  8.6076e+26,
             6.8183e+26,  2.1156e+26], ...

I am assuming the value ranges should more like in the 1 frame tensor...

Okay I’ve identifieed the problem. It's kind of pytorch bug, or mismatch between pytorch expectation vs MPS behavior, and this mismatch is only happening on prior to macOS 15 because macOS 15 is using native stride.
I've updated this pull request with fix, but can't test on macOS 14.
@YAY-3M-TA3 if you have time, try this new change!

@YAY-3M-TA3 if you have time, try this new change!

Yes, I m very happy to help you! - you have done a fantastic job!  So, far I just tested a 2 frame and it worked!  I'm now going to do 9 frame, and 16 frame.
FYI: It also still works with torch nightly (2.6.0)...
I will update you in a couple of hours as I finish the other video renders!

@niw
OK! all confirmed - its working for SONOMA 14.7, with Torch Nightly (2.6.0)
Mac M2 with 24Gb
Python 3.10.13
Pyramid 384 model
9 frames (~45 minutes to render)
https://github.com/user-attachments/assets/1e6d8267-6b41-45ae-8c4c-29a7aceabeff
16 frames (~85 minutes to render)
https://github.com/user-attachments/assets/08f4fe8e-b6a6-4805-ba3a-06cd7032bf63
Considering no modern video diffusion model was working on Macs until now!  And now we can even render a 16 frame, 640x384 with less than 24Gb - this is quite a milestone!
I never would have seen that
sample = sample.contiguous() and
hidden_states = hidden_states.contiguous() would have been the key towards the solution - Great work!

@YAY-3M-TA3 Thanks for the confirmation! I'm glad that it sovled the problem.

Thanks for supporting miniFLUX @niw

Still I'm testing niw, though, it seems working as expected. I think the patch working well with Apple Mac thus, If you can test it doesn't break on cuda environment (and if you are confortable with the change, of course!) feel free to merge into main.

The output seems better than sd3 ones! It'a really impressive that such video can be generated quickly on laptop locally.

a old man standing in front of a wall clock at train station with crowded peopel.

https://github.com/user-attachments/assets/dbd823c1-c7d8-4678-b5be-da87282c0a17

thank you all for the help. do you have any idea why the memory would be fine during the generation and then as soon as it hits 100% (generated all frames), it suddenly spikes to like 4x of what it was running at during gen?

thank you all for the help. do you have any idea why the memory would be fine during the generation and then as soon as it hits 100% (generated all frames), it suddenly spikes to like 4x of what it was running at during gen?

This is likely due to VAE decoding, see https://github.com/jy0205/Pyramid-Flow/issues/5#issuecomment-2404411873.

Hi, any plans to merge this in so mac users can also use this? Or is there a reason why it's not merged in yet?

@cocktailpeanut nice change! let me do that.

@niw I have cloned use_mps_on_apple_silicon, but after 100% of generation I got this error abut FFmpeg not installed or looking for FFmpeg EXE
[INFO] Text-to-video generation completed.
[ERROR] Error exporting video: No ffmpeg exe could be found. Install ffmpeg on your system, or set the IMAGEIO_FFMPEG_EXE environment variable.

Any help about this issue

@PakanAngel It's unrelated to this repo but for the node that makes video from each frame images generated. Recommend to see the doc for that node yet likely you need to install ffmpeg by using such as Homebrew.

Hi, any plans to merge this in so mac users can also use this? Or is there a reason why it's not merged in yet?

@cocktailpeanut  I just found that pytorch>2.4.1 whch is used for this branch doesn't work with cuda and failed to generate latents. It may not good to merge this change to main at this moment.

@niw couldn't we create a separate file requirements_mac.txt?
TBH even the current requirements.txt file doesn't even work for CUDA anyway (it will install CPU torch instead of CUDA torch if you just run pip install -r requirements.txt). Anyway, even if we ignore that, if this is the only thing that's blocking this branch from merged in, I think this is a good solution.
See liveportrait as an example: https://github.com/KwaiVGI/LivePortrait/blob/main/requirements_macOS.txt

@niw couldn't we create a separate file requirements_mac.txt?
TBH even the current requirements.txt file doesn't even work for CUDA anyway (it will install CPU torch instead of CUDA torch if you just run pip install -r requirements.txt).

Oh hmm. At least it worked (amd64/linux) to me tho. Also I addressed the pytorch 2.5.1 changes in previous commit so this requirements should work both on cuda and mps.
But as you mentioned, probably it’s better to have a separated setup path for on mac, which is only for inference as well.


Codemirror editor and streams

I would like to know whether codemirror editor can be used for editing streams tiddlers. Only when a node is opened as a seperate tiddler, codemirror editor was working. Not when I tried to edit the node within the streams tree itself.

I once had a working prototype for RapidTiddler, but it stopped working
with newer versions of streams
https://groups.google.com/forum/embed/#!topic/tiddlywiki/wvjYv36S2YU
arunnbabu81<EMAIL_ADDRESS>schrieb am Mi. 30. Dez. 2020 um
23:45:

I would like to know whether codemirror editor can be used for editing
streams tiddlers. Only when a node is opened as a seperate tiddler,
codemirror editor was working. Not when I tried to edit the node within the
streams tree itself.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/saqimtiaz/streams/issues/11, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABJGCW72IAUSPU5MHMOH5XTSXOURPANCNFSM4VOZJJUA
.

Streams needs to extend the default editor to be able to allow some of the features users have requested, such as splitting a text at the caret when saving a node. Therefore trying to use an alternative editor would likely be problematic.
There are fairly significant updates to Streams coming in the next couple of weeks which may also help some what in this regard. I can take a look at CodeMirror integration after that but I cannot promise that this will be implemented. It will depend on how easily extensible CodeMirror is to do what is needed, which is getting the caret position in the text, the selection if any and also allowing to specify the caret start position when the editor is first rendered.

Thanks Saq. Take your time for looking into this matter.
MeneDev, I had checked RapidTiddler before posting here. The concept looks interesting, but was buggy. May be because i am using the latest version of streams. Hope you can correct the bugs after the new streams update which may come out of beta soon. Best wishes.

Yes, RapidTiddler is just a POC at the moment, my plan is to look into it
again once streams' interface becomes more stable
arunnbabu81<EMAIL_ADDRESS>schrieb am Do. 31. Dez. 2020 um
09:43:

Thanks Saq. Take your time for looking into this matter.
MeneDev, I had checked RapidTiddler before posting here. The concept looks
interesting, but was buggy. May be because i am using the latest version of
streams. Hope you can correct the bugs after the new streams update which
may come out of beta soon. Best wishes.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/saqimtiaz/streams/issues/11#issuecomment-752891404,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABJGCW5LF5PK46364COUIKLSXQ2TRANCNFSM4VOZJJUA
.

There is an attempt at CodeMirror integration in the updated preview for Streams 0.2:
https://saqimtiaz.github.io/sq-tw/temp/streams-0.2-preview.html
When the CodeMirror plugin is installed, you can choose the editor in the Streams settings.
Testing and feedback is needed to make sure this works as expected, both using codemirror as the editor and using the default editor.
The area most likely to be affected is splitting a node by hitting Enter with the text cursor/caret in the middle of some text, at the beginning of some text, or hitting backspace with the cursor at the beginning of a text.

There is an attempt at CodeMirror integration in the updated preview for Streams 0.2:
https://saqimtiaz.github.io/sq-tw/temp/streams-0.2-preview.html
When the CodeMirror plugin is installed, you can choose the editor in the Streams settings.
Testing and feedback is needed to make sure this works as expected, both using codemirror as the editor and using the default editor.
The area most likely to be affected is splitting a node by hitting Enter with the text cursor/caret in the middle of some text, at the beginning of some text, or hitting backspace with the cursor at the beginning of a text.

@MeneDev the requirements for a new editor to work with Streams are:

they keyboard-plus widget needs to be able to get the selection information from the editor when handling keydown events. https://github.com/saqimtiaz/streams/blob/performance/plugins/streams/widgets/keyboard-plus.js#L149
in addition to the core requirements for an editor engine, it needs to support setting the caret position in the text at startup https://github.com/saqimtiaz/streams/blob/performance/plugins/streams/core-tweaks/codemirror-tweak.js

If the CodeMirror integration works as expected, your RapidTiddler should be able to make use of it as long as it doesn't change anything related to the two above requirements.

@MeneDev the requirements for a new editor to work with Streams are:

they keyboard-plus widget needs to be able to get the selection information from the editor when handling keydown events. https://github.com/saqimtiaz/streams/blob/performance/plugins/streams/widgets/keyboard-plus.js#L149
in addition to the core requirements for an editor engine, it needs to support setting the caret position in the text at startup https://github.com/saqimtiaz/streams/blob/performance/plugins/streams/core-tweaks/codemirror-tweak.js

If the CodeMirror integration works as expected, your RapidTiddler should be able to make use of it as long as it doesn't change anything related to the two above requirements.

@saqimtiaz Thanks for the CodeMirror integration with Streams. i was asking for this because, l use streams for making nodes with multiline contents also. In such cases, i liked Codemirror editor's UI better than the default editor. I have started used Codemirror recently only. So far i am liking it. In case you dont plan to implement Codemirror in main streams plug in, please give it as an supplemental plug in so that users who need it can use it.
I tried the new streams beta in my personal wiki in my windows laptop and iphone 7 with Codemirror editor selected. It worked fine except that in iphone 7 I had to long press in the editor area for the keyboard to appear. Splitting a node by hitting Enter with the text cursor/caret in the middle of some text, hitting Enter with the cursor at the beginning of some text and hitting backspace with the cursor at the beginning of a text -  all worked fine in laptop and in iphone.
When i was comparing the default editor and CodeMirror editor, i found some something strange. With "exit editing on saving an empty node" not selected and default editor is selected instead of codemirror editor, suppose i create 3 to 4 empty nodes at the end of a streams tiddler, i was unable to delete the empty nodes except for the last empty node. I found this behaviour in the old versions of streams also. I noticed this only in my iphone. Not in my windows laptop or in the android phone which i used to test this. This was also not seen when CodeMirror editor was selected.

@saqimtiaz Thanks for the CodeMirror integration with Streams. i was asking for this because, l use streams for making nodes with multiline contents also. In such cases, i liked Codemirror editor's UI better than the default editor. I have started used Codemirror recently only. So far i am liking it. In case you dont plan to implement Codemirror in main streams plug in, please give it as an supplemental plug in so that users who need it can use it.
I tried the new streams beta in my personal wiki in my windows laptop and iphone 7 with Codemirror editor selected. It worked fine except that in iphone 7 I had to long press in the editor area for the keyboard to appear. Splitting a node by hitting Enter with the text cursor/caret in the middle of some text, hitting Enter with the cursor at the beginning of some text and hitting backspace with the cursor at the beginning of a text -  all worked fine in laptop and in iphone.
When i was comparing the default editor and CodeMirror editor, i found some something strange. With "exit editing on saving an empty node" not selected and default editor is selected instead of codemirror editor, suppose i create 3 to 4 empty nodes at the end of a streams tiddler, i was unable to delete the empty nodes except for the last empty node. I found this behaviour in the old versions of streams also. I noticed this only in my iphone. Not in my windows laptop or in the android phone which i used to test this. This was also not seen when CodeMirror editor was selected.

I'm not really convinced as to the value that using CodeMirror in Streams brings.
I suspect codemirror is an attractive editor for the normal tiddler interface due to things like indenting and support for the tab key, as well as Vim keyboard shortcuts, all of which will clash with Streams.
As for iOS specific issues, moving forwards I think my policy for iOS is going to be that if certain features don't work then so be it, and I'll only debug issues that make the plugin entirely unusable. iOS is far too limited and restricted a platform for me to dedicate the time and effort to, especially considering that I do consider Streams and in fact TiddlyWiki to be of limited utility on mobile.

I'm not really convinced as to the value that using CodeMirror in Streams brings.
I suspect codemirror is an attractive editor for the normal tiddler interface due to things like indenting and support for the tab key, as well as Vim keyboard shortcuts, all of which will clash with Streams.
As for iOS specific issues, moving forwards I think my policy for iOS is going to be that if certain features don't work then so be it, and I'll only debug issues that make the plugin entirely unusable. iOS is far too limited and restricted a platform for me to dedicate the time and effort to, especially considering that I do consider Streams and in fact TiddlyWiki to be of limited utility on mobile.

In streams also, I liked the better color coding given by CodeMirror plus plug in by adithya. This is useful when i am writing multiline contents within a single node especially if i use formatting . Since i haven't used the entire keyboard shortcuts of CodeMirror, i am not aware of all the clashes with streams. However because the keyboard shortcuts of streams are customizable, i guess clashes could be avoided. I am not sure though.
If you think it wont work out, don't implement it in the main streams plug in. Consider whether it could be given as add on. In the mean time, I will use this beta more in my TW and report if its useable without clashes.
Another thing which i forgot to mention in the above post  -  if i unselect "Double click in editor to exit editing", it is working only when default editor is used. When CodeMirror editor is selected, i was still able to double click in editor to exit editing. I am ok with it if that behavior is not modifiable.
Regarding the iOS related issues, it not a big thing actually. I just reported it to you since it was not seen in other systems.
An iOS related bug which i had reported in reddit is not seen with the beta released today. when editing the Child node 1, the margins of the Child node 1 which is in edit mode is partly overlapping the subnode 1

In streams also, I liked the better color coding given by CodeMirror plus plug in by adithya. This is useful when i am writing multiline contents within a single node especially if i use formatting . Since i haven't used the entire keyboard shortcuts of CodeMirror, i am not aware of all the clashes with streams. However because the keyboard shortcuts of streams are customizable, i guess clashes could be avoided. I am not sure though.
If you think it wont work out, don't implement it in the main streams plug in. Consider whether it could be given as add on. In the mean time, I will use this beta more in my TW and report if its useable without clashes.
Another thing which i forgot to mention in the above post  -  if i unselect "Double click in editor to exit editing", it is working only when default editor is used. When CodeMirror editor is selected, i was still able to double click in editor to exit editing. I am ok with it if that behavior is not modifiable.
Regarding the iOS related issues, it not a big thing actually. I just reported it to you since it was not seen in other systems.
An iOS related bug which i had reported in reddit is not seen with the beta released today. when editing the Child node 1, the margins of the Child node 1 which is in edit mode is partly overlapping the subnode 1

These are my suggestions after using streams beta with codemirror for a few days

In the documentation you may suggest to those people who use codemirror editor and editor autolist plug in along with streams to change the conflicting keyboard shortcuts. I think such conflicts can be overcomed since everything is customizable.
Codemirror editor add much value when I am dealing with multi line content in child nodes because of keyboard shortcuts. This holds true in desktops and laptops.
I think codemirror should be disabled in mobile by default since selecting text doesn’t work when codemirror editor is active in mobile. Also because codemirror shortcuts won’t work in mobile. I came across a code by buggyj  for automatic disabling of codemirror in mobile. But this doesn’t disable codemirror editor in streams nodes in the mobile mode. Can that code be modified to disable codemirror in mobile for nodes in the streams tree also.
When the wiki is opened in mobile and the code mirror editor is selected, currently there is no option to disable double click to exit editing. But since I recommend not using codemirror in mobile, it doesn’t matter. But in desktop or laptop it would be a greater addition if the option to double click to exit editing is given while using codemirror editor also.
For a node with long mutli line text content, when codemirror is selected as editor and we double click to edit the node, the focus of the node in edit mode will shift to the end of that node with caret position also at end of node. But if the default editor is selected, focus of the node in edit mode will remain at the top of the node with the caret position still at the end of the node. I am not saying that I routinely use such long multi line text in a single node of streams tree, but sometimes when I am preparing notes I will first paste the relevant text into a single node before splitting it into separate nodes. This behavior while using codemirror editor is not a big problem bcos I can just use keyboard shortcut to return to the top.

These are my suggestions after using streams beta with codemirror for a few days

In the documentation you may suggest to those people who use codemirror editor and editor autolist plug in along with streams to change the conflicting keyboard shortcuts. I think such conflicts can be overcomed since everything is customizable.
Codemirror editor add much value when I am dealing with multi line content in child nodes because of keyboard shortcuts. This holds true in desktops and laptops.
I think codemirror should be disabled in mobile by default since selecting text doesn’t work when codemirror editor is active in mobile. Also because codemirror shortcuts won’t work in mobile. I came across a code by buggyj  for automatic disabling of codemirror in mobile. But this doesn’t disable codemirror editor in streams nodes in the mobile mode. Can that code be modified to disable codemirror in mobile for nodes in the streams tree also.
When the wiki is opened in mobile and the code mirror editor is selected, currently there is no option to disable double click to exit editing. But since I recommend not using codemirror in mobile, it doesn’t matter. But in desktop or laptop it would be a greater addition if the option to double click to exit editing is given while using codemirror editor also.
For a node with long mutli line text content, when codemirror is selected as editor and we double click to edit the node, the focus of the node in edit mode will shift to the end of that node with caret position also at end of node. But if the default editor is selected, focus of the node in edit mode will remain at the top of the node with the caret position still at the end of the node. I am not saying that I routinely use such long multi line text in a single node of streams tree, but sometimes when I am preparing notes I will first paste the relevant text into a single node before splitting it into separate nodes. This behavior while using codemirror editor is not a big problem bcos I can just use keyboard shortcut to return to the top.

I was wrong about the point number 4. When codemirror editor is active in streams in mobile, double click to exit editing is not enabled by default or is not working. But since text selection won’t work in codemirror editor in mobile, it’s better to let codemirror editor be disabled by default in mobile.
But in desktop or laptop with codemirror editor active for streams, double click to exit editing functions even if its disabled in the streams setting. I am not much bothered about this because if someone is using codemirror, there is always a keyboard shortcut to select text easily. So I won’t complain if it’s difficult to disable double click to exit editing when using codemirror editor in desktop or laptop.

I was wrong about the point number 4. When codemirror editor is active in streams in mobile, double click to exit editing is not enabled by default or is not working. But since text selection won’t work in codemirror editor in mobile, it’s better to let codemirror editor be disabled by default in mobile.
But in desktop or laptop with codemirror editor active for streams, double click to exit editing functions even if its disabled in the streams setting. I am not much bothered about this because if someone is using codemirror, there is always a keyboard shortcut to select text easily. So I won’t complain if it’s difficult to disable double click to exit editing when using codemirror editor in desktop or laptop.

Closing this as Streams 0.21 has experimental support for CodeMirror.


fix: #6334 updating requestgroup caused render error

changelog(Fixes): Fixed issue #6334 where a error was thrown when updating a Request Folder via its Settings.
Closes #6334
The way we were calling update request group (e.g. when renaming or moving a request folder, or changing its description) wasn't properly setup. This PR fixed that. Also added a test.
Pending problems

[ ] when moving a request group to another workspace, we need to either redraw the current workspace view or open the other workspace view, otherwise we still see the moved request on the sidebar
[ ] there's another bug when picking different options of the workspaces to move/copy to on the request

We need to standardise on a update request convention. Thanks for the fix


Use the doctrine paramconverter

Use the doctrine paramconverter instead of manually fetching from db. This simplifies the code and moves more logic out of the controllers

It's indeed a bit of magic, but no different than the Request typehint we already use. It removes some extra logic in the controllers so that was enough for me to move to the paramconverter :smile:
Let's strip some more code out of the controllers :smile:


Merge prTesting_201910271259 branch to master branch

Docs Build status updates of commit dbd44dd:
:clock10: Pending: waiting for processors (5 builds ahead of you)
:warning: Docs Build is busy, currently there are 5 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit dbd44dd:
:warning: Validation status: warnings

File
Status
Preview URL
Details

split/updateTestRef.md
:warning:Warning

Details

split/updateTestRef.md

[Warning] Missing attribute: author. Add the current author's GitHub ID.
[Warning] Missing attribute: title. Add a title string to show in search engine results.
Line 1: [Suggestion] H1 is required. Use a single hash (#) followed by a space to create your top-level heading. NOTE: This Suggestion will become a Warning on 11/25/2019.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Add mapping for SteamDeck controller

Add steam deck controller mapping
Some remarks:

the button 5 is not identified (maybe steam button)
the back triggers L4/L5/R4/R5 are not recognized (maybe we should use steam deck SDK to use them)
L2 and R2 are not recognized as buttons but axis (Axis 4 and 5)
there seem to be no vibration

Oups, you are right I missed the const declaration, I saw someone else made a PR for this
https://github.com/nathanhoad/godot_input_helper/pull/53


Support object literal and object inference

Summary

SDK Version: 3.0.8
Actual behavior: object literals and object inference won't import anything
Expected behavior:

somehow define the correct type
use any as return type
or, at least, throw an error or a warn explaining the limitation

Example 1: type defined on controller
input
@Controller('/hello')
export class HelloController {
  constructor(private readonly helloService: HelloService) {}

  @Get()
  hello(): { message: string } {
    return { message: 'Hello World' };
  }
}

output
import type { __type } from "./../../../modules/hello/hello.controller";

export namespace hello
{
    export type Output = Primitive<__type>;
}

Example 2: type from service
input
@Injectable()
export class HelloService {
  world(): { message: string } {
    return { message: 'Hello World' };
  }
}

@Controller('/hello')
export class HelloController {
  constructor(private readonly helloService: HelloService) {}

  @Get()
  hello() {
    return this.helloService.world();
  }
}

output
import type { __type } from "./../../../modules/hello/hello.service";

export namespace hello
{
    export type Output = Primitive<__type>;
}

Example 3: type inference
input
@Injectable()
export class HelloService {
  world() {
    return { message: 'Hello World' };
  }
}

@Controller('/hello')
export class HelloController {
  constructor(private readonly helloService: HelloService) {}

  @Get()
  hello() {
    return this.helloService.world();
  }
}

output
import type { __object } from "./../../../modules/hello/hello.service";

export namespace hello
{
    export type Output = Primitive<__object>;
}

@samchon depending on your thoughts about how this should be solved, I would be happy to help with this one too.
My only limitation is that I don't know deeply about reflection yet. ~I actually just learnt what it is, with your code~ 🙈

Then, I will solve this problem by myself. Thanks for reporting.

@samchon is it possible to bring whole type declarations to the output source, instead of just import?
In projects that use nest as private modular packages, it would also be useful to create a whole new independent package for the SDK.

@SkyaTura
Writing whole DTO without import statement is enough possible llike below:

swagger.json generated by nestia
typescript-json

However, need to distinguish which to import or re-write the whole DTO.
Have you any plan to dstinguish them? What about distinguish them by object-literal-expression?
//----
// IMPORT
//----
// WHEN DTO TYPE IS CLEARLY DEFINED
export interface IFormalDto {
    id: string;
    name: string;
}

//----
// WHOLE RE-WRITE
//----
// OBJECT-LITERAL-EXPRESSION
class SomeController {
    someMethod(): Promise<{ id: string; name: string; }>
}

// ALIAS TYPE FOR OBJECT-LITERAL-EXPRESSION
type SomeAlias = {
    id: string;
    name: string;
};

@samchon
Would be nice to have a configuration flag that allows you to switch between "always rewrite", which could be used to create a fully independant package, or "import when possible", which would work like you said.
If I didn't misunderstood, the implementation os the second option is harder because the type isn't always clear, right?

Thinking further about it, would be useful as well if, in the future, we could, optionally, pass each type to a transformer, or some kind of hooks, defined in nestia.config, that could be used to create helper functions like form validation and object generators.
May I create an issue to elaborate how could this be useful and how could be the usage, or do you think it's out of the project scope?

@samchon how about using Custom AST Transformers?
@ksaldana1 had a pretty similar scenario as we do, and as demonstrated here, and in an awesome blog post, which describes the development of the idea in steps.
I think this could also be useful in #141, if approved.

@SkyaTura I'd been busy by another open source project. I'll try this on this sunday.
However, when nestia re-writes all DTO, have you any idea to where those new DTOs to be placed in?
Also, do you have any idea if duplicated DTO name exists?

Hmm... I think, ideally a SDK should have unique type definitions, and by extension, I would always try to avoid duplicates in the API as well.
But this is definitely not a well adopted as standard and should be supported by a generator.
Maybe using namespaces to replicate the original path tree, but I really need to wonder a lit bit about this question.
If we had infinite resources I would suggest multiple strategies, but unfortunitely we should also think what would be more feasable.

Then which directory name would be better when generating DTO automatically.
Also, how to do if duplicated directory exists?


Vegetation model update

PULL REQUEST
Purpose and Content
This PR brings several changes to the vegetation model, which solves for fluxes and water content as a function of height in a tree. We changed the flow (mol s-1) function to flux (m s-1), and use the Bristow approximation instead of the Wang approximation for flux, which errored when pressure values in adjacent compartments were equal, or when running with certain boundary conditions which led to negative flow estimates and negative values inside the log() of the flow function. It also sets fluxes at the faces of the compartments, which are calculated by using pressure values at compartment midpoints, which is more accurate than previous implementation. Previously, pressure and flux were defined at compartment surfaces. We also changed the state variable from relative water content to augmented liquid fraction to allow for expansion in the water holding capacity of the tree (vartheta > 1). We also made a few name changes.
Benefits and Risks
Unit conversion between soil and vegetation models not necessary going forward as uniform, all SI. Bristow approximation works better as described above.
Linked Issues
(Provide references to any link issues. Use closes #issuenum to automatically close an open issue)
#70
PR Checklist

[x] This PR has a corresponding issue OR is linked to an SDI.
[x] I have followed CliMA's codebase contribution and style guidelines OR N/A.
[x] I have followed CliMA's documentation policy.
[x] I have checked all issues and PRs and I certify that this PR does not duplicate an open PR.
[x] I linted my code on my local machine prior to submission OR N/A.
[x] Unit tests are included OR N/A.
[x] Code used in an integration test OR N/A.
[x] All tests ran successfully on my local machine OR N/A.
[x] All classes, modules, and function contain docstrings OR N/A.
[x] Documentation has been added/updated OR N/A.

looks good, Anna! let me know if you need help fixing any of the tests, etc.

Thank you! I'll let you know.
On Tue, 6 Sept 2022 at 15:47, kmdeck @.***> wrote:

looks good, Anna! let me know if you need help fixing any of the tests,
etc.
—
Reply to this email directly, view it on GitHub
https://github.com/CliMA/ClimaLSM.jl/pull/89#issuecomment-1238582490,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AOX4JOD3LRXHCAU62C36ZW3V46NU3ANCNFSM56STLOFQ
.
You are receiving this because you authored the thread.Message ID:
@.***>

bors r+

yaaaayyyyy


Nightmare (Solo) no longer gives experience.

🐛 Bug Report
Hello
I have noticed that for about 3 days now, i no longer receive exp, or i receive an extreme minimal amount of exp from doing Nightmare (only tried this on Solo).
I've added 2 images which show my stats before a kill and after a kill.
As you can see, i only gain 4 exp for an entire trip, which doesn't seem like how it should be.
I have asked for help in the osbot discord, and they forwarded me here for this issue.
I hope this helps you, and helps me regain exp-rates during my nightmare grind.
Praenthos.

As i think of it, it's been more than 3 days, nearly a week. A friend of mine also doesn't gain any more experience from killing the Nightmare.

What style are you training?

Shared, but i have tried it on shared, attack only and strength only

it only gives me 4-5 exp for an entire trip


www.abebooks.com

Energized Protection - block ⚡

Let's make an annoyance free, better open internet, altogether!

Issue Submit Form

  Provide the following infos properly, which will help us to resolve your issue quickly.

Issue(s):
Type x in between [ ] and make sure there isn't any space between brackets. Example; for Your Selected Issue(s), type like this - [x]
You can select more than one category of issues if you need to!

[x] Whitelist
[ ] Blacklist
[ ] App Broken
[ ] Website Broken
[ ] Request
[ ] Other Issue

Pack(s):
Write the name of the pack(s) you are using.

Pack(s) Name: Unified

Extension(s):
Also name the extension(s) if you are using any. If you don't then leave it blank.

Extension(s) Name:

Domain(s):
If you are submitting this issue for whitelist/blacklist issue, send us the domain(s) for whitelisting/blacklisting here. Kindly use the Code Tag to prevent tracking.

Domain(s):

www.abebooks.com

Your Config:
Just to ensure there is no issues or conflicts with other app/software/magisk module/extension/source list.
Make sure you are running Energized Protection Service only.

Client: dnscrypt-proxy

Version: 20.07.200

Details:
Write us a lil bit more about your issue or query. You can attach any screen shot or log of the issue or advert, this will help to highlight it.

Your Issue Detail:
Not sure why www.abebooks.com is blocked. It's simply an online store for books. No idea why it would be on the porn list.

Thank you for making Energized Protection great, with your kind help!
A project by Ador with ❤

Apparently a whitelisting was successfully added to Energized's centralised whitelist sometime in the past month, having previously sourced a blocking entry from Airelle's Anti-Sex Hosts.
However, www.abebooks.co.uk still remains blocked and not-whitelisted.


Website components not synchronized with current NPM version

hi, I see that the latest components added (ie Tooltip) are still not available on the main site?

Tooltip will be available in the next release ;)
Sent from my iPhone

On 06 Nov 2015, at 17:00, Cristiano Belloni<EMAIL_ADDRESS>wrote:
hi, I see that the latest components added (ie Tooltip) are still not available on the main site?
—
Reply to this email directly or view it on GitHub.

mmmh so, I'm a bit confused :)
react-toolbox on npm seems to be @ v0.11.3, while here on Github the only tag I see is 0.10.20. What is the main site referring to?

Yes, sorry about that. The thing is we just released a first approach to be able to show the site while presenting it @ Reactive 2015 but we are not into semantic versioning either tagging releases in Github. We have an open PR to use a command line tool to make versioning easy and stop messing up everything.
I didn't expected the project to get that popular so we need discipline with versioning and releasing. This weekend I'll have time to get everything straight. Sorry about that!

No problem! Just wanted to raise the issue for you to be able to pick it up when you could :)
Thank you for your answer!

So sorry @janesconference we will fix that ASAP.

It should be everything ok now!


[HOLD for payment 2023-06-21] mWeb - Workspaces are not immediately displayed when transitioning from OD to ND

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Navigate to OD and log in with an account that has a existing workspace
Click on Create a company for free

Expected Result:
The user is redirected to the Workspaces list in NewDot and all existing workspaces are displayed
Actual Result:
The user is redirected to NewDot Create a Workspace screen but the existing workspaces are not displayed immediately, it takes >10 seconds to load
Workaround:
Can the user still use Expensify without this being fixed? Have you informed them of the workaround?
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[x] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.24-4
Reproducible in staging?: Yes
Reproducible in production?: No - New flow introduced in this PR https://github.com/Expensify/App/pull/20112
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/44479856/9e582b61-ba04-401b-8cc3-27ae87307b8f
Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal team
Slack conversation:
View all open jobs on GitHub

:wave: Friendly reminder that deploy blockers are time-sensitive ⏱ issues! Check out the open StagingDeployCash deploy checklist to see the list of PRs included in this release, then work quickly to do one of the following:

Identify the pull request that introduced this issue and revert it.
Find someone who can quickly fix the issue.
Fix the issue yourself.

@francoisl @srikarparsi assigning you since it looks like it coming from this PR https://github.com/Expensify/App/pull/20112#issuecomment-1578994733

Yeah, this is also how it works on production. The delay is indeed probably because the app has to load workspaces data. As an improvement, we can look into keeping the loading indicator visible while the data is loading, but it's not a blocker / regression.

Production
Staging

@francoisl is this a broader performance issue? Should we be tackling it here?

This is just a "visual" issue if you will, we're showing the wrong state of the workspaces list page while data is still loading. I'll handle internally.

@francoisl @kevinksullivan Could you assign the issue to me since I got assigned to the PR? Thanks!

@mollfpr offer sent
https://www.upwork.com/jobs/~01147597d87da7cb51

[@mollfpr] The PR that introduced the bug has been identified. Link to the PR:
[@mollfpr] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:

No offending PR. This is a visual improvement while transitioning.

[@mollfpr] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:

The regression step should be enough.

[@mollfpr] Determine if we should create a regression test for this bug.

Yes!

[@mollfpr] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.

Only available for web/mWeb

Make sure you're signed out of NewDot
On OldDOt, sign in to an account that has at least 1 existing workspace
After entering your magic code / OTP, tap on "Set up my company for free"
Verify A new tab opens to NewDot, and you get automatically signed in
Verify you do not see the "Create a new workspace" view for a brief moment before your existing workspaces get listed
👍 or 👎

👍  but small correction, this flow only happens on mobile web – not web, so the regression steps also only apply to mobile web.

Updated the test proposal. Thanks for the correction @francoisl 👍

Internal issue for regression testing opened here. @kevinksullivan can you take care of payment so we can close this please?

All set


chore: Typo in variable fixed

Variable renamed from dapService to dapsService

hey, thanks. I think you need to sign the ECA to contribute, then we're good to go :)

Thanks Paul for the hint. I signed the ECA.


Recommend storing current_user in assigns

Mithril's authentication system (powered by Authority) is a token-based system. You exchange a username and password for a token.
<EMAIL_ADDRESS>"password"})
# => %MyApp.Accounts.Token{token: "d88fb81c-1843-4696-932c-a5b5d779d854"}

You can then later use that token to verify and fetch a user's identity.
MyApp.Accounts.authenticate(token)
# => {:ok, %MyApp.Accounts.User{...}}

Current Recommendations
Mithril recommends that you store only this token in your Phoenix session. It specifically forbids you to store user IDs in the session.
In addition, Mithril currently recommends that you use the token instead of a user_id for function calls. This means that every function that needs to know which user is logged in would accept a token argument.
MyApp.CMS.create_page(token, params)

Internally, the function would have to exchange this token for a user_id, either by calling authenticate directly or using some kind of protocol.
# Using authenticate directly
def create_page(token, params) do
  with {:ok, user} <- Accounts.authenticate(token) do
    # ...
  end
end

# Using an Identity protocol
def create_page(identity, params) do
  with {:ok, user_id} <- Identity.user_id(identity) do
    # ...
  end
end

This adds a lot of complexity to functions, and it can be inefficient. Each function call revalidates and refetches the current user.
Proposed Recommendations

Continue forbidding storing current_user or current_user_id in the session
Encourage storing current_user or current_user_id in the assigns

This would have several benefits:

The logic app can still cancel sessions, because only tokens are stored in the Phoenix session, not the current user
Efficiency: domain functions don't have to accept tokens or refetch user ids. MyApp.CMS.create_page(conn.assigns.current_user.id, params)
Simplicity: it's more the way people expect to do things, and there's no real harm in it.
We could integrate rzane/authenticator, perhaps making it into authority_plug.

I think this new approach makes total sense. The current convention could mean loading the current user from the DB an unknown number of times. It also disrupts pipeline flows.
Two 👍  👍  from me.

I'd be happy to rename authenticator 👍

One the mithril-based project I've been working on, I've started following the second approach, after a fashion.
I think to ensure that the user is always loaded using the proper method (w/authentication), we could add a function to the phoenix side (e.g. authenticated_user/1) that pulls the user from the assigns.
def authenticated_user(conn) do
  conn.assigns.current_user
end

We could also add a requirement that any user passed to a domain function has a virtual attribute authenticated with a value of true, which can be set whenever a user is authenticated. So Repo.get(User, id) will not return an 'authenticated' user, and a user loaded in that method cannot be passed to a domain function requiring an authed user.
defmodule MyApp.Conversations do
  def archive_conversation(%{authenticated: true} = user, conversation_id) do
    # archive conversation
  end

  def archive_conversation(_user, _id), {:error, :unauthorized}
end

@danielberkompas Thoughts?
I've run into the pain points of passing tokens everywhere, and I like the second approach. There are really  two concerns at play regarding authenticating an action:

Ensuring that the user is authenticated
Ensuring that the user has permissions to perform the action

I think it makes sense to have domain functions only concern themselves with the second, as the first is handled by authority. An 'archive conversations' function should not need to know how to authenticate a user, for example, only whether any given user can archive a given conversation.

@silasjmatson good thoughts! I agree that domain functions should only concern themselves with permissions, not authentication.
For that reason, it probably isn't necessary to have a virtual authenticated attribute. As long as Phoenix isn't storing the user_id in the session, the domain function can be confident that the user it receives is being authenticated, because the only way Phoenix will have a user ID is if it authenticated the token.
I'm thinking we'll probably use rzane/authenticator once it's renamed to authority_plug for basic plugs that ensure that current_user is present in the assigns.

Seeking advice. Currently, Authenticator is a generic solution for this sort of Plug level authentication. It doesn't really care if you're using Authority or Guardian. As such, it feels weird to rename it to "Authority.Plug". I'm still not entirely opposed to the idea though, because I really like the idea of forming a community around authority.  🤷♂
However, I added Authenticator.Authority, which provides a macro that should handle the 90% case of using Authenticator in combination with Authority:

https://github.com/rzane/authenticator#usage-with-authority
https://github.com/rzane/authenticator/blob/3abe1b24140941bf01500c0a3e9f8d0d4d7f3e12/lib/authenticator/authority.ex

On second thought, I'm back to thinking renaming is a fine idea. Authenticator takes kind of a similar approach to Authority, so I think this seems fitting. And, I think I can rename Authenticator.Authority to Authority.Plug.Template.


gulp task 'library:build:esm' build failed on Windows 10 64bit and NodeJS 6.9.5

Bug, feature request, or proposal:
Bug: gulp task 'library:build:esm' build failed on Windows 10 64bit and NodeJS 6.9.5
What is the expected behavior?
build success
What is the current behavior?
when trigger the gulp serve:devapp, after invoking task 'library:build:esm'
it shows:
Error: Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:7:53: Cannot find module '.corea11ylive-announcer'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:8:66: Cannot find module '.corecoordinationnique-selection-dispatcher'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:8:88: Hexadecimal digit expected.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:9:89: Cannot find module '.coreoverlayoverlay-container'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:10:44: Cannot find module '.coreoverlaypositionoverlay-position-builder'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:11:103: Cannot find module '.coreoverlayposition
iewport-ruler'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:12:89: Cannot find module '.coreoverlayscrollscroll-dispatcher'.
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:13:32: Cannot find module '.coreoverlayscrollscrollable'.
ipple-renderer'.spaces/Frontend/material2/src/lib/index.ts:14:36: Cannot find module '.core
Error at D:/Workspaces/Frontend/material2/src/lib/index.ts:15:154: Cannot find module '.grid-listgrid-tile'.
What are the steps to reproduce?
On windows 10 64bit

clone the latest master branch of master2
npm install
gulp serve:devapp

the error will show up
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: https://goo.gl/DlHd6U
What is the use-case or motivation for changing an existing behavior?
Which versions of Angular, Material, OS, browsers are affected?
Is there anything else we should know?

I believe this is due to https://github.com/angular/angular/issues/15403; follow that issue for further updates.

@jelbourn - we are still getting this issue. Is it resolved?


Command and current path artifacts printed when used with Neovim terminal

I get the following output when I type echo foo and echo bar using the built-in Neovim terminal:
❯ echo "foo"
echofoo
..thbox/dbx-api
~/Code/Healthbox/dbx-api master
❯ echo "bar"
echobar
..thbox/dbx-api
~/Code/Healthbox/dbx-api master

I've narrowed the issue down to oh-my-zsh.
The problem doesn't occur in iTerm or Terminal.app, or anywhere else that I can find, but it does go away when I disable oh-my-zsh.

Hi, and thanks for reporting.
Were you, by any chance, able to track this down further, for instance link it to a particular plugin or theme?
Is the behavior happening for every command or just echo?

@ncanceill thanks for the quick reply!

Were you, by any chance, able to track this down further, for instance link it to a particular plugin or theme?

The problem occurs when I disable everything but oh-my-zsh itself.

Is the behavior happening for every command or just echo?

It happens with every command.
Please let me know if there's anything else I can do to help diagnose this issue.

I have to admit I am quite baffled here.
Can you please gist your ~/.zshrc for me (along with other startup files you may have) please?
Also paste the output of $SHELL --version.

@ncanceill - here is my .zshrc file. I've disabled all plugins except for oh-my-zsh. I am using antigen, but I don't think that is related because the problem goes away if I omit antigen use oh-my-zsh from the file.
Here is the output of $SHELL --version:
zsh 5.0.7 (x86_64-apple-darwin14.0.0)

I don't have any other startup files.
Thanks for your help with this!

Hi @lynndylanhurley, what theme are you using with oh-my-zsh? Also follow these instructions to generate a debugging file.

@mcaserta - I normally use pure, but the issue still occurs when I don't use any theme at all.
Thank you for linking to the troubleshooting instructions. Here is output from the debug log when I run it from the Neovim terminal.

Sorry, I can't make enough of that file with the command I gave you. Try this one instead:
zsh -xv &> >(tee ~/omz-debug.log 2>/dev/null)

Here is the result of that command.

On the last one you didn't run any command to reproduce the behavior, could you retry again?
Also, this might be related to auto-title functionality. Try running the same offending commands before and after running:
function omz_termsupport_cwd omz_termsupport_precmd omz_termsupport_preexec { }

and see what happens.

Updates?

I can partially reproduce. Looks like this is due to a bad $TERM value being set in your ~/.zshrc, not an OMZ or neovim issue.
It appears that your ~/.zshrc is forcibly setting the $TERM terminal type when it is run from inside tmux or nvim.
if [ "$TMUX" ] || [ ! $ZSHRC_LOADED ] || [ "$NVIM_LISTEN_ADDRESS" ]
then
   ...

  export TERM=screen-256color

If I put export TERM=screen-256color in to a fresh ~/.zshrc copied from the OMZ template, and run zsh in iTerm, then each command that I run results in the command name getting echoed as well, like you have.
> echo foo
echofoo

The problem is that the by forcing $TERM, your ~/.zshrc is causing programs, including OMZ's lib/termsupport.zsh, to use the screen-256color escape sequences. But nvim's terminal emulator, along with iTerm/Terminal (unless you have reconfigured them) emulate xterm-256color, which uses different escape sequences.
What you are seeing appears to be the terminal commands to update the window title with the running program's name, and change the title back to the trailing part of the current directory after the command finishes, but being issued with the wrong escape sequences, so the terminal doesn't interpret them correctly, and echoes the contents as text.
It is an error to set $TERM to a value that doesn't match your terminal. That value is usually supplied by the program that's launching the shell, so you don't need to set it yourself in most cases. Fix your ~/.zshrc to not set $TERM, at least not when running inside nvim, and this problem should go away.

@apjanke - that was the issue. I removed export TERM=screen-256color from my ~/.zshrc and now everything works as it should. Thanks!!!


[$500] [Payment card / Subscription] Implement “Change billing currency” action

Drafting cc @fabioh8010
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~011778e5bbd693de04
Upwork Job ID:<PHONE_NUMBER>632573440
Last Price Increase: 2024-03-19

This is not ready for proposals @ShridharGoel, we'll apply the help wanted label again when it is - Thanks!

Hi, I'm Fábio from Callstack - expert agency - and I would like to work on this issue.

Hi, I'm Nicolay from Callstack - expert agency - and I would like to work on this issue.

@MitchExpensify The Link in the doc was not proper - the proper task for this part of the work - https://github.com/Expensify/App/issues/38630
This issue is about the gluing of the  "Change billing currency" screen with "Your card section"

Today updates - start working on implementation for re-usable form (same will be for this one https://github.com/Expensify/App/issues/38630)
WIP

Today report - build re-usable form to use in both places for change currency. Tomorrow plan to add latest code peaces and open a PR for a review

@narefyev91 does not require payment (Contractor)
@situchan requires payment (Needs manual offer from BZ) ~I have sent you an job offer. please accept!

@situchan please also propose the regression test!

No regression test proposal needed I think. This is a new feature and we've written a suite of regression tests to be used by applause once the full project/feature is ready.

@zanyrenney I accepted invite.

waiting on @situchan accepting offer

payment summary
Contributor: @narefyev91 is from an agency-contributor and not due payment
ROLE: @situchan paid $(250) via Upwork - https://www.upwork.com/nx/wm/workroom/37844612/overview?nav_dir=pop


Make extracellular mechanism available

I'd like to be able to use CoreNeuron to simulate extracellular electrical stimulation, but this is currently not possible. Would it be feasible to make the extarcellular mechanism available in CoreNeuron, in order to allow these kinds of simulations?

@nrnhines: could you comment if this feature can be implemented and how much efforts this requires? Joseph Tharayil is working at Blue Brain and using NEURON+CoreNEURON for simulations including LFP,EEG calculations.

Extending CoreNEURON to implement extracellular is a fairly extensive undertaking (the tree matrix requires a general sparse solver). It is not needed for calculation of LFP and EEG as that can be done with https://nrn.readthedocs.io/en/latest/python/simctrl/cvode.html#CVode.use_fast_imem which is presently understood by CoreNEURON.  As to transforming i_membrane_ into an LFP or EEG calculation, that can be done by a mod file, but we would have to deal seriously with the GPU and Intel reduction issues that are mentioned in https://github.com/BlueBrain/CoreNeuron/pull/772#issuecomment-1032496400
I believe it would be better to implement extracellular (along with cvode and LinearMechanism) after integrating CoreNEURON back into NEURON where all calculation will be carried out with Structure of Array memory layout and pointer management will become automatic.

thank you Michael for quick feedback!

It is not needed for calculation of LFP and EEG as that can be done with https://nrn.readthedocs.io/en/latest/python/simctrl/cvode.html#CVode.use_fast_imem which is presently understood by CoreNEURON.

Just to clarify - we have already using necessary features for LFP computations. I was just mentioning features that he is using in his simulations.

Extending CoreNEURON to implement extracellular is a fairly extensive undertaking (the tree matrix requires a general sparse solver).

Ok. In that case we will have internal discussion to see what are alternatives considering the use case from Joseph.
@jjt20 : it would be great if we could sync internally on Slack. Tagging @WeinaJi and @jamesgking here.


Standalone AWS deployment server

Implements an AWS deployment engine as a standalone independent unit (decoupled from the nitric CLI).
This can be tested with a gRPC client (until we integrate it with the CLI).
If we port the remaining providers to this method this makes it possible to decouple our CLI from pulumi as well (could deliver pulumi within docker containers instead.
This also allows providers to be pulled as part of a stack definition (rather than being mapped directly to CLI implementations.
Example Stack file:
name: my-stack
provider: aws

Could potentially turn into something like:
name: my-stack
# Define remote engine to pull and execute (this doesn't have to be docker, just an example
# Mainly demonstrating that this opens to door to building completely custom providers, even in different
# languages/deployment tech
provider: docker://nitrictech/aws-cloud

name: my-stack
# Define remote engine to pull and execute (this doesn't have to be docker, just an example
# Mainly demonstrating that this opens to door to building completely custom provders.
provider: docker://nitrictech/aws-cloud

I like this idea for defining and loading providers

name: my-stack
# Define remote engine to pull and execute (this doesn't have to be docker, just an example
# Mainly demonstrating that this opens to door to building completely custom provders.
provider: docker://nitrictech/aws-cloud

I like this idea for defining and loading providers. If we go ahead with something like this, maybe the current providers (e.g. aws) can become aliases of something like docker://nitrictech/aws-cloud as a way to enable backwards compatibility?

that sounds perfect! easy to implement and explain.
The part I'm currently wrapping my head around is configuration for mapping to something like docker.
So if we're using something like terraform/pulumi in a docker container, ensure we properly respect locally configured settings (by mapping configuration directories from the host to the container). So if a user is deploying with a local backend their config doesn't get nuked everytime and existing stacks are properly updated.

Shouldn't we be able to define the provider per resource rather than per stack?
What if you wanted all built in resources, except this one?

Shouldn't we be able to define the provider per resource rather than per stack? What if you wanted all built in resources, except this one?

@asalkeld it depends on where control of the specific resources should be.
Also the idea of a single provider isn't mutually exclusive with swappable/pluggable resources, there are no reasons that a provider couldn't give multiple alternatives for a single resource type within the scope of that provider, that provider could even define its own interface for plugins as well and incorporate them into a build/wrapping process that dynamically includes its own plugins on request.
I feel like pulling all the complexity of individually pluggable resources forward into the stack level might just end up introducing an inner platform effect of configuration into stack rather than that configuration being defined within the scope of a concrete provider.
If you have some ideas on how we could achieve that level of pluggability with high cohesion that would be awesome, how do you think something like that would look?

Codecov Report
Base: 54.81% // Head: 54.81% // No change to project coverage :thumbsup:

Coverage data is based on head (bde8505) compared to base (348a405).
 Consider uploading reports for the commit 4781a95 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@                   Coverage Diff                    @@
##           feat/framework-refactor     #364   +/-   ##
========================================================
  Coverage                    54.81%   54.81%
========================================================
  Files                           24       24
  Lines                         1578     1578
========================================================
  Hits                           865      865
  Misses                         649      649
  Partials                        64       64

Flag
Coverage Δ

unittests
54.81% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Testing on apache libarrow-glib results in false positive result?

I'm trying to bind C (GObject based) interface of libarrow https://arrow.apache.org/install/
A manual partial binding already exist in nim repo: https://github.com/emef/nimarrow_glib
How I successfully compile an example file
$ pkg-config --cflags --libs arrow-glib
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -DUTF8PROC_EXPORTS -larrow-glib -lgobject-2.0 -lglib-2.0 -larrow
$ gcc -o build build.c $(pkg-config --cflags --libs arrow-glib)
$ ./build
length: 3
array[0] = 29
array[1] = 2929
array[2] = 292929

nimarrowglib.nim
import futhark

importc:
  absPath "/usr/lib/clang/10/include"
  path "/usr/include"
  path "/usr/include/glib-2.0"
  path "/usr/lib/x86_64-linux-gnu/glib-2.0/include"
  define UTF8PROC_EXPORTS
  "arrow-glib/arrow-glib.h"

nimarrowglib.nims
--passL: "-larrow-glib -lgobject-2.0 -lglib-2.0 -larrow"

Running (had to raise --maxLoopIterationsVM:100000000, default was 10000000)
nim c --maxLoopIterationsVM:100000000 nimarrowglib.nim
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.0/config/nim.cfg' [Conf]
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.0/config/config.nims' [Conf]
Hint: used config file '/home/jack/nim/nimarrowglib/nimarrowglib.nims' [Conf]
........................................................................................................
/home/jack/.nimble/pkgs/futhark-0.3.1/futhark.nim(479, 12) Hint: Running: opir -I/usr/lib/clang/10/include -I/usr/include -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include /home/jack/.cache/nim/nimarrowglib_d/futhark-includes.h [User]
/home/jack/.nimble/pkgs/futhark-0.3.1/futhark.nim(486, 8) Hint: Parsing Opir output [User]
/home/jack/.nimble/pkgs/futhark-0.3.1/futhark.nim(499, 8) Hint: Generating Futhark output [User]
/home/jack/.nimble/pkgs/futhark-0.3.1/futhark.nim(599, 10) Warning: imported and not used: 'macros' [UnusedImport]
CC: stdlib_digitsutils.nim
CC: stdlib_assertions.nim
CC: stdlib_dollars.nim
CC: stdlib_system.nim
CC: stdlib_times.nim
CC: stdlib_os.nim
CC: nimarrowglib.nim
Hint:  [Link]
Hint: gc: refc; opt: none (DEBUG BUILD, `-d:release` generates faster code)
83287 lines; 24.231s; 1.267GiB peakmem; proj: /home/jack/nim/nimarrowglib/nimarrowglib.nim; out: /home/jack/nim/nimarrowglib/nimarrowglib [SuccessX]

Apparently is a success! But no functions seems available (for example garrow_int32_array_builder_new()).
 Opir will read all symbols in your C project, but Futhark will only import those which are directly imported. This might seem a bit weird, after all it's not how C does imports, and I agree. But the reason why I did it this way was to give the user more control over what code to generate. While I was making Futhark I tried to build it in such a way that if something doesn't work you can simply override or ignore it, no need to mess around with the original C sources. In my early testing I had cases which failed when it simply generated code for everything that Opir output (not sure if this is still the case) so I implemented this way of importing as an escape hatch of sorts.
Futhark is now a lot more stable, and the cases where it fails to generate proper code based on some Opir output are fewer and fewer. Maybe it is time to switch to a "import everything and explicitly ignore the stuff you don't want" approach, it would certainly make it a lot more "it just works" than it currently is (at least if it is able to wrap everything properly). One of the issues is that this will create code for everything that is imported, and this includes the core C libraries.. Maybe it would be better to be able to specify a folder to allow imports from. That way you could specify arrow-glib as the folder, and then import arrow-glib.h and it should import all those other files for you.
Looking forward to feedback on this!

not all symbols seems imported correctly
my test.nim (where nimarrowbindings is symlink to futhark cache nim)
import nimarrowbindings

var error: ptr Gerror

var builder = garrow_int32_array_builder_new()

discard garrow_int32_array_builder_append_value(builder, 29, error.addr)
discard garrow_int32_array_builder_append_value(builder, 2929, error.addr)
discard garrow_int32_array_builder_append_value(builder, 292929, error.addr)

#[
  proc garrowarraybuilderfinish*(builder: ptr Garrowarraybuilder_452986635;
    error: ptr ptr Gerror_452987699): ptr Garrowarray_452986259 {.
    cdecl, importc: "garrow_array_builder_finish".}

  proc Garrowarraybuilderproc*(ptrarg: gpointer_452985237): ptr Garrowarraybuilder_452986635 {.
    cdecl, importc: "GARROW_ARRAY_BUILDER".}
  ]#
var myarray = garrow_array_builder_finish(Garrowarraybuilderproc(builder), error.addr)

error in linking:
nim c -r test.nim
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.0/config/nim.cfg' [Conf]
Hint: used config file '/home/jack/.choosenim/toolchains/nim-1.6.0/config/config.nims' [Conf]
Hint: used config file '/home/jack/nim/nimarrowglib/test.nims' [Conf]
............................................................
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(809, 23) Warning: use `char` or `uint8` instead; cuchar is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(6796, 5) Warning: guint8_452985884 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(809, 23) Warning: use `char` or `uint8` instead; cuchar is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(6796, 5) Warning: guint8_452985884 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(17413, 15) Warning: guint8_452985884 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(25089, 73) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(25236, 39) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(27729, 67) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(27884, 72) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(30329, 46) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(32501, 67) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(33968, 18) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(36209, 14) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(38943, 51) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(40482, 37) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(42845, 37) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(45714, 77) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(45816, 18) Warning: guint8_452985885 is deprecated [Deprecated]
/home/jack/.cache/nim/nimarrowglib_d/futhark_4DE1D3EA70247952.nim(2, 6) Warning: imported and not used: 'macros' [UnusedImport]
/home/jack/nim/nimarrowglib/test.nim(16, 5) Hint: 'myarray' is declared but not used [XDeclaredButNotUsed]
Hint:  [Link]
/usr/bin/ld<EMAIL_ADDRESS>in function `NimMainModule':
@mtest.nim.c:(.text+0x2ea): undefined reference to `GARROW_ARRAY_BUILDER'
collect2: error: ld returned 1 exit status
Error: execution of an external program failed: 'gcc   -o /home/jack/nim/nimarrowglib/test  /home/jack/.cache/nim/test_d/stdlib_digitsutils.nim.c.o /home/jack/.cache/nim/test_d/stdlib_assertions.nim.c.o /home/jack/.cache/nim/test_d/stdlib_dollars.nim.c.o /home/jack/.cache/nim/test_d/stdlib_system.nim.c.o<EMAIL_ADDRESS> -larrow-glib -lgobject-2.0 -lglib-2.0 -larrow  -ldl'

culprit seems Garrowarraybuilderproc -> GARROW_ARRAY_BUILDER of array-builder.h

After reading some C code I've endeded up with a working solution
GARROW_ARRAY_BUILDER macro expands to a cast, and this seems to be for all
static inline GArrowArrayBuilder * GARROW_ARRAY_BUILDER(gpointer object) {
    return G_TYPE_CHECK_INSTANCE_CAST(object, garrow_array_builder_get_type(), GArrowArrayBuilder);
}

This compiles and executes
import futhark

importc:
    absPath "/usr/lib/clang/10/include"
    path "/usr/include/arrow-glib"
    path "/usr/include/glib-2.0"
    path "/usr/lib/x86_64-linux-gnu/glib-2.0/include"
    define UTF8PROC_EXPORTS
    # "arrow-glib.h" # imports nothing by itself
    "arrow-glib/gobject-type.h"
    "arrow-glib/version.h"
    "arrow-glib/array.h"
    "arrow-glib/array-builder.h"
    "arrow-glib/chunked-array.h"
    "arrow-glib/codec.h"
    "arrow-glib/compute.h"
    "arrow-glib/data-type.h"
    "arrow-glib/datum.h"
    "arrow-glib/enums.h"
    "arrow-glib/error.h"
    "arrow-glib/expression.h"
    "arrow-glib/field.h"
    "arrow-glib/record-batch.h"
    "arrow-glib/scalar.h"
    "arrow-glib/schema.h"
    "arrow-glib/table.h"
    "arrow-glib/table-builder.h"
    "arrow-glib/tensor.h"
    "arrow-glib/type.h"
    "arrow-glib/file.h"
    "arrow-glib/file-mode.h"
    "arrow-glib/input-stream.h"
    "arrow-glib/output-stream.h"
    "arrow-glib/readable.h"
    "arrow-glib/writable.h"
    "arrow-glib/writable-file.h"
    "arrow-glib/ipc-options.h"
    "arrow-glib/metadata-version.h"
    "arrow-glib/reader.h"
    "arrow-glib/writer.h"
    "arrow-glib/file-system.h"
    "arrow-glib/local-file-system.h"
    # manually added as not included in arrow-glib.h
    "arrow-glib/basic-array.h"
    "gobject/gobject.h"

var error: ptr Gerror

var builder = garrow_int32_array_builder_new()

discard garrow_int32_array_builder_append_value(builder, 29, error.addr)
discard garrow_int32_array_builder_append_value(builder, 2929, error.addr)
discard garrow_int32_array_builder_append_value(builder, 292929, error.addr)

var myarray = garrow_array_builder_finish(cast[ptr Garrowarraybuilder](builder), error.addr)
g_object_unref(builder)

var n = garrow_array_get_length(myarray)
echo n

for i in 0..<n:
    var val = garrow_int32_array_get_value(cast[ptr Garrowint32array](myarray), i)
    echo val

g_object_unref(myarray)

output:
3
29
2929
292929

I just pushed a new version of Futhark, and it should solve the folder import issue. Your sample should now simply be:
import futhark

importc:
    sysPath "/usr/lib/clang/10/include"
    path "/usr/include/arrow-glib"
    path "/usr/include/glib-2.0"
    path "/usr/lib/x86_64-linux-gnu/glib-2.0/include"
    define UTF8PROC_EXPORTS
    "arrow-glib.h" # imports nothing by itself
    "arrow-glib/basic-array.h"
    "gobject/gobject.h"

var error: ptr Gerror

var builder = garrow_int32_array_builder_new()

discard garrow_int32_array_builder_append_value(builder, 29, error.addr)
discard garrow_int32_array_builder_append_value(builder, 2929, error.addr)
discard garrow_int32_array_builder_append_value(builder, 292929, error.addr)

var myarray = garrow_array_builder_finish(cast[ptr Garrowarraybuilder](builder), error.addr)
g_object_unref(builder)

var n = garrow_array_get_length(myarray)
echo n

for i in 0..<n:
    var val = garrow_int32_array_get_value(cast[ptr Garrowint32array](myarray), i)
    echo val

g_object_unref(myarray)

Unfortunately I'm unable to install arrow-glib on my machine because the package is unmaintained and out of date in AUR so I can't actually test this myself. Please open another issue if the above doesn't work.


Occasional stack overflow when using InventoryGrid

  Occasionaly this error comes up:

Originally posted by @filipinyo in https://github.com/peter-kish/gloot/discussions/41#discussioncomment-3833107
  Little bi more of the debugger errors:

  seems like it goes in the infinite loop occassionaly.

Originally posted by @filipinyo in https://github.com/peter-kish/gloot/discussions/41#discussioncomment-3833219
The issue seems to be related to inventory sorting when items are modified.  The sorting moves items around, modifying them, which recursively triggers the sorting again.

Should be fixed in a0f6ce9def81596fe5cd141b5e40cc7a2d275dee


Remove random_shuffle in favor of SliceRandom's shuffle

Motivation
As much as this approach works, we already have a shuffle function in SliceRandom. Some points in favor of SliceRandom's shuffle:

This code executes multiple swaps between two randomly selected indices in the array, SliceRandom::shuffle performs an optimized Knuth shuffle which does O(n) swaps
Some of these multiple swaps could also be redundant, which is less efficient in number of operations and randomness quality, becase multiple swaps also can potentially create a bias: since some elements can be swapped more times than others, we won't have every permutation be equally likely

Proposal
Use SliceRandom::shuffle instead
Test Plan
CI
Release Plan

Nothing to do / These changes follow the usual release cycle.

#2618  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @andresilva91 and the rest of your teammates on  Graphite

Merge activity

Oct 14, 11:41 AM EDT: A user started a stack merge that includes this pull request via Graphite.


Allow trailing , in macro arguments at 0.13.0-alpha.3

This is rejected:
counter!("processing_errors", 1,
    "component_kind" => "transform",
    "component_type" => "add_fields",
);

While this works:
counter!("processing_errors", 1,
    "component_kind" => "transform",
    "component_type" => "add_fields"
);

Used to work before at 0.12.

I think we can close this now with the latest alphas having this.