💭 iOS photo quality vs native camera

Question
Fist of all, thanks for the latest updates, they were great.
When comparing pictures taken with the example app of the repository against native images, I can see a difference on quality. This difference is not so noticeable without zooming, but once you zoom into some object, there is quite noticeable difference between the two pictures.
Im attaching a screenshots with the comparison when zooming on the lock and also the picture metadata for both cases. Left if Vision Camera Example, right is iPhone Native.
There is any way to get the same result on vision camera and in native?
Or maybe iOS is doing some post processing after capturing?
Im using back triple camera with iPhone 15 Pro
Although Camera Devices Back Triple Camera says picture is being taken in 4032x3024, as you can see in the screenshot, but actual image size is 2160x3840.

Thanks!
What I tried
No response
VisionCamera Version
4.1.0
Additional information

[ ] I am using Expo
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar questions in the issues page as well as in the discussions page and found none.

After capturing, iOS applies post-processing (which can be seen, as the photo takes a little time to load/improve in the gallery right after being taken).
If you want to reproduce the same thing with react-native-vision-camera, you would need to apply algorithms yourself to enhance the quality of the image after taking the shot.

There are two different formats used. iOS native Camera seems to shoot in 4284x5712 (5k+), and you selected a 2160x3840 (4k) format. This is a HUGE difference. You should be able to also select the 5k+ format in VisionCamera.
I can see that the iOS native Camera shot in P3, and VisionCamera shoots in sRGB. There is currently no option to change that, but I could add it. A bit of effort though.

And also; yes exactly what Jonathan said; I also think that they are doing some post-processing. That's not something a Camera library should do imo.


Fix README headers spacing

GitHub recently altered the Markdown parser to disallow for spaces on the "#" headers.
So a header "###Example" does not render:
###Example
...while a "### Example" does.
Example
This PR simply adds the necessary spaces on the headers so that the formatting is kept.

Thanks @Yuri-M-Dias !


Model predict get label name

training data as follows:
class,gender,age,fare,label
1,female,50.0,28.7125,died
1,female,2.0,151.55,died
1,female,25.0,151.55,died
2,female,29.0,26.0,survived
2,female,33.0,26.0,survived
2,female,36.0,26.0,survived
2,female,42.0,26.0,survived
test_data as follows:
class,gender,age,fare
3,male,24,7.1417
1,male,47,227
I use  tfdf.keras.RandomForestModel predict test data:
output = model.predict(test_ds)

[[0.6399997 ]
 [0.13999999]]

How can I get specific category names 'died' or 'survived' according to probability?

when I use sklearn RandomForestClassifier, the probability of two classifications will be output
prediction = model.predict_proba(vec.transform(data))
[[0.27861378 0.72138625]]
[[0.99693835 0.00306163]]

adv_args = tfdf.keras.AdvancedArguments(
    predict_single_probability_for_binary_classification=False
)

model = tfdf.keras.GradientBoostedTreesModel(verbose=0, advanced_arguments=adv_args)

I solved this problem by this code.


stdlib: fix compilation error when compiling with an old compiler

Fix wrong/missing check for $BuiltinUnprotectedAddressOf around the new "unprotectedAddressOf" builtins.
rdar://99713099

@swift-ci smoke test and merge


i don't have a phone number so I can't access Microsoft Fabric

I was wondering if I haven't explored all avenues for accessing Fabric.

did you get a chance to print out the paper? https://arxiv.org/abs/2312.17653


Do not build SwiftReflectionTest when we are not including tests.

What's in this pull request?

Resolved bug number: (SR-)

Before merging this pull request to apple/swift repository:

[ ] Test pull request on Swift continuous integration.

Triggering Swift CI
The swift-ci is triggered by writing a comment on this PR addressed to the GitHub user @swift-ci. Different tests will run depending on the specific comment that you use. The currently available comments are:
Smoke Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please smoke test
    All supported platforms     | @swift-ci Please smoke test and merge
    OS X platform               | @swift-ci Please smoke test OS X platform
    Linux platform              | @swift-ci Please smoke test Linux platform

A smoke test on macOS does the following:

Builds the compiler incrementally.
Builds the standard library only for macOS. Simulator standard libraries and
device standard libraries are not built.
lldb is not built.
The test and validation-test targets are run only for macOS. The optimized
version of these tests are not run.

A smoke test on Linux does the following:

Builds the compiler incrementally.
Builds the standard library incrementally.
lldb is build incrementally.
The swift test and validation-test targets are run. The optimized version of these
tests are not run.
lldb is tested.

Validation Testing
    Platform     | Comment
    ------------ | -------------
    All supported platforms     | @swift-ci Please test
    All supported platforms     | @swift-ci Please test and merge
    OS X platform               | @swift-ci Please test OS X platform
    OS X platform               | @swift-ci Please benchmark
    Linux platform              | @swift-ci Please test Linux platform

Lint Testing
    Language     | Comment
    ------------ | -------------
    Python       | @swift-ci Please Python lint

Note: Only members of the Apple organization can trigger swift-ci.

This avoids a cmake warning.

@swift-ci Please smoke test

@bitjammer Hey Dave. What do you think of this?

Seems fine!

@swift-ci Please smoke test

@swift-ci Please smoke test linux

The linux test failure here is unrelated.


Clickable zone (like screen markers)

It would work like screen markers plugin, except clicking anywhere not inside the marker will have no clicking (as if clicking on part of menu where nothing happens). Im mostly interested in this when using remote desktop app while doing afk things, since this activity requires me to click on same spots the marker stays, but clicking anywhere else leads my character to moving and ruining it's afkness.

Jagex's recent statement prohibits Any movement or resizing of click zones for 3d components, which this would fall under


[COURIER] Shipment should not be edited multiple times

After first edit, the shipment holds the bill object and attached documents (attached by COURIER, and shipment comments).
The edit Shipment button should not be available, instead an "Add message" button should be available

@Mastaleru "Add message" -> "Add Shipment Comment"?


Bug: Threads.Notify undefined column

Comments Not Loading and Uneditable in APT Due to Migration Issues
Summary:
There's an issue in the front-end of the APT application, preventing comments from loading and being edited. It appears that the migration for these columns may have been deployed and then reverted in the same commit. The related migration files can be found here:

Migration Deployment
Migration Revert

Potential Solution:
To investigate and resolve this issue, perform the following steps:

Run the migration on the staging site to verify that the migration works as expected and does not cause any unintended side effects.
Once the migration has been successfully tested on the staging site, apply the migration to the MCP deployment instance.
Test the comment loading and editing functionality in the APT application to ensure that the issue has been resolved.

After these steps, the comments should load and be editable as expected in the APT application.

Sqitch notes (may be outdated)
1. View past migrations:
To see the list of past migrations and their current status, use the sqitch log command:
2. Test the migration on the staging site:
To test the migration on the staging site, first, ensure that you have the latest version of the codebase and then deploy the migration using the sqitch deploy command:
After running the migration, verify that the migration works as expected and does not cause any unintended side effects.
3. Apply the migration to the MCP deployment instance:
If the migration works as expected on the staging site, apply it to the MCP deployment instance using the sqitch deploy command:
4. Test the comment loading and editing functionality:
After applying the migration, test the comment loading and editing functionality in the APT application to ensure that the issue has been resolved.
5. Rollback the migration (if necessary):
In case the migration causes any issues or you need to roll it back for any reason, use the sqitch revert command:
Replace TARGET_NAME with the appropriate target name (e.g., staging or production).
After these steps, the comments should load and be editable as expected in the APT application.

I'll go through these steps on the MCP instance to set up comms with the DB in MCP.
https://sqitch.org/docs/manual/sqitchtutorial/

To fix the immediate issue, we should run the latest migrations against the production db. I think the db/sqitch bash script documented above is outdated. But I'm fairly sure we can just use the sqitch command shipped with ubuntu like we do in the setup script.
For a long term fix, we should make sure that the migrations run automatically during deployment. But since our RDS databases can be accessed from whitelisted IP addresses only, I can see 2 options:

We can whitelist the ip address range used by github actions (for automated deployments through the ci) and the ip of our jumpbox (for manual deployments) and include the sqitch migration command in the ci and deployment script.
Or we can add a lambda backed custom resource to the CDK deployment script that will run the migrations inside a lambda during cdk deploy

Hey there!
In regards to the sqitch executable in the db/ folder:  Another option would be to use the sqitch docker image with a custom command to specify the location of the database against which to run the migrations, that way you can use the same docker container to boostrap the locally running running API using docker-compose or use the container as a standalone tool to run the migrations in any of the deployed stacks.
In regards to running the migrations during deployment: I had opened issue #452 but never got around to it. The list of IP addresses used by Github CI can be found in the github API's /meta endpoint (under actions). I had a discussion with MCP folk about opening the VPC to a specific set of IPs and I think the weren't too keen on it due to the huge number of IPs in the list. I would definitely recommend the lambda backed custom resource route - it's very much a solved problem, @edkeeble has even published a CDK construct for it!

Thanks @leothomas! That's super helpful! I think it's totally understandable that MCP admins are hesitant to add too many ips to the whitelist. I'll go ahead and implement option 2 (a lambda backed custom resource to run migrations automatically)

Checking to see if this is ready to close @sunu

Yea @naomatheus, it's safe to close now. I'll handle the long term fix in https://github.com/NASA-IMPACT/nasa-apt/issues/452.


certrotation: protect metrics regstration with sync.Once

/hold
this can try to come back in 4.2

We're reverting in https://github.com/openshift/library-go/pull/346

/close
not needed anymore


Apply the layout twice: main layout + layout of a specific div

Hi,
I'm using Mui Treasury Layout on my app and I have the following question: is it possible to use it twice, for the layout of the app and also for the layout of a specific div?
I'm trying to apply the layout to a specific div, but I have a problem with the sidebar getting a fixed position and therefore covering the main sidebar. Any suggestions?
Many thanks in advance.
Mauro

@siriwatknp  I noticed that activity on the repository in the last few months has dropped, hope you are healthy.
Is there any workaround I can adopt to achieve what I want (eg. forcing style on elements)?
Many thanks in advance,
Mauro

I don't quite get the picture, do you have any repo or sandbox or design that I can look at?


Added poetry to manage dependencies, made directory structure.Removed…

… move_certificate() function as its not needed now.
pyproject.toml should be filled out by the auther : authors = ["Your Name<EMAIL_ADDRESS>

Looks good
thank you 👍


Populate the initial views with the file data

Now that the file metadata is handled by the backend, populate the frontend views with the metadata and the addresses, physical and logical views.
Sending data from the VS Code extension to the webview is done via message passing as follows: https://code.visualstudio.com/api/extension-guides/webview#passing-messages-from-an-extension-to-a-webview

Here are my thoughts on the data population...
The extension starts off with a reference to a file, which it opens and collects various metrics like the file size, mime type, and ASCII character count.
The webview is the expert on the slice of data it needs, and the desired encoding, so once the webview loads, the extension should pass it the file metadata, which then the webview requests a slice (offset and length) and the desired encoding.  The extension then sends a message with the requested data slice in the requested encoding.

Here are my thoughts on the data population...
The extension starts off with a reference to a file, which it opens and collects various metrics like the file size, mime type, and ASCII character count. The webview is the expert on the slice of data it needs, and the desired encoding, so once the webview loads, the extension should pass it the file metadata, which then the webview requests a slice (offset and length) and the desired encoding. The extension then sends a message with the requested data slice in the requested encoding.

I think I may need clarification on what sources you're mentions as extension and webview. Currently, I have the webview.ts reading the file data after the file selection has been made. It then packages a message to post to the svelte webview panel. I moved some of the helper functions from the svelte side and into the extension side. I used these functions to formulate the webview.ts messsage to post:
      this.panel.webview.postMessage({
        command: 'loadFile',
        metrics: {
          type: fileExtensionType(this.fileToEdit),
          size: data.length,
          asciiCount: 0 //TODO
        },
        display: {
          physicalOffset: makeOffsetRange(16, 2),
          physical: encodeForDisplay(data, 16, 16),
          address: makeAddressRange(0, Math.ceil(data.length / 16), 16, 16)
        }
      });

In my head I was refering to the webview as the svelte display HTML and the extension as the webview.ts. Are we thinking the same thing?


Image placeholder could have the same size as target image

I don't know if it could be enhanced, but currently in the blog post where {<< img src="" width="" >>} is used (also if height is precised) the placeholder displayed when a page is rendered is much bigger (higher) than the target image. Those hundreds of millisecond (depending on the network connection) when the image is being loaded doesn't look (text is moved back when the image is loaded).
Maybe it could be tweaked with some CSS magic?

It seems that the placeholder is also displayed for a moment if my browser has that image cached (there is request for it which returns 304). Could be the image displayed from cache right away in that case?

the placeholder displayed when a page is rendered

Did you mean this placeholder image?

I made some changes. But the placeholder image is square, and most of the image is rectangular so I think we can't avoid the mismatch of the height.
Regarding cache, I added option lazy
{{< img src="" lazy="false">}}

Since, I use the lazysizes for lazy loading images, the library not support serving cached image. So, if we disable lazy image load, the rest of a job for serving cached image is the responsibility of the web browser

I made some changes. But the placeholder image is square, and most of the image is rectangular so I think we can't avoid the mismatch of the height.

Thanks for clarification. The rectangular image was the issue. However, having that knowledge I am able to start defining height instead width for my in-post images and now it is almost invisible (the most destructive effect was moving the text under the image back and forth).
Regarding laziness, it in fact prevents a place holder to be displayed (although, with the above finding it is less important). However, for {{< img src="" >}} laziness seems to be disabled by default. From your comment I understand it has to be disabled on demand. Have you done it intently?

Hmm, I'll test it more after I get home.
It should be disabled on demand.
Something went wrong on the img shortcode

There might also some something on my side, but I don't see a placeholder with the clear img shortcode. Firefox 73.

Can you check again?

Currently img by default doesn't show a placeholder. However, if I add lazy="false" or lazy="true" (it doesn't matter) it starts.

I misunderstood 😅
It's fixed now

:+1:


Berlin Meetup

It would be great to have a more regular Berlin meetup. I know that there have been events in the past - at Blueyard, the DApp meetup, etc - and there seems to be a vibrant community of IPFStronauts in Berlin. Would anyone be interested in more of a monthly get together, hack meeting, possibly with speakers?
cc @lgierth @hsanjuan @haadcode

Organizing a meetup is not a pleasurable task (for me at least). It is very hard to do a good meetup. It should a purpose and add real value to IPFS other than socializing (IMHO).
So the question is: does the IPFS community in Berlin need such a space? Apart from the logistic effort (venue, drink, food, advertisement, chasing down speakers, agenda), can Berlin provide [monthly] content for those meetups? Is that content going to come from other people than the core contributors? Are there interested players which could sponsor (providing venue, content etc) such meetings?
I am not sure about those things. Sure some people can just hang out and hack, but this usually does a bad job in reaching new audiences or making an impact that justifies the organizational effort.
This is my personal impression right now, but my points are arguable anyway so I could be convinced it is a good idea if people see it differently. 😸

I am not sure about those things.

I am not sure, either. However, I suspect that Berlin may be the best place to do this, currently. And I want to know if others think so, too. I feel that all of the points you raise can be sufficiently met given the right amount of interest.

I think we can shrink the overhead of organizing quite a bit by having it at c-base.

c-base has a bar, for meetups in the past we simply provided EUR200 worth of drinks via the bar.
I don't think we need food.
c-base has all the equipment needed, and it has worked very well in the past. (chairs, projector, cables, microphones, etc.)
At least 3 longtime c-base members are active in the IPFS community, and there's an amazing physical IPFS installation there :)

Content-wise, I think we can definitely come up with at least 1 presentation from the core of IPFS per month. It doesn't have to be as polished as e.g. the devcon presentations. And I'd personally be very happy with presentations that aren't specifically about IPFS or its APIs, other p2p projects do just as interesting stuff, and I'd also love for more people to talk about the politics of the internet+web.
There are also plenty of formats for meetups to pick from and try out :)
Anyhow, this would leave us with the following recurring tasks:

coordination with c-base
moderating the meetup
acquiring presentations

And I'd personally be very happy with presentations that aren't specifically about IPFS or its APIs, other p2p projects do just as interesting stuff, and I'd also love for more people to talk about the politics of the internet+web.

So this would be something like "ipfs + p2p + x"

We can probably also get most of the presentations recorded. @copernicus-mogley has a track record of making excellent recordings and live streams from ipfs meetups and other events in the p2p community.
That way there could be a benefit beyond the local community in berlin.

I can certainly record and stream a meetup at any time, though I should point out that C-Base has equipment and skills to livestream as well. It might be easier and better quality for them to do it. In any case, I'd love to attend an IPFS meetup and visit with you all.

I can certainly record and stream a meetup at any time, though I should point out that C-Base has equipment and skills to livestream as well. It might be easier and better quality for them to do it. In any case, I'd love to attend an IPFS meetup and visit with you all.

@jbenet
I am in Berlin and hacking on multi / ipfs.
Very interesting what's happing.
Did the Berlin move forward ?

I think a presentation on the multi stack is needed. It's amazing tech and getting to understand the use cases envisioned with this is a god starting point to get to see why it's useful to use it

thanks-- i dont think this happened, but there's enough interest for something regular.
I haven't yet given talks about multiformats specifically, but i usually wrap it up in other talks. eg https://www.youtube.com/watch?v=soUG72j7kB0

I am currently in Berlin, and will be until May 12th.
Anyone fancy an IPFS Un-meetup meetup at a nice bar? Decentralized web chats?

Hi all
There is an informal meetup in Berlin next week with some discussion of use-cases for how IPFS might apply in the context of preserving research data: https://www.meetup.com/ScienceB0/events/248464271/. It is being organised by the "Blockchain for Science" group, but the emphasis should be on distributed storage and dweb rather than blockchain. It would be great to have some core IPFS/Protocol devs there, @hsanjuan @lgierth @haadcode?
@hsanjuan ipfs-cluster is extremely relevant to the use case and we've been experimenting with it on a small scale. It would be great if you could make it and possibly give a brief overview of ipfs-cluster or just discuss our use case and whether we're on the right track?
Apologies for short notice on this! A few of us are in Berlin for the Research Data Alliance conference (wednesday - friday), so took the opportunity to meetup with some of the Berlin dweb community.
Cheers
cc @flyingzumwalt @diasdavid @aboynejames @parfenov @soenkeba

@eocarragain I'll be there. It will be really good to know your use cases first hand!

@hsanjuan brilliant - thanks!

Hi all. I'm starting new meetups in Berlin, next in October 11th, 2018. Check #339


resolve issue-461

Resolve issue-461

A duplicate PR has been created. So closing this PR- https://github.com/openBackhaul/ApplicationLayerTopology/pull/500


Resolve bug in group details update

Fixes #1161.
Changes in this pull request:

Fixed a bug for group details update. A super user can now edit group details (name, email and description) without being part of the group and without removing existing members of the group.

This doesn't seem to be working for me anymore in my local environment, when I try to update a group email it's still removing the other group members. @JoshSEdwards @Aravindh-Raju @Jay07GIT could you guys try it out and see if it works for you?

This doesn't seem to be working for me anymore in my local environment, when I try to update a group email it's still removing the other group members. @JoshSEdwards @Aravindh-Raju @Jay07GIT could you guys try it out and see if it works for you?

I also just tried it out locally and it seemed to work fine. See the screenshots, after email update, the group members are the same.

This doesn't seem to be working for me anymore in my local environment, when I try to update a group email it's still removing the other group members. @JoshSEdwards @Aravindh-Raju @Jay07GIT could you guys try it out and see if it works for you?

I found out there was an NPM error causing issues with my local environment, with that resolved the change works as expected.


add dynamically configurable responses

e.g. take a look at this
need

trigger phrase(s)
response(s)
rarity of response

multiple rarity levels / response groups

ordered responses


Using Presets in multiple rigs

Hi! This is not really a bug, but something that is being problematic in my actual project. I'm working with multiple rigs I'd really like to have a system that let me separete the saved presets by each rig, right now I have to see all the presets for all my rigs at all times and it's a little tricky.
I'm not into script myself, for that reason really apreciate this addon, thanks a lot!

yes, I think it's a nice solution to the problem. Also I was thinking, becouse I always use custom properties on my rigs, to be able to control the snaps by drivers. However, I don't know how difficult is to do that.

My addon registers two operators, SnapIKToFKOperator and SnapFKToIKOperator. You can call these operators as long as the following attributes are present:
bpy.context.scene.armature_name,
bpy.context.scene.FK_control_upper_name,
bpy.context.scene.FK_control_lower_name,
bpy.context.scene.FK_control_end_name,
bpy.context.scene.IK_control_upper_name,
bpy.context.scene.IK_control_lower_name,
bpy.context.scene.IK_control_end_name,
bpy.context.scene.IK_control_name,
bpy.context.scene.IK_control_pole_name

Hi! This is not really a bug, but something that is being problematic in my actual project. I'm working with multiple rigs I'd really like to have a system that let me separete the saved presets by each rig, right now I have to see all the presets for all my rigs at all times and it's a little tricky.
I'm not into script myself, for that reason really apreciate this addon, thanks a lot!

For the meantime, you could prefix your preset with the name of the armature to keep the presets seperate from each other in the preset menu. It might be cluttered at this stage but you'll be able to keep different presets.

I'll do as you say. Thanks!!


(materialised) views are created before the tables

When the database has (materialised) views, setting up the database throws an error because the underlying tables do not exist.
This issue is happening for alembic_utils versions >= 0.2.16 and it gives this error message:
File ".../lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "my_table" does not exist
LINE 26:         FROM my_table
                      ^
....

(Background on this error at: http://sqlalche.me/e/13/f405)

I tested the same schema with versions <= 0.2.15 and it works fine.

There isn't enough information for me to attempt to reproduce
please include:

Does this issue still exist in the latest version alembic_utils==0.5.3?
Does this error when the migration is being generated, or when you attempt to apply the migrations to the database?
postgres version
python version
a table definition and a materialized view definition i can use to reproduce the error

Closing for inactivity
Please re-open if you'd like to continue


feat(evm): tracer option is read from the config

Does this close https://github.com/NibiruChain/nibiru/issues/1919?


[WIP] Support for live previews of Pages

This PR adds initial support for a "Preview" tab that presents a live preview of the page currently being edited on the frontend without requiring the page or changes to be saved first.

TODO:

[x] Dynamically add all fields on the Page form (even those that have been dynamically added by other plugins) to the dependsOn array of the preview field
[ ] Thoroughly test editing all content types
[x] Test switching layouts
[x] Ensure that change events are triggered by the richeditor
[ ] Perhaps make the "Preview" button in the top bar open the preview page in a new tab (will need to make sure that the preview logic does a session::get() instead of session::pull() in that case, could also make the preview button an AJAX request so that the latest form data is forced to be saved to the session before the URL is returned, that's probably the safest bet)
[ ] Improve the styling of the preview tab / controls
[ ] Improve the Workshop theme's test cases for static pages
[ ] Adjust the sizing of the iframe per device to be more accurate to the device buttons, perhaps also allow for dynamic zooming of the iframe content window so that it will show exactly how it'll look on the desired device, regardless of the size of the user's screen.

I've created a plugin that allows just that:

https://gitlab.com/studiobosco/wintercms/pagepreview
https://packagist.org/packages/studiobosco/wn-pagepreview-plugin

@der-On cool! Do you have a screenshot of what the UX looks like?

@der-On cool! Do you have a screenshot of what the UX looks like?

Sure:

The plugin also allows to add a preview to any model form with a single line of php code in the plugins register method.

@LukeTowers, just pushed a commit to prevent the preview being displayed by default so that people have to "opt in" to using it, any thoughts?
Also I fixed some display issues.
Is there anything else that needs doing before this can be merged?

@jaxwilko any reason why it should be opt-in instead of just always enabled?

@LukeTowers not really, just thought it would make sense for people to opt in

Meh, if it's opt in no one's going to see it. If you can think of a reason why someone would want it disabled then we could look at providing that as an option but at the moment I'm fine leaving it always enabled.

@jaxwilko todo before we can merge:

thorough testing of the editing all content types because of the refactor from controller to objecthelper
look into the todo (https://github.com/wintercms/wn-pages-plugin/pull/19/files#diff-30c49f6ba6a738fbd3fed3c4e7ddacb319ace9478f52ac56ecc56bec67e32ebfR68)
remove the config option
remerge in latest main

@LukeTowers all of there points have now been complete, tested everything and it all looks good, just need to add unit tests


Wrong result using .ONNX in ML .NET

Hi,
I'm trying to make a prediction with the ONNX midas model.
About the input i'm resizing my image, and scaline / offsetting (but i don't know if i am using the right values as there is no documentation, and values are not the same depending where we look at).
My prediciton is made this way :
public Bitmap predict()
        {
            var mlContext = new MLContext();
            var imageList = new List<ImageInputData>();
            var dataView = mlContext.Data.LoadFromEnumerable(imageList);

            //Here we resize image, offset and scale each channels and apply "model-f6b98070.onnx" model.
            var pipeline = mlContext.Transforms.ResizeImages(resizing: ImageResizingEstimator.ResizingKind.Fill, outputColumnName: "ImageResized", imageWidth: ImageSettings.imageWidth, imageHeight: ImageSettings.imageHeight, inputColumnName: nameof(ImageInputData.Image))
                .Append(mlContext.Transforms.ExtractPixels("Red", "ImageResized", colorsToExtract: ColorBits.Red, offsetImage: 0.485f * 255, scaleImage: 1 / (0.229f * 255)))
                    .Append(mlContext.Transforms.ExtractPixels("Green", "ImageResized", colorsToExtract: ColorBits.Green, offsetImage: 0.456f * 255, scaleImage: 1 / (0.224f * 255)))
                    .Append(mlContext.Transforms.ExtractPixels("Blue", "ImageResized", colorsToExtract: ColorBits.Blue, offsetImage: 0.406f * 255, scaleImage: 1 / (0.225f * 255)))
                    .Append(mlContext.Transforms.Concatenate(onnxModel.ModelInput, new string[] { "Red", "Green", "Blue" }))
               // .Append(mlContext.Transforms.ExtractPixels(outputColumnName: onnxModel.ModelInput, inputColumnName: onnxModel.ModelInput, interleavePixelColors: false, scaleImage: 1 / 255.0f))
               .Append(mlContext.Transforms.ApplyOnnxModel(modelFile: onnxModel.ModelPath, outputColumnName: onnxModel.ModelOutput, inputColumnName: onnxModel.ModelInput))
               ;

            var model = pipeline.Fit(dataView);

            //Just watching data. it is a tensor [1x3x384x384]
            ConsoleHelper.ShowDataViewInConsole(mlContext, dataView);

            //My image to predict.
            ImageInputData imgToTst = new ImageInputData()
            {
                Image = new Bitmap("C:\\Users\\Thibaud\\Desktop\\ml\\dataset\\data\\demo\\img_nyu2.png")
            };

            var predictEngine = mlContext.Model.CreatePredictionEngine<ImageInputData, OutputImageData>(model);

            //Get my prediction
           var pred = predictEngine.Predict(imgToTst);

            //As prediction ouptut is a big  1D float[] and values are not between 0-255, we normalize values between 0-255
           var formatted = formatOuput(pred.Output);

            //And then, we reconstruct bitmap from the 1D float[] array;
              var bitmap1 = FromDoublesToGrayscal(formatted, ImageSettings.imageHeight, ImageSettings.imageWidth);

            return bitmap1; //I should get a grayscal image with depth here !
        }

OutPut is :

I don't know what is wrong.
For the output, i'm reconstructing the bitmap from a 1D array given by the network (as seen in an other issue here) :
public static float[] formatOuput(float[] fin)
        {

            //From :https://github.com/intel-isl/MiDaS/issues/81
            var min = fin.Min();
            var max = fin.Max();

            for(int i = 0; i < fin.GetLength(0); i++)
            {
                fin[i] = (fin[i] - min) / (max - min);
            }

            var maxForm = fin.Max();

            for (int i = 0; i < fin.GetLength(0); i++)
            {
                fin[i] = (int)(fin[i] * 255 / maxForm);
            }

            return fin;

        }

        public static Bitmap FromDoublesToGrayscal(float[] doubles, int Width, int Height)
        {
            Bitmap newBitmap = new Bitmap(Width, Height);

            for (int j = 0; j < Height; j++)
            {
                for (int i = 0; i < Width; i++)
                {

                    newBitmap.SetPixel(i, j, Color.FromArgb( 0, (int)doubles[i + j], (int)doubles[i + j], (int)doubles[i + j]));
                }
            }
            return newBitmap;
        }

If someone see what is wrong i would be grateful,
Thanks

There are a lot of  midas .onnx models. I'm giving a try to each of them but none of them have inputs documentation.

https://github.com/intel-isl/MiDaS/pull/35

https://github.com/intel-isl/MiDaS/pull/35

https://github.com/intel-isl/MiDaS/tree/master/tf (the official which seems to be completly outdated)

There are a lot of  midas .onnx models. I'm giving a try to each of them but none of them have inputs documentation.

https://github.com/intel-isl/MiDaS/pull/35

https://github.com/intel-isl/MiDaS/pull/35

https://github.com/intel-isl/MiDaS/tree/master/tf (the official which seems to be completly outdated)


Removing Specta from dependencies

I'm using a forked version of Specta repository (Taptera/specta). Putting Specta in ios-snapshot-test-case-expecta podspec file results in target has frameworks with conflicting names: Specta. error.
As this project is a FBSnapshots extension for Expecta there is no need to add Specta to it.

It uses Specta to generate test case names, CocoaPods will resolve that version of Specta to your fork fine, what does your podfile look like? ( Also why fork, I don't see anything unique to your Specta fork? )

In the fork I use there is an additional block called action which is executed between beforeEach and it - I like to use it as it allows to extract the part of code that you're testing in each describe \ context block from configuration & preparation which is handled in beforeEach.
My Podfile:
platform :ios, '9.0'

use_frameworks!

source 'https://github.com/CocoaPods/Specs'
source 'https://github.com/Taptera/cocoapods-specs'

target 'ProjectName' do

target 'ProjectNameTests' do
    inherit! :search_paths

    pod 'Specta-Taptera'
    pod 'Expecta+Snapshots'
end
end

After running pod install I get:
Downloading dependencies
Using Expecta (1.0.5)
Using Expecta+Snapshots (3.0.0)
Using FBSnapshotTestCase (2.1.1)
Using Specta (1.0.5)
Installing Specta-Taptera (1.0.5)
[!] The 'Pods-CallrestoBusinessTests' target has frameworks with conflicting names: Specta.

Hrm, ideally you should be using pod "Specta", :git => "https://github.com/Taptera/specta.git" here, by re-naming Specta you're gonna get into problems like this, as we don't have real namespacing in objc/swift.
I'm not removing the Specta dep ( in CP 1.0 you cannot do faux weak-linking anymore ) as the features it offers are basically 90% of my use case for the library.

Ok, thank you!


multivar support for Config

Are there plans for adding multivar support for Config?
https://github.com/libgit2/libgit2/blob/master/src/config.c#L1032-L1043
Perhaps something like set_multivar(name: &str, regexp: &str, value: &str) or set_multivar_string?

Sure! If an API is omitted from git2-rs it's almost guaranteed not to be intentional, just haven't gotten around to it yet :). Would you be interested in sending a PR for this?

Sure, I'd be happy to give it a shot! Does set_multivar work for the function name?

Sure yeah, I typically just mirror libgit2

So it was pretty easy to implement, but I don't have the rust-fu to figure out how to test it.
    cfg.set_multivar("foo.bar", "^$", "baz").unwrap();
    cfg.set_multivar("foo.bar", "^$", "qux").unwrap();

The above works, but I can't get the borrow checker to play nice with making a collection of values from ConfigEntries. Any suggestions?

Ah it's ok to omit tests, it's likely just a small shim around the libgit2 function anyway.
Want to send a PR so I can take a peek at the error and some code?

Done.
The error was when I was trying to test it - I couldn't figure out how to make a slice/vec of entries from ConfigEntries (to make sure that multiple values for a single name are present in the config) while appeasing the borrow checker.

Non-compiling test: https://github.com/kejadlen/git2-rs/blob/40d0978996dcf67b068a1e9a0953a099286b0395/src/config.rs#L559-L565

Added in https://github.com/alexcrichton/git2-rs/pull/170


Update slf4j-nop to 1.7.31

Updates org.slf4j:slf4j-nop from 1.7.25 to 1.7.31.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.slf4j", artifactId = "slf4j-nop" } ]

labels: library-update, semver-patch

Superseded by #139.


WIP: Allow multiple inputs

Hi!
I'm exploring using your project as I migrate away from Ansible to a more static config generation tool that I can keep track with git. One of the features I enjoyed in my setup in ansible was a way to combine multiple directories into one when templating, using with_filetree. I.e., for HomeAssistant, I'd have a "common" directory that is distributed to all servers, and a "per-server" directory with additional files for just that server. I could probably make something work with globs here, but I think this might be slightly more elegant.
I've only just barely tested this with the included test pattern, using
[makejinja]
input = "./input"
output = "./output"
data = ["./config"]
loaders = ["loader:Loader"]
additional_inputs = ["./input"]

[makejinja.delimiter]
block_start = "<%"
block_end = "%>"
comment_start = "<#"
comment_end = "#>"
variable_start = "<<"
variable_end = ">>"

❯ makejinja
Remove 'output' from previous run
Skip 'input/empty.yaml.jinja'
Copy 'input/ui-lovelace.yaml' -> 'output/ui-lovelace.yaml'
Render 'input/views/home.yaml.jinja' -> 'output/views/home.yaml'
Skip 'input/empty.yaml.jinja'
Copy 'input/ui-lovelace.yaml' -> 'output/ui-lovelace.yaml'
Render 'input/views/home.yaml.jinja' -> 'output/views/home.yaml'

I could probably clean it up, but figured I'd share a WIP first to see thoughts.

I saw that you added a second commit that uses a different handling of multiple inputs with Jinja's FileSystemLoader. In the newer revision, you pass it a list of input folders which should make the app faster. At the same time, I think this may lead to issues since Jinja only uses relative paths afterwards. As such, if we have a file like dashboard.yaml in both input1 and input2, only one dashboard.yaml would be rendered. At least that is what I understood from the documentation.

This was actually the intention! The idea is that you can run makejinja multiple times. I.e., have
input/common/
input/server1/
input/server2/
output/server1/
output/server2

with a common .makejinja.toml config that just specifies common, and then run it once each with an additional input, outputting to server1 or server2. This way you can specify server specific overrides or additional files (or even remove files by putting an empty template in, with the skip_empty option!).
I got it working for my purpose, and I'm pretty happy with it!

Thank you for the update, I am happy to see that it is working for your use case. I will review your PR sometime this week.
One question straight away: Could you briefly describe how you use the exclude_pattern? I fear that the input_pattern being a glob and the exclude_pattern being a list of regexes is somewhat confusing to users. So to make an informed decision on that matter, I would like to get your perspective on this option 😄

Fair enough - I defaulted to regex, as someone who is very comfortable with regex, in order to have its power! But in my use cases, I only ever needed glob. Conveniently, the Path objects we are working with as a match function. We can just use that. I also flip-flopped some loop logic to reduce the depth of if nesting. Stylistic choice of mine, but if you like the changes generally, feel free to do whatever you'd like!

Thank you for the new commit, I really like the new way of handling the copy_tree feature. I refactored your code a bit:

The copy tree test data was removed since there was no corresponding test for it. I would propose to add a nested folder to the existing test data if necessary.
A new option copy_metadata has been added so that users can choose whether they want to maintain the input file attributes. This applies to rendered files as well!
The inputs of the test data have been swapped since the inputs are no longer reversed.

Looks good to me. Did you encounter any bugs with this version of the code in your config?

I did not. I just reran after the latest log-line change, and set copy-metadata to true, and no files were changed in my output directory according to git.

Great to hear that! I will do some more testing on my personal Home Assistant config during the next days. If all goes well, I will merge this PR.
Thanks again for all the effort you put into this project, I am more than happy with the end result and also the communication with you along the way.

This PR is now merged into the beta branch. Once the release action has finished, you can install it via pip (using the --pre option).


Unable to download JSON metadata on Raywenderlich.com

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2020.01.24
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl --verbose https://www.raywenderlich.com/4743-beginning-rxswift
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.raywenderlich.com/4743-beginning-rxswift']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.01.24
[debug] Python version 2.7.16 (CPython) - Darwin-19.2.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2, rtmpdump 2.4
[debug] Proxy map: {}
[RayWenderlichCourse] 4743-beginning-rxswift: Downloading webpage
[download] Downloading playlist: Beginning RxSwift
[RayWenderlichCourse] playlist Beginning RxSwift: Collected 39 video ids (downloading 39 of them)
[download] Downloading video 1 of 39
[RayWenderlich] 4743-beginning-rxswift/1: Downloading webpage
[RayWenderlich] 4743-beginning-rxswift/1: Downloading JSON metadata
[vimeo] 266135871: Downloading webpage
[vimeo] 266135871: Extracting information
[vimeo] 266135871: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 404: Not Found
[vimeo] 266135871: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 266135871: Downloading fastly_skyfire m3u8 information
[vimeo] 266135871: Downloading akfire_interconnect_quic MPD information
[vimeo] 266135871: Downloading akfire_interconnect_quic MPD information
[vimeo] 266135871: Downloading fastly_skyfire MPD information
[vimeo] 266135871: Downloading fastly_skyfire MPD information
[debug] Default format spec: bestvideo+bestaudio/best
[download] Introduction-266135871.mp4 has already been downloaded and merged
[download] Downloading video 2 of 39
[RayWenderlich] 4743-beginning-rxswift/2: Downloading webpage
[RayWenderlich] 4743-beginning-rxswift/2: Downloading JSON metadata
[vimeo] 266136175: Downloading webpage
[vimeo] 266136175: Extracting information
[vimeo] 266136175: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 404: Not Found
[vimeo] 266136175: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 266136175: Downloading fastly_skyfire m3u8 information
[vimeo] 266136175: Downloading akfire_interconnect_quic MPD information
[vimeo] 266136175: Downloading akfire_interconnect_quic MPD information
[vimeo] 266136175: Downloading fastly_skyfire MPD information
[vimeo] 266136175: Downloading fastly_skyfire MPD information
[debug] Default format spec: bestvideo+bestaudio/best
[download] Hello RxSwift-266136175.mp4 has already been downloaded and merged
[download] Downloading video 3 of 39
[RayWenderlich] 4743-beginning-rxswift/3: Downloading webpage
[RayWenderlich] 4743-beginning-rxswift/3: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

$

Description

This does require account credentials but I am unable to provide them as the account is a shared account and I am not the primary owner.

I have the same issue on Udemy:
youtube-dl -u user_ -p password_ https://www.udemy.com/course/credit-risk-modeling-in-python/ --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-u', 'PRIVATE', '-p', 'PRIVATE', 'https://www.udemy.com/course/credit-risk-modeling-in-python/', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.02.16
[debug] Python version 3.8.1 (CPython) - Linux-5.4.6-2-MANJARO-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.1, ffprobe 4.2.1, rtmpdump 2.4
[debug] Proxy map: {}
[udemy:course] Downloading login popup
[udemy:course] Logging in
[udemy:course] course: Downloading webpage
[udemy:course] 2474716: Downloading course curriculum
[download] Downloading playlist: 2474716
[udemy:course] playlist 2474716: Collected 62 video ids (downloading 62 of them)
[download] Downloading video 1 of 62
[udemy] Downloading login popup
[udemy] 15585666: Downloading webpage
[udemy] 15585666: Downloading lecture JSON
[udemy] 15585666: Downloading lecture JSON
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2237, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

After debugging I have found several issues with the current raywenderlich.py

Currently, RWL, short for RaywenderLich, is using cookies sessions with the state to get USER_TOKEN on webpage.

    <script>
//<![CDATA[

      window.CAROLUS_ENV = {
        KERCHING_BASE_URL: "https://store.raywenderlich.com/",
        BETAMAX_BASE_URL: "https://videos.raywenderlich.com/api/v1",
        GUARDPOST_BASE_URL: "https://accounts.raywenderlich.com/v2",
        CONTENT_PERMISSIONS_REQUIRED_COOKIE_DOMAIN: ".raywenderlich.com",
        USER_TOKEN: "*"
      };
//]]>
</script>

The 403 JSON error is coming from L106, L116. The correct way to get the JSON is:

GET /api/v1/videos/3712.json
Accept: application/json, text/javascript, */*; q=0.01
Authorization: Token $USER_TOKEN
Origin: https://www.raywenderlich.com
Referer: https://www.raywenderlich.com/
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_3) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.2 Safari/605.1.15
X-Requested-With: XMLHttpRequest
X-CSRF-Token: *

You can bypass by parse the USER_TOKEN with a parameter (--video-password) and get that value into raywenderlich.py

The current implementation is using the thumbnailUrl in HTML's meta tags to get the lessonId. Apparently, there is no other way to get that 3712 except getting from the thumbnail. And there are videos that don't have a thumbnail.

<meta property="og:image" content="https://files.betamax.raywenderlich.com/attachments/videos/3712/f0a9b08b-3919-4b5a-aad7-40676ce0fa1f.png">


Merge 1.0 into dev branch

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[ ] The toolchain has been rebuilt successfully (or no changes were made to it)
[ ] The toolchain/worker package manifests are up-to-date
[ ] Any updated packages successfully build (or no packages were changed)
[ ] All package sources are available
[ ] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[ ] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[ ] All source files have up-to-date hashes in the *.signatures.json files
[ ] sudo make go-tidy-all and sudo make go-test-coverage pass
[ ] Documentation has been updated to match any changes to the build system
[ ] Ready to merge

Summary

Merge 1.0 (up to the March update) into dev branch.
Where the specs diverge between 1.0 and dev, I generally took the changes from 1.0 and squashed them into one release on top of the dev version of the spec.
Change Log

See list of commits for details on which changes have been merged

Does this affect the toolchain?

YES

My changes are looking good.

script to build kubernetes container is OK


iterrows (by chunk)

Hello,
I noticed that with blaze it's possible to iterate by row.
In [200]: for row in dat[0:10]:
    print(row)

tuples are print
maybe an iterrows method should be create (like Pandas).
it could yields
an integer and a dict (or a pd.Series like)
iterrows could have a chunksize parameter.
It will be very useful if data is too big.
Kind regards

You can already get this functionality through odo:
In [1]: ds = bz.Data('postgresql://localhost/test::test')

In [2]: ds
Out[2]:
    a   b   c
0   0   1   2
1   3   4   5
2   6   7   8
3   9  10  11
4  12  13  14

In [3]: for chunk in odo(ds, chunks(pd.DataFrame), chunksize=2): print(chunk)
   a  b  c
0  0  1  2
1  3  4  5
   a   b   c
2  6   7   8
3  9  10  11
    a   b   c
4  12  13  14

Generally things that involve data conversion live in odo

I wonder if it doesn't convert to a DataFrame and after iterates over rows ?

Maybe I misunderstood, how would your proposed iterrows work differently than the current iteration protocol or the code I showed?

My idea is to iterate over slices of a big table (as blaze.expr.expressions.Slice) and to iterate over rows of slices
Not sure I'm clear enough
The goal is to quickly start process (no to put data in memory) even with a very big table

with the odo(..., chunks(pd.DataFrame)) you would only ever have one chunk in memory at a time. If you want them as symbols, you can map(Data, odo(...)). Some computation would need to happen to know how many elements to yield, otherwise it would yield slices that are no longer in the dataset.

Compare with a big table (200000 rows for example)
# (very) long at startup

for i, chunk in enumerate(odo(dat, chunks(pd.DataFrame), chunksize=10)):
    print("chunk %d" % (i + 1))
    for i, row in chunk.iterrows():
        print(row)

with
# Short at startup

def iterrows(dat, chunksize):
    n = int(dat.nrows)
    i = 0
    while(i<n):
        slice = dat[i:i + chunksize]
        for row in slice:
            yield(row)
        i += chunksize

for row in iterrows(dat, 10):
    print(row)

I still need some improvement to have row a dict like.

Why is the first one long at startup, it lazily reads from the database and yields as soon as 1 dataframe is ready

I don't think so... try to change chunsize. Even with a very low value of chunksize it's very long to see first rows.

sorry you may be true.

I know that there was an issue that some database iterator paths would pull data eagerly; so this might actually be slow due to another bug.


Dynamic Programming added in hackerrank

Hey, appreciate the contribution, but this is my private repo and I don't accept PRs :)


Add Twitter Feed

As a person trying to market Employee Empowerment, it would help if my posts on social media appeared on the home page.
AC:

Tweets appear on homepage

This is now on Master


vulkan-headers: <IP_ADDRESS> -> 1.1.85

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/vulkan-headers/versions.
meta.description for vulkan-headers is: '"Vulkan Header files and API registry"'.
Compare changes on GitHub

Checks done (click to expand)

built on NixOS

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 1.1.85 with grep in /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85

directory tree listing: https://gist.github.com/d667edd8ddacdede99fed524d0e66b21

du listing: https://gist.github.com/f96fd91ef4ed2870ae09a29657a05ab7

Instructions to test this update (click to expand)

One-time optional setup to skip building using Cachix:

Install cachix from https://cachix.org/
Use r-ryantm's cache:cachix use r-ryantm

Test this update by entering a nix shell, seeing what is inside the
result, and if applicable, running some binaries:
nix-shell --pure -I nixpkgs=https://github.com/r-ryantm/nixpkgs/archive/3cc6b6518c8f77c7b1f3d75c00f453b5ca451fa2.tar.gz -p vulkan-headers
ls -la /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85

cc @ralith for testing.

Success on aarch64-linux (full log)
Attempted: vulkan-headers
Partial log (click to expand)
-- Installing: /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85/share/vulkan/registry/reg.py
-- Installing: /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85/share/vulkan/registry/genvk.py
-- Installing: /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85/share/vulkan/registry/generator.py
-- Installing: /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85/share/vulkan/registry/cgenerator.py
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85
strip is /nix/store/428gs2z4b8f9byvghzlpbjwjb3a7jwxx-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85
checking for references to /build in /nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85...
/nix/store/imv819lh0rdbv5p9xlkycdgkvld1dz51-vulkan-headers-1.1.85

No attempt on x86_64-darwin (full log)
The following builds were skipped because they don't evaluate on x86_64-darwin: vulkan-headers
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-linux (full log)
Attempted: vulkan-headers
Partial log (click to expand)
-- Installing: /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85/share/vulkan/registry/reg.py
-- Installing: /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85/share/vulkan/registry/genvk.py
-- Installing: /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85/share/vulkan/registry/generator.py
-- Installing: /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85/share/vulkan/registry/cgenerator.py
post-installation fixup
shrinking RPATHs of ELF executables and libraries in /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85
strip is /nix/store/dxf1m7dhc4qb655bdljc1fsd74v1nag3-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85
checking for references to /build in /nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85...
/nix/store/mlrrdsqh8ziscv77cz32sa98818sxv7s-vulkan-headers-1.1.85

@xeji Please don't merge automated updates without doing at least some checking. In particular, @r-ryantm does not check if dependent packages still build. I've had to fix NixOS modules in the past because arguments changed and the package bump was blindly merged. Furthermore, changelogs should also be read to check if this release fixes any security vulnerabilities that need to be backported etc.

@fpletz sorry about that. I appreciate your feedback and will try to be more careful in the future.
In this particular case I don't see what I should have done differently. ofborg said no other packages are affected (probably because it doesn't catch eval errors), so there was no indication this could break stuff.

reverted in ac6f4851480e3a7016a2257a62c34ed46dcea789


chore: fix security vulnerabilities

Details

This PR fixes the security issues as reported by npm audit. More details in the linked issue.
Fixed Issues

$ https://github.com/Expensify/App/issues/48327
PROPOSAL: https://github.com/Expensify/App/issues/48327
Tests

[x] Verify that no errors appear in the JS console

Testing Steps:

Verify the general usage of the app doesn't break

Offline tests

QA Steps

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/aa0a1cac-8494-4031-9089-36ff9e6a051b

Android: mWeb Chrome

https://github.com/user-attachments/assets/bc495fe4-cad9-432c-b61b-e0a3d037980d

iOS: Native

https://github.com/user-attachments/assets/a854fed5-99a1-4ec9-b83f-b04630e301ee

iOS: mWeb Safari

https://github.com/user-attachments/assets/2c5398e7-54ae-4af0-8367-54f093bd6703

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/c5e56574-c67f-4cdd-b39e-098cdcd61d41

MacOS: Desktop

https://github.com/user-attachments/assets/9f59e8d4-542d-4a4b-9ab2-3574fe4057a5

Reassure tests are failing because we have updated the patch version. In reassurePerformanceTests.yml we execute following line, since we have updated the patch version, we have to remove node_modules and re-install dependencies BUT the code fall into npm install --force and doesn't execute the right side of OR conditon.
https://github.com/Expensify/App/blob/8a35256bff4d66d0f0958fd9100e191a8fda404d/.github/workflows/reassurePerformanceTests.yml#L55

When I run reassure locally it works all fine. Following is the change I suggest to fix this:
diff --git a/.github/workflows/reassurePerformanceTests.yml b/.github/workflows/reassurePerformanceTests.yml
index d4a25a6395..a9ae6351e5 100644
--- a/.github/workflows/reassurePerformanceTests.yml
+++ b/.github/workflows/reassurePerformanceTests.yml
@@ -52,7 +52,7 @@ jobs:
           git switch --force --detach -
           git merge --no-commit --allow-unrelated-histories "$BASELINE_BRANCH" -X ours
           git checkout --ours .
-          npm install --force || (rm -rf node_modules && npm install --force)
+          rm -rf node_modules && npm install --force
           NODE_OPTIONS=--experimental-vm-modules npx reassure --branch

       - name: Validate output.json

Nice

When I tried to upload pdf on mSafari, I got this crash, Can you take a look @hurali97 ?
https://github.com/user-attachments/assets/8f3fd817-a35a-4a3b-973e-a1d6f436514e

When I tried to upload pdf on mSafari, I got this crash, Can you take a look @hurali97 ?
Screen.Recording.2024-09-09.at.16.21.00.mov

@hungvu193 So it was happening because of a missing polyfill for Promise.withResolvers. Since this polyfill is available via corejs and we have it installed already, so it's just a matter of importing it. Here's this solution shared in the issues of react-pdf.
https://github.com/user-attachments/assets/e3f34810-e032-42fa-9176-5cc97be697ad

Reviewer Checklist

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I included screenshots or videos for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/df71d4a2-893f-47e1-847f-89a21e49cd6f

Android: mWeb Chrome

iOS: Native
https://github.com/user-attachments/assets/33c1c301-71e6-4c61-8bc9-7f8e99b225e4

iOS: mWeb Safari

https://github.com/user-attachments/assets/e16fd983-7106-4d35-a686-ccf7879b4c3d

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/4bc5887b-aca8-42ba-ac7c-5923694b910a

MacOS: Desktop

https://github.com/user-attachments/assets/cba0f091-b589-4a99-a209-188a294be3d4

@hurali97 We have conflicts here

@grgia this PR is ready for your review!

@hungvu193 After pulling in the latest main, I see 28 vulnerabilities added. Should we fix those in this PR or leave as is?

Hmm I think Yes

@hungvu193  so I have it fixed now, below is the diff:
diff --git a/package.json b/package.json
index 3f685cc1efe..c636a18aaf8 100644
--- a/package.json
+++ b/package.json
@@ -345,11 +345,14 @@
     "loader-utils": "2.0.4",
     "follow-redirects": "1.15.6",
     "fast-xml-parser": "4.4.1",
-    "express": "4.19.2",
+    "express": "4.20.0",
     "elliptic": "6.5.7",
     "fast-json-patch": "3.1.1",
     "webpack": "^5.94.0",
-    "@blakeembrey/template": "1.2.0"
+    "@blakeembrey/template": "1.2.0",
+    "body-parser": "1.20.3",
+    "path-to-regexp": "0.1.10",
+    "send": "0.19.0"
   },
   "expo": {
     "autolinking": {

I will push it shortly 👍

Thanks for your quick work 🚀

@hungvu193  After pulling in the latest main CI and local both are failing because of:

I don't see any changes in my PR which could affect this, so I am looping in @kirillzyusko as I see some commits from him in the history for react-compiler-healthcheck. @kirillzyusko could you maybe know why this could be happening? I have tried removing node_modules and re-installing but the error persists.

@hurali97 have you tried to lock the version, i. e. change ^0.0.0-experimental-ab3118d-20240725 -> 0.0.0-experimental-ab3118d-20240725 in package.json? And reinstall node_modules?

@hurali97 have you tried to lock the version, i. e. change ^0.0.0-experimental-ab3118d-20240725 -> 0.0.0-experimental-ab3118d-20240725 in package.json? And reinstall node_modules?

Yeah did that but no luck. The error says that the patch is for the previous version then the one installed. Which means that we maybe missed updating the patch to target this newer version?

Failed to apply patch file react-compiler-healthcheck+0.0.0-experimental-b130d5f-20240625+001+initial.patch.

From git history,

@hurali97 there wasn't a difference between 0625 and 0725 versions, so the patch still could be applied successfully.
I tend to think that the reason why it fails is because you have this lock file content:
    "node_modules/react-compiler-healthcheck": {
      "version": "0.0.0",
      "resolved": "https://registry.npmjs.org/react-compiler-healthcheck/-/react-compiler-healthcheck-0.0.0.tgz",
      "integrity": "sha512-b+WqemUSkGcAag7+xEEl/pVCULXnP8tWsZsqDx8MgCETkaC+yxPzOwpsScVkTiRJ/Z75e+xdAa9W67S/hx6dSw==",
      "dev": true
    },

While expected content is:
"node_modules/react-compiler-healthcheck": {
      "version": "0.0.0-experimental-b130d5f-20240625",
      "dev": true,
      "license": "MIT",
      "dependencies": {
        "@babel/core": "^7.24.4",
        "@babel/parser": "^7.24.4",
        "chalk": "4",
        "fast-glob": "^3.3.2",
        "ora": "5.4.1",
        "yargs": "^17.7.2",
        "zod": "^3.22.4",
        "zod-validation-error": "^3.0.3"
      },
      "bin": {
        "react-compiler-healthcheck": "dist/index.js"
      },
      "engines": {
        "node": "^14.17.0 || ^16.0.0 || >= 18.0.0"
      }
    },

So you installed the first published version 0.0.0 while we use slightly more latest versions.

@kirillzyusko  Oh man ! Thanks I will check this out 🚀

@hungvu193 All is green now.

Can you merge main please? @hurali97  😄

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/grgia in version: 9.0.37-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

@hungvu193 Can you please share the QA steps for this or full regression suite is enough for this which we normally do

@m-natarajan Sure.
Regression test:

Go to any chat.
Click Plus button => Add Attachment => Choose any pdf.
Verify that you can preview that pdf. Click Send.
Verify that you send the pdf without any issue.
Send a few other pdf files then click on one of the pdf to view the pdf.
Press the left and right arrow to switch between pdf files.
Verify you can switch between pdf files.

🚀 Deployed to production by https://github.com/grgia in version: 9.0.38-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Add missing i18n attribute for translation / add user filter in car list / delete in list cars

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.mahdibenromdhane seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

You should not rush on the last requested changes.

Update was not saving the converter type (CarSecurity class bug): Should be tested more.
Type naming was confusing
All converter fields should be explicitly set when saved in storage
You could use the Converter type in building converter name
Missing some converter properties in the drop down like nbr of phases
Type naming is confusing: CarConverter (from CarStorage) and Converter (from Car)
Column converter should display all converter props
-> Take your time and deliver quality like you're use to

okey, no rush next time :)

okey, no rush next time :)

Thanks! 👍


[Doc][2.2-2.6]Incorrect Swagger Definition in RESTful API doc in APIM

Description:
Came across RESTfulApi docs pointing to outdated swagger files. (Ponting swagger files in  the v6.1.66 code base )
This is valid for APIM 2.1.0.
But from 2.2.0 to 2.6.0 these have to be updated with yaml files in the following carbon-apimt version.
v6.2.201 - APIM 2.2.0
v6.3.95 - APIM 2.5.0
v6.4.50 - APIM 2.6.0
Correct links for 2.6.0:
https://docs.wso2.com/display/AM260/apidocs/store/

https://raw.githubusercontent.com/wso2/carbon-apimgt/v6.4.50/components/apimgt/org.wso2.carbon.apimgt.rest.api.store/src/main/resources/store-api.yaml

https://docs.wso2.com/display/AM260/apidocs/admin/

https://raw.githubusercontent.com/wso2/carbon-apimgt/v6.4.50/components/apimgt/org.wso2.carbon.apimgt.rest.api.admin/src/main/resources/admin-api.yaml

https://docs.wso2.com/display/AM260/apidocs/publisher/

https://raw.githubusercontent.com/wso2/carbon-apimgt/v6.4.50/components/apimgt/org.wso2.carbon.apimgt.rest.api.publisher/src/main/resources/publisher-api.yaml

Suggested Labels:

Suggested Assignees:

Affected Product Version:
2.2. - 2.6
OS, DB, other environment details and versions:
Steps to reproduce:
Related Issues:

We will not be fixing this as this issue is based on an older version of the product.


Added login page icon to the Events page

Close #975
Closes #975

Description
Added login page icon to the Events page and linked it accordingly. It was previously not there
Screenshots

Checklist:

[X] I have linked the PR to the correct issue.
[X] I have read the Contribution Guidelines
[X] I have built and tested the changes, and they do not break or show any errors.

@Vaishnavi-Patil2211 Please merge this


fix: replace Webpacker with Shakapacker

Summary
closes https://github.com/shakacode/react_on_rails/issues/1621 by replacing Webpacker.config.shakapacker_precompile? in configuration to Shakapacker.config.shakapacker_precompile?
Pull Request checklist
Remove this line after checking all the items here. If the item is not applicable to the PR, both check it out and wrap it by ~.

[x] Add/update test to cover these changes
[ ] ~Update documentation~
[x] Update CHANGELOG file

Other Information
Remove this paragraph and mention any other important and relevant information such as benchmarks.

This change is

Summary by CodeRabbit

Refactor

Updated internal references from Webpacker to Shakapacker to ensure compatibility and improve maintainability.

Tests

Modified test cases to reflect the transition from Webpacker to Shakapacker, ensuring consistent behavior and error handling.

@Judahmeek please review.

@adriangohjw can you please add a changelog entry.

@adriangohjw can you please add a changelog entry.

@justin808 oops my bad, saw that it has been addressed in https://github.com/shakacode/react_on_rails/pull/1622/commits/9ae6922153b8144c9d36553ae12434f23bdaf987 - thanks @Judahmeek !

I'll ship once CI passes!

@adriangohjw please check CI failures.

@adriangohjw Shakapacker v6 & v8 are incompatible since v6 uses the Webpacker constant. Shakapacker v7 contains both Shakapacker & Webpacker constants & should be compatible with v8, so let's just drop support for v6.

@Judahmeek

i'm not sure if i'm making the correct changes
i'm also unsure if I should change WebpackerUtils to ShakapackerUtils

can you advise? thank you!

i'm not sure if i'm making the correct changes
Your changes look promising. We've got to get CI to pass though.
i'm also unsure if I should change WebpackerUtils to ShakapackerUtils
Yes, please do.

@Judahmeek

CI good to go (Reference: https://github.com/adriangohjw/react_on_rails/pull/2) + Addressed comments from the automated code review
given that we are dropping support for shakapacker v6, perhaps the new version should not be 14.0.2?

CI good to go (Reference: Fix uninitialized constant webpacker adriangohjw/react_on_rails#2) + Addressed comments from the automated code review

I greatly appreciate your work on that. 👍

given that we are dropping support for shakapacker v6, perhaps the new version should not be 14.0.2?

Correct. Any time we change supported dependency version ranges, then the next release of ReactOnRails will be a  new major version since it is a breaking change.

@Judahmeek any update on this? I think the workflow got stuck (might be due to the Github actions incident last week - https://www.githubstatus.com/incidents/nj47vccwm2zj)

@justin808 I've tested this branch, and it did work fine.

Thanks for this - I gave it a try locally.
The file lib/generators/react_on_rails/templates/base/base/config/shakapacker.yml generates config that is no longer in-line with the Shackapacker v8 default lib/install/config/shakapacker.yml file.
After running rails generate react_on_rails:install, and running ./bin/dev, Webpack (webpack-cli 5.1.4) does not compile, instead raising:
wp-client | [webpack-cli] Invalid options object. Dev Server has been initialized using an options object that does not match the API schema.
wp-client |  - options has an unknown property 'https'. These properties are valid:
wp-client |    object { allowedHosts?, bonjour?, client?, compress?, devMiddleware?, headers?, historyApiFallback?, host?, hot?, ipc?, liveReload?, onListening?, open?, port?, proxy?, server?, setupExitSignals?, setupMiddlewares?, static?, watchFiles?, webSocketServer? }
wp-client | Exited with code 2

This is because Webpack dropped the https: true key in favor of server: https.
The config in the Shackapacker shakapacker.yml were changed in https://github.com/shakacode/shakapacker/pull/277

This is because Webpack dropped the https: ... key in favor of server: https.

thanks @timkrins - will make the change
@justin808 should we copy-paste the config from https://github.com/shakacode/shakapacker/blob/main/lib/install/config/shakapacker.yml?

Yes, as it's the current one and the current version is the default.

@adriangohjw Once again, thanks for your work.
Justin decided that he wanted to use an adapter module instead of having to make any breaking changes so I'm incorporating work from an old PR of mine.

@G-Rath any opinions on this PR?

Closing, since the issue has been fixed by: https://github.com/shakacode/react_on_rails/pull/1629
CC: @justin808

@vaukalak #1629 doesn't resolve all the incompatibilities regarding Shakapacker v8.
It unfortunately doesn't even resolve the generator issues because the Examples job was failing to actually run the expected test suite due to an id mismatch between changed-files-specific and steps.changed-files.outputs.any_changed


About the bonferroni correction in pairwise_logrank_test

Lifelines provides pairwise_logrank_test in lifelines.statistics for multicomparison of logrank between various group pairs.
I found the issue mention about this function provide the bonferroni correction in default.
However, I could not find the options or implementation in current version.
Is this function deprecated? Or some reason this method are removed from pairwise_logrank_test.

Hi @lycantrope, looks like it was removed. It should be possible to do outside lifelines using the results from pairwise_logrank_test.


Licensing Information Missing for GitBase

Hi @qiayue, The project looks great. I truly appreciate the work.
The Problem:
The GitBase's ReadME states it’s open-source under the MIT license, but the provided link returns a 404 error. Please update the link or confirm the open-source license status.
Thanks!
Waleed Durrani
Creator of InsTech Newsletter

Hi, I made a mistake before and now I have added the MIT license.

Great, thanks a lot @qiayue. By the way, the reason I specifically asked for the license was; I wanted to feature your tool in my #InsTechNewsletter as an Open Source Tool.
Please do check it out. Your tool is amazing 🚀.


File browser filters on group page not working correctly

On the screenshot geo-coordinates filter didn't work

also, why the artifacts are hard-coded?
2022.12.01
https://databus.dbpedia.org/marvin/generic
en

I don't think they are :). The query is generated just like this with artefact names as consts...

there are many more artifacts and the question is why they are not in the query?

The query generator was being passed the wrong array of artifacts (the currently displayed ones instead of all)
The first issue still persists, the query generator is not working correctly here.
there's no hard-coded SPARQL, the queries are generated.

what @manonthegithub wrote seems duplicate to this here https://github.com/dbpedia/databus/issues/124
but @m1ci thing might be something different. but I am not sure. so maybe rename/clarify/clean up the issue if its not already fixed.

possible fix deployed on dev