Player.load does not respect startTime for certain content

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
4.7.9
Can you reproduce the issue with our latest release version?
yes
Can you reproduce the issue with the latest code from main?
yes
Are you using the demo app or your own custom app?
my own app, since I don't think that I can use demo to force start from certain second
What browser and OS are you using?
Chrome + MacOS
What are the manifest and license server URIs?
https://edge1.motv.eu/s1/channels/recordings/eurosport-2-clear-40-71/2024/02/10/40-65bf6e622eed0-2024-02-10-11-45-cyklistika/data/40-65bf6e622eed0-2024-02-10-11-45-cyklistika-clear.mpd
Hopefully this URL will work for some time, if not, I will share a new one
What did you do?
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Video Player</title>
    <!-- Shaka Player Scripts -->
    <script src="https://cdnjs.cloudflare.com/ajax/libs/shaka-player/4.7.9/shaka-player.compiled.debug.min.js" integrity="sha512-C0WsjfEA5l9y+QmVK7BkSxEWGnLD4FE2qslS20ZeRq6V91ryFO8h2sq0PURIatqGPsj/4uXmOXtnlqnPSyHImg==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/shaka-player/4.7.9/shaka-player.ui.debug.min.js" integrity="sha512-9veM994T5eBzgybgFwqis2eGd+gzAYHPlYNaYnsnFxHxPw+2vbw3sQlTgLubEI4Bhs5q8b0O+MaFykG7QdykgA==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <!-- Shaka Player Styles -->
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/shaka-player/4.7.9/controls.min.css" integrity="sha512-4RC6gy+J9tI06Vqu8sntJdArqFk86ufXyDaDIzvAp/zMTcFzC0IQiH2iWOsL1i5/2+r6igS0xxtKCJZnPL2NTA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
</head>
<body>
    <div id="channel-player" style="max-width: 800px;">
        <video id="video" style="max-width: 500px;" autoplay></video>
    </div>

    <script type="text/javascript">
        // Initialize Shaka Player
        const videoElement = document.getElementById('video');
        const localPlayer = new shaka.Player(videoElement);

        const ui = new shaka.ui.Overlay(localPlayer, document.getElementById('channel-player'), videoElement);

        const contentStartTimestamp = 60;

        localPlayer.load('https://edge1.motv.eu/s1/channels/recordings/eurosport-2-clear-40-71/2024/02/10/40-65bf6e622eed0-2024-02-10-11-45-cyklistika/data/40-65bf6e622eed0-2024-02-10-11-45-cyklistika-clear.mpd', contentStartTimestamp);
    </script>
</body>
</html>

It is really very basic app, it just opens the MPD and attempts to start from 60th second.
What did you expect to happen?
I expected the content to start to be played at 60th second.
What actually happened?
It started at zero second.
And I know why. It is because the content is "special" - it does not start from zero.
<SegmentTemplate timescale="90000" initialization="audio_clear_eng_stereo.mp4" media="a_clear_eng_stereo/$Number$.mp4" startNumber="1" presentationTimeOffset="71994">
          <SegmentTimeline>
            <S t="6479452576" d="549120"/>
            <S t="6480001696" d="864000" r="754"/>
          </SegmentTimeline>
        </SegmentTemplate>

the SegmentTimeline starts from nonzero t and with nonzero PTS.
Calling player.load(url, 60) does not work. In this case, I'd have to call player.load(url,<PHONE_NUMBER> / 90000 + 71994 + 60 and it works like a charm and starts at 60th second.
The content was created using shaka-packager by capturing static part of live stream. Unfortunately, I cannot tell shaka-package to start the MPD from zero and therefore I was hoping it would be possible to resolve in Shaka-player.  For example, exoplayer (also google product but from another team) handles this seamlessly and it works with timeStart = 60 out of the box.
I know I could wait for the content to load, check for example seekRangeStart and update video.currentTime = player.seekRange().start + offset but its far from ideal. Because the player loads the chunks twice, once from beginning and once from given offset.
Do you please have any idea how to achieve seamless player.load(url, 60) in shaka-player?
Thank you for any advice
Jakub

videoElement.addEventListener('loadedmetadata', () => {
          videoElement.currentTime = localPlayer.seekRange().start + 60
        });

this works but as I said, its not ideal + working with video.currentTime gets difficult because its starts for every catchup from different value. But shaka player is somehow capable of calculating the correct value - since it shows in the UI correct time

Don't use the video element's metadataloaded event, instead use the player's event e.g. in your case you want to listen to the streaming one: https://shaka-player-demo.appspot.com/docs/api/shaka.Player.html#.event:StreamingEvent .

@absidue thank you, but unfortunately, that does not help.
does not work, does not seek at all
player.addEventListener('streaming', () => {
          player.updateStartTime(60)
        });

works, seeks to 60th second
player.addEventListener('streaming', () => {
          player.updateStartTime(6479452576 / 90000 + 71994 + 60)
        });

The goal is to be able to be able to seek without the complicated formula

Didn't realise I had to spell every single step out for you, as I thought that it was obvious that you would need to use seek range start as well (that's the earliest point that your manifest indicates that can be seeked to). shaka-player uses the timeline that you specify in your manifest, so if you manifest doesn't start at zero, you cannot reasonably expect to be able to seek to a position starting at 0 either.

 Your advice to use streaming event instead of loadedmetadata was completely useless. As per my first post, I knew how to work around the manifest timing.
I was asking whether its possible to add some configuration to shaka-player so that it would ignore the timeline offset and start from zero - like some other players, such as ExoPlayer, do out of the box without any additional configuration (or the player that Roku uses). If others can do it, shaka surely could too, therefore yes, I could reasonably expect it.
Thank you

Also, shaka already kind of does the recalculation, in the UI, it shows the start from zero and not from 77381. So I think either shaka should honor the time in all places, that is in the calculations as well as in the UI or not. Right now it displays one thing while showing another in the background. Shouldn't it be consistent?

Your advice to use streaming event instead of loadedmetadata was completely useless.

It's most certainly not useless, it addresses this part of your original post:

I know I could wait for the content to load, check for example seekRangeStart and update video.currentTime = player.seekRange().start + offset but its far from ideal. Because the player loads the chunks twice, once from beginning and once from given offset.

Additionally this should have been opened as a question or a feature request, as your misunderstanding of how developer facing APIs work is most certainly not a shaka-player bug.
Also no the developer facing API is correct in using absolute values (that's also what videoElement.currentTime uses), as that's what you specified in your content. Imagine how horrible it would be for live streams if the value you passed it, would seek to a different position every time. shaka-player is respecting your wish as a developer to use those large offsets and correctly exposing those implementation details in the developer facing APIs, but as the user of the UI/the viewer couldn't care less about how the actual content is setup, the UI does a bunch of extra calculations to hide those implementation details.
The easier option would be to fix your content (if you don't want shaka-player to use large offsets, don't use them in your manifest), otherwise you'll want to replicate the UI calculations in your own code.

Hey, it looks like this discussion is getting heated.  Let's avoid rude language that implies that others are less capable:

"Didn't realise I had to spell every single step out for you"
"I thought that it was obvious"

@absidue, please consider this a formal warning.  If someone misunderstands you, please do not disparage them over it.  Every one of us has misunderstood something at one time or another, and this is a big, complex project where it can be hard to get to the root of any issue.
I know we participate here because we want to be helpful to one another, so let's let our language reflect that spirit of community.  Thanks!

@jakubvojacek, I can explain what is happening in the UI vs the player.
There are several distinct timelines involved in playback.  The most relevant ones are:

The media timeline: the timestamps in actual media segments.
The presentation timeline: the timestamps in the "presentation".  The timestamps seen by MediaSource and the video element, which may differ from the media timestamps.  These are adjusted when consumed by MSE.  For DASH, the formula is (media timestamp + period start time -  presentation time offset).
The seek range: the available content you can seek to.  This is a subset of the presentation timeline.
The UI range: the seek range, but offset to start at 0 for VOD content.

It's a little messy, but there's a diagram showing some of these relationships here: https://shaka-player-demo.appspot.com/docs/design/timeline.svg

The player API and the video element API (used for seeking) do everything in terms of the presentation timeline.  So when you seek with video.currentTime = 60 or load content with player.load(url, 60), you are specifying a presentation timestamp.  What you may not know is that the player will clamp that timestamp into the seek range.
Because 60 is before the start of the seek range, clamping that timestamp will move it to the earliest available time stamp.  This looks like it's ignoring your time because it's starting at the beginning.  But if you had picked a timestamp past the end of the seek range, such as 152000, it would move the playhead to the end of the seek range.
The UI can see that the content is VOD, and VOD content that doesn't start at 0 doesn't make much sense to the end user.  So it offsets the displayed timestamps so that the seek range starts at 0:00.

The reality of the "presentation" and the timestamps in the presentation timeline is dictated by the DASH manifest, which says:
  <Period id="0" start="PT71996S" duration="PT79249S">
<!-- ... -->
        <SegmentTemplate timescale="90000"
                         initialization="audio_clear_eng_stereo.mp4"
                         media="a_clear_eng_stereo/$Number$.mp4"
                         startNumber="1"
                         presentationTimeOffset="71994">
<!-- ... -->
            <S t="6479452576" d="549120"/>

You have a single period starting at timestamp 71,996 seconds, with a duration of 79,249 seconds.  This puts the end of the presentation at time 151,245, which is what the player reports for the end of the seek range.
The first segment of this audio track is at 6,479,452,576 (in the media timeline, in timescale units) with an offset of 71,994 (also in timescale units).  The timescale is 90,000, and the period start is 71,996 (in seconds).  Putting all this together, the first audio segment is at presentation time ((6479452576 - 71994) / 90000) + 71996 = 143,989.118 seconds, which is what the player reports for the start of the seek range.
Note that although the period start is 71,996 seconds, the first segment is at 143,989.118 seconds.  The segments get clipped to the period range, but it is the segments within the period that define availability and the seek range, not the period itself.

So if you want to start at a specific timestamp in the presentation timeline, use player.load(url, timestamp).  If you want to start at a specific timestamp relative to the seek range, and you don't know it before loading the content, you should use the streaming event, as suggested by @absidue:
  localPlayer.addEventListener('streaming', () => {
    videoElement.currentTime = localPlayer.seekRange().start + 60;
  });
  localPlayer.load(url);

I hope this all helps to explain how it works and why.
And to address your earlier comment:
@jakubvojacek wrote:

So I think either shaka should honor the time in all places, that is in the calculations as well as in the UI or not. Right now it displays one thing while showing another in the background. Shouldn't it be consistent?

I can understand why you might think that.  But from my perspective, it is consistent... it's just that there are multiple constituencies and views, each of which is self-consistent.
The API is consistent in its use of presentation timestamps and the presentation timeline for everything.  The API is for developers, who need to know how content works.
If we used the UI timeline in the API, it could make certain app features and decisions more difficult.  For example, if you know an actual timestamp from your DASH presentation, perhaps extracted from another service, and you want to use it to seek to a position or overlay metadata or whatever, using the UI timeline in the player would require even more info about the presentation timeline, which the player wouldn't be using and therefore wouldn't make available in this scenario.
The UI is consistent in showing the content that is available to the user.  The UI is for users, who just want to watch it and shouldn't have to know or care about how the underlying content works.
If we used the presentation timeline in the UI, users would be mad that they couldn't seek to "0", or wouldn't understand why their video started at time 39:59:49 (in the case of your sample content above).
Does this all make sense now?

Hiya @joeyparrish
I'm sorry for using such aggressive language. It rubbed me the wrong way seeing a question/feature request being reported as a bug. I'll refrain from commenting on issues on this repository in the future to avoid a similar situations arising.
As I can't contribute any code because of the CLA (having to doxx myself to contribute to an open source project is rather undesirable, also know that many people see CLAs as going against the spirit of open source), I thought I could make myself useful on the issue tracker, so that I can at least help out in some way, I'm sorry that I made things worse.

@absidue, the CLA is Google policy that is out of my control, and I respect your choice not to sign it.
You are welcome to continue to contribute here on the issue tracker.  Just please use respectful language and remember that everyone, at some point, will need to have something patiently explained to them, including you.  That's just the nature of human communication.  And likewise, everyone, at some point, has been impatient or rude to others, including me.  I believe everyone has been in situations like this before, from every side, and we can all learn and move on productively.  I hope you won't let this small moment keep you from being a member of the community.

Hello @joeyparrish
thank you for the detailed explanation. I was aware that I might have to workaround the content timing in code. Its just that we heavily use exoplayer and it works very differently there and the timing for such content starts from zero. That is why I was hoping that such a feature could be ported shaka player, perhaps under some flag. Since both players are from Google, it would make some sense that they'd behave the same I guess.
Once again, thank you 🙏

I understand.  Exoplayer is a great project, but Exo & Shaka are maintained by two very different teams and we differ in philosophy in some areas.  We apologize for any inconvenience, and we thank you for your understanding.


DataServiceItemDefinitionHandler NPE fix

@mswiderski Could you look at this?

@sutaakar I made a byteman test! And I improved another test by using the CyclicBarrier class. If you have any questions, please let me know. :)

Thanks @mswiderski.
Merged @ https://github.com/droolsjbpm/jbpm/commit/c7666467421b2c2d6021c4d840e83101039c0da5


More minimal options for installing requirements

Not sure if the requirements should be pinned or not

We have this in setup.py, will create a new issue if we want the ability to do this with pinned requirements


style: rename menu items for copying and sharing

Closes #857

Copy Public Gateway URL → Copy shareable link
Copy IPFS Path → Copy content path

Would “Copy IPFS content path” be more explicit? There’s room for it.

@jessicaschilling updated:

Nice! Oh: “Resource” in the pinning menu item should be lowercase for consistency. Otherwise LGTM 😊


Android Keyboard Not Opening Sometime on Search Screen - Resolved#9827

@rushatgabhane @sketchydroide
Details

The keyboard on android was not opening few times due to inconsistent node handling native side. Setting a timeout should be giving android time to handle keyboard easily.

Fixed Issues

$ https://github.com/Expensify/App/issues/9827
Tests

[x] Verify that no errors appear in the JS console

PR Review Checklist

Contributor (PR Author) Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] iOS / native
[x] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product was added in all src/languages/* files
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] Any functional components have the displayName property
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] I have checked off every checkbox in the PR author checklist, including those that don't apply to this PR.

PR Reviewer Checklist
The Contributor+ will copy/paste it into a new comment and complete it after the author checklist is completed

[x] I have verified the author checklist is complete (all boxes are checked off).
[x] I verified the correct issue is linked in the ### Fixed Issues section above
[x] I verified testing steps are clear and they cover the changes made in this PR

[x] I verified the steps for local testing are in the Tests section
[x] I verified the steps for Staging and/or Production testing are in the QA steps section
[x] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[x] I checked that screenshots or videos are included for tests on all platforms
[x] I verified tests pass on all platforms & I tested again on:

[x] iOS / native
[x] Android / native
[x] iOS / Safari
[x] Android / Chrome
[x] MacOS / Chrome
[x] MacOS / Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I verified proper code patterns were followed (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product was added in all src/languages/* files
[x] I verified any copy / text that was added to the app is correct English and approved by marketing by tagging the marketing team on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I verified that this PR follows the guidelines as stated in the Review Guidelines
[x] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] If a new component is created I verified that:

[x] A similar component doesn't exist in the codebase
[x] All props are defined accurately and each prop has a /** comment above it */
[x] Any functional components have the displayName property
[x] The file is named correctly
[x] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[x] The only data being stored in the state is data necessary for rendering and nothing else
[x] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[x] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[x] All JSX used for rendering exists in the render method
[x] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(themeColors.componentBG)

[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

QA Steps

On All Report List Page press on search.
Search page will open, here keyboard should open.
Now try above steps 10-15 times and every time it should open the keyboard on search page.

[x] Verify that no errors appear in the JS console

Screenshots

Web
https://user-images.githubusercontent.com/78416198/179116621-51a38803-7c2f-4afe-a30f-0e9251d940a4.mov
Mobile Web
https://user-images.githubusercontent.com/78416198/179116656-b55313d2-dad5-4943-a8c7-476b8a5f1e4c.mov
Desktop
https://user-images.githubusercontent.com/78416198/179117428-d26e21fc-1a7d-4c80-9d2e-a26084ce4677.mov
iOS
https://user-images.githubusercontent.com/78416198/179116958-db45198e-58e9-41ab-ba74-5bf3212865e7.mov
Android
https://user-images.githubusercontent.com/78416198/179025618-5ae4d851-e8b2-44a6-9ff8-0cf1c3413577.mp4
I have read the CLA Document and I hereby sign the CLA

I have read the CLA Document and I hereby sign the CLA

@rushatgabhane @sketchydroide @parasharrajat I have closed the old PR an this the updated PR with Signed Commits and all videos for Issue https://github.com/Expensify/App/issues/9827

@rushatgabhane Done 👍
Fixed android keyboard issue - for new chat and new group page too.
https://user-images.githubusercontent.com/78416198/179138130-c598f02c-febb-4167-9fef-945859d2a33b.mp4

@sketchydroide LGTM. I'm wondering if we should always delay focus when autoFocus is true.
Because it's easy to forget about shouldDelayFocus when using OptionsSelector in a new page.
The tradeoff is that we'll always have a 300ms delay when focusing. And this won't be ideal for pages with no transitions.
cc: @jeet-dhandha

@rushatgabhane I think that is a bigger discussion we can have in the slack channel, and explain the why there.
It might makes sense in other transitions, but it might not. I don't really know that just by myself tbh.

@rushatgabhane We can put comments for that for now as other developers will refer to the older  code only. So through comments they might be able to figure out why we used that prop.

@rushatgabhane Awaiting Response on this PR. If everything is completed please update on when this PR will be merged 👍.

happening here https://expensify.slack.com/archives/C01GTK53T8Q/p1658026341800479

@jeet-dhandha
Great job getting your first Expensify/App pull request over the finish line! I'll merge it assuming all the tests pass.
I know there's a lot of information in our contributing guidelines, so some points to take note of:
Once your PR is merged, you can be hired for another issue. Once you've completed a few issues, we may start hiring you for more than one issue at a time.
Once your PR is deployed to our staging servers, it will undergo quality assurance (QA) testing. If we find that this doesn't work as expected or causes a regression, you'll be responsible for fixing it. Typically we would revert this PR and give you another chance to create a similar PR without causing a regression. (I don't imagine this will happen with this PR, but it's something to be aware of)
Once your PR is deployed to production, we start a 7-day timer. After it has been on production for 7 days without causing any regressions, then we pay out the Upwork job.
So it might take a while before you're paid for your work, but we typically post multiple new jobs every day, so there's plenty of opportunity. I hope you've had a positive experience contributing to this repo!

🚀 Deployed to production by @luacmartins in version: 1.1.85-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Telegram Bot API

Hi, Is it possible to add support for Telegram Bot API? How?
https://core.telegram.org/bots
https://core.telegram.org/bots/api

I'm working on some documentation about building your own custom client but the best way is to study the https://github.com/Bottr-js/Bottr/blob/master/lib/clients/facebook-messenger-client.js
Basically right now the client listens out for a webhook event when somehting hits the /webhook endpoint and triggers the events.
It then implements some methods for sending a method etc.
I can invite you to our slack though if you have any questions on implementation.


[Clipboard] Error when pasting in browser

Describe the bug of the extension
The extension: Clipboard
The problem: When I try to use the action "Get text from the clipboard" from the extension and am testing on a browser via a development build (using "Publish" then "Generate link" in the app), I get nothing except an error in the console.
Steps to reproduce the behaviour:

Use the ReadTextCrossPlatform (full name Get text from the clipboard) from the Clipboard extension
Publish the game via the app, using "Publish" then "Generate link"
Open the game, from the generated link, in any browser
Try using the action

Other details

Please include a link to a game if possible!

https://liluo.io/games/bc0b3558-80af-4aac-a8da-d9094913f728
game.zip

If applicable, add screenshots to help explain your problem.

On Firefox:

Include any OS/browser version/smartphone that you're using:

Mozilla Firefox 103.0.1 (64-bit)

Which version of GDevelop are you using? The desktop app or the web-app?

I am using what I think is the last version (5.0.139) of the GDevelop desktop app

I might try to look into this myself, but I am not sure I have the time/skill. Any hints appreciated anyway :).

As far as I could understand the problem, it seems that the iframe containing the game on Liluo should have the allow-storage-access-by-user-activation attribute set to true to allow the browser to even ask the user if they want to allow copy/pasting.
I am guessing that this means a potential fix is in the hands of the GDevelop Team? Also that this might be intentional and done for security reasons?


Unable to cast object of type 'System.Func`2 using entity framework core. v1.1.1

var entry = _context.Entry(obj);  foreach (var property in entry.OriginalValues.Properties) { var current = entry.CurrentValues.GetValue<object>( property.Name );  //error occurring this line entry.Property(property.Name).IsModified = current != null; }

@irowbin Not sure why the exception is being thrown when using GetValue<object>, but if you want to get the value in a non-generic way use var current = entry.CurrentValues[property.Name]; instead.

@ajcvickers it worked. Thanks

@ajcvickers The InvalidCastException is being thrown because propertyBase.GetPropertyAccessors().CurrentValueGetter in InternalEntityEntry.cs is a delegate representing a Func<InternalEntityEntry, T> where T is the type of the property being accessed. Since in most cases T will not be an object, an exception will be thrown.
You could replace
        public virtual TProperty GetCurrentValue<TProperty>(IPropertyBase propertyBase)
            => ((Func<InternalEntityEntry, TProperty>)propertyBase.GetPropertyAccessors().CurrentValueGetter)(this);

with
        public virtual TProperty GetCurrentValue<TProperty>(IPropertyBase propertyBase)
            => (TProperty)propertyBase.GetPropertyAccessors().CurrentValueGetter.DynamicInvoke(this);

But I'm not sure this is worth doing, since as you already suggested you can use the non-generic method.


[fix][broker] Fix broker is not able to load topic with broken schema ledger

Motivation
If due to any operation reason if schema ledger metadata gets deleted then broker is not able to handle that non-recoverable error and not able to load the topic. So, if one of the schema ledger is broken/deleted or ledger metadata is missing , then broker logs below exception because ledger without metadata gives NoSuchLedgerExistsOnMetadataServerException which is not considered as non-recoverable error.
2023-03-21T10:40:17,930-0700 [main-EventThread] ERROR org.apache.pulsar.broker.service.schema.SchemaRegistryServiceImpl - [prop1/ns1/t2] Put schema failed
java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.schema.exceptions.SchemaException: No such ledger exists on Metadata Server -  ledger=8 - operation=Failed to open ledger
        at java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:368) ~[?:?]
        at java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:377) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:1097) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
        at org.apache.pulsar.broker.service.schema.SchemaRegistryServiceImpl.lambda$trimDeletedSchemaAndGetList$30(SchemaRegistryServiceImpl.java:553) ~[pulsar-broker-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
        at org.apache.pulsar.broker.service.schema.BookkeeperSchemaStorage.lambda$openLedger$41(BookkeeperSchemaStorage.java:603) ~[pulsar-broker-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.apache.bookkeeper.client.LedgerOpenOp.openComplete(LedgerOpenOp.java:267) ~[bookkeeper-server-4.15.4.jar:4.15.4]
        at org.apache.bookkeeper.client.LedgerOpenOp.lambda$initiate$0(LedgerOpenOp.java:118) ~[bookkeeper-server-4.15.4.jar:4.15.4]
        at java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:990) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:974) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
        at org.apache.pulsar.metadata.bookkeeper.PulsarLedgerManager.lambda$readLedgerMetadata$2(PulsarLedgerManager.java:208) ~[pulsar-metadata-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
        at org.apache.pulsar.metadata.impl.ZKMetadataStore.handleGetResult(ZKMetadataStore.java:269) ~[pulsar-metadata-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.apache.pulsar.metadata.impl.ZKMetadataStore.lambda$batchOperation$5(ZKMetadataStore.java:219) ~[pulsar-metadata-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.apache.pulsar.metadata.impl.PulsarZooKeeperClient$3$1.processResult(PulsarZooKeeperClient.java:489) ~[pulsar-metadata-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]

Modifications
Handle NoSuchLedgerExistsOnMetadataServerException non-recoverable error and successfully load such topic with broken schema ledger.
Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

[ ] Dependencies (add or upgrade a dependency)
[ ] The public API
[x] The schema
[ ] The default values of configurations
[ ] The threading model
[ ] The binary protocol
[ ] The REST endpoints
[ ] The admin CLI options
[ ] The metrics
[ ] Anything that affects deployment

Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

Matching PR in forked repository
PR in forked repository:

The PR of type fix should be deferred to 3.0.1.

Codecov Report

Merging #19882 (1bd2776) into master (1e44ba1) will increase coverage by 44.95%.
The diff coverage is 74.72%.

@@              Coverage Diff              @@
##             master   #19882       +/-   ##
=============================================
+ Coverage     24.42%   69.38%   +44.95%
- Complexity      291     3652     +3361
=============================================
  Files          1603     1873      +270
  Lines        124343   152550    +28207
  Branches      13571    18653     +5082
=============================================
+ Hits          30369   105841    +75472
+ Misses        89490    37775    -51715
- Partials       4484     8934     +4450

Flag
Coverage Δ

inttests
25.77% <23.07%> (+1.35%)
:arrow_up:

systests
25.12% <21.95%> (?)

unittests
71.75% <82.27%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...extensions/reporter/TopBundleLoadDataReporter.java
71.64% <0.00%> (+71.64%)
:arrow_up:

...broker/service/schema/BookkeeperSchemaStorage.java
78.72% <ø> (+17.32%)
:arrow_up:

...nsions/policies/AntiAffinityGroupPolicyHelper.java
37.77% <38.46%> (+37.77%)
:arrow_up:

...metadata/coordination/impl/LeaderElectionImpl.java
73.14% <66.66%> (+20.24%)
:arrow_up:

...ar/metadata/coordination/impl/LockManagerImpl.java
89.47% <71.42%> (+31.57%)
:arrow_up:

...java/org/apache/pulsar/common/util/FutureUtil.java
80.55% <76.92%> (+50.02%)
:arrow_up:

...dbalance/extensions/scheduler/TransferShedder.java
66.40% <80.00%> (+66.40%)
:arrow_up:

...ker/loadbalance/extensions/models/TopKBundles.java
90.78% <87.17%> (+90.78%)
:arrow_up:

...org/apache/pulsar/broker/ServiceConfiguration.java
98.66% <100.00%> (+0.76%)
:arrow_up:

... and 1583 files with indirect coverage changes

/pulsarbot run-failure-checks

On the first question, I asked the same question in PR 18010. Whether it should be at the time of enable autoSkipNonRecoverableData, to get NoSuchLedgerExistsOnMetadataServerException as a recoverable error.
https://github.com/apache/pulsar/pull/18010#discussion_r1140357628

@rdhabalia Could you please take a look at @poorbarcode 's comment?


[Feature Request] Implement a proper pause/resume

Thanks for this amazing software!
The only issue is that if your server crash for any reason / the script stop working you have to restart from the 0.
Would be asking too much to implement a proper pause / resume after crash system?
Also your script seems to just stop running and exit without any error or issues...it just exit...
I'm running your script with python batch_download.py DOI_60000000-64871768.txt false 1 > python.log &
Every time it exists I see nothing on python.log no error, nothing.

Hi and many thanks for utilizing the script.
Apologies for any inconvenience you may have experienced, due to my amateur implementation.
I've tried to address the issue of "proper pause / resume after crash", by listing the PDF files saved in the working directory and excluding them from the downloading process (when scripts runs anew).

No need to apologize for anything lol :)
I think the biggest issue is why it's just exiting without any error, anything really. The only time I got a error message in my console it was :
File "batch_download.py", line 109, in <module>
    work(dois)
  File "batch_download.py", line 91, in work
    download_doi_pdf(doi)
  File "batch_download.py", line 41, in download_doi_pdf
    response = urllib2.urlopen(req)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 502: Bad Gateway

I can help you debug this, what is the proper way to log the output of the script then? So far seems to be working stable now, I left my putty session open but I don't know if was that or no ( I still send them to the background with & )


Port highfive to python3

r? @kennytm or @pietroalbini
I don't know python well enough, nor am I sure we should be moving to python3 yet -- I want to discuss it in an infra meeting first.

I don't know if my voice counts but this is very nice clean change. Python 2 is going to be deprecated next year (end of support), now it's good time to switch to 3.


The list.collapseAll keybinding stopped working after 1.40 version

VSCode Version: 1.40 (1.41.0-insider)
OS Version: Linux Mint 19.2

Steps to Reproduce:

Go to files Explorer and use ctrl+leftArrow keybinding
Nothing happens :man_shrugging:

Keyboard Shortcuts say it should work

Does this issue occur when all extensions are disabled?: Yes

My bad.
Workaround until fix comes out: In keyboard shortcuts find list.scrollLeft, right click and select Remove Keybinding
Fix will also come soon on Insiders.


Timestamp, Adobe Reader & "date/time are from the clock on the signer’s computer"

Hi!
First, congrats for the project! It's fantastic!
I have a quirk about the time stamp and, well, I don't know if it's a feature of the project or some wrong implementation I did.
I'm getting a valid timestamp from a server and embedding it in the code. However, when I sign the PDF, Adobe Reader/Acrobat reports that “signature date/time are from the clock on the signer’s computer”.
Do the output PDF (digital signature + timestamped) have "correct" timestamp validation when opened in Adobe (“The signature includes an embedded timestamp”), or is it anyway - i.e., will this message always appear?

That's an excellent question, and one that I should probably address more fully in the documentation.
The short answer is: yes, pyHanko supports embedding timestamps, and here's how to get that to work.
The slightly longer answer is that, as the diagnostic message in Acrobat implies, the point of embedding timestamps is to ask a trusted third party to tell you what time it is, instead of forcing the signature's validator to take your word for it. In order to do that, you basically need to tell pyHanko who to ask. There's no "default" way to get timestamp tokens, so unless you provide pyHanko with a TSA endpoint, it'll just put the time on your computer's clock into the signature container.
The way this works on the technical level is by sending a digest of the signature to a time stamping authority, who will reply with a signed, timestamped statement that certifies that the signature existed at that time. This token is then stapled to the signature. There are quite a few widely trusted timestamp authorities that provide their services to the public for free. You can find a list of those here (note: possibly some of these are no longer operational, I haven't tried them all). Alternatively, if you're working with government paperwork, chances are that your government PKI also supplies a TSA service.

Hi, @MatthiasValvekens!
Thanks for the fast support :)
Hmmm, strictly speaking, my question is different. I don't think I was clear enough, sorry. To put it in other terms:
If all the assumptions made are true (correct encoding, correct implementation of the time stamp, TSA trusted according to Adobe etc), the output PDF from pyHanko would be “certifiable” (ie, Adobe would show the message that The signature includes an embedded timestamp)? Or, even so, with everything “right” according to pyHanko, Adobe would still show the message that signature date/time are from the clock on the signer’s computer?
I know that for many this would be a silly, trivial validation. But in my case it is essentially important. Hence the question.

Yes, if everything is done correctly, Acrobat should display "The signature includes an embedded timestamp".
And you're quite right that timestamp validation is important :)

I'll close this question now, feel free to reopen if you encounter related issues later.


CASMINST-5830 Enable backport-command on csm repo

Summary and Scope
Enables backport-command workflow on csm repo. For any existing PR (open or closed), add a comment /backport target-branch and this workflow will create a new PR, which backports original PR to target-branch.
Issues and Related PRs

Resolves [issue id](issue link)

Testing
No testing can be done until workflow is added to main branch.
Risks and Mitigations
Low - adds workflow, which is invoked on pr comment only. If it breaks, it may only generate additional comments to the PR.

Merging as failures in Jenkins are not related to this PR.


Fixed #13910 -- Added generator based rendering (streaming) to templates and supporting views.

Rebased and optimized https://github.com/django/django/pull/4783

Thanks for taking care of cleaning that up @pglotov :+1: . I'm thrilled to see that in 3.0 :smile: .

Yes, it seemed easier than rigging something up to handle timing out requests :)

I have no idea why pr-mariadb/database=mysql,label=mariadb,python=python3.7 fails.

The failed test doesn't seems to be on our end: ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?

it would need you to squash commits in a single one using co-authored with tag in git I guess.

Ok that should be all good then.

It's still red:
test_chunked_data (builtin_server.tests.ServerHandlerChunksProperly) ... Killed
Build step 'Execute shell' marked build as failure
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Finished: FAILURE

Looks like several other recent PRs fail this test with same exact message.

Performance: master vs pr:
Server Software:        WSGIServer/0.2
Server Hostname:        <IP_ADDRESS>
Server Port:            8001

Document Path:          /polls/streaming/
Document Length:        268906 bytes

Concurrency Level:      1
Time taken for tests:   16.633 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      26909300 bytes
HTML transferred:       26890600 bytes
Requests per second:    6.01 [#/sec] (mean)
Time per request:       166.333 [ms] (mean)
Time per request:       166.333 [ms] (mean, across all concurrent requests)
Transfer rate:          1579.88 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       0
Processing:   141  166  28.4    153     279
Waiting:      140  166  28.3    153     278
Total:        141  166  28.4    153     279

Percentage of the requests served within a certain time (ms)
  50%    153
  66%    163
  75%    172
  80%    189
  90%    207
  95%    232
  98%    262
  99%    279
 100%    279 (longest request)
######################################################
Server Software:        WSGIServer/0.2
Server Hostname:        <IP_ADDRESS>
Server Port:            8001

Document Path:          /polls/streaming/
Document Length:        268906 bytes

Concurrency Level:      1
Time taken for tests:   15.726 seconds
Complete requests:      100
Failed requests:        0
Total transferred:      26909300 bytes
HTML transferred:       26890600 bytes
Requests per second:    6.36 [#/sec] (mean)
Time per request:       157.262 [ms] (mean)
Time per request:       157.262 [ms] (mean, across all concurrent requests)
Transfer rate:          1671.01 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   140  157  15.4    153     217
Waiting:      140  157  15.4    153     216
Total:        140  157  15.4    153     217

Percentage of the requests served within a certain time (ms)
  50%    153
  66%    157
  75%    161
  80%    163
  90%    174
  95%    190
  98%    215
  99%    217
 100%    217 (longest request)

What are the next steps to see this PR merged ?

I guess uncheck the button patch needs improvement and need docs test etc in the related ticket so that the core fellows could do the final round of review to merge it.

I don't think i have permissions to uncheck the flag, is anyone able to? https://code.djangoproject.com/ticket/13910

done

Thanks :+1:

done

made changes

For reference, a link @Gagaro's benchmarking project for this change. https://github.com/Gagaro/django_benchmark_streaming_templates

Hi @carltongibson, i'm not sure it's a good idea to only leave Template.render(... stream=True) as the only way to use template streaming. I was actually looking forward to just replace TemplateView with StreamingTemplateView as base class in a bunch of views which tend to cause timeout on aws.

Hi @pglotov. Sure. You're welcome to include versions of SreamingTemplateResponse and StreamingTemplateView in your own project.

Hmm.. isn't the point of Django to make users' life easier?

It definitely would NOT make my life easier .. I would like to get a second opinion on this. How should i go about it (I'm not familiar with power hierarchy in Django project).

I removed StreamingTemplateView related classes, and I'd like to find a solution (possibly adding stream=True parameter to TemplateView) to be able to easily use template streaming with a class based view.

made changes

@carltongibson @felixxm We seem to have a problem with pull requests for older Django Trac tickets closing unrelated open pull requests, e.g. this one.
This is because we structure commit messages with the prefix Fixed #11157 -- ... which GitHub takes to mean that issue or PR #11157 is resolved by that commit. See here.
I fear that this will become more common. Is it time to consider changing the format of commit messages, e.g. Fixed T11157 -- ... or Fixed ticket-11157 -- ...? (Or something better than those...)

Hey @pope1ni. Yes, it happens from time to time. 🤔
Let's close due to inactivity. Great if someone wants to pick this up.


ability to include/exclude relation data in payload (fixes #63)

This PR fixes #63 by allowing an included attribute to be specified as part of the relations option. Previously, the relations option was either a boolean or an array of strings. This has now been changed to either a boolean or an object containing options that determine how a relationship is processed.
The array of relation names has now been moved into a fields property. Example:
relations: {
  fields: ['rel1', 'rel2']
}

The new syntax also allows for an included option to determine whether the relation data is included as a compound document, which has been the default thus far. Setting included: false will not include the actual relation data in the payload.
Scenarios:
relations : true // all in, to maintain the current behaviour
relations : false // nothing
relations : { included: true } // relation data included in payload
relations : { included: false } // relations defined, but data not included in payload

@chamini2 @ShadowManu would love your feedback on this.

Note that I haven't updated the README yet, but will do so after getting your feedback.

@chamini2 thanks for the thorough review. I just pushed a few fixes from above.

If you're good with the changes, LGTM 👍

1.0.0-beta.4 is live


SPR-124 GC Notify

Description
This PR includes the following proposed change(s):

Implemented GC Notify, currently only sending message when new request is submitted.
Replaced use of HOSTNAME env. ENVIRONMENT was already needed for Keycloak, and provided the same function.

Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[x] Refactoring / Documentation
[ ] Version change
[ ] Dependencies added/removed

Aspect(s) of Project Affected

[ ] Frontend
[x] API
[ ] Database
[x] Workflows
[x] Documentation

Best Practices Checklist

[x] I have performed a self-review of my own code
[x] My code follows the Airbnb React Style Guidelines
[x] Documentation has been updated to reflect my changes
[ ] New and existing tests pass locally with my changes

NOTE: Tests have had the endpoints updated, but there's a big change in how requests are handled coming soon. At that point, a rework of any /requests tests must be done.
How Has This Been Tested?
Please describe the tests that you ran to verify your changes.
Tested locally. Containers build and run expectedly.

We may want to rethink implementing GC Notify as I've only recently found out that it doesn't comply with BC Gov guidelines  / legislation.


Make ipaddress dependency conditional on python2

Note, right now this patch will be an no-op because txtorcon has not
yet released a new version which contains this patch.  Once it has
done so, however, this will make the build dependency exclude
ipaddress on python3 environments.

Do we know why ipaddress was needed in the first place? I see it showed up when we added tor, and originally it was only included in the extras_require= [tor] feature. Then I did some refactoring (2d949f23) that (accidentally?) made it an unconditional requirement. Is this something that txtorcon needs? Should txtorcon be expressing this requirement internally? It doesn't look like there's any code in magic-wormhole that imports ipaddress.

Local unit tests pass if I just remove ipaddress from setup.py entirely. I haven't tried actually using the --tor support, though.

txtorcon does express this requirement: https://github.com/meejah/txtorcon/blob/master/requirements.txt#L5


Revert "Move log config file from log service dir into log dir"

Reverts hw-cookbooks/runit#82

@mkropat This fails testing currently.  Let's take another pass at this.
      * Parent directory /var/log/plain-defaults does not exist.
      ================================================================================
      Error executing action `create` on resource 'template[/var/log/plain-defaults/config]'
      ================================================================================

           Chef::Exceptions::EnclosingDirectoryDoesNotExist
           ------------------------------------------------
           template[/var/log/plain-defaults/config] (dynamically defined) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /var/log/plain-de
faults does not exist.

[2015-02-12T06:56:45+00:00] ERROR: runit_service[plain-defaults] (runit_test::service line 53) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: template[/var/log/plain-defaults/config] (dynamically defined) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /var/log/plain-defaults does not exist.

I'm having a heck of a time trying to get bundle exec kitchen test to run on my Linux Mint 17 box. It seems to be some conflict between the system vagrant and my rbenv instance of Ruby.
I'd love to see this through, but it might be a little while before I'll be in a place to do so.

I took another crack at this today and I got test kitchen running on my Windows box.
But then I got stuck trying to make a change to always create the log directory (since the config file is always created there). The change causes the tests to fail with a weird error:
Failures:

  1) Chef::Provider::Service::Runit actions action_enable enables the service with memoized resource creation methods
     Failure/Error: provider.run_action(:enable)
     NameError:
       directory[/var/log/getty.service] (dynamically defined) had an error: NameError: uninitialized constant Chef::Pro
vider::File::Content
     # ./libraries/provider_runit_service.rb:124:in `configure_service'
     # ./libraries/provider_runit_service.rb:93:in `block in action_enable'
     # ./libraries/provider_runit_service.rb:92:in `action_enable'
     # ./test/spec/libraries/provider_runit_service_spec.rb:482:in `block (4 levels) in <top (required)>'

  2) Chef::Provider::Service::Runit actions action_enable new resource conditionals creates the env dir and config files
 if env is set
     Failure/Error: provider.run_action(:enable)
     NameError:
       directory[/var/log/getty.service] (dynamically defined) had an error: NameError: uninitialized constant Chef::Pro
vider::File::Content
     # ./libraries/provider_runit_service.rb:124:in `configure_service'
     # ./libraries/provider_runit_service.rb:93:in `block in action_enable'
     # ./libraries/provider_runit_service.rb:92:in `action_enable'
     # ./test/spec/libraries/provider_runit_service_spec.rb:661:in `block (5 levels) in <top (required)>'

  3) Chef::Provider::Service::Runit actions action_enable new resource conditionals creates the control dir and signal f
iles if control is set
     Failure/Error: provider.run_action(:enable)
     NameError:
       directory[/var/log/getty.service] (dynamically defined) had an error: NameError: uninitialized constant Chef::Pro
vider::File::Content
     # ./libraries/provider_runit_service.rb:124:in `configure_service'
     # ./libraries/provider_runit_service.rb:93:in `block in action_enable'
     # ./libraries/provider_runit_service.rb:92:in `action_enable'
     # ./test/spec/libraries/provider_runit_service_spec.rb:668:in `block (5 levels) in <top (required)>'

The stack trace gives me no clue as to why it's throwing that NameError.  So that's where I'm stuck, in case anyone's curious.

This PR seems to be in an odd state following the merge of #107. Can you please retest the issue and adjust as needed?

Closed by #135


Support check_run for Github/Cirrus

Adds support for https://developer.github.com/v3/activity/events/types/#checkrunevent which
allows Cirrus events to display.
There's plenty of room for improvement here but it works for now.

There's some "check_suite" event that also comes through that isn't very useful as far as I can tell.
The lack of showing the git hash can be confusing when multiple runs are involved. The PR URL is
not sent unless the PR is from the same repo. This flaw exists in the existing "status" hook as well. I think most of the notifications should include a shortened hash and branch name.

Example output (with verbose queued enabled):
12:28:42 <@    gitlogs>| [poudriere] Check Run for freebsd11 Queued. https://cirrus-ci.com/task/5662741731213312
12:33:12 <@    gitlogs>| [poudriere] Check Run for freebsd11 completed: Success. https://cirrus-ci.com/task/5662741731213312

12:33:56 <@    gitlogs>| [poudriere] Check Run for freebsd11 queued. https://cirrus-ci.com/task/5654801989238784
12:34:07 <@    gitlogs>| [poudriere] Check Run for freebsd11 completed: Cancelled. https://cirrus-ci.com/task/5654801989238784

12:39:18 <@gitlogs> [poudriere] bdrewery opened pull request #725: TEST - https://git.io/JectH
12:39:19 <@    gitlogs>| [poudriere] Check Run for freebsd11 queued. https://cirrus-ci.com/task/5152479666241536
12:39:21 <@    gitlogs>| [poudriere] Check Run for freebsd12 queued. https://cirrus-ci.com/task/6278379573084160
12:49:23 <@    gitlogs>| [poudriere] Check Run for freebsd12 completed: Success. https://cirrus-ci.com/task/6278379573084160
12:52:28 <@    gitlogs>| [poudriere] Check Run for freebsd11 completed: Success. https://cirrus-ci.com/task/5152479666241536

Looks good from a quick skim, thanks for the contribution. I need to add some tests for github but this should go live on n.tkte.ch on Saturday.


Updated maven pom

Updated Maven version to 3.3 (to newer version than pull request 132) left compile target at 1.7
Removed com.keyboardsamurais.maven as no longer necessary in newer Maven
versions
Updated Jacoco version for Java 8 (same as pull request 191 by
XDarklight)
Merged duplicate entries to prevent build warnings
Added maven-clean-plugin version to prevent build warning
Minor update of Solr version to 4.10 (note updating to Solr 5 causes
problems with dependency injection)

@csokol did some more pom.xml fixes (2dddd8efec702ca36c950211287cb144a7038c43 and 1f61cfccb65a80e6ca42035c134706a1b4936c65).
Could you please update this PR (since there are merge conflicts now)?

Pull request updated to fix merge conflicts.

Thanks, @JamesSullivan!


Add license lint into CI

Task description:
Previously in https://github.com/kurator-dev/kurator/pull/490, i added a license lint checking script. But did not add the check in CI.
Solution:
Who can join or take the task:
The good first issue is intended for first-time contributors to get started on his/her contributor journey.
After a contributor has successfully completed 1-2 good first issue's,
they should be ready to move on to help wanted items, saving the remaining good first issue for other new contributors.
How to join or take the task:
Just reply on the issue with the message /assign in a separate line.
Then, the issue will be assigned to you.
How to ask for help:
If you need help or have questions, please feel free to ask on this issue.
The issue author or other members of the community will guide you through the contribution process.

This issue has already been processed.

/close


Hide the Table internals in Table and DB_Table

Since #9692 has been merged, we can finally properly hide internal implementation details of our types! 🎉 (including their internal data/state)
We should use that to make sure the internal implementation of our Table and DB_Table types is properly encapsulated.
To do so:

[x] make the constructors of Table and DB_Table private
[ ] additionally, we could revise internal helper methods on these types and try to move them into private modules as well. These are e.g. as_subquery, updated_context_and_columns etc.

Waiting for private methods


Platform does not match error when uploading an Android app to App-Center

I'm trying to upload an Android app to a new Organisation and I get the following error:
{"status"=>"error", "message"=>"Platform does not match. Expected value: iOS. Actual value: Android."}
The organisation has no apps yet and I couldn't find anything in the settings to tell it's an iOS only (We wish it to be both Android & iOS)
Appreciate your help.
appcenter_upload(
api_token: ENV["APPCENTER_API_TOKEN"],
owner_name: ownerName,
owner_type: "organization",
app_os: 'Android',
app_platform: 'Java',
app_name: appName,
apk: "./outputs/apksV5.x.x/" + filename,
notify_testers: true
)

thanks for the issue @golanshy, is this when using the fastlane plugin?

Yes, when using the fastlane plugin

Thanks for clarifying! We're taking a deeper look at this.

@golanshy we already have a ticket from you about this https://github.com/microsoft/fastlane-plugin-appcenter/issues/111
Have you reached out to our support? Also take a look at this comment, maybe it is the same case https://github.com/microsoft/fastlane-plugin-appcenter/issues/111#issuecomment-526215976

I'm closing this as a duplicate of https://github.com/microsoft/fastlane-plugin-appcenter/issues/111


selective_scan_cuda

Hi, author, where can i find the module named  'selective_scan_cuda'?

Did you follow the installation instruction in the README?
It's here and should be installed if you do pip install mamba-ssm: https://github.com/state-spaces/mamba/tree/main/csrc/selective_scan

Sorry,
but when i followed the installation, i failed to successfully install mamba-ssm, i met the following issue:

I tried both pip install and conda install, and i also tried to pip install . from this repository, all of them couldn't work.

Can you try downloading that wheel URL manually to check if the networking works?

@gnorZgnoD I run into the same problem, have you solved it? I already do pip install mamba-ssm, but it still report "no module named 'selective_scan_cuda' "

I run into the same problem,

抱歉，但是当我进行安装时，我无法成功安装mamba-ssm，我遇到了以下问题：我尝试了pip install和conda install，并且还尝试了pip install。从这个存储库中，所有这些都无法工作。
This is because the cuda version is too low and requires cuda 11.6 and above.

抱歉，但是当我进行安装时，我无法成功安装mamba-ssm，我遇到了以下问题：我尝试了pip install和conda install，并且还尝试了pip install。从这个存储库中，所有这些都无法工作。
This is because the cuda version is too low and requires cuda 11.6 and above.

I solved the issue by following operations:

I downloaded the mamba_ssm.whl file from https://github.com/state-spaces/mamba/releases, then i pip install the whl file in my working environment.
I updated my torch version to 2.0.0, and my torchaudio version to 2.0.1, and i used the cudatooklit11.8.

Did you follow the installation instruction in the README? It's here and should be installed if you do pip install mamba-ssm: https://github.com/state-spaces/mamba/tree/main/csrc/selective_scan

I have done pip install mamba-ssm successfully , but I still have this problem.

 Hello author, how should I solve this error? I downloaded the 1.1.4 version of the whl package and installed it.

I fixed the problem, you can't download the latest whl file, you can just recompile it with the previous whl file, Such as mamba_ssm - 1.1.3. Post1 + cu118torch2. 1 cxx11abifalse - cp310 - cp310 - linux_x86_64. WHL

FWIW, i also had the same problem and was able to solve it by pip install mamba-ssm. Earlier, I tried with pip install .

 Hello author, how should I solve this error? I downloaded the 1.1.4 version of the whl package and installed it.

hi, I met the same issue. Have you solved yet?

and mine here is:

Did you follow the installation instruction in the README? It's here and should be installed if you do pip install mamba-ssm: https://github.com/state-spaces/mamba/tree/main/csrc/selective_scan

I have done pip install mamba-ssm successfully , but I still have this problem.

Have you solved this problem?I meet the same question,can you tell me how to solve it please

I have done pip install mamba-ssm successfully, but selective_scan download is not successful

 Hello author, how should I solve this error? I downloaded the 1.1.4 version of the whl package and installed it.

I have the same question, have you be done this.

 Hello author, how should I solve this error? I downloaded the 1.1.4 version of the whl package and installed it.

I fixed the problem, you can't download the latest whl file, you can just recompile it with the previous whl file, Such as mamba_ssm - 1.1.3. Post1 + cu118torch2. 1 cxx11abifalse - cp310 - cp310 - linux_x86_64. WHL

I have the same question, i have pip install mamba_ssm but also meet this.

ImportError: /home/yida/miniconda3/envs/mambair/lib/python3.9/site-packages/selective_scan_cuda.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

I have the same question,have you finish this problem？

我又重新弄了环境，现在好了，你们可以看看下面这篇文章，我是使用vision mamba的。然后下载的whl进行安装的应该都差不多。
`wget https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.3.post1/causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
wget https://github.com/state-spaces/mamba/releases/download/v1.1.1/mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
`
https://blog.csdn.net/weixin_45667052/article/details/136311600

I have fixed the environment again, now it's okay. You can take a look at the following article. I'm using vision mamba. You can try installing it using a whl file, maybe you don't need to reinstall the environment. Pay attention to matching the versions of PyTorch and Python, etc.
wget https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.3.post1/causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
wget https://github.com/state-spaces/mamba/releases/download/v1.1.1/mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

pip install causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

https://blog.csdn.net/weixin_45667052/article/details/136311600

Thanks very much, I have down this question by your answer. Can success follow this link. The key is 'cp -rf mamba-1p1p1/mamba_ssm  /home/yida/miniconda3/envs/mamba/lib/python3.10/site-packages'.

Hi,
I figured out the problem with import mamba_ssm prompting import error:
I found out that it was the problem with using the newest torch.__version__=2.3.
I tested with cuda versions 12.2, 11.8; python 3.9, 3.10; torch 2.1, 2.2, 2.3. Only torch 2.3 is not able to import mamba_ssm.
Pictures are shown below.
Hope you can find this useful!

v1.2.1 now includes wheels for pytorch 2.3 so it should hopefully fix this issue

v1.2.1 now includes wheels for pytorch 2.3 so it should hopefully fix this issue

Thank you.

@tridao the problem exist for v2.0.3 for pytorch 2.3

我重新修复了一下环境，现在没问题了。你可以看看下面的文章。我用的是 Vision Mamba。你可以尝试使用 whl 文件安装，也许不需要重新安装环境。注意匹配 PyTorch 和 Python 的版本等。
wget https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.3.post1/causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
wget https://github.com/state-spaces/mamba/releases/download/v1.1.1/mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

pip install causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

https://blog.csdn.net/weixin_45667052/article/details/136311600

非常感谢，我通过你的回答解决了这个问题。可以成功点击此链接。密钥是“cp -rf mamba-1p1p1/mamba_ssm /home/yida/miniconda3/envs/mamba/lib/python3.10/site-packages”。

我重新修复了环境，现在没问题了。你可以看看下面的文章。我用的是 Vision Mamba。你可以尝试使用 whl 文件安装，也许不需要重新安装环境。注意匹配 PyTorch 和 Python 的版本等。
wget https://github.com/Dao-AILab/causal-conv1d/releases/download/v1.1.3.post1/causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
wget https://github.com/state-spaces/mamba/releases/download/v1.1.1/mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

pip install causal_conv1d-1.1.3.post1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
pip install mamba_ssm-1.1.1+cu118torch2.1cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

https://blog.csdn.net/weixin_45667052/article/details/136311600

非常感谢，我通过你的回答解决了这个问题。可以成功点击此链接。密钥是“cp -rf mamba-1p1p1/mamba_ssm /home/yida/miniconda3/envs/mamba/lib/python3.10/site-packages”。
您好，我已经成功安装以上版本的causal-conv1d和mamba-ssm,但是依旧报错ImportError xxxx causal_conv1d_cuda.cpython-xxx-linux-gnu.so undefined symbol的错误，我将对应版本的源码已替换到site-packages后，依旧报同样的错误，不知道是不是我的操作有问题，如：我下载源码mamba-1.1.1，做cp -rf mamba-1.1.1/mamba_ssm ./***/site-packages .还请您抽空解惑下，非常感谢

I am getting the same error.

do not work with torch 2.4.1...


Break endless loop

Hi. Is there a way to break an endless loop?
In example, I have some script that contains endless loop, and I can't check it before execution.
I want to destroy a runtime instance after delay, but I'm getting exception with message "A runtime that is still in use cannot be disposed".

JsDisposeRuntime doesn't stop execution it only frees the runtime object - which it obviously mustn't do if it's in use.
You should try using JsDisableRuntimeExecution to stop execution.
And you'll also want to do JsSetCurrentContext(JS_INVALID_REFERENCE); before disposing of the runtime.

Thanks a lot. To use it I've created a runtime instance with 'JsRuntimeAttributeAllowScriptInterrupt' flag.


Support for non-channel/server hilights

Hello!
Would it be possible to add a support for hilights which don't belong to any channel/server? Some plugins just have a window where they print their output, but even when a message has MSGLEVEL_HILIGHT and MSGLEVEL_MSGS flags, it doesn't make it to notification server (these messages have empty $lastMsg variable because they don't emit message {public,privmsg,dcc,etc.} signals).
Thank you.

Does highlights apply to those messages anyhow?

As I said - hilights for these messages meet the same criteria as for the 'regular' (channel) hilights (they have same level - MSGLEVEL_MSGS|MSGLEVEL_HILIGHT). The only problem is that they don't emit any message * signal (these signals can't be emitted for messages without a valid server object), thus the $lastMsg variable here is empty and the message won't get pushed into the notification queue.

Hopefully fixed by #165, needs some testing

Thanks for the info! Unfortunately, now irssinotifier.pl has a strange behavior when a non-channel hilight occurs. Instead of that hilight it sends (probably) the last message from a channel pool to the push queue. So instead of a non-channel hilight I see some random (from my point of view) message in my IrssiNotifier app (or push notification).

I thought that might happen, but hasn't happened for me so far. Do you have any idea how to reproduce it?

If it's only tripping the print_text signal then we need to realize that somehow by inspecting $dest? Alternately, turn what is print_text now into a helper function that each signal handler calls, including a new print_text handler (for non-server hilights), and instead of saving global variables, we just pass the msg, nick, dest, etc content to the other function. I just didn't want to overhaul it that much in my last pull request.
I don't get double messages from having hilightwin.pl running, so I'm sorry I didn't notice this.

Here's a simple plugin, which reproduces aforementioned issue. Just set irssinotifier_screen_detached_only and irssinotifier_away_only options to OFF and load the script. You should receive a push message every 2 seconds containing a last message from your channel pool.

You seem to be pretty handy with Perl scripting, maybe you could fix it
yourself and send a pull request? :)
If not, I'll take a look at this some day...
On Sat, 26 Mar 2016 21:50 Frantisek Sumsal<EMAIL_ADDRESS>wrote:

Here's a simple plugin
https://gist.github.com/mrc0mmand/4425e70c991cf670d43d, which
reproduces aforementioned issue. Just set
irssinotifier_screen_detached_only and irssinotifier_away_only options to
OFF and load the script. You should receive a push message every 2
seconds containing a last message from your channel pool.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/murgo/IrssiNotifier/issues/178#issuecomment-201914090

I can definitely look into it, but it won't be in the nearest future. Nevertheless, thank you both for all your effort so far, patching will be much easier now thanks to the information you provided.


[Bug]: toEqual() fails jest-worker with "Do not know how to serialize a BigInt" when reporting a failed test

Version
27.4.7
Steps to reproduce
expect(BigInt(0)).not.toEqual(BigInt(0));
Expected behavior

The expectation should produce the typical expected/received message.

When an individual test fails catastrophically, the failing test should be identified to the user and the entire suite should continue with any remaining tests.

Actual behavior
Summary of all failing tests
FAIL  tests/parsers.test.ts
● Test suite failed to run

        TypeError: Do not know how to serialize a BigInt

            at stringify (<anonymous>)

          at messageParent (node_modules/jest-worker/build/workers/messageParent.js:34:19)

Test Suites: 1 failed, 46 passed, 47 total
Tests:       285 passed, 285 total
Snapshots:   0 total
Time:        9.718 s, estimated 10 s
Ran all test suites.

Additional context
Possibly related issues:

11617 - jest-worker fails with "Do not know how to serialize a BigInt" in messageParent.ts instead of showing the actual assertion error message when there are 2 or more test suite files executing in parallel

11624 - [jest-worker]: bigInt serialization in jest-worker

Environment
System:
    OS: Linux 5.13 Linux Mint 20.3 (Una)
    CPU: (12) x64 Intel(R) Core(TM) i5-10400 CPU @ 2.90GHz
  Binaries:
    Node: 14.19.0 - /usr/bin/node
    npm: 8.4.1 - /usr/bin/npm
  npmPackages:
    jest: ^27.4.7 => 27.4.7

Please respond to my second expectation. It would have been easier for me to track down if the test name was included with the other information. Is it possible to add a catch handler inside the loop running the individual tests?

This actually fails outside of the context of a single test (it fails when jest tries to pass the test result from a separate thread/process back to the main process), so we don't know which specific test failure caused this error.
I hope to address this issue in Jest 28 (see https://github.com/facebook/jest/pull/11467#issuecomment-1040849566 for latest info)

If no object has both the test name and result to format at the same time, is it possible to pass the test name along to include in the error message?


Add inbuilt support for all kubernetes versions

Is your feature request related to a problem? Please describe.
Move2Kube internally has cluster.yaml only for one kubernetes version
Describe the solution you'd like
All cluster yamls for all kubernetes versions
Describe alternatives you've considered
Doing a move2kube collect and specifying it for each version.

Supporting and maintaining the list will be a recurring exercise. Instead, we will use move2kube collect to support all versions.


Updated UI to not print the commands all the time

UI now prints the available commands with "help"-command. All the available commands are not printed every time the user is asked to enter a command.

Codecov Report

Merging #6 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #6   +/-   ##
=========================================
  Coverage     89.97%   89.97%
  Complexity      101      101
=========================================
  Files             9        9
  Lines           409      409
  Branches         49       49
=========================================
  Hits            368      368
  Misses           33       33
  Partials          8        8

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b685d42...6bd139b. Read the comment docs.


New version: Dagger v0.12.0

Registering package: Dagger
Repository: https://github.com/JuliaParallel/Dagger.jl
Created by: @jpsamaroo
Version: v0.12.0
Commit: e6b3cdca6a50b983a51250fd479dfae17f7944a6
Reviewed by: @jpsamaroo
Reference: https://github.com/JuliaParallel/Dagger.jl/commit/e6b3cdca6a50b983a51250fd479dfae17f7944a6#commitcomment-54485260

@jpsamaroo please see the message above.  Once you fix the automerge issues you can register again the new revision without changing the version number and this pull request will be automatically updated.
[noblock]


[WIP] Framework Modification needed for ChainStore

PR Description
added the ability to cache and retrieve BeaconState in ChainStore.  T…his caused a massive series of circular dependencies that were a result of bad design.  the end result is an improved structure
Fixed Issue(s)

part of #368

Added WIP label until team is aware to prevent accidental premature merging. Feel free to remove when you're ready @jrhea.


build(app): fix artifacts exclusion for other app

The same github workflow builds and optionally deploys app builds for robot-stack and ot3. This process was almost right but had two bugs in it, both around the intermediate artifact handling that links building the app and deploying it.
The robot-stack and ot3 apps are built in separate jobs. There is one job per combination of OS and stack - so ot3 on linux, robot stack on mac, ot3 on windows, etc. Each is a separate job. Each builds the app and an electron-updater release manifest. The app artifact is always copied to artifact storage; the release manifest is copied if and only if this is a release build for this stack. The app and optional release manifest are put in the same directory, which is zipped up and named opentrons-$os.zip for the robot stack and opentrons-ot3-$os.zip for the ot3 stack.
There is a separate deploy job for each stack. Each deploy job depends on the build jobs for its stack, and therefore is guaranteed not to run unless the build jobs for its stack have completed successfully. However, there is no causal link between each stack's deploy job and the other stack's build jobs, and artifact storage is shared - so when each deploy job runs, it will certainly have its own stack's artifacts, but may also have all, some, or none of the other stack's artifacts.
That means that each deploy job must make sure that it does not deploy the other stack's artifacts, and here are the bugs:

the OT3 deploy job only copies over OT3 artifacts. However, it did this by copying over only the app artifacts themselves, and thus while it would only copy over the OT3 app, it will only ever copy over the OT3 app - not the release manifest. Therefore, release jobs will not result in app update notifications.
The OT2 deploy job removes the OT3 artifacts before copying over the remaining files. However, it did this by removing only the app artifacts themselves, and thus while it would not copy over the OT3 app, it would only not copy over the OT3 app - any release manifests in the OT3 artifacts would get deployed to the robot-stack storage.

Both these bugs can be fixed by specifying "ot3 artifact" by the name of the folder it unzips to rather than the name of the specific files inside that folder.
Testing and Review
As you can tell from a 500 word PR description accompanying a two-line change, this is an annoying area of code to deal with and needs to be deeply inspected for knock-on effects.
Unfortunately, the only way to test this I believe is to do some releases. Let's do those as alphas to make sure they don't get out, and let's try and do them in a time that won't obstruct robot stack release testing if they go wrong.
You can check whether OT3 builds were mistakenly uploaded to the OT-2 bucket by checking this link: http://s3.amazonaws.com/opentrons-app/builds/alpha-linux.yml and inspecting the file your browser downloads
You can check whether an OT3 build got uploaded correctly to the OT-3 bucket by checking this link: http://ot3-development.builds.opentrons.com/app/alpha-linux.yml and inspecting the file your browser downloads (or noticing that it's an access error because it hasn't successfully been uploaded yet)

I think I tenuously understand this. Let me see if I have it right.
This workflow creates a set of GitHub Actions artifacts like this:

opentrons-macos-11/

Opentrons-blahblah-version-blahblah.dmg
latest-mac.yml (should only be present if this run is intended to update the live stable channel)
beta-mac.yml (should only be present if this run is intended to update the live beta channel)
alpha-mac.yml (should only be present if this run is intended to update the live alpha channel)

opentrons-ot3-macos-11/

Ditto.

opentrons-ubuntu-22.04/

Opentrons-blahblah-version-blahblah.AppImage
latest-linux.yml (should only be present if this run is intended to update the live stable channel)
beta-linux.yml (should only be present if this run is intended to update the live beta channel)
alpha-linux.yml (should only be present if this run is intended to update the live alpha channel)

etc. for other combinations of platform and stack.

At the end of the workflow, it copies all of this from GitHub Actions artifact storage to AWS S3 storage. This copying is split between two jobs: one for uploading to OT-2 AWS S3 storage, and one for uploading to OT-3 AWS S3 storage.
Each job downloads everything from GitHub Actions artifact storage and is responsible for sifting through it so it only uploads the proper things to its designated target. The bugs are that it doesn't do that totally right. So OT-3 stuff can leak into the OT-2 storage, and some things can be missing from the OT-3 storage.

I'm gonna do those in a followup PR though - this needs to get merged to produce a build for testing.


JsonSerializerOptions.Clone fixed

Added missing properties.

@mehyaa, thanks for the commit!  I've updated the reference and republished the package as a patch.


Turborepo crashing with pre-commit hooks

Verify canary release

[X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue
git commit -m ""
What package manager are you using / does the bug impact?
pnpm
What operating system are you using?
Mac
Which canary version will you have in your reproduction?
2.0.1, 2.0.2-canary.1
Describe the Bug
I have a pre-commit hook. I am using lefthook for it. I am using Biome for Linting and Vitest for testing. Here is the file..
pre-commit:
  parallel: true
  commands:
    fmt:
      run: pnpm fmt
    test:
      run: pnpm test

If I run a git commit -m "<COMMIT-MSG>" it crashes.  Here are some Logs that turborepo prompted me to post..
   1   │ name = "turbo"
   2   │ operating_system = "Mac OS 14.4.1 [64-bit]"
   3   │ crate_version = "2.0.1"
   4   │ explanation = """
   5   │ file 'crates/turborepo-lib/src/commands/run.rs' at line 57
   6   │ """
   7   │ cause = "render thread panicked: JoinError::Panic(Id(26), ...)"
   8   │ method = "Panic"
   9   │ backtrace = """
  10   │
  11   │    0: 0x102deaaa4 - __mh_execute_header
  12   │    1: 0x1033a8d6c - __mh_execute_header
  13   │    2: 0x103419ec4 - __mh_execute_header
  14   │    3: 0x103415194 - __mh_execute_header
  15   │    4: 0x1035eabcc - __mh_execute_header
  16   │    5: 0x10381c850 - __mh_execute_header
  17   │    6: 0x103819108 - __mh_execute_header
  18   │    7: 0x103268ff4 - __mh_execute_header
  19   │    8: 0x103267ae4 - __mh_execute_header
  20   │    9: 0x1032686e0 - __mh_execute_header"""

Expected Behavior
I shouldn't need to run the commands individually in order for the commit hook to run the underline command without issue.
To Reproduce
I guess, we need a Monorepo with Pnpm, Turbo 2.0.1 and Lefthook. Important part here that this used to work as expected.
Additional context
No response

Hi, this is a duplicate of #8323, this should be fixed in the latest canary (2.0.2-canary.0). Feel free to re-open the issue if that's not the case.

I tried with a Canary after what y suggested. It's not working.

Gotcha, could you try adding TURBO_UI=0 to the turbo command?

Yeah, with the old UI it is working.. I can leave with that for now.. Thanks!

parallel: true

Can you remove this line and see if that changes anything?
Could you also try to provide a minimal reproduction repo?

parallel: true

Can you remove this line and see if that changes anything?
Could you also try to provide a minimal reproduction repo?

It's not working even without parallel. For the repro, I will try tomorrow evening as I am away from computer for a couple days. Sorry!

I can confirm the bug with Turbo 2.0.3 on macOS. It did work prior 2.x
my lefthook.yml:
pre-commit:
  parallel: false
  commands:
    check:
      run: pnpm run check

pre-push:
  parallel: true
  commands:
    check:
      run: pnpm run check
    test:
      run: pnpm run test

name = "turbo"
operating_system = "Mac OS 14.5.0 [64-bit]"
crate_version = "2.0.3"
explanation = """
file 'crates/turborepo-lib/src/commands/run.rs' at line 57
"""
cause = "render thread panicked: JoinError::Panic(Id(24), ...)"
method = "Panic"
backtrace = """

   0: 0x104d2eaa4 - __mh_execute_header
   1: 0x1052ecdec - __mh_execute_header
   2: 0x10535d7f4 - __mh_execute_header
   3: 0x105358e4c - __mh_execute_header
   4: 0x10573bd88 - __mh_execute_header
   5: 0x10577edc4 - __mh_execute_header
   6: 0x10577b4dc - __mh_execute_header
   7: 0x1051ace68 - __mh_execute_header
   8: 0x1051ab958 - __mh_execute_header
   9: 0x1051ac554 - __mh_execute_header"""

me too, in this issue!
I can confirm the bug with Turbo 2.0.3 on macOS.
turbo: 2.0.3
lefthook: 1.6.15
lefthook.yml:
# EXAMPLE USAGE:
#
#   Refer for explanation to following link:
#   https://github.com/evilmartians/lefthook/blob/master/docs/configuration.md
#
pre-push:
  commands:
    # type-check:
    #   tags: type check
    #   run: yarn type:check

pre-commit:
  parallel: true
  commands:
    # eslint:
    #   tags: lint code
    #   run: yarn lint
    format:
      tags: format code
      run: yarn format

and then this error log generated
name = "turbo"
operating_system = "Mac OS 14.5.0 [64-bit]"
crate_version = "2.0.3"
explanation = """
file 'crates/turborepo-lib/src/commands/run.rs' at line 57
"""
cause = "render thread panicked: JoinError::Panic(Id(36), ...)"
method = "Panic"
backtrace = """

   0: 0x10020b535 - __mh_execute_header
   1: 0x1008f38ef - __mh_execute_header
   2: 0x10097f8e3 - __mh_execute_header
   3: 0x10097c0fe - __mh_execute_header
   4: 0x100c7f0e1 - __mh_execute_header
   5: 0x100ddb195 - __mh_execute_header
   6: 0x100dd6573 - __mh_execute_header
   7: 0x100775887 - __mh_execute_header
   8: 0x1007743a6 - __mh_execute_header
   9: 0x100774eca - __mh_execute_header"""

Could everyone who encountered this issue try to use 2.0.4-canary.3? #8457 should address the issue

Could everyone who encountered this issue try to use 2.0.4-canary.3? #8457 should address the issue

That worked but I don't think it used the new UI when running via the gut hook.. Maybe I am wrong though.

Yeah, that's intentional. Pre-commit tools like left hook open up a fake terminal that doesn't play well with our new UI. Unfortunately that does mean we need to fall back to the previous stream output

Looks like this is resolved, closing.


management_charge total stored on submissions (Zendesk #10151)

Changes in this PR:
https://dxw.zendesk.com/agent/tickets/10151
The client complained that viewing submissions for 'CORPORATE TRAVEL MANAGEMENT (NORTH) LIMITED' on either the supplier app or the admin app was resulting in a 504 and the entire RMI site crashing. Investigation revealed that for every submission, the management_charge total is calculated on the fly from a total of all the management_charges on the submission's entries.
It was suggested that we add the management_charge for a submission to the database. However doing so would require backfilling the management charges for 74157+ submissions and would probably bring down the production server.
I propose we change the management_charge() method on the submission to either attempt to get the value from the database, or if that is nil, backfill it on the fly once only. Because many submissions are never viewed after they're completed, this should be low overhead for the production servers (the management charge is only calculated once when requested for the first time).
For the troublesome supplier who precipitated this issue - 'CORPORATE TRAVEL MANAGEMENT (NORTH) LIMITED' - I have written a script to be executed once this PR is merged. It may be an expensive process but it will only need to be run once, for one supplier. We can test it first on staging to see if it has acceptable speed/consumption.

How about setting management_charge_total in SubmissionManagementChargeCalculationJob#calculate ?

@leeky @Floppy  Ready for another look please 🙏


Fix Wishlist Functionality

Issue: Fix Wishlist Functionality

Description:
Currently, the wishlist functionality on our e-commerce website is not functioning. To address this, we can utilize Firestore to store wishlist data in the backend and render it on the wishlist page. The layout used for the cart page can be repurposed to display wishlist items.
Proposed Solution:

Implement Firestore integration to store wishlist data.
Develop functionality to add and remove items from the wishlist.
Modify the wishlist page layout using the cart page layout as a template.
Ensure seamless rendering of wishlist items on the wishlist page.
Test thoroughly to ensure proper functionality.

Assignee:
Firebase access will be provided for implementation.
Documentation of Firestore : Firestore
Labels:

Enhancement
Firestore
Wishlist

I want to work on this. I request that you assign it to me. I think I have enough firestore experience to implement this. Let me do it.

give me your email , i will give you the access

Email<EMAIL_ADDRESS>

@dev-rsiva check your mail

I have received mail. Let me have a look at it by morning.
Regards,
Siva
On Fri, 10 May, 2024, 10:57 pm Rishi Mondal, @.***>
wrote:

@dev-rsiva https://github.com/dev-rsiva check your mail
—
Reply to this email directly, view it on GitHub
https://github.com/MAVRICK-1/e-commerce_website/issues/6#issuecomment-2104996833,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/A7N36NWVOYQT33UYDVT3OM3ZBT7RDAVCNFSM6AAAAABHPGKJJWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBUHE4TMOBTGM
.
You are receiving this because you were mentioned.Message ID:
@.***>

@dev-rsiva check your mail

I have received mail. Let me have a look at it by morning.

@dev-rsiva any updates ? or should i pass it to @Hemu21

Sorry for the delayed response. I am understanding the code now. I have few query.
Why did you not having user specific data in firebase backend. I am not seeing it. I am just seeing the carts collection, seller collection only.
Whether you have any specific reason for it.
What my concern is we need to finalise how the data structure to be followed. Whether user- specific data or
Common data where each data link with the users who associated.
Have you worked on it.

@dev-rsiva any updates ? or should i pass it to @Hemu21

Sorry for the delayed response. I am understanding the code now. I have few query.
Why did you not having user specific data in firebase backend. I am not seeing it. I am just seeing the carts collection, seller collection only.
Whether you have any specific reason for it.
What my concern is we need to finalise how the data structure to be followed. Whether user- specific data or
Common data where each data link with the users who associated.
Have you worked on it.

@dev-rsiva #60 will solve this one , and we get the user data from fireauth

@dev-rsiva we got uid as primary key , so it is associate to it's respective users

@dev-rsiva we got uid as primary key , so it is associate to it's respective users

Ok let me work and complete.

I have worked on this task and submitted. I request you to review the pull request.

closed by #94


Added initial definitions for Typescript Support

The following PR provides basic code hinting for Typescript users. It can be further improved, but serves well for a start. See issue #10.

Awesome! Thanks!