[Harness-Automation] Fix issues until harness r40

Main changes:

[x] Fix using golden device as DUT when one golden device fails.
[x] Workaround an sniffer detection issue on DELL laptop.
[x] Add RF shield simulation option.
[x] Add Router_5_2_7 test case back.
[x] Support using Harness's default values.
[x] Refine code by not using map()/filter() function.
[x] Update document.

Codecov Report

Merging #1348 into master will decrease coverage by -0.92%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1348      +/-   ##
==========================================
- Coverage   70.13%   69.21%   -0.92%
==========================================
  Files         168      171       +3
  Lines       25163    26827    +1664
  Branches     3163     3369     +206
==========================================
+ Hits        17647    18569     +922
- Misses       6182     6843     +661
- Partials     1334     1415      +81

Impacted Files
Coverage Δ

third_party/mbedtls/repo/library/ctr_drbg.c
72.05% <ø> (-0.74%)
:x:

src/core/thread/mle.cpp
80.4% <ø> (-0.08%)
:x:

src/ncp/ncp_base.cpp
39.14% <ø> (ø)

src/ncp/ncp_uart.cpp
85.13% <ø> (ø)

src/ncp/hdlc.cpp
76.34% <ø> (ø)

third_party/mbedtls/repo/library/bignum.c
51.09% <ø> (+0.13%)
:white_check_mark:

src/core/thread/mesh_forwarder.cpp
76.22% <ø> (+0.23%)
:white_check_mark:

src/core/common/message.cpp
92.54% <ø> (+0.41%)
:white_check_mark:

examples/platforms/posix/radio.c
78.9% <ø> (+0.42%)
:white_check_mark:

src/core/thread/mle_router.cpp
80.11% <ø> (+0.48%)
:white_check_mark:

... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e81219...753785b. Read the comment docs.


src/Illuminate/Encryption/Encrypter.php: openssl_cipher_iv_length not found

Laravel Version: 6.18.2
PHP Version: 7.4.4
Database Driver & Version: MariaDB 10.4.x?

Description:
As soon as I moved to php7.4 the Encrypter class is producing error:
Call to undefined function Illuminate\Encryption\openssl_cipher_iv_length() on line 84
Steps To Reproduce:
I don't know which function excatly calls this (couldn't catch it in backtrace) but here's code of working with cookies:
app/Providers/RouteServiceProvider.php:
$lang = \cookie('user_lang', session()->get('user_lang') ?? Cookie::get('user_lang') ?? config('app.locale'), 60 * 24 * 30);
Cookie::queue($lang);
$this->app->setLocale($lang->getValue());

Hi there,
Thanks for reporting but it looks like this is a question which can be asked on a support channel. Please only use this issue tracker for reporting bugs with the library itself. If you have a question on how to use functionality provided by this repo you can try one of the following channels:

Laracasts Forums
Laravel.io Forums
StackOverflow
Discord
Larachat
IRC

However, this issue will not be locked and everyone is still free to discuss solutions to your problem!
Thanks.


Why does pusher-websocket-react-native sometimes work and sometimes just doesn't?

onConnectionStateChange. previousState=DISCONNECTED newState=CONNECTING
onConnectionStateChange. previousState=DISCONNECTED newState=CONNECTING
onConnectionStateChange. previousState=DISCONNECTING newState=DISCONNECTED
onConnectionStateChange. previousState=DISCONNECTING newState=DISCONNECTED
onConnectionStateChange. previousState=CONNECTING newState=CONNECTED
onConnectionStateChange. previousState=CONNECTING newState=CONNECTED
This is the connection data on my console
And this is the data that tells me the subscription was successful
onConnectionStateChange. previousState=DISCONNECTED newState=CONNECTING
onConnectionStateChange. previousState=CONNECTING newState=CONNECTED
This is a success I guess
onSubscriptionSucceeded: userChannel-3
PusherChannel {
  "channelName": "userChannel-3",
  "me": undefined,
  "members": Map {},
  "onEvent": [Function onEvent],
  "onMemberAdded": undefined,
  "onMemberRemoved": undefined,
  "onSubscriptionCount": undefined,
 "onSubscriptionSucceeded": undefined,
}

When I try sending data be it through postMan or the App, it refuses to do a single thing, doesn't give me any errors either, it only works sometimes.
This might seem vague, but I haven't genuinely gotten any signal of why this is happenning, I'm logging every possible action that could be responsible, it just seems like the onEvent method is never triggered at all, yet sometimes it works perfectly.
  const pusherCtx = useContext(PusherContext);

  useEffect(()=>{
    async function getPusher(){

   pusherCtx.startPusher(`userChannel-${userID}`, 'ThisEvent');

    }
   getPusher();
  }, [data])

The context
import {useState, useContext, useEffect, createContext} from 'react';
import { Pusher, PusherEvent } from '@pusher/pusher-websocket-react-native';
import { APP_CLUSTER, APP_ENDPOINT, APP_KEY } from '../util/pusher_credentials';
import { QueryClient, useQueryClient } from '@tanstack/react-query';

export const PusherContext = createContext({
    pusher: '',
    apiKey: APP_KEY,
    cluster: APP_CLUSTER,
    authEndpoint: APP_ENDPOINT,
    port: 6001,
    refetchThis: '',
    thisData:{},
    onConnectionStateChange: (currentState, previousState)=>{},
    onError: (message, code, error)=>{},
    onEvent: (event)=>{},
    onSubscriptionSucceeded: (channelName)=>{},
    onSubscriptionError: (channelName, message, e)=>{},
    onDecryptionFailure: (eventName, reason)=>{},
    subscribeToChannel: (channelName)=>{}
});

function PusherProvider({children}){
    const queryClient= useQueryClient();
    const [pusher, setPusher] = useState(Pusher.getInstance());
    const [thisData, setData] = useState({});
    const [apiKey, setApiKey] = useState(APP_KEY);
    const [cluster, setCluster]= useState(APP_CLUSTER);
    const [authEndpoint, setAuthEndpoint]= useState(APP_ENDPOINT)
    const [port, setPort]= useState(6001)
    const [refetchThis, setRefetchThis] = useState('');

    function onEvent(event: PusherEvent){
        console.log("Technically, I should be called, but that's not working")
        console.warn(`onEvent> ${event}`)

      setData({...event})
      console.log(thisData)

      }

    function onSubscriptionSucceeded(channelName){
        console.log("This is a success I guess")

        console.log(
          `onSubscriptionSucceeded: ${channelName}`);

        const channel= pusher.getChannel(channelName);

        if (!channel) {
         console.log("Error");
         return
        }
        const me = channel;
      //  onChangeMembers([...channel.members.values()]);
        console.log(me);
      }

     function onError(message, code, error){
        console.log(`onError: ${message} code: ${code} exception: ${error}`);
      };

      function onSubscriptionError(channelName, message, e){
        console.log("Yikes this ian Error")
        console.log(`onSubscriptionError: ${message}, channelName: ${channelName} e: ${e}`);
      };

      function onDecryptionFailure(eventName, reason){
        console.log(`onDecryptionFailure: ${eventName} reason: ${reason}`);
      };

      function onConnectionStateChange (currentState, previousState) {
        console.log(
          `onConnectionStateChange. previousState=${previousState} newState=${currentState}`
        );
      };

      async function startPusher(channelName, refetch){
        setRefetchThis(refetch);
        console.log(refetchThis)
        console.warn(refetchThis)
        await pusher.init({
            apiKey:APP_KEY,
            cluster: APP_CLUSTER,
            authEndpoint: "http://<IP_ADDRESS>:8000/pusher/broadcasting/auth",
            port: 6001,
            onConnectionStateChange,
            onError,
            onEvent,
            onSubscriptionSucceeded,
            onSubscriptionError,
            onDecryptionFailure,

        }).catch(error=>console.error(error));
        await pusher.subscribe({channelName:channelName, onEvent });
        await pusher.connect();
      }
      async function subscribeToChannel(channelName, refetchThis){
        await pusher.subscribe({channelName:channelName, onEvent });
        setRefetchThis(refetchThis);
         }
      const value ={
        pusher:pusher,
        apiKey:apiKey,
        cluster: cluster,
        authEndpoint: authEndpoint,
        port:port,
        refetchThis: refetchThis,
        thisData: thisData,
        onConnectionStateChange: onConnectionStateChange,
        onError: onError,
        onEvent: onEvent,
        onSubscriptionSucceeded: onSubscriptionSucceeded,
        onSubscriptionError: onSubscriptionError,
        onDecryptionFailure:  onDecryptionFailure,
        subscribeToChannel: subscribeToChannel,
        startPusher: startPusher
      }
      return <PusherContext.Provider value={value}>{children}</PusherContext.Provider>
}

export default PusherProvider

@xutopia there's a known issue in our underlying Swift websocket library waiting rarely waiting indefinitely for the device to reconnect (but the iOS device has disposed of the connection. So it will wait forever, hence hanging on reconnecting). There's an experimental fork that should mitigate this.
@AntonioA-D-C Your case is more complex, and I suspect its because you are using a Context. I've used contexts in a provider before in React, but not in React Native. So although your app is establishing the connection and subscribing to the channel, the RN context probably isn't behaving as expecting when your client receives an event. Could you restructure your Pusher implementation away from using Contexts, and just initialize it in a useEffect(() => {...}, []); like our example?

@benjamin-tang-pusher are there any plans to merge the changes from the experimental fork? we would like to migrate to the new library but with pusher being a critical step in our business operations, we cannot afford to risk the flaky connections.

@AntonioA-D-C  Using a context with pusher will give you a very odd behaviour,  you can refactor your code into a hook then  consume where ever needed if possible.

So I can't use Pusher Data globally?

@xutopia there's a known issue in our underlying Swift websocket library waiting rarely waiting indefinitely for the device to reconnect (but the iOS device has disposed of the connection. So it will wait forever, hence hanging on reconnecting). There's an experimental fork that should mitigate this.
@AntonioA-D-C Your case is more complex, and I suspect its because you are using a Context. I've used contexts in a provider before in React, but not in React Native. So although your app is establishing the connection and subscribing to the channel, the RN context probably isn't behaving as expecting when your client receives an event. Could you restructure your Pusher implementation away from using Contexts, and just initialize it in a useEffect(() => {...}, []); like our example?

Alright, I stopped using context, and decided to initialize it in a useEffect like the example you mentioned
Now, Initially it works with zero issues, but when I travel to another screen, and come back it doesn't work anymore, why?

I also encountered a similar issue and after debugging I found out that

After hot reload pusher.channels is still populated even though those channels are not connected
Not sure if this is the preferred approach but this is how I fixed it
const pusher = Pusher.getInstance()
const init = async () => {
  // Rest of the code
  if (pusher.channels.size > 0) {
    const promises = [];

    pusher.channels.forEach(({channelName}) =>
      promises.push(pusher.unsubscribe({channelName})),
    );

    await Promise.allSettled(promises);
  }
  pusher.init({ ... })
  // Rest of the code
}

The reason behind a large number of logs for CONNECTED, DISCONNECTED and all the other states is that ON_CONNECTION_STATE_CHANGE listener is not being removed in removeAllListeners
https://github.com/pusher/pusher-websocket-react-native/blob/067f8ff1133c780aedb257659f62d569eaf0c2b7/src/index.tsx#L307-L315
Just add the following code in removeAllListeners and the events will be logged only for one connection
    this.pusherEventEmitter.removeAllListeners(
      PusherEventName.ON_CONNECTION_STATE_CHANGE
    );

If you want to use a patch file follow the steps below
Steps to use the patch file

Create a folder named patches in your root directory
Create a file named @pusher+pusher-websocket-react-native+1.3.0.patch

1.3.0 is the version (latest) I am using make sure to check you are using in your package.json

Past the following code in the file

diff --git<EMAIL_ADDRESS><EMAIL_ADDRESS>index 5c04965..6bff3b7 100644
---<EMAIL_ADDRESS>+++<EMAIL_ADDRESS>@@ -312,6 +312,9 @@ export class Pusher {
     this.pusherEventEmitter.removeAllListeners(
       PusherEventName.ON_MEMBER_REMOVED
     );
+    this.pusherEventEmitter.removeAllListeners(
+      PusherEventName.ON_CONNECTION_STATE_CHANGE
+    );
   }

   public async reset() {

Run  the following command

npx patch-package

You can read more about patch-package and how to configure it in your project here.

We recently updated the library to use a newer Swift dependency that resolves the issue mentioned in https://github.com/pusher/pusher-websocket-react-native/issues/111#issuecomment-1773252321
@AntonioA-D-C I can see you raised https://github.com/pusher/pusher-websocket-react-native/issues/123 as a follow up so I will close this thread

I solved the issue
The problem was I was calling Pusher with each re-render which caused pusher to act in erratic ways
This was fixed
Sorry for the inconvenience


(amplify-graphql-types-generator) : Failing during snapshot testing

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
yarn
If applicable, what version of Node.js are you using?
14.0.0
Amplify CLI Version
5.1.0
What operating system are you using?
Windows
Amplify Categories
Not applicable
Amplify Commands
Not applicable
Describe the bug

Failed snapshot matches
'start' and 'end' represent positions in the file, thus seems to be a mismatch of line positioning between the document being processed and snapshot file.
Expected behavior
To have snapshot testing pass, without any mismatch in line position between the two files.
Reproduction steps

cd packages/amplify-graphql-types-generator
yarn test

GraphQL schema(s)

# Put schemas below this line

Log output

# Put your logs below this line

Additional information
No response

we do run all unit tests on windows as part of our release flow.


(Re)implement reporting functionality

Is the plan to be able to template in a scheduled report of some sorts?

Reporting code is unused & removed as of a84053a00f2cf50f3e5b842c516b7f1967a71843.


Remove custom fonts

This is an OS-integration idea.
I think it would be better not to use Source Sans Pro for UI fonts, and only use OS-specific fonts, such as WordPress backend or GitHub:
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol";
The font is better rendered and it integrates better with the various OS's out there.

I like the idea in general, might be worth testing it out.

I created my own theme, which replaces the vanilla fonts with the ones above. The theme is called The Greyhound and I have also released a plugin, called The Greyhound Companion, which changes more areas of the IDE, inaccessible to a regular theme.

Good work @wolffe. Honestly I quite liked the idea of using a different font. But along with this we should try to resolve one more problem where in high dpi monitors rendering of file tree and status bar is quite useless. We need to use a user customizable font size for control panes as well. Could you think about some solution in that line?

Maybe. I can only test it with 2 monitors, one 1440 and one 1920, but I'll have to check if I can add:

Font size controls in the Theme panel.
REM units for fonts, so they scale dynamically with the resolution.

I'm actively working on my theme and my plugin, as I'm coding on 3 different computers and I need them to look and behave the same.

@wolffe I have seen this problem in my 4k 28inch monitor. Editor font size can be increased in Brackets but not the rest of the things. I am working on matching the code hint font size with that of code view, but the rest of the UI should also evolve.

So, first of all, the change I am proposing is to change the current IDE font (Source Sans Pro) with the font stack in my original post.
Second, a bit offtopic, and this should have its own issue, the font size configuration. I agree with your points above. That's why I created a plugin to change the fonts that cannot be changed via a regular theme.
Please check The Greyhound theme and the The Greyhound Companion plugin.

It seems there is also the system-ui font.
But maybe we should update the fonts, see #12663


enhancement: Lag when using over high-latency connections

Is your feature request related to a problem? Please describe.
It's more of a UX request.
Describe the solution you'd like
When using open-webui over high-latency connections (eg. 300-350ms), certain page actions are incredibly delayed giving the user the impression that he hasn't clicked a button or submitted certain actions.
This is most noticeable when sending a text to a model, where the text remains in the textbox even after it's being sent, and will only clear once the model answers. This becomes even worse when a image is chosen.
It's also noticeable when deleting a chat. Clicking the "yes" check to delete a chat gives no immediate answer until the server responds, which makes me sometimes wonder if I really clicked it. A simple spinner would solve this.
Furthermore, it's also noticeable when the page initially loads. The user is presented with a completely blank page and the "loading" spinner from the browser itself ceases, giving the impression that the page really has loaded, and is blank. However, several seconds later, the content actually loads. Again, a simple spinner hardcoded in HTML/CSS would solve this.
Describe alternatives you've considered
-
Additional context
My only server that runs 24/7 is (unfortunately) pretty far away, and thus I notice this all the time.

Related: #1550 Could you try our dev branch and check if anything's changed for you?

I currently built from dev and the same issues persist. Nothing's changed.
I have since relocated the UI to another server closer to me. However, I saw that the exact same behaviour can be archieved using DevTools/Network and creating a "High Latency" profile.


turn on autoescape for flask.templating.render_template_string

Previous pull request was #1176

Could you shorten the commit msg a bit? It's so long that github truncates
it. Usually the first line should be 51 chars or less (with the 2nd line
being empty and anything else in the following lines (max 72 chars per line)

@ThiefMaster sure, done. What's the eta on this getting merged, I have other pull requests I would like to submit as well.

I'm not sure how, but I think the current behavior should be better documented.

In addition to some docs, could you add a changelog and a note in the Upgrading document?

I've updated the docstring which will show up in this doc, but I should regenerate docs or add more info somewhere?
For changelog, should I add to version 0.10.2 or 1.0?

1.0

Finished all the changes, everything looks good?

I would squash all the commits (besides the whitespace fixes) into a single one

I've pinged mitsuhiko on IRC if there's a good reason for the current behavior. I assume not and will merge this if he doesn't respond.

@ThiefMaster squashed.

thanks! (and oops, looks like i forgot the trailing backtick in the commit message suggestion)

Thanks, rebased and merged!


Add Sketch 47+ Libraries Support

Overview
This PR adds the ability to use the new Libraries feature in Sketch v47 (Link)
Changes

Added new changeIdentifier property to the Symbol model.

This new property gets auto generated by a new generateNumberId function which ensures a unique Number (Int) by reading the current date and keeping track of previously generated numbers.

Dependencies

Waiting on PR: https://github.com/airbnb/react-sketchapp/pull/183

Related Issues
https://github.com/airbnb/react-sketchapp/issues/172

@jongold,
With the release of v0.1.2 of sketchapp-json-plugin, this PR no longer has any extra dependencies and should be ready to go.
Let me know if there is anything else that needs changing. Thanks!


chore(docs): Add a goproxy settings description in docs/kitex

What type of PR is this?

Check the PR title.

[ ] This PR title match the format: <type>(optional scope): <description>
[ ] The description of this PR title is user-oriented and clear enough for others to understand.

(Optional) Translate the PR title into Chinese.
在docs/kitex中，添加设置代理的说明。
(Optional) More detailed description for this PR(en: English/zh: Chinese).

en:
zh(optional):
(Optional) Which issue(s) this PR fixes:

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Move HTTP API classes into a common package tree

Resolves current situation where HTTP APIs are distributed throughout the source tree.

*Resource.java => api package
*Info.java => api.types package

The separate API package code should only change when a user/CLI-facing interface change is expected. Having a separate dedicated package for these endpoints provides some assurance that changes to that API only occur when expected.
Having a common package for all the API endpoints also makes them a bit more discoverable by developers/operators that may be browsing the source tree.

@mrbrowning:  We tried having the APIs defined on a per sub-component basis, but it was scattering the API definition so it was hard to figure out if everything that should be exposed etc. was being covered.  I think this is @nickbp's discoverability point.


Perfect translation

Improve the untranslated, errors, etc. based on the English version of the document.

Thank you for your translation support.
This PR has a lot of content, so it may take some time to check. Please wait patiently.

@CainLuo
Thanks again for helping to translate and sync these documents, it will help a lot of Chinese developers.
Also I've added some comments to the PR, if you have free time please check and consider revising. 🍻

@0xTim
Hi Tim
I found that the previously merged PR did not trigger the execution of the github action script, so the translated content was not updated to the website, such as this PR: https://github.com/vapor/docs-cn/pull/15/files.
Do you know the reason?

@Jinxiansen looks like the GH action was triggered. Does it need updating? What's missing?

@CainLuo
Thanks again for helping to translate and sync these documents, it will help a lot of Chinese developers.
Also I've added some comments to the PR, if you have free time please check and consider revising. 🍻

@Jinxiansen OK, where do I see your comment? I also found some small details on the problem, is currently being revised one by one, and will submit a PR again after all the changes, please check again then, thank you ~

@CainLuo

Thanks again for helping to translate and sync these documents, it will help a lot of Chinese developers.

Also I've added some comments to the PR, if you have free time please check and consider revising. 🍻

@Jinxiansen OK, where do I see your comment? I also found some small details on the problem, is currently being revised one by one, and will submit a PR again after all the changes, please check again then, thank you ~

Sorry, I came back late from overtime. You can see this PR in your browser and scroll down.

@Jinxiansen looks like the GH action was triggered - https://github.com/vapor/docs-cn/runs/4508150654?check_suite_focus=true. Does it need updating? What's missing?

Sorry, Tim, I'm not familiar with this part of the script. Tanner originally added this part of the action script task.
I am a little late after working overtime today, I will check it in my spare time.

@CainLuo

Thanks again for helping to translate and sync these documents, it will help a lot of Chinese developers.

Also I've added some comments to the PR, if you have free time please check and consider revising. 🍻

@Jinxiansen OK, where do I see your comment? I also found some small details on the problem, is currently being revised one by one, and will submit a PR again after all the changes, please check again then, thank you ~

Sorry, I came back late from overtime. You can see this PR in your browser and scroll down.

@Jinxiansen Sorry, I didn't see a reply on PR, only your conversation with 0xTim. Am I not operating correctly somewhere?

@CainLuo
Please recheck these comments, you should be able to see it by now.

@CainLuo
Please recheck these comments, you should be able to see it by now.

好的，等我检查完之后再次提交合并~

@CainLuo
Hi, CainLuo.
Do you have time to process these comments? It's not urgent.
I would love to be able to merge your PR after you finish revising it.

@0xTim
This is not included in the main repo yet, and looks like CainLuo is a little busy and may not have time to deal with this PR.
I will try to copy the correct content in the PR and submit the PR to main Repo by the end of the week.


Reference in new issue modal: dont pre-populate issue title

Examples like https://github.com/go-git/go-git/issues/384 and https://github.com/go-gitea/gitea/issues/17182 highlight, that good defaults matter, to frame it positively.
btw, interesting that this malformed selector worked in the first place!

Codecov Report

Merging #17208 (b2799a9) into main (47193db) will decrease coverage by 0.00%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main   #17208      +/-   ##
==========================================
- Coverage   45.29%   45.29%   -0.01%
==========================================
  Files         779      779
  Lines       87263    87263
==========================================
- Hits        39530    39522       -8
- Misses      41320    41324       +4
- Partials     6413     6417       +4

Impacted Files
Coverage Δ

modules/notification/mail/mail.go
35.29% <0.00%> (-2.95%)
:arrow_down:

modules/queue/queue_disk_channel.go
60.94% <0.00%> (-1.78%)
:arrow_down:

modules/queue/queue_channel.go
95.00% <0.00%> (-1.67%)
:arrow_down:

modules/notification/ui/ui.go
60.86% <0.00%> (-1.45%)
:arrow_down:

models/notification.go
65.28% <0.00%> (-0.88%)
:arrow_down:

models/repo_list.go
77.64% <0.00%> (-0.79%)
:arrow_down:

services/pull/pull.go
41.78% <0.00%> (ø)

models/issue_comment.go
51.90% <0.00%> (+0.29%)
:arrow_up:

modules/queue/workerpool.go
49.61% <0.00%> (+0.76%)
:arrow_up:

modules/git/utils.go
70.83% <0.00%> (+2.77%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47193db...b2799a9. Read the comment docs.


Relations refacoring ideas

Continuing from #162:

Wow thank you so much for your advice! This is super helpful! I do think having indexes for the FK records are interesting and will boost the performance when retrieving data from the state, though I'm not so sure how that would impact the overall user experience.

How do you mean?

I'm not sure how this would work. Because the database Vuex ORM is using is Vuex Store State, not RDB. Will sql.js and knex run on browser against JSON object as a database...?

I think you misunderstood what I meant. I'm saying that vuex-orm probably shouldn't go overboard in the types of queries that it does - because relational databases could do them much better anyway. So if a user needed that level of complexity, they'd be better off going with sql.js or something else that's a "true" relational database. Vuex is for reactive state information, not emulating a full blown database, and I think it's helpful to bear that in mind so we don't feature-creep too much.

This will indeed could be nice when loading data, but at least on my usage, I tend to refresh the Store State a lot. Maybe I do so more than I retrieve data from the state. In this case, maybe it would cost more to create indexes than lookup data each time? Also we have to make sure we delete those index when deleting record or update them when updating the record.

This is a valid point. A possibility is to support both and let the user opt-in to indexing. Vuex-orm could then use the index if it exists or fall back to a linear record scan if it doesn't. That said, my suspicion is that creating and maintaining the indexes should be fairly cheap in terms of cost. Hash maps are constant-time-access, so if FK targets are guaranteed to be unique (a safe assumption), then hash maps can be used and the index maintenance cost becomes essentially inconsequential.

The second reason is, I think the data stored at Vuex ORM is quite small at most of the time (it was completely wrong with your case though :p). Because even if you have like thousands of record stored at your state, you could only show few in a single page.

This doesn't impact the need for indexing. In fact, in that huge-app-dataset (:P) I am showing only a few items at a time on any one page - but the cost-savings of looking up those few items is massive.

This is interesting. I'm actually thinking to store such "prototyped" data into the Vuex State directly too. In fact, at very beginning of Vuex ORM, I was saving actual class instance to the Vuex State.

The reason I stoped doing that is was because Vue official document tells to store only "plain object" to the store. I'm still not sure what exactly "plain" means, but I did asked at Vue's chat and the core member told me that there might be chance it wouldn't work in future when they implement Proxy based reactivity (Vue 3).

But as you know, it works and I think it would work in future too. If we could do that, then yes we can eliminate the need of instantiating object each time we retrieve the data!

I won't claim to know the ins and outs of Vue's future plans, but I bet there's a workaround for that too. Vue class-based components use properties for computed methods, so that'll have to be addressed anyway.

How do you mean?

I meant mainly about the performance impact on when creating the records (indexes).

I think you misunderstood what I meant. I'm saying that...

Oh yeah I see. That true.

This is a valid point. A possibility is to support both and let the user opt-in to indexing.

Yeah that could be a nice option. If the cost is too hight, maybe we can extract it as a plugin later.

I won't claim to know the ins and outs of Vue's future plans, but I bet there's a workaround for that too. Vue class-based components use properties for computed methods, so that'll have to be addressed anyway.

Hah, that's valid point I guess. I totally forgot about Class component lol.

OK so I like to try out this idea. It might take a while since it's a quite large architecture change since we must be bale to manage the indexes on every CRUD action. Maybe I can start off by storing class instance directly to the store to eliminate the needs for instantiating model each time user uses the getters.

I still think the cost should be quite small for a hashtable-style index.
 I think storing instances directly and then making a new relation system is the way to go :)

OK, I'm having trouble that maybe storing class instance to the state wouldn't make many benefits.
The reason is even if we store class instance to state, we have to clone that instance in the getter to support eager loading the relationships since we have to "mutate" the record and we can't do that inside getters.
If we're going to clone the instance (and it's not that easy in JavaScript), maybe it would make more sense to create instance each time a user fetches the data out of the state.

The way I solved that was by doing away with eager/lazy loading completely. Instead, each property was just a lookup each time - but thanks to the hashmap indexes, the lookups were almost instantaneous. As a reminder:
                        Object.defineProperty(mat, 'item', {
                            enumerable: true,
                            get() {
                                return rootGetters['data/db/items'][mat.itemId];
                            }
                        });

I'd argue that eager/lazy loading doesn't make much sense in that context. Actually, I'm not sure it makes much sense at all to eager-load anything in vuex-orm, at least in the traditional ORM sense. Eager loading is made to avoid multiple trips to the database, instead fetching all related objects into program memory up front. But in vuex-orm, the entire database is in memory already, so there's nothing to eagerly load.
This may be a case of a feature that becomes obsolete in the new system :)

Thank you for your feedback!
I understand having lookup approach via getter, but how can we filter the relationship individually upfront?
Let's say we have User that has many Post, and in Vue Component we want to have 2 users collection with different posts filtering.
export default {
  computed: {
    userWithAllPosts () {
      return this.$store.getters['entities/users/find'](1)
    },

    userWithLimitedPosts () {
      return this.$store.getters['entities/users/query']()
        .where('id', 1)
        .with('posts', (query) => {
          query.where('published', false)
        })
        .get()
    }
  }
}

With above example, both computed property references same user record where id of 1, but it has to have a different number of posts. How can we achieve this without cloning the class instance...?
Let's say we stop using eager loading, how would then the API look like on above example? Do we have to use general JS map and filters?
user.posts.where('published', true) // We can't do this

Maybe we should create the Collection class and wrap relationships with it instead of making them simple Array so that the user can use such handy features (like where and orderBy). I don't know :p

Eager loading is made to avoid multiple trips to the database, instead of fetching all related objects into program memory up front.

That's true but one another thing to consider is that it fetches the all records upfront and do the iteration only once for the entire collection. It depends on how you create your loop, but I think you can gain performance by eager loading the records when you need to fetch lots of them.
For example, let's say we have 100 posts, and we need to filter the posts. If we do eager loading we only have to filter posts once no matter how many users record we have, but otherwise, we must iterate post records for each user record.
<template>

{{ user.name }}

        {{ post.title }}

</template>

With above code, if you have 100 users, then you have to iterate posts 100 times. But if you eager load, you only have to iterate post once. Even if we have a hash index, because we are filtering the posts by its field (maybe get only posts where published is true), we must iterate them over and see if the specified value matches.
Well, of course, the performance will depend on how many records are attached to the user (and by many other factors), so I'm not saying eager loading will always be faster, but I believe there're situations where it will be.
What do you think?

For example, let's say we to filter the posts. If we do eager loading we only have to filter entire posts once no matter how many users record we have, but otherwise, we must iterate post records for each user record.

This is true, but this is a good thing. In fact, I don't see how you could effectively do it otherwise. With an index-based relation system, fetching all the posts for a user is as fast as a lookup (to get the list of post IDs for that user) and then n lookups to get the post instances. Worst case, each post belongs to one user, so loading the user list means M + N performance (where M is user count and N is post count). This is the same as the full scan approach of iterating all users and then all posts and doing the sorting that way.
BUT, if we only load a subset of users, an index-based system would only need to look up a fraction of users and thus only the faction of posts that are associated with that user. The full-scan thing would still require completely iterating both.
So, I think the index/entity-based filtering is the way to go. This also removes complexity in the case of something like this:
   publishedPosts (user) { return user.posts.where(p => p.published) }
   unpublishedPosts (user) { return user.posts.where(p => !p.published) }

In that case, we'd somehow have to detect that they actually may iterate all posts, and load them all up front regardless of published status. Otherwise, we'll pay a hit later that nullifies the (slight) benefit of doing the filtering up front. Even if we managed that, we'd have to maintain two different arrays and decide which to pull from... and that's just for booleans!

Have you ever used C#? C# has a feature called LINQ which is almost exactly what you describe here. In C#, LINQ is a set of methods that act on anything enumerable. Arrays, maps, etc. It's a lot like lodash chaining, actually.
LINQ works with enumerators/generators, so foos.Where(f => f.num > 4).OrderBy('bar'), etc. The key thing is that all LINQ functions return enumerables themselves - so you can chain arbitrarily without evaluating at each step. Only when you iterate over the LINQ query object does it actually hit the collection.
I think we should put the onus on the end developer to cache the results of a query. LINQ does this, too. You can call ToArray on a LINQ query to force evaluation and storage of the results in an array - which of course you can iterate repeatedly then. IMO automatically returning an array by default would be worse for performance, because then you're guaranteed to iterate query results twice - once for the library to run the query and build the array, and again for the developer to use the results. Most queries are probably once-and-done deals, where the developer iterates once and then tosses the result. Putting things in an array would be double the work at that point.
We could, of course, re-implement LINQ in JS... but that seems like a ton of work and I'm honestly not sure if JS supports exactly the same language features, so there'd have to be some transliteration that went on to adapt it.
But... lodash already does this. So does underscore, if anyone still uses that library. And so will the next library that comes out. Lodash even has built in support for so-called "lazy evaluation".
My suggestion here is not to bundle lodash with vuex-orm. Instead, we take care of the "vuex" part and load the relations into an array. With proper indexing, this should be nice and fast. Then, we suggest pairing vuex orm with lodash (or any other similar library) which gives a number of benefits:

Smaller code footprint for vuex-orm, so fast load time and less to maintain ^.^
Not reinventing the wheel (I doubt we could get performance near lodash!)
Developer flexibility on what filtering to use (lodash, underscore, array prototype methods, custom functions, etc)

Tbh this idea didn't occur to me until just now, but I think it solves the problems nicely and maintains a nice separation of concerns :)

BUT, if we only load a subset of users, an index-based system would only need to look up a fraction of users and thus only the faction of posts that are associated with that user. The full-scan thing would still require completely iterating both.

Well, you got a point. Maybe there wouldn't be a much of a benefit on eager loading the relation if each item belongs to the only subset of related records. And I'm pretty sure that that's is the nearly 100% the case.

About the Lodash idea
I like the approach where we can do the on-demand loading of the records, but I was thinking to add that feature as an addition to the eager loading. Because with this approach, you have to create a dedicated method that does that operation, and I feel like it's a bit cumbersome. For example with the below code;
export default {
  computed: {
    users () {
      return this.$store.getters['entities/users/all']()
    }
  },

  methods: {
    publishedPost (user) {
      return user.posts.filter(post => post.published)
    }
  }
}

With eager loading, we don't need to create publishedPost method. We can just do;
export default {
  computed: {
    users () {
      return this.$store.getters['entities/users/query']().with('posts', (query) => {
        query.where('published', true)
      }).get()
    }
  }
}

Also, if we are to use lodash, or even if we make some Collection class our selves, we still can't chain the query because, in order to chain, we have to return same Collection class instanse, which means we have to modify the state that Collection holds. Which you can't because that Collection class instance might be referenced from another place, and also you can't change the state of the object which is stored inside Vuex.
In order to make it chainable, I think we need to make a copy of the Vuex State and set it to the lodash or any other Collection class. And if we were to make a copy of state anyway, then maybe it's not worth putting class instance into Vuex since we still have to create new instance when copying them.
Maybe I need some code to make my thoughts more clear on this, so I'll try to write very simple example that would simulate the flow of the operation and see if I can find some good way to handle this : )

My point here is, that I'm not against having a hash index and each record look up to those indexes. The point is I'm thinking that it's quite hard to use Vuex State directly by referencing them.
I think we need to first make the copy of the state anyway. I mean that's what other ORM's does. It'll first fetch records from DB via SQL, then do something with those records. Instead of fetching data from the DB, I think Vuex ORM has to copy the record from the Vuex State.

I'm short on time for the next day or two, but I wanted to briefly respond.
First, yeah, indexes and filtering/storing instances are two distinct but related problems. We've now expanded beyond my original intent haha, but I think it's a good thing to look at all of this!
Philosophically, I think that composing like you did it the 2nd way makes for better code since you can do other things with the users property and the display logic is clearly embedded in the view.
FWIW I also do something (like, I'm omitting some syntax) this:
export default {
  computed: {
    filteredPostsByUser () {
      return _.map(this.$store.getters['entities/users/all'](), u => { user: u posts: _.filter(u.posts, 'published'));
    }
  },
}

IMO since vuex orm is an extension to vuex, the same rule about modifying objects returned from vuex is still in play. This means that I (as a developer) would never try to mutate any object coming out of vuex orm - which would eliminate the need to clone things in the first place. If they mutate things, it's just like (well, it is..) they mutated the vuex store directly... so undefined behavior that maybe we don't need to worry about? If someone wants to modify state directly, they can worry about cloning, eg with something like _.clone.

Also tbh implementing filtering falls under what I think is something that traditional ORMs do but this library shouldn't. I think that kind of operation opens a whole can of worms and "gotchas" since we don't have an SQL server (and the associated optimizations and query planning) that goes with it. I think a few simple things like where filtering could be implemented easily, but I also think that any non-trivial use case would quickly expand beyond that and the user would end up reaching for lodash and custom functions anyway.
From that standpoint it makes much more sense to me to worry about providing seamless relations traversal geared toward allowing downstream APIs to easily consume related entities and leave the sorting and filtering to them in the first place.
I'm going to be quite busy for a few days so if I don't reply for a while, that's why :P

Well yeah you got a valid point. I'll try organize the overall flow on data creation to fetching them and see how the architecture should be.

So here is the more detailed planning on how to achieve this. Let's see how it goes.
Simple Relationship Creation
Let's say we have "User has many Post" situation, and server response is as below (we'll define by directional relationship so Post belongs to User as well).
// This is the user object.
{
  id: 1,
  posts: [
    { id: 1 },
    { id: 2 }
  ]
}

When passing above data into Vuex ORM, we'll first normalize the data. This is done mostly by Normalizr but Vuex ORM will attach any necessary foreign field as well so which means the user_id field on Post records will be generated. After normalization, we get the following data.
{
  users: {
    '1': { id: 1, posts: ['1', '2'] }
  },
  posts: {
    '1': { id: 1, user_id: 1, user: '1' },
    '2': { id: 2, user_id: 1, user: '1' }
  }
}

Next, we'll loop through those object and try to create the hash indexes. The simple index should look something like this.
// Indexes
{
  users: {
    '1': {
      posts: ['1', '2']
    }
  },
  posts: {
    '1': {
      user: '1'
    },
    '2': {
      user: '1'
    }
  }
}

OK, this is simple enough. Creating this indexes shouldn't be hard in this case. Now, what would happen if a user creates above data separately?
Let's say the user created just User and then later creates Post that belongs to that User.
// User first creates this record with out post.
{ id: 1 }

There's no Post so the index will not be created. That's OK. Next, the user will create Post.
// User creates post record.
{ id: 1, user_id: 1 }

Now, we need to check if there's a User with an id of 1, and if there is, we'll create the index for that. This requires a full scan of the User records.
Also, we need to create indexes for the User as well (reverse index). This could be a bit more complicated because we have to determine if the User relates to Post or not.
Q. Can't we just automatically assume that User also belongs to Post?
Maybe we can, but not sure.
Do we really have to create indexes for all relation in both directions? Maybe there's a good chance that users might not use that relation at all. So maybe it makes more sense to just create indexes that user has specified explicitly. We could implement auto relationship generation option or something in the future if we really need it.
Complicated Relationship Creation
OK how about many to many relationships where Post belongs to many Tag and Tag belongs to many Post. There's a pivot table of PostTag as well.
Now the index is not that different, but we need to consider the pivot table as well. So we might have to have indexes as an object like below?
{
  posts: {
    '1': {
      tags: {
        target: ['1', '2'],
        pivot: ['1', '2']
      }
    }
  },
  tags: {
    '1': {
      posts: {
        target: ['1'],
        pivot: ['1']
      }
    },
    '2': {
      posts: {
        target: ['1'],
        pivot: ['1']
      }
    }
  },
  postTag: {
    '1': {
      post: 1,
      tag: 1
    },
    '2': {
      post: 1,
      tag: 2
    }
  }
}

Hmm... Now it's a bit complicated here. But still simple enough I guess.
Updateing and Deleting
When updating the relationship ID or deleting the record, we need to go check if there's index or not and delete and also update them. The tough part is that we also have to check if there's any other relation that is pointing to the record to be deleted or not.
In this point of view, maybe it's better to create the indexes bidirectional every time the data is being created.
Retrieving Record
Retrieving should be easy now since we have indexes and all we need to do is reference them. I'm not sure if there's a situation where we can't find the index of the record? In such case we have to go through the ordinal process but... maybe it wouldn't happen. I don't know.
At Last
Now it seems pretty implementable. I think we can consider whether we should store class instance or not later. Making indexes should increase the performance at read operation.
Maybe we can start implementing this structure and see how it feels like.

I have created the PoC project to check to see how it would feel like to have indexing system. Common public API and the architecture is pretty much same, but I've rewritten every code since I had few ideas I wanted to try.
It is not fully featured and we should definitely consider more about the architecture, but at least it's working as expected. The core ideas I've tested is as follows.

Create relationship indexes when adding data to the store. I've created and tested the "has one" and the "belongs to" relationship. The bidirectional relationship is tested as well.
Instantiate model before persisting the data to the store, and just reference them when fetching the records (model instances).
Resolve relationship referencing the indexes created at (1) by attaching getter property to the model that has logic to look up the appropriate index (the lazy load feature).

I'll describe my thoughts about implementing these features, and idea how we should proceed from here.
About Indexes
First and most, indexing is the hardest and most complicated work. Although it works as expected, and sure it should be performant when retrieving data, it was much complicated than I thought.
Complexity of Creating and Updating Indexes
Creating indexes out of normalized data is easy since most of the work is done by Normalizr. However, the tough part is, when creating a record, we must go through and check every relationship and create indexes for those relationships as well. This is quite complicated.
For example, if User has one Phone, and Phone belongs to User, when inserting Phone records we must;

Check all existing User records and see if there's User[owner key] of Phone[foreign key], if there is, create an index for Phone to that User.
Check if User also has the relationship to the Phone, if it has, create an index for User to the Phone (reverse index).

There should be many ways to tackle this problem, but the thing is that we need to implement this feature in all of the relationship types including more complicated relations such as hasManyThourgh and morphTo.
Also, we must manage the index when updating and deleting any records.
Performance Concern
Since we are creating index creation, the speed on record creation will decrease somewhat. It shouldn't be noticeably slow considering the average amount of data that the frontend application handles, however, it might be a problem when you have lots of data with lots of relationships.
For example, if the User model is being referenced by 5 other models, when creating a single User record, we must go through all of the records of 5 other models to ensure that their index is updated. In the worst scenario, this requires full data scan of other 5 models.
For the "Write Intensive" application that creates more data than reading them, it might raise the same issue we're having currently with retrieving data in term of performance.
I'm not saying hence indexing is useless. I just think we need a way to turn indexing feature off or make it optional to be as flexible as possible.
Store Model Instance to The Store
Storing model directly to the state was good. I haven't encountered any huge problem. However, since there's a lazy load feature via getter property, we need to be careful when handling records because it might lead to an infinite loop.
For example, if User and Phone has a bidirectional relationship and if we reference data deeply, it causes the circular reference.
How We Should Proceed from Here
I've listed mostly the downsides, but the base idea is good. And I wouldn't think performance issue is going to be a big problem and even if it did, it should be a rare case. So I would like to proceed in the following manner.

Store class instance to the store directly.
Implement lazy loading feature by the getter method.
Implement indexing feature.

I'm thinking (1) and (2) should be included in Vuex ORM version 1.0.0, and (3) could start from version 1.1.0.
That means, we'll keep eager loading system, and current relationship loading mechanism (search records each time we fetch). The reason why I think we should keep eager loading and full scan data loading is not that of the performance concern but more because of the hardness of the indexing feature.
If we were to implement indexing feature at once and rely on that, it's going to take a long time before we can launch the next version of Vuex ORM. Until then, we must stop any other improvement to the library because we can't release the new version until the indexing feature is complete or users will lose functionality that Vuex ORM offers right now, or even worth, they'll encounter the dirty read issue when fetching the data.
Instead, if we can fetch data by scanning the existing records, there may be a performance issue but still, the library can work. We can then add indexing feature one by one for each relationship (for example, we can first release indexing system just for the HasOne relation and other relationship later on). In this case, we should have eager loading system to make the retrieving relationships more performant when fetching lots of data.
It shouldn't be hard to combine the full scan and indexing since all we have to do is check if there's an index, then retrieve it else do the scan.
In the end, I felt really like indexing is more of a cache system (we still need full scan capability internally anyway because we need it when creating indexes). Maybe it is worth considering to implement cache system into Vuex ORM. For example, maybe we can create index not when creating data, but fetching the relationship for the first time? Or even both? I feel like there are cool ideas behind the indexing approach and I'm willing to discuss more on what the best way to implement it.
It has been a long post, but I'm pretty happy about the result of the PoC project and my mind got quite clear on this issue. I'm more than happy to hear your thought on this.

This all sounds good, with one comment and two suggestions.
Comment is that indexing isn't the same as caching and it'd produce weird behavior to use an index instead of a cache. For example, if you cache on access (with memoize or similar), you'd end up with the following psudocode which would be a great source of head-banging bugs:
entities.store({ id: 1, foo: 'bar'});
var x = entities.get(1);
console.log(x.foo); // bar, as expected
entities.store({ id: 1, foo: 'something else'}); // Overwrite data
console.log(entities.get(1).foo); // Still prints 'bar' since it's returning the cached object!

To fix this, you'd need to evict the old entity from the cache when storing a new one, and at that point you've recreated a (less efficient) index :P
My first suggestion is to make reverse indexing explicit (ie, Bar.HasOne('foo').WithReverse('bars'). That way, you can make two different indexes (one on Foos, and one on Bars) and treat both as "forward" indexes. The idea is also to build up a type of relation map at startup, so you know at the outset which types have which relations and indexes.
My second suggestion is to simplify how you're creating the indexes. If you take the view that indexes only need to be updated when storing a record or deleting one, the whole indexing problem (to me) simplifies down to a "put" and "delete" method duo. Instead of trying to figure out what indexes to add where and when and all that, just run every entity operation through the put and delete methods.
These two methods can take care of scanning the record for relations, converting the relations into the appropriate property type (index or full scan), adding the record to the appropriate indexes, and finally putting the entity into the store. Combined with my first suggestion, each type will know exactly which indexes it needs to update. Delete will remove from indexes and the store.
So Normalizr will give you (essentially) an array of entities - just shove these through the put method and your indexes are all up to date!
Imo, you could say that updates direct updates to an indexed property are not supported (which they shouldn't be, following vuex conventions) and instead say an "update" is actually a delete of the old record followed by a store of the new one. Since this uses the methods already defined, indexing is consistent by default.
Mutating the store directly (ie, without put/delete) is undefined behavior.

Thanks for the feedback!

Comment is that indexing isn't the same as caching and it'd produce weird behavior to use an index instead of a cache.

Oh yeah, sorry for that, I meant we can treat indexing like a hash in term of management them so I thought maybe we can borrow some ideas from a general caching system. I'm not planning the cache the result of the query, but if we miss updating the indexing, that is going to return the wrong result so we have to manage the index similar to the cache.
And thank you for the other advice too! There are many ways to design how indexing system works and your's looks good. I'll go deeper when I actually start implementing this!

@tvillaren Thanks for the detailed feedback! And yeah basically that is what we should do.
It's just we need to take time to implement this because there're some nasty edge cases for complex relationship such as HasManyThrough or belongsToMany. In those relations, we must update when pivot table got modified as well. Sure we can do that, but it's going to take some work until we finish everything.
So what I think we should do is just release index mapping for each relation one by one, for example first with hasOne, next with belongsTo and such.
I agree we should have this feature and soon as we hit version 1.0.0, I think it7s time to work on to this.

what is the status of this?


Some help getting my data in the right format

Hi Boris,
SpikeShip itself is running smooth and fast but getting my spiking data in the right input format for SpikeShip is currently taking very long and I was wondering if you have conversion functions already. The required input format is very unintuitive to me and I'm losing a lot of time in for loops but I can't easily see how to make it faster.
My data is in the format used by the IBL (and I believe Allen as well). It consist of an array 1xN spike_times which contains all the times of N spikes. Then there is a 1xN spike_clusters array which contains the neuron identity of each spike. Then I have a Nx2 events array which contains the start (first column) and stop (second column) times of all my N events.
Do you happen to have a conversion script to go from this data format to the one required by SpikeShip or do you have an idea on how to do this efficiently?

Hi Guido,
I uploaded an example of how to convert data with the format you mentioned.
Please check SpikeShip/notebooks/example_data_conversion.ipynb and let me know if you have any questions.

Thanks, this is super helpful! However I do think there is a small bug in the code. Instead of
temp_ii_spike_times.append([current_count, (count_st+current_count)])
current_count += current_count

it should be
temp_ii_spike_times.append([count_st, (count_st+current_count)])
count_st += current_count

It's also still quite slow. For 45 neurons with 958697 spikes and 312 epochs it takes 21.1 seconds to run. I'll see if I can find ways to speed it up.

That's correct. Thx and great that it helped!

I optimized your function and it now runs much faster! Also I never realized the spike times should be relative to the epoch onset. Maybe good to emphasize in the other example notebooks? Thanks again for your help
def to_spikeship_dataformat(raw_spike_times, neuron_ids, epoch_intervals):
    """
    Processes spike times relative to epoch intervals and return `spike_times` and
    `ii_spike_times` indices for spikeship's data format. `raw_spike_times` is an
    array with all the spike times of a dataset, `neuron_ids` is the neuron to
    which such spike belongs to, and `epoch_intervals` contains the start and end
    time to consider for defining each epoch.

    Parameters
    ----------
    raw_spike_times : numpy.ndarray
    Array of 1xn which contains all the spike times (n).
    neuron_ids : numpy.ndarray
    Array of 1xN labels which correspond to neurons identity of each spike (i.e., N
    neurons).
    epoch_intervals : numpy.ndarray
    Array of `Mx2` values. `M` corresponds to the number of trials or epochs.
    First and second columns correspond to start and stop times, respectively.

    Returns
    -------
    spike_times : numpy.ndarray
        Array with relative spike times.
    ii_spikes_times: numpy.ndarray
        `(M,N,2)`-Array with indices per neuron (N) and epoch (M).
    """

    M = epoch_intervals.shape[0]

    spike_times = np.empty(0)
    ii_spike_times = np.zeros((epoch_intervals.shape[0],
                               np.max(neuron_ids) + 1, 2)).astype(int)

    # we can save time by excluding spikes outside of the outer epoch boundaries
    neuron_ids = neuron_ids[(raw_spike_times >= np.min(epoch_intervals))
                            & (raw_spike_times <= np.max(epoch_intervals))]
    raw_spike_times = raw_spike_times[(raw_spike_times >= np.min(epoch_intervals))
                                      & (raw_spike_times <= np.max(epoch_intervals))]

    for i_M in range(M):
        t_start = epoch_intervals[i_M][0]
        t_end = epoch_intervals[i_M][1]

        temp_spike_train = raw_spike_times[(raw_spike_times >= t_start)
                                           & (raw_spike_times < t_end)]
        # spike times are relative to trial onset
        temp_spike_train = temp_spike_train - t_start
        temp_neurons = neuron_ids[(raw_spike_times >= t_start)
                                  & (raw_spike_times < t_end)]

        this_order = np.argsort(temp_neurons)
        temp_spike_train = temp_spike_train[this_order]
        temp_neurons = temp_neurons[this_order]

        for i_N in np.unique(temp_neurons):
            ii_spike_times[i_M, i_N, :] = (np.array([np.argmax(temp_neurons >= i_N),
                                                    np.sum(temp_neurons <= i_N)])
                                           + spike_times.shape[0])
        spike_times = np.append(spike_times, temp_spike_train)

    ii_spike_times = ii_spike_times[:, np.unique(neuron_ids), :]

    return spike_times, ii_spike_times

Great! Yeah it will run much faster if you remove those spike times that are not included in your epochs, so it was a good idea to implement that. Also using NumPy arrays rather than python lists ;)
Let me know if you have any other questions.
Boris.


Deprecate the Measure shortcut in favour of MeasureFock

Context:
Currently, we have two ways of specifying a Fock measurement. Instantiating a MeasureFock() operation class, or using the pre-initialized Measure object (which is simply identical to MeasureFock().
This is confusing, doesn't really serve much purpose, and makes the documentation and code examples more confusing.
Description of the Change:
Deprecates Measure in favour of MeasureFock(). Using ops.Measure now raises a warning, instructing the user that this shortcut is now deprecated, and instead to use MeasureFock().
Documentation and tests have also been cleaned up to remove reference to Measure.
Benefits:
Cleaner documentation, tests, and serialization of Blackbird code.
Possible Drawbacks:
The line from strawberryfields.ops import * will raise the deprecation warning, as the user is importing the Measure object implicitly. I think this is unavoidable, due to our use of module level attributes. There isn't anywhere to place the deprecation message apart from
Related GitHub Issues: #123

Seems to be still one reference to Measure in examples/GaussianBosonSampling.ipynb

Fixed.


SPIKE - Grafana alert for wired only errors on NAC

As a DevOps engineer I want to investigate the possibility of having alerts produced through Grafana that identify when a rogue device is attempting to access our Network through a wired access point only.
Value/Purpose
As an engineer I want to distinguish wired only access errors to allow me to be notified when a device is unable to access the network through a wired cable.
Additional information
As an engineer, once I have been notified of the error I need to establish a process to deal with these errors effectively across the MOJ estate.  This task will not form part of this spike as this needs to be addressed separately.

Spike completed we have a way forward using the freeradius logline module.
https://wiki.freeradius.org/guide/eduroam-logging


Screenshots update

Screenshot tests use AndroidJUnit4, and Prod build

Added Readme
App now only uses one intrumentation runner
Emulator has a preset resolution and density

[Current coverage][cc-pull] is 16.13%

Merging [#682][cc-pull] into [master][cc-base-branch] will increase coverage by +<.01%

@@           master       #682   diff @@
========================================
  Files         461        461
  Lines       19762      19762
  Methods         0          0
  Branches     2869       2869
========================================
+ Hits         3173       3188    +15
+ Misses      16217      16199    -18
- Partials      372        375     +3

2 files (not in diff) in ...obile/module/db/impl were modified. more

Misses -10
Partials +2
Hits +8

1 files (not in diff) in ...rg/edx/mobile/module were modified. more

Misses -6
Partials +1
Hits +5

1 files (not in diff) in .../java/org/edx/mobile were modified. more

Misses -2
Hits +2

[][cc-pull]

Powered by Codecov. Last updated by 176933d
[cc-base-branch]: https://codecov.io/gh/edx/edx-app-android/branch/master?src=pr
[cc-pull]: https://codecov.io/gh/edx/edx-app-android/pull/682?src=pr

[Current coverage][cc-pull] is 16.13%

Merging [#682][cc-pull] into [master][cc-base-branch] will increase coverage by +<.01%

@@           master       #682   diff @@
========================================
  Files         461        461
  Lines       19762      19762
  Methods         0          0
  Branches     2869       2869
========================================
+ Hits         3173       3188    +15
+ Misses      16217      16199    -18
- Partials      372        375     +3

2 files (not in diff) in ...obile/module/db/impl were modified. more

Misses -10
Partials +2
Hits +8

1 files (not in diff) in ...rg/edx/mobile/module were modified. more

Misses -6
Partials +1
Hits +5

1 files (not in diff) in .../java/org/edx/mobile were modified. more

Misses -2
Hits +2

[][cc-pull]

Powered by Codecov. Last updated by 176933d
[cc-base-branch]: https://codecov.io/gh/edx/edx-app-android/branch/master?src=pr
[cc-pull]: https://codecov.io/gh/edx/edx-app-android/pull/682?src=pr

[Current coverage][cc-pull] is 16.13%

Merging [#682][cc-pull] into [master][cc-base-branch] will increase coverage by +<.01%

@@           master       #682   diff @@
========================================
  Files         461        461
  Lines       19762      19762
  Methods         0          0
  Branches     2869       2869
========================================
+ Hits         3173       3188    +15
+ Misses      16217      16199    -18
- Partials      372        375     +3

2 files (not in diff) in ...obile/module/db/impl were modified. more

Misses -10
Partials +2
Hits +8

1 files (not in diff) in ...rg/edx/mobile/module were modified. more

Misses -6
Partials +1
Hits +5

1 files (not in diff) in .../java/org/edx/mobile were modified. more

Misses -2
Hits +2

[][cc-pull]

Powered by Codecov. Last updated by d03096d
[cc-base-branch]: https://codecov.io/gh/edx/edx-app-android/branch/master?src=pr
[cc-pull]: https://codecov.io/gh/edx/edx-app-android/pull/682?src=pr

@aleffert Take a pass?

@BenjiLee cursor? Like the textedit insertion point?

This seems good. There are a couple style things we need to resolve for the project as a whole:

Test names
Prefixing instance members with m.

Again, I think we should just follow the AOSP guidelines (even though I personally think the m thing is pointless).

@aleffert Yep... that stupid textedit cursor.

Again, I think we should just follow the AOSP guidelines (even though I personally think the m thing is pointless).

I agree that this style guideline is mostly pointless. It seems to be designed for the benefit of people who're reading the code in a basic text editor instead of an IDE, which always provides a hint of the variable declaration level (local vs. object vs. static). Unless we also want to accommodate this use-case, we can probably leave out following this guideline.


Edited markdown files for better readability.

@MGlolenstine thank you for proofreading.

You're welcome.
Keep in mind that I'm not native and might've missed something 😁
.
Keep up good work!


Problem with showing "open pull request" shortcut

The screenshot speaks for itself.
I've cleared my cache and tried other pages but with the same result. (Chrome 65.0)

Thanks

same here

This was caused by some recent CSS removal. #1183
PRs welcome, it’s 3 lines of code with z-index


Printing

The appropriate CSS styles need to be put into place to ensure that verses are not cut across pages when printing.

To add to this: when a song is more than one page long, it just cuts off at one page and that's it. This might be a problem caused by the songdown-app styling but it's related to this.


Pester Task: Allow recursive for CodeCoverageFolder

Azure DevOps Extension you are using
Pester Task
Where are you running it?
Cloud Azure Pipeline
Version of Extension/Task
Latest
Expected behaviour and actual behaviour
I want to set the CodeCoverageFolder for a specific Folder recursively.
I tried following in my yml file
CodeCoverageFolder: '$(Build.SourcesDirectory)\Files\*'
CodeCoverageOutputFile: 'PesterCodeCovarge.xml'

I think you pass it directly to Pester and in this case, I cannot make a pattern to read the folder recursively! Read here more https://github.com/pester/Pester/issues/490
A solution would be
$files = Get-ChildItem .\File -File -Recurse -Include *.ps1
Invoke-Pester -Path \Files.Test -CodeCoverage $files

Is it possible to change in my YML something that he will use all files recursively inside the CodeCoverageFolder folder?
Steps to reproduce the problem

I think we can handle this in a few ways, we can add a switch to the task to tell it to recurse (defaulting to false) and then handle that in the task to automatically figure out what files you want to do code coverage on and pass that to Pester.
It shouldn't take too much work and I'll see if I can get it done over the next week or so.

Digging into the code I found we should already be handling this as we've got the following lines:
$Files = Get-ChildItem -Path $CodeCoverageFolder -include *.ps1, *.psm1 -Exclude *.Tests.ps1 -Recurse |
        Select-object -ExpandProperty Fullname

What happens if you drop the * from the end of your CodeCoverageFolder? It shouldn't make a difference but it's worth a try.

You are right it worked now!
Probably my fault sorry and thanks for the hint.


Icon request: Grist

URL for the brand / site / app
https://www.getgrist.com/
Logo resources for the icon
https://www.getgrist.com/wp-content/uploads/2021/07/logo-white.svg
Anything else?

Confirmation

[X] I have read and followed the issue guidelines.

Here you go!
https://github.com/aegis-icons/aegis-icons/blob/master/SVG/Grist.svg

:question: How to add icon(s) to Aegis?
:star: Support this project by giving a star! Just adding like to this comment is also appreciated.


Network error handling

Description

Fixes network error page
Display internal kuzzle error
Once connectivity is back, remove error page and restore previous navigation state

Boyscout

Upgrade Kuzzle SDK from 5.0.0 to 5.0.3

Codecov Report

Merging #284 into 2.x will decrease coverage by 0.69%.
The diff coverage is 60%.

@@            Coverage Diff            @@
##              2.x     #284     +/-   ##
=========================================
- Coverage   76.67%   75.97%   -0.7%
=========================================
  Files          39       39
  Lines         763      766      +3
=========================================
- Hits          585      582      -3
- Misses        178      184      +6

Impacted Files
Coverage Δ

src/services/kuzzle.js
100% <ø> (+42.85%)
:arrow_up:

src/vuex/modules/common/kuzzle/actions.js
89.79% <0%> (ø)
:arrow_up:

src/vuex/modules/common/kuzzle/store.js
81.81% <100%> (ø)
:arrow_up:

src/services/kuzzleWrapper.js
51.09% <61.11%> (-4.29%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5809ea...7a4875a. Read the comment docs.


Fix naming (minor)

You probably want “Apple TV” or “tvOS”. 📺
PS: thanks for this great article!

I do - thanks 👍


run multiple authenticators in parallel

In order to allow humans and service accounts which are not oauth2 compliant yet to authenticate, LDAP and OIDC authenticators must run in parallel.
This PR also introduce a yaml configuration for ease of configuration and also fixes some linting issues.

Why naming the 3rd commit "tmp rework (...)"?
Is this really temporary?

My mistake, I thought I had modified the commit message before pushing. I will do it.

Why naming the 3rd commit "tmp rework (...)"?
Is this really temporary?

My mistake, I thought I had modified the commit message before pushing. I will do it.

I suggest we eventually squash the PR instead of me rewriting the history. We don't really need the intermediary commits in the history.


Left shift with saturation

Current spec requires 4 vector instructions to implement a left shift with saturation.
# v0 is data
# v1 is shift
# a0 is vl

vsetvli x0, a0, e32, m1
vmv.v.i v2, 1
vsll.vv v3, v2, v1
vwmulsu.vv v4, v0, v3
vnclip.wv v5, v4, 0

However, with widening instructions, this method cannot work with SEW 64 for most hardwares. If we have a specified vector instruction like the following
vssll.vv vd, vs2, vs1, vm   # vd[i] = clip(vs2[i], vs1[i])
vssll.vx vd, vs2, rs1, vm   # vd[i] = clip(vs2[i], x[rs1])
vssll.vi vd, vs2, uimm, vm  # vd[i] = clip(vs2[i], uimm)

vssllu.vv vd, vs2, vs1, vm   # vd[i] = clip(vs2[i], vs1[i])
vssllu.vx vd, vs2, rs1, vm   # vd[i] = clip(vs2[i], x[rs1])
vssllu.vi vd, vs2, uimm, vm  # vd[i] = clip(vs2[i], uimm)

Only 1 instruction is needed.
vsetvli x0, a0, e32, m1
vssll.vv v5, v0, v1

@aswaterman
What's the reason that the spec only has "vssrl" but no left shift with saturation?
We could use widening instruction and vnclip to do the work and the "vxsat" could be set correctly by "vnclip".
But this approach is not work for SEW 64 input if the platform doesn't support SEW 128. It's hard to check and setup vxsat correctly.

My recommendation is to use a widening multiply and clip for the cases where SEW < ELEN.  The overhead of multiply vs. shift is usually not a concern, since vector units will nearly always provide fully pipelined multipliers.
For the SEW=ELEN case, I think it's totally reasonable to use a multi-instruction sequence (compare against 2^N, perform left shift, and, using the comparison result as a mask, overwrite some elements with -1).

I think it would be a performance issue if SEW equals to ELEN.
For example, with SEW and ELEN are 64, the code would be
# v1 is data
# v2 is shift
# a0 is vl

vsevli x0, a0, e64, m1
vsll.vv v3, v1, v2          # input do shift
li a1,<PHONE_NUMBER>854775807  # INT64_Max
vmv.v.x v4, a1
vsrl.vv v4, v4, v2          # INT64_Max / (2 ^ shift)
vmsgt.vv v0, v1, v4         # overflow if data > (INT64_Max / (2 ^ shift))
vmerge.vvm v3, v3, a1, v0   # INT64_MAX
li a1, -9223372036854775808 # INT64_Min
vmv.v.x v4, a1
vsra.vv v4, v4, v2          # INT64_Min / (2 ^ shift)
vmslt.vv v0, v1, v4         # overflow if data < (INT64_Min / (2 ^ shift))
vmerge.vvm v3, v3, a1, v0   # INT64_MIN

# v3 is result

Yeah, that's a fairly substantial implementation. Can you give more details how this shows up in applications?
A slight variation is to construct INT64_MIN from INT64_MAX, something like
# v1 is data
# v2 is shift
# a0 is vl

vsevli x0, a0, e64, m1
vsll.vv v3, v1, v2
li a1,<PHONE_NUMBER>854775807
vmv.v.x v5, a1              # change: use v5
vsrl.vv v4, v5, v2          # change: don't overwrite INT64_MAX
vmsgt.vv v0, v1, v4
vmerge.vxm v3, v3, a1, v0   # change: fix typo (.vxm suffix)
vnot.v v4, v5               # change: create/splat INT64_MIN in one instruction
vsra.vv v4, v4, v2
vmslt.vv v0, v1, v4
vmerge.vvm v3, v3, v5, v0   # change: use v5

# v3 is result

This avoids a scalar instruction (or two?) and scalar-to-vector register movement, but consumes another vector register.
You can also do a similar thing using vmul and vmulh, although that seems at least as costly.
I was hoping for a trick involving vsmul.vv but I didn't see it.
I think @aswaterman's earlier comment referenced the unsigned case, which involves roughly half the code.

If I didn't screw it up, I was able to improve on the algorithm a bit (9 -> 6 vector instructions):
vsevli x0, a0, e64, m1
li t0, (1<<63)             # -inf
vsll.vv v3, v1, v2
vsra.vv v4, v3, v2
vmsne.vv v0, v1, v4        # true if +/- overflow
vmerge.vxm v3, v3, t0, v0  # set to -inf if +/- overflow
vmsge.vi v0, v1, 0, v0.t   # true if +overflow
vnot.v v3, v3, v0.t        # set to +inf if +overflow

It also needs the additional instructions to check and setup the vxsat status.
https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#38-vector-fixed-point-saturation-flag-vxsat
The right shift instruction could do the similar things in one instruction without the sew==elen problem.
https://github.com/riscv/riscv-v-spec/blob/master/v-spec.adoc#134-vector-single-width-scaling-shift-instructions

There should be a later vector extension with greater support for fixed-point operations, and don't want to add more to vector spec before 1.0.


Call GetId on Cognito Identity when Access Control is available

Authorization may not be included in some Cognito operations.
Not including Authorization results in an error when Access control is enabled.
moto server version: 4.2.14
stack trace.
Error on request:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/werkzeug/serving.py", line 362, in run_wsgi
    execute(self.server.app)
  File "/usr/local/lib/python3.11/site-packages/werkzeug/serving.py", line 323, in execute
    application_iter = app(environ, start_response)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/moto_server/werkzeug_app.py", line 262, in __call__
    return backend_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1488, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1466, in wsgi_app
    response = self.handle_exception(e)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask_cors/extension.py", line 176, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
                                                ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/core/utils.py", line 111, in __call__
    result = self.callback(request, request.url, dict(request.headers))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/core/responses.py", line 254, in dispatch
    return cls()._dispatch(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/core/responses.py", line 455, in _dispatch
    return self.call_action()
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/core/responses.py", line 538, in call_action
    self._authenticate_and_authorize_normal_action(resource)
  File "/usr/local/lib/python3.11/site-packages/moto/core/responses.py", line 172, in _authenticate_and_authorize_normal_action
    self._authenticate_and_authorize_action(IAMRequest, resource)
  File "/usr/local/lib/python3.11/site-packages/moto/core/responses.py", line 156, in _authenticate_and_authorize_action
    iam_request = iam_request_cls(
                  ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/moto/iam/access_control.py", line 191, in __init__
    "Credential=", ",", self._headers["Authorization"]
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/werkzeug/datastructures/headers.py", line 493, in __getitem__
    return self.environ[f"HTTP_{key}"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'HTTP_AUTHORIZATION'

Hi @nonchan7720, thanks for raising this. I've identified the following operations where this problem could occur:

"AWSCognitoIdentityProviderService.ConfirmSignUp"
"AWSCognitoIdentityProviderService.GetUser"
"AWSCognitoIdentityProviderService.ForgotPassword"
"AWSCognitoIdentityProviderService.InitiateAuth"
"AWSCognitoIdentityProviderService.SignUp"

They have now all been fixed with #7331
Please let us know if we've missed anything.

Ah, that's my  bad @nonchan7720. I only looked at the operations for CognitoIdentityProvider, but GetId falls under CognitoIdentity.
#7335 should fix that (for real this time).

Can you release it in v4?
I haven't been able to migrate to v5 yet, so I don't want to be released with v5... 🙇

@nonchan7720 No, I'm not planning any V4 releases anymore

Okay.


docs(ha): Add paragraph indicating that Redis endpoint must include protocol and port.

@dorbin @jtk54 @lwander

@dorbin Updated to direct to doc describing redis URI standard


Scrolling and Spreads options not available

Attach (recommended) or Link to PDF file here: Link
Configuration:

Web browser and its version: Firefox Developer Edition 61.0b11 (64-bit)
Operating system and its version: Windows 10  1803 Build 17134.81
PDF.js version: Built-in
Is a browser extension: No

Steps to reproduce the problem:

Open any PDF file from PC ('Menu>Open File...' or Drag and drop PDF file into browser window)

What is the expected behavior? (add screenshot)

What went wrong? (add screenshot)

Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):
https://mozilla.github.io/pdf.js/web/viewer.html

The scrolling modes are merged quite recently, so they will most likely be in a later Firefox version, probably Firefox 62.

Note that there is a version 2 pre-release now: https://github.com/mozilla/pdf.js/releases/tag/2.0.550


[QUESTION] Scan depth

Hello, I have been using this excellent tool to scan Javascript projects and I have a question regarding the depth of the scan.
One of my projects (let's call it "X") had a dependency on a library( let's call this library "Y").
When I ran the scan in "X", it detected 3 vulnerabilities related to "Y".
I decided to fork from the "Y" repository and fix those vulns myself.
The thing is that when I ran the scan in the "Y" repo for the first time, it detected approximately 15 vulns. It means that, for the same code, there was a different scan result depending on if I was scanning it from a client or in the repo itself.
Is there any stuff related to the scan depth that am I not understanding? Thank you.

Check the scope of the dependencies that you scanned, this tool does not scan code it scans a dependency tree


Make a install.php for systems like MAMP

We don't want people touching command line. Humans can download files and put them in folders. Developers can run bash scripts.

could also help w/ installation adoption.


fix: create upload dir during build 🐛

@ciprian-cgr
@adrian-mih
@ClimenteA
This is how the issue is fixed. For those interested in the topic, here's a good link to start.


Optimistic UI pattern for portfolio create, update and delete actions.

fixes: https://projects.engineering.redhat.com/browse/SSP-347
Although this issue is suggesting to use spinners, we (me and Susan) would like to move towards optimistic UI pattern - we expect that the action will success and render UI accordingly and the user work with the app and not wait for server synchronization which is happening in the background. For instance don't wait until portfolio is created on back end and add new card immediately.
Changes

added optimistic UI patter to porfolios actions

create portfolio
update portfolio
remove portfolio

storing previous state before adding/editing/deleting portfolio

this allows us to restore state if the action fails

added portfolio card disabled state
added another error message key to notifications middleware config

cc @sbuenafe-rh

@lgalis we have been talking about this with Susan a couple of weeks ago. We should be using it instead of loaders and spinners which can take over the screen very easily.
What we don't know just yet, is how to show cards in disabled state. That is probably a placeholder for now.

Looks good! Will look into other options for the show cards in disabled state.


addMessage is calling the error handler with a HTTP status code of 201

A successful call to currentUser.addMessage ends up rejecting instead of resolving on success.
It looks like the request is successful, but the response parsing logic is wrong in executeNetworkRequest in the core. A 201 response is a successful object creation, but the handler is looking for status codes of 200 only, and rejecting otherwise.
In the chrome backtrace,
xhr.onreadystatechange = function () {
            if (xhr.readyState === 4) {
                if (xhr.status === 200) {
                    resolve(xhr.response);
                }
                else if (xhr.status !== 0) {
                    reject(network_1.ErrorResponse.fromXHR(xhr));
                }
                else {
                    reject(new network_1.NetworkError("No Connection"));
                }
            }
        };

(I don't have pusher's sourcemaps on, so not quite sure where this is.)
Not the worst, but surprising that a successful create would end up with the promise rejecting instead of resolving.

This is also causing currentUser.createRoom to fail.

It looks like you just need to upgrade they version of pusher-platform as this was fixed there:
https://github.com/pusher/pusher-platform-js/blob/0e54667e4b19e0b01e222de15748a22401a6092a/src/request.ts

@rrcobb sorry I didn't see this issue sooner - I had neglected to watch the repo.
@troygoode is correct and we're going to release a new version of pusher-platform-js and the Chatkit JS SDK today!
And thanks for the report!

Should be fixed in 0.1.2 🙂


move plugin network creation/removal into weaveutil

The main gotcha is that we lose the ability to specify a different mesh network name, or disable it altogether. That was always poorly supported and IMO is not worth losing sleep over until we / our users have a real need for it.
Closes #1897.

btw, weave stop time has dropped from 1.5s to 1.1s for me with this change.

we run the plugin with restart=always, so it will get restarted

Nitpicking: it does not get restarted immediately if stopped by the user, e.g. docker stop. It does get restarted on the next reboot.

CreateNetwork does sometimes(always?) not return ErrNetworkAlreadyExists

Tracked at https://github.com/docker/docker/issues/18363


Improved: the types for product and order(#2eh9zgt)

Dependent on #45

Closing as will done differently


cob_substitute: 0.6.9-1 in 'kinetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository cob_substitute to 0.6.9-1:

upstream repository: https://github.com/ipa320/cob_substitute.git
release repository: https://github.com/ipa320/cob_substitute-release.git
distro file: kinetic/distribution.yaml
bloom version: 0.9.3
previous version for package: 0.6.8-1

cob_docker_control
* Merge pull request #55 <https://github.com/ipa320/cob_substitute/issues/55> from fmessmer/feature/python3_compatibility
  [ci_updates] pylint + Python3 compatibility
* python3 compatibility via 2to3
* Contributors: Felix Messmer, fmessmer

cob_reflector_referencing

No changes

cob_safety_controller
* Merge pull request #55 <https://github.com/ipa320/cob_substitute/issues/55> from fmessmer/feature/python3_compatibility
  [ci_updates] pylint + Python3 compatibility
* python3 compatibility via 2to3
* Contributors: Felix Messmer, fmessmer

cob_substitute

No changes

Holding for sync


Curious what causes inconsistent pronunciation of certain words

This is not an issue per-se but I noticed that both on my model and on LJ,  certain words like, "Can't" vary between the brittish way of pronouncing and american. EG: C ANT vs C AUNT and sometimes I get both in one sentence. I thought that by using phonemes it would be consistent?  Do you know what causes this sort of issue? Is it fixed by simply having more training data? LJ is 24 hours, mine is 8.5.
Thank you

Hi, I would think that it might be inconsistent in the dataset then. Even with phonemes the model is learning to interpret the input in a ceartain way, although it should be more consistent. If you have two opposing examples you could grep though the dataset and look for similar texts, maybe you see some differences. I did that for weird prononciations in our German dataset and found some inconsistencies, e.g. the word 'Union' was ambiguous (German and English).

Hi
Thank you for the information!
As it turns out, can't does not appear at all in my training set! That would explain it for mine.
I will generate some sample sentences, look at failure points then probably add to my training set where there are weaknesses.
But I checked ljspeech and it does have a few occurrences which were all pronounced properly. That said, that may not be enough samples for it to learn what to do when it appears twice in 1 sentence.

Interesting - it could also be a matter of training longer, I found that it is really easy to judge too quickly, sometimes the model converges after 400k steps only.

Hi,
I have trained my model to 272K and it still has a lot of the same funny problems. I synthesized the same sentence on the pretained model and it experienced a lot of the same issues:
Here is mine: https://vocaroo.com/eT1Q7T8MQsl
Here is LJSpeech: https://vocaroo.com/1lTbRTS4vhE
The sentence: "Let's walk, and talk, go on the internet, get 100 percent on a test, then use data to buy pizza. After that, we will cause the 1 percent per capita to listen to Mariah, because we are very tired, and want, to go, to sleep..."
It gives this strange Boston accent where words ending in 'a' like data are pronounced dater, capiter, pizzer Mariher, etc.
My dataset is the first 14 chapters of LJSpeech. Maybe it's caused by not enough examples of those types of vowel sounds....

Not sure how many such words differ (and in what way) between UK and US English but probably enough to trip it up a bit.

Good point @m-toman, I should probably default the LJSpeech hparams to en-us. That would take another week of training though...


mgmt, firstbeta for devopsinfrastructure from TypeSpec

Description
Issue: https://github.com/Azure/sdk-release-request/issues/5139
namespace review: https://github.com/Azure/azure-sdk-pr/issues/1908
Strangely pipeline hang at generating... Retried few times, hang still. Figuring out the cause.. Use manual generation this time.
https://dev.azure.com/azure-sdk/internal/_build/results?buildId=3813389&view=logs&j=12f1170f-54f2-53f3-20dd-22fc7dff55f9&t=dee4e6ec-be1b-5671-4678-ab293b4004c5
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[ ] The pull request does not introduce [breaking changes]
[ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
[ ] I have read the contribution guidelines.

General Guidelines and Best Practices

[ ] Title of the pull request is clear and informative.
[ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[ ] Pull request includes test coverage for the included changes.

/azp run prepare-pipelines

/azp run java - devopsinfrastructure - ci

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure.resourcemanager:azure-resourcemanager-devopsinfrastructure


Formatting of content required for displaying message

Hi Tobias,
Its been a long time to have a discussion with you. Hope you doing well.
Yesterday I had a thought that the maintenance message which we generally set to display to non admin user should have some sort of formatting. Such as bold/ italics/ increment/decrements of font size/ hyperlinks should behave as intended. Formatting which redmine provides while putting description against a tracker
I think, it would be better to incorporate these features in upcoming version.
Regards,
Lajish Lakshmanan

textile or markdown formatting will be supported in the next version. added in 3091f5e on the deface-views branch.
you're welcome to test the changes by checking out the new deface-views branch.
But attention! You will first have to install the Redmine Base Deface plugin! follow their installation notices, then update this plugin and then run rake redmine:plugins:migrate RAILS_ENV=production

see upgrade instructions for details

the fix is merged and available in plugin 2.0.0

Hi there,
Thanks for adding the requested feature to plugin. I will surely test it and return back to you if i found any glitch in it.
Regards,
Lajish Lakshmanan


Proposal: Support Application.ShutdownMode for WinUI 3

Proposal: Support Application.ShutdownMode for WinUI 3

Summary

Add the Application.ShutdownMode property from WPF to WinUI 3, for finer-grained control of app lifecycle.
Rationale

A large number of apps are 'daemon-like' and need to run in the background as a system tray icon or otherwise until invoked. However, WinUI 3 in its current state does not allow for this as it terminates the application when all its windows are closed. Adding Application.ShutdownMode would allow developers to configure this and run in the background without resorting to ugly workarounds.

Scope

Capability
Priority

This proposal will allow developers to control the automatic termination behaviour of WinUI
Must

Important Notes
None

Open Questions

This isn't a perfect solution since apps running without any sort of UI can offer poor UX since the user is unaware that it is still running. In the future, the framework itself may need to support background apps in a user-friendly way (perhaps with microsoft/WindowsAppSDK#713?)

The Windows App SDK 1.5-experimental2 release from a few days ago has added an experimental API, Application.DispatcherShutdownMode, with the options OnLastWindowClose and OnExplicitShutdown.

For anyone who really needs this now, it's doable by:

getting your hwnd via IWindowNative
subclassing or otherwise hooking the hwnd
intercepting WM_CLOSE and starting your shutdown; call this->Close() (or ShowWindow() on the hwnd)
intercepting WM_DESTROY and ignoring it - the WM_DESTROY message is what triggers the application exit on v1.4
When you actually want to exit, call PostQuitMessage() from the main thread
Leave a TODO/issue/task to remove this when you upgrade to v1.5 :)

This is included in 1.5. See the 1.5 release notes for more details.


✨ feat: better tag extraction (#897)

Checklist

[ ] The title of my PR starts with a Conventional Commit prefix (fix:, feat:, docs: etc).
[ ] I have read the Contributor Guide and followed the process outlined for submitting PRs.
[ ] I have updated/added tests for new/updated/fixed functionality.
[ ] I have updated/added dartdoc comments with ///.

Breaking Change

[ ] Yes, this is a breaking change.
[ ] No, this is not a breaking change.

Related Issues

Codecov Report

Merging #898 (789d8c5) into main (32127c9) will decrease coverage by 0.68%.
The diff coverage is n/a.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
@@            Coverage Diff             @@
##             main     #898      +/-   ##
==========================================
- Coverage   82.25%   81.57%   -0.68%
==========================================
  Files         255        3     -252
  Lines        3218       38    -3180
==========================================
- Hits         2647       31    -2616
+ Misses        571        7     -564

see 252 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more