Refactoring Stacking, Multiplexer, Ensembler and TransformedTarget Forecasters

Reference : https://github.com/alan-turing-institute/sktime/issues/955
-- This is a copy of 965 without changes to reducer.py (a time when there was no bug)
We need to slowly re-factor all forecasters to be compliant with the new interface (see PR #912). i.e., no overrides of fit etc and descendant classes only implementing core logic, while the BaseForecaster implements check etc logic.
Currently this is running due to various downwards compatibility workarounds, which should be removed once all forecasters are ready (in "no-override" state)
The forecasters that still need a downstream refactor are:

[x] EnsembleForecaster
[x] MultiplexForecaster
[x] TransformedTargetForecaster
[ ] _Reducer and related code
[x] StackingForecaster

looks like checks are all fine, so do you want to convert to "proper" PR and request reviews?

and you may want to close #955 of which this is now a duplicate

I'm not really sure what the best solution is. A few thoughts:

I see tags as being defined on the class (not object) level, conceptually I think of them as describing algorithm properties, so I would prefer not to update them dynamically
In the cases of the above mentioned (composition) algorithms, for the implementation (i.e. input validation of the forecasting horizon), the tag should be set to False, the composite doesn't require the forecasting horizon during fitting, it merely passes it on to the components, which have their own tag and raise an error accordingly if they require the forecasting horizon during fitting
For documentation, when users may use tags for finding relevant models, setting it to False may be misleading, but conceptually still makes sense, as it refers to the composition algorithm, rather than a particular instance of the composite

ok, agreed - let's split off the larger tags discussion, and just do a "safe" refactoring here.
Outstanding changes:

the _SktimeForecaster removal is not tags related and should be done
multiplexer tag should be changed to requires fh in fit (see above)

the test examples are now failing - they ran before.
Can you please check what's happening?

I think it's the multiplexer - the tutorial notebook passes the fh too late

I'd suggest:

don't change the notebook, since #972 is currently under review, and the change to notebook will just cause very fiddly merge issues
instead, change the tag back, and open a new issue that it needs to be changed once #972 is merged, together with the notebook

What do you think, @mloning?

@fkiraly Setting requires-fh-in-fit: to True is the cause

@fkiraly Setting requires-fh-in-fit: to True is the cause for the failure of the multiplexer

You mean, to False, no? We want to set it back to True according to my suggestion - please confirm?

pinging @ninamiolane

@mloning this I ready for review :)


Fail on cjs files in a node_modules folder somewhere out of the project

Hi, first of all, a really nice job!
I just came across an issue when I have to import a cjs file in another node_modules folder out of the target project. The error message is like:
caught SyntaxError: The requested module '/@fs/Users/jinjiang/xxx/packages/bar/node_modules/foo/index.js?v=f73c5dad' does not provide an export named 'foo' (at main.js?t=1682654369388:8:10)

The minimized reproduction is here to refer to:
https://github.com/Jinjiang/reproductions/tree/vite-plugin-commonjs-20230428
As it mentions in the main.js:
// // this would work
// import { foo } from './foo'

// // this would work
// import { foo } from '../foo'

// this would fail
import { foo } from '../bar/node_modules/foo'

Thanks.

Maybe the way can solution problem for you 🤔
https://github.com/vite-plugin/vite-plugin-commonjs/pull/20/files#diff-a2a171449d862fe29692ce031981047d7ab755ae7f84c707aef80701b3ea0c80R63
commonjs({
  filter(id) {
    if (id.includes('node_modules/foo')) {
      return true
    }
  },
})

If this doesn't work, I'll iterate on the next version.

@caoxiemeihao I tried again. Unfortunately, it doesn't work. Not sure where the broken point of the logic is.

Okay! I'll try to Debug it.

Update: I debugged a little bit and found this case bit 2 walls at the same time:

/node_modules/ which might be skipped by filter
extensions (the native path.extname is only for file systems which can't handle the query string like xxx.js?v=f73c5dad)

However, I don't have a big picture of your tech design yet. Not sure which way to solve this is the best and won't involve other issues. So I'd like to leave the eventual bugfix to you.
Thanks.

It works to me @ v0.7.0 - coming soon

大佬看看代码不(帮忙 review 下呢 🥺)，放假回家了先，明天下午 or 晚上有时间我发个版本

Would you mind checking one more case that I found? It's about package regexpu-core. If we have import rewritePattern from '../other-project/node_modules/regexpu-core', it will get this error:
[vite] Internal server error: invalid import "require(`regenerate-unicode-properties/${ path }.js`)". Variable bare imports are not supported, imports must start with ./ in the static part of the import. For example: import(`./foo/${bar}.js`).

However, import rewritePattern from 'regexpu-core' has no such a problem.
This is also about requests to other node_modules. So I just put it in this same GitHub issue.
Thanks.

v0.7.0 is out, can you re-provide a minimal repo base on v0.7.0 😂

I'm afraid the error is still there on v0.7.0 as:
Uncaught SyntaxError: The requested module '/@fs/Users/xxxx/reproductions/packages/bar/node_modules/foo/index.js?v=d571362a' does not provide an export named 'foo' (at main.js?t=1682991288619:8:10)

Here is the reproduction
https://github.com/Jinjiang/reproductions/tree/vite-plugin-commonjs-20230502
I added the case about regexpu-core there too.
Thanks.

This error comes from the cache.

Uncaught SyntaxError: The requested module '/@fs/Users/xxxx/reproductions/packages/bar/node_modules/foo/index.js?v=d571362a' does not provide an export named 'foo' (at main.js?t=1682991288619:8:10)

regexpu-core requires support mono-repo(pnpm) - TODO

@caoxiemeihao I tried it from a brand-new workspace, including re-generated lock-file or not. The first error was still there. Would you mind double-checking?

@Jinjiang 👋
Everything is right, except that our small range of judgment causes problems :(

// It works on my local machine 😆
commonjs({
  filter(id) {
    if (id.includes('node_modules/foo')) {
      return true
    }
    if (id.includes('node_modules/regexpu-core')) {
      return true
    }
    if (id.includes('node_modules/regenerate-unicode-propertie')) {
      return true
    }
  },
}),

So perhaps we can be lazy
commonjs({
  filter(id) {
    if (id.includes('node_modules')) {
      return true
    }
  },
}),

@caoxiemeihao I added the filter config and updated the reproduction repo. The error was still there. And this time the terminal also had errors:
[plugin:vite-plugin-commonjs] invalid import "require(`regenerate-unicode-properties/${ path }.js`)". Variable bare imports are not supported, imports must start with ./ in the static part of the import. For example: import(`./foo/${bar}.js`).
<EMAIL_ADDRESS>
$ pnpm dev

> reproduce@ dev /xxx/reproductions
> pnpm --filter vite-project dev

><EMAIL_ADDRESS>dev /xxx/reproductions/packages/vite-project
> vite

  VITE v4.3.2  ready in 201 ms

  ➜  Local:   http://localhost:5173/
  ➜  Network: use --host to expose
  ➜  press h to show help
Error:   Failed to scan for dependencies from entries:
  /xxx/reproductions/packages/vite-project/index.html

  ✘ [ERROR] invalid import "require(`regenerate-unicode-properties/${ path }.js`)". Variable bare imports are not supported, imports must start with ./ in the static part of the import. For example: import(`./foo/${bar}.js`). [plugin vite-plugin-dynamic-import:pre-bundle]

    ../../node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1421:21:
      1421 │         let result = await callback({
           ╵                      ^

    at dynamicImportToGlob (file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-dynamic-import@1.3.2/node_modules/vite-plugin-dynamic-import/dist/index.mjs:321:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async globFiles (file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-dynamic-import@1.3.2/node_modules/vite-plugin-dynamic-import/dist/index.mjs:508:10)
    at async DynaimcRequire.generateRuntime (file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-commonjs@0.7.0/node_modules/vite-plugin-commonjs/dist/index.mjs:284:26)
    at async transformCommonjs (file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-commonjs@0.7.0/node_modules/vite-plugin-commonjs/dist/index.mjs:430:20)
    at async file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-commonjs@0.7.0/node_modules/vite-plugin-commonjs/dist/index.mjs:379:30
    at async requestCallbacks.on-load (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1421:22)
    at async handleRequest (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:723:13)

  This error came from the "onLoad" callback registered here:

    ../../node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1279:20:
      1279 │       let promise = setup({
           ╵                     ^

    at setup (file:///xxx/reproductions/node_modules/.pnpm/vite-plugin-commonjs@0.7.0/node_modules/vite-plugin-commonjs/dist/index.mjs:372:17)
    at handlePlugins (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1279:21)
    at buildOrContextImpl (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:968:5)
    at Object.buildOrContext (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:776:5)
    at /xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:2172:68
    at new Promise (<anonymous>)
    at Object.context (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:2172:27)
    at Object.context (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:2012:58)
    at prepareEsbuildScanner (file:///xxx/reproductions/node_modules/.pnpm/vite@4.3.2/node_modules/vite/dist/node/chunks/dep-7efa13d7.js:43041:26)

  The plugin "vite-plugin-dynamic-import:pre-bundle" was triggered by this import

    main.js:20:27:
      20 │ import rewritePattern from '../bar/node_modules/regexpu-core'
         ╵                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    at failureErrorWithLog (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1636:15)
    at /xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1048:25
    at runOnEndCallbacks (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1471:45)
    at buildResponseToResult (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1046:7)
    at /xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1058:9
    at new Promise (<anonymous>)
    at requestCallbacks.on-end (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:1057:54)
    at handleRequest (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:723:19)
    at handleIncomingPacket (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:745:7)
    at Socket.readFromStdout (/xxx/reproductions/node_modules/.pnpm/esbuild@0.17.18/node_modules/esbuild/lib/main.js:673:7)

Here is the update:
https://github.com/Jinjiang/reproductions/tree/vite-plugin-commonjs-20230502
https://github.com/Jinjiang/reproductions/commit/fb71ab36ab18dad065345df46d25914ee1a8f408

v0.7.1 is out!

It works now. Thanks.


how to use  highchats option chart{event:load:function(){}}?

how to use? please help me!
thank you
<EMAIL_ADDRESS>

is this planning to be fixed?  We are running into this issue as well.  I also saw a reference to this issue in this posting - http://stackoverflow.com/questions/24233173/explanation-of-why-i-need-to-wrap-a-highcharts-jquery-event-in-an-angular-timeo

HI,
Is there any solution on the above issue ?
I am trying to achieve the below functionality using highcharts-ng but to no avail.
http://jsfiddle.net/LHSey/129/
All events are fired before the chart is drawn. But if I use a timeout with 0 sec it works.
Please help.

Hi,
I found a "trick" for this issue few months ago, using redraw event
chart: {
    events: {
        redraw: this.completeGrowthGraph
    }
}

I create a variable for each graph
var growthRendered = false;

and I test the variable in the completeGrowthGraph preventing the function to execute twice
this.completeGrowthGraph = function() {
    if (growthRendered) {
    } else {
        growthRendered = true;
        // your load function code here
    }
}

Hope it helps.


Update the win conditions so they're no longer broken.

Probably requires: https://github.com/evinism/lambda-explorer/issues/17

Most tutorial should be good at this point-- the challenges need some work tho.

all necessary tutorial win conditions are correct as of now.


Increase the limit for API list

Is your feature request related to a problem? Please describe.
I need to make queries larger for more than 1000 entities without using nextToken.
Describe the solution you'd like
Increase the limit to AWS AppSync's forEach to more than 1000.
Describe alternatives you've considered
I could implement my logic with nextToken but that would take days, because it would add a lot more complexity into the app.

This is an issue with AWS Appsync, not AWS Amplify. You should contact the AppSync team through AWS support to suggest that increase.
Frankly, I would highly recommend using nextToken to implement your query because AWS probably won't change that limit for a long time.

Why not just manually set limit to 1000? That is the easiest fix.

@ajhool Thank you for letting me know. Guess I will have to use nextToken then 😢
@YikSanChan I need more than a thousand, which according to ajhool is apparently impossible.

@janhesters No, it is possible, simply set the limit in your graphql query.
An example can be found https://docs.aws.amazon.com/appsync/latest/devguide/using-your-api.html:
query list {
    getTodos(limit:2){
        todos {
            id
            name
            status
            comments{
                commentid
                content
            }
        }
        nextToken
    }
}

Change to limit: 2 to limit: 5000 simply works, though ugly.

@YikSanChan No, it doesn't (at least for me it didn't).  Click on the datasource and post the resolver code here?
When I look at my resolvers, I don't actually see them using the  forEach operator on the retrieved items, so I believe that my resolvers would allow for more than 1000 items to be returned.

@ajhool Sure, I'll give you everything. Here is the query I wrote:
query list {
  listContacts(limit: 1001) {
    items {
      id firstName lastName
    }
  }
}

Here is the respose I get:
{
  "data": {
    "listContacts": null
  },
  "errors": [
    {
      "path": [
        "listContacts"
      ],
      "data": null,
      "errorType": "MappingTemplate",
      "errorInfo": null,
      "locations": [
        {
          "line": 32,
          "column": 3,
          "sourceName": null
        }
      ],
      "message": "List size cannot exceed 1000"
    }
  ]
}

Here is the resolver for the request:
#set( $limit = $util.defaultIfNull($context.args.limit, 10) )
{
  "version": "2017-02-28",
  "operation": "Scan",
  "filter":   #if( $context.args.filter )
$util.transform.toDynamoDBFilterExpression($ctx.args.filter)
  #else
null
  #end,
  "limit": $limit,
  "nextToken":   #if( $context.args.nextToken )
"$context.args.nextToken"
  #else
null
  #end
}

And for the response:
## No Static Group Authorization Rules **

## [Start] If not static group authorized, filter items **
#if( ! $isStaticGroupAuthorized )
  #set( $items = [] )
  #foreach( $item in $ctx.result.items )
    ## No Dynamic Group Authorization Rules **

    ## [Start] Owner Authorization Checks **
    #set( $isLocalOwnerAuthorized = false )
    ## Authorization rule: { allow: owner, ownerField: "owner", identityField: "cognito:username" } **
    #set( $allowedOwners0 = $item.owner )
    #set( $identityValue = $util.defaultIfNull($ctx.identity.claims.get("username"), $util.defaultIfNull($ctx.identity.claims.get("cognito:username"), "___xamznone____")) )
    #if( $util.isList($allowedOwners0) )
      #foreach( $allowedOwner in $allowedOwners0 )
        #if( $allowedOwner == $identityValue )
          #set( $isLocalOwnerAuthorized = true )
        #end
      #end
    #end
    #if( $util.isString($allowedOwners0) )
      #if( $allowedOwners0 == $identityValue )
        #set( $isLocalOwnerAuthorized = true )
      #end
    #end
    ## [End] Owner Authorization Checks **

    #if( ($isLocalDynamicGroupAuthorized == true || $isLocalOwnerAuthorized == true) )
      $util.qr($items.add($item))
    #end
  #end
  #set( $ctx.result.items = $items )
#end
## [End] If not static group authorized, filter items **

$util.toJson($ctx.result)

@ajhool

I would actually say that this is an Amplify schema bug for allowing limits > 1000 if authorization is required.

Yes, I think that is the case right here. But still a greater limit would be neato! :)

I'd recommend contacting that AppSync team via AWS support and closing this issue as Amplify can do nothing about the limit

@ajhool Thank you.

@ajhool I closed the issue, but I do have two more questions, maybe you know a solution for these 🤗 .

I'm basically building a contact list app. Up until now, no user had more than a thousand contacts. I display all contacts at once at several points in the app. Let's say I use a useEffect hook for example in the list of all contacts to fetch all contacts iteratively at once using the nextToken (until its null).

useEffect(() => {
  iterativelyFetchContacts();
}, []);

The user can also update and create contacts.
export default compose(
  // Using AppSync's helper methods for optimistic UI
  graphqlMutation(gql(createContact), ListContacts, 'Contact'),
  graphqlMutation(gql(updateContact), ListContacts, 'Contact'),
  // ... other HOCs
)(List);

If the user now updates a contact or creates one, the ListContacts query gets invoked which can only fetch 1000 contacts. This results in some contacts missing from the list, if he has more than a thousand. Is there a way to fix this, or do I have to implement optimistic updates myself if I have more than a thousand entities?

Is there a way to sort the incoming list based on custom fields? E.g. the entities in my app are sorted on the client by firstName and lastName. But if I had to add pagination the first thousand contacts in the database would obviously not be sorted alphabetically. I know there is a sortField in the GraphQL API, but only for connection's and you can only sort based on one field.

Not sure. If you're using Apollo, you can use the update function https://www.apollographql.com/docs/react/essentials/mutations#update to call the iterative query after a mutation. You could also just use an isDirty flag in your react component that recalls the iterative query. It's hard for me to imagine an application where a contact list of +1000 needs to worry too much about the few seconds after adding a contact

I'm actually not sure if AppSync returns items based on alphanumeric sorting of the primaryKey (or sortKey if primaryKeys are the same). But, if so, you might be able to make it work by setting the ID! of your schema like so:

// The userId prevents conflicts across users. (Eg. my John Doe contact is different from your John Doe contact).
const userId = getUserId(); // from Auth ??

// The contact's name.
const firstName = 'john';
const lastName = 'doe';

// A uuid in case a user has multiple John Doe contacts.
const uniqueId = uuid().substring(0, 5)

// concat those details in such a way that they are in alphanumeric order.
let primaryKey = userId.substring(0, 8) + firstName + lastName + uniqueId;

createContact(
  {
     id: primaryKey
     ...
  }
)

@ajhool Thank you very much for your help. I'm going to look into these approaches 🙏


Translate the site navigation into Spanish

The content for the navigation is in this file…
https://github.com/foodoasisla/site/blob/master/_data/locales.yml
And you can see an example Spanish page here…
https://foodoasis.la/es/
Halina can help with the translation.

https://github.com/foodoasisla/site/pull/64
Nice work on this one, @MarioEsparza !

Once the site is redeveloped we will re-translate it.


Add support for generating IntrospectionFragmentMatcher data

If you use unions or interfaces you need to generate and import some JSON in Apollo client:
http://dev.apollodata.com/react/initialization.html#fragment-matcher
It would be great apollo-codegen had a command for generating that JSON file, instead of having to copy the approach described in the docs. Something like this:
apollo-codegen fragment-types --output fragmentTypes.json

const { graphql } = require("graphql");

const QUERY = `
  {
    __schema {
      types {
        kind
        name
        possibleTypes {
          name
        }
      }
    }
  }
`;

async function generateFragmentTypes(schema) {
  const result = await graphql(schema, QUERY);

  const fragmentTypes = {
    ...result.data,
    __schema: {
      ...result.data.__schema,
      types: result.data.__schema.types.filter(
        type => type.possibleTypes !== null,
      ),
    },
  };

  return JSON.stringify(fragmentTypes, null, 2);
}

Actually, you don't need any changes to apollo-codegen.
Schema retrieved by apollo-codegen has sufficient data to generate introspection query result data.
Just do following:
import apolloCodegenData from '../schema/schema.json';

const introspectionQueryResultData = {
  ...apolloCodegenData.data,
  __schema: {
    ...apolloCodegenData.data.__schema,
    types: apolloCodegenData.data.__schema.types.filter(
      type => type.possibleTypes !== null
    ),
  },
};

const fragmentMatcher = new IntrospectionFragmentMatcher({
  introspectionQueryResultData,
});

Hi! It’s been a litte while since I worked with Apollo and GraphQL – have I missed an update that does this, or did you decide not to do it? I’m fine with either, just wanted to know your decision :)

The latest documentation suggests using https://graphql-code-generator.com/ for generating the fragment schema:
https://www.apollographql.com/docs/react/advanced/fragments#fragment-matcher

Just updating to say instead of apolloCodegenData.data and apolloCodegenData.data.__schema, use apolloCodegenData and apolloCodegenData.__schema respectively.
eg.
import apolloCodegenData from '../schema/schema.json';
import { IntrospectionFragmentMatcher } from 'apollo-cache-inmemory';

const introspectionQueryResultData = {
  ...apolloCodegenData,
  __schema: {
    ...apolloCodegenData.__schema,
    types: apolloCodegenData.__schema.types.filter(
      type => type.possibleTypes !== null
    ),
  },
};

const fragmentMatcher = new IntrospectionFragmentMatcher({
  introspectionQueryResultData,
});

Also, the docs are still out-of-date [1] and suggest that https://graphql-code-generator.com/ is the correct solution.
[1] https://www.apollographql.com/docs/react/data/fragments/#fragments-on-unions-and-interfaces


Added output feature to the main script

Code like this is impossible to test, we should discuss how we handle this?
Internal error codes? Return error strings Haskell Style?

https://github.com/Porthmeus/MeMoMe/pull/113 Adds a new parameter to the match function, we should add a parameter to the main script for this as well


Porting to ASP.NET Core - Chapter 2

Summary
Adding Chapter 2. Ignore Chapter 1 files as these are under discussion in PR #21042

Hi @ardalis ,  Chapter 2 pages are not getting listed in Internal Review site. That could be because of extra dash ("-") in toc.yaml line 21 and missing dash ("-") in line 56. Could you please check ?

@ardalis Can you merge the change from the branch to this PR, please? The chapter isn't reflecting in the internal preview site.

merged previously accepted PRs into this branch so fixes already dealt with are here.

Chapters 2-3 in this PR should be ready to merge.

@mairaw @mjrousos @scottaddie Are you good with the changes?

@mairaw @scottaddie :shipit:  ?


refactor: extract _iterate_lines_to_cache

Codecov Report
Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (5fc14fc) 87.39% compared to head (c24246c) 87.46%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #316      +/-   ##
==========================================
+ Coverage   87.39%   87.46%   +0.07%
==========================================
  Files          19       19
  Lines        1039     1045       +6
==========================================
+ Hits          908      914       +6
  Misses        131      131

Files
Coverage Δ

seagoat/sources/ripgrep.py
88.54% <87.50%> (+0.76%)
:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Run Functional Tests using NUnit

The Deploy TestAgent task succeeds but when running the RFT task, all tests remain Pending. I see a lot of IEDriver and Internet Explorer processes being spawned at the Server but no tests seem to actually run? See the RFT task configuration in attached screenshot.
2017-07-28T10:17:19.2036651Z ==============================================================================
2017-07-28T10:17:19.2036651Z Task         : Run Functional Tests
2017-07-28T10:17:19.2036651Z Description  : Run Coded UI/Selenium/Functional tests on a set of machines (using Test Agent)
2017-07-28T10:17:19.2036651Z Version      : 1.0.52
2017-07-28T10:17:19.2036651Z Author       : Microsoft Corporation
2017-07-28T10:17:19.2036651Z Help         : More Information
2017-07-28T10:17:19.2036651Z ==============================================================================
2017-07-28T10:17:19.2036651Z Preparing task execution handler.
2017-07-28T10:17:19.5630415Z Executing the powershell script: F:\agent01_work_tasks\RunVisualStudioTestsusingTestAgent_d353d6a2-e361-4a8f-8d8c-123bebb71028\1.0.52\RunDistributedTests.ps1
2017-07-28T10:17:20.7036636Z DistributedTests: Environment WinRm Protocol HTTP.
2017-07-28T10:17:20.7192886Z DistributedTests: Creating run for selected test assemblies with following parameters
2017-07-28T10:17:20.7192886Z DistributedTests: SourceFilter: CustomerTestAutomation.dll TestCaseFilter:
2017-07-28T10:17:20.7192886Z DistributedTests: Run title: TestRun_Customer-CI_Release-137
2017-07-28T10:17:20.7192886Z DistributedTests: is automated: True
2017-07-28T10:17:20.7192886Z DistributedTests: build location: F:\Test\customer
2017-07-28T10:17:20.7192886Z DistributedTests: build id: 5477
2017-07-28T10:17:20.7192886Z DistributedTests: test configuration mapping:
2017-07-28T10:17:22.1411706Z DistributedTests: Test Run with Id 364 Queued
2017-07-28T10:23:56.9235570Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:56.9235570Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:02.8287490Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:02.8287490Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:12.9068955Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:12.9068955Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:23.0618427Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:23.0618427Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:33.2024870Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:33.2024870Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:43.4056431Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:43.4056431Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:18:53.5462858Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:18:53.5462858Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:03.6400600Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:03.6400600Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:13.7494393Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:13.7494393Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:23.8432336Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:23.8432336Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:33.9369803Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:33.9369803Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:44.0151279Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:44.0151279Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:19:54.1088965Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:19:54.1088965Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:04.2026653Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:14.2964406Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:14.2964406Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:14.2964406Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:24.3902078Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:24.3902078Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:34.4839754Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:34.4839754Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:44.8746320Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:44.8746320Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:20:54.9527647Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:20:54.9527647Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:05.0934102Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:05.0934102Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:15.1888829Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:15.1888829Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:25.2826357Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:25.2826357Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:35.3607896Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:35.3607896Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:45.6420502Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:45.6420502Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:21:55.5327094Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:21:55.5327094Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:05.6264709Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:15.7202588Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:15.7202588Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:15.7202588Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:25.8296340Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:25.8296340Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:35.9546474Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:35.9546474Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:46.0327934Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:46.0327934Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:22:56.1109389Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:22:56.1109389Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:06.2203316Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:06.2203316Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:16.3922359Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:16.3922359Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:26.5172468Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:26.5172468Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:36.6578869Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:36.6578869Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:46.7516616Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:46.7516616Z DistributedTests: Total Tests : 36, Passed Tests : 0
2017-07-28T10:23:56.9235570Z DistributedTests: Test run '364' is in 'InProgress' state.
2017-07-28T10:23:56.9235570Z DistributedTests: Total Tests : 36, Passed Tests : 0

@nigurr @kaadhina
Here are the corresponding DTA logs (with sensitive information obscured):
DTAExecutionHost.exe.log.txt

@ygeelen-tb Can you attache the build log? Is the task failed with some error? I do see that tests are getting executed and few of the tests failed and updated the test run with the same information.

In fact I had to cancel the Release because it seemed to be hanging..

RFTlog.txt

@ygeelen-tb From the logs, everything seems to be working fine.
From Test Agent machine:
DTAExecutionHost.exe Information: 0 : Publish.Run : Publishing to tcm finished
DateTime=2017-07-28T10:27:46.9672582Z
From Release log:
2017-07-28T10:24:07.2360815Z ##[error]The operation was canceled.
You seems to cancelled the operation before even tests are completed.

@nigurr I don't think it's normal that the RFT task runs for almost 7 minutes without finishing any test? They all stayed in Pending status, that's why it was cancelled.

@ygeelen-tb Currently it's limitation that it will report at the end of the test run. We have improved to live reporting using VsTest v2 - https://blogs.msdn.microsoft.com/devops/2017/03/26/vstest-task-dons-a-new-avatar-testing-with-unified-agents-and-phases/

@nigurr Does the VsTest v2 task replace the Run Functional Tests task? Does it support Selenium browser tests as well?
When waiting for 25+ minutes I managed to see the results (it's the same for all tests):
OpenQA.Selenium.WebDriverException : Unexpected error launching Internet Explorer. IELaunchURL() returned HRESULT 80070012 ('There are no more files.') for URL 'http://localhost:23946/'
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(ICommandExecutor commandExecutor, ICapabilities desiredCapabilities)
at OpenQA.Selenium.IE.InternetExplorerDriver..ctor(InternetExplorerDriverService service, InternetExplorerOptions options)
at MyCustomerTestAutomation.AuthenticationSteps.GivenIAmOnTheAuthenticationpage() in C:\agent_work\120\s\MyCustomerTestAutomation\Steps\AuthenticationSteps.cs:line 34
at lambda_method(Closure , IContextManager )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(IContextManager contextManager, StepInstance stepInstance)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
at MyCustomerTestAutomation.Features.AuthenticationFeature.ScenarioCleanup()
at MyCustomerTestAutomation.Features.AuthenticationFeature.AuthenticationFailed(String username, String password, String[] exampleTags) in C:\agent_work\120\s\MyCustomerTestAutomation\Features\Authentication.feature:line 23
--TearDown
at MyCustomerTestAutomation.AuthenticationSteps.DisposeWebDriver() in C:\agent_work\120\s\MyCustomerTestAutomation\Steps\AuthenticationSteps.cs:line 108
at lambda_method(Closure , IContextManager )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireScenarioEvents(HookType bindingEvent)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnScenarioEnd()
at TechTalk.SpecFlow.TestRunner.OnScenarioEnd()
at MyCustomerTestAutomation.Features.AuthenticationFeature.ScenarioTearDown()

@ygeelen-tb Yes. It will be used for running running unit tests as well as functional tests. The error seems to be very specific to Selenium/Driver implementation
https://github.com/SeleniumHQ/selenium/issues/4135


Standalone Flash executable

Hi all,
Is there a way to generate a standalone binary for the flash portion of PyOCD? It looks like in the how_to_build.md there are instructions for building a standalone executable but I see that it is outdated, as the option for building the flash tool on its own has been removed.
We've adopted the DAP-Link project as our debugger/programmer of choice in our Engineering team and my use case is as follows: Our hardware engineers would like to be able to take a .hex file given to them and flash a target at their bench without requiring too much setup (This is mostly a severe allergic reaction to having to open a CL :roll_eyes:). The same goes for our clients who are not as technically adept, but would like to be able to update their devices without sending them back to our office.
I would like to create a GUI application that will build the latest PyOCD Flash tool and package it along side our application. (Think J-Flash for the J-Link suite of tools). That way we can send a DAP-Link probe, Installer/Executable and their new Firmware and the end user is able to update without too much hassle.
Is this an existing feature or is it a potential new feature that could be added (again)?
Thanks!

Whew, really sorry it took so long to get back to this one! 🙇🏽
Having a GUI flasher is (somewhere) on the todo list. 😄 It's something I'd like to get to, but right now there are lots of other priorities for me. But I'd be super happy to held guide an effort.
Also on the list is releasing a standalone executable of pyocd. The how_to_build docs are unfortunately totally out of date. 😦  I have a work in progress branch feature/binary_build with updates to the pyInstaller script, but it's not working. There are issues with some of the dependencies that I didn't have time to finish debugging.
Another option for a binary executable is PyOxidizer, but I haven't tried it. No idea if it's better.

@flit no worries on the delay. I know your a busy person! I am more than happy to help and contribute to this. What did you have in mind when creating this GUI? Did you intend to ship it with the pyocd utility or have it has a separate project/repo and then the pyocd executable is a dependency?
My idea was the latter. We have been on an ElectronJS kick lately and created quite a few applications with standalone executables as binaries so this wouldn't be much of a stretch to get SOMETHING working.
I'm realizing as I type this out though that I would need to either duplicate your effort on the Pack manager or instead generate a standalone .exe for the entire PyOCD tool and then put a GUI front-end on it instead of the "Flash" portion only.
Happy to hear your thoughts and help out.

Having it be a separate project is probably best, but I'd be happy to host it within the pyocd GitHub org.
Personally, I was thinking it would be easiest to use PyQt5 or PySide to build the UI, so it stays entirely Python. Then use PyInstaller or PyOxidizer to build a standalone executable. Not sure how realistic the executable is, though, I haven't even been able to build pyocd itself as a standalone executable lately (I'm sure it's possible, just haven't had time to follow through debugging), and the Qt libs will only add complication.
So an Electron based project might be easiest, especially if you're already familiar! If you can't put the time into building pyocd as an executable, something like miniconda might work. That's how pyocd is packaged for Mbed Studio. That would also cover the Pack manager.
One question is how to communicate with the pyocd process. It's perfectly possible to just parse the current stdout+stderr output, and that should be the starting point. But for a long time I've wanted a better machine interface, something like a simple JSON Lines based interface over stdio. This would obviously be a long term goal, however.

I'll take a look at miniconda and/or generating a standalone binary. That's the only real hurdle to getting a minimal viable product.
Our current implementations are just event listeners on the 'std' pipes. Based on our current "state/operation" we know what to be parsing & looking for in the output strings. Creating a standard interface over stdout/err intended for communicating to other processes would be totally fine and we can definitely look into it. I would agree it's much more robust and is the "correct" way to go about this type of problem in the long run.

The feature/binary_build branch on my fork is as far as I got with PyInstaller. IIRC, the last? hurdle was prettytable for some reason.
Yeah, I also had never heard of the JSON Lines project/spec until very recently even though I've used it (more or less) in pyocd for the remote probe server/client.

Any news on this? I want to make an standalone programmer for the py32f002. The idea is to make a port of the arduino core. Tks


[BUG] 'autotrain llm' does not save checkpoints in the project folder

Prerequisites

[X] I have read the documentation.
[X] I have checked other issues for similar problems.

Backend
Local
Interface Used
UI
CLI Command
autotrain llm --train --project-name ftLlama --model meta-llama/Llama-2-7b-hf --data-path . --peft --lr 2e-4 --batch-size 12 --epochs 5 --trainer sft
UI Screenshots & Parameters
No response
Error Logs
Checkpoints are not saved
Additional Information
I'm fine-tuning "meta-llama/Llama-2-7b-hf" on a remote GPU using autotrain llm, but after completing the training, the checkpoints are not being saved. The project folder contains files like autotrain-data, adapter-config, tokenizer, training-params, etc., but the checkpoints are missing.
Here's the content of training-params file:
{     "model": "meta-llama/Llama-2-7b-hf",     "project_name": "ftLlama",     "data_path": "ftLlama/autotrain-data",     "train_split": "train",     "valid_split": null,     "add_eos_token": true,     "block_size": 1024,     "model_max_length": 2048,     "padding": "right",     "trainer": "sft",     "use_flash_attention_2": false,     "log": "none",     "disable_gradient_checkpointing": false,     "logging_steps": -1,     "eval_strategy": "epoch",     "save_total_limit": 1,     "auto_find_batch_size": false,     "mixed_precision": null,     "lr": 0.0002,     "epochs": 5,     "batch_size": 12,     "warmup_ratio": 0.1,     "gradient_accumulation": 4,     "optimizer": "adamw_torch",     "scheduler": "linear",     "weight_decay": 0.0,     "max_grad_norm": 1.0,     "seed": 42,     "chat_template": null,     "quantization": "int4",     "target_modules": "all-linear",     "merge_adapter": false,     "peft": true,     "lora_r": 16,     "lora_alpha": 32,     "lora_dropout": 0.05,     "model_ref": null,     "dpo_beta": 0.1,     "max_prompt_length": 128,     "max_completion_length": null,     "prompt_text_column": "autotrain_prompt",     "text_column": "autotrain_text",     "rejected_text_column": "autotrain_rejected_text",     "push_to_hub": false,     "unsloth": false,     "distributed_backend": null }
Am I missing something or is this a bug?
This is the command I'm using:
autotrain llm --train --project-name ftLlama --model meta-llama/Llama-2-7b-hf --data-path . --peft --lr 2e-4 --batch-size 12 --epochs 5 --trainer sft

which checkpoints do you want to save? it saves only one checkpoint by default. you can change it by changing save_total_limit argument. llm doesnt use validation data.

information for all arguments are in docs btw: https://hf.co/docs/autotrain

I also tried the below code but it doesn't save checkpoints too.
from autotrain.trainers.clm.params import LLMTrainingParams
from autotrain.cli.utils import llm_munge_data
from autotrain.project import AutoTrainProject

params = LLMTrainingParams(
    model="meta-llama/Llama-2-7b-hf",
    project_name="autotrain-finetune-Llama",
    data_path=".",
    trainer="sft",
    push_to_hub=False,
    save_total_limit = 1,
    lr=2e-4,
    epochs=2,
    batch_size=16,
    peft=True,
)
params = llm_munge_data(params, local=True)
project = AutoTrainProject(params=params, backend="local")
project.create()

sorry, i overlooked and gave you incorrect response. in llm training validation data isnt used. so, checkpoints are not saved. ill take a look at this behaviour again and will come back to you.


Write ATS for TG requirement 46

Style shall be mapped to the wms:Style element. The humanreadable name shall be mapped to the wms:Title element and the Unique Identifier shall be mapped to the wms:Name element.

Agreed


NS_ERROR_FAILURE in Firefox

I have somewhat the same problem with the other guy.
I am using version 3.1.1
I get an NS_ERROR_FAILURE error on line 361 in the unminified version of the file. This only
happens in Firefox only and not in Chrome.
...
    // Force updating CTM
    this.updateCTMOnNextFrame()

    this.options.svg.removeAttribute('viewBox')
} else {
    var bBox = this.viewport.getBBox(); // <-- error is triggered here.

    // Cache viewbox sizes
    this.viewBox.x = bBox.x;
    this.viewBox.y = bBox.y;
    this.viewBox.width = bBox.width
    this.viewBox.height = bBox.height
...

This fiddle might be a clue on what is going on.

Thanks for this bug report but from what I can see it is not related to svg-pan-zoom library.
From what I found there are similar reports on bugzilla.mozilla.org. Some of them are related to this error when element is not displayed display="none". Now I don't see why you would wont to have your viewport hidden. The sole reason of viewport is to interact with it.
Maybe you want to achieve something specific that may be done in some other way?
p.s. You should be able to safely update to 3.1.3 (3.1.2 contains a bug fix for Firefox). There are no breaking changes from 3.1.1 to 3.1.3

After further investigation, you are right. I styled the root attribute to overflow:hidden; and it is probably the cause of this error.


Run code Error ：Can't find class

when i complier the project. In class AlwaysSameSpeakerIDLabeler and AlwaysSameSpeakerIDLabeler have an error.
show
package com.google.audio can't find class SpeakerIDInfo

I use the android studio under windows, I found the SpeakerIdInfo.java code generated by the protocol buffer compiler, but automatically generate the Class SpeakerIdInfo's "D" to lowercase.

Thanks! Some folks on my team observed the same thing when they dug into this. I think they have a fix on the way.

This should be fixed. Please verify.


Roadmap

Now that 0.4.0 is released to PyPI and the most pressing matters are resolved, we can look ahead to the future and get a stable version published.
There is a milestone for the 1.0 release and any issues that must be resolved before that release are attached to it.
Note that jumping from the 0.x.y versioning scheme to 1.0.z allows for introducing breaking changes. The breaking changes towards 1.0 will likely require updating your code, but we should aim to do these as soon as possible if they are needed to get them out of the way.
Breaking changes after that will require a bump to 2.0, 3.0 etc.. Dropping support for no-longer maintained Django versions technically qualifies as a breaking change, so we will handle them as such. Those changes typically won't require you updating your code. If, however, we do introduce future breaking changes, they will be clearly communicated.
Now, for the roadmap - these are roughly the goals/reworks that must be completed before we can publish 1.0

Solving the Javascript integration properly (#15, #49). In short, there should probably be a move towards a JSON view for async fetching of the cookie groups/configuration and rewrite the cookiebar JS (and supporting JS) to handle this. Inline JS is annoying in general in combination with Content-Security-Policy too, so in general this should be a good move.
Scripting cleanup and tooling: ideally, people should be able to drop the shipped scripts/modules in Webpack or their bundler/toolchain of choice. I'd also like to explore <script type="module" ... /> to support modern browsers. This may not drop support for simple staticfiles integration and will still be available. We may set up a bit more of a toolchain for JS (and possibly look into Typescript so we can also ship types for users on a typescript toolchain).
Fixture support (for moving between staging/prod environment) via natural primary key #103
Good defaults but extensible architecture - see #34

@sergei-maertens are there people actively working on this project ?

@sergei-maertens are there people actively working on this project ?

I definitely am! Of course I'm distributing my time over a couple of projects, so my development work is typically in bursts

That is great to hear, I am using it on production on a project.
Are there any area of the roadmap that you need help with ?

I think a critical look at the docs wouldn't hurt and checking if we can
clean up the implementation of backend code.
The Big Thing is really the JS support at the moment which I'd like to sort
out myself, but I can always use people to test it out!
On Thu, 21 Sep 2023, 17:37 Fabien Le Frapper, @.***>
wrote:

That is great to hear, I am using it on production on a project.
Are there any area of the roadmap that you need help with ?
—
Reply to this email directly, view it on GitHub
https://github.com/jazzband/django-cookie-consent/issues/98#issuecomment-1729833225,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABKDJVTYATHQKNDDEQ3JUSTX3RNMTANCNFSM6AAAAAAZCMVE44
.
You are receiving this because you were mentioned.Message ID:
@.***>

@fabienheureux I've been ticking off some items and that resulted in a couple more issues on the backlog with concrete tasks.
Either way, the good first issue filter has some easy-pickings, and the milestone issues also have some less-straightforward things if you're comfortable with that.
I will assign the issues to myself that are being covered with the JS rework so it's clear they're being worked on :) Feel free to pick up anything else if you'd like!


Dartium very slow

When I run this example of a Grid/Datasheet component https://github.com/bwu-dart/bwu_datagrid/blob/1656186567358b77fbe4567ed33277eae9e83273/example_/web/checkbox_row_select.html
it takes quite a while until the data area is rendered.
The column titles provide a context menu to show/hide columns. When I click a checkbox it takes 10-15 secs until the view is built and Dartium responds to clicks.
When I load the same example with Chrome there is no noticable delay at all.
Both loaded from pub serve (using WebStorm)

Dart VM version: 1.14.0-edge.64e71a0b2e74cbb8a2a840b18fa100007ec1c43f (Fri Nov 20 02:13:48 2015) on "linux_x64"

Dartium

Dart	1.13.0-edge.517a44348143fc0af59d6f3e49fc7b9a06e5a8f4

I just started again working again on this project (didn't do anything since about 6 weeks), therefore I can't tell what version exactly introduced this slowness.

With the switch to using JS Interop, things that are doing many calls back and forth could easily slow down compared to calling through C++. We've seen remarkably few performance issue with it so far, but you might have found one. Thanks for the repro.

I can't reproduce today (after update to newest bleeding edge).
I add another comment when I run into it again.
Feel free to close.

Sounds good to me.


Added fixes to check variable instance of JsFunction for issue 2013 fixes

Description
Added fixes to check if the provided object is an instance of JsFunction and then assign value and type accordingly.

Relevant Issues :  Retry Until failure leads to afterscenario not running #2013
Type of change :

[ ] New feature
[x] Bug fix for existing feature
[ ] Code quality improvement
[ ] Addition or Improvement of tests
[ ] Addition or Improvement of documentation

@bipin-k build failed

@ptrthomas I need your help in getting the context for the test failures as with the above changes on this PR, it is getting harder for me to debug and figure out the reasons for the test failures with below logs,
The value 'DynamicObject<JSFunction>@12be8021' cannot be passed from one context to another. The current context is 0x7502434d and the argument value originates from context 0x14cac35a.

sorry I can't help more here. welcome to my world :)
you can ignore this one for now as I plan to do some deep dive on the JS stuff shortly

@bipin-k I just made the long-pending changes to the JS engine, that hopefully solves a lot of the JS function problems. in case you have time, it would be great if you can re-look at #2013 - it may be fixed


Refactor join implementation to improve readability

Summary or solution description
Refactored join built-in method.
Part of an overall refactoring to unify code generator logic and built-in methods generation.
How to Reproduce
https://github.com/CityOfZion/neo3-boa/blob/c7168432e2dd2196178b1ee1525ea83641894c4a/boa3_test/test_sc/string_test/JoinStringMethodWithDictionary.py#L1-L8
https://github.com/CityOfZion/neo3-boa/blob/c7168432e2dd2196178b1ee1525ea83641894c4a/boa3_test/test_sc/string_test/JoinStringMethodWithSequence.py#L1-L8
Tests
https://github.com/CityOfZion/neo3-boa/blob/c7168432e2dd2196178b1ee1525ea83641894c4a/boa3_test/tests/compiler_tests/test_string.py#L1203-L1257
Platform:

OS: Windows 10 x64
Python version: Python 3.8

Task linked: CU-864eq2t23 Refactor join implementation to improve readability


Improve UX when using older Electron versions

Currently if an Electron version > 2 versions behind the current version is used, the service fails with 404 errors whilst attempting to download Chromedriver.
The bare minimum of effort here would be simply to explicitly mention this issue and relevant workarounds in the chromedriver configuration doc.
A better solution would be to have the service throw a specific error with a link to the service chromedriver configuration page for the case where 404s are received and the detected Electron version is > 2 behind the latest.
We can also improve upon the chromedriver manual configuration section of the docs by adding specific steps such as detailed here.
Update: After a bit more investigation I raised https://github.com/webdriverio/webdriverio/issues/12630 to handle this  -- the service is no longer responsible for CD download and management so I think we deal with it in WDIO.

We decided to handle this via documentation (See PR #511 above).

Released in 6.4.1.


How do nodes join cluster?

Hello Verdigris,
First, thanks for writing this cookbook. Indeed a great help.
I"m  trying to get my head across how do a node join cluster?
The cluster recipe only runs when there is a change in cookie. Also, when the node is first launch. However, since sever list is populated at run time and it's only happening at line 101, no template change is happening. It is only restarting the node and resetting it, making it fresh installation.
That explain why /etc/rabbitmq/rabbitmq.config file  has cluster node list empty. However, surprisingly when i run rabbitmqctl cluster_status it shows me all the connecting nodes. I cannot figure out how did it happen. Please help me, if you could recall anything anyhow
Setup info:
env: aws
cluster: true
layer_name:

Hi can you brief me where to set the cluster true flag in opswork is it an environment variable? thanks


Build Pipeline: CI:trigger-build-everything does not consider changes on master

Bug Description
The CI:trigger-build-everything is an important flag to tell the build pipeline to build every artifact from a specific branch/PR which can then be used during integration tests. Artifacts that went through code changes are freshly bulid, all other artifacts are just retagged. However, changes made on the master branch are not considered (even if you do a git rebase).
Workaround
In order to make sure that every artifact is up to date with the current state of the master branch, a new PR needs to be created.

Fixed in #5943


'capture region of the screen' not working

'capture region of the screen' not working

It should be fixed. Please test the last version.

Thanks
On Wed, Feb 14, 2018 at 12:37 PM, Aminadav Glickshtein <
<EMAIL_ADDRESS>wrote:

It should be fixed. Please test the last version.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Aminadav/screenshot-extension/issues/54#issuecomment-365685491,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AXomRPDimbljbKI4lvfHD5GKxlplFpB-ks5tUxnbgaJpZM4LV7QE
.


changed out IE8 for IE9 in home page and feature page fixes issue #899

https://github.com/donejs/donejs/issues/899

Coverage remained the same at 96.396% when pulling d8b713dc22121454a6d24d618bd85d01319d0f9e on GeneralMeow:master into 884284e8c2eb412f73de7ae9195cff7822d3f859 on donejs:master.

Coverage remained the same at 96.396% when pulling 2fa9f397b2fb8e1edb1ae82e948ea32041c00957 on GeneralMeow:master into 884284e8c2eb412f73de7ae9195cff7822d3f859 on donejs:master.


#5187 Move all framework Shortcode Providers to the same namespace.

Current coverage is 70.24% (diff: 83.78%)

Merging #6126 into master will increase coverage by <.01%

@@             master      #6126   diff @@
==========================================
  Files           416        418     +2
  Lines         35711      35715     +4
  Methods        4688       4690     +2
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          25083      25087     +4
  Misses        10628      10628
  Partials          0          0

Powered by Codecov. Last update 7663102...e94e7bf

This code looks fine to me!

I'll be adding commits, so it's easier to track the changes, until it's a go, then squash it. That okay with you folks?

@sminnee I don't mind the new namespace. Is SilverStripe\View\Parsers ok with you? If so I'll merge.

Have rebased and squashed into a single commit. Separately, the tests are fine. Unsure of anything I rebased on broke other tests though.

@tractorcow It's not a "new" namespace. It's the same namespace as where the ShortcodeParser Interface resides.

My concern here is that this makes the assets module a dependency of the view module, which doesn't make sense to me.

tl;dr: It makes OembedProvider a bit a hovering orphan that needs a good namespace.
I understand and also partially agree with your concern @sminnee.
What I however wish to accomplish, is unity. Hence also the move of the Oembed provider to the same namespace.
If we put the File and Image providers in Assets\ViewSupport, I think the Oembed one, should also move to a similar namespace instead of HTMLEditor\Form.
I'm happy with the ViewSupport namespace, but unclear on how to make it consistent across all providers (same goes for CMS, which would probably then go to CMS\Model\ViewSupport?)

I'm a bit split on this but I'm learning on agreement with sam here. We should be placing parsers in namespaces relevant to the parser, not lumping them together in one.

I can live with that @dhensby, I'm trying to find a good concensus, where the View was my first thought on unifying.
I'm just still stuck on where the Oembed provider should go. As having a single Oembed\ViewSupport namespace for it, sounds silly to me.

You can leave the Oembed provider where it is if you want.

To unify, Do you mind if I make the Oembed SilverStripe\View\ViewSupport and create a folder for it like that?
I want the end namespace to be as uniform as possible, to avoid confusion.

@Firesphere just do it :) Let's get this merged

Agreed and fixed your change request @sminnee

needs rebasing

Will do when I have time tomorrow or this weekend :)

I completely lost sight of this PR, have rebased but it seems there are still merge conflicts?

@Firesphere Any progress here?

Closing. Reopened as:

https://github.com/silverstripe/silverstripe-assets/pull/40
https://github.com/silverstripe/silverstripe-framework/pull/7085


Zwave set temperature fix

On a simple zwave thermostat, when the device was in off state set_temperature did not work.

Do you want to set setpoint temperature when when thermostat is off?
Follow up question: Can you set the temperature on the physical device when it is off?

The thermostat I have is the Horstmann hrt4zw and it always seems to be reporting that it is off. It is just a simple thermostat that you can set the setpoint at anytime on the physical device and will call for heat when needed.
So as far as I can see you need to set the setpoint when it is off and you can always change the setpoint on the physical device.

Ah, I see. This change will also affect any other device that offers the operating_mode off. And allow them to set a setpoint in off, and we don't know which setpoint is being set.
I would rather see a workaround code for this specific device, using like with the zxt_120.
if hrt4zw and current_operation == 'Off':

This should be under the statement you have changed.

Thanks for the feedback, I don't have any experience with other devices so did not know the repercussions of my changes. I agree that it should be done as a workaround and have now implemented as such.
It seems to be failing the CI build but on an unrelated error:
tests/test_remote.py::TestRemoteClasses::test_statemachine_remove_from_slave FAILED

@turbokongen Giving you final say on this, looks good to me.


Grouping Fields

Hey,
I read a lot of issues asking this but all were closed in the end.
Is there a way of grouping fields inside a div container? It's really hard to align fields next to each other in case you are using a horizontal form for example without any support for grouping which is crucial for bootstrap forms...
Basically this: http://stackoverflow.com/questions/12201835/form-inline-inside-a-form-horizontal-in-twitter-bootstrap

No plans to bake anything like this into the package. It's really meant just to speed up quick prototyping stuff where you aren't trying to make your forms look at all fancy.
If you need anything different than what comes out of the box, you're really way better off just writing the markup instead of using the package :+1:


tests/api_connexion/endpoints/test_dataset_endpoint.py ~18 Failed

What happened?

The query parameter validator of Connexion v3 doesn't work properly on the dataset/events endpoint. It raises extra query parameter errors even for parameters registered in open API(v1.yaml file). (I believe the v1.yaml file is set correctly.)
Here is the source code of Connexion.
@potiuk

Fixed by https://github.com/sudiptob2/airflow/pull/94


Remove timestamp examples

What were you trying to do?
Cedar docs examples used timestamp. We should remove the timestamp because it is not supported.
What is wrong and why?
Cedar currently does not support timestamp.  The following example does not work with Cedar.
when {
  context.time.now < timestamp("2022-01-01T10:15:00.021-05:00")
}

What do we need to do to fix this?
Remove the timestamp references.

Can you just make this change and submit it as a PR?

Is there any way to compare the time, or should I just remove that example altogether?

Thanks! Fixed in PR 59.


feat(GODT-1643): RFC5322 Parser in C++

Rewrite of https://github.com/ProtonMail/go-rfc5322 using C++.
We also add the d postfix to debug builds to avoid having debug binaries linked into go.

I'd like you to also include the XML test set from the go-rfc5322 package here. They were very valuable. Should be simple enough to test them from the go side.


Running pods panel in Global dashboard

Currently, "Running Pods" panel uses the expression sum(kube_pod_container_info), which sums the containers, but not the pods. I believe the metric kube_pod_info would be the best for this panel.
Should be updated here: https://github.com/dotdc/grafana-dashboards-kubernetes/blob/793cb68b3c8474dca19f686c63e11f96b01f13b4/dashboards/k8s-views-global.json#L868-L881
P.S. Thank you for the dashboards, they look awesome!

It's fixed, thank you @mashun4ek for reporting the issue!
Glad to see the dashboards are usefull :wink:

:tada: This issue has been resolved in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


CI: Missing KIND features to have the same coverage as Vagrant

This issue tracks all shortcomings of KIND which contribute to lower test coverage of the CI ginkgo suite when compared to when running on Vagrant.

[x] XDP hack for veth ifaces (described in https://github.com/cilium/cilium/blob/master/test/l4lb/test.sh#L12).
[ ] Add secondary iface to KIND network, so that we can test the NodePort via secondary iface.

Add secondary iface to KIND network, so that we can test the NodePort via secondary iface.

We'll also need the secondary iface to exercise the FIB lookup path when coming from bpf_lxc:
https://github.com/cilium/cilium/blob/c11c9f7b68292b09d32be6f577895677a424acfb/bpf/bpf_lxc.c#L1111

XDP hack has been in place for a while now, unassigning myself.

Add secondary iface to KIND network, so that we can test the NodePort via secondary iface.

Can we add secondary iface like this and test the NodePort via secondary iface on it?
docker network create -d=bridge kind-cilium-secondary
NODES=$(docker ps -a --filter label=io.x-k8s.kind.cluster=kind --format {{.Names}})
for NODE in $NODES; do
  docker network connect kind-cilium-secondary $NODE
done

docker exec kind-worker ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
9: eth0@if10: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:12:00:04 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fc00:c111::4/64 scope global nodad
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe12:4/64 scope link
       valid_lft forever preferred_lft forever
20: eth1@if21: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:13:00:05 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe13:5/64 scope link
       valid_lft forever preferred_lft forever

@ysksuzuki Yep! PR please :-)

Now we've got the secondary network in the kind network. Can we move the N/S load balancing test to the kind-based test?

@ysksuzuki Good point! Let me close this issue. We will do the coordination of moving in a separate issue.


Unable to open database file: out of memory (14) - in windows (wails) app

Error creating table: "unable to open database file: out of memory (14)":
CREATE TABLE IF NOT EXISTS test
Any suggestion how to resolve this?

Prod build requires a different paths to access local files.


Update sap-destination-with-rest.md

@MarkvanMents would be good to update the shared glossary with the feedback in these PRs.


Error

If i try to type #bump i saw error
(node:12816) UnhandledPromiseRejectionWarning: ReferenceError: rawEmb is not defined
at bump (Z:\partnership-help-reload\commands\bump.js:88:15)
(Use node --trace-warnings ... to show where the warning was created)
(node:12816) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag --unhandled-rejections=strict (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:12816) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
How to fix it??

oh yeah some files were´t the correct one, sry ^^" it is fixxed now ^^"

Thx :)


Module '"@remix-run/node"' has no exported member 'Response'

Describe the bug
I copy pasted the code on the README and got this error.
Module '"@remix-run/node"' has no exported member 'Response'.ts(2305)
Your Example Website or App
local development
Steps to Reproduce the Bug or Issue
Copy past the codes on readme on Remix 2 tutorial app.

Error

Expected behavior
Response to be exported. There are response like types are exported but they are not classes so new Response doesn't workç
Screenshots or Videos
No response
Platform

OS: Windows
Browser: Chrome
Version: "remix-i18next": "^5.3.0",

Additional context
My package json:
"dependencies": {
"@prisma/client": "5.3.1",
"@remix-run/node": "2.0.0",
"@remix-run/react": "2.0.0",
"@remix-run/serve": "2.0.0",
"i18next": "^23.5.1",
"i18next-browser-languagedetector": "^7.1.0",
"i18next-fs-backend": "^2.2.0",
"i18next-http-backend": "^2.2.2",
"isbot": "^3.6.13",
"match-sorter": "^6.3.1",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-i18next": "^13.2.2",
"remix-auth": "^3.5.1",
"remix-auth-form": "^1.3.0",
"remix-i18next": "^5.3.0",
"sort-by": "^1.2.0",
"tiny-invariant": "^1.3.1"
},
"devDependencies": {
"@remix-run/dev": "2.0.0",
"@remix-run/eslint-config": "2.0.0",
"@types/react": "^18.2.20",
"@types/react-dom": "^18.2.7",
"eslint": "^8.47.0",
"eslint-plugin-unused-imports": "^3.0.0",
"prettier": "^3.0.3",
"prisma": "^5.3.1",
"typescript": "^5.1.6"
}

Remix 2 documents changes needed here
https://remix.run/docs/en/main/start/v2#removal-of-exported-polyfills
Remove "import Response ..." from entry.server.tsx and add "import { renderToPipeableStream } from 'react-dom/server';"
create a variable stream like so const stream = createReadableStreamFromReadable(body); and use that object instead of body

let body = new PassThrough();
const stream = createReadableStreamFromReadable(body);
responseHeaders.set('Content-Type', 'text/html');

resolve(
  new Response(stream, {
    headers: responseHeaders,
    status: didError ? 500 : responseStatusCode,
  })
);

@yukosgiti it would be nice to update the README...


Component instances are destroyed before a transition leave done callback is called

Vue.js / vue-router versions
2.2.6 / 2.3.1
Reproduction Link
https://jsfiddle.net/pyh4uzxb/3/
Steps to reproduce

Wrap a transition component around the router-view with an attribute leave="leaveFn"
Call the done method after a few seconds
Observe that the DOM with components is still rendered but the component instances are already destroyed.

What is Expected?
The component would still be instantiated.
What is actually happening?
The component is still rendered in the DOM but the component instances are already destroyed.

This is actually how Vue works (it happens without vue-router), the DOM element is still there for the transition but it's not reactive anymore since there's no Vue instance anymore


Do another crawling with data from the last one

Hey !
I would like to get some data in an array from a page A (scraped with cheerio), return an array of url to populate the next queue for a completely different crawler.
But it's look like the second crawler get triggered before the first one occur.
I try to add .on('drain') to lunch the second crawling but it doesn't seem to work.
Any idea on this ?

@gfelot Can you attach your code here so I can better understand your situation? In most cases we don't declare two crawler instances.

Close due to inactive more than a month

Hey !
I would like to get some data in an array from a page A (scraped with cheerio), return an array of url to populate the next queue for a completely different crawler.
But it's look like the second crawler get triggered before the first one occur.
I try to add .on('drain') to lunch the second crawling but it doesn't seem to work.
Any idea on this ?

You can do like this
const c = new Crawler({     maxConnections: 10,     // This will be called for each crawled page     callback: function (error, res, done) {         if (error) {             console.log(error);         } else {             const $ = res.$;             // $ is Cheerio by default             //a lean implementation of core jQuery designed specifically for the server             const tags = $('div.group-list a');             const links = tags                 .map((index, el) => ({                     uri: ${res.request.uri.protocol}//${res.request.host}${el.attribs.href},                     callback: function (error, res, done) {                         if (error) {                             console.log(error);                         } else {                             const $ = res.$;                             const content = $('div.content p:nth-child(1)').html();                             data.push({ content });                         }                         done();                     }                 }                 ))                 .toArray();             c.queue(links);         }         done();     } });


add noop catch for duplicate module paths

fixes #1208
catching and noop seems to work fine for this error; not sure if there is a way to check for an existing model; will spend a little more time combing through monaco api but this solution works for now

Found a getModel check we can safely use. Pushing commit now for that instead

👍🏻 great!


File input  - Invariant Violation

When I am trying to use library to add file to Form I get:

Invariant Violation: A state mutation was detected between dispatches, in the path invoice.invoice_file.0.lastModifiedDate. This may cause incorrect behavior.

The Field for the file looks like this:
<Field className="btn" model="invoice.invoice_file">
            Plik z fakturą
            <input type="file" name="invoice_file" />
  </Field>

Also couldn't find anything about fileInput in the documentation. It would be nice if we could get an example.
Also I was trying to get this going with custom handler:

            Plik z fakturą
            <input type="file" name="invoice_file" onChange={this.handleFileUpload}/>

...
  handleFileUpload(e){
    e.preventDefault();
    let file = e.target.files[0];
    this.props.mergeModel('invoice.invoice_file', file);
  }

...

function mapDispatchToProps(dispatch) {
  return {
    saveInvoice: bindActionCreators(saveInvoice, dispatch),
    changeModel: bindActionCreators(actions.change, dispatch),
    mergeModel: bindActionCreators(actions.merge, dispatch)
  };
}

But this also causes Invariant Violation error. Did anyone got this right with file input?

Can you put it up on www.esnextb.in with the minimally reproducible code?
Also, for reference, here is the unit test for <input type="file" />: https://github.com/davidkpiano/react-redux-form/blob/master/test/field-component-spec.js#L489-L521
I'm working on the V1 docs right now, and will definitely include this.

Ok, so I put this on esnextbin.in and it was alright. So I checked what are the differences between my project and esnextbin setup and I found out I was using redux-immutable-state-invariant
https://github.com/leoasis/redux-immutable-state-invariant
At that was causing the error to show in console.


chore: track script to make local test images

Can be used for local development. Might be worth integrating into npm commands.
Remembering to start the local registry before running tests locally is cumbersome. Here's a shell script, copied essential from the docs, to do it for us.
Questions:

Should this script be instead moved to an npm target?
Should this script be extracted from test setup into its own task?
How should we handle failure modes, e.g. where there's already a container named registry or a port is in use?
Should we totally dispense with the local registry and just push test images to a container registry somewhere?

PR is a conversation starter, in other words. But this script has been kicking around locally for a while and I use it, so I wanted to share.

Should we totally dispense with the local registry and just push test images to a container registry somewhere?

this would require that local development requires non local resources for testing, which isn't ideal

Should this script be instead moved to an npm target?

yes -- I think the idiomatic way to deal with stuff like this is as a new script entry here
https://github.com/anchore/sbom-action/blob/72370e18af3add17e587ca8533fab7d28d2b0bee/package.json#L7

Should this script be extracted from test setup into its own task?

can it be both? That is, have the test automatically set it up if its not already running, or use an already running one?

How should we handle failure modes, e.g. where there's already a container named registry or a port is in use?

My vote would be to name it sbom-action-registry with a unique-enough static port (so probably not 5000) and always use it (if already created then use that)

Questions:

Should this script be instead moved to an npm target?

This should be integrated into the tests so running npm test does any required setup.

Should this script be extracted from test setup into its own task?

No, it should be integrated with the tests.

How should we handle failure modes, e.g. where there's already a container named registry or a port is in use?

Fail the tests.

Should we totally dispense with the local registry and just push test images to a container registry somewhere?

This would be a fine solution, too -- it could be a GHCR package linked to this repo.

It sounds like consensus is to move startup code from
https://github.com/anchore/sbom-action/blob/72370e18af3add17e587ca8533fab7d28d2b0bee/.github/workflows/test.yml#L68-L73
to a target in https://github.com/anchore/sbom-action/blob/72370e18af3add17e587ca8533fab7d28d2b0bee/package.json#L7
It might be nice to go further and make npm test call the new functionality, but that has a bit of difficulty:

A developer wants to run npm test on every code change
A developer needs to run npm run start-test-registry (or whatever we call it) only if the images change / once per session, which is way less often than item 1.

Therefore, for  npm test to also start the test registry, we'd need to make starting the test registry idempotent, which in turn would be annoying because then someone would have to restart the test registry if they changed an image file.
I'll see how hard it is to make an npm run start-test-registry command idempotent, and failing that at least add it as a script and document it.

These images are not really ever changing, the main purpose was just to validate that the command is actually running and generating files and that the output isn't changing drastically. I strongly favor just pushing these images to GHCR and updating the tests to point to them.

I strongly favor just pushing these images to GHCR and updating the tests to point to them.

This is the way, but that's a totally different approach, so just closing this for now.


update vendor for k8s default version

Closing in favor of https://github.com/rancher/rancher/pull/15642


Implement Focus Timer

Implement focus timer that plays music.
Focus timer should have the following features:

Ability to choose length of focus time and breaks
Ability to choose number of focus periods

Created focus timer. Functionality handled within timer.html template. Had to use script tag within the file for javascript.


Why item node should contain only block objects?

https://github.com/GitbookIO/slate-edit-list/blob/master/lib/validation/schema.js#L14
Why not inlines?
nodes: [{ objects: ['block', 'inline'] }],

I can make a pr with this, if it is ok.

Yes, I think it's a good practice to have blocks that can either contain other blocks, or contain inlines/texts, but never a mix of these. Especially since a default rule in Slate is for blocks to never contain both blocks and inlines at any given time.


Debug/fix filelock test on macos

The following test sometimes fails on macos

def test_update_well_metadata_concurrency(
    tmp_path: Path,
    testdata_path: Path,
    monkeypatch,
):
    """
    Run _update_well_metadata in parallel for adding N>1 new images to a given
    well. We artificially slow down each call by INTERVAL seconds, and verify
    that the test takes at least N x INTERVAL seconds (since each call to
    `_update_well_metadata` is blocking).

    In the last section of the test, we verify that a timeout error is raised
    when the timeout is too short.
    """

    N = 4
    INTERVAL = 0.5

    # Copy a reference zarr into a temporary folder
    raw_zarrurl = (testdata_path / "plate_ones.zarr").as_posix()
    zarr_url = (tmp_path / "plate.zarr").resolve().as_posix()
    shutil.copytree(raw_zarrurl, zarr_url)

    # Artificially slow down `_update_well_metadata`
    import fractal_tasks_core.tasks._zarr_utils

    def _slow_load_NgffWellMeta(*args, **kwargs):
        time.sleep(INTERVAL)
        return load_NgffWellMeta(*args, **kwargs)

    monkeypatch.setattr(
        fractal_tasks_core.tasks._zarr_utils,
        "load_NgffWellMeta",
        _slow_load_NgffWellMeta,
    )

    # Prepare parallel-execution argument list
    well_url = Path(zarr_url, "B/03").as_posix()
    list_args = [(well_url, "0", f"0_new_{suffix}") for suffix in range(N)]

    # Run `_update_well_metadata` N times
    time_start = time.perf_counter()
    executor = ProcessPoolExecutor()
    res_iter = executor.map(_star_update_well_metadata, list_args)
    list(res_iter)  # This is needed, to wait for all results.
    time_end = time.perf_counter()

    # Check that time was at least N*INTERVAL seconds
    assert (time_end - time_start) > N * INTERVAL

    # Check that all new images were added
    well_meta = load_NgffWellMeta(well_url)
    well_image_paths = [img.path for img in well_meta.well.images]
    debug(well_image_paths)
    assert well_image_paths == [
        "0",
        "0_new_0",
        "0_new_1",
        "0_new_2",
        "0_new_3",
    ]

    # Prepare parallel-execution argument list with short timeout
    well_url = Path(zarr_url, "B/03").as_posix()
    list_args = [
        (well_url, "0", f"0_new_{suffix}", INTERVAL / 100)
        for suffix in range(N, 2 * N)
    ]
    with pytest.raises(Timeout) as e:
        res_iter = executor.map(_star_update_well_metadata, list_args)
        list(res_iter)  # This is needed, to wait for all results.
    debug(e.value)

What a fun rabbithole, thanks for exploring & solving into this!


Segregate Julia notebook from Python/R notebook

The Julia dependencies are conflicting with the conda dependencies for both R and Python3. Users that use Julia for their assignments rarely need both Julia and R and only need the basic Python packages.

No longer relevant. These specific implementations should be handled in new end-user repos created from this template repo.


Flutter v1.12.2: Error: The method 'AdvancedNetworkImage.load' has fewer positional arguments than those of overridden method 'ImageProvider.load'.

Error:
[+15740 ms] [+16243 ms]
[   +4 ms]             Compiler message:
[   +1 ms] [   +2 ms]
../../../../AppData/Roaming/Pub/Cache/hosted/pub.dartlang.org/flutter_advanced_networkimage-0.5.0/lib/src/provider/flutt
er_advanced_networkimage.dart:143:24: Error: The method 'AdvancedNetworkImage.load' has fewer positional arguments than
those of overridden method 'ImageProvider.load'.
[   +4 ms] [        ]   ImageStreamCompleter load(AdvancedNetworkImage key) {
[   +3 ms] [        ]                        ^
[   +1 ms] [        ] /C:/flutter/packages/flutter/lib/src/painting/image_provider.dart:403:24: Context: This is the
overridden method ('load').
[   +3 ms] [        ]   ImageStreamCompleter load(T key, DecoderCallback decode);
[   +3 ms] [        ]

The updated overriden method:
@protected
  ImageStreamCompleter load(T key, DecoderCallback decode);

Flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel dev, v1.12.2, on Microsoft Windows [Versione 10.0.18362.476], locale it-IT)

[√] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[√] Chrome - develop for the web
[!] Android Studio (version 3.3)
    X Flutter plugin not installed; this adds Flutter specific functionality.
    X Dart plugin not installed; this adds Dart specific functionality.
[√] VS Code (version 1.40.1)
[√] Connected device (3 available)

! Doctor found issues in 1 category.

Try 0.6.0-alpha.1.

Can we keep this issue open until it is released?

@mchome 0.6.0-alpha didn't fix the issue
check it here
`Target dart2js failed: Exception: ../../../Development/flutter/.pub-cache/hosted/pub.dartlang.org/cached_network_image-1.1.3/lib/src/cached_network_image_provider.dart:42:24:
Error: The method 'CachedNetworkImageProvider.load' has fewer positional arguments than those of overridden method 'ImageProvider.load'.
ImageStreamCompleter load(CachedNetworkImageProvider key) {
^
../../../Development/flutter/packages/flutter/lib/src/painting/image_provider.dart:403:24:
Info: This is the overridden method ('load').
ImageStreamCompleter load(T key, DecoderCallback decode);
^
Error: Compilation failed.
`

我也是报相同的错误，但是用 @protected
ImageStreamCompleter load(T key, DecoderCallback decode);替代了@override
ImageStreamCompleter load(AdvancedNetworkImage key) {
return MultiFrameImageStreamCompleter(
codec: _loadAsync(key),
scale: key.scale,
informationCollector: () sync* {
yield DiagnosticsProperty('Image provider', this);
yield DiagnosticsProperty('Image key', key);
},
);
}这个方法还是报错。请问各位大佬有什么解决方法


Command options --current-user , --current-orgaization can not be set by username, name

OroPlatform 3.0-rc
In the ConsoleContextGlobalOptionsProvider filter_var is used with FILTER_VALIDATE_INT, this returns false if a string value is used.
$userId = filter_var($user, FILTER_VALIDATE_INT);
$organizationId = filter_var($organization, FILTER_VALIDATE_INT);

Was my fault, read the code wrong. Not an issue, can be closed.


Pre-built Docker images

Tool version
N/A
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
I wanted to deploy the solution and noticed that I need to build the Docker image myself. It would be awesome if this project could publish a Docker image to either Docker Hub or as a GitHub package so that users can easily deploy the container and keep up-to-date with the project. That would make it super easy for all the container scenarios to deploy the application.
Describe alternatives you've considered
N/A
Additional context
There is a complete description here that builds the image and pushes it as a GitHub artifact:  https://docs.github.com/en/actions/publishing-packages/publishing-docker-images#publishing-images-to-github-packages
Thank you!

This will be added to Backlog and considered for a future release.

Bryan

Just to provide an update, we determined that the GitHub org we are currently hosted in does not allow "Public" packages. Without that capability, we are unable to host a pre-built image for the tool. If that ever changes, we will definitely add this functionality in a future release.

Bryan

Thanks for looking into that Bryan! Is it possible to get them hosted at the Microsoft docker repo like this one? https://hub.docker.com/_/microsoft-dotnet-sdk/

We have looked into that, however, it would quire us hosting the tool in a different Org, along with other changes.  The Patterns & Practices org (our current home) gives us a lot of flexibility and autonomy, so we will be here for the foreseeable future.

Bryan

Does look like Microsoft has started publishing things to https://mcr.microsoft.com/en-us/catalog?alphaSort=asc&alphaSortKey=Name with the recent Docker Hub licence changes - might be a good place for a prebuilt image to end up?


Bodymovin plugin (Lottie animations)

enhancement
It think that would be great for having really good json animations if WebSlides add bodymovin as plugin, so we can use Lottie  animations
Great work, by the way!

@nelbu Thanks for your suggestion! We'll look into this further.


Error: undefined

On your live version it works great. However, when I download the code from your github. I'm getting plenty of errors.
My logs are stating that in search.php
!isset($response->listings[0]) is triggering, which means that looking for a listing in response is returning null
This only occurs when using scripts.js or the index.html page. When accessing the php file directly and echoing responses from there, everything is fine.

I reproduced the issue and pushed a fix.
The error occurred on the initial page load when no URL parameters were present. In this case the backend tried to query the field Featured_on_Website__c which is not part of the default configuration.
Thanks for letting me know!


After volume.balance moved a volume: stat idx file .idx failed

On Snapraid: Version 8000GB 1.77 linux amd64 (both machines involved)
During volume.balance, moved volumes' idx files became unrecognizable somehow:
> volume.balance -force
moving volume 2 <IP_ADDRESS>:8995 => <IP_ADDRESS>:8526
2020-05-17 04:15:49.992744 I | copying volume 2 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526
2020-05-17 04:20:24.665441 I | tailing volume 2 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526
2020-05-17 04:20:36.823121 I | deleting volume 2 from <IP_ADDRESS>:8995
2020-05-17 04:20:40.512024 I | moved volume 2 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526
moving volume 3 <IP_ADDRESS>:8995 => <IP_ADDRESS>:8526
2020-05-17 04:20:40.512072 I | copying volume 3 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526
2020-05-17 04:25:18.733374 I | tailing volume 3 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526
2020-05-17 04:25:30.890895 I | deleting volume 3 from <IP_ADDRESS>:8995
2020-05-17 04:25:32.675848 I | moved volume 3 from <IP_ADDRESS>:8995 to <IP_ADDRESS>:8526

A few minutes I notice that this was moved rather than replicated, set the respective replication to 010, and try to re-replicate (multiple times, to show all moved volumes were affected):
> volume.fix.replication
replicating volume 2 010 from <IP_ADDRESS>:8526 to dataNode <IP_ADDRESS>:8995 ...
error: rpc error: code = Unknown desc = stat idx file .idx failed, stat .idx: no such file or directory

> volume.fix.replication
replicating volume 3 010 from <IP_ADDRESS>:8526 to dataNode <IP_ADDRESS>:8995 ...
error: rpc error: code = Unknown desc = stat idx file .idx failed, stat .idx: no such file or directory

The odd thing is that if I look at the filesystem, these files are present and seem to have been moved when it was indicated:
-rw-r--r--+  1 seaweedfs seaweedfs<PHONE_NUMBER>8 May 17 04:20 2.dat
-rw-r--r--+  1 seaweedfs seaweedfs     2098344 May 17 04:20 2.idx
-rw-r--r--+  1 seaweedfs seaweedfs          60 May 17 04:43 2.vif
-rw-r--r--+  1 seaweedfs seaweedfs<PHONE_NUMBER>8 May 17 04:25 3.dat
-rw-r--r--+  1 seaweedfs seaweedfs     2139450 May 17 04:25 3.idx
-rw-r--r--+  1 seaweedfs seaweedfs          60 May 17 04:43 3.vif

Is there a way to make it use these .idx files? Also: Maybe seaweedfs should run a sanity check prior to deleting remote volumes?

I could not reproduce it, but I added some more error checking before copying files.

The server startup seems to be ok with loading the idx files:
I0518 07:27:19 81450 volume_loading.go:104] loading index /mnt/seaweedfs-data/3.idx to memory. Even volume.fsck doesn't mind the respective volumes.  It's apparently mainly volume.fix.replication and volume.balance -force?
I spotted two strange things though.

Shouldn't the messages actually say something like:
replicating volume 2 010 from <IP_ADDRESS>:8526 to dataNode <IP_ADDRESS>:8995 ...
error: rpc error: code = Unknown desc = stat idx file 2.idx failed, stat 2.idx: no such file or directory
or
replicating volume 3 010 from <IP_ADDRESS>:8526 to dataNode <IP_ADDRESS>:8995 ...
error: rpc error: code = Unknown desc = stat idx file 3.idx failed, stat 3.idx: no such file or directory

At least if I understand this correctly, it should have the idx file name, right?  But it missing from the error.

The mask for the ACLs on the respective files was wrong for unknown reasons:

> getfacl 2.idx
# owner: seaweedfs
# group: seaweedfs
user::rw-
user:seaweedfs:rwx              #effective:r--
group::r-x                      #effective:r--
group:seaweedfs:rwx             #effective:r--
mask::r--
other::r--

This doesn't match the default ACLs. Unfortunately changing it back to effective rwx doesn't actually make volume.fix.replication and volume.balance -force work again, even after restarting seaweedfs. But it is quite strange regardless. Maybe it caused issues at some point.


Distance searches

Due to a wide variety of reasons a common use of this package will be to find a closest match given a distance tolerance.
Maybe a macro. The idea is that the below would return a subset of ZIPCODES, or maybe some other dataframe, with a "distance" column appended to it.
# Return top 10 records of ZIPCODES with a tolerance of 50km
@distance(:zip .== "01234", 50.00, 10)

# Return furthest 1 record from the centroid of OH records
@distance(:state .== "OH", 1e10, -1)

# Return all records within 500km
@distance(:city .== "Los Angeles", 500)

If other dataframes are supported:
@distance(df, :timezone .== -4, 50.0, -5)

Could be interesting to combine this with DataFamesMeta.jl:
@linq ZIPCODES |>
     @where(:state .== "CA", :city .== "San Francisco") |>
     @distancefrom(TripLength = :zip .== "01234")

Alternatively (or in conjunction) there could be a standard function that performs a vector operation.
distance(coord, coords, tol=50, returntop=25)

Canceling this. There's a lot of overlap with much more mature solutions like DataFramesMeta.jl and Query.jl.