Insert causes :memory: not an error

Hi,
I was wondering if you see anything wrong with my code or maybe it's actually a bug somewhere?
#from main.nim
import sqliteral
import "helpers/database""

when not defined(release): db2.setLogger(proc(db2: SQLiteral, msg: string, code: int) = echo msg)
db1.openDatabase("main.db", SiteSectionsSchema)
db2.openDatabase(":memory:", UserSessionsSchema)

#from helpers/database
import sqliteral

const SiteSectionsSchema* = "CREATE TABLE IF NOT EXISTS SiteSections(id INTEGER PRIMARY KEY, title TEXT NOT NULL, link_title TEXT, nav_menu_inclusion TEXT, landing_type TEXT, thumbs_type TEXT, parent_id INTEGER, status TEXT)"
const UserSessionsSchema* = "CREATE TABLE IF NOT EXISTS UserSessions(id INTEGER PRIMARY KEY, session_id TEXT NOT NULL, user_id INTEGER NOT NULL)"

var
  db1*, db2*: SQLiteral
  preparedDb1* {.threadvar.}: bool
  preparedDb2* {.threadvar.}: bool

template prepareSQL*(db: untyped, sqlStatements: untyped): untyped =
  if not preparedDb1:
    db.prepareStatements(sqlStatements)
    preparedDb1 = true
  if not preparedDb2:
    db.prepareStatements(sqlStatements)
    preparedDb2 = true

#from postRouteHandler.nim
var userSession: UserSession
userSession.sessionId = sessionId
userSession.userId = registeredUser.id
createUserSession(userSession)

#from /models/user_session_model.nim
import sqliteral
import "../helpers/database""

type
  SqlStatements = enum
    Insert = """INSERT INTO UserSessions (session_id, user_id) VALUES (?,?)"""

proc createUserSession*(userSession: UserSession) =
  {.gcsafe.}:
    prepareSQL(db2, SqlStatements)
    db2.transaction:
      discard db2.insert(Insert, userSession.sessionId, userSession.userId)

#from debug output
PRAGMA encoding = 'UTF-8'
PRAGMA foreign_keys = ON
PRAGMA locking_mode = EXCLUSIVE
PRAGMA journal_mode = WAL
CREATE TABLE IF NOT EXISTS UserSessions(id INTEGER PRIMARY KEY, session_id TEXT NOT NULL, user_id INTEGER NOT NULL)
:memory: opened
exec Pstmt []
--- BEGIN TRANSACTION
INSERT INTO UserSessions (session_id, user_id) VALUES (QFs0MywgNjAsIDI0OCwgMzgsIDExOSwgMTA0LCAyMjAsIDE4Ny,1)
not an error
exec Pstmt []
--- ROLLBACK
:memory: not an error

I've got too many problems with my implementation, some of which were still not shown in this report, even with the verbosity of this example. I'm going to close this report and work on it a while (more) by myself and if i can't get it fixed and have a simpler example, maybe i'll reopen. thanks anyways.


Update weaver-cats, weaver-scalacheck to 0.8.1

Updates

com.disneystreaming:weaver-cats
com.disneystreaming:weaver-scalacheck

from 0.7.15 to 0.8.1.
GitHub Release Notes
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.disneystreaming" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "com.disneystreaming" }
}]

labels: library-update, early-semver-major, semver-spec-minor, commit-count:1

Doing this one myself.


exports is not defined

exports is not defined
ReferenceError: exports is not defined
    at /node_modules/svelte-scrolling/dist/index.js:5:23

Using SvelteKit with Vite, clean project with TS.
svelte.dev/repl shows
'default' is not exported by<EMAIL_ADDRESS>imported by<EMAIL_ADDRESS>

Thanks for reporting this issue. A fix was sent in v1.2.2.


Got an exception and I can't find the reason

LiteDB Version : 2.04
System Version: Microsoft Windows XP Service Pack 3
Call Stack:
System.InvalidCastException: Unable to cast object of type 'LiteDB.EmptyPage' to type 'LiteDB.DataPage'.
at LiteDB.PageService.GetPage[T](UInt32 pageID, Boolean setDirty)
at LiteDB.PageService.GetFreePage[T](UInt32 startPageID, Int32 size)
at LiteDB.DataService.Insert(CollectionPage col, Byte[] data)
at LiteDB.DbEngine.<>c__DisplayClass21_0.b__0(CollectionPage col)
at LiteDB.DbEngine.WriteTransaction[T](String colName, Boolean addIfNotExists, Func2 action)    at LiteDB.DbEngine.Insert(String colName, IEnumerable1 docs)
at LiteDB.LiteCollection1.Insert(IEnumerable1 docs)
Model:
public class Point
{
    /// <summary>
    /// Gets or sets the code.
    /// </summary>
    /// <value>The code.</value>
    public string Code { get; set; }

    /// <summary>
    /// Gets or sets the time.
    /// </summary>
    /// <value>The time.</value>
    public DateTime Time { get; set; }

    /// <summary>
    /// Gets or sets the value.
    /// </summary>
    /// <value>The value.</value>
    public double Value { get; set; }

    /// <summary>
    /// Gets or sets the total.
    /// </summary>
    /// <value>The total.</value>
    public double Total { get; set; }

    /// <summary>
    /// Gets or sets the other.
    /// </summary>
    /// <value>The other.</value>
    public double Other { get; set; }

    /// <summary>
    /// Gets or sets a value indicating whether this <see cref="Point"/> is ok.
    /// </summary>
    /// <value><c>true</c> if ok; otherwise, <c>false</c>.</value>
    public bool OK { get; set; }

    /// <summary>
    /// 0=浓度 1=参数 2=流量
    /// </summary>
    /// <value>The type.</value>
    public int Type { get; set; }
}

Some advice?

When I remove the data file, there is no problem.
It seems a compatibility problem...

Hi @lidanger, there is no difference in datafile between 2.0 and 2.0.4. Changes in datafile are made only for v3. There are some bugs in v2.0.0 on concurrency access, but was fixed (as I know) in v2.0.4. If persist, let me know.

No need maybe.I'll never change the version in the fulture in this project.
It's just a feedback now.


make fail on mac osx

does aflpin support on MAC?
$sudo PIN_ROOT=/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac make
c++ -shared -w -Wl,-exported_symbols_list -Wl,/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/source/include/pin/pintool.exp -stdlib=libstdc++    -o obj-intel64/AFLpintool.dylib obj-intel64/AFLpintool.o  -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib-ext -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/runtime/glibc -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/extras/xed2-intel64/lib -lpin -lxed -lpindwarf
clang: error: no such file or directory: 'obj-intel64/AFLpintool.o'
make: *** [obj-intel64/AFLpintool.dylib] Error 1

I am not sure, I was never able to test it.  It appears that it if failing to build the ALFpintool it self.  Do you have more error logs?

I make with the following cmd after del "obj-intel64/AFLpintool.o" in c++ build cmd:
c++ -shared -w -Wl,-exported_symbols_list -Wl,/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/source/include/pin/pintool.exp -stdlib=libstdc++    -o obj-intel64/AFLpintool.dylib   -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib-ext -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/runtime/glibc -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/extras/xed2-intel64/lib -lpin -lxed -lpindwarf
**ld: library not found for -lxed**
clang: error: linker command failed with exit code 1 (use -v to see invocation)

beacause no such dir "/extras/xed2-intel64/lib", so I modify "/pin-3.0-76991-clang-mac/extras/xed2-intel64/lib" to "/pin-3.0-76991-clang-mac/extras/xed-intel64/lib" and rebuild:
c++ -shared -w -Wl,-exported_symbols_list -Wl,/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/source/include/pin/pintool.exp -stdlib=libstdc++    -o obj-intel64/AFLpintool.dylib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib-ext -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/runtime/glibc -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/extras/xed-intel64/lib -lpin -lxed -lpindwarf
**ld: library not found for -lpindwarf**
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I found that no libpindwarf in pin 3.0 for mac, it would build error if you del "-lpindwarf":
c++ -shared -w -Wl,-exported_symbols_list -Wl,/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/source/include/pin/pintool.exp -stdlib=libstdc++    -o obj-intel64/AFLpintool.dylib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/lib-ext -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/intel64/runtime/glibc -L/Volumes/Macintosh/Users/riusksk/Reverse-Engineering/pin-3.0-76991-clang-mac/extras/xed-intel64/lib -lpin -lxed
Undefined symbols for architecture x86_64:
  "_OS_DeregisterCurrentThread", referenced from:
      LEVEL_PINCLIENT::BsdThreadTerminateProbeMode(void*, unsigned long, unsigned int, unsigned int) in libpin.a(probe_instrument_mac.os)
  "_OS_GetKernelRelease", referenced from:
……
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)


Add support for multiple hash types (rebased)

Add support for multiple hash types with negotiation of common hash types for comparison.
Manually rebased version of #277 (see discussion there)

I've merged this!  Sorry it has take a while - life rather hectic right now!
I made a small fix to the integration tests here e8daab7971fd6dcf120d7c8bb88a8be4aeb5a4b4


FAQ describing how to access fasta files for non-CDS features

See Nick's request
My answer here seems like the easiest way
https://rt.sanger.ac.uk/SelfService/Display.html?id=533148
I'll also open a ticket to create these files.....

done on dev

closing because done, even though not synced to www


The File Activity proto is receiving two additional fields—Attacks and Malware—that are not present in the OCSF schema for File Activity.

Will need to remove this 2 fields, Attacks and Malware

This occurs because Attacks and Malware are part of the Security Control profile.
By default, all profiles will be active if no profiles are configured. You might want to use cloud and container profile for your context.
To configure the profile use the following URL.
https://github.com/valllabh/ocsf-tool?tab=readme-ov-file#to-set-profiles


Error: MODULE_NOT_FOUND (undefined) Error: Cannot find module 'dotenv'

Summary
Hi, I'm trying to use stryker with typescript. But when running stryker throws this error:
Error: MODULE_NOT_FOUND (undefined) Error: Cannot find module 'dotenv'
Stryker config
module.exports = function(config){
  config.set({
     mutator: 'typescript',
    mutate: [
      './components/<component-name>/**/*.ts',
      '!./components/<component-name>/_tests/*.ts',
      '!./components/<component-name>/**/*.unit.test.ts',
      '!./components/<component-name>/**/*.integration.test.ts',
      '!./components/<component-name>/index.ts',
      '!./components/<component-name>/infrastructure/**/*.ts',
      '!./components/<component-name>/lib/*',
      '!./components/<component-name>/<filename>.ts',
    ],
    packageManager: 'npm',
    reporters: ['clear-text', 'progress', 'html'],
    testRunner: 'mocha',
    // transpilers: ['typescript'], ---> I comment the transpiler because we currently use ts-node
    tsconfigFile: 'tsconfig.json',
    testFramework: 'mocha',
    coverageAnalysis: 'off',
    maxConcurrentTestRunners: 2,
  });
}

Test runner config
I'm using mocha configuration at package.json
"mocha": {
    "timeout": 50000,
    "reporter": "progress",
    "exit": true,
    "ignore": [
      "**/node_modules/**/*"
    ],
    "require": [
      "ts-node/register/transpile-only",
      "source-map-support/register",
      "./modules/test-utils/MochaBootstrap.js",
      "./modules/test-utils/mocha.env.js"
    ]
  }

Stryker environment

├─┬<EMAIL_ADDRESS>│ ├─┬<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>├─┬<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>deduped
├─┬<EMAIL_ADDRESS>│ ├──<EMAIL_ADDRESS>deduped
│ ├──<EMAIL_ADDRESS>deduped

Test runner environment
+--<EMAIL_ADDRESS>
Test command

npx stryker run

Your Environment

software
version(s)

node
10.14.1

npm
6.4.1

Operating System
Linux - ArchLinux

Add stryker.log

stryker.log

Hi @jtlimo 👋 thanks for opening this issue.
I did a quick search in our repo, but we're not using dotenv anywhere in our codebase. I conclude that it must be something specific to your setup. Could you please try to find where it is used?

Hi @nicojs, dotenv is used at a specific file, called environmentVariables.ts. This file has the below code:
if (env !== 'production') {
  require('dotenv').config({ path: dotenvPath });
}

I thinking about stryker loading this file, probably I need to not load this file at mutate, right?

This happen to me as well on first setup.
I think main problem could be, that this package was not copied to stryker temp directory. Please check if node_modules in sandbox contain this package.

Hi @HonzaMac , actually the dotenv package is not on node_modules at sandbox, but I don't have this dependency in my package.json.
In the project, we use lerna to create modules at our monolith system, and the dotenv is a dependency at one of modules.
The stryker probably does not recognize others package.json? It's possible to have a workaround for this situation? :thinking:

Hi @HonzaMac , actually the dotenv package is not on node_modules at sandbox, but I don't have this dependency in my package.json.
In the project, we use lerna to create modules at our monolith system, and the dotenv is a dependency at one of modules.
The stryker probably does not recognize others package.json? It's possible to have a workaround for this situation?

I discover more about this, and our project basically do this:

in modules/api-config has a package.json with dotenv dependency.
we use lerna and they installs dotenv at api; although dotenv does not necessary to have on package.json global;

@nicojs Do you have some solution for this?
I put dotEnv into root package.json, but we have another modules/packages created by lerna... like this one:
@enterprise-name/package-name
and stryker doesn't recognize them. I'm really sad with that. :crying_cat_face:

As far as I know, the only way to let require('dotenv') not load from the node_modules directory (or one of its parent directories node_modules) is by setting the NODE_PATH environment variable. See https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
Could you please try to figure out the following things?

What does the test script in your package.json look like?
Can you find the modules directory in your sandbox?
 If not, try to figure out what's wrong.

Sorry, it's hard to debug these kinds of things from remote. If you can provide a small reproduction repository with a similar setup, I would be able to help more.

As far as I know, the only way to let require('dotenv') not load from the node_modules directory (or one of its parent directories node_modules) is by setting the NODE_PATH environment variable. See https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
Could you please try to figure out the following things?

What does the test script in your package.json look like?
Can you find the modules directory in your sandbox?
 If not, try to figure out what's wrong.

Sorry, it's hard to debug these kinds of things from remote. If you can provide a small reproduction repository with a similar setup, I would be able to help more.

package.json test script --> "NODE_ENV=${NODE_ENV:=test} npm-run-all --silent lint test-integrity test-public-api test-private-api"

Yes, I can find the modules directory in my sandbox;

I run the npm test at sandbox, and throws some errors:

2:31  error  Unable to resolve path to module '@enterprise-name/module-name'  import/no-unresolved
The errors above are examples and I omit the others import errors from rest of modules.

Well, the only thing I can tell you is that it is not something directly related to Stryker.
You can either:

Figure out why tests don't run in sandboxes, try. Hint: probably has something to do with NODE_PATH or a dependency setting NODE_PATH. (sorry :/)
You can wait for #2163 to be solved, as that would most probably also solve your issue.
Recreate the issue in a small example repository and attach that here, or publish that on GitHub, so I can take a look.

@nicojs I found a problem here, inside stryker sandbox the nested packages doesn't have yours node_modules.
In this image below, I point to missing node_modules at nested packages on stryker sandbox.

So if I understand correctly, the node_modules was supposed to be in storage, but wasn't?
Is tis your setup?
packages/
   package_under_test/
      .stryker-tmp/sandbox123
                             node_modules/ (symlink to packages/package_under_test/node_modules
      node_modules/
      storage/ (symlink to below)
   storage/
      node_modules

Sorry, but this keeps me guessing. A small reproduction repository would be helpful.

My setup:
project_folder/
    node_modules/
    components/
    modules/
         storage/
             node_modules/
    ...others_folders/

Right?
Okay, but inside .stryker-tmp/sandbox, in nested packages like storage/ stryker doesn't generate a symlink to storage/node_modules... This occurs for every lerna package at my project. I will try to reproduce this behavior on small repository as soon as possible.
And here has my .stryker-tmp/sandbox:
project_folder/
     .stryker-tmp/
           sandbox/
               node_modules ---> symlink to node_modules from project_folder root.
               components/
               modules/
                   storage/
                       example.ts
                       package.json
                       package-lock.json
                       ...other_files
                       **here is missing the node_modules for this lerna package**

So if I understand correctly, the node_modules was supposed to be in storage, but wasn't?
Is tis your setup?
packages/
   package_under_test/
      .stryker-tmp/sandbox123
                             node_modules/ (symlink to packages/package_under_test/node_modules
      node_modules/
      storage/ (symlink to below)
   storage/
      node_modules

Sorry, but this keeps me guessing. A small reproduction repository would be helpful.

My setup:
project_folder/
    node_modules/
    components/
    modules/
         storage/
             node_modules/
    ...others_folders/

Right?
Okay, but inside .stryker-tmp/sandbox, in nested packages like storage/ stryker doesn't generate a symlink to storage/node_modules... This occurs for every lerna package at my project. I will try to reproduce this behavior on small repository as soon as possible.
And here has my .stryker-tmp/sandbox:
project_folder/
     .stryker-tmp/
           sandbox/
               node_modules ---> symlink to node_modules from project_folder root.
               components/
               modules/
                   storage/
                       example.ts
                       package.json
                       package-lock.json
                       ...other_files
                       **here is missing the node_modules for this lerna package**

@nicojs this structure above helps, or it's best to you I create the repo?
I'm trying to create this repo but I does not have time yet, sorry.

What is the status of the reproduction?

What is the status of the reproduction?

I'm does not have time yet to reproduce in a public repo. I wish to do in this quarter.

@jtlimo good news 🎉, version 4.4 just released with --inPlace mode. It will solve your issue since it will mutate your files in place. See https://stryker-mutator.io/docs/stryker/configuration/#inplace-boolean
I'm very curious to know what you think 🙌

@jtlimo good news 🎉, version 4.4 just released with --inPlace mode. It will solve your issue since it will mutate your files in place. See https://stryker-mutator.io/docs/stryker/configuration/#inplace-boolean
I'm very curious to know what you think 🙌

@jtlimo good news tada, version 4.4 just released with --inPlace mode. It will solve your issue since it will mutate your files in place. See https://stryker-mutator.io/docs/stryker/configuration/#inplace-boolean
I'm very curious to know what you think raised_hands

Wow, I'm very happy to see this happen!
I'll try this version asap!
Thanks @nicojs !

@jtlimo good news tada, version 4.4 just released with --inPlace mode. It will solve your issue since it will mutate your files in place. See https://stryker-mutator.io/docs/stryker/configuration/#inplace-boolean
I'm very curious to know what you think raised_hands

Wow, I'm very happy to see this happen!
I'll try this version asap!
Thanks @nicojs !


Does it support multithreading? Is it lock free?

Hi @loongs-zhang . I believe you already found the answer before my response. Still do not want to leave it unanswered:

The priority queue traits are designed for single threaded usage.
Likewise, the d-ary heap implementations are not meant to be Send across multiple threads.

I did work on implementing lock free concurrent collections but have never worked on a concurrent priority queue represented as a heap. Sounds challenging but also very interesting. Hope I can focus on this at some point.

I'm currently using crossbeam_skiplist::SkipMap<c_longlong, Vec<T>>, but it is not exactly what I want because the priority may be the same. The second type of SkipMap uses Vec, which still cannot fully utilize CPU cache. Speaking of which, I think you can refer to crossbeam_skiplist.


Explore PostgreSQL Datasource with JSONB array ERROR

Error in Explore view when the underlining query filters a JSONB array in PostgreSQL that contains integer values.
Workaround
Convert the integer value inside the JSONB array to string
what you expected to happen.
Since the query is working as expected in SQL Lab, we should be able to create a chart utilizing it.
what actually happens.
Screenshots

How to reproduce the bug

Create a query in SQL Lab that filters a JSONB array in PostgreSQL, where the value is integer (example below)
Create a datasource out of this query
Error in Explore view when trying to create a chart

select  * from jsonb_datasource WHERE jsonb_datasource.transactions @> '[{"transaction_status":"SUCCESS","transaction_id": 1}]'
Environment

superset version: 0.37.0rc1

Interesting! @villebro @dpgaspar this looks up your alley. Thank you for the bug report!

@GiannisDimitriou I am unable to reproduce the bug on master branch against a Postgres 10.4 database. The query
SELECT *
FROM (SELECT '[{"transaction_status":"SUCCESS","transaction_id": 1}]'::jsonb AS abc) AS x
WHERE x.abc @> '[{"transaction_status":"SUCCESS","transaction_id": 1}]'

renders fine in both SQL Lab and when in Explore view:

Are there perhaps additional filters that you're trying to apply on JSONB columns? I propose breaking out any fields you need using JSON get operators: https://www.postgresql.org/docs/10/functions-json.html. For example, if your JSONB column contains an object with the key transaction_id, you can easily break that out into a separate column that can then be used as a regular column in your datasource. Example below of how superset renders the query when applying a filter on the (integer cast) value of transaction_id:

@villebro  thanks for trying to reproduce this.
I don't use any additional filters on the JSONB column.
Also the end query is a bit more complicated so exposing the transaction_id on a different column will make things a lot more complicated.
Can you try running the below query in SQL Lab and then press EXPLORE in order to visualise it.
This is what I see following the above steps:
SELECT  * FROM ( SELECT  * FROM ( VALUES  (1::int,'[{"transaction_status":"SUCCESS","transaction_id":1},{"transaction_status":"SUCCESS","transaction_id":2}]'::jsonb), (2::int,'[{"transaction_status":"PENDING","transaction_id":3},{"transaction_status":"CANCEL","transaction_id":4}]'::jsonb) ) s(id, transaction_details) ) t WHERE transaction_details @> '[{"transaction_status":"SUCCESS","transaction_id":1}]'

Thanks for the detailed reproduction case @GiannisDimitriou. I was able to reproduce the error, will investigate.

This can be solved by putting a space next to the colon character. In addition to making it cleaner JSON, it avoids making SQLAlchemy think it's a bound parameter:
SELECT *
FROM (
SELECT *
FROM (
VALUES (1::int,'[{"transaction_status":"SUCCESS","transaction_id": 1},{"transaction_status":"SUCCESS","transaction_id": 2}]'::jsonb),
(2::int,'[{"transaction_status":"PENDING","transaction_id": 3},{"transaction_status":"CANCEL","transaction_id": 4}]'::jsonb) )
s(id, transaction_details) ) t
WHERE transaction_details @> '[{"transaction_status":"SUCCESS","transaction_id": 1}]'

By adding the space it works as expected but this is a workaround as well.
In real life adding a space after the colon character for any incoming JSON field before storing it in the db or manipulating the JSONB field after ( jsonb_agg() function does not add any spaces), is kinda counter intuitive.

After checking PostgreSQL behaviour I see that a space is added automatically after the colon character when using the jsonb_agg function. I haven't check though what is the behaviour when trying to insert a json string explicitly in an json/jsonb column.
So for this example adding the space on the filter clause only, its fine.
Yeah I get your point I will check if there is any way to escape JSON and reply back.
@villebro I don't think inserting non-spaced JSON objects in a JSON or JSONB column will cause trouble, it's only the query that needs to have the space.
In fact the JSON object it self must have a space as well for this to work.
Check this example..same error
SELECT *  FROM (  SELECT *  FROM (  VALUES  (1::int,'[{"transaction_status":"SUCCESS","transaction_id":1},{"transaction_status":"SUCCESS","transaction_id":2}]'::jsonb),  (2::int,'[{"transaction_status":"PENDING","transaction_id":3},{"transaction_status":"CANCEL","transaction_id":4}]'::jsonb) )  s(id, transaction_details) ) t  WHERE transaction_details @> '[{"transaction_status":"SUCCESS","transaction_id": 1}]'

What I'm mostly trying to say is that the JSON data will usually be coming from a table, not a static query as in the example. In that case, the only requirement is that your query has the appropriate spaces in the JSON objects.

Correct, I tried explicitly adding a json string to a jsonb column and the space after the colon is added automatically.
I am closing this issue as resolved.
Thanks for your help @villebro


Chapter 2: minikube image load failed

Hi,
I have an issue with Docker 25 that not being able to load images into my local cluster via the minikube image load command, and I was able to resolve by downgrading to Docker 24 from here
see the issue for more information: https://github.com/kubernetes/minikube/issues/18021
Below is my environment that I was having the issue.
OS: macOS (intel chip)
Docker Engine: 25.0.3
Minikube: v1.32.0
Kubectl Server Version: v1.28.3
this is the part of the log
$ minikube image load catalog-service:0.0.1-SNAPSHOT

❌  Exiting due to GUEST_IMAGE_LOAD: Failed to load image: save to dir: caching images: caching image ".../catalog-service_0.0.1-SNAPSHOT": write: unable to calculate manifest: blob sha256:... not found

Had the same issue, also on an intel Mac. I tried several times recreating the cluster and the catalog-service image. Eventually I gave up. I've removed minikube and installed kind. All is good now.
@hyeonjerry  thanks for the tip. I thought it was something wrong with minikube and did not consider for a second that it might actually be docker's fault.

Yes, as pointed out above by @hyeonjerry, it's an issue with Docker 25 that affects the current version of Minikube. They are working on a fix. For now if you are using Minikube, this workaround below works. Make sure to ignore *.tar in your .gitignore file to prevent error when trying to push to GitHub due to the big size of the tar.
docker image save -o image.tar my_image:tag
minikube image load image.tar

For example:
docker image save -o catalog-service.tar catalog-service:0.0.1-SNAPSHOT
minikube image load catalog-service.tar

Thank you all for reporting this issue and providing some workarounds. I didn't manage to reproduce the error on macOS with Apple Silicon when using Docker 26.1.1 and Minikube v1.33.1. I wonder if the error is only available on macOS with Intel or if something changed in Docker 26 (or more recent versions of Minikube) that fixed the issue.
In any case, I have included that information in a dedicated section of the repo: https://github.com/ThomasVitale/cloud-native-spring-in-action/commit/48b171adda29642c150284c46e55512c27e917f4. Thanks again!


Automated cherry pick of #10845: fix(region): optimized vm account name code

Cherry pick of #10845 on release/3.5.
#10845: fix(region): optimized vm account name code

Codecov Report

Merging #10848 (c6b9ceb) into release/3.5 (6a15e02) will increase coverage by 2.85%.
The diff coverage is 0.34%.

@@               Coverage Diff               @@
##           release/3.5   #10848      +/-   ##
===============================================
+ Coverage         5.98%    8.83%   +2.85%
===============================================
  Files              796      417     -379
  Lines           125693    51797   -73896
===============================================
- Hits              7522     4576    -2946
+ Misses          117504    46790   -70714
+ Partials           667      431     -236

Flag
Coverage Δ

aFlag
8.83% <0.34%> (+2.85%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/apigateway/handler/auth.go
0.00% <0.00%> (ø)

pkg/apis/input.go
0.00% <ø> (ø)

pkg/baremetal/cronjobs.go
0.00% <0.00%> (ø)

pkg/baremetal/utils/ipmitool/ipmitool.go
0.00% <0.00%> (ø)

pkg/cloudcommon/db/db_dispatcher.go
0.00% <0.00%> (ø)

pkg/cloudcommon/db/jointbase.go
0.00% <ø> (ø)

pkg/cloudcommon/db/metadata.go
5.04% <0.00%> (ø)

pkg/cloudcommon/db/opslog.go
1.36% <0.00%> (-0.01%)
:arrow_down:

pkg/cloudcommon/db/quotas/errors.go
0.00% <0.00%> (ø)

pkg/cloudcommon/db/quotas/handler.go
0.00% <0.00%> (ø)

... and 392 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cd2207b...c6b9ceb. Read the comment docs.


Customizing Language?

Hi,
is it possible/how to set the L.drawLocal object for customizing the language label?
Thanks

Have you tried setting properties on the L.drawLocal variable in your component that's using leaflet-draw?
L.drawLocal is missing from the typings, but this worked for me:
import { Component, OnInit } from '@angular/core';

import * as L from 'leaflet';
import 'leaflet-draw';

@Component({
	selector: 'leafletDrawCoreDemo',
	templateUrl: './core-demo.component.html'
})
export class LeafletDrawCoreDemoComponent implements OnInit {

	options = {
		layers: [
			L.tileLayer('http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', { maxZoom: 18, attribution: 'Open Street Map' })
		],
		zoom: 5,
		center: L.latLng({ lat: 46.879966, lng: -121.726909 })
	};

	featureGroup = L.featureGroup();
	layers = [ this.featureGroup ];

	drawOptions = {
		position: 'topright',
		draw: {
			marker: {
				icon: L.icon({
					iconSize: [ 25, 41 ],
					iconAnchor: [ 13, 41 ],
					iconUrl: 'assets/marker-icon.png',
					shadowUrl: 'assets/marker-shadow.png'
				})
			},
			polyline: false,
			circle: {
				shapeOptions: {
					color: '#aaaaaa'
				}
			}
		},
		edit: {
			featureGroup: this.featureGroup
		}
	};

	constructor() {
		(L as any).drawLocal.draw.toolbar.buttons.polygon = 'Draw a sexy polygon!';
	}

	ngOnInit() {
		this.featureGroup.addLayer(L.circle([ 46.95, -122 ], { radius: 5000 }));
	}
}

I thank you 👍 it works

and the other button? (save, cancel, ....)


Feature/save to specific album

This PR adds the option to saveToCameraRoll to save a picture to a specific album.
It replaces the "type" param with "options", which is optional and contains "type"(optional) and "album"(optional). Because it replaces an old parameter, it's a breaking change.
Furthermore it fixes getPhotos for Android by checking the platform before setting the default value for groupType, because otherwise it throws an exception with "groupTypes is not supported on Android" if groupTypes is not undefined. Maybe we should just remove this exception instead.
I tested the PR on iPhone Xr(Simulator) and OnePlus3T(Device)

@SimonErm thanks for this! I like the approach in general and the ability to extend the options without introducing future breaking changes. This is still possible without the current breaking change though, so would you be interested in reworking your implementation to match that?

i messed up the rebase. I am going to open a new PR.


feat!: integrate AVM proving

#6775  👈
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @fcarreiro and the rest of your teammates on  Graphite

Benchmark results
Metrics with a significant change:

protocol_circuit_witness_generation_time_in_ms (root-parity): 64.5 (+38%)
protocol_circuit_proving_time_in_ms (base-rollup): 77,926 (+21%)
protocol_circuit_proving_time_in_ms (root-parity): 40,457 (+19%)
protocol_circuit_proving_time_in_ms (public-kernel-setup): 44,750 (+19%)
protocol_circuit_proving_time_in_ms (public-kernel-tail): 181,635 (+17%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,742
1,559
715
745 (-2%)
775 (-1%)

proof_construction_time_sha256_30_ms
11,444
3,097
1,391
1,419
1,456 (-1%)

proof_construction_time_sha256_100_ms
44,031
11,855 (-1%)
5,442 (-3%)
5,520
5,377 (-2%)

proof_construction_time_poseidon_hash_ms
78.0
34.0
34.0
58.0
87.0 (-1%)

proof_construction_time_poseidon_hash_30_ms
1,518
417
202
225 (-2%)
268

proof_construction_time_poseidon_hash_100_ms
5,754
1,580
732 (+1%)
784
797 (-1%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
8 txs
32 txs
64 txs

l1_rollup_calldata_size_in_bytes
1,412
1,412
1,412

l1_rollup_calldata_gas
9,464
9,464
9,476

l1_rollup_execution_gas
616,105
616,105
616,117

l2_block_processing_time_in_ms
1,285
4,827 (+1%)
9,598 (+1%)

l2_block_building_time_in_ms
44,455
174,167 (-1%)
348,252 (-1%)

l2_block_rollup_simulation_time_in_ms
44,258
173,438 (-1%)
346,804 (-1%)

l2_block_public_tx_process_time_in_ms
41,115 (+6%)
170,185
343,394

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 16 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
9,484 (+1%)
14,420

node_database_size_in_bytes
14,467,152
21,307,472

pxe_database_size_in_bytes
18,071
29,868

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
512 leaves
1024 leaves
2048 leaves
4096 leaves
32 leaves

batch_insert_into_append_only_tree_16_depth_ms
10.4
17.0
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.7
31.8
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.603
0.521
N/A
N/A
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
48.7
76.3
247
476
934 (+1%)
1,846
N/A

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
543
1,055
2,079
4,127
N/A

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.496
0.470
0.448
0.444
0.442 (+1%)
0.442
N/A

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
58.7 (+1%)
113 (+1%)
357
701
1,395 (+1%)
2,766
N/A

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
106
208
692
1,363
2,707
5,395
N/A

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.507
0.507 (+1%)
0.483
0.481
0.482 (+1%)
0.481 (+1%)
N/A

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
62.9

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
107

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
N/A
N/A
N/A
N/A
N/A
N/A
0.555

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
84,053
665,267

Transaction size based on fee payment method
| Metric |  |
| - |  |


[VS Code] An error occurred while sending the request.

Error:
An error occurred while sending the request.

Info:

login with GitHub
user code doesn't work
forever and ever 'Signing in...'

Version Data

extensionName
VSLS

extensionVersion
0.3.319

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.24.1

platformName
MacOS

platformVersion
17.6.0

Error:
An error occurred while sending the request.

Info:

login with GitHub
user code doesn't work
forever and ever 'Signing in...'

Version Data

extensionName
VSLS

extensionVersion
0.3.319

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.24.1

platformName
MacOS

platformVersion
17.6.0

@Lewih Thanks for reporting this! We're tracking this issue via #597 , so I'm going to close this is a duplicate. On #597, could you attach the log file you get after running the Live Share: Export Logs command in the command palette? Thanks!


add verify markup pass

This replicates the functionality of metamath.exe's VERIFY MARKUP command. New checks:

Labels should not contain _
Labels should not start with mm
Labels should not match reserved windows filenames CON, PRN, AUX etc
If there is an axiom ax-foo with a corresponding axfoo axiom/theorem then they should match
Axioms that prove |- ... should have a name ax-* or df-*
Tokens should not use the @ and ? characters
Every token should have a htmldef command
Use of the author ?who? is forbidden
Dates should parse
Parentheticals should come in chronological order
Axioms and theorems (except for syntax axioms) should have a Contributed by comment
There should not be two Contributed by comments
The Contributed by comment should be the first one in the list
Proof modification is discouraged` is not allowed on axioms
*OLD and *ALT theorems should be marked as (Proof modification is discouraged.) and (New usage is discouraged.)
Lines should have at most 79 characters (TODO: make an exception for lines with URLs)
There should be no trailing whitespace
Tabs should not appear anywhere
There should not be two htmldef (and althtmldef, latexdef) commands for the same token
Tokens appearing in htmldef should be declared
The exthtmllabel command should reference an existing label
Header comments (defined as starting with $( <whitespace> #*#*...) should parse as header comments (decorator, newline, one line of text, newline, decorator, markup text)
The ~ and ` characters, when used as delimiters in markup, should have whitespace around them
Characters [, ~ and ` must be doubled if they could not be interpreted as special in markup
<HTML> and </HTML> must appear in all caps, not nested, and in balanced pairs
Tokens inside ` math strings in markup must exist in a $c/$v statement
Math strings must be closed before the end of the comment
<HTML> must be closed before the end of the comment

Interestingly, (10) is not implemented in exactly the same way in metamath.exe, and running this on set.mm has unearthed two issues:

ssfin4 has two revised-by attributions out of order
cshwidxm1 has a revision date prior to its contributed-by date

Spectacular! Quick question: This is something you can enable or not per database, instead of ALWAYS requiring it, yes?
Eventually I'd like to also allow ISO dates in YYYY-MM-DD form, but there's no need to do that now.

I think the dates are even specified as being YYYY-MMM-DD in the metamath book, but metamath.exe and set.mm obviously don't match that.
The verify markup pass is completely separate from other activities of metamath-knife, like metamath.exe's VERIFY MARKUP. You have to explicitly enable it independently of proof verification, which you can do by setting the -m flag in the command line. You can specify -v -m or -v or -m depending on what you want, and call it on the databases you like.

I added comment markup checking, and rules 22-28 listed in the PR comment. This has found quite a few more issues, 535 reported issues in set.mm, to the extent that we may want to relax some of these checks. The majority of reported errors are violations of rules 23 (mandatory whitespace around `) and 24 ([ should be doubled in regular text). Besides those, three more errors have been found:

The <HTML> block in section "Ring isomorphism between polynomial matrices and polynomials over matrices" is not closed
The <HTML> block in section "GRAPH THEORY" was closed with <HTML> instead of </HTML>
The conventions comment contains $d and $a inside math strings even though they are not defined tokens

Here are some examples of rule 23, 24 violations:
warning: Markup character requires surrounding whitespace
     --> ../mm/set.mm:24152:65
      |
24152 |   correspond respectively to the logical exclusive disjunction (~ df-xor ) and
      |                                                                 - Put spaces around this character
      |
warning: Markup character requires surrounding whitespace
     --> ../mm/set.mm:30270:63
      |
30270 |        Lammen, 30-Dec-2017.)  Reduce symbol count in ~ nfex , ~hbex .  (Revised
      |                                                               - Put spaces around this character
      |
warning: Invalid escape character
     --> ../mm/set.mm:81549:33
      |
81549 |        more general sets than R [the real numbers] and to other objects.".  The
      |                                 - This escape character should be doubled
      |
      = note: This character has special meaning in this position, but it was not interpretable here.
      = note: Use ~~ or [[ or `` if you mean to include the character literally
warning: Markup character requires surrounding whitespace
      --> ../mm/set.mm:182964:29
       |
182964 | ` -. 2 || N  ` to say that "` N ` is _odd_" (under the assumption that
       |                             - Put spaces around this character
       |
warning: Markup character requires surrounding whitespace
      --> ../mm/set.mm:182965:11
       |
182965 | ` N e. ZZ `).  The previously proven theorems about even and odd numbers, like
       |           - Put spaces around this character
       |
warning: Invalid escape character
      --> ../mm/set.mm:279456:55
       |
279456 | "[Cramer's rule] ... expresses the [[unique] solution [of a system of linear
       |                                                       - This escape character should be doubled
       |
       = note: This character has special meaning in this position, but it was not interpretable here.
       = note: Use ~~ or [[ or `` if you mean to include the character literally
warning: Markup character requires surrounding whitespace
      --> ../mm/set.mm:279463:2
       |
279463 | (~ https://en.wikipedia.org/wiki/Cramer's_rule#A_short_proof), is as follows:
       |  - Put spaces around this character
       |

Looking at these examples, it seems like spaces are usually used on the inside of a math string, rather than on both sides, and similarly there are many occurrences of  (~ label ) with space after but not before the ~. For the doubled character error, [ is the usual culprit, and examples like L.279456 above show that users are depending on the behavior that [foo bar] is not treated as a bibliographic reference and puts a literal [ character, but [foobar] is a bibref (which is why the [[ in [[unique] is doubled).
What do people think? Should we update set.mm with these more stringent requirements or drop / weaken the requirement to decrease the number of false positives?

The verify markup pass is completely separate from other activities of metamath-knife, like metamath.exe's VERIFY MARKUP. You have to explicitly enable it independently of proof verification, which you can do by setting the -m flag in the command line. You can specify -v -m or -v or -m depending on what you want, and call it on the databases you like.

Perfect!

I added comment markup checking, and rules 22-28 listed in the PR comment. This has found quite a few more issues, 535 reported issues in set.mm, to the extent that we may want to relax some of these checks. The majority of reported errors are violations of rules 23 (mandatory whitespace around `) and 24 ([ should be doubled in regular text). Besides those, three more errors have been found:

Wow. Well, you've proven once again that automated checking can detect problems :-).
I think this should be raised to the mailing list for discussion. We should try to steer all tools to eventually agree on what the checks are :-). I'm torn on whether or not we should actually make them stringent. I'm okay with eventually making the rules more stringent, especially if they'll eliminate potential ambiguities, but there should be some reason to enforce a rule. If we enforce rules not previously enforced, we ought to do it in stages - update the .mm files, give people time to adjust their copies, and only later start enforcing them.

Okay, this PR is now feature complete, with some omissions:

omitted: check that files in IMG SRC="..." of html_defs exist
omitted: check that html_home has HREF="..." and IMG SRC="..."
omitted: check that ext_html_home has HREF="..." and IMG SRC="..."
omitted: top date check

Additionally, bibliography checking is allowed by the API but there is no way to trigger it from the command line currently. The issue is what to do about locating files (specifically: the bibliography files in the htmlbibliography command and the GIF files in the checks above), as I would rather not just naively assume everything is in the current directory like metamath.exe. Suggestions on a good command line input here are welcome.
The next step is to get set.mm passing the checks. Some of the checks have been weakened per the metamath group discussion, so we are down to 112 issues, most of which are easy to fix.

It is now possible to give bibliography files on the command line, via --biblio FILE. This command can be used twice; the second will supply the exthtmlbibliography for set.mm.
Markup is also fixed in metamath/set.mm#2472.

Thanks for all the answers and explanations, LGTM!


Update youtubeoauth.py (TODO part)

This modification introduces a new method _extract_oauth_credentials() which dynamically extracts the client ID, client secret, and scopes from the YouTube TV web page and its associated JavaScript file. Here's a breakdown of the changes:
We've added instance variables self._CLIENT_ID, self._CLIENT_SECRET, and self._SCOPES to store the extracted values. The _extract_oauth_credentials() method:
Downloads the YouTube TV web page
Extracts the URL of the base.js file
Downloads the base.js file
Uses regular expressions to extract the CLIENT_ID, CLIENT_SECRET, and SCOPES from the JavaScript content
We call self._extract_oauth_credentials() at the beginning of both initialize_oauth() and authorize() methods to ensure we have the latest credentials before proceeding. If the extraction fails for any reason, we fall back to the original hardcoded values (represented by _SCOPES in this case).
This approach makes the code more maintainable and adaptable to changes in YouTube's authentication system. However, please note that this method relies on the current structure of YouTube's web pages and JavaScript files. If YouTube significantly changes their frontend, you might need to update the extraction logic.

Does this still work?   I am not well versed in ytdlp plugins so I hope I just messed up trying to use it.
Would providing our own API credentials be a good idea or is it better off as is?

Does this still work? I am not well versed in ytdlp plugins so I hope I just messed up trying to use it.
Would providing our own API credentials be a good idea or is it better off as is?

Your own API creds won't work iirc, it requires the Youtube TV client.

From a quick look I don't think this PR works - it doesn't look like the code is hooked up to anything. The creds do not come from the player js either..
It kind of looks like this was written by AI..?

ok, no problem


Customize labels

Is there a way to the change text, e.g. Card number, Card number is invalid, etc?

Hi @snowcxt,
You are able to change placeholder text by using custom renderers https://github.com/medipass/react-credit-card-input/blob/master/readme.md#custom-input-renderer-usage.
However, there is no way to change error-based help text at the moment. I'll mark this issue as a 'good first issue' if anyone wants to add custom error messages.


WIP Changed template names and folder structures for branch, user, client, group. And reverse url in the views for HttpResponse Redirect.

Coverage increased (+0.004%) to 86.86% when pulling 7ef373bedd331a763bc9546fdd76e98ebb65007b on swethanaretla:code_cleaning into edd46acd58af4861a8f51870188dd371d6d8951d on MicroPyramid:master.


Replace InitOnLoad with MRTK > Utils > WMR > Check Config

This change continues the work started by @wiwei to replace InitOnLoad classes with discrete menu options.
The new Mixed Reality Toolkit > Utilities > Windows Mixed Reality > Check Configuration menu item checks to see if the Dot Net WinRT NuGet package has been imported and appropriately sets the DOTNETWINRT_PRESENT flag.

/azp run

/azp run

added doc updates.

/azp run

/azp run

/azp run

/azp run

/azp run

/azp run


[CT-2260] [Feature] Explainable state modified

Is this your first time submitting a feature request?

[X] I have read the expectations for open source contributors
[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature
Problem:
I want to be able to explain a state:modified selection with fine grained diffs. I can code up a lot of ways to solve this but let me riff on what I think would be ideal.
Solution:
There are methods returning primitive types that should ideally be wrapper types that duck-type the effect of the primitive. Specifically, the methods I am calling out are planted all over the sort-of core dataclasses, and they return bools when, in an ideal world, they should return a specific class which implements __bool__ but is, by effect of not being a binary primitive, significantly more extensible. We can do this whilst maintaining the contract with dependents of the method.
All of the same_* methods from same_body to same_database_representation
These should pass around a Comparator which has a baked in method called output diff which leverages difflib to generate a log-able diff of the node. This means Comparator's will carry references to old and new, so that it is available as needed. And lastly, the diff is opt-in based on where we decide to plant the method calls. I can see the -vv verbosity level being the impetus to output the diff.
Describe alternatives you've considered
So long as the same_* methods are restricted to bools, there is no alternative.
Who will this benefit?
Anyone troubleshooting why a model is running when they don't think or know if it was modified by providing a highly detailed unified_diff.
Are you interested in contributing this feature?
Yes
Anything else?
Slack Thread: https://getdbt.slack.com/archives/C50NEBJGG/p1675216311733109

Thanks for opening the issue @z3z1ma! I know folks will be excited about this :)
(Internal note: Even though this has to do with node selection, which is generally Team:Execution, the changes will largely be in updates to our node classes, so Team:Language feels like the better fit.)

In progress PR: #7102


Isolating frontend dependencies.

Problem
Most of FoundationPress front-end dependencies are scattered. This includes sass, css, javascript, and other assets. Although quite common in other themes, I think a more modern approach would be to isolate each into a parent folder. This makes it clear where all front-end dependencies live.
Solution
We recently implemented FoundationPress as parent for a child theme. Within our child theme, we adapted the following pattern:
assets
├── css
├── img
├── js
│   ├── custom
│   └── vendor
├── scss
│   ├── config
│   └── site
└── type
    └── fontawesome
        ├── fonts
        └── scss

12 directories

This makes it clear where everything lives. Granted, we had to adjust Grunt, SASS, and functions file to correctly import all dependencies, but the result was good.
Discussion
This, of course, is an option that could be left to the child theme. But I think it would be beneficial if we considered this for parent. It mirrors other popular applications and how they manage their frontend. One of the questions raised is whether or not to include bower components in this folder. We chose not to as it follows the same paradigm like node modules and taking a step further, a vendor folder from composer.
If this is something your interested in, I can do a PR on the initial solution.

Hello. I agree that it makes sense to organise static files in a separate folder for assets. I am also accustomed to this from a number of other projects. The folder for bower_components (and node_modules) should be kept outside and lie directly on the root. If you want to make a pull request, I am open to merge it into the master branch. It would be nice if you could update the README.md with the new folders structure while you're at it. Thanks!

Sure! I will try and get this to you this week. And I'll provide a sub-heading for folder structure in the readme as well.

Sorry about the delay on this, been busy with the current project using this theme as parent. Hopefully next week I can do the PR.

No worries. @THEBRANDYMAN made the changes and created a pull request https://github.com/olefredrik/FoundationPress/pull/454 :)

@THEBRANDYMAN thanks for the PR!


If no driver is installed - Exception uncatched

If no suitable driver is installed on the machine this code throws exception:
https://github.com/CirrusRedOrg/EntityFrameworkCore.Jet/blob/4dda3e41d5e2e55beb87b2efb20a576c7dcd7c9a/src/EFCore.Jet.Data/JetConnection.cs#L586
The message in log in this case does not say the problem:
An error occurred using the connection to database '""' on server '"File.accdb"'.
System.InvalidOperationException: Sequence contains no elements

A better error message is now thrown in beta 1


S2 Increase Python Unit Tests

Added to Sprint 2 (milestone) during 9/17/18 Sprint planning meeting. Assigned to dev9 team.

Issue
Increase Python Unit Tests

T-shirt size: S(mall), M(edium), L(arge)
M

Story Points - Fibonacci (1,2,3,5,8,13)
5

Working on it now

I have a branch out with details in the commit message. Quick summary: Extracted out reusable parts, changed test loader to load all tests in the directory, renamed dir per best practices... PR coming soon.

review the user story and decompose around the feature. the unit testing will be a part of user stories. 8 story points. (add 2 tasks: unit and integration tasks).

Closing, as there is no completion criteria for this story.


Add letters for the asset amount

Add letters to indicate the number of assets

Now billion (B) is the last on. Should add more.

Added like this. Should be enough

checked


Special char in sender's name is displayed incorrectly

Hi,
in current version <IP_ADDRESS> there is a problem showing special chars in the sender's name in the message list.
The sender's name in the following example is "Lands' End".

This only applies to the message list. In the real message it is shown correctly:

Other special chars like umlauts or "ß" are shown correctly as well.

Thanks.
Yes, I know about this problem, this fix will be available in the next release and already available in the beta version.

fixed in version 1.10.3.
Thanks!


System.TypeLoadException when using with GenericRepository

Hello,
When used with generic repository
System.TypeLoadException: 'Could not load type 'MongoDbGenericRepository.IMongoDbCollectionIndexRepository' from assembly 'MongoDbGenericRepository, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'.'
error occurs on:
services.AddIdentity<ApplicationUser, ApplicationRole>()                 .AddMongoDbStores<ApplicationUser, ApplicationRole, Guid>(taskDbContext)                 .AddDefaultTokenProviders();
It is probably because of this package's internally dependency on IMongoDbCollectionIndexRepository while GenericRepository has the same interface with different assembly version

Are you using MongoDbGenericRepository 1.3.8?
Le 21 avr. 2019 15:54, Galip Erdem<EMAIL_ADDRESS>a écrit :
Hello,
When used with generic repositoryhttps://github.com/alexandre-spieser/mongodb-generic-repository
System.TypeLoadException: 'Could not load type 'MongoDbGenericRepository.IMongoDbCollectionIndexRepository' from assembly 'MongoDbGenericRepository, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'.'
error occurs on:
services.AddIdentity<ApplicationUser, ApplicationRole>() .AddMongoDbStores<ApplicationUser, ApplicationRole, Guid>(taskDbContext) .AddDefaultTokenProviders();
It is probably because of this package's internally dependency on IMongoDbCollectionIndexRepository while GenericRepositoryhttps://github.com/alexandre-spieser/mongodb-generic-repository has the same interface with different assembly version
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/alexandre-spieser/AspNetCore.Identity.MongoDbCore/issues/13, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC4IAQTI2G67B6N62EHKLCLPRR5Z7ANCNFSM4HHLZSHQ.

Hello,
It is 1.4.0. I think i tried to downgrade to 1.3.8 also it didn't work.

Try downgrading and clear your cache/ build folder. 1.4.0 is not compatible yet because I renamed this interface. For the downgrade please use the version referenced in the AspNetCore.Identity.MongoDbCore nuget package.
Le 21 avr. 2019 21:54, Galip Erdem<EMAIL_ADDRESS>a écrit :
Hello,
It is 1.4.0. I think i tried to downgrade to 1.3.8 also it didn't work.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/alexandre-spieser/AspNetCore.Identity.MongoDbCore/issues/13#issuecomment-485281735, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AC4IAQWVHSVF34MANRIK5I3PRTH63ANCNFSM4HHLZSHQ.

I have upgraded the package dependencies.
https://github.com/alexandre-spieser/AspNetCore.Identity.MongoDbCore/releases/tag/1.1.1


Container padding MouseWheel

Hi!
Look a this : http://jsfiddle.net/91mbbt7k/3/
Difference between 1 and 2 is that I can wheel on "padding" first container but NOT on the second...
I try to find a way to resolve that ^^'

Okay!
I'va changed the Mouse Wheel bind.
Lines 1472 to 1474
mCustomScrollBox.bind("mousewheel."+namespace,function(e,delta){
	_onMousewheel(e,delta);
});

change by
mCustomScrollBox.parent().bind("mousewheel."+namespace,function(e,delta){
	_onMousewheel(e,delta);
});


extend decoder with depth field to limit memory that can be allocated by recursive types

consider to following type. i can craft such message that will recursively allocate as much memory as i want by encoding length of the collection just below the limits enforced by the user.
type Nested struct {
    Collection []Nested
}

Resolved with https://github.com/spacemeshos/go-scale/pull/60


OCM-6281 | chore: Ensure mocks, tests and other directories are excluded from Codecov statistics

/remove-approve

/lgtm


NC | Noobaa log is not performing logrotate

Explain the changes

Enhance logrotate function to support syslog-ng if it used
logrotate will happen daily and logrotate_noobaa.config will be used for the rotation. Outchannel is removed from rsyslog config and rotation will not happen when log file reach size limit after this changes.
ratelimit for rsyslog is disabled
rsyslog and logrotate config files are unchanged by RPM, if edited : link

Issues: Fixed #xxx / Gap #xxx

https://github.com/noobaa/noobaa-core/issues/8098
https://github.com/noobaa/noobaa-core/issues/8153

Testing Instructions:

install noobaa RPM
check logs are getting rotated daily basis when logs are reachs configured size

[X] Doc added/updated
[ ] Tests added

Looks good to me.
Just change the name of the file in first description comment from logrotate_noobaa.config to logrotate-noobaa


Add command to support for Paste cell above current cell

Blocked by upstream https://github.com/microsoft/vscode/issues/114720
Hacky work around posted here https://github.com/microsoft/vscode/issues/114720#issuecomment-764826535
Will wait for feedback from VSC.

Not sure this is a blocker for Feb, but @claudiaregio added this list as a blocker (but these are new features, hence not sure about priority of this particular issue).

Not sure this is a blocker for Feb, but @claudiaregio added this list as a blocker (but these are new features, hence not sure about priority of this particular issue).

Validated. Command is there and works like a charm:


Will this work with github?

I am new to this kind of thing, so I want to know if it is possible to use this on github?

Yes. I use my fork which adds static media collection to deploy to Github pages.

Sweet! If you dont mind me asking, what kinds of steps should I be taking to get this to github? I dont think Im understanding this medusa app very well. I understand that its a static site generator but I dont know where I should be inputting my posts, my themes and how I should set up my site for deployment. For instance, if we look at a generic static site generator set up, its something like this:
 ▸ content/
 ▸ layouts/
 ▸ static/
 config.toml

In Medusa, I think that there is some configs that I would have to go through, reasonably, but I am stuck on the renderers. Are they supposed to derive from models in your app? I created a method via Python/Django to render markdown documents and displays them in the web, but I dont know if thats how medusa works, or if I can make some integrations.
On another note, when using Hugo (a diff staticsitegen), I noticed that if I place any markdown file in the posts section, the site generation produces as much folders with index.html in them, with that being the webpage. Just wondering, is it like this in Medusa? .

With Medusa, you develop a Django site as you would any other, with the consideration that you're not going to be able to process dynamic content, handle POST requests, or anything else a "normal" Django site would do.
Any time you make a change to the content of your site, presumably when running the project using the local development server, you will compile out those changes to static code, which can then be pushed to your Github Pages enabled repository.
If your site has blog posts for example, you would/could edit those using Django admin running locally, then compile and push.
Renderers simply take paths, follow them, and save the result of calling the URL as a static file. Sometimes these paths come from a model instance, sometimes they are just the result of a view. For each path, a directory structure is created to mimic segmented URLs.
As an example, on my personal site: https://alsoicode.github.io/, I have a view: /work/ which returns samples of my graphic design and web projects. Each work sample has a detail: https://alsoicode.github.io/work/texas-tribune/ for example.
Medusa will create an index.html that sits in a /work directory, with links to each work sample, that come from the .get_absolute_url() method of each. Each URL segment, like /texas-tribune, becomes a directory, and the result of the work sample detail view is saved as an index.html in that directory.
I hope that gives you a better understanding of what's going on behind the scenes.

It does, thanks for this. My only issue is that get_absolute_url seems to be a very model-based method. Does this mean I have to create something from django-admin? It seems like this is what I would need to do, but if there is a way I can link urls without getting the absolute url or without models I would prefer that. I've had lots of issues trying to work with get_absolute_url(), so Ive usually worked without them.

How you get the URL patterns is up to you. get_absolute_url is leveraged by Django admin, but it's not a hard requirement; it's just a convenient, documented way to encapsulate getting the URL pattern for an instance of your object.
Ultimately, you're going to need to resolve the URL pattern for an object instance somehow. You may as well use the built-in mechanisms to do so, instead of re-inventing the wheel.
It sounds like perhaps you just need a bit of practice with Django's URL pattern matching, which is what get_absolute_url leverages. For what it's worth, I always work my patterns most to least specific and when/wherever possible, I namespace my URLs to avoid collisions.

Well its been days trying to find a solution and Im all out. I really dont understand whats going on with this app or what I ought to be doing with the urls. I tried using models, just to see if it works and it doesnt. I keep running into NoReverseMatch errors with get_absolute_url, even though I can see my posts just fine in localhost. So Im out of ideas. Hosting in general gets me so frustrated I want to quit...but quitting now would be admitting that I wasted all this time, so Im not quitting.
For comparing and contrasting, can you share with me your models.py, urls.py and views.py? Something that I am doing is clearly not in line with django medusa.
My models
from django.db import models
from django.db.models import permalink
from django_markdown.models import MarkdownField
from django.core.urlresolvers import reverse
from django_bootstrap_markdown.widgets import MarkdownInput

class Category(models.Model):
    # article = models.ForeignKey('models.Post', related_name="categories", related_query_name="category")
    category_title = models.CharField(max_length=140, default='DEFAULT VALUE', blank=True, null=True)
    def __str__(self):
        return self.category_title
    class Meta:
        verbose_name = "Category"
        verbose_name_plural = "Categories"

class EntryQuerySet(models.QuerySet):
	# This will be used for filtering everything that is published
	def published(self):
		return self.filter(publish=True)

class Post(models.Model):
    title = models.CharField(max_length=200)
    body = MarkdownField() # formerly MarkdownField(), and TextField()
    slug = models.SlugField(max_length=200, unique=True)
    publish = models.BooleanField(default=True)
    created = models.DateTimeField(auto_now_add=True)
    modified = models.DateTimeField(auto_now=True) # Keeps track of everytime it gets saved
    objects = EntryQuerySet.as_manager() #Im allowed to do this because of the QuerySet class

    def __str__(self):
        return self.title

    @permalink
    def get_absolute_url(self):
        """ The URL to this post in the detail-based view...or in other words,
        the format in which the URL of the blog post is presented. """
        # ::: mysite.com/entry/the-posts-respective-slug
        return reverse('entry', (), kwargs={'slug': self.slug })

My views.py
from django.shortcuts import render
from . import models
from django.views.generic.detail import DetailView # from the 16 min vid
from django.utils import timezone
from django.core.paginator import Paginator, InvalidPage, EmptyPage, PageNotAnInteger

class BlogDetail(DetailView):
    model = models.Post
    template_name = "post2.html"
    context_object_name = "post"

def blog(request):
	entries = models.Post.objects.all()
	paginator = Paginator(entries, 3)
	page = request.GET.get("page", '1')

	try:
		entries = paginator.page(page)
	except PageNotAnInteger:
		entries = int(page)
	except ValueError:
		page = 1
	except (InvalidPage, EmptyPage):
		entries = paginator.page(paginator.num_pages)

	# because index is out of the basic layout
	template = 'blog.html'
	context = {'posts': entries}
	return render(request, template, context)

My urls.py
from django.conf import settings # for development environment
from django.conf.urls import include, url
from django.conf.urls.static import static # for development environment
from django.contrib import admin
from blog.views import BlogDetail

url1 = url(r'^admin/', include(admin.site.urls))           # 1. url for "admin" page
url2 = url(r'^$', 'joins.views.home', name='home')         # 2. url for "joins" page
url3 = url(r'^markdown/', include('django_markdown.urls')) # 3. url for the "django_markdown" module
url4 = url(r'^blog/', 'blog.views.blog', name='blog') # 4. url for the "blog" page
url5 = url(r'^entry/(?P<slug>\S+)/', BlogDetail.as_view(), name="entry")

urlpatterns = [url1, url2, url3 ,url4, url5]

# For development environment
if settings.DEBUG:
	urlpatterns += static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
	urlpatterns += static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Sure, I'd be happy to help. I'm pretty slammed at the moment, but I'll do my best to take a look for you as quickly as possible.

@JellisHeRo Sorry to take so long to get back to you. Got slammed with some refactoring. Should have time to help this evening if you still need it.

Yes! please, i could really use it. I'd like to see your models, views, urls modules to compare with mine. Right now idk what else I need to do to get this working.

Ok, finally have some time to help.
First, I always do my URL patterns from most to least complex. This decreases the chance of colliding patterns, so I would do:
urlpatterns = [
    url(r'^markdown/', include('django_markdown.urls')),
    url(r'^entry/(?P<slug>[-\w]+)/$', BlogDetail.as_view(), name="entry"),
    url(r'^admin/', include(admin.site.urls)),
    url(r'^blog/$', 'blog.views.blog', name='blog'),
    url(r'^$', 'joins.views.home', name='home')
]

Make sure you terminate patterns with $.
I wouldn't allow any non-whitespace character for the slug. Instead I use [-\w]+ which is almost the same, but a bit more restrictive.
On the call to reverse, try:
class Post(models.Model):
    . . .

    @permalink
    def get_absolute_url(self):
        return reverse('entry', args=(), kwargs={'slug': self.slug })

As I'm apparently still watching this repo:
wrapping your get_absolute_url underneath the permalink decorator is not necessary - you're already doing what permalink does by calling reverse.
You haven't shown your medusa renderers - does the site work except for the the medusa stage, or are you still wrapping your head around Django itself? (if so, the official docs, the users mailing list, and the IRC channel are all good support options)

@alsoicode  thanks for the advice regarding the urls module. With your help along with kezabelle, I got rid of that error, and now Im facing a AppRegistryNotReady error. I dont rememeber encountering this kind of error before, but its a long traceback (let me know if you would like to see it); here is the renderers.py file that I used.
from django_medusa.renderers import StaticSiteRenderer
from blog.models import Post
from django.core.urlresolvers import reverse

class BlogPostsRenderer(StaticSiteRenderer):
    def get_paths(self):
        # A "set" so we can throw items in blindly and be guaranteed that
        # we don't end up with dupes.
        paths =["/entry/"]

        items = Post.objects.filter(publish=True).order_by('-created')

        for item in items:
            paths.append(item.get_absolute_url())

        return paths

renderers = [BlogPostsRenderer, ]

@kezabelle My issue is really medusa. I found a way to get around the get_absolute_url issue when I decided to just render my django app around mere markdown files instead of a models. When I run it, it shows and it works, from what I know. I am only trying the get_absolute_urls now because I just want to get a good idea on how Medusa works with url patterns on models since I cant seem to make it work for the urls and my markdown documents in a mere folder (assuming that Medusa can function that way). Getting rid of the permalink helped, though, so the initial error I had with the get_absolute_url is gone. Thanks.


sharded histogram filling with dask-histogram

Right now dask-histogram generates a complete copy of the histogram-to-be-filled on each worker node that wants to fill a histogram.
This means we are presently limited in histogram size by the memory available to an individual dask worker.
However, with #16 noted as a goal for the near term future, this means we can aggregate worker-node memory to have a larger upper bound on the size of histograms we can produce. This suggest yet another direction, which is very oriented towards HPC-like distributed computing, where we scatter the bins of the histogram across the memory of all workers in a dask-like clusters. This brings the histogram-size limit to its logical extremum, which is nearly the total aggregate memory of the accumulated dask cluster.
Aim would be to discuss how to achieve this. I personally think this is important since it means you can achieve multi-gigabyte histograms on 2 GB / 1 core provisioned worker nodes, and it stands to wildly flatten the advantage of the "big machine" analysis workflow.

+1

+1

+1


ia.fetch() only returns one image for U3V camera

Describe the Issue
I am trying to stream a U3V machine vision camera from SVS-Vistek. The call to ia.fetch() is contained in a loop which returns a single image. The second time through the loop ia.fetch() never returns and ends up timing out. At least this is what happens when using the transport layer file mvGenTLProducer.cti provided by MatrixVision. If I use the file sv_u3v_tl_x64.cti provided by SVS-Vistek, the first call to ia.fetch() fails due to "Invalid layout of buffer attached to chunk parser" exception.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Sample Code
I can show a piece of code that demonstrates the reported phenomenon:

[ X] Yes
[ ] No

If yes, please provide a sample code:
h = Harvester()
        # h.add_file('C:/Program Files/MATRIX VISION/mvIMPACT Acquire/bin/x64/mvGenTLProducer.cti')
        h.add_file('C:/Program Files/SVS-VISTEK GmbH/SVCam Kit/TLUsb/bin/sv_u3v_tl_x64.cti')

        h.update()
        for i in range(len(h.device_info_list)):
            print(h.device_info_list[i])
        ia = h.create({'serial_number': '66860'})

        width = ia.remote_device.node_map.SensorWidth.value
        height = ia.remote_device.node_map.SensorHeight.value
        ia.remote_device.node_map.AcquisitionMode.value = "Continuous"

        try:
            ia.start()
            i = 0
            done = False
            while not done:
                with ia.fetch(timeout=3) as buffer:
                    # I do some stuff with the image
                    fps = ia.statistics.fps
                    print("FPS: ", fps)

                    i = i + 1
                    # buffer.queue()
        except Exception as e:
            traceback.print_exc(file=sys.stdout)
        finally:
            ia.stop()
            ia.destroy()
            print('fin')
            h.reset()

If applicable, please paste the actual output (its whole traceback, etc) here:
This is the output using the transport layer from MatrixVision:
False
FPS:  0.0
False
Traceback (most recent call last):
  File "D:/Documents/WA/svf_testbed/harvester.py", line 83, in <module>
    with ia_left.fetch(timeout=3) as buffer:
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\harvesters\core.py", line 2386, in fetch
    raw_buffer = self._fetch(
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\harvesters\core.py", line 2262, in _fetch
    raise TimeoutException
_gentl.TimeoutException

This is the output using the transport layer from SVS-Vistek:
False
Traceback (most recent call last):
  File "D:/Documents/WA/svf_testbed/harvester.py", line 83, in <module>
    with ia_left.fetch(timeout=3) as buffer:
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\harvesters\core.py", line 2389, in fetch
    buffer = self._finalize_fetching_process(raw_buffer, is_raw)
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\harvesters\core.py", line 2320, in _finalize_fetching_process
    self._update_chunk_data(buffer=raw_buffer)
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\harvesters\core.py", line 2203, in _update_chunk_data
    self._chunk_adapter.attach_buffer(buffer.raw_buffer[:size])
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\genicam\genapi.py", line 2963, in attach_buffer
    self._adapter.attach_buffer(self._buffer, statistics)
  File "D:\opt\Anaconda3\envs\svf_testbed\lib\site-packages\genicam\genapi.py", line 3130, in attach_buffer
    return _genapi._ChunkAdapterU3V_attach_buffer(self, pBuffer, pAttachStatistics)
_genapi.RuntimeException: Invalid layout of buffer attached to chunk parser! : RuntimeException thrown (file 'ChunkAdapterU3V.cpp', line 95)
fin

Expected Behavior
I expect multiple frames to be grabbed from the camera and displayed using OpenCV (OpenCV display code was removed to keep this short).
Configuration

OS: [Windows 10]
Python: [3.8.13]
Harvester: [1.3.8]
GenTL Producer: [mvGenTLProducer.cti from MatrixVision] and [sv_u3v_tl_x64.cti from SVS-Vistek]
Camera: [SVS-Vistek SVCam exo183CU3 U3V camera]

Reproducibility
This phenomenon can be stably reproduced:

[X ] Yes
[ ] No.

This happens every time I run the code.
Actions You Have Taken

[X ] I've read the Harvester FAQ page.
[X] I have tried queuing the buffer with buffer.queue() as suggested in #270 with no effect.
[X] I have tried turning off Chunk mode as suggested in #337 using ia.remote.node_map.ChunkModeActive.value=False and ia.remote_device.node_map.ChunkModeActive.vale=False, but this value doesn't exist.

Additional context
I can stream the camera using a provided GUI from SVS-Vistek and their transport layer file.
There is a script provided by MatrixVision, that has the same behavior as my code using their transport layer file (I get one image then wait forever for the second).
When coding in C/C++ with the SVS-Vistek SDK, I have to declare the number of buffers at 4; I don't know if this relevant or how I would do this using harvesters.
Also, in the print out you will notice a couple of False prints. I put this statement in the fetch() method on line 2379 of core.py. It prints the result of self.thread_image_acquisition.is_running(), so the acquisition thread is not running.

Our camera also doesn't support Chunk transfers.  And ia.remote.node_map.ChunkModeActive does not exist.  Sticking with harvesters 1.3.2 until we can figure this out.
OS: windows 10
Harvester: 1.3.8
Genicam: 1.1.0
Python (Anaconda): 3.8.12
GenTL Producer: Critical Link
GenTL Viewer: 2.7.0
Camera: MityCAM-35MMFHDXS

In [5]: dir(ia.remote_device.node_map)
Out[5]:
['AcquisitionControl',
 'AcquisitionFrameCount',
 'AcquisitionFramePeriod',
 'AcquisitionFrameRate',
 'AcquisitionMode',
 'AcquisitionStart',
 'AcquisitionStop',
 'AdcBlkClamp',
 'AdcCdsGain',
 'AdcSelection',
 'AdcVgaGain',
 'AmpGain',
 'BadPixelReplacementEnable',
 'BadPixelReplacementMap',
 'DeviceControl',
 'DeviceFirmwareVersion',
 'DeviceManufacturerInfo',
 'DeviceModelName',
 'DeviceReset',
 'DeviceSFNCVersionMajor',
 'DeviceSFNCVersionMinor',
 'DeviceSFNCVersionSubMinor',
 'DeviceSelect',
 'DeviceSerialNumber',
 'DeviceVendorName',
 'DeviceVersion',
 'DigitalIOControl',
 'ExecRead',
 'ExecWrite',
 'ExposureTime',
 'FpgaVersion',
 'FrameCountMax',
 'Fx3Version',
 'Height',
 'HeightMax',
 'ImageFormatControl',
 'LineMode',
 'LineSelector',
 'LineSource',
 'LineStatus',
 'Max10CtrlVersion',
 'Max10Version',
 'MinFramePeriod',
 'OffsetX',
 'OffsetY',
 'PayloadSize',
 'PixelFormat',
 'RegAddress',
 'RegValue',
 'Root',
 'SensorHeight',
 'SensorPeekPoke',
 'SensorWidth',
 'SoftwareBuildDate',
 'TestControl',
 'TestPattern',
 'TestPendingAck',
 'TransportLayerControl',
 'TriggerActivation',
 'TriggerMode',
 'TriggerSelector',
 'UserOutputSelector',
 'UserOutputValue',
 'VSkip',
 'Voltage',
 'VoltageSelect',
 'Width',
 'WidthMax',
...


modify superserial to output bytes for unicode string to store in trie

The problem was: superserial serialize object to a JS string (could contain utf16 chars). However, the state could only store bytes. In our modified c runtime, JS string is flatten (into bytes) and stored, so it can be stored. However, when deserialize it back, it deserialize to bytes instead of original string. There's no way to tell if the original content is the flatten bytes or the utf16 string.
The repro test in this PR shows above fact. '👋'.length is 2 in JavaScript (two utf16 chars), but it is 4 in this repro test. Another serious problem, if one of two bytes flattened from the utf16 character is ", then it would unexpectedly end the string and cause deserialization fail.
I had patch node_modules superserial locally and that fix the problem. I still need to figure out the build and release as npm package (superserial is a deno package).  The fix is about checking if it contains utf16 chars, and serialize/deserialize differently for utf16 string and bytes: https://github.com/denostack/superserial/commit/7b73f21047cfd7444fde40ecd16095e1c3ff3648. I'm not very happy with the fix, if you have a good idea that would be great! @volovyk-s

With JSON.serialize/parse it behaves differently.
{"ExecutionError":"Smart contract panicked: unexpected end of input\\n    at <input>:1\\n    at parse (native)\\n    at get (build/lookup-map.js:158)\\n    at store_and_get_utf16_string (build/lookup-map.js:283)\\n    at store_and_get_utf16_string (build/lookup-map.js:299)\\n"}

With this commit it should fix the serialization of a serialized JSON string: https://github.com/denostack/superserial/commit/7b73f21047cfd7444fde40ecd16095e1c3ff3648, because it encode string as length + data, instead of looking for ". But as we decide to not using 3rd party serialization library in SDK, I'll close it


make the whole index page responsive

Description
I make the whole index page responsive. As you can seen in the attached video.

This PR does the following:

Updates ... The index.ejs page for responsiveness.

Related Issues
Link any related issues using the format Fixes #issue_number.
This helps to automatically close related issues when the PR is merged.

Placeholder: "Fixes #236 "

Screenshots (if applicable)
Add any screenshots that help explain or visualize the changes.
https://github.com/user-attachments/assets/16ce9d73-d352-44e2-9cfe-a5be23ceb841
Checklist
Make sure to check off all the items before submitting. Mark with [x] if done.

[x] I have performed a self-review of my code
[x] I have commented my code, particularly in hard-to-understand areas
[x] My changes generate no new warnings
[x] I am working on this issue under GSSOC

@Soujanya2004  Marge it. And don't forget to add all the labels as per the ISSUE #236

@Suvadip-sana conflicts

@Soujanya2004   Solved. Marge it.


Move Expression creation to ExpressionUtils

this PR is not backwards compatible, so no backport is provided

@johnktims @Shredder121 Is this good to merge?

It looks good to me.

I'll merge this then.


Data retrieval (queries) from the project server gets extremely slow for large data sets

🔍 Before submitting the issue

[X] I have searched among the existing issues
[X] I am using a Python virtual environment

🐞 Description of the bug
While working on the saf-osl-app-builder, we experienced that the performance of the data retrieval from the osl project server through pyosl drops considerably when the data set gets very large (> 4 MB).
For small data sets (smaller than around 3-4 MB), the time required to transfer the same data set through pyosl (e.g. osl_server.send_command(server_queries.full_subtree_status_info(root_node_uid), timeout=10000)) and through the optislang web services (e.g. requests.post(f"{BASE_URL}/get_subtree_status_info", auth=HTTPBasicAuth(NAME, PASSWORD), headers=HEADERS, json=input) with input = {"project_id": PROJECT_ID, "uid": root_node_uid}) are comparable:

For larger datasets (I tested up to 140 MB), the time required to transfer the data with pyosl grows nonlinearly, while it shows a linear growth with ows (as expected). This nonlinear growth leads to a huge performance drop with pyosl compared to ows:

This seems to be caused by the relatively small buffer size set in the osl_server.py (2^12). When I increased the buffer size to 2^16, the performance improved considerably:

📝 Steps to reproduce

Run a big osl project which produces many DPs (e.g. sensitivity_calc_with_sig_2.owa). In my tests, I ran the project through the ows (uploaded the .owa as a wizard, then started it from the wizard), so I did not need to register it as externally started in the ows for my comparison requests to ows.
Adjust port, host id, file path etc. in the following script and then execute it. It will continuously query the subtree status info for the root node through pyosl and through ows and log the time as well as the data size to two .csv files.

import time
from ansys.optislang.core import Optislang
from ansys.optislang.core.tcp import server_queries
import requests
from requests.auth import HTTPBasicAuth
import json

# Before starting this script, the optislang project needs to be started
# through the ows.
# Ideally do not monitor it actively and disable auto-updates etc. so the
# server is not busy when calling it.
# The osl project server is expected to run on localhost with port 5310.

PORT = 5310  # to be adjusted
HOST = "<IP_ADDRESS>"
PROJECT_ID = "10"  # to be adjusted
PROJECT_NAME = "sensitivity"  # to be adjusted
BASE_PATH = "C:\\temp\\pyosl_vs_owa_timings\\"  # to be adjusted
NAME = 'myowsusername'  # to be adjusted
PASSWORD = 'test'  # to be adjusted

BASE_URL = "http://<IP_ADDRESS>:5015"
HEADERS = {"user-agent-custom": "optiSLang_web_client", "Content-Type": "application/json; charset=utf-8", "Accept": "application/json"}

times_and_sizes_pyosl_large_file_name = f"pyosl_large_{PROJECT_NAME}.csv"
times_and_sizes_ows_large_file_name = f"ows_large_{PROJECT_NAME}.csv"

def get_subtree_project_status_info_ows(uid):
    input = {"project_id": PROJECT_ID, "uid": uid}
    response = requests.post(f"{BASE_URL}/get_subtree_status_info", auth=HTTPBasicAuth(NAME, PASSWORD), headers=HEADERS, json=input)
    return response.json()["message"]

osl = Optislang(port=PORT, host=HOST)
osl_server = osl.osl_server
root_node_uid = osl.project.root_system.uid

times_and_sizes_pyosl_large = open(f"{BASE_PATH}{times_and_sizes_pyosl_large_file_name}", "w")
times_and_sizes_ows_large = open(f"{BASE_PATH}{times_and_sizes_ows_large_file_name}", "w")

times_and_sizes_pyosl_large.write("Size, Duration\n")
times_and_sizes_ows_large.write("Size, Duration\n")

while osl.project.get_status() == "PROCESSING":

    start_time = time.time()
    data_large_ows = get_subtree_project_status_info_ows(root_node_uid)
    end_time = time.time()
    duration_large_ows = end_time - start_time
    print(f"duration for large dataset with ows: {duration_large_ows}")

    start_time = time.time()
    data_large_pyosl = osl_server.send_command(server_queries.full_subtree_status_info(root_node_uid), timeout=10000)
    end_time = time.time()
    duration_large_pyosl = end_time - start_time
    print(f"duration for large dataset with pyosl: {duration_large_pyosl}")

    file_large = open(f"{BASE_PATH}data_ows_large.json", "w")
    size_large_ows = file_large.write(json.dumps(data_large_ows))
    file_large.close()
    print(f"Size for large dataset with ows: {size_large_ows}")

    file_large = open(f"{BASE_PATH}data_pyosl_large.json", "w")
    size_large_pyosl = file_large.write(json.dumps(data_large_pyosl))
    file_large.close()
    print(f"Size for large dataset with pyosl: {size_large_pyosl}")

    times_and_sizes_pyosl_large.write(f"{size_large_pyosl},{duration_large_pyosl}\n")
    times_and_sizes_ows_large.write(f"{size_large_ows},{duration_large_ows}\n")

    time.sleep(5)

times_and_sizes_pyosl_large.close()
times_and_sizes_ows_large.close()

To check the influence of the pyosl osl_server buffer size, adjust the _BUFFER_SIZE in osl_server.py

💻 Which operating system are you using?
Windows
📀 Which ANSYS version are you using?
24R2
🐍 Which Python version are you using?
3.10
📦 Installed packages
ansys-optislang-core==0.7.1
certifi==2024.6.2
charset-normalizer==3.3.2
Deprecated==1.2.14
idna==3.7
psutil==5.9.8
requests==2.32.3
urllib3==2.2.1
wrapt==1.16.0

@rfahlberg This might help us a lot to improve the update performance of the saf-osl-app-builder for big cases, and considering the impact of the buffer size it might be easy to improve the performance for large data transfers considerable. However, I'm not aware of the full implications and possible drawbacks of increasing the buffer size beyond what's currently used.

Another interesting observation: In case of a busy osl project server (e.g. when other data requests are coming in from another client, or parallel tasks), the performance drop with pyosl is much more pronounced than with ows. This was unexpected - I'd expect a similar absolute performance "penalty" for both of them intuitively:

@clguenthe Great analysis, thanks. We should consider raising this buffer size. I wouldn't have expected such an impact. As for the other observation, are you referring to another client in a different python context or same context?

@rfahlberg For the "busy project server" case, I just kept the project open on the Status Overview page of the root node in the ows portal with "auto-update" active, so the owa was constantly querying for the "large data" data as well. So the client was running in a different context in this case.

@clguenthe Thank you for the comprehensive study. I was able to reproduce it. Basically, I got the same results, see figure below. @rfahlberg I would suggest to increase the buffer size. It should not have any side effect.


Compiling dawn error: set_property could not find TARGET update_mappings

Hi,
First of all thanks for this cool project! I've been following the Learn WebGPU, although it's still WIP it's been very useful so far! And I'm amazed by how quickly bugs got fixed. Previously I ran into the markupsafe missing error but I noticed that it had been already fixed.
I'm seeing another error right now while compiling dawn on Linux (NixOS) for Wayland:
CMake Error at webgpu/cmake/FetchDawn.cmake:77 (set_property):
  set_property could not find TARGET update_mappings.  Perhaps it has not yet
  been created.
Call Stack (most recent call first):
  webgpu/CMakeLists.txt:21 (make_dawn_available)

I see that it was due to 	set_property(TARGET update_mappings PROPERTY FOLDER "External/GLFW3")  in webgpu/cmake/FetchDawn.cmake but I'm not sure how to fix it. Could you please help? Thanks!

Okey I guess we can wrap this in some if (TARGET update_mappings) because the existence of the target may depend on the OS!

@eliemichel Thanks for the tip! Build succeeded. Ran into another error at run time. Not sure if it's me or something else. I will open another issue if I couldn't figure it out myself.

Well, during cmake . -B build, I saw the following log:
-- Dawn build Wayland support: OFF
-- Dawn build X11 support: ON

Does it mean wayland support for Dawn is turned off?

Did you check whether other vulkan-based program run correctly on your system?

@eliemichel You are right. I pulled in some vulkan dependencies and now it's working. Instance created. Surface created. Adapter requested with non-zero value. There are only two warnings, which I think are probably fine.
Warning: terminator_CreateInstance: Failed to CreateInstance in ICD 0.  Skipping ICD.
Warning: terminator_CreateInstance: Failed to CreateInstance in ICD 3.  Skipping ICD.

And these are the Adapter features logged:
Adapter features:
 - 5
 - 6
 - 7
 - 4
 - 3
 - 1006
 - 1
 - 2
 - 1005
 - 8
 - 10
 - 11
 - 1002
 - 1004
 - 1007

Nice :) For the record, can you give some details about "pulling in some vulkan dependencies"? Anything worth sharing to other people in a similar situation that I could add to the guide?

@eliemichel Sure. I'm using NixOS and this is a flake-based project. My flake.nix is here. Basically it pins any dependencies that are needed for the particular project.
The Vulkan dependencies that I added are:
            # vulkan
            glslang # or shaderc
            vulkan-headers
            vulkan-loader
            vulkan-validation-layers # maybe?
            vulkan-tools

(Note: there are definitely unnecessary dependencies in my flake.nix but this is what works for me for now.)


Is this repo still maintained?

I get the below error when i run
yarn build:ui
Module build failed (from ../../node_modules/css-loader/dist/cjs.js):
Error: PostCSS received undefined instead of CSS string

Hi @samelamin,
Please use node v16 or v18 and ensure that you run yarn install before run yarn build:ui.


moved Filetype validation

filetype validation was triggering during edit mode. Moved this validation to the bind event for when the file input changes instead of the text input.
Made a change in recordCreate where an error without a message (I saw it as an error with code 0) would throw a terminal error.

Last commit included a change that fixed a test that only fails locally.


Update to CMAK <IP_ADDRESS>

yahoo has released another version, which might fix the memory leak: https://github.com/yahoo/CMAK/issues/458#issuecomment-593802665
Release:
https://github.com/yahoo/CMAK/releases/tag/<IP_ADDRESS>

nice one!


refine help message; other minor refinement

fix #23
fix #22
fix #17

@xscript,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Codecov Report

Merging #26 into master will increase coverage by 0.09%.
The diff coverage is 93.33%.

@@            Coverage Diff            @@
##           master     #26      +/-   ##
=========================================
+ Coverage    78.3%   78.4%   +0.09%
=========================================
  Files          27      27
  Lines         710     704       -6
  Branches       90      88       -2
=========================================
- Hits          556     552       -4
+ Misses        129     127       -2
  Partials       25      25

Impacted Files
Coverage Δ

.../maven/webapp/handlers/JavaRuntimeHandlerImpl.java
64.28% <ø> (-4.14%)
:arrow_down:

...a/com/microsoft/azure/maven/webapp/DeployMojo.java
100% <100%> (ø)
:arrow_up:

...in/java/com/microsoft/azure/maven/FTPUploader.java
44.26% <100%> (-2.62%)
:arrow_down:

...osoft/azure/maven/webapp/DeployFacadeBaseImpl.java
74.07% <100%> (+2.64%)
:arrow_up:

...azure/maven/webapp/DeployFacadeImplWithCreate.java
100% <100%> (ø)
:arrow_up:

...crosoft/azure/maven/webapp/AbstractWebAppMojo.java
88.23% <100%> (+0.73%)
:arrow_up:

...a/com/microsoft/azure/maven/AbstractAzureMojo.java
65.57% <66.66%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e1b3d6f...7eb9178. Read the comment docs.


feat(types): add IPublicTypeInstanceOf to prop-types

fixed https://github.com/alibaba/lowcode-engine/issues/2644

1.2.3-beta.1