wrong semantics on lookup vindex for insert ignore and upsert queries

When handling INSERT IGNORE or INSERT ... ON DUPLICATE KEY UPDATE ... queries on a table with a lookup vindex, vtgate fails the operation if an entry already exists in the secondary index table.
Instead I think the right behavior should be:

For INSERT IGNORE it should succeed even if the secondary vindex query fails for any reason.
For upsert queries where the fields in the secondary index are not included in the update clause, it should try the insert but ignore errors if the key already existing in the table.
For upsert queries where the fields in the secondary index are included in the update clause, it should probably pass through a sensible update statement but this also seems quite unlikely so IMO it seems ok to just fail to handle this case for now.

Design notes
There are actually three possible combinations beyond the simple INSERT:

INSERT IGNORE: This statement ignores 'all' errors including dup keys, which is usually not what the app intends.
INSERT...ON DUPLICATE KEY: This statement performs an update if there is duplicate key, but returns an error if the update encounters dup keys.
INSERT IGNORE... ON DUPLICATE KEY: This statement attempts to insert. On dup key, it attempts to update. If the update has dup key errors, then it ignores the failures.

We can disallow construct 3. There shouldn't be many use cases for it, and app-level workarounds can be coded if such a need really arises.
CAVEAT: The solution below opens a hole in the owned vindex design: It will allow the possibility of multiple owner rows for a single vindex entry. If this happens, deleting one of the owner rows will render the other owner orphaned. The use cases that are requesting this feature will generally not do this. But it's a caveat to keep in mind. The reason why we should still allow this is because the lookup vindex is mainly a convenience feature that allows you to simplify the app. It's not meant to be an integrity enforcer in the relational sense.
INSERT IGNORE
For the first construct, the following changes need to be made:

For unowned vindexes with values supplied: we perform a sanity check that the supplied values map to the target keyspace id. With the ignore construct, we can return success with 0 rows affected.
For unowned vindexes with no values supplied: we try to compute the value if a reverse map is available. If this fails, we can return success with 0 rows affected.
For owned vindexes: We try to create an entry. If this fails, then the code needs to switch to behavior 1.

The additional complexity is that the inserts may have multiple values. If so, the rules should be individually applied for each row, but the whole operation still needs to be atomic. For example, the first row may need a vindex row to be created, but the second row may see a duplicate, but the sanity check could fail. If so, the previously created vindex must be rolled back. This can be achieved with the following steps:

Perform the sanity checks for rows that are already present.
Compute which rows are absent (may be derive it as side-effect of step 1).
Insert just entries for the rows that are absent.

If any failure is encountered while performing the above, vtgate can fail the entire insert ignore and return 0 rows updated.
As a future improvement, we could remove the insert entries that generated errors and just send the ones that succeeded.
Additional possible improvement: MySQL ignores all errors on INSERT IGNORE. We could instead ignore only data errors, but return failure on other system-level errors.
INSERT...ON DUPLICATE KEY
For this construct, we do everything that INSERT IGNORE performs. However, we additionally have to verify that the the dup key statements don't update the vindex columns. This rule is already enforced for all updates, and should be enforced the same way. This will be detected at the time of analysis and the failure will be before any statements are executed.
Possible improvement: For this construct, we should only ignore data errors. Other system errors will need to be returned back to the client.

There are a few more corner cases to worry about:

After a successful insert in the lookup, the main insert can fail for reasons other than dup key. If so, no error will be returned, but the lookup entry will survive without an owner row. This will cause subsequent regular inserts to chronically fail. However, an INSERT IGNORE that succeeds will fix this inconsistency. The other way out is to manually drop the row.
Behavior for unique and non-unique lookup may have to diverge: In the case of a unique lookup, a key being present but the value not matching means that we want to skip the insert. In the case of a non-unique lookup, we need to create a new association and proceed with the insert.

This further encourages us to redefine the meaning of INSERT IGNORE to return an error if the failure is anything other than a dup key error. If this happened, the distributed transaction watcher can detect partial executions and abort such transactions, thereby preventing dangling rows.

After some more study, I think an UPSERT would be a challenge to implement in a 100% MySQL compatible way. But we can come up with a decent trade-off that will work as intended for practical use cases:

For a table that owns a vindex, the upsert should try to create an entry for the lookup vindex. On duplicate key, we have to verify that there is no routing conflict: there must exist a route from the vindex key to the keyspace id of the row. If such a route does not exist, then the request is inconsistent, and we must not route that part of the request. This means that no rows will be affected. Contrasting this with MySQL, this is equivalent to the case where a table has multiple unique keys; MySQL will choose one of those rows and update it.
As mentioned previously, if an entry already exists for an owned vindex, and the keyespace id matches, then we can route that statement to the correct shard. However, it is possible that the UPSERT would eventually insert a row instead of updating one, which will cause two rows to own a vindex entry. We've decided that allowing this is an acceptable trade-off.
For an unowned vindex, if the keyspace id doesn't match, we'll return success with 0 rows affected, just like case 1. If the keyspace id matches, then we route the query to that shard.

Simply put, a row can have only one keyspace id. If an operation can result in this rule being broken, upserts will not be forwarded and will instead return 0 rows affected. Should we instead return an error?
I'll post a separate brainstorm on how to handle multi-value upserts on the vttablet side.

For the vttablet I continued @demmer's previous work:

If a table has only one unique key, then the query is passed through. Also, multiple rows are allowed.
If the updates modify any of the primary key columns, then those values are also added to the stream comment.
Rewrote VALUES(column) handling to compute and print out row-wise values for multi-row upserts.
In the case where a table has more than one unique key, we continue with the plan that breaks it up into two statements. We allow only one row in such cases. Previously, the generated update used a Resolved field, but the new implementation uses a custom formatter instead. Changes can be previewed here https://github.com/youtube/vitess/pull/3042.

To implement the functionality on the VTGate side, the vindex API needs to be extended: A lookup vindex will need to support a CreateIgnore function. This function will create what it can and will not return errors on dup key.
The Verify API will need to be changed to return a []bool (instead of a single bool), one for each association. This will allow VTGate to decide which rows must be dropped.
While inspecting lookup, I found a minor bug in its Verify function that would have caused it to falsely fail some multi-row inserts: If identical pairs were sent as input, the number of results returned will be less than the input length (because two identical conditions matched still returns only one row), which would cause it to think that everything didn't verify. This is a very uncommon use case, which is why no one has seen it yet.
This can be fixed one of two ways:

Perform the verification one row at a time. This will be the most correct, but could slow down multi-row inserts.
Pull all results at once, build a map and performs a hash comparison. This will be fast. However, hash comparison is lexical and will not work correctly for text columns.

For now, I'll go with option 1 for the sake of correctness. There are ways to address issues raised in option 2, but the solutions are more elaborate. So, we'll have to tackle that separately.


Use GitHub Actions for python-client (instead of Jenkins)

I really dig these github action workflows! The declerative style and YAML syntax remind me very much of Azure DevOps. (I do genuinely appreciate Microsoft's engineering in this DevOps space 😜)


Functionality for getting duplicated coordinates

I'd like to follow up on this discussion from pypwsqc as this is also relevant for CMLs.
We should implement a function to identify duplicated coordinates. This is an issue with Netatmo PWS when users don't allocate the correct location of their PWS in the web interface. This is quite frequent in PWS data and indicates that all PWS with duplicate coordinates are not placed correctly. In this case these PWS need to be removed. For CMLs, it might be of interest to find multiple CMLs on one tower for example. In this case, the IDs of these CML should be returned.
So far, the easiest option to identify duplicated coordinates is using pandas.DataFrame.duplicated but apparently there's nothing similar available in xarray...
Therefore I suggest to write a function that identifies duplicated coordinates and either returns indices or discards these entries in a data set.

Yes thanks, this sounds good, To aspects for further discussion:

Should we transform/backtransform from xarrayto pandas (which is tempting but might also bear some risks..) or try to figure out something that works in xarraydirectly?
I think it's safe to set the keep=False option as default for our purposes because then all duplicates coordinates are returned (otherwise the first or last duplicate is kept).

Here's somethings that seems to work.  I added an option to explicitly select the coordinates and id dimension by names if a data set is not according to our OS convention. In the end this is a 2-liner, does this justify a function?
ddef get_duplicate_coordinates(ds, id=id, keep=False, lat='lat', lon='lon'):
    # lat and lon can be set if a dataset does not follow our OS cenvention
    df = ds.id.to_dataframe() # assumes that there is an 'id' dimension

    # maybe check if we use a CML (line) or PWS dataset, to be discussed

    # return a boolean series which can easily be used for indexing in the initial `ds`
    return df.duplicated(subset=[lon, lat], keep=keep)
    # maybe we could transform the output into a xr.DataArray with the correct dimension
    # to make it 100% clear what is what

In the end this is a 2-liner, does this justify a function?

Yes, that is a good question. It could even fit on one line... Hence, I am not sure how to proceed. Maybe adding this to notebook with "xarray and pandas recipes" is the best option.

I added an option to explicitly select the coordinates and id dimension

Please note, that you are not using the variable id from your argument inside the function. You would have to do something like
def foo(ds, id_var_name='id'):
    ds[id_var_name].to_dataframe()
    # and so on...

Thanks for pointing this out. I'm also away next week, let's see what happens in the next meeting...

@JochenSeidel I think we can close this because you show the code to do it in #55. Is so, please close.


Add rand(numRows: Int, numCols: Int) functions

What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
add rand(numRows: Int, numCols: Int) functions to DenseMatrix object,like breeze.linalg.DenseMatrix.rand()

we can use it to replacebreeze.linalg.DenseMatrix.rand(numRows: Int, numCols: Int)

Can one of the admins verify this patch?

Doesn't that need a JIRA, test and PR descriptio maybe?

I add test : https://github.com/apache/spark/pull/14424

I guess it might be ncer if the contributions you make follow https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark.just like other contributions.

@HyukjinKwon Thank you.
This is my first time to push request to spark, Sorrry, I will follow the https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark  later.

Please don't open other PRs for related changes. Usually, you also need a JIRA. Although I understand the use for these methods, is there any use for them in Spark? we probably wouldn't add these things unless Spark itself needed them a few places, or else it was clearly a common task. I am not clear these are.

@srowen sorry, please close the issue. I will learning more  before next PR. The PR is only because breeze have the function. In spark ,there is no use for them.
Could you tell me some issue for starter? Please.

@xubo245 you need to close the issues, we can't do so directly.

ok


Feedback on diskette configuration

Describe the bug
Projects that have upgraded to cmsplugin-blocks>=1.3.0 (currently we are pinned on v1.2.0 but the latter versions are compatibles) will have a wrong diskette definition and their dump will fail to dump.
Note: This is not a bug only related to cmsplugin-blocks, it can happen with any other app. We just only faced with cmsplugin-blocks because of some project using a more recent release.
Also there are some cms plugin apps that are not marked to depend from djangocms, so they would be ordered to be loaded before djangocms, this will lead a loading error (at least, it can lead to other problem outside of diskette). These apps are:

djangocms-audio
djangocms-video

Environment
Describe your environment:

Plateform: Any
Python version: Any
Bireli version: >=0.3.12

To Reproduce
Steps to reproduce the behavior:

Upgrade to cmsplugin-blocks>=1.3.0
Create at least a plugin from cmsplugin-blocks (empty dump won't raise issue)
Dump data
Try to load data
It fails on a message alike "Integrity error: ForeignKey missing"

Expected behavior
I've thought that explicit app model definition for diskette would be better but i think it would lead more often to failures than with relaxed definition (just giving the app name) for majority of apps. Sometime apps may have breaking change that bring an undesirable model but it is probably to happen more rarely than legit new models.
All app configurations should be reviewed to be relaxed and only known undesirable model should be excluded and we stop to explicitly define all app models.
Additional context
I've reported an issue on Diskette https://github.com/emencia/diskette/issues/9 for a check command that would have helped to see this kind of issue.

Also Diskette v0.4.0 has been released and it now involves django sendfile, its settings has to be fixed for the right paths.

Bireli 0.4.0 includes now all fixes from feedbacks


AUR not Supported

The AUR (Arch User Repository) is a major feature for arch users (and kinda analagous to ppas, which are already supported in chef), and I see no reason why they should not be supported natively by the chef client.

closing due to lack of interest in cleaning up the associated PR


Runtime error when paths are provided in .swcrc

I am trying to add path aliases through .swcrc by specifying paths
Here is my .swcrc
{
  "jsc": {
    "parser": {
      "syntax": "typescript"
    },
    "target": "es2021",
    "baseUrl": ".",
    "paths": {
      "@connectors/*": ["./src/connectors/*"],
      "@operations/*": ["./src/operations/*"]
    }
  },
  "module": {
    "type": "commonjs"
  }
}

When paths are specified and I attempt to use the path-alias for an import, the build is successful but I get the following runtime error
 Error: Cannot find module '../../../../../../../../../../.cache/bazel/_bazel_victor/b76e982e01c6243367a3fb2de72a73de/execroot/[redacted]/bazel-out/k8-fastbuild/bin/backend/packages/grpc-error/index'

grpc-error is one of our other packages built with the bazel js-library rule.
It seems like "paths" is actually working, because it allows me to use the path-aliases, the build doesn't crash. However the resulting import path isn't valid, the file doesn't exist, and I am guessing there is some bazel magic I don't understand going on in the background. The resulting import path when paths is not specified and I don't attempt to use the path-alias looks like this @[package-name]/grpc-error, which is the name of the library.
To summarize
//.swcrc
{
...
    "paths": {
      "@connectors/*": ["./src/connectors/*"],
      "@operations/*": ["./src/operations/*"]
    }
...
}

results in
var _grpcError = require("../../../../../../../../../../.cache/bazel/_bazel_victor/b76e982e01c6243367a3fb2de72a73de/execroot/energy_services/bazel-out/k8-fastbuild/bin/backend/packages/grpc-error/index");

and a run-time error
and
// .swcrc
{
...
    // "paths": {
    // "@connectors/*": ["./src/connectors/*"],
    // "@operations/*": ["./src/operations/*"]
    // }
...
}

results in
var _grpcError = require("@ingka/grpc-error");

As with the TypeScript compiler, the paths configuration is just to help it resolve paths at compile-time. In order to run the resulting code, the runtime also has to know where the paths should resolve.
I think the JS code with all the ../.. imports is not desirable. What JS imports do you want to have at runtime?
If you could make a minimal repro (especially as a new folder under examples/ here in this repo) then I'm happy to understand more.
Note SWC has a bug https://github.com/swc-project/swc/issues/3028 but from staring at your output path I don't think this bug is involved.

The resulting path I want is var _grpcError = require("@ingka/grpc-error");, I don't expect swc to change this path since it is not in paths.
I also expect
import {
  grpcBadRequestError,
  grpcNotFoundError,
  grpcUnauthenticatedError,
  grpcUnhandledError,
} from '@ingka/grpc-error';
import { BadRequestError } from '@connectors/@types/errors/BadRequestError';
import { NotFoundError } from '../../../connectors/@types/errors/NotFoundError';
import { UnauthenticatedError } from '../../../connectors/@types/errors/UnauthenticatedError';

to transpile to
var _grpcError = require("@ingka/grpc-error");
var _BadRequestError = require("../../../connectors/@types/errors/BadRequestError");
var _NotFoundError = require("../../../connectors/@types/errors/NotFoundError");
var _UnauthenticatedError = require("../../../connectors/@types/errors/UnauthenticatedError");

instead of
var _grpcError = require("../../../../../../../../../../.cache/bazel/_bazel_victor/b76e982e01c6243367a3fb2de72a73de/execroot/energy_services/bazel-out/k8-fastbuild/bin/backend/packages/grpc-error/index");
var _BadRequestError = require("../../../../../../../../../../.cache/bazel/_bazel_victor/b76e982e01c6243367a3fb2de72a73de/sandbox/linux-sandbox/7526/execroot/energy_services/backend/electricity-subscription/services/-partner-svc/src/connectors/@types/errorsBadRequestError");
var _NotFoundError = require("../../../connectors/@types/errors/NotFoundError");
var _UnauthenticatedError = require("../../../connectors/@types/errors/UnauthenticatedError");

It feels like swc is resolving the paths based on a different baseUrl than I specify in .swcrc. "." in the bazel sandbox environment becomes something different than I expect. I expect it to be the root of my project folder (one level up from src), but in reality it becomes the root of the sandbox environment, and the folder structure in that environment is different both than my project structure, and the output folder. I hope this makes sense, I feel like I am having trouble explaining what I mean.
I'll try to create a repro to illustrate the problem.

We are just rolling out a new ruleset, https://github.com/aspect-build/rules_js which is specifically to solve problems related to nodejs or nodejs tools behaving differently when they see Bazel's filesystem layout. Take a look at the README there for a quick explanation.
I suspect the problem is solved there - looking forward to trying it on your repro :)

Hey, there is a repro at https://github.com/GV94/rules_swc/pull/1
Not sure if this is the preferred way of doing this, this is my first ever repro. Let me know if I should do it differently or if you have any other issues @alexeagle

fwiw we're running into the same issue when trying these rules, and the repro seems representative of our setup.
@alexeagle how do rules_js help rules_swc? as far as i can tell, the issue is what @GV94 pointed out: when SWC rewrites the paths it seems it ends up using some unexpected location for the baseUrl. is rules_js supposed to help rules_swc run in some other way?

thanks for the repro @GV94
Bazel is just running swc like so:
$ bazel build //examples/paths:transpile --subcommands
...
BAZEL_BINDIR=bazel-out/k8-fastbuild/bin \
    SWC_BINARY_PATH=../../../external/default_swc_linux-x64-gnu/swc.linux-x64-gnu.node \
  bazel-out/k8-opt-exec-2B5CBBC6/bin/swc/_cli_launcher.sh --source-maps false --config-file examples/paths/.swcrc examples/paths/src/index.ts --out-file examples/paths/src/index.js -q

We're not doing anything funny there, and if you cd into that folder and run SWC you don't get the strange path.
That leaves bazel sandboxing as the difference - when bazel spawns an action, the default strategy is to run within a filesystem sandbox, trying to help you discover non-hermeticity when the action depends on some undeclared input. To confirm, you can add --spawn_strategy=local to your bazel run. You'll see the imports are written the way you expect.
So I think this is simply an SWC bug when the input files are symlinks, which is already reported: https://github.com/swc-project/swc/issues/4057

Note that using --spawn_strategy=local turns off sandboxing for ALL actions in your build, which is too broad. I added a commit on top of your repro demonstrating a nicer fix, just adding the no-sandbox tag to all swc transpile actions.
https://github.com/aspect-build/rules_swc/pull/33

@alexeagle Thanks for the response. I agree, seems like that bug is the cause of all this.
Yes, feel free to use my commits in any way you please.

Do you mind signing the CLA over on that PR? I think for legal I could just point to your approval on this thread, but that would be easier for me :)

@alexeagle I think I already did?


The app is crashing when accessing date object

How frequently does the bug occur?
Sometimes
Description
In our app we have a model class Show which is similar to the following:
final class Show: Object {
...
...
   @objc dynamic var server_date_refreshed: Date?
...
...
   func expired() -> Bool {
        if let server_date_refreshed = server_date_refreshed, server_date_refreshed.timeIntervalSinceNow > -Show.ttl {
            return false
        }

        return true
   }
...
...
}

In the stacktrace, you can see that line number 43 is pointed out in Show.swift which is where expired() is written above. Any help to figure out the possible solution to this issue would be really appreciated.
The frustrating part is, we are not able to reproduce this crash on our side but there are a lot of reports for these crashes on Firebase.
Stacktrace & log output
Crashed: com.apple.main-thread
0  Realm                          0x13e7e8 realm::Allocator::translate_less_critical(realm::Allocator::RefTranslation*, unsigned long) const + 68
1  Realm                          0x156124 realm::ArrayIntNull::init_from_ref(unsigned long) + 200
2  Realm                          0x15d478 realm::ArrayTimestamp::init_from_mem(realm::MemRef) + 48
3  Realm                          0x752d8 realm::ArrayTimestamp::init_from_ref(unsigned long) + 144
4  Realm                          0x1eb824 realm::Timestamp realm::Obj::_get<realm::Timestamp>(realm::ColKey::Idx) const + 100
5  Realm                          0x8ba0 objc_object* (anonymous namespace)::getBoxed<realm::Timestamp>(RLMObjectBase*, unsigned long) + 136
6  Stitcher                       0x19e170 LibraryRepository.loadShowsIfNeeded(userInitiated:) + 43 (Show.swift:43)
7  Stitcher                       0x199208 specialized closure #1 in LibraryRepository.sync(context:types:wasCompleted:block:) + 463 (LibraryRepository.swift:463)
8  Stitcher                       0x19b0f0 closure #1 in closure #1 in LibraryRepository.doGet(context:type:sinceDate:page:fetchDate:block:) + 313 (LibraryRepository.swift:313)
9  Stitcher                       0x1adae0 partial apply for closure #1 in closure #1 in LibraryRepository.doGet(context:type:sinceDate:page:fetchDate:block:) +<PHONE_NUMBER> (<compiler-generated>:4303919840)
10 Stitcher                       0x132c0 closure #1 in closure #1 in closure #1 in DB.updateData(dict:completion:) +<PHONE_NUMBER> (<compiler-generated>:4302238400)
11 Stitcher                       0x29d39c closure #1 in static NSThread.ensureMainThread(block:) + 19 (Thread.swift:19)
12 Stitcher                       0x107a04 thunk for @escaping @callee_guaranteed () -> () +<PHONE_NUMBER> (<compiler-generated>:4303239684)
13 libdispatch.dylib              0x24b4 _dispatch_call_block_and_release + 32
14 libdispatch.dylib              0x3fdc _dispatch_client_callout + 20
15 libdispatch.dylib              0x127f4 _dispatch_main_queue_drain + 928
16 libdispatch.dylib              0x12444 _dispatch_main_queue_callback_4CF + 44
17 CoreFoundation                 0x9a6f8 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 16
18 CoreFoundation                 0x7c058 __CFRunLoopRun + 2036
19 CoreFoundation                 0x80ed4 CFRunLoopRunSpecific + 612
20 GraphicsServices               0x1368 GSEventRunModal + 164
21 UIKitCore                      0x3a23d0 -[UIApplication _run] + 888
22 UIKitCore                      0x3a2034 UIApplicationMain + 340
23 Stitcher                       0x5734 main + 22 (AppDelegate.swift:22)
24 ???                            0x1d0044960 (Missing)

Can you reproduce the bug?
Not yet
Reproduction Steps
No response
Version
10.25.2
What SDK flavour are you using?
Local Database only
Are you using encryption?
No, not using encryption
Platform OS and version(s)
iOS 16, 15,14
Build environment
Xcode version: 14.0
Dependency manager and version: Cocoapods 1.11.3

Hi @ayush-ttn can you please update to our latest version of RealmSwift https://github.com/realm/realm-swift/releases/tag/v10.32.3, since 10.25.2 we had a lot of fixes on timestamp which could be causing this issue. After the update please report if you still seeing this crashes.

Thanks @dianaafanador3. Is there a way I can make sure the crash is fixed by updating the SDK because currently I am not able to reproduce this crash on my side but there are a lot of entries for it on Firebase from the users. I just want to make sure that if I release with the updated SDK, it will not be a shot in dark 😅. Hope this makes sense.

@ayush-ttn The only way we can assure this is fixed is that we have a repo case. But it is very hard to diagnose without taking a look at the code, and having more information about the conditions during the crash.

have same issue:
Crashed: com.apple.root.background-qos
0  libsystem_kernel.dylib         0x79c4 __pthread_kill + 8
1  libsystem_pthread.dylib        0x7434 pthread_kill + 268
2  libsystem_c.dylib              0x1ff64 abort + 164
3  Realm                          0x2d58d4 realm::util::terminate(char const*, char const*, long, std::initializer_list<realm::util::Printable>&&) + 10
4  Realm                          0x2d5bbc realm::util::terminate_internal(std::__1::basic_stringstream<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) + 260
5  Realm                          0x2d5a1c realm::util::terminate(char const*, char const*, long, std::initializer_list<realm::util::Printable>&&) + 328
6  Realm                          0x16b67c realm::ArrayIntNull::init_from_ref(unsigned long) + 224
7  Realm                          0x17286c realm::ArrayTimestamp::init_from_mem(realm::MemRef) + 32
8  Realm                          0x1f79bc realm::Timestamp realm::Obj::_get<realm::Timestamp>(realm::ColKey::Idx) const + 100
9  Realm                          0x9834 objc_object* (anonymous namespace)::getBoxed<realm::Timestamp>(RLMObjectBase*, unsigned long) + 94 (timestamp.hpp:94)
10 RealmSwift                     0xede8 protocol witness for static _Persistable._rlmGetProperty(_:_:) in conformance Date + 128
11 RealmSwift                     0x5de3c Persisted.get(_:) + 182 (PersistedProperty.swift:182)
12 RealmSwift                     0x5db88 static Persisted.subscript.getter + 126 (PersistedProperty.swift:126)
13 Me                             0x4efea0 RealmCallLog.toDictionary() +<PHONE_NUMBER> (<compiler-generated>:4378443424)

class RealmCallLog: Object {

    override static func primaryKey() -> String? {
        return "id"
    }
    override static func indexedProperties() -> [String] {
        return  ["id", "toRemove", "fullPhoneNumber", "phoneNumber", "name", "apiPhoneNumber", "type", "tag", "calledAt", "synchronized"]
    }

    @Persisted var id = UUID().uuidString
    @Persisted var phoneNumber = ""
    @Persisted var apiPhoneNumber: APIPhoneNumber = 0
    @Persisted var fullPhoneNumber = ""
    @Persisted var name = ""
    @Persisted var type = ""
    @Persisted var calledAt: Date = Date()
    @Persisted var userUUID: String?
    @Persisted var tag = ""
    @Persisted var synchronized = false
    @Persisted var toRemove = false

    func toDictionary() -> [String: Any]? {
        guard apiPhoneNumber != 0 else { return nil }
        var data: [String: Any] = [
            "phone_number" : apiPhoneNumber,
            "name" : (name.isEmpty ? nil : name) as Any,
            "type" : type,
            "called_at": calledAt.toAPIString()
        ]

        if tag == CallLogEntity.CallLogTag.identified.rawValue || tag == CallLogEntity.CallLogTag.spam.rawValue {
            // API Accepts only `identified` or `spam` values. For `regular` it should not be sent.
            data["tag"] = tag
        }

        return data
    }

@mr-grey I had updated my app to use 10.32.3 and after the release, the date object-related crashes were reduced. I recommend updating the realm in your project if you have not done already.

@ayush-ttn  thank you

@mr-grey let us know if after the update you stop getting this crashes.

@mr-grey do you have any update after updating to the latest version?

@mr-grey let us know if this issue still persist, and will proceed to reopen the issue.


Exposing all tenants name to users

I have an installation of OpenShift with Hawkular configured.
Just realized that any user could ask Hawkular to list which tenants are available and obtain the list of available projects (not just his project). This leaking of info does not seem appropriate.

Which version are you using? The /tenants endpoint is admin-protected for a while now, cf https://issues.jboss.org/browse/HWKMETRICS-478

OpenShift 3.5.0. Probably and old version of Hawkular.
Thanks

Actually, I'm surprised that users can see all tenants in OS 3.5.
Is it through the /tenants endpoint, or in a different way?

@mwringe any idea about it?

This is what I have executed:
$ curl -H "Hawkular-Tenant: MYPROJECT" -H "Authorization: Bearer xxxxxxxxxxx" https://hawkular-metrics.apps.xxx.com/hawkular/metrics/tenants
[{"id":"AAA"},{"id":"BBB"},{"id":"CCC"},{"id":"network-diag-global-ns-3z7v5"},{"id":"DDD"},{"id":"openshift-infra"},{"id":"_hawkular_admin"},{"id":"network-diag-ns-vpj5d"},{"id":"_system"},{"id":"logging"},{"id":"network-diag-ns-3vrkr"},{"id":"EEE"},{"id":"network-diag-global-ns-6bmt8"},{"id":"FFF"}]

Don't discard some error in my side, I'm quite new using openshift.

Ok, then I expect this call to work if and only if the provided authorization token is an admin one. Using some user, non-admin token here should result in a 'forbidden' answer.
Which means that this token/query should obviously not to be shared to other users.

Then token come from my user account.


pexamp.get_example_data('./') returns 404 error

I don't know if this function should be deprecated or updated since it returns a 404 error.

Sorry, somehow that data got lost.  It should work now.  Thanks for the heads up


router meta data support push change

What problem is this solving
In case of cms website, we suppose to defined router with parameter and defined menu from server side, when click the menu, the better way for change document title and breadcrumbs with the menu title not the router meta title;
so the better way to change the meta data is support router.push function with merge the meta data;
Proposed solution
router.push support the meta to defined customer meta value and merge into router meta
Describe alternatives you've considered
in my way, when start to init dynamic router, save menuMap to bind path with menu info, then change document title and breadcrumbs with menuMap[path] to get then title, but in some case like we just want to open a new page without a menu option, then this method will fail to support;
Please consider adding this capability!

This is what we call transient state (state available only during navigation) and is planned in a way that would align with the upcoming navigation API (https://github.com/WICG/navigation-api#example-using-info). It would require going through an RFC first though

Big +1 for transient state!

It has some useful applications (not the least being force reloading a component when the route doesn't change);
and it's incredibly difficult to work around, the router code goes to great lengths to "normalize" everything passed in the route and only selectively copies properties.

There are few places where you could pass "hidden" state, most likely params, but in most cases only known keys will passthrough. I say "in most cases" because the code around this is relatively inconsistent. For example if you're modifying the current route and pass no name, params doesn't get normalized (for unclear reasons, seems pretty inconsistent).


pdf completely rendered blank

Attach (recommended) or Link to PDF file here:
Configuration:

Web browser and its version:
Chrome Version 63
Operating system and its version:
Windows 7
PDF.js version:
Your actual Demo Project
Is a browser extension:

Steps to reproduce the problem:
1.
2.
What is the expected behavior? (add screenshot)

What went wrong? (add screenshot)

4149180-355989.pdf
Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):

Interesting. I have not seen the "too much recursion" exception before. Okular on Linux handles it just fine. The console log is:
Warning: getOperatorList - ignoring errors during task: RenderPageRequest: page 0
Warning: Unable to decode image: InternalError: too much recursion
Warning: Dependent image isn't ready yet
Warning: getOperatorList - ignoring errors during task: RenderPageRequest: page 1
Warning: Unable to decode image: InternalError: too much recursion
Warning: Dependent image isn't ready yet

This appears to be a "regression" from PR #6112, even though the underlying issue seem to be older.

@gHensges , Did the fix works for you? since I have same problem and it seem still not working fro me.
Thanks

@shiguanglu , Yes this fix solved my problem.


[GSSoC23] : Gesture Volume Control

💥 Proposal
It is a project that falls under Image Processing using OpenCV. It allows the user to control the system volume using our gestures.
Tech Stack : OpenCV, Python

Please assign me this issue under GSSOC23.

Please tell us which dataset you will be using.

Okay, thank you for informing.
Am closing it as it's already done.


RSDK-2021 - Add the cartographer AppImage

Ticket: https://viam.atlassian.net/browse/RSDK-2021
Done:

Adds orbslam3-module app image creation to Makefile & github workflow

Tested on:

[x] arm64
[ ] x86_64 linux

Ticket: https://viam.atlassian.net/browse/RSDK-2021
Done:

Adds orbslam3-module app image creation to Makefile & github workflow

Tested on:

[x]  arm64
[ ]  x86_64 linux

Would you mind also confirming that this works on mac?


get_string yields invalid UTF-8 string when input only contains low surrogate escape sequence.

Describe the bug
The comment of get_string says "The string is guaranteed to be valid UTF-8." However, if the input only contains low surrogate escape sequence, it may return a string that is rejected by its own simdjson::validate_utf8.
To Reproduce
#include "simdjson.h"
#include <iostream>

using namespace simdjson;

int main() {
  ondemand::parser parser;
  auto json = R"("\uDC00")"_padded;
  auto doc = parser.iterate(json);
  auto str = doc.get_string().value();
  std::cout << validate_utf8(str) << '\n';
}

The expected behavior of the code is throwing an exception at get_string().value() since \uDC00 is not really a valid escape sequence: valid surrogate pairs must consist of one high surrogate character (0xD800–0xDBFF) followed by one low surrogate character (0xDC00–0xDFFF), but it only has the low surrogate character.
The actual behavior is outputting a 0, since the result is not a valid UTF-8 string, both logically and in the eyes of simdjson::validate_utf8.
simjson release
v1.1.0 (commit 8c1bfe782b718d128fd655f2070971b52abfc9b0).
Configuration (please complete the following information if relevant)

OS: macOS Monterey
Compiler Apple clang version 13.0.0 (clang-13<IP_ADDRESS>) x86_64-apple-darwin21.4.0
Version: 12.3.1
Optimization setting: None

I am terribly sorry. The issue was sent to the wrong repository.


[BUG] Receiving duplicate messages during consumer rebalancing to new partitions

Describe the bug
I'm using EventHub build-in endpoint in IoT Hub to read device messages by backend service. (1 consumer group and 4 partitions).
When there is 1 consumer application, then everything works fine.
After scale up instances to 2, change partition ownership is starting. During this process, the consumer apps start receiving duplicate messages. After partition ownership changed, everything works fine. The problem is with duplication. The process of changing partition ownership takes some time and I got more than 150 duplicate messages during that time. Could you please advise how to avoid duplication?
Before I worked with Kafka and there the rebalancing process works very fast and I didn't have any problems with duplication on scaling.
Why EventHub still sending messages to consumer applications within one consumer group? Could you please suggest?
Expected behavior
The EventHub should not send the same message to all consumer applications within one consumer group
Actual behavior (include Exception or Stack Trace)
Receiving duplicate messages during consumer rebalancing to new partitions.
See logs

-- Instance 1
[20-12-18 22:40:37.37Z INF] Application started.
[20-12-18 22:40:38.38Z INF] Assigned partition: 1
[20-12-18 22:40:47.47Z INF] Assigned partition: 3
[20-12-18 22:40:57.57Z INF] Assigned partition: 2
[20-12-18 22:41:07.07Z INF] Assigned partition: 0
[20-12-18 22:44:51.51Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>64. Payload: {"EquipmentNumber":"3333333","CabinPosition":1}. Partition: 1.
[20-12-18 22:44:54.54Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>76. Payload: {"EquipmentNumber":"2222222","CabinPosition":1}. Partition: 1.
[20-12-18 22:44:55.55Z INF] Unassigned partition: 2
[20-12-18 22:44:59.59Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>84. Payload: {"EquipmentNumber":"7777777","CabinPosition":1}. Partition: 1.
[20-12-18 22:45:00.00Z INF] Unassigned partition: 1

-- Instance 2
[20-12-18 22:44:38.38Z INF] Application started.
[20-12-18 22:44:39.39Z INF] Assigned partition: 2
[20-12-18 22:44:48.48Z INF] Assigned partition: 1
[20-12-18 22:44:51.51Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>64. Payload: {"EquipmentNumber":"3333333","CabinPosition":1}. Partition: 1.
[20-12-18 22:44:51.51Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>76. Payload: {"EquipmentNumber":"2222222","CabinPosition":1}. Partition: 1.
[20-12-18 22:44:59.59Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>84. Payload: {"EquipmentNumber":"7777777","CabinPosition":1}. Partition: 1.
[20-12-18 22:45:01.01Z INF] Event State received from DevEmulator.dev. Offset:<PHONE_NUMBER>24. Payload: {"EquipmentNumber":"2222222","CabinPosition":2}. Partition: 1.

As you can see, Instance 1 started and has owned all partitions (1,2,3,4) .
Then, start the Instance 2 and start listening to Partition 1. At this time my Instance 1  and Instance 2 are listening to the same Partition 1. After Unassigned partition: 1  occurs in Instance 1, the problem with duplicates is gone.
To Reproduce

Create a simple EventHub consumer

_eventProcessor = new EventProcessorClient(
	storageClient,
	EventHubConsumerClient.DefaultConsumerGroupName,
	options.EventHubConnectionString,
	options.EventHubName);

_eventProcessor.ProcessEventAsync += ProcessMessage;
_eventProcessor.ProcessErrorAsync += ExceptionReceivedHandler;
_eventProcessor.PartitionInitializingAsync += AssignPartition;
_eventProcessor.PartitionClosingAsync += UnassignPartition;

where ProcessMessage is:
private async Task ProcessMessage(ProcessEventArgs eventArgs)
{
	EventData eventData = eventArgs.Data;
	string equipmentNumber = eventData.SystemProperties["iothub-connection-device-id"].ToString();
	string message = Encoding.UTF8.GetString(eventData.Body.ToArray());

	_logger.LogInformation(
		"Event State received from {EquipmentNumber}. Offset: {Offset}. Payload: {Payload}. Partition: {Partition}.",
		equipmentNumber, eventData.Offset, message, eventArgs.Partition.PartitionId);

	...

	await eventArgs.UpdateCheckpointAsync(eventArgs.CancellationToken);
}

Start sending messages
Run the second instance

Environment:

Name and version of the Library package used: Azure.Messaging.EventHubs 5.1.0, Azure.Messaging.EventHubs.Processor 5.1.0,
Hosting platform or OS - docker, linux
.NET runtime version 3.1
IDE and version : Visual Studio 16.8.3

Hi @Marusyk.  Thank you for your feedback and we regret that you're experiencing difficulties.   In this case, what you're seeing is the expected behavior when processor instances are scaling.
When a processor is starting up, it will attempt to balance the work for processing with other processors active for the same consumer group.  In the case where a processor isn’t able to reach its fair share of the work by claiming unowned partitions, it will attempt to steal ownership of partitions from other processors.  During this time, the new owner will begin reading from the last recorded checkpoint.  At the same time, the old owner may be dispatching the events that it last read to the handler for processing; it will not understand that ownership has changed until it attempts to read the next set of events from the Event Hubs service.

As a result, you will see some duplicate events being processed when processors are started or stopped which will subside when the processors have reached a stable state with respect to load balancing.  The duration of that window will differ depending on the configuration of your processor and your checkpointing strategy.

Could you please advise how to avoid duplication?

An important call-out is that Event Hubs has an at-least-once delivery guarantee; duplication is possible when consuming events regardless of the client used.
It is highly recommended that you ensure that your processing is resilient to event duplication in whatever way is appropriate for your application scenarios.
The most common technique is for publishers to inject a property that contains a unique identifier for the event, and consumers track the ids that have been processed for a rolling window of time, discarding duplicates.  Another common approach is to deploy a static set of EventProcessorClients that can handle the maximum expected load; those will still see duplication during the initial startup but won't see additional periods, as there is no scaling.
Some applications with extreme sensitivity to duplication will choose to bypass using the EventProcessorClient and instead use the PartitionReceiver,  managing checkpointing, load balancing, resilience, and fail-over at the application-level.  This approach shifts much of the responsibility handled by the processor to the host application, but allows the application to have full control.
It is also possible to implement a more robust method of load balancing that uses a central point of coordination or a peer-to-peer mesh approach by extending the EventProcessor<TPartition>,  though that would involve trade-offs around additional hosting cost, increased complexity, and lower throughput.
Just to mention again, any of these approaches would still be subject to the potential for duplication due to the service guarantee.

Thank you very much for such a detailed answer. Really appreciate it. Thanks


SSL/TLS required

My XMPP server is required to use SSL/TLS configuration. how do I configure xmpp_plugin to be ready for my server.
this my console:
I/System.out(22306): [socket]:check permission begin!
D/flutter_xmpp(22306):  Something went wrong while connecting ,make sure the credentials are right and try again:
W/System.err(22306): org.jivesoftware.smack.SmackException$SecurityRequiredByServerException: SSL/TLS required by server but disabled in client

thank you,
Best Regards.

I am also facing same issue i tried lot of stuff yet no luck, my same server is working with XMPP jabber client app,its responding but when I try with the given code here adding proper credentials I get an error-
W/System.err(26207): org.jivesoftware.smack.SmackException$SecurityRequiredByServerException: SSL/TLS required by server but disabled in client

@gnusagit @sid9890 Right now plugin supported only for the SSL connection and mapped jid.
I am expecting to make it usable for all of the options very shortly.

@gnusagit @sid9890 Right now plugin supported only for the SSL connection and mapped jid.
I am expecting to make it usable for all of the options very shortly.

thank you for your answer. I hope this feature can be implemented soon.
best regards.

@gnusagit try with version1.1.15

Hello @vavadiyahiren
thank you for your update the last few hours, it helped me a lot.
but I've try your example application from this git and still no luck. Is the change i have to do is just add a line of code below?
"requireSSLConnection": true,
"autoDeliveryReceipt": true,

or is there some other process i have to do?
because, even though I'm using the new version of this plugin, it still has a information in my log like:
D/flutter_xmpp(12803):  Something went wrong while connecting ,make sure the credentials are right and try again:
W/System.err(12803): org.jivesoftware.smack.SmackException$SecurityRequiredByServerException: SSL/TLS required by server but disabled in client

Thank you


ONNX Graduation

ONNX graduation proposal: https://github.com/apache/incubator-mxnet/issues/20063
This pr does 1, 2, and 4 of the proposed changes.
Also this pr adds a simple Readme.md under python/mxnet/onnx/ which we can later expand on with other prs

1.Graduate the MX2ONNX module (exporting to ONNX format) from the mx.contrib.onnx namespace to a regular and shallower directory (such as mxnet.mx2onnx).
2.Deprecate the ONNX2MX module (importing from ONNX format)
3.Only support ONNX 1.7, 1.8, and future releases for MX2ONNX.
4.Add a setup.py for MX2ONNX.
5.Up to date, accurate, and better documentation.

@szha @leezu would you help review this pr?

@josephevans @waytrue17 Can you please review the PR? Thank you.


Improve similar libraries links

Changed log

Improve similar libraries links and it can let developers/users access these source code project links easily (directly).

The links are referring to the same packages.


docs(deep-sleep-stub.rst): Added documentation about re-entering deep sleep from wake stub (IDFGH-12569)

This PR adds documentation to the deep sleep wake stub docs detailing how to re-enter deep sleep from the wake stub.

@Lupindakaas
Thank you for pointing out this flaw in the IDF deepsleep stub example! The configuration of the wake-up source in the example  is hidden in the call to the esp_sleep_enable_timer_wakeup function and sleep process in the main core app main, and then subsequent wake-ups in the stub inherit this enablement. If the wake-up source needs to be changed while the stub is running, other users is hard to obtain relevant information.
But the description in this file should be applicable to all esp chips. For some other esp32 series chips, the registers name and function have changed. Do you mind if I restart an update to this doc internally?

Hi, that would be awesome!


Key derivation not working

Getting an error during key derivation. Occurring on Ubuntu using Python 3.5.2.
2017-12-12 20:38:40,276 INFO: Deriving keys...
Traceback (most recent call last):
  File "nowallet.py", line 1065, in <module>
    main()
  File "nowallet.py", line 1029, in main
    wallet = Wallet(email, passphrase, connection, loop, chain)  # type: Wallet
  File "nowallet.py", line 293, in __init__
    create_root_keys(salt, passphrase)
  File "nowallet.py", line 214, in inner
    func(*args, **kwargs)
  File "nowallet.py", line 270, in create_root_keys
    t = derive_key(salt, passphrase)  # type: Tuple[int, bytes]
  File "/home/user/nowallet/nowallet/keys.py", line 38, in derive_key
    digestmodule=hashlib.sha256).read(key_length)  # type: bytes
  File "/home/user/.local/lib/python3.5/site-packages/pbkdf2.py", line 161, in read
    block = self.__f(i)
  File "/home/user/.local/lib/python3.5/site-packages/pbkdf2.py", line 173, in __f
    U = self.__prf(self.__passphrase, self.__salt + pack("!L", i))
  File "/home/user/.local/lib/python3.5/site-packages/pbkdf2.py", line 146, in _pseudorandom
    digestmod=self.__digestmodule).digest()
  File "/usr/lib/python3/dist-packages/Crypto/Hash/HMAC.py", line 211, in new
    return HMAC(key, msg, digestmod)
  File "/usr/lib/python3/dist-packages/Crypto/Hash/HMAC.py", line 105, in __init__
    self.outer = digestmod.new()
AttributeError: 'builtin_function_or_method' object has no attribute 'new'

Going to start using tox to test all supported versions of Python.
Probably going to use pycrypto here instead of hashlib.

So, I tried it on Ubuntu 17.10, which has Python 3.6.3
First off, separate issue, I noticed that running make init gives me:  RuntimeError: aiohttp requires Python 3.4.2+
even though I have 3.6.3. Is the make init trying to use Python 2.7 pip by accident?
To get around this, I had to install pip3 and then manually ran: pip3 install -r requirements.txt
This completed successfully, but I haven't looked at the make file to see if it still had stuff to do after that
Then I ran the following:
cd nowallet
python3 nowallet.py
That gets me as far as I got on Mint, without having to comment out ClassVar, but I still get the same digestmod.new() error, even with 3.6.3 on Ubuntu

Switched to pycrypto in 6fffd2e04889378c54c294e707b183791509f191 and got rid of ClassVar stuff because it wasn't correct anyway. Go ahead and do a git pull and see if it works for you. Still haven't got tox going yet.

README is looking better!
39d1ab4acefa2d2fedad2573a17124d37eccd923


Want to get list of roles in power bi

Hi,
My concern is I have MVC web application and PowerBI reports, I want to sync PowerBI Row level security list to my own database table. How it possible,, provide me an right solution.
https://docs.microsoft.com/en-us/power-bi/service-admin-rls

Closing as dup of https://github.com/MicrosoftDocs/powerbi-docs/issues/1284


Could you join efforts with processing-p5js-mode from Fathom Information Design?

Instead of having a separate IDE for P5.js, why not integrate P5.js as a new "mode" of standard Processing IDE like this project (https://github.com/fathominfo/processing-p5js-mode) does?
Thanks

I'm a bit confused, there is already a p5.js mode for Processing.

To give further background, we are actually planning to stop support of this editor once we launch the new web editor we've been focusing our development energy on. I see the p5.js mode for Processing as a good holdover or alternative until that is ready, but they are separate projects.

My wish would be having one tool "to rule them all". One Processing IDE with "Java mode", "Android mode" and "Javascript/P5.js Mode" would be excellent: desktop, mobile and web all in one

I understand. However, p5.js and Processing are separate projects. While we talk frequently about shared goals between them, we'd prefer not to merge them but to instead give each project the space to grow and develop independently and organically.


Make 'pyleus build' faster

@patricklucas:

Waiting for a topology to build during development is tedious, let's try to speed it up.

Re-use pyleus_venv from previous build if requirements.txt hasn't changed
Otherwise cache wheels (curdling?)

Moved from PYLEUS-35

This is definitely frustrating. Have a small typo, build takes a few minutes and then local dies.
@poros is there a timeline for exploring this more?

@karan there is no timeline afaik (I'm not actively working on the project anymore). So feel free to have a stab at it :)


Azure Data Tables readme and samples issues

Section link:

Suggestion:
Add " after the myTable. Update to "myTable"

Section link:

Suggestion:
Add ' after the product_name. Update to b'product_name'

Section link:

Suggestion:
Update RowKey to PartitionKey. Because in the code snippet "Creating entities", PartitionKey ='RedMarker', RowKey= '001234'.

Section link:

Reason:
Wrong link.
Suggestion:
Update the link to https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/tables/azure-data-tables/samples/sample_query_table.py and https://github.com/Azure/azure-sdk-for-python/blob/master/sdk/tables/azure-data-tables/samples/async_samples/sample_query_table_async.py.

Section link:

Suggestion:
Update to "Table could not be found".
@jongio for notification.

The issue is in the wrong place. close it.


net/url: QueryEscape escapes its own escape characters

What version of Go are you using (go version)?

$ go version
go1.19.2

But really, affects 1.18 and other versions too
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
It's universal anywhere, since it's a bug in logic implementation of internal shouldEscape method
What did you do?
Playground link: https://go.dev/play/p/WXXx4EiSy_E
What did you expect to see?
Already escaped string should not be escaped again (example behavior: Javascript encodeURIComponent method and similar)
What did you see instead?
It gets escaped as many times as the method is called.

The + sign carries semantic meaning within url query escaping, in general it won't be possible to know if it has been escaped before.
Closing as working as intended.

It also escapes % sign as shown in the playground, it is not working as intended.

The JavaScript encodeURIComponent function also escapes a % character in its input string, encoding it as %25.
We aren't going to make any changes here.


A next generation CLI

Maintaining a fully featured single application generator has proven very difficult and time consuming. Every new feature or prompt adds an exponential growth in code and test complexity. Templates are difficult to customize since they are buried in a third party module and difficult to maintain, especially when having to account for all kinds of customizations somebody would want to do.
A more flexible approach is used by hygen.io which puts generator templates right into your application. If you want to change it, edit it to what you need and all future generator calls will reflect it.
To further improve modularity, each Feathers module will ship with the templates that can be used to generate its integration in a standard JavaScript/TypeScript Feathers setup. You can either just run it or eject the templates into your own application to customize.
For example
feathers generate authentication

Will look for templates that can be generated in the following modules:

./_templates/authentication (locally)
@feathersjs/authentication/_templates
feathers-authentication/_templates
authentication/_templates

Templates can be ejected into your application with
feathers customize authentication

This will allow any third party module to provide templates to generate its integration into a standard Feathers application setup and any user to customize what is being being generated to fit their own application needs (like creating templates that initialize a common set of protection hooks when generating a new service).
The steps necessary to allow this would roughly be

[ ] Port the existing generator templates to be usable as a Hygen generator so that v4 apps can still be supported
[ ] Use Hygen programmatically in the Feathers CLI and expand the template lookup mechanism to the one described above
[ ] Update the templates to v5 and put them into their individual modules

@daffl we should define a clear action plan for migration and rewrite. As I said in the other issue, I'll try to come up with something in the morning.
Is there a project board for the feathers somewhere?
On a semi-related note: Are you planning to merge my CLI enhancements, or would you rather keep things as-is and add this functionality indirectly through use of hygen templates?

Some observations I made already:

Feathers generator (:any) does not respect settings in eslintrc files

Example: See this PR's description

Feathers generator (:app) automatically adds static files like .gitignore, .editorconfig which might be unwanted
Feathers generator (:service) does not differentiate between collections and entities correctly

Example: Generating a service "users" results in the collection resource users being registered correctly, the corresponding model and class should be called User though (capitalized singular form)

List is not final

New CLI will be moved to the main repo and is currently being worked on in https://github.com/feathersjs/feathers/tree/hygen-cli
In general, the templates will provide a very minimal setup that can be fully customized to anything you want by running something like feathers customize app (or feathers customize authentication etc.) and ejecting the module specific templates into your local project.


Words in excluded_words being flagged as typos

Hi @MikeRogers0,
Not sure if this is just me configuring it incorrectly.
I'm using the following in my repos, located in .github/.typo-ci.yml, with the TypoCI GitHub App (not within a workflow job):
dictionaries:
    - en
    - en_GB
    - javascript

# Any files/folders we should ignore?
excluded_files:
    - ".github/.typo-ci.yml"
    - "node_modules/**/*"
    - "yarn.lock"

# Any typos we should ignore?
excluded_words:
    - fastify
    - fhir
    - hl7
    - nhs

# Would you like filenames to also be spellchecked?
spellcheck_filenames: true

During pull requests it will flag fastify and fastify-plugin as potential typos, even though it's in the array of excluded_words.
Example here: https://github.com/Fdawgs/fastify-disablecache/pull/2/checks?check_run_id=1687500011

@Fdawgs - Is this for the TypoCI/spellcheck-action or the GitHub Marketplace app?
I haven't updated the GitHub marketplace App to look in .github/.typo-ci.yml yet (It is on my todo). I'll get on it this weekend :)
Though in the interim you don't need javascript in the dictionaries, it knows most programming languages methods. I think if you put this in .typo-ci.yml it should work.
dictionaries:
    - en
    - en_GB

# Any files/folders we should ignore?
excluded_files:
    - ".github/.typo-ci.yml"
    - "node_modules/**/*"
    - "yarn.lock"

# Any typos we should ignore?
excluded_words:
    - fastify
    - fhir
    - hl7
    - nhs

# Would you like filenames to also be spellchecked?
spellcheck_filenames: true

@Fdawgs - Is this for the TypoCI/spellcheck-action or the GitHub Marketplace app?
I haven't updated the GitHub marketplace App to look in .github/.typo-ci.yml yet (It is on my todo). I'll get on it this weekend :)
Though in the interim you don't need javascript in the dictionaries, it knows most programming languages methods. I think if you put this in .typo-ci.yml it should work.
dictionaries:
    - en
    - en_GB

# Any files/folders we should ignore?
excluded_files:
    - ".github/.typo-ci.yml"
    - "node_modules/**/*"
    - "yarn.lock"

# Any typos we should ignore?
excluded_words:
    - fastify
    - fhir
    - hl7
    - nhs

# Would you like filenames to also be spellchecked?
spellcheck_filenames: true

Ah, that makes sense @MikeRogers0, I'm using the GitHub App so that explains it!

Ah, that makes sense @MikeRogers0, I'm using the GitHub App so that explains it!

@Fdawgs Thanks for reporting though, I'll ping you when I have a update to the app complete :)

@Fdawgs Thanks for reporting though, I'll ping you when I have a update to the app complete :)

@Fdawgs I just pushed an update to the GItHub Marketplace app, they both now support .github/.typo-ci.yml & .typo-ci.yml as a location for the configuration file.

@Fdawgs I just pushed an update to the GItHub Marketplace app, they both now support .github/.typo-ci.yml & .typo-ci.yml as a location for the configuration file.

Thanks @MikeRogers0 !

Thanks @MikeRogers0 !


[spaceship] Enable trusted number SMS verification

There are currently 2 PRs implementing this, but both have problems.
https://github.com/fastlane/fastlane/pull/13507 + https://github.com/fastlane/fastlane/pull/13554
I created this issue to extract the discussion from the PR.

So on my end, when I add two trusted phone numbers - they both send 4 digit SMS message in the TrustedDevices API response, not 6 digit TrustedPhoneNumber codes.
With that set up I have no way of knowing that there are both TrustedDevices and TrustedPhoneNumbers.
In both mine and @mariannegru 's setup, I don't think there's a way to know that we need to give both options to the user.

Ok, I know a bit about this as I just rewrote the login logic for spaceship:

Everything 4 digit is "2 step verification": https://support.apple.com/en-us/HT204152
Everything 6 digit is "2 factor authentication": https://support.apple.com/en-us/HT204915

In https://github.com/fastlane/fastlane/pull/13507#issuecomment-440794407 I described the process for 2 factor authentication with 6 digit codes: You login in and get your push, if you tell the system you can't access that you get the option to send SMS.
How does it work for your with 2 step verification?

I suggest to add a last option to the devices list (if any present into the response) as  SMS authentication that will show the phones. Can I have an example of two step verification account response? Unfortunately I do not have one.

What do you mean exactly with "add a last option to the devices list (if any present into the response) as SMS authentication that will show the phones"?
Are you on 2FA @mariannegru, 6 digit codes?
And you @jgadbois, you are using 4 digit codes so 2 step verification, correct?

For 2 factor authentication, which I am using, on the web UI directly after username + password you get the token via push and should enter it. Only if you click another button, you get the option to choose a phone to receive an SMS (See https://github.com/fastlane/fastlane/pull/13507#issuecomment-440794407) - we will have to figure out a way to go from "input 6 digit code" to "choose phone to receive sms" but I am not really sure how, without make the login process harder for everyone (I guess 99% of all logins will use the 6 digit code via push, not via SMS).

What does the 2 step verification flow look like exactly?

@janpio Yes I'm on 2FA, the account is not mine so I need to select the phone number that is not the default number. As I understand you want to leave the current flow but to be able to support selecting a different phone number. Therefore we have the following options:

Add a command line option that will enable the phone selection when used so by default will ask for code
When asking for code to support adding a sentence like 'phone' that will show phone list

@jgadbois can you post an authorized response to see the diffrencies? I think the difference is in the security code definition.

As I understand you want to leave the current flow but to be able to support selecting a different phone number.

I see no other option how to implement this. The web UI can offer the input and a link to the phone flow at the same time, but in the console we have to go to the token input and allow a good "exit" to the phone flow.

For CI integrations the command line option would be preferred, so we might have to implement both options command line for CI and an exit for console.

Can you elaborate how that would help? On CI you need to use a app specific password anyway as you can't input anything, right? Why do you need a flag to enable login via sms sent token? (Or do I miss something?)

@janpio is this similar to the following issue? https://github.com/fastlane/fastlane/pull/13743

No @rlam3. That PR fixes the use of FASTLANE_SESSION which was semi broken. Here we are talking about how to best implement a way to let users send their 2FA/2SV tokens via SMS as they can in the Apple web login UI.

@janpio Thank you for clearing this up. I removed the link.

Random thought: Could Ctrl+S (or similar) work as a way to trigger the SMS flow?

Ok, so I thought about this some more: The only proper way to "escape" the normal x digit input is to type e.g. sms and hit Enter, then the new "display SMS numbers" logic takes over. (For CI we could implement a new option or env variable that skips the normal input and goes to SMS directly).
I am currently working on getting an account with 2 step verification, so I can test and understand both flows better.

Ok, got it via a super old iPhone 4S with iOS6 running...
Here was my documentation on how 2FA works:
https://github.com/fastlane/fastlane/pull/13507#issuecomment-440794407
And here is it for 2SV:

So for 2FA after login you automatically get a 6 digit code pushed to your devices. You can manually switch to the list of phone numbers and select one to retrieve a text message with a 6 digit code.
2SV works differently and you get a selection of methods directly after login: You can chose a device or a phone number (by default there is only a phone number after registration, you can add devices that you added to "Find my Phone" manually). Depending on what you chose, you get the 4 digit code pushed to your device or sent a text message.

Status update before I leave for the holidays 🎄: I have it all connected together, using bits of #13507 and #13554 plus the missing pieces and some general cleanup on the 2SV/FA code.
Everything works, but during testing I got Apple to block my account from login which is currently not handled by the login code but probably should be ;) Probably a new issue though, so expect a PR with just adding this functionality first thing next year 🎆

Awesome, thanks @janpio !

https://github.com/fastlane/fastlane/pull/14011 is ready for testing.

Hey everyone that posted here, I am currently working on a first improvement to the 2FA sms handling of fastlane/spaceship: A way to define which phone number should automatically be selected for the 2FA token sms.
I need some help with the output on the Apple UI though: #13980 (comment) Would be great if some of your with "trusted phone numbers" in their accounts could take a look and reply. Thanks!


Never receive messages or finish connecting when using AWS Managed Kafka (MSK) with IAM Credentials.

Description
Streamiz.net: 1.5.1
AWS Managed Kafka: Apache Kafka 3.5.1
Using the OAUTHBEARER IAM Credentials Provider
MacOS or AWS Linux
In order to use the OauthBearer I created a KafkaSupplier supply Kafka Consumers and Producers.
The application never gets to reading messages.
Full Log
In the logs I get a lot of messages like this:
%7|1707168712.622|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:main]: sasl_ssl://b-1-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Selected for cluster connection: coordinator query (broker has 0 connection attempt(s))
%7|1707168712.622|CGRPQUERY|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:main]: Group "test-app-2-restore-group": no broker available for coordinator query: intervaled in state query-coord
%7|1707168712.622|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:sasl_ssl://b-1-public..kafka.us-east-1.am]: sasl_ssl://b-1-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Received CONNECT op
%7|1707168712.622|STATE|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:sasl_ssl://b-1-public..kafka.us-east-1.am]: sasl_ssl://b-1-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1707168712.622|BROADCAST|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:sasl_ssl://b-1-public..kafka.us-east-1.am]: Broadcasting state change
%7|1707168712.631|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:main]: Not selecting any broker for cluster connection: still suppressed for 41ms: coordinator query
%7|1707168712.631|CGRPQUERY|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:main]: Group "test-app-2-restore-group": no broker available for coordinator query: intervaled in state query-coord
%7|1707168712.631|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-restore-consumer#consumer-3| [thrd:main]: Not selecting any broker for cluster connection: still suppressed for 41ms: no cluster connection
%7|1707168712.639|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-consumer#consumer-4| [thrd:main]: sasl_ssl://b-3-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Selected for cluster connection: coordinator query (broker has 0 connection attempt(s))
%7|1707168712.639|CGRPQUERY|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-consumer#consumer-4| [thrd:main]: Group "test-app-2": no broker available for coordinator query: intervaled in state query-coord
%7|1707168712.639|CONNECT|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-consumer#consumer-4| [thrd:sasl_ssl://b-3-public..kafka.us-east-1.am]: sasl_ssl://b-3-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Received CONNECT op
%7|1707168712.639|STATE|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-consumer#consumer-4| [thrd:sasl_ssl://b-3-public..kafka.us-east-1.am]: sasl_ssl://b-3-public..kafka.us-east-1.amazonaws.com:9198/bootstrap: Broker changed state INIT -> TRY_CONNECT
%7|1707168712.639|BROADCAST|test-app-2-ce7cb5c5-e2db-4853-889e-92caf67bf887-StreamThread-0-streamiz-consumer#consumer-4| [thrd:sasl_ssl://b-3-public..kafka.us-east-1.am]: Broadcasting state change

How to reproduce
Config
var configMSK = new StreamConfig<StringSerDes, StringSerDes>
{
    ApplicationId = "test-app-2",
    SessionTimeoutMs = 45000,
    BootstrapServers = "servers",
    SaslMechanism = SaslMechanism.OAuthBearer,
    SecurityProtocol = SecurityProtocol.SaslSsl,
    NumStreamThreads = 1,
    AutoOffsetReset = AutoOffsetReset.Earliest,
    AllowAutoCreateTopics = true,
    StateDir = Path.Combine(".", Guid.NewGuid().ToString())
};

Topology
StreamBuilder builder = new StreamBuilder();

builder
	.Table("streams-plaintext-input", InMemory.As<string, string>("table-store"))
	.ToStream()
	.Print(Printed<string, string>.ToOut());

Topology topology = builder.Build();

I have a repo that outlines has all of the code.
https://github.com/monkeyboy64/msk-iam-steamiz-bug
Checklist
Please provide the following information:

[x] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
[x] A code snippet with your topology builder (ex: builder.Stream<string, string>("topic").to("an-another-topic");)
[x] Streamiz.Kafka.Net nuget version.
[x] Apache Kafka version.
[x] Client configuration.
[x] Operating system.
[x] Provide logs (with in debug mode (log4net and StreamConfig.Debug) as necessary in configuration).

Hi @monkeyboy64 ,
Did you try to create a simple consumer in .net to consume your source topic ? I guess is more an issue to the Kafka Client more than the high level library which is Streamiz.
Best regards,


Not able to build Vite (SSR) with react-query

Describe the bug
This small example fails when running yarn serve (essentially yarn build && yarn serve): https://github.com/duerdod/react-query-vitejs.git.
The build works fine when removing these lines https://github.com/duerdod/react-query-vitejs/commit/ef110a6d4fb1b93c72448e1d48278a782e9324c4#diff-1ba718c1eb8aa39cd20c2562d92523068c734d75f54655e97d652b992d9b4259R71-R77.
Issue seems to be react-query is loading the development version of react-dom:

Basically a reproduction of this ol badboy: https://github.com/facebook/react/issues/13276#issuecomment-408232369
Your minimal, reproducible example
https://github.com/duerdod/react-query-vitejs.git
Steps to reproduce

Clone https://github.com/duerdod/react-query-vitejs.git
Run yarn && yarn serve
Go to http://localhost:7456/

Expected behavior
React-query to load the production version of react-dom and not fail to build
How often does this bug happen?
Every time
Screenshots or Videos
No response
Platform
macOS
react-query version
4.5.0
TypeScript version
4.8.2
Additional context
Same issue on previous react-query versions.
yarn dev:server works fine (I guess it's because all deps are loaded in development)

As discussed on Discord, react-query doesn't decide which version (prod or dev) it loads of react-dom - we just import react like everybody else 🤷♂. So it seems more like an issue for the bundler...
react-query has react dom as optional peerDependency because you could also use it with react native - Maybe the bundler doesn't like that

anything we can do here @duerdod ?

closed because stale


redoc scroll issue

related issue

https://github.com/axnsan12/drf-yasg/pull/543
https://github.com/axnsan12/drf-yasg/issues/548
https://github.com/Redocly/redoc/issues/1167

i seem to package.json is fixed version, but static directory isn't.
please update static file
static/drf_yasg2/redoc/redoc.min.js

I confirm, have wrong behavior in chrome and ok in firefox.

This should be fixed in #69. If you point pip at this branch, you should be able to test it out on your project: https://github.com/johnthagen/drf-yasg/tree/update-js-deps

@JoelLefkowitz Could we release a 1.19.3 that includes #69 so that users can take get this fix in their projects?

@junshoong @garanews Please try the 1.19.3 release.

1.19.3 solved this issue for me.


investigate filtering posts with phone numbers

at it's simplest, filter titles with \b[0-9]{3|7|10}\b (or an equivalent valid regex) will filter titles that include a three (area code), seven (local number), or ten (full number) digit number. there's probably a mess of automod code in r/automoderator or wherever with a canned semi-working solution.

https://www.reddit.com/r/AutoModerator/wiki/library#wiki_dox_detection_and_user_safety


Update readme to reflect monorepo

I briefly described what the repo contains
Added brief getting started section
Replace "Dependencies" section with pointer to compiled summary of third-party licenses

The README is basic, but I think this is enough. @bilalshaikh42 can we close this?

Yes, I removed the references and directions for the old code structure. The new structure can be explained in #975


Only push named app.

Avoid downtime on multi-application manifests. See #23 for details.
[Resolves #23]

@xoebus Does this look reasonable?
Alternatively it could rename every app in the manifest I suppose?

Looks good. Doing zero downtime deployments of all of the applications would be nice to have but that's a large change that no one is asking for yet and is do-able with multiple runs with this pull request.

Cool. I'll go ahead and merge then. Thanks @jmcarp!

@xoebus Is it still the case that if there are multiple applications defined in the same manifest file, Autopilot will NOT perform zero-downtime autodeployment for all of them?
The way I am planning on using your plugin with Travis CI, we'll have a prod-manifest.yml and a dev-manifest.yml, each one with at least 2 app definitions, and it sure would be convenient if my script could invoke cf zero-downtime-push and simply pass it the appropriate manifest file to have all apps therein deployed with zero downtime.
Should I open an issue for that, if that feature is missing?

It won't do all of them but you could iterate through every application in the manifest and use autopilot to deploy each of them.

@xoebus Yes, but unless I'm missing something, it would require a priori knowledge of all the names of the apps in the manifest, which is not ideal.
For instance, say my dev-manifest.yml contains descriptors for three CF apps: app1, app2 and app3, either my .travis.yml or my deploy script would need to "know" the names of all those apps so it can execute the zero-downtime push command for each one. If tomorrow I added app4 to dev-manifest.yml, I would have to once again modify either my .travis.yml or my deploy script to let it know it should also deploy app4.
An ideal scenario is where the manifest file is truly the "source of truth": I should be able to just do a zero-downtime push of dev-manifest.yml and it should deploy whatever apps are there with zero-downtime.

Right, you could parse the manifest in the deploy script to get the list of applications.

@xoebus Right. Just to be clear, because I'm relatively new to CloudFoundry: when you say "parse," are there existing CF tools that easily parse manifest.yml files, making this a relatively trivial thing, or do you mean literally parsing the manifest file with awk and grep? :-)

Yeah, some YAML parsing fun. I don't think there is an existing tool for that.
YAML.load_file(manifest_path).each { |app| puts app.fetch("name") }

@vito and/or @xoebus: would you mind cutting a new release with this?

@cmc333333 Looks like it got included in 0.0.3 a while ago: https://github.com/contraband/autopilot/commit/2ee2da2e36671ce3522079d998393a2ac717dd80

@afeld, yep @xoebus just published that over the weekend. Thanks again!


isoDate should only have date

The current isoDate should really only look for a date, and we could introduce a new matcher (isoDateTime maybe) to match date and time. Or at the very least isoDate should take a time: false or something similar.
What do you think?

:+1: for clarity.
This was supposed to be "typical" ISO 8601, but the RFC has many valid formats, including (but not limited to)

2015-03-24
2015-03-24T06:57:23+00:00
2015-03-24T06:57:23Z

Not sure what we should aim to cover here. I like your idea of time. What about:

time: true/false? (ensures there's either a T component, or nothing)
gmt: true/false? (ensures there's either a Z, or an offset/nothing)

For all cases we should probably accept the widest possible ISO format. After all it's up to another library to parse it, so strummer should not be restrictive.

gmt or timezone, maybe. I'll write something up.


nodePackages: update

Motivation for this change
The neovim plugin coc-eslint needs to updated so it can be used in projects using the latest ESLint version 8.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 153586 run on x86_64-linux 1

3 packages marked as broken and skipped:

hyperspace-cli
teleprompter
timedoctor

66 packages built:

antora
balanceofsatoshis
bitwarden-cli
castnow
commitlint
concurrently
create-cycle-app
deltachat-desktop
discourse
discourseAllPlugins
emojione
epgstation
fast-cli
flexoptix-app
image_optim
iosevka
joplin
lessc
lumo
mastodon-bot
parity-ui
pm2
postcss-cli
pulp
pyright
redoc-cli
reveal-md
shepherd
shout
slack
sloc
styx
teams
thelounge
vimPlugins.coc-css
vimPlugins.coc-eslint
vimPlugins.coc-git
vimPlugins.coc-go
vimPlugins.coc-html
vimPlugins.coc-json
vimPlugins.coc-metals
vimPlugins.coc-pairs
vimPlugins.coc-prettier
vimPlugins.coc-pyright
vimPlugins.coc-rust-analyzer
vimPlugins.coc-snippets
vimPlugins.coc-solargraph
vimPlugins.coc-stylelint
vimPlugins.coc-tslint
vimPlugins.coc-tsserver
vimPlugins.coc-vetur
vimPlugins.coc-vimtex
vimPlugins.coc-wxml
vimPlugins.coc-yaml
vscode
vscode-extensions.matklad.rust-analyzer
vscode-extensions.ms-python.vscode-pylance
vscode-extensions.vadimcn.vscode-lldb
vscode-fhs
vscode-with-extensions
vscodium
vscodium-fhs
wasm-text-gen
whalebird
yaml-language-server
zx

Rebased and fixed conflicts be re-creating node-packages.nix.


Vitaly/nested container

support additional option for el - container will be used instead of this.container.
this allows building nested html tags easily.
also:

fixed Luton argument bug (typescript complained)
fixed a couple of build time warnings from typescript

It may be possible to improve this API further to be ergonomic for nested elements, but you can always use dv.renderValue and document.createElement in the worst place. This should do for now.


What's the default CORS-safelisted response headers？

please see docs: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Expose-Headers。
By default, only the 6 CORS-safelisted response headers are exposed: Cache-Control, Content-Language, Content-Type, Expires, Last-Modified, Pragma.
But I found the Content-Length header could be get by: xhr.getResponseHeader("Content-Length").
I want to know what the standard.

"Content-Length" was added to the spec after the initial six: Safelist Content-Length header. Since this issue was opened the necessary changes to MDN have already been done. Issue can be closed.

Thanks @mikoMK ! Closing.


Current build of tensorflow is not compatible with travis

The docker image used by the tensorflow team uses a version of glibc that is too recent for travis (ubuntu 12.04).
Update the scripts in https://github.com/tjhunter/tensorframes-artifacts to use directly ubuntu 12.04 as the base docker image.

Using beta version of travis solves the issue


Chart not expanding

The chart is not occupying all the space it could, even with HorizontalOptions="FillAndExpand"  it is only taking up a portion of the space, and is only enlarged if more data points are added
Xamarin.Forms

I'm seeing this issue as well, any update?

Also having this issue.

+1

Can you provide some more information please? A repro would be best.


Improve user-facing error text for debug app launch from iOS 14 homescreen

Current text:

In iOS 14+, Flutter application in debug mode can only be launched from Flutter tooling, IDEs with Flutter plugins or from Xcode.
Alternatively, build in profile or release modes to enable re-launching from the home screen.";

Proposing a change to:

In iOS 14+, debug mode Flutter apps can only be launched from Flutter tooling, IDEs with Flutter plugins or from Xcode.
Alternatively, build in profile or release modes to enable launching from the home screen.";

Google style guide is "apps" rather than "applications"
Pluralize apps
launching is appropriate here

Fixed in https://github.com/flutter/engine/pull/21362

thanks!

what do we do if we're getting this screen from vscode? its running the app fine on my ipad pro ios 14


Install EmailAuthorization extension completely

It turns out that EmailAuthorization doesn't get fully installed by Canasta, because it requires calling "git submodule update --init":
https://www.mediawiki.org/wiki/Extension:Email_Authorization
I would think the best way to fix this is to add this call to the Dockerfile, as is already done for VisualEditor:
https://github.com/CanastaWiki/Canasta/blob/master/Dockerfile#L92

Sounds right to me.


When Modal is dismissed keyboard comes back up and down

Hey there!
I'm trying to implement Introspect to focus on a textfield when a modal appears. It works great when the modal appears! However, when I dismiss the modal, the keyboard initially dismisses until the modal is fully gone, then comes back up and down on the ContentView.
running Xcode 11.5
Put together a quick setup so you can see what I'm doing:
https://github.com/MichaelMifsudSweeney/SwiftUI-Introspect-Modal-Bug

You can set a flag so textField.becomeFirstResponder() is only called once.

I have the same problem. Is there something I miss.?

Hi, as @vukrado has pointed out you can fix it like this:
struct ModalView : View {

    @State private var usernmae =  "hello"
    @State private var password =  "password"
    @State var shouldBecomeFirstResponder = true

    var body : some View {
        VStack(alignment: .leading) {
            Text("hello!")
            TextField("test", text: $usernmae)
                .introspectTextField { textField in
                    if shouldBecomeFirstResponder {
                        textField.becomeFirstResponder()
                        shouldBecomeFirstResponder = false
                    }
                }

        }
        .padding(.horizontal, 50)

    }
}

That said this has never been an issue with Introspect and I will close this issue now.


Alternative to building a new repository/code base

Hi Harry,
thank you very much for the time you invested into the code of this repository!
I have had to walk the very stony path of figuring out how OpenID Connect and a resource server in FastAPI could work together, myself. And this helped quite a lot in understanding some details.
I also came up with a solution and was wondering what you were thinking about it. My approach was to use a maximum of the standard (existing) libraries to avoid any self-introduced security issues.
Also, I have tried to use the security models from FastAPI in order to automatically have them listed in the OpenAPI docs.
Basically, the two tools I am using are authlib (for verification of the access_token, which in my case is a Json Web Token (JWT)) and the FastAPI security model.
Here is how it should look once https://github.com/swagger-api/swagger-ui/issues/3517 is fixed.
The code should already work (more or less - my final version is a bit more complex since I still try to work around the mentioned swagger-ui problem). Just a login via swagger-ui is not possible yet.

from fastapi.security.open_id_connect_url import OpenIdConnect
from starlette.middleware.sessions import SessionMiddleware
from authlib.integrations.starlette_client import OAuth
from fastapi import HTTPException
from fastapi import Request
from fastapi import Depends
from fastapi import status
import string
import random

# create your app first....
# then

# middleware required for authlib - though if you look into the code it is only used to obtain the "nonce"
# which is in this case always None and should not be checked anyways
# (since we are the resource server and the nonce is specified by the identity provider)
# but to reuse the code in authlib, we need to have the middleware installed
app.add_middleware(
    SessionMiddleware,
    secret_key="".join(random.choice(string.ascii_letters) for _ in range(16)),
)

oicd_discovery_url = "...." # the url before .well_known/...

authlib_oauth = OAuth()
authlib_oauth.register(
    name="myapp",
    server_metadata_url=oicd_discovery_url,
    #client_kwargs={"scope": "openid email offline_access profile"},
    #client_id="some-client-id", # if enabled, authlib will also check that the access token belongs to this client id (audience)
)

fastapi_oauth2 = OpenIdConnect(
    openIdConnectUrl=oicd_discovery_url,
    scheme_name="My Authentification Method",
)

async def current_user(
    request: Request, token: Optional[str] = Depends(fastapi_oauth2)
):
    # we could query the identity provider to give us some information about the user
    # userinfo = await self.authlib_oauth.myapp.userinfo(token={"access_token": token})

    # in my case, the JWT already contains all the information so I only need to decode and verify it
    try:
        # note that this also validates the JWT by validating all the claims
        user = await authlib_oauth.myapp.parse_id_token(
            request, token={"id_token": token}
        )
    except Exception as exp:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail=f"Supplied authentication could not be validated ({exp})",
        )
    return user

@app.get( "/me" )
def me( user = Depends(current_user)):
     return user

It seems to me that authlib also checks a few more claims than you do in your auth.py.
Comments highly appreciated. Maybe it also helps.
Best regards,
Carsten

Hey @camold
Thanks so much for opening this and letting me know the repo was useful (at least in a demonstrative sense). It's been a minute since I wrote this but I'll take a peak tonight. I'm all for avoiding an unnecessary repo and dependency where possible.
Cheers,
Harry

After review I think you've got a point about your excerpt above being equivalent to the repo as is. There still might be some value to the types.py and having a repo for fastapi-oidc code though. I'll add a warning on import that references this issue so people are at least made aware of a straightforward DIY implementation.

Hey @HarryMWinters,
thanks for the review and also letting me know I didn't get anything wrong.
I'm still buffled by the fact that implementing a ressource server in Python with oidc took me a long time to figure out - really there is no good documentation on such a seemingly simple task out there.
Let's hope we spare some other people some time with this.

For anyone else looking into this
oicd_discovery_url = "...." # the url before .well_known/...

This comment is incorrect. It should include the the "well_known" e.g.
oicd_discovery_url = "http://localhost:8080/auth/realms/my-realm/.well-known/openid-configuration"

Hello @des1redState,
I'm glad glad you found this thread helpful. Accolades to @samedii and @camold for their work. It is surprising how complicated it is to implement OIDC w/ FastAPI.
I want to caveat my answer with the acknowledgment that it's been a long time since I worked on middleware and that I've not used authlib much at all. That said here are my thoughts.

Have you tested this? I have not but I assume it's working as intended when you test with Postman or cURL or other tools.

This is my biggest point. It seems that in your auth function/endpoint after validating the OIDC token you dump the information into the session cookie. I don't think this is a security flaw per-se since the cookie is signed but it might be worth considering what the benefits to doing so are versus continuing to rely on the token provided by the Identity Provider. StackOverflow

I think your middleware approach is sound. Especially for bigger apps where it might be easy to forget to manually add identity checks to an endpoint. If you think it would be useful I'd be happy to add an example using this library but that might be of limited use if you're using AuthLib which actually has token generating functionality.

Take all the above with a healthy skepticism and hopefully we got more eyes on this. I'm curious what I missed. Good luck and thanks for posting 🤙


POST/PUT/DELETE routed controller actions always come back forbidden

Sails version:v1.0.1
Node version:v8.9.4
NPM version:5.6.0
DB adapter name: disk
DB adapter version: N/A
Operating system: osx
This is sailsjs v1.0.1 with blueprints disabled.
When I create POST routes, they all come back 403 forbidden
Here's my code:
// routes.js
module.exports.routes = {
  "GET /contact/*": {
    controller: "contact",
    action: "helm"
  },
  "POST /contact/*": {
    controller: "contact",
    action: "helm"
  },
  "PUT /contact/*": {
    controller: "contact",
    action: "helm"
  },
  "DELETE /contact/*": {
    controller: "contact",
    action: "helm"
  }
};

// ContactController.js
module.exports = {
helm: function(req,res){
    if(req.method == "GET") return res.send("index <form method='post'><input type='submit'></form>")
    return res.send("OK!")
  }
}

// policies.js
module.exports.policies = {
  '*': true,
};

GET request:

POST requests always come up forbidden:

This happens with POST/PUT/DELETE.  Any help

Ok, fixed!

Interestingly, in order to isolate the issue, I created a new project:

sails new test2 - using "empty"
created the API for that project sails generate api contact
copied the code above
seems to work fine.

Then I tried again using the "web app" option:

sails new test3 - using "web app"
created API for that project sails generate api contact
copied the code above, set global policy to true
I now can replicate the error, as described above.  All post requests return "forbidden"

quick update: the webapp template has CSRF enabled, whereas the "empty" template has it commented out.
Interestingly changing it to csrf:false didn't fix the issue but comment it out did.

Closing this for now.


Firebase Firestore

Prisma is doing great with all the most used Databases like PostgreSQL, SQLite, and the upcoming MongoDB support... But how about support for Firebase's Firestore, which is no less than the others in the race, and similar to MongoDB's collection and document format. Adding this feature is gonna be a great help for all the Firebase geeky people who want authentication, storage, and Database at a single place that Firebase support
Kindly see into the idea
By @GNVageesh

any news on this ?

Following up on this

We're creating an open source project for fast MVPs and would like my team to give this a try! Can any of the maintainers point me to the documentation on how to create adapters for new databases?
cc @sorenbs @apolanc @laplab @Weakky @SevInf


Protectli v1x10 testing

pre-commit.ci run

I checked all the commits, it looks good to me. Please make Pre-commit happy.

Locally pre-commit was happy.

@macpijan please resolve the comments if you have some spare time. I am basing the VP2410 validation on this code, so it would be good to land it either on master or on develop (still not sure which one I should target TBH).

I rebased and fixed the QEMU self-tests to work.


Migrate from RecyclerViewAdapter to ListAdapter

Migrate the following class from recycleview adapter to list adapter.

Resources:
https://developer.android.com/reference/kotlin/androidx/recyclerview/widget/ListAdapter

solved by @mawindavic #4


Update to v0.11.0 and rm pypy

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Fails only on pypy


Versions in master are confusing

At the moment, the latest release is 3.0.1.
But also at the moment, if you build from master -- which contains everything in 3.0.1, and maybe more -- it reports that it's 3.0.0. Which is doubly confusing: it's NOT 3.0.0, and also in a sense, feature-wise, it is AFTER the current release.
I'm not sure exactly what solution we should adopt, but it should be (a) numerically indicative of the version it will eventually become, and (b) have an explicit way to to tell it's a development branch and not a release.  3.0.99? 3.1.0-dev?
This is no emergency, just putting in this placeholder to remind us to choose a version reporting policy for master.

I thought we had talked about making that 3.0.9999 or something of the sort to indicate a devel / main build?

I thought we could set the OPENEXR_VERSION_EXTRA to "master" (or "dev") so the version would be 3.0.1-master, but I can't seem to get the OPENEXR)VERSION_EXTRA option to work at all, CMake complains:
CMake Error at CMakeLists.txt:24 (project):
VERSION "3.0.1-master" format invalid.
And indeed the CMake docs say the version must be all unsigned int's and dots.  What was the intention here, and has this ever worked?  I do prefer 3.0.1-dev to the other alternatives, if that can be made to work.

The VERSION has to be numbers only.

OPENEXR_VERSION_EXTRA can be used to end up in the headers as some kind of identifier, but don't pass it to the cmake project() function.

I took a stab at simplying the version-setting logic Imath: https://github.com/AcademySoftwareFoundation/Imath/pull/139. If that looks reasonable, we can replicate that here.

#1008 hopefully addresses this. Building off the master branch will generate "3.0.2-dev". Assuming this works as expected, the release process will be, just before a release:

bump the version on the RB-3.X branch (with OPENEXR_VERSION_EXTRA cleared to the empty string)
bump the version on the master branch to be one ahead of the RB-3.X branch (with OPENEXR_VERSION_EXTRA cleared to the empty string)


Move waitForStart() to before while loop

Moves waitForStart() to a more sensible location

messed up the git stuff (x2)


[Bugfix] Fix gptq failure on T4s

Fix for https://github.com/vllm-project/vllm/issues/7240, bug was introduced by: https://github.com/vllm-project/vllm/commit/a8d604ca2a2912b3a5352821c53c080383580df1, min_capability was refactored incorrectly leading the override_quantization_method to think it was always on a Ampere system

/ready