Experiment nimble dependencies management

After a discussion with @gmega, we decided to give a try to managing the dependencies with Nimble instead of using sub-modules.
The problem we currently have with sub-modules is the pain of updating because you have to go manually for each dependency, update the version, and the versions of the sub-dependencies.
Nimble would make our lives easier because it would fetch the dependencies automatically, and we could lock the versions into a nimble.lock file.
So the idea would be to fetch the dependencies with Nimble and keep building with the Nimbus build system.
We may need to create a symbolic link from the Nimble pkgs2 folder to the vendor folder to make the Nimbus build system recognize the dependencies (similarly to what the Nimbus build system does in the pkgs folder).

To give context: you'll only need to do the symbolic linking as long as you want to run hybrid builds. If you could move all deps to nimble, then that'd no longer be needed.

I'm not sure this is needed, we aren't moving away from the nimbus-build-system in codex any time soon, nimble should be only used for the third party dependencies that live under the vendors folder.

Yeah, the reason is that @2-towns asked me why we don't do lock files and nimble, and I explained it to him that it was quite painful to manage things this way, but that I haven't tried to do it in a while. So I gave him some overall directions on the why of nimbus-build-system, and some short leeway to experiment with nimble if he wanted (also for him to be convinced that it's not the right approach).

I am still not convinced :). Currently, in my attempt, Nimble is not perfect, but it is working (almost well) when trying to retrieve the sub-dependencies (which is the main reason to use it in our case). So the idea is not to move away from the nimbus-build-system, but to integrate Nimble and manage the dependencies only.
However, I am fine if you prefer to maintain the current management with the sub-modules.
I think I have updated half of the dependencies now. So which direction should I take? Should I continue with Nimble, or should I abandon this experiment and update the sub-modules?

Okay, so actually, nimbus-build-system has a hard-coded version of nimble, which is 0.13.1.
A PR was created to make the NIMBLE_COMMIT customizable, but it was closed yesterday.
So the summary I can make at this point is:

Nimble works great when retrieving sub-dependencies. You can have multiple versions of a sub-dependency without conflict because the resolution is based on the commit hash.

In our case, many dependencies don't have versions, so is it worth it?

The nimbus-build-system uses an old version of nimble (0.13.1) that does not support .lock files and uses the pkgs folder (which has since been moved to pkgs2). Therefore, we will need to maintain symbolic links between pkgs2 and pkgs.

The nimbus build system doesn't rely on nimble, what we should do is bump the deps in the vendor directory manually and make sure things compile and run.

A https://github.com/status-im/nimbus-build-system/pull/71 was created to make the NIMBLE_COMMIT customizable,

That PR changed a bit more than is necessary, but the aim is to upgrade n-b-s to nimble 0.16.4 as soon as that is released. It's waiting for a fix in nimble so that it can be compiled with nim 1.6. PR:s welcome (the above closed PR contained other changes that are not relevant for the moment)


First row behaviour is different

Hello,
i found a problem with rendering first row in grid. my code:
` public function getEntries()
{
$cfg = (new GridConfig())
->setDataProvider(
new EloquentDataProvider(
(new Entry)->newQuery()
)
)
->setPageSize(15)
->setColumns([
(new FieldConfig)
->setName('id')
->setLabel('ID')
->setSortable(true)
                    ->setCallback(function ($val) {
                        //dd($val);
                        $icon = ' ';
                        return $val;
                    }),
            (new FieldConfig)
                    ->setName('title')
                    ->setLabel('Title')
                    ->setCallback(function ($val) {
                        return $val;
                    })
                    ->setSortable(true)
                    ->addFilter(
                        (new FilterConfig)
                            ->setOperator(FilterConfig::OPERATOR_LIKE)
                    ),
            (new FieldConfig)
                    ->setName('description')
                    ->setLabel('Description')
                    ->setSortable(true)
                    ->setCallback(function ($val) {
                        return $val;

                    }),
            (new FieldConfig)
                    ->setName('is_active')
                    ->setLabel('Active')
                    ->setSortable(true)
                    ->setCallback(function ($val) {
                        return $val;

                    }),
            (new FieldConfig)
                    ->setName('order')
                    ->setLabel('Order')
                    ->setSortable(true)
                    ->setCallback(function ($val) {
                        return $val;

                    }),
            (new FieldConfig)
                    ->setName('id')
                    ->setLabel('Actions')
                    ->setCallback(function ($val) {
                        return  Form::open(array('url' => 'entries/'.$val))
                                    .Form::hidden('id',$val)
                                    .Form::hidden('_method', 'DELETE')
                                    .Form::token()
                                    .Form::submit('Delete', array('class' => 'btn btn-outline btn-danger'))
                                .Form::close()
                                .HTML::linkRoute('entries.edit', "Edit", [$val],array('class' => 'btn btn-outline btn-warning'));

                    })

            ])
            ->setComponents([
                (new THead)
                    ->setComponents([
                        (new ColumnHeadersRow)
                    ])
                ,
                (new TFoot)
                    ->setComponents([
                        (new TotalsRow(['posts_count', 'comments_count'])),
                        (new TotalsRow(['posts_count', 'comments_count']))
                            ->setFieldOperations([
                                'posts_count' => TotalsRow::OPERATION_AVG,
                                'comments_count' => TotalsRow::OPERATION_AVG,
                            ])
                        ,
                        (new OneCellRow)
                            ->setComponents([
                                new Pager,
                                (new HtmlTag)
                                    ->setAttributes(['class' => 'pull-right'])
                                    ->addComponent(new ShowingRecords)
                                ,
                            ])
                    ])

        ]);
    $grid = new Grid($cfg);
    $text = "
Constructing grid programmatically
";
    return [$text,$grid];
} `

When grid is rendered all rows have delete form, but in first row  tag is missing.
Is it a bug or i have a problem in my code?

I think that the problem resides in nesting forms.
All the grid id wrapped inside a form to manage filters
FORM -> TABLE -> (THEAD + TBODY + TFOOT )
You cannot render form(s) nested.
personally I use this solution https://gist.github.com/crystrk/32f1fb5d32102537e534b75d443ae297/
with some modification

anyway @Nayjest a very small fix in  resources/views/default/grid.php
move the input outside the table
<input type="submit" style="display: none;" />

        </form>

    <input type="submit" style="display: none;" />
        </form>

Hi!
How moving input outside the table helps? Or it's just because input tag placed direclty inside table tag looks strange?

An input is not a valid child of table (w3c) and may raise some problem with javascript, and also nesting forms is proibited.
there is a nice workaround but only for html5  see http://stackoverflow.com/questions/5967564/form-inside-a-table
I'm working on it

Using "form attribute of input and select" fixes the issue,
but will only work for html5 compliant browser
see compatibility table: http://www.w3schools.com/tags/att_input_form.asp
@gasperthegracner  if you want you can use a different template changing only few things
filters/select.php filters/input.php and the grid.php


[DO NOT MERGE] Policy service integration test

This PR must not be merged. It is here to demonstrate an error I am getting with a prerelease integration of the policy service.
The pass-ember README should detail the changes I made to proxy and sp containers to enable this setup to run. Brief instructions on how to invoke the service and reproduce the error are included in the README as well.

Closed for being dumb. Turned out there was simply an extra } here https://github.com/OA-PASS/pass-ember/pull/907/files#diff-a27a4520238e3375c58fca16c99c519cR35


Normalize path for "Save As".

"Save As" isn't working for me on Windows, as the path isn't normalised so forward slashes are lost.
Using the existing normalisePath function seems to have fixed this for me.

Change looks great! I'll merge it in when the build completes. Thanks for the fix @CrossR !

Cheers! Think that was the fastest Issue > PR > Finished PR I've done!

Haha ya, as soon as I saw there was the issue, there was already a PR fixing it 😄 Awesome!


working local pytorch transform

This PR adds the transform logic required to run an inference call on a trained pytorch model without triton in an ensemble graph.

rerun tests

rerun tests


Delay type class.

I'm just offering this because I have it and type aliases make me vaguely uncomfortable. I don't feel strongly about its necessity, so please merge or close as you see fit.

Yeah, I like this. Never did like the implicit natural transformation thing (although, we still have that for the time being – but now we have a path away from it).

Current coverage is 72.67% (diff: 0.00%)

Merging #52 into master will decrease coverage by 0.10%

@@             master        #52   diff @@
==========================================
  Files            37         38     +1
  Lines           665        666     +1
  Methods         652        649     -3
  Messages          0          0
  Branches         13         17     +4
==========================================
  Hits            484        484
- Misses          181        182     +1
  Partials          0          0

Powered by Codecov. Last update f2fe465...af8bacb


find_highest_exit, in djiskra, work backward

The entity go on target instead of fleeing.
In the #[cfg(not(feature = "threaded"))], a and b seem to be in the bad order.
But the #[cfg(feature = "threaded")] version it seem to be ok.

#69, I already opened a pull request for it.

I just merged the PR, it looks fixed now. Thanks everyone!


[Tool] SQL-Tester add check for empty result of SQL

What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[x] Tool

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.1
[ ] 3.0
[ ] 2.5
[ ] 2.4

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

@Mergify rebase

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

@@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.1 branch-3.0 branch-2.5


Scope logic in base Visitor class

The base Visitor implements the scope logic that defines when a new scope is opened or an existing one is closed. The scopes build a hierarchy, i.e., scopes are linked with each other while a scope refers to an outer scope and one or multiple nested scopes. This hierarchy is determined while visiting the statements, hence it is important that the classes call the base Visitor's visit method.
The initial design did not take into account that moving nodes may require updating the scope hierarchy. This is important as searching for a variable's value may otherwise fail if a variable declared in an outer scope cannot be found.
Also, sometimes nodes are visited out of order, for example, during partial loop unrolling in the CompileTimeExpressionSimplifier. This was not considered either. As an example, look into the doPartialLoopUnrolling method that moves the For loop's initializer into a Block but visits the initializer directly by skipping the block statement. This causes a wrongly assigned variable scope (but does not lead to break partial unrolling).

This has been resolved in v2.0, where there are now functions ScopedVisitor::enterScope,  ScopedVisitor::visitChildren and   ScopedVisitor::exitScope that can be used by Visitors derived from the basic visitor to manually decide when to enter/not enter a scope. This doesn't eliminate the possibility of these kinds of issues occurring, but it does provide the tool to prevent and/or fix them.


Evaluate literal expressions in vsa

Adds local assign expressions that can be evaluated to literals to the VSA
Functionally this is just merging the constant propagation results into the vsa
Before commit 93d99378cd96185c71f677f72a7bc3dce3cc6a20 the VSA tracked values of registers, but unclear if this was broken in a merge later or something else.

@yousifpatti I think generally you were trying to delegate to the constantprop results rather than duplicate in the VSA is this a correct fix to make?
We could validly repeat the same analysis in the VSA to get the same results, or just get the result directly from constant prop

Good catch. You are right I was trying to not duplicate the results in the VSA but I think doing so will give us the more accurate VSA implementation (should be tracking constants too) which will save us from having to merge the results of constant prop and VSA.
Your fix LGTM 👍


Move sticked elements to their position on page load

For when you load a page that has a #hash in the URL, the sticked elements are not moved and sticked in their proper position until you scroll. This fixes that issue.

Can you submit a pull request with the current version and I'll merge it in.


Evaluate server needs for this year

Since the current server expires on February the 2nd, I have some thought to go:

my private use is currently only limited to darkkeepers.dk and affiliated services
everything else drives Inexor
since the nooblounge will be migrating to new servers, we might think about migrating servers for capacity as well as for costs

As for the downtimes, the reason is probably capacity issues with disk IO caused by MongoDB/RocketChat.
The current server is hosted at NetCup, pretty much the same as the RS 1000 G7 SE, the HDD flavor.
When evaluating the infrastructure, please keep the following in mind

we need a central service to distribute SSL certificates (for Let's encrypt)
we need a low-performance host for website and FTP

both can be on a cheap HDD storage

we need high performance host for Rocket.Chat and Jenkins

both are preferred to use a SSD
specifically the Jenkins machine does need a lot of developer packages, it is best suit to separate it from the infrastructure

Since the infrastructure varies between different services and different load scales (e.g: during build time), the infrastructure should be capable of handling extremly high performance, but for most of the time is expected to run at very low usage.
Because of that nature cloud hosting could both be cheaper and more efficient for us to use..

Since I've got no feedback so far, I will be migrating some of the services to uberspace.
That means:

the main site will run on uberspace more permanently
since it doesn't take a lot of space, probably a new etherpad will also run there
darkkeepers.dk will also run there (including game servers)
the mail servers are running there (relieving mtmedia)
as cloud.darkkeepers.dk and the ftp server are very storage intensive, I am looking forward to provide the means with a S3 storage, which is really cheap for long term storage
we might be using CodePipeLine as an alternative to Jenkins, or alternatively run a seperate EC2 instance for Rocket.Chat and Jenkins

What does this mean:

I have already issued a DNS update to use uberspace and the website already runs there
It'll take some time to figure out Let's Encrypt
You can contact me for mail adresses, as you need them
Optionally, a Mumble server may be hosted there as well .. if we need that!

please wait & mumble...

We never decided to use Rocket.Chat, please post your opinion here: #352.

Why did we just migrate our emails to mtmedia to not have to manage that anymore on our own, just to migrate it again now?

Optionally, a Mumble server may be hosted there as well .. if we need that!

Again, see #352. Seems like we don't need Mumble anymore.

Since I've got no feedback so far

Please be more patient then ~20hours.

We never decided to use Rocket.Chat
Just because we never decided something does not mean we won't give it a try..

Why did we just migrate our emails to mtmedia to not have to manage that anymore on our own, just to migrate it again now?
If you've checked your email inbox quiet recently, mtmedia has already troubles with finances. I think they are better of if we don't use their resources for something we can get cheap and stable somewhere else.

Please be more patient then ~20hours.

I've been announcing this quiet a couple of time via IRC and Rocket.Chat

This would mean another thing to maintain. I guess we are better off when we use existing public pads and download important pads sometimes. It's sometimes possible to create a team. So let's create an Inexor team somewhere and let's go?

That is true. But it also grants us more privacy and freedom what happens with the Inexor ideas.
Furthermore, as far as I can see, I do not expect any changes in the setup for a decent amount of time.

I'd just wanted to note down as well that TitanPad, that we've been using quiet exhaustingly, is closing down soon, which might be additional reason to host a pad ourselves.

@Fohlen I can't remember ever seeing a TitanPad, did you use that while on Mumble?

I've been announcing this quiet a couple of times without much of reactions via IRC and Rocket.Chat, during the past year

You reminded us that we need to pay for the server again, not a detailed action plan what we could change.

The website is now on uberspace running fine so far. Closing this for now.

@Fohlen Website doesn't seem to update.

The NoobLounge will provide resources for Jenkins and Nightlies :-)


Authorization not working

I'm using the 2.2.0 version and whenever I define a Gate, the system returns a 403 Forbidden page.
In App/Providers/AuthServiceProvider:
public function boot()
{
    $this->registerPolicies();

    Gate::define('viewLarecipe', function($user, $documentation) {
        return true;
    });
}

Captured Gate response:
[
  "ability" => "viewLarecipe"
  "result" => null
  "user" => null
  "arguments" => "[0 => Object(BinaryTorch\LaRecipe\DocumentationRepository)]"
]

Any ideas? Thank you!

Can I see your config file?

Your question has put me on track to realize that I needed to force update the config file with php artisan vendor:publish --tag=larecipe_config --force which adds the 'middleware' => ['web'] param.
Sorry for bothering you and thank you for your work!


Image pull issue: failed to handle layer

CoCo 0.8.0
A simple test yaml:
apiVersion: v1
kind: Pod
metadata:
  name: distroless-12
  annotations:
    io.containerd.cri.runtime-handler: kata-qemu
spec:
  runtimeClassName: kata-qemu
  containers:
  - name: app
    image: gcr.io/distroless/base-debian12:debug
    command:
      - sh
      - -c
      - "sleep 10000"

error:
Warning  Failed     9s (x2 over 18s)   kubelet            Error: failed to create containerd task: failed to create shim task: failed to handle layer: hasher sha256: channel: send failed SendError { .. }: unknown

seems the issue happens on 'image-rs/src/stream.rs#channel_processing'
  tx.send(buffer)
            .map_err(|e| anyhow!("channel: send failed {:?}", e))?;

BTW, if change the image to 'gcr.io/distroless/base-debian11:debug', then no such issue.

This is usually caused by running out of memory in the guest. These distroless images are pretty small, but that is still the first thing I would check. You can adjust guest memory size via the Kata config file.

I tried a large image. The error is "No space left on device", this should be caused by the '/run/image' size limitation. However just adjust the 'image-rs/src/stream.rs#CAPACITY' to 65536, 'gcr.io/distroless/base-debian12:debug' works well.
Maybe there is a bug about the image unpacking.

I think with PR https://github.com/confidential-containers/guest-components/pull/602, the error message could be more clear.


CoachMark appearing off screen.

I had a  coachMark had the bottom cut off - the cut off was in the middle of the last row of text. I worked around it by  manually setting the arrow direction to bottom to put the coachMark on top of what it was explaining.
I don't currently have time to look for the problem to fix it - I might later - but I believe what happened was the size of the text fit underneath the area the coachMark was applied to, but after adding the arrow, it was pushed slightly off the bottom of the screen.
Here is a screenshot of the problem.

Hello @DougWatkinsDev, thanks for reporting the issue, that's quite strange. Would you be able to make a sample project demonstrating the issue, so I can investigate?

Closing due to inactivity. Might be fixed as well when #160 and #152 will be fixed.


modifying or overloading UserManager.LoginAsync() to accept collection of claims

i want to add custom claims while logging in,  for example EmployeeId to use it in the system
I need to create session class that will get properties from claims as you do with AbpSession.TenantId
Thank you :)

How todo:

Override CreateIdentityAsync to add new claims.
Derive a class from AbpSession and register it for IAbpSession on PreInitialize.

i registered it like that
IocManager.IocContainer.Register(Component.For<IAbpSession>().ImplementedBy<Al7Session>());

and resolve it like that
Session = (Al7Session)IocManager.Instance.IocContainer.Resolve<IAbpSession>();

and it's working, is this the way to do it ?

Hi,
It's not good to resolve a dependency like that. You can not constructor-inject it. Do it like that:
IocManager.IocContainer.Register(Component.For<IAbpSession,Al7Session>().ImplementedBy<Al7Session>());

Then you can inject it anywhere using Al7Session and also directly resolve it without cast if you want like that:
Al7Session session = IocManager.Instance.Resolve<Al7Session>();

Thank you dear

It looks easier to create an extension method than overriding CreateIdentityAsync. In some cases, the argument passed in is not included in User entity at all. Then I created the following extension for my case (serviceLine)
public static class UserManagerExtensions
{
    public static void AddServiceLineClaim(UserManager.AbpLoginResult result, string serviceLine)
    {
        if (!string.IsNullOrEmpty(serviceLine))
        {
            result.Identity.AddClaim(new Claim(BVClaimTypes.ServiceLine, serviceLine));
        }
    }
}

Then I call it after GetLoginResultAsync is called in AccountController.Login[HttpPost]
        var loginResult = await GetLoginResultAsync(
            loginModel.UsernameOrEmailAddress,
            loginModel.Password,
            loginModel.TenancyName
            );
        AbpOne.Users.UserManagerExtensions.AddServiceLineClaim(loginResult, loginModel.ServiceLine);
        await SignInAsync(loginResult.User, loginResult.Identity, loginModel.RememberMe);

btw, I also changed LoginViewModel by adding the new property called ServiceLine.
Hope this is helpful for others.

Please take a look at this question
1- How can that custom session can be used in Application Service layer ?
2- Is there a way to use custom authenticate ?

If you mean how the custom claims can be used, inject IPrincipalAccessor and use it like this:
var userIdClaim = PrincipalAccessor.Principal?.Claims.FirstOrDefault(c => c.Type == AbpClaimTypes.UserId);

See the documentation on External Authentication.

I mean, suppose I have a custom session like :
public class MarcoSession : IAbpSession
    {
        public Token MainSession { get; set; }
        ....
    }

and register it like
IocManager.IocContainer.Register(Component.For<IAbpSession, MarcoSession>().ImplementedBy<MarcoSession>());
Is it possible to initiate it in Application Layer like
public class UserAppService : ApplicationServiceBase, IUserAppService
{
    public Authenticate_Box Authenticate(Authenticate_Box_In box_In)
    {
        //Here that session can be initiated
    }
}

Or Should I do something like
[HttpPost]
public JsonResult Login(string username, string password)
{
    //Custom login by service
    var auth_c = ac.LoginAndAuthorize(argLogin);
    var loginResult = auth_c.LoginResult;
    if (loginResult != null)
    {
        switch (loginResult.Result)
        {
            case Result_C.AccountLocked:
                break;
            case Result_C.Succees:
                // 1- Here created AbpSession
                // 2- In second step get it for sample
                // 3- or Token token = HttpContext.Session["Token"] as Token;
                break;
            default:
                break;
        }
    }
    return Json(true);
}

Have you seen that: https://gist.github.com/hikalkan/67469e05475c2d18cb88

Yes I have seen that but I think that is complicated,
I have an ASP.Net web form with some generic handler and a login page with ASP.Net submit button by clicking that Btn I am creating a custom session...
in the other side I am using aspnetboilerplate App service just to proxy all abp app service in JavaScript (I don'nt want make ajax but using apbservice.method() directly)
So I want to make a relation between my custom session and abp session,
1- I can't access created session in ASP.Net page from an ApplicationService.
2- I don't want everyone can use my abp application services directly unless ASP.Net forms authenticate occurred.


chore: split decimal cast

Which issue does this PR close?

Part of https://github.com/apache/arrow-rs/issues/5125
Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

Could we instead put the decimal module underneath cast, as described in the ticket. i.e. create a directory called cast and place it under that?

Could we instead put the decimal module underneath cast, as described in the ticket. i.e. create a directory called cast and place it under that?

I misunderstand the ticket. 😢

Marking as draft to signify this is not waiting on review, feel free to unmark when you would like me to take another look


Dataproc API client fails to cancel due to wrong discovery of parameters

We (@adrianAhne and I) have found a small bug on the Python client code.
When using the Google API Python client to cancel a job, the Python code fails because it expects a body. However, the API documentation clearly states that there is no body to this request (https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/projects.regions.jobs/cancel ). I do not know if the discovery process is wrong or it is something else.
Here is a simple excerpt to reproduce the problem:
from oauth2client.client import GoogleCredentials
from googleapiclient.discovery import build

credentials = GoogleCredentials.get_application_default()
service = build('dataproc', 'v1beta2', credentials=credentials)
request = service.projects().regions().jobs().cancel(
                  projectId='my_project_id',
                  region='global',  # cannot be anything else than global at the moment
                  jobId='my_job_id')
result = request.execute()

Which fails when creating the request with the following error:
TypeError: Missing required parameter "body"

There is a workaround, which is to remove the body from the request:
request = service.projects().regions().jobs().cancel(
                  projectId='my_project_id',
                  region='global',  # cannot be anything else than global at the moment
                  jobId='my_job_id',
                  body='')  # Set body to something, None does not work
request.body = None
request.body_size = 0
result = request.execute()

This is also the case when calling projects.models.versions.setDefault:
credentials = GoogleCredentials.get_application_default()
ml = discovery.build('ml', 'v1', credentials=credentials)
ml.projects().models().versions().setDefault(
        name=model_version,
        body={} # Must be set, even if empty.
    )

It seems that body must be passed even if it is empty. In other methods, body is simply omitted if it is not required, but in setDefault, omitting the body parameter gives an error:
/Projects/pythonenv/lib/python3.6/site-packages/googleapiclient/discovery.py in method(self, **kwargs)
    734         if name not in _PAGE_TOKEN_NAMES or _findPageTokenName(
    735             _methodProperties(methodDesc, schema, 'response')):
--> 736           raise TypeError('Missing required parameter "%s"' % name)
    737
    738     for name, regex in six.iteritems(parameters.pattern_params):

TypeError: Missing required parameter "body"


Verify the integration of coloredlogs is fixed

Closes #159

Codecov Report

Merging #174 (14daa2c) into main (90cb3af) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #174   +/-   ##
=======================================
  Coverage   87.78%   87.78%
=======================================
  Files          12       12
  Lines        2186     2186
=======================================
  Hits         1919     1919
  Misses        267      267


On Sequential() LSTM, .evaluate not outputting results

Hi there,
First of all I should mention that Keras is brilliant. Keep up the great work.
I'm training two stacked LSTM layers for a vector-2-vector mapping. The model is:
model = Sequential()
model.add(LSTM(embedding_size, hidden_size, activation='sigmoid', truncate_gradient=-1, return_sequences=True))
model.add(LSTM(hidden_size, output_size, activation='sigmoid', truncate_gradient=-1, return_sequences=True))
model.compile(loss='mse', optimizer='adam')

Training the model works fine. For one epoch:
  Epoch 0
  9500/9500 [==============================] - 125s - loss: 0.0054 - acc: 0.2139 - val_loss: 0.0038 - val_acc: 0.0475

However, the evaluation code:
model.evaluate(Xtest, Ytest, batch_size=batch_size, show_accuracy=True, verbose=1)

Does not output anything informative:
  Evaluation...
  401/401 [==============================] - 1s

The above is all that happens before ending, without any loss/accuracy values.
Any suggestions?
Thanks!

evaluate does not print the loss / accuracy (maybe it should, though). Instead, it returns it (as a scalar if show_accuracy=False, else as a (loss, acc) tuple). So just print the output of evaluate instead.


should support editorconfig by default

Actually subj.
Another advanced feature would be to integrate with linter

Depending on your config, a possible workaround is to set beautify.tabSize to the same value as indent_size in your .editorconfig file.

well, this is not a workaround. This is exactly the opposite of using .editorconfig. The latter makes it simple to configure lots of users directly.
I mean yes, problem with tabulations is somehow "solved", but this has nothing to do with editorconfig

In the last version (2.3.1) we removed "tabSize" from the extension options and tabSize is get from the configuration of the active file or from the editor.tabSize


Cu 32h28rg improve token account handling

Changes
Adds an account getter using Brett's token account function. Replaces parseInt() with comparison of anchor BN to avoid potential overflow.
Checklist

[ ] UAT
[ ] Passes final QA checks

Task linked: CU-32h28rg [FACTORY] Improve token account handling


can the feature of enabling native resolution be added?

this game forces super resolution to be turned on, is there any way to turn off super-resolution to experience true native game graphics?

Turn the super resolution slider to 100% and the game renders in native resolution.


Impact of subnet size

What is the impact of increasing or decreasing the subnet size?
For example, if I set the size to /27, what specific restrictions do I have? If the number of endpoints available changes, we would like it to be clearly stated in the documentation. Thanks!

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: b312425f-a486-6c4b-9608-cca8ccd30937
Version Independent ID: 1d306c78-99e8-8f2c-d0f3-1aa5bf60d794
Content: What is Azure DNS Private Resolver?
Content Source: articles/dns/dns-private-resolver-overview.md
Service: dns
GitHub Login: @greg-lindsay
Microsoft Alias: greglin

@torumakabe
Thank you for your feedback! We have assigned this issue to the author to review further and take the next course of action.

Hi - 10 IP addresses are needed to accomodate the maximum number of allowed inbound endpoints (5) and the reserved IP addresses in the subnet (another 5).  That's why the minimum subnet size is /28 (16 IP addresses).  I will check to see if you are able to use larger subnets for anything, or if these extra IP addresses must remain unused.

@greg-lindsay Thank you for your comment!
The current limit for the number of inbound and outbound endpoints is 5, so is there no benefit to be gained "at this time" by providing  /27 or more subnets?
I would appreciate it if you could check and clarify this in this documentation.

@greg-lindsay Thank you for your comment!
The current limit for the number of inbound and outbound endpoints is 5, so is there no benefit to be gained "at this time" by providing /27 or more subnets?
I would appreciate it if you could check and clarify this in this documentation.

I have confirmed that yes - at this time - there is no benefit to a larger subnet than the minimum of /28. In the future, greater capacity could be enabled (no current plans), so larger subnets are allowed.

I will add a note in the article saying something similar to what is above.

#please-close


Quality scores lost

I recently ran TranscriptClean on my PacBio reads, and it ran fine after making the changes you suggested (thanks!). However I noticed that the quality scores present in the input SAM file are not preserved in the output SAM. Would it be possible to add this feature at some point?
Of course, the quality string would need to be edited correspondingly with whatever indels are corrected. For insertions, the corresponding qual value would be removed, and for deletions you'd need to impute something (maybe "!"?).
If you're OK with a pull request I'd be happy to take a stab at it :)

When I initially developed the code, the PacBio reads I was using lacked a quality score altogether, so it wasn't something I thought a lot about. This seems like it could be a reasonable feature to add- you're welcome to give it a try! I'm working on a TranscriptClean upgrade right now (Python 3 among other things) which I hope to have out by next week, so maybe it would make sense to do a pull after that.

Sounds great, I'll keep an eye out for it!


[REQUEST] Extraneous param validation in config

Is your feature request related to a problem? Please describe.
I've had a few times where I've thought I've been configuring DeepSpeed to do something, but it turns out that I misspelled a field name in the config. Ideally, DeepSpeed should be able to detect when this happens.
Describe the solution you'd like
When a DeepSpeedConfig is created from a dictionary, it should validate that all keys in that dictionary are actually recognized. If not, it should ideally raise an error, or at least surface a loud warning.
Describe alternatives you've considered
Client-side validation of config is hard because DeepSpeed is a rapidly evolving project.
Additional context
:pray:

Hi @jbloxham - this should be supported more now but will be better once we get PyDantic v2 updated so we can validate those fields.


Fix cube arithmetic dtype

See https://github.com/SciTools/iris/pull/2492#issuecomment-297005131

I'm going to look into this. I can't assign myself though, or set it to in progress...

Closed by #2526.


fixed usage of atomic operation lib for armcc compiler, added conditi…

…onal compilation

Due to lack of response from @suiauthon, I'm assuming this to be ready to merge.


feat: added get videos to functions

Signed-off-by: veralex<EMAIL_ADDRESS>feat: added get videos to functions

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind user-interface

/kind content

/kind translation

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area blog

/area documentation

/area videos

What this PR does / why we need it:
Fixes youtube videos date issue
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: alexakvelon
Once this PR has been reviewed and has the lgtm label, please assign jasondellaluce for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: alexakvelon, Issif
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Issif]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: ab439c84a31824ce09e1a6c199898279665c5f51


chore: enable swc for emotion jsx

Experiment based on this question: https://github.com/belgattitude/nextjs-monorepo-example/issues/1535
Not yet prime time
CI blocks on bundle size increase. Hypothesis: babel transpilation reads browserlist, swc does not and transpile to es5.

Closing in favour of https://github.com/belgattitude/nextjs-monorepo-example/pull/2187


Update http4s-blaze-server, http4s-circe, ... to 0.21.24

Updates

org.http4s:http4s-blaze-server
org.http4s:http4s-circe
org.http4s:http4s-dsl

from 0.21.4 to 0.21.24.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Applied Migrations

dependency:v0_21@org.http4s:http4s-scalafix:0.21.5

Documentation:

https://github.com/http4s/http4s/releases/tag/v0.21.5

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s" } ]

labels: library-update, semver-patch, scalafix-migrations

Superseded by #90.


Documenting use of vagrant provisioning for troubleshooting installation haults

With reference to the issue #1886
Suggests use of vagrant provision when vagrant machine fails due to lost connectivity or power failure in  the https://zulip.readthedocs.io/en/latest/dev-env-first-time-contributors.html.
Let me know if improvement or additions needed.

Automated message from Dropbox CLA bot
@sonali0901, it looks like you've already signed the Dropbox CLA. Thanks!

@christi3k  I have incorporated the changes you suggested. Kindly review them and let me know if something more is needed.

@sonali0901 did you intend to close this pull request?  It looks like you maybe accidentally did so by deleting the branch it was based from.

@timabbott  Actually I had to delete all my branches because they were created from my master whose commit history was messed  . I got my master cleaned up now so that it is even with zulip:master and will be creating branches now.


Prevent runtime mismatch

Follow-on from discussion at https://github.com/capitalone/cloud-custodian/pull/1420#discussion_r128206725, includes #1545.

Not sure where Travis (and therefore Coverage) are, but this is otherwise ready for review @kapilt.

Got Travis back. It's red for py36, investigating ...

Rebased and squashed with a new placebo_data.zip, was cd59afaf55512bc7025f8a90c52bc0e67146b6da.

This is green and ready for review.

Rebased, was 5ad9345921c6264ee7ab7402942a6da7a53a6db8.

i'm a bit wary of doing implicit deltas, as its common to have multiple actors potentially running custodian, and doing an implicit delta on their execution env feels quite dangerous, albeit this effectively raises an error. But for example how would i upgrade the lambda policies from 2.7 to 3.6 with this in play? afaics the only reason we care is if there's compiled resources in the lambda (afaik none of the custodian policy lambdas need that).

afaics the only reason we care is if there's compiled resources in the lambda (afaik none of the custodian policy lambdas need that).

There are different strategies for porting code from Python 2 to 3. We are using a single-codebase strategy, but some other projects use a split-codebase strategy. In general the single-codebase approach (ours) won out in the community, but there are still projects with split codebases.
PyYAML has a split codebase (though see ruamel.yaml). Core Custodian doesn't require PyYAML, but the mailer does, which gives rise to a broken mailer when specifying a remote runtime different from the local runtime, since we end up bundling the wrong version of PyYAML.

how would i upgrade the lambda policies from 2.7 to 3.6 with this in play

Delete the old lambda and republish the new one? This only throws an error when local and remote differ. If there is no remote then create works fine.

If you want explicit runtime configuration then we could also throw an error on create, i.e. check locally for runtime match between config and environ before we even ping remote.

Or at least a warning?

I lied. Core Custodian does require PyYAML, though we don't bundle it in custodian_archive.
https://github.com/capitalone/cloud-custodian/blob/1daaec83624f473ad45bfb3fd395817b23b09b4d/requirements.txt#L6
https://github.com/capitalone/cloud-custodian/blob/1daaec83624f473ad45bfb3fd395817b23b09b4d/c7n/mu.py#L198-L200

What about a devexp as follows?
$ custodian run policy.yml
Runtime mismatch! Policy specifies python3.6, your local is python2.7. Proceed? (y/n) n
$ custodian run policy.yml --ignore-runtime-mismatch
$

Probably in the mailer we'd stick with a hard-fail since we know it matters there?

Could even perhaps add an allow_runtime_mismatch key to policy config next to runtime?

@kapilt on call:

We have three sources of truth:

config file
local environ
remote lambda

We need to look at config.
When we're not actually updating code ... we don't even care.

Alternate strategy in https://github.com/capitalone/cloud-custodian/pull/1611 ... always use python3.6.

Closing, will try a workaround in the mailer.


Python 3 support

Adds support for Python 3, while retaining full support for Python 2.
There are 2 non-python 3 related changes in this:

I added *~ to .gitignore, since my text editor outputs *~ files as backups
I fixed some type errors (specifically len(non_array_variable)) and an undefined key error when I tried calling getUsernameInfo

There is also the addition of an extra file, Utils/Compat.py, for various Python 2/3 compatibility functions (currently only contains a urllib/urllib.parse support block). I'm not sure if this follows the coding standards of the project though.

Merged,
had to take out this commit though @17a0891d738bb651358246e069687bf3572ee9f1 due to the contribution guidelines
Thanks very much


Replace scraping mechanism with something faster

Maybe pull the database records directly from a mongodb gravity connection? It fetches pages and does a lot of empty pages towards the end.

👏  for the Cinder sitemap work! Please feel free to delete this repo/heroku app at your convenience.
I'll leave some quick historical context/clarity around Fusion here for those interested. Fusion was an experiment pre-Metaphyiscs to address two things—

Web Engineering had kicked around the idea of an orchestration API on top of Gravity to serve cached blobs of data that avoided all the ugly crawling/caching code in Force & friends (GraphQL was barely a thing at time).
We were implementing a bunch of a new sitemaps and needed to get them from multiple services (namely, but not necessarily limited to, Positron and Gravity).

These two "efficiently serve aggregated data from multiple services" use cases seemed close enough to attack with the same layer so I went off to try Fusion out on solving 2 with the intent to expand to 1 and ironically Damon simultaneously felt it was about to time to try out a solution for 1—we had a chuckle and discussed how to merge our efforts. Of course Metaphysics became a big success and we never got around to folding Fusion's use case back into it.
I explained some of the reasoning for choosing dynamic sitemaps powered by APIs here—but in summary it was simpler to maintain, with fresher data, and didn't require direct database access (at the cost of running the occasional expensive query on said APIs). Fusion's scraping silliness was necessary to work around not being able to, and not wanting to expose the performance impact of, querying artworks by date range in Gravity. Having a slowly updated stale cache of Gravity API data also seemed like a useful feature for an orchestration layer and hence the reason for exploring that approach.
Hope that helps clear some things up about Fusion. Glad to see us take this service out of the mix and leveraging Cinder for this work now 👌.

Thanks for the story @craigspaeth that makes a lot of sense! I think in many ways Cinder/Spark/Hadoop fundamentally are the same thing when it comes to data, a copy of all artsy data from all over the place. Spark adds a parallel processing layer making things extremely fast, ie. generating sitemaps in a non-dynamic way with a turnaround of a few minutes, so we don't need to query live data anymore to generate them except in the rare small cases like news articles.


Document recent ufunc enhancements

This should add suitable user and developer documentation for the latest set of ufunc enhancements, specifically DUFunc's and array operators/expressions.
This PR currently depends on #1110, though it may also document later changes (such as #1148).

Do you have any additional changes you want to make to this PR?

Yes,but no longer.

I think this is close enough to merge.


NPM login deny with access list

Describe the bug

Access List not working with NPM page itself

What version of Nginx Proxy Manager is reported on the login page?
2.7.3 (Docker)

To Reproduce
Steps to reproduce the behavior:

Access List >> Add Access List
add a username: user1
check 'Satisfy Any'
Authorization>> enter a username, and password
Access >> Allow > enter a public IP address
click Save
go to Proxy Hosts > edit the NPM subdomains> Access List > select the list name
click Save
go to NPM URL: https://npm.mydomain.com

Expected behavior

Go to the NPM url: npm.mydomain.com
Authentication window popup: username,  password
enter credentials, enter email address and password.
Log into the NPM dashboard

Screenshots
If applicable, add screenshots to help explain your problem.
Operating System

Ubuntu 20, Docker

Additional context
After I enter the credentials in the auth pop up box, it denied.
It goes into a loop after I enter the credentials even it's correct.
If I click cancel, the domain would direct to: https://npm.mydomain.com/login.
It then let me enter my email, and password, but it wouldn't let me log in.
I've tried the 'Access'  in the Access List with IP and no ip, but error persist.
I've also tried turn off proxy option in Cloudflare as well.
However, the access list works with other services.
NPM current setup:

Cloudflare
Google VM
web admin port: 81
http: 80:80
https: 443:443

Duplicate of https://github.com/jc21/nginx-proxy-manager/issues/334

I have the same problem with npm Auth Loop . Version 2.9.3

Hi,
I'm currently using v2.9.6, and the issue still persist.
The proxy access list works for other subdomains except NPM itself.
I don't understand why.
However, if I use the below proxy codes, the access list box will not prompt and let me log in.
location / {     include conf.d/include/force-ssl.conf;     proxy_pass http://<IP_ADDRESS>:81;     proxy_set_header Upgrade $http_upgrade;     proxy_set_header Connection $http_connection;     proxy_http_version 1.1; }

This issue was closed as a duplicate. Please only use the open issue to add helpful information, since it will get lost here.


Improve dependency extractor

Summary
This refactors the default dependency extractor to make regular expressions less error-prone (building them with a custom DSL) and fixes some bugs:

require("foo",) (notice the trailing comma), which is completely valid, wasn't detected.
import typeof {foo} from 'foo' was detected as a dependency, when it should've been ignored like import type {foo} from 'foo';.

It also uses negative lookbehind when it's supported.
Performance-wise, it's faster than the previous version as it removes unnecessary capture groups (tested internally at Facebook).
Test plan
Updated unit tests.

Tests seems to be failing on Node 6?

Tests seems to be failing on Node 6?

There's a case that was failing before:
foo.require.requireActual('foo');

I used a negative lookbehind to fix it, but they're only supported in Node 9+. I disabled the test cases that don't work with Node 6 to make CI pass. Once we drop support for Node <10 we can re-enable them.

Once we drop support for Node <10 we can re-enable them.

Heh, so in like 3 years :P
Would it be possible to detect support and activate if it's available? Or would that give us different behaviors on different nodes which might be confusing to the user?

@SimenB what I disabled was the test, not the support for the feature. I still use feature detection and enable it if it's available.

Ah, cool! Could we conditionally disable the test as well, then? To make sure we don't break it 🙂

Codecov Report

Merging #7385 into master will increase coverage by 0.05%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #7385      +/-   ##
==========================================
+ Coverage   66.72%   66.78%   +0.05%
==========================================
  Files         241      242       +1
  Lines        9356     9372      +16
  Branches        6        6
==========================================
+ Hits         6243     6259      +16
  Misses       3110     3110
  Partials        3        3

Impacted Files
Coverage Δ

...ckages/jest-haste-map/src/lib/isRegExpSupported.js
100% <100%> (ø)

...ages/jest-haste-map/src/lib/dependencyExtractor.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b62b08...512b0e2. Read the comment docs.


Add browser support

Ensure that @iso4217/json can be used in a browser environment via adding a <script> tag.

[ ] ISO4217JSON === window.ISO4217JSON === window["@iso4217/json"] – global namespace object;
[ ] ISO4217JSON.data – the parsed contents of data.json file;
[ ] Figure out whether we need contents of the data-grouped-by-*.json files in browsers, and if so, how to distribute them to browsers.

Pending: unclear until data-grouped-by-*.json files are figured out


Typo "resposneDecoder"

Version
.NET API 4.9.4
Description
In DropboxRequestHandler.cs, there are several occurrences of the misspelled name resposneDecoder:
/// <param name="resposneDecoder">The response decoder.</param>
IDecoder<TResponse> resposneDecoder,
It should obviously read responseDecoder.

Thanks! I'll ask the team to fix that up.


Add Chile scraper

https://desarrolladores.mercadopublico.cl/docs/services/5a9ed02f718ed712f4403e75/operations/5a9ed1fc718ed712f4403e78

Year 2008 onwards

@yolile  will check on the first full run of this scraper

The first full run crashed, see #241

@odscjames  The error is that some json files have:
{  "NumeroError": -2146233079,  "Mensaje": "The operation has timed out"}
instead of a release-package

If it's timed out does that mean it's an intermittent error? In save_url function, could we look for "NumeroError" and retry a few times?


build: add cross-platform support to plugins artifacts

What type of PR is this?
/kind design
/kind feature
Any specific area of the project related to this PR?
/area source-plugins
/area extractor-plugins
/area build
What this PR does / why we need it:
This includes the platform in the name of plugins artifacts for both pre-release and release builds. Accordingly, the version of all plugins has been bumped to trigger a rebuild.
Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
build: add cross-platform support to plugins artifacts

LGTM label has been added.  Git tree hash: 51b65cbb8ce17272da48612635272f4bf1acef97

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


I could use a fast alternative to JSONEncoder or PropertyListEncoder and it feels like protobuf may be able to offer a similar API as those two for serialization?

Protobuf includes its own spec for JSON support, which doesn't always line up with what folks want. The protobuf one uses strings for some numeric types because it strives for interop, and some implementation (javascript), can have issues with larger numerical values.
So at the moment, we have a json interface exposed, it isn't Codable, but adding a second seems like it would be somewhat confusing.

I'm sorry, maybe my question was confusing. I'm not looking to convert anything to JSON.
My understanding is that Swift 4 Codable allows for a unified interface that structs/classes can conform to to allow serialization.
Which format we encode Codable objects to is completely up to what Encoder/Decoder we implement, it's just that the only encoders included in Foundation are currently PropertyListEncoder and JSONEncoder. The Encoders aren't conforming to anything, it's just that they are built to accept objects that are Codable (=Serializable).
Both are really slow and bulky, and I'm currently looking for a smaller & faster format to serialize to.
I thought that maybe Protobuf could be a format that serializable objects could be serialized to.

Understood, some people (historically) have expected numbers to always end up as numbers, and so the protobuf JSON isn't a match for them.  I'm not sure if supporting Codable would lead to any more of this.
Part of our stance has been if you want Protobufs, you want the protobuf behaviors, and it doesn't provide completely generic support for JSON encoding like some might expect.  Likewise, the keys in the JSON go through transforms from what is in the proto files, so that also can make it less of a fit for folks.

Supporting Codable would be somewhat complicated, because the protobuf spec describes multiple types that map into a single Swift type. For example, uint64 and fixed64 are both represented as UInt64 in Swift, so we would need a way to discriminate them for serialization.
If we really wanted to do it (that's a big "if"), it's probably possible—generated messages could have their own CodingKey enum that tags each field with the additional information needed for each field (its field number and its wire format), and then the proto-specific encoders/decoders could pull that information and do the right thing. Whether that alone is enough is hard to say though, and there could be other gotchas.
However, it's extremely likely that such an implementation that has to go through all of the coding plumbing would be significantly slower than the implementation here, which has been specifically tuned for speed—and when using the binary encoding, speed is what you want.
In short, supporting the Codable APIs seem like a large amount of effort for relatively little gain. Codable is meant to allow arbitrary data structures to support serialization in multiple formats. Protocol buffers are specifically generated data structures that support serialization in a finite number of spec-defined formats. The use cases simply don't line up.

I could use a fast alternative to JSONEncoder or PropertyListEncoder...

I believe the performance of JSONEncoder and PropertyListEncoder is constrained primarily by the performance overhead of the Codable protocol itself.  SwiftProtobuf implementation included many iterations of profiling and redesigning the API between the generated objects and the encoders/decoders.  I'm not at all certain that adapting our encoders/decoders to work against the Codable protocols would provide any performance advantage.  Codable also lacks a number of key concepts that would be needed to properly support protobuf's binary serialization.  (For example, it has no concept of varying wire encodings for the same in-memory data type.)

@allevato thank you, that made the differen use cases much clearer for me.
@tbkka Yes and no, from my profiling it seems the issue is as much with Codable as with the fact that JSONCoder uses NSJSONSerialization internally, which converts everything to NSNumber  and NSString etc. Similar issue for PropertyListEncoder.

The concept of "varying wire encodings for the same in-memory data type" seems to have come up twice here as the only real reason not to implement Codeable conformance (I'm assuming this is the same thing as was meant with "uint64 and fixed64 are both represented as UInt64 in Swift"?)
I'm not sure I see how this is a hinderance to the goal as a level of consistency. The codec would indeed need a source of truth (defined externally to the Codeable type) about which protobuf tags to use, and which protobuf types map to which Swift types etc. But is it unreasonable to implement the codec such that it takes either a Swift data structure generated from a .proto file, or even the .proto file itself?
let decoder = ProtobufDecoder(schema: myRootSchema) // <-- here
let myInstance = try decoder.decode(MyType.self, from: binaryData)

Or is the criticism more of a "what do we gain for our efforts here?" I don't claim to be an expert at Codeable, and I'm especially not an expert in protobuf. Just trying to figure out why it'd be a non-goal to fit in with the forward momentum of Swift development in this regard. I was made interested in this idea after listening to a podcast with Chris Lattner recently, who seemed to suggest that Codeable compatibility for Protobuf was already available.
As a way of promoting use of of this library, it'd be pretty great to support this as a (admittedly slightly more complicated) drop-in addition to the existing JSON and PropertyList codecs.

So I'm starting to wonder if folks are talking about different things here.
I was thinking these requests were around supporting mapping the individual fields of a message to the encode/decode methods (int32, double, etc.).  So it would get another api to get the tree of JSON for example; i.e.- the library would be calling the encode/decode methods on a per field basis.  But now I'm starting to wonder if folks were thinking of having their Messages encoded as a single (binary) blob in the encoder/decoder and not having the fields expressed/exposed.  This would just allow a message to be captured within something Codable, but wouldn't expose the details of how it is encoded.
The comments about JSONEncoder or PropertyListEncoder performance; I guess folks could even be talking not so much about directly supporting Codable as much as trying to have this library expose things that conform the the Encoder/Decoder (and I guess the matching Containers), hoping that some how lets types that support Codable use our endcode/decode to get better performance?
Anyway, I guess I need folks commenting to better spell out what they mean by saying they want Swift4 Codable support so we can be sure everyone is talking about the same thing.
Or it is just me confused and everyone can ignore this comment.  😄

@ephemer

As a way of promoting use of of this library, it'd be pretty great to support this as a (admittedly slightly more complicated) drop-in addition to the existing JSON and PropertyList codecs.

This is exactly what I meant.
@thomasvl
I was asking for protobuf to become an alternative to JSONEncoder/ PropertyListEncoder for serialization of objects conforming to Codable.

I was asking for protobuf to become an alternative to JSONEncoder/ PropertyListEncoder for serialization of objects conforming to Codable.

Do you want to exchange the serialized result with a server or other protobuf-using system?

If so, then you have a proto file and you would have to find a way to ensure the result conformed to that schema (which has information that Codable by itself cannot represent).

If not, then you might not need to conform to a proto file, but the result would not be protobuf and it would be misleading to call it that.

If the intent is just to avoid the performance of JSONEncoder / PropertyListEncoder, then it seems like the better thing to do would be to file bugs against those.  Protobuf can get some of its performance between we have a proto definition to provide the scheme and use a different set of interfaces.  It doesn't seem right to take on the burden of trying to support a different api just to work around someone's implementation speed (especially when some of the speed could be because of the api one is trying to duplicate).

It’s not a complaint about performance of encoding or decoding JSON or plists so much as an interest in the benefit of what protobuf can give us. I’m interested in the safety guarantees and also size efficiency in storing and exchanging data as protobuf instead of in JSON. Or why else are we using protobuf at all??
As is, the data I’m interested in has (among other things) arrays that have hundreds of JSON objects with a few keys each. Their structure is such that using protobuf would allow a 90%+ OTW saving compared to the current JSON implementation. And yes, coding/decoding protobuf is faster too.
So, I could use Swift-protobuf as is, but the suggestion in this thread, as I understood it and continued, is that we make ProtobufDecoder / ProtobufEncoder types that work alongside JSONDecoder / JSONDecoder (etc) without having to think too much about new APIs. This would set a great precedent for Swift Codable types, given that Codable is now a first class citizen in Swift land.

@ephemer:  As discussed earlier, the Codable protocol has some inherent limitations that make it very difficult to accurately support the official protobuf spec.  Since our primary goal is to accurately support that spec, providing a Codable-based encoder/decoder is not really appropriate for the SwiftProtobuf project.

It might be technically possible for us to generate appropriate coding keys with the necessary supplemental information (field number and wire format) to let a hypothetical ProtobufEncoder/Decoder work, but without diving into the details, it's hard to say what stumbling blocks someone doing so would hit because of their different design goals. Codable is considerably more general than protobuf, so it's likely that there would be some degree of runtime checks to ensure that nobody used the protobuf encoders/decoders with incompatible data types.  There may also just be places where the design principles between the two APIs can't fit together.
I'd still be concerned about performance, too. If a Codable-based implementation is significantly slower than the one we have now, that's kind of a deal breaker. There would be no benefit to offering and maintaining both a "full speed" binary encoder for users who need all the performance and a "kind of ok" binary encoder for users who just want to use a different API.
Since the protobuf spec defines a very specific JSON representation (e.g., the way 64-bit integers are handled), we would need to provide our own encoders/decoders for those as well, because the ones in Foundation wouldn't be appropriate. So that's awkward at best—if users want to communicate with a server that specifically uses JSON-encoded protobufs, they would need to use our encoder/decoder, not the standard one. And we would need to implement a Text Format encoder/decoder as well.
It could certainly be worth investigating, but what you're asking for is a complete redesign of the library internals and heavy code generation changes to fit a new API—that's a lot easier said than done. If somebody passionate about it wants to experiment with that and give us concrete data showing that all of the concerns stated above can be alleviated, feel free...

@tbkka maybe I missed it, but I didn’t see any discussion following the suggestions I made to remedy the issues you mention, namely feeding the codegen’d Schema / proto file into the hypothetical ProtobufDecoder(schema: xyz) initialiser.
The Codable type itself doesn’t need to know anything about the protobuf implementation, only the Swift types. This would fit alongside the other Codable implementations perfectly, and makes more sense anyway given that the extra info needed by protobuf is only relevant to protobuf and not to other Codecs (JSON, plist etc).
If this seems like nonsense to you it’s possible we’re talking about different things: can you elaborate as to what you mean?

@allevato thanks for your response, I’m not referring to JSON protobufs (nor was the OP from what I can tell), but rather the standard binary implementation.
Are you suggesting there are no runtime checks as to whether the data types are correct in Swift-Protobuf? I’m surprised by that revelation, but if it’s true you’re probably right that the implementations would differ enough to put this into the “too hard” basket.

I mentioned binary protobufs as well—we need to carry additional information than just the Swift type in order to encode it (field number and wire format), and our implementation is already highly tuned for performance because it's intentionally not generic in a lot of places and my concern is that a Codable-based implementation would have significant difficulty hitting those same numbers.

For future reference for anyone keen to implement a binary protocol for Codable, including deciding to map to Protobuf:
Mike Ash wrote a great article about it on 2017-07-28 including source on GitHub
I understand the appeal very strongly as a way for people to easily get into Protobuf but share concerns that it bypasses Protobuf's toolchain.
An interesting alternative would be using the Codable protocol to generate .proto files, then use the rest of the toolchain.

My $.02... This comment very succinctly seems to describe the basic problem with using protobufs in any problem domain:

Codable is meant to allow arbitrary data structures to support serialization in multiple formats. Protocol buffers are specifically generated data structures that support serialization in a finite number of spec-defined formats. The use cases simply don't line up.

Having said that, it seem therefore reasonable to limit the use cases of protobufs with Codeables to protobuf's own use cases. In other words, if one were to try to serialize a Codeable object whose properties were not protobuf-compatible, the system should alert the programmer with a fatal error at build time.
Finally, it seems to me that this suggestion would be the best path forward:

An interesting alternative would be using the Codable protocol to generate .proto files, then use the rest of the toolchain.

Codable is meant to allow arbitrary data structures to support serialization in multiple formats.

Except that it doesn't really support arbitrary structures - it's only for flat structures, failing to support polymorphism via classes or protocols. Unless I have massively misunderstood something, you have to do a lot of work writing your own wrappers to get polymorphic decoding. It only recreates base classes.
I explored this a bit in [some playgrounds[(https://github.com/AndyDentFree/perswifting) ending up doing my own implementation to get easy complex graph support.
(I hate wheel re-invention or any approximation to it, so would love someone to correct my misunderstandings.)

I think like many others up thread, the hope was that something like protobuf would come along and take our existing Codable models, which were up until now serialized to JSON, and instead use a binary format. Unfortunately, maybe protobufs isn’t a good fit for this use case.

FWIW, MsgPack (in particular, this implementation) seems to be both faster (10%) and smaller (about 40%) than the PropertyListEncoder.

Looks like MessagePack follows a different wire specification, so it isn't an exact match.

Yes, this comment was written based on the premise that protobuf and Codable is not a good match due to the dynamic/static mismatch.

I'm inclined to close this.
To summarize:  There are a few key obstacles for this idea, fundamental mismatches between how Codable works and how Protobuf works:

Protobuf serialization records field numbers.  (Indeed, this is a major part of protobuf's size efficiency.)  Swift Codable has no such concept; it traffics in field names.
Protobuf has multiple wire formats for each Swift data type.  For example, protobuf has several different "int32" encodings that are optimized for values that are mostly positive, mostly small, or often large.  Codable has no such concept.

To resolve the above issues, there aren't a lot of options:

You could generate the requisite information automatically. (e.g., number fields in definition order, choose a single encoding for each swift format). But this would preclude interoperating with other protobuf-using systems, which have that information explicitly specified in a .proto file.  I think of this as "not really protobuf."
You could work out some way to augment the Swift definition (using macros, perhaps).  This would require every data type used by your system to be explicitly annotated for use with this coder, which would make the coder useless with other Swift data types that supported Codable.  I think of this as "not really Codable."

I think it would be quite interesting for someone to start a new project that used protobuf concepts to develop a new encoder that works with Codable.  You're even welcome to copy code from SwiftProtobuf (within the restrictions of our Copyright License, of course).  But I still see no point in the SwiftProtobuf project attempting this.


Epns juicebox goodkarma

Add EPNS, Juicebox and GoodkarmaDAO

@Rashmi-278 I have a qn about: protocols/graphprotocol.eth/gip.png is it the correct logo? Should it be named logo.png?

Hey, it's not logo it's an image for overview page. Logo is different.

Hey, it's not logo it's an image for overview page. Logo is different.

Do we care if logos are png or jpeg? 👀  @ yungapesquad logo


feat(NODE-3938): Add support for pre/post images in change streams

Description
What is changing?

Adds support for pre/post images in change streams (https://jira.mongodb.org/browse/DRIVERS-1915)
Adds tests for create collection and modify collection with pre/post image (https://jira.mongodb.org/browse/NODE-4210)
Includes changes from https://github.com/mongodb/specifications/commit/d1458823bd810014df9da16d3a5354d2269ab865 (this commit is linked in NODE-4210)

Is there new documentation needed for these changes?
No.
Double check the following

[x] Ran npm run check:lint script
[x] Self-review completed using the steps outlined here
[x] PR title follows the correct format: <type>(NODE-xxxx)<!>: <description>
[x] Changes are covered by tests
[x] New TODOs have a related JIRA ticket

@baileympearson could you rebase against main to get the versioned api tests passing, just to have a little more green? Then I'll merge. Thanks!


Unify module type formatting

Among other things, this fixes #2057
There's still some cleanup left to do, but I'd love to get some initial feedback on this.

Alright so how's this one?

The logic for module type presented in the PR:

break if there's a "row" with an attribute
break if there's more than one "row"

This is brings module type X = {...} closer to module X = {...}. The printing output is a very nice improvement.
There's one small comment that could improve the code by using a feature provided by easy_format  instead of doing that logic in userland.

I’ll look into @IwanKaramazow’s suggestion this evening

Since we’re gonna release a 3.3.1 https://github.com/facebook/reason/pull/2071, we can get this one in too

Just addressed the code review item – that part of the code looks much simpler now.

Has my blessing :)

Wooo


No aligned malloc implementation on MinGW

When building on Windows with MinGW, there's no available aligned malloc implementation:
configure:
checking for aligned_alloc... no
checking for posix_memalign... no
checking for memalign... no

meson:
Checking for function "aligned_alloc": NO
Checking for function "memalign": NO
Checking for function "posix_memalign": NO

This means that alloc_align() in src/bench/matrix.c ends up returning an uninitialized pointer. The implementation needs to be like this to actually show the bug at compile-time:
diff --git a/src/bench/matrix.c b/src/bench/matrix.c
index 2621243..d2d6a0a 100644
--- a/src/bench/matrix.c
+++ b/src/bench/matrix.c
@@ -58,6 +58,8 @@ alloc_align (gsize n,
   res = aligned_alloc (alignment, real_size);
 #elif defined(HAVE_MEMALIGN)
   res = memalign (alignment, real_size);
+#else
+#error "Need some type of aligned allocation function"
 #endif

   g_assert (res != NULL);

I noticed this because of this warning:
[40/61] Compiling c object<EMAIL_ADDRESS>../src/bench/matrix.c: In function 'alloc_align':
../src/bench/matrix.c:50:9: warning: unused variable 'real_size' [-Wunused-variable]
../src/bench/matrix.c:63:3: warning: 'res' is used uninitialized in this function [-Wuninitialized]

Maybe graphene should be built with werror=true in meson.build:project() so that -Werror is enabled, or perhaps that option can be enabled with a version check inside meson.build.

It looks like the recommended way to allocate aligned memory on MinGW is the same as with MSVC: _aligned_malloc. I have a patch that does this, but I'm still getting a crash in the matrix benchmark. Need to confirm it's not because of this before I send a PR.
PS: I tried building Graphene with MSVC and got approximately 1 million errors 😉. Will go through them and submit PRs.


update dataset & model to include HTML tags

builds on https://github.com/Police-Data-Accessibility-Project/data-source-identification/pull/27
we have a dataset: https://huggingface.co/datasets/PDAP/urls

[ ] include data from the HTML tag collector in the huggingface URLs dataset
[ ] train the model and see if it's improved

This will consist of adding the ml-url-classifier to the identification_pipeline and then including the tags as features in the model

@bonjarlow you mentioned this working—maybe not perfectly, but working—a few updates back. If you point to the code where that's accomplished in HF or GitHub, we can close this 👀


Update ArtifactOntology.ttl

Addressing https://github.com/CommonCoreOntology/CommonCoreOntologies/issues/160 .
I performed a scan of all IBEs that are said to bear some subrelation of aboutness and revised the wording.

@cameronmore  Mind taking a look at this one when you have a moment?

This shouldn't be done separately from the IBE/ICE refactor, which is being worked on in the CCO weekly grad student weekly, in particular by at least @avsculley and myself in the https://github.com/CommonCoreOntology/CommonCoreOntologies/tree/ice-iba-refactor. Let's coordinate.
It doesn't really deconflict in that e.g/ Chart probably shouldn't be in IBE at all.

@alanruttenberg  I was executing here only to update the definitions and not the broader of issue of what should and shouldn't be an IBE. Good to know you've been tackling this more comprehensively; however, I see your branch is now 35 commits behind (can you and @avsculley pull from development? Also, are @APCox @johnbeve  and @mark-jensen  tracking this refactor?

I believe they are aware. It is a WIP, but if not they are now :-)
Agree it should pull to update. But it should be noted that it isn't exclusive to us (there are already other students involved in requirements gathering) and I am hoping for wider participation in the discussion. Since it is a big change, it needs to be coordinated, which is the main point I wanted to make.

To be clear, I don't think these changes should be made now/here. They suggest progress on the IBE/ICE issue that isn't in sync with current thinking. It is an open question as to whether or not to retain a corresponding IBE terms for each ICE term. For example, it is rare to have an IBE chart about which we want to say something interesting. An example of where we might use such a term might be in referring to an original historical chart for which there is one copy. Or if access to specific printed copies of a chart needed to be tracked, though I would expect such tracking to be at a higher level like document.

@alanruttenberg "They suggest progress on X that isn't in line with current thinking, where current thinking considers X an open question" --seems like a recipe for the perfect murdering the good.
Can you give me an ETA for when your branch will be ready to merge in? We're planning to release a 1.6 just to get this repository looking respectable (maybe in the next month), and then a major 2.0 release where we seek truth and justice and break some things. Would you target the first or the second for this update?

I recently got a new job at SKS and haven't stayed on top of this. But I can set time aside this weekend to make sure that the work we've done is both consistent with the most recent updates to CCO, and also suitable for viewing.
Currently, the ICE/IBA refactor work is too tangled up with work on information structures that we were also doing. We decided to separate them since the ice/iba refactor stuff is more straightforward, and, if the community wants it to be implemented, could be ready much sooner than work on information structures. This became even more apparent in the recent NCOR CCO working group session on information.

From: Alan Ruttenberg @.>
Sent: Monday, August 5, 2024 9:33 PM
To: CommonCoreOntology/CommonCoreOntologies @.>
Cc: Alec Sculley @.>; Mention @.>
Subject: Re: [CommonCoreOntology/CommonCoreOntologies] Update ArtifactOntology.ttl (PR #319)
To be clear, I don't think these changes should be made now/here. They suggest progress on the IBE/ICE issue that isn't in sync with current thinking. It is an open question as to whether or not to retain a corresponding IBE terms for each ICE term. For example, it is rare to have an IBE chart about which we want to say something interesting. An example of where we might use such a term might be in referring to an original historical chart for which there is one copy. Or if access to specific printed copies of a chart needed to be tracked, though I would expect such tracking to be at a higher level like document.
—
Reply to this email directly, view it on GitHubhttps://github.com/CommonCoreOntology/CommonCoreOntologies/pull/319#issuecomment-2270195393, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATJNTNYK7OQNGL7IIYMCQEDZQARU7AVCNFSM6AAAAABL7K4X52VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENZQGE4TKMZZGM.
You are receiving this because you were mentioned.Message ID: @.***>

Thanks @avsculley!

If you are aiming for 1.6 soon then I would make the IBE/ICE changes in 1.7. As I said, I don't think this pull cleans anything up, because the major flaws remain and these changes don't move in the direction changes are going to happen, so they aren't making progress, IMO. I say 1.7 because the change will require several people to review and because we haven't made one of the decisions, namely whether or not to keep all corresponding IBE terms or deprecate most of them. I lean towards the latter. If that many deprecations happens it might be polite to give a longer warning. Which reminds me that we really need a discussion list or two. Probably a general questions and discuss list as well as an announce list, for messages like: There's a new release and maybe for the deprecation (though in the latter case requesting comments, if any, get directed to the discuss list). If a mailing list is set up, please make sure it has a public archive. Google groups is easiest. There's a milestone capability in Github. I haven't used it but it might be worth looking into for planning things like this.

@alanruttenberg Certainly agreed on the need for a listserv, foreshadowing, and the rest of it. I'd like to spend some time finding the best options here and not the biggest fan of google groups. Certainly github has a lot of bells and whistles too that we've only scraped the surface of.
On the matter of this change: I hear you, but I disagree. I think fixing these definitions aids in clarifying that these are not ICEs and that this will be marginally helpful to users in the interim between now and whenever the larger and more important matter is settled.

We agree to disagree. However I believe we aim for consensus. I don't think my claims unreasonable, and I have been spending time working in this area.

Hey Alan, can we maybe discuss briefly over the phone? Can be whenever.

I'm still working on the refactor. It should be coming sometime this week.

I agree with @alanruttenberg on the need for refactoring; I'm not exactly happy with the IBE subclasses...I'm looking eagerly towards @alanruttenberg's and @avsculley's results.
Even so, I'm inclined to update these IBE definitions (see my proposed revisions to @neilotte's updates) because the original definitions exhibit serious confusions, e.g. an information bearing entity can't represent. I'm worried that these confusions - if left unaddressed - will spread among new users who might be leveraging CCO before the refactoring is implemented.
With respect to these updates suggesting progress on the IBE/ICE issues that aren't aligned with current thinking, perhaps we can make it clear here and elsewhere that:

IBE/ICE refactoring is being researched
These updated classes may not persist through the refactoring
They're being updated nonetheless because of internal issues with the definitions, independent of the refactoring effort

@johnbeve Ready to review.

I agree with @alanruttenberg points re making changes here when there is good likelihood that they will be superseded by the much more comprehensive refactor of ICEs and IBA/Es in CCO. However, the timeline to completion and adoption is uncertain. Which makes sense. The refactor is requiring taking a more definitive stance of the nature of GDCs and ICEs. It will be a substantial update to CCO and require a decent amount of vetting by the community.
These changes reflected in this MR are small improvements, they are needed ones, the work has already mostly been done (see my suggestions), and I think they will transfer nicely into the ongoing refactoring efforts to move IBAs up and over to ICEs. Therefore, I approve them, as short-lived as they may be.

@neilotte, I'm on board with @mark-jensen's re-revisions and I believe the revisions overall address @cameronmore's concerns as well.
Folks on the thread agree refactoring is needed and we're looking forward to seeing what @alanruttenberg and @avsculley have done. The majority seem in favor of making these minor updates while waiting for the more substantial work to come.
I believe we all agree that these minor updates should not suggest to anyone that more substantial revisions to the information content in CCO are undesirable. We're simply revising terms in CCO that exhibit obvious issues while awaiting more substantial revisions.
So, barring further objections, I'm happy to approve this PR once the latest updates have been made.
FYSA @APCox @oliviahobai


In IL view, the empty opcode bytes space is highlighted when clicked

Obviously only happens when the Show opcode bytes option is on. Maybe treat this option as always off when switching to IL view?

Appears to be already fixed in dev.


welcome bot fix /signoff

This is the fix for the issue that was assigned to me at layer5io/meshery-linkerd
Signed-off-by: tonyanant<EMAIL_ADDRESS>

Thank you for contributing, @tonyanant!

Ha! You're work in-action, @tonyanant. 😆

@leecalcote, how about adding an invite to slack channel in the welcome message?

@JubayerJoy Sounds great.

Weldone @tonyanant this PR fixes #80


test_loader in train.py

Is the test_loader used in train.py actually validation data?
I'm wondering because it seems to be a data leakage if test data is used as validation.

Ok, thanks


Error in jsonrpc_stream_cost_estimate()

File "site-packages/twisted/internet/defer.py", line 498, in errback
File "site-packages/twisted/internet/defer.py", line 565, in _startRunCallbacks
File "site-packages/twisted/internet/defer.py", line 651, in _runCallbacks
File "site-packages/twisted/internet/defer.py", line 1355, in gotResult

--- <exception caught here> ---
  File "site-packages/twisted/internet/defer.py", line 1297, in _inlineCallbacks
File "site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
File "site-packages/lbrynet/lbrynet_daemon/Daemon.py", line 1700, in jsonrpc_stream_cost_estimate
File "site-packages/twisted/internet/defer.py", line 1297, in _inlineCallbacks
File "site-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
File "site-packages/lbrynet/lbrynet_daemon/Daemon.py", line 956, in get_est_cost_from_name
File "site-packages/twisted/python/threadpool.py", line 246, in inContext
File "site-packages/twisted/python/threadpool.py", line 262, in <lambda>
File "site-packages/twisted/python/context.py", line 118, in callWithContext
File "site-packages/twisted/python/context.py", line 81, in callWithContext
File "site-packages/lbryum/commands.py", line 127, in func_wrapper
File "site-packages/lbryum/commands.py", line 763, in requestvalueforname
File "site-packages/lbryum/network.py", line 838, in synchronous_get
File "Queue.py", line 176, in get

Queue.Empty:

This means that lbryum failed to get a response from lbryum-sever within the timeout of 30 seconds. Can happen on any command that requires lbryum-server.
Should be caught and logged properly so we can keen track of the frequency of these errors.

fixed in https://github.com/lbryio/lbryum/pull/82


CRUD operations

In issue https://github.com/jas-/crypt.io/issues/7 there is a discussion about updating and deleting entries. While it focused on an issue with outdated dependencies the main question did not seem resolved. (Or maybe I just read it too fast?)
Anyway, to clarify: Does Crypt.io have a full CRUD cycle built in? If so is there documentation for this?
Thank you!

No it doesn't. Setting and getting are the only two methods.

I'm trying to retreive all the encrypted keys but when the encryptPassword doesn't match it kills the function with uncaught exception: CORRUPT: ccm: tag doesn't match.
function testValue() {
    cleanStore();
    storage = cryptio;
    for (var i = 0, len = localStorage.length; i < len; ++i) {
        keyID = localStorage.key(i);
        keyObject = localStorage.getItem(localStorage.key(i));
        console.log(keyID + keyObject);
        // console.log(encryptPassword);
        storage.get({ passphrase: encryptPassword }, keyID, function(err, results) {
            console.log(err);
            console.log(results);
        });
    }
}
testValue();

Any idea how I would for loop through all the keys to see which one the encryptPassword matches? The idea is that the user could use multiple encryptPassword's for multiple entries in localStorage. Two users can use the same site on the same browser but with different results.

This functionality is outside of the scope this module provided

Fair enough. I'll likely write something similar to crypt.io but with CRUD then. Thanks.

Sounds like you don't want to use the nosql interface available with localStorage and sessionStorage and would prefer to use the indexedDB (since depreciated) to deal with CRUD.
The CCM error you are experiencing needs to be handled in the module but isn't currently

@spencerthayer v1.0.5 should address the CCM errors but I did not test it thoroughly due to my reservations about implementing this as a way to brute force encrypted stored data through a browser extension per domain.

NICE! Thank you.

Did that resolve your issue?

Sorry. I won't be able to look at this until Monday. I'm certain it did though.


⚠ ClamAV has degraded performance

In 339f2f6, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 204 ms

Resolved: ClamAV performance has improved in 7a6df70 after 5 minutes.


Better document parsing

Consider parent directories of resources, include all resource types, etc.

Even fix existing parsing, getting:
Unable to parse HTML document: http: read on closed response body


Send NewSessionTicket messages from server

From https://tools.ietf.org/html/draft-ietf-quic-tls-32#section-4.6.1:

To communicate their willingness to process 0-RTT data, servers send
a NewSessionTicket message that contains the early_data extension
with a max_early_data_size of 0xffffffff.  The TLS
max_early_data_size parameter is not used in QUIC.  The amount of
data that the client can send in 0-RTT is controlled by the
initial_max_data transport parameter supplied by the server.
Servers MUST NOT send the early_data extension with a
max_early_data_size field set to any value other than 0xffffffff.  A
client MUST treat receipt of a NewSessionTicket that contains an
early_data extension with any other value as a connection error of
type PROTOCOL_VIOLATION.

Closing for now until 0-RTT is scoped out.


Add lax_deserialize feature

CC ruma/ruma#99 @MTRNord @DevinR528 @iinuwa @poljar
Left to do:

docs
tests

Nice. Just a thought, is it worth being able to configure how lax the deserialization is? Specifically, 5.0 -> 5 doesn't lose any data, but 5.1 -> 5 does. Could the former be considered "non-lax"?

I did think about that. It would be possible, but then I should probably hand-roll deserialization instead, so 10000.00000000000000001 is rejected, since that would be accepted with f64 deserialization + checking whether the fractional part is 0. It may be the better path forward either way, since I'm not entirely sure MIN_SAFE_INT - 1 and. MAX_SAFE_INT + 1 are handled properly right now (they probably are).

Thanks for this, I'm already fine with the initial implementation under the assumption that MIN/MAX_SAFE are handled correctly.
Checking the fractional part sounds nice as well so no objections here.

After thinking more about this, since js_int is not tied to deserialization from JSON, hand-rolling parsing is impossible or at the very least doesn't make sense. I don't see there being a good solution for "only-if-fractional-equals-zero".

This is what I was thinking at first (this is more just in case, I am fine with the current implementation also).
This has a lot of allocation for such a small task
impl<'de> Deserialize<'de> for UInt {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let val = f64::deserialize(deserializer)?.to_string();
        match val.split('.').collect::<Vec<_>>().as_slice() {
            &[int] => {
                let num: u64 = int.parse().map_err(D::Error::custom)?;
                Self::new(num).ok_or_else(|| {
                    D::Error::invalid_value(
                        Unexpected::Unsigned(num),
                        &"an integer between 0 and 2^53 - 1",
                    )
                })
            }
            &[int, dec] => {
                if !dec.replace("0", "").is_empty() {
                    return Err(D::Error::custom("expected an integer found a float"));
                }

                let num: u64 = int.parse().map_err(D::Error::custom)?;
                Self::new(num).ok_or_else(|| {
                    D::Error::invalid_value(
                        Unexpected::Unsigned(num),
                        &"an integer between 0 and 2^53 - 1",
                    )
                })
            }
            _ => Err(D::Error::custom("found invalid number")),
        }
    }
}

This is what I was thinking at first (this is more just in case, I am fine with the current implementation).
This has a lot of allocation for such a small task
impl<'de> Deserialize<'de> for UInt {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let val = f64::deserialize(deserializer)?.to_string();
        match val.split('.').collect::<Vec<_>>().as_slice() {
            &[int] => {
                let num: u64 = int.parse().map_err(D::Error::custom)?;
                Self::new(num).ok_or_else(|| {
                    D::Error::invalid_value(
                        Unexpected::Unsigned(num),
                        &"an integer between 0 and 2^53 - 1",
                    )
                })
            }
            &[int, dec] => {
                if !dec.replace("0", "").is_empty() {
                    return Err(D::Error::custom("expected an integer found a float"));
                }

                let num: u64 = int.parse().map_err(D::Error::custom)?;
                Self::new(num).ok_or_else(|| {
                    D::Error::invalid_value(
                        Unexpected::Unsigned(num),
                        &"an integer between 0 and 2^53 - 1",
                    )
                })
            }
            _ => Err(D::Error::custom("found invalid number")),
        }
    }
}

Out of interest, do you need to convert the float to a string? Can't you check if the float is a number using is_finite() and then get to the fractional part using fract()?
Converting a string to a f64 and then back to a string seems a bit much, am I missing something?

I've added tests and they are passing, at least when increasing serde_jsons float parsing accuracy 😅
CC serde-rs/json#692

I thought all tests passed locally. Seems unrelated to the new logic, will investigate later.

Out of interest, do you need to convert the float to a string? Can't you check if the float is a number using is_finite() and then get to the fractional part using fract()?

Yeah, that would have been much better. I didn't look closely enough at the f64 methods thanks for the tip @poljar that'll come in handy.

Can't you check if the float is a number using is_finite() and then get to the fractional part using fract()?

I guess I didn't make this clear enough before: the reason I don't like this (which equally applies to @DevinR528's solution though) is that floats with a non-zero fractional part that can't be represented by f64 would still be accepted, so the check would be imperfect. At that point I'd rather throw away the fractional part altogether.
Examples:
10000.00000000000000001f64 == 10000.0f64
9007199254740991.1f64 ==<PHONE_NUMBER>740991.0f64

Can't you check if the float is a number using is_finite() and then get to the fractional part using fract()?

I guess I didn't make this clear enough before: the reason I don't like this (which equally applies to @DevinR528's solution though) is that floats with a non-zero fractional part that can't be represented by f64 would still be accepted, so the check would be imperfect. At that point I'd rather throw away the fractional part altogether.

Oh the question wasn't really pointed at you. I didn't mean to ask why you're not checking it that way, rather it was pointed at @DevinR528 since the string dance confused me and seemed pointless.
Sorry I thought that quoting the code snippet would make it clear.

And my explanation was mostly aimed at devin too 😉 😆


manuscript edits

I'm not sure how I cloned the repo in the previous PR but I think this should work now. I was mindful of the changes that Dr. Meyer made on the manuscript and retained those edits as well.
On the discussion.md file of the manuscript, I had looked back on previous merges and fixed the sentence to say Also, the model is dependent on the distance between the existing states and may perform poorly when the states are close or when the susceptible cells possessed high variance in parameter estimation because it was incomplete before. Please let me know if this change should be edited! :)

@Farnazmdi No worries! I kept the conflicts that were from the master branch and removed the ones from jc-manuscript! Please let me know if there's anything else I should adjust :)

Was there a change to the AIC figure? This PR was building earlier but now it is not.


Update the adaptive demo

Title:
Adds code examples using the adaptive optimizer to the adaptive demo.
Summary:
Relevant references:
Possible Drawbacks:
Related GitHub Issues:

I am fixing the issue, it is not related to this PR.
I'll let you know here when it's ready :)

Is there a reason this demo is being made to dev and not master?

Is there a reason this demo is being made to dev and not master?

I created it, locally, before the release. Will be rebased to master.

Thanks for the update @soranjh ! 🚀😄
Sorry to have postponed the merge of this so much


what is wss_link?

I cannot find a reference on wss_link. Please update the README?

wss_linkis the Websocket address for connect to Microsoft's server.

In order to build a new proxy server when the API interface is banned in China.