new --inpaint_replace option for inpainting

add a --inpaint_replace option that fills masked regions with latent noise. This allows radical changes to inpainted regions at the cost of losing context.
fix up readline, arg processing and metadata writing to accommodate this change
fixed bug in storage and retrieval of variations, discovered incidentally during testing
update documentation

There's definite changes.
Inpainting this image to give it a green eye. This is without inpaint_replace

This is with inpaint_replace

I like this control. Generated quite a few images. Don't seem to have any obvious bugs. In a lot of cases I felt like inpaint_replace should actually be the default.
What do you think about adding an option called inpaint_strength -- with inpaint_strength 1 does what inpaint_replace does and the values under will control the strength at which the inpainting is placed on the base canvas?

Also inpaint_replace seems to work only when the transparency on the image is 100% ... Which mean its all the more an inpaint_strength 1  feature.

I have changed the --inpaint_replace option to take a strength argument ranging from 0.0 to 1.0. At 0.0 levels it behaves like default inpainting and pays full attention to the image under the mask. At 1.0 levels it behaves like the proposed inpaint_replace and completely ignores what was under the mask. Intermediate values lead to interesting effects.


charterafrica production deployment

Description
This PR updated charterAfrica production deployment.
Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

@kelvinkipruto this PR looks good to me, but before merging we'll need to have a production db snapshot taken then run migrations dokku run charterafrica pnpx payload migrate


Insert.py get rid of -w flag

Insert.py get rid of -w flag

@sankalpatimilsina12 @GportDev Was this merged? I remember discussing this but don't remeber the resolution.

I think it was. The -w flag is working as expected

Expected outcome - client uploads to one node (e.g., node5). After insertion, node5 notifies others. The other nodes get the data from node5 and not from the client.


feat(FSADT1-751): code cleanup and external libs

Description
Please provide a summary of the change and the issue fixed. Please include relevant context. List dependency changes.
Fixes # (issue)
Type of change

[x] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] Documentation update

How Has This Been Tested?

[x] Updated existing tests

Checklist

[x] I have read the CONTRIBUTING doc
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] Any dependent changes have already been accepted and merged

Further comments

Thanks for the PR!
Deployments, as required, will be available below:
Any successful deployments (not always required) will be available here
Please create PRs in draft mode.  Mark as ready to enable:

Analysis Workflow

After merge, new images are deployed in:

Merge Workflow

Closing this task. Due to the way our organization and repositories are set, the Maven packages published are owned by the BCGov GitHub organization. Due to that and some other maven package issues on GitHub, it is not possible to allow other repos to consume the packages without the use of PAT (personal access tokens).
Maybe in the future, we get back to this initiative and once again refactor the code to allow this, but for now, we are closing this as a failed attempt.


Better Error reporting if loaded vue has an issue

Is your feature request related to a problem? Please describe.
When errors happen on loaded vue3-sfc-loader there is no way to know the exact issue or line causing the issue.
Describe the solution you'd like
I am loading multiple components dynamically to render my pages. I have a page called User.vue that loads the following sections, top.vue, search.vue, table-list.vue, and footer.vue. If an Issue happens on any of the loaded vue there is no way to tell the file causing it. The error shown reads:
runtime-core.esm-bundler.js:244 SyntaxError: Unexpected token (12:33) (at vue3-sfc-loader.js:13:179309)
    at $e._raise (vue3-sfc-loader.js:13:179309)
    at $e.raiseWithData (vue3-sfc-loader.js:13:179248)
    at $e.raise (vue3-sfc-loader.js:13:178704)
    at $e.unexpected (vue3-sfc-loader.js:13:198674)
    at $e.checkExpressionErrors (vue3-sfc-loader.js:13:199845)
    at $e.parseMaybeAssign (vue3-sfc-loader.js:13:209562)
    at vue3-sfc-loader.js:13:208690
    at $e.allowInAnd (vue3-sfc-loader.js:13:238585)
    at $e.parseMaybeAssignAllowIn (vue3-sfc-loader.js:13:208669)
    at $e.parseExprListItem (vue3-sfc-loader.js:13:234507)

Describe alternatives you've considered
If the vue3-sfc-loader could catch the error and raise another highlighting the file causing the issue it would help.

Same here, any idea how to get usable error text?

Hi, do you pass a log function in the options ?
ex.
	log(type, ...args) {

		console.log(type, ...args);
	},

If so, can you give me an example where error is not properly reported ?

Hi, I finally solved it with the same code you sent, and it's good enough to debug, only SyntaxError: Unexpected token errors are hard to find, as dedanirungu has mentioned too. I got the same error, and I don't know what causes exactly the syntax error (e.q. ";" missing somewhere).
Anyway I very-very appreciate your work, I made a very complex CMS and online code development framework with version control system, and your sfc-loader has a huge part of it, so big thanks to you!
The only disadvantage is the speed performance, if you have any idea how to speed it up please give it some efforts. With some visual tricks I made our system to look a bit quicker, so it's not so bad, but a little speeding up would be great.

Hi grcrozzo, thank you for your support !
Since this lib compiles code on the fly, client-side, I don't have many options for improving performance.

Hi, you're welcome, I know you have much more work in it than that little paypal support.
Yeah, I know compiling on the fly will be always slower, anyway by time it will be faster automatically with newer and faster phones and notebooks, so I'm satisfied with it and with the visual tricks I made on loading and page navigation.
Great work man, I give compliment rarely, but you and the component are awesome!

on second thought, the compiled version of the code may already be saved on a server using the async compiledCache API...

First thanks for thinking about our problems, it's very kind from you!
Well, I was thinking about it before, but as I mentioned, it's a very complex version control and publishing system either, so the time we could win with it we would lose on server side while generating page and component sources for different branches/versions. We make this dynamic generation in stored procedures from codes stored in SQL tables, and it's very hard to handle theses trees, so we would lose time there. Storing compiled versions seems obvious and a good idea, but could be hard to integrate into our system and our servers would use much more resources while serving the requests.
But as I mentioned it's not so slow, noone said that our system is slow, it's only a bit slow to me because I compare it to precompiled apps I made before, but none of our partners said that it's slow, all of them are pretty satisfied with it.

Dear Franck,
first of all: I need some advice, but of course I don't expect it for free, if you have some idea I would be glad to pay for your help.
I can load a component from SFC, works well, but is there a way to export the loaded/compiled component to e.q. base64 string? I tried to save it with JSON.stringify(component), but functions (e.q. methods...etc.) are not exported, so it doesn't work.
My goal is to load and SFC, make the component with your loadModule function, save the result component to DB, and later in another page load I would load this saved (precompiled) component into the app (not the SFC). Is this possible? I would not save/compile the whole app, I need compiled version of a lot of components, and dynamically load some of them into the app in main.js.
Here is what I mean exactly, this should be saved and later loaded:

Please let me know if it's possible and if you could help with this, as I mentioned I would be glad to donate for your assistance.
Best regards,
Roland

Hi Roland, I accept to give you assistance. Fell free to contact me at<EMAIL_ADDRESS>or open a new topic in discussions if you prefer a public discussion.


Updated layout to accomodate manual installs

Added github to all

!pull

Help.
For some reason, even though version 0.3.7 is on kerbalstuff, this is failing because it is downloading 0.3.4 of the unmanned contracts

My problem was a lowercase D in Gamedata.
It might make sense to add a check for the proper case for the gamedata, and alert the user if it is wrong

You could cut the adresses for KS and KSP forums down to not include the flavourtext after the modnumber and threadnumber. They'll still work and won't look weird in the future if the name of the mod on KS or the name of the thread on forums change. Seeing as you're changing stuff around that change might make sense to do now rather than later :)

Understood, I'll do that the next time I change things.
Thanks

Seeing as you closed this, am I to understand that you don't want these changes merged?

oops. Ummm, I just pulled everything and checked, I don't see the flavourtext there anymore.  Did you make those changes?

For some reason, it is pulling the wrong version of spacetux:
Downloading "https://github.com/jrossignol/ContractConfigurator/releases/download/0.7.11/ContractConfigurator_0.7.11.zip" (libcurl)
Downloading "https://ksp.sarbian.com/jenkins/job/ModuleManager/lastSuccessfulBuild/artifact/ModuleManager-2.6.0.zip" (libcurl)
Downloading "https://kerbalstuff.com/mod/656/Spacetux Shared Assets/download/0.3.2" (libcurl)
When version 0.3.3 is available for the Spacetux Shared Assets

#rebuild

I haven't done anything to this PR other than commenting on it. What flavourtext are you refering to?

Umm, I think we are crossing messages.  I was referring to your earlier message, and then got confused (I think) by your comment.

#rebuild

Either way, the reason it's downloading 0.3.2 is because it seems 0.3.3 hasn't been pushed to CKAN-meta by the bot yet. Someone will probably give it a swift kick again in the future and it'll fix itself.

Ok, so I'll wait until this evening.
Thanks

#rebuild

#rebuild

For some reason, it is pulling the wrong version of spacetux:
Downloading "https://github.com/jrossignol/ContractConfigurator/releases/download/0.7.11/ContractConfigurator_0.7.11.zip" (libcurl)
Downloading "https://ksp.sarbian.com/jenkins/job/ModuleManager/lastSuccessfulBuild/artifact/ModuleManager-2.6.0.zip" (libcurl)
Downloading "https://kerbalstuff.com/mod/656/Spacetux Shared Assets/download/0.3.2" (libcurl)
When version 0.3.3 is available for the Spacetux Shared Assets

#rebuild

#rebuild

#rebuild

#rebuild

#rebuild

Will ModuleManager be added back or was it removed due to not Being required anymore?

It wasn't required, i assume you are referring to the grandtour?
On 4/16/2015 9:43 AM, Willhelm Rendahl wrote:

Will ModuleManager be added back or was it removed due to not Being
required anymore?
—
Reply to this email directly or view it on GitHub
https://github.com/KSP-CKAN/NetKAN/pull/933#issuecomment-93737201.

I was indeed, everything looks fine so I merge this now :)

Thank you
On 4/16/15 9:48 AM, Willhelm Rendahl wrote:

Merged #933 https://github.com/KSP-CKAN/NetKAN/pull/933.
—
Reply to this email directly or view it on GitHub
https://github.com/KSP-CKAN/NetKAN/pull/933#event-283065483.


[FeatureRequest]

Feature Description / 功能描述
A clear and concise description of what you want to happen. / 请描述你想建议加入的新功能。
Reason / 理由
Please describe why this feature should be added. / 请描述为什么应该加入这个功能。
Suggest Solution / 实现方案
(Optional) Possible ways to implement this feature / （可选）如果你已想出新功能的实现方案，请在这里写出。

It seems to be a mistake. Please feel free to create a new issue.


[Bulk] Validation: Fix Odatadocs 2

This PR fixes 3 issues in Odatadocs.

Docs Build status updates of commit 0814c21:
:warning: Validation status: warnings

File
Status
Preview URL
Details

Odata-docs/connectedservice/generating-code-for-vb-clients.md
:warning:Warning
View
Details

Odata-docs/connectedservice/t4-templates.md
:warning:Warning
View
Details

Odata-docs/webapiauth/overview.md
:warning:Warning
View
Details

Odata-docs/changelog/odatalib-7x.md
:bulb:Suggestion
View
Details

Odata-docs/webapi/work-around-for-singleresult-create.md
:bulb:Suggestion
View
Details

Odata-docs/changelog/aspnetcoreodata-8x.md
:white_check_mark:Succeeded
View

Odata-docs/changelog/odatalib-5x.md
:white_check_mark:Succeeded
View

Odata-docs/changelog/odataopenapi.md
:white_check_mark:Succeeded
View

Odata-docs/changelog/webapi-7x.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/create-data.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/data-model.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/delete-data.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/derived-types-adv.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/get-data.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/queryoptions-overview.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/queryoptions-usage.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/update-data.md
:white_check_mark:Succeeded
View

Odata-docs/concepts/url-components.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/accessing-endpoints-with-auth.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/choosing-what-to-generate.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/generating-multiple-files.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/getting-started.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/reference.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/updating-generated-code.md
:white_check_mark:Succeeded
View

Odata-docs/extensions/migration.md
:white_check_mark:Succeeded
View

Odata-docs/includes/appliesto-odataclient-v6.md
:white_check_mark:Succeeded
View

Odata-docs/includes/appliesto-odataclient-v7.md
:white_check_mark:Succeeded
View

Odata-docs/includes/appliesto-odatalib-v6.md
:white_check_mark:Succeeded
View

Odata-docs/includes/appliesto-odatalib-v7.md
:white_check_mark:Succeeded
View

Odata-docs/includes/appliesto-webapi-core-v7.5.md
:white_check_mark:Succeeded
View

Odata-docs/connectedservice/generating-code-for-vb-clients.md

Line 20, Column 97: [Warning: file-not-found - See documentation] Invalid file link: '../getting-started'.

Odata-docs/connectedservice/t4-templates.md

Line 94, Column 66: [Warning: file-not-found - See documentation] Invalid file link: '../updating-generated-code'.

Odata-docs/webapiauth/overview.md

Line 19, Column 15: [Warning: file-not-found - See documentation] Invalid file link: './getting-started'.

Odata-docs/changelog/odatalib-7x.md

Line 1477, Column 3: [Suggestion: column-header-missing - See documentation] Column headers are required for tables. Add appropriate header text, or remove any extra table cells that cause an extra column.

Odata-docs/webapi/work-around-for-singleresult-create.md

Line 2, Column 1: [Suggestion: description-missing - See documentation] Missing required attribute: 'description'.

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Add Computer Architecture/Organization courses

Prerequisite for both is CS1-2. Either course (Berkeley and Carnegie Mellon) is suitable as an alternative to nand2tetris and/or 6.004 as the required course of choice for Computer Architecture/Organization. The Princeton course that's already here would have one of these 4 courses as a prerequisite and is suitable as an elective.

I'm happy to include these for now. There is future work to do to clean up the extras pages. The current criteria is essentially: Any online course that is good that is not in the official curriculum. That seems overly broad. Once we have a more appropriate criteria, we can examine which courses belong and which do not.


Allow JX-serialization of cyclic objects

I know JSON specifically doesn't allow cyclic objects to be serialized, but it would be useful if JX could do so.  Obviously round-trip serialization would be impossible, but maybe Duktape could put some sort of backreference in the output when it encountered a cycle, similiar to how it adds { _func:true } tags for otherwise unserializable functions.
Currently variable viewing in the minisphere debugger is limited because there are no commands to walk objects, so I have to rely on Duktape.enc and an Eval command to display the full object.  This fails, however, in the case of a cyclic object (which it turns out happens a lot in the game I'm making).

This should be relatively simple to implement. Here's a quick hack branch of one approach (it's a hack in that it breaks JSON.stringify() output for example):

https://github.com/svaarala/duktape/compare/cyclic-jx-gh331

Example:
$ ./duk
((o) Duktape 1.2.99 (v1.2.0-583-g81c2e5d-dirty)
duk> obj = {};
= [object Object]
duk> obj.foo = {}
= [object Object]
duk> obj.foo.bar = {}
= [object Object]
duk> obj.foo.bar.quux = { ref: obj.foo }
= [object Object]
duk> Duktape.enc('jx', obj)
= {foo:{bar:{quux:{ref:{_loop:true}}}}}

The plumbing is a bit awkward so maybe reorganizing the helpers a bit would be needed.

Even more useful would be if there were some way for the _loop tag to specify what it looped back on, but that would probably be too difficult to implement and/or hurt performance.
As long as all the unique objects are present in the output I'll be happy. :)

Looked at the diff, I'm impressed at how few changes were needed for a minimal implementation.  I thought this would be a lot more difficult!

One possible format would be something like:
{_loop:(0xdeadbeef)}

The pointer there could be the duk_heaphdr address of the object. But since it's not present in the serialized form of the target object, this only really serves to indicate if there are multiple separate loops.
A more fancy version would be to serialize the _loop tag and then perhaps serialize the target shallowly (just one level) which would mostly allow identifying the target. For example if the target object has an .id or .type property or similar, that would be shown. But this will essentially serialize certain objects twice so it seems quite messy to me.

I was thinking more a "path" of sorts to identify the loop (e.g. _loop:"foo.bar").  This would always be meaningful since by definition a cycle is self-contained.

Oh, and additionally, it would allow deserialization with the circular refs intact.

Oh, I see. That information would need a walk through the loop to get the property name list.
This is not as easy as it seems, because there may be multiple "branches" in the walk path and multiple independent loops.

It would also be quite interesting with accessors where every time you tried to walk the property chain you might get a different result from each accessor. Corner case, sure, but the JSON code needs to deal with all cases.

Yeah, like I said it's probably not possible to implement without some major changes under the hood.  Just figured I'd throw it out there. :-)

By the way, regarding the debugger heap walking, I think the easiest approach to ensure safety of heap walking would be to prevent both mark-and-sweep passes and refzero queue freeing while the debugger is in paused state. That way all heap object references would always point to valid memory (even when their refcounts dropped to zero during debugging - e.g. for the result of an Eval command).
The downside would of course be that you could run out of memory while paused if you did too many Evals for example.

Actually, getting the property name list would probably be possible through the loop detection stack; it already stores the objects being processed (for the slow path this is a hash map but it could also have an additional array). It would then need to store the keys through which the loop stack has been accumulated. This would then give the property list when one walked back the stack until the loop target was found in the stack.
But yeah, it'd be quite a lot of work and unnecessary book-keeping when one didn't specifically want to have it.

Pushed a slightly better hack which doesn't break JSON (should only affect JX):
$ ./duk
((o) Duktape 1.2.99 (v1.2.0-584-g75705c8-dirty)
duk> obj1 = {}; obj2 = {}; obj1.ref1 = obj2; obj2.ref1 = obj1; obj2.ref2 = obj1;
= [object Object]
duk> Duktape.enc('jx', obj1)
= {ref1:{ref1:{_loop:true},ref2:{_loop:true}}}
duk> JSON.stringify(obj1)
TypeError: cyclic input
	duk_bi_json.c:1432
	stringify  native strict preventsyield
	global input:1 preventsyield

Agreed on suspending GC during debugger pause.  With enough ingenuity it would probably be possible to segfault the target through the debugger, which is obviously not optimal. ;)

It doesn't need ingenuity really - if you just use Eval to arrive at a result object, that object won't be reachable once the Eval is complete. If Eval was changed to provide a "walkable reference" it would immediately have a stale pointer if refzero queue was freed.
A less invasive approach would be to have commands to "enable walking" and "disable walking" which would control the gc freeze. Slightly tedious for the debug client, but would limit the out-of-memory risks only when walking was actually done.

Alternatively, there could be a command the client could send that would be equivalent to Duktape.gc(), which could be requested every once in a while to drive the GC and prevent running out of memory.
Anyway, this should probably go in a separate issue.

Just tried this out, it seems to work well.  I was quite surprised to see where the loops in the object were, they weren't where I expected them to be!  This is very useful for debugging.
It did serve to illustrate how much JX encoding would benefit from the recent JSON speedups, though, the objects in question (representing battlers in an RPG) turned out to be quite large and took a few seconds to serialize.

Wow, so either there's a bug in this causing it to not detect a loop properly, or the object I'm trying to view is hella complex--when attempting to view one the above-mentioned battler objects, minisphere's RAM usage shot up to nearly 2GB for about almost a full minute, and then the debug client locked up the machine trying to deal with what must have been an enormous string... :stuck_out_tongue_closed_eyes:

There might be e.g. a value stack handling bug there, it was just a quick test.
But it's quite easy to generate much larger output than input with just loop detection in place. For example, suppose you have some object that serializes to 10kB of JX output. Then create a list of 1000 such objects (1000 separate references to that object). While the heap is not much larger (the references take around 16kB), the output is 1000x larger, i.e. 10MB. Then create another list that references that 1000-object list. Each reference there costs 16 bytes of RAM but generates 10MB of output. There are no loops - each object is just visited a lot of times.
This is actually why loop detection alone is not very useful IMO for debugging or logging values (I'll post a separate note on that). That's my experience anyway, with just loop detection you'll sometimes get ridiculously large output because of some accidental reference somewhere which causes some object to reference some self-contained but huge object (say some config or state object).
Anyway, for debugging, heap walking would be the best approach because the debugger could then allow the user to expand values, trigger getter calls, etc.

Anyway, for JX formatting I had roughly the following option list:

Detecting loops and stubbing them as {_loop:true}
Detecting a depth limit and stubbing objects beyond that limitas {_depth:true}
Detecting visited objects and avoiding their re-serialization (even if they're outside loops), stubbing them as {_visited:true}
Avoiding getter calls and stubbing them as {_getter:true}
Avoiding Proxy traps and stubbing them as {_proxy:true}
Avoid .toJSON() calls, as they cause side effects
Truncate large strings and buffers to specified limit, indicate truncation and real length, e.g. |deadbeef|(256).
Serialize strings as raw byte data (allows exact serialization of non-UTF-8 strings)
Detect a maximum serialization output length, and truncate whole output at that point (basically throw an error, but keep the output, and stick a ... after)
Serialize non-enumerable properties too; but perhaps only of the level 1 object and not of parents (might need an option for parents or depth limit for non-enumerable properties)

In a project unrelated to Duktape we have most of these incorporated in our serializer, and it's quite useful as the serializer is usable in much more situations than plain JSON would be. But it does mean quite a lot of options, because these contexts are quite different.
There's a place for these in Duktape.enc() so that one could e.g.:
Duktape.enc('jx', val, {
    loop: true,   // detect loops
    depth: 3,     // restrict to max depth 3
    size: 4096,   // limit max output to 4kB
});

This would then also need to be available for the C API (if one is added) where things get quite annoying: representing those options in a version extensible C API is a bit of an issue.
I originally looked into this for logging: it would be very useful if no side effects were triggered and there was a guaranteed maximum output size (it's rarely useful for a log entry to be beyond a few kB longer, even if legitimate). For JX to be truly useful for logging, one will need most of the above options to cover all relevant cases occurring in application code.
I left these out from Duktape 1.0 originally because there's some footprint impact. Also some features need new tracking state (like detecting "visited" objects and not just "looped" references). Finally, I didn't have time to figure out the C API for accessing these options (using a boxed option object seemed lazy :) and ended up leaving JX/JC out of the C API at that time.

I'll have to look into the objects in question to see exactly what properties they maintain.  If I really am generating 2GB of data by serializing it, that's going to be an issue for game saving. :)

The only reason a non-cyclic structure would be "amplified" if some objects get repeatedly serialized, i.e. they are referenced multiple times. That's obviously a problem for saving game state, because when you load it back each such re-serialization will be loaded back into a separate object which is probably not intended.
If the are multiple references to the same object for some legitimate reason, making all but one of the references non-enumerable causes JSON and JX to skip the property during serialization.

For one, a dump/load mechanism should handle at least DAGs (non-looping structures where individual objects are referenced multiple times) and preferably arbitrary object graphs. That's why using JX, modifications or not, won't ultimately be good in the debugger because it loses the information on object identities and doesn't support arbitrary graphs.
Debugger heap walking would preserve that identity information correctly. It'd then be up to the debug client to figure out how to pull in the graph - level by level as the user expands nodes, or as an entire graph, etc. But it'd be nice to do generic graph handling in the debug client and not on the potentially low capability target.
For non-debugger use a generic dump/load handling graphs would be useful but it would be a separate mechanism with probably quite little code sharing with JSON/JX code. It would need to represent property attributes, etc.

Yeah, so it turns out that each BattleUnit holds a backreference to the battle engine object, which in turn holds ALL of the units in the battle, plus the Session data which is quite a large object itself.  Long story short, nearly the entire game state is reachable through a single BattleUnit object.
I'll agree that JX is far from optimal for my use case, but at least for right now, it's all I have if I want objects to display meaningful values in the debugger. :-)

Right, if you want a quick fix for that, make that backreference non-enumerable using Object.defineProperty(obj, 'backRef', { enumerable: false }).

Thanks for the quick hack, this seems adequate until you can come up with a better solution. :)

Well there really isn't much of a better solution unless there's an actual "visited already" map (which is very different from "currently visiting"). Not to mention that is definitely not what one wants by default, because JSON behavior is serialize objects multiple times if need be.
So supporting a "visited already" list would need a separate option, and in general adding options to JX serialization. It's still a good idea I think but not a trivial change :)

So will this be merged as-is or is it contingent on #332?

It's technically an incompatible change so at least it needs some thought.

Hm, I don't know if I would consider this specific hack incompatible as it doesn't break any existing code (it's merely lifting a restriction), it's more like adding functionality which is allowed for minor releases under semantic versioning.  But you're the boss.

JSON.stringify() guarantees an error throw from cyclic objects. Why wouldn't that guarantee matter for some code for JX too?

But then, maybe that JSON guarantee is simply there to say "don't get stuck in cycles" which would be the other, ahem, implementation option.

Ah.  I didn't realize the error throw was specifically part of the contract.  See,I tend to view exceptions as, well, exceptional, rather than part of an interface.

Well it's part of JSON.stringify() contract. But the E5 specification doesn't really differentiate between "intentional" contracts and "accidental" contracts (which is quite hard to do in a specification).
For example, if the algorithm for a certain function specifies a certain argument coercion order, is that order an intentional guarantee or just there because there's no other easy way to specify algorithm than in steps?
Similarly, I'm not sure if anyone is actually relying on the cyclic object error. I guess someone could use it for something like:
function isCyclicHack(obj) {
    // Close enough :)
    try { Duktape.enc('jx', obj); } catch (e) { return true; }
    return false;
}

Many E5 features are actually used in a surprising way. This was interesting enough that I added a test case for it, for example: http://ecmanaut.blogspot.fi/2006/07/encoding-decoding-utf8-in-javascript.html :-)
(Granted JX is not an E5 features so there's much less existing code base and thus more leeway.)

Also, I'll have to look at the issue list and see if a 1.4 release makes sense before a 2.0 release. If not, then incompatible changes can start going into master and things like this can be merged as long as we agree they're good longer term changes.
I like semantic versioning but it's sometimes a bit hard to draw the line where bug fixes start and feature changes begin ;-)
Anyway, I'm off to sleep, cu :)

I guess this could be landed in 2.0 since it's an incompatible change.
I actually have less use for this now that heap inspection is in master, but cyclic JX would still be a nice thing to have. :)  We'd just have to figure out the most useful implementation.

@svaarala Now might be a good time to do something with this, otherwise we won't be able to do anything with it till 3.x ;)


Optional path params and path params with default value not working

The code generated for /{p:path} is if contains(urlPath, re2("^/([\\S]+)$")): can match correctly.
But /{p?:path} generates incorrect regex re2("^/{p?:path[m]}$"), path params with default value have the same issue.

Thanks for the quick reply.


Not an issue but how do I donate?

@abuibrahim2 Hello sir, I was trying to find a way to contact you but this is the only way. Just wanted to donate to show my appreciation because I’ve been looking for this feature for a long time

@menawi  thank you so much for the appreciation and thoughtful question. I'm sorry I haven't been posting updates since the original but there are more features and fixes I have pending. I'll set up a donate link soon and update the readme accordingly but until then, your dua is much appreciated.


Overall code quality

This would not be required for the acceptance of the paper, but checkcode("fff_segmenter.m") outputs the following suggestions:

L 631 (C 20-27): To improve performance, use 'isscalar' instead of length comparison.
631    if aux == 1
L 1154 (C 4-9): Consider using ISA instead of comparing the class name.
1154    if strcmp(class(res),'double')

Also some loops could be removed using indexes, like replacing:
for i = 1:length(dirX)
    if dirX(i) > 2
        dirXAdjusted(i) = 5;
    else
        dirXAdjusted(i) = 0;
    end
end

for i = 1:length(dirY)
    if dirY(i) > 2
        dirYAdjusted(i) = 5;
    else
        dirYAdjusted(i) = 0;
    end
end

with:
dirXAdjusted(dirX > 2) = 5;
dirXAdjusted(dirX < 2) = 0;
dirYAdjusted(dirY < 2) = 0;
dirYAdjusted(dirY > 2) = 5;

Hey @olivecha
The authors appreciate the issue.
We have made the required alterations to the fff_segmenter code.
Please, if additional alterations are necessary, let us know.
Att,
Thiago

Closing this as this was not necessary for publication and was addressed by the authors.


Possible DoS vulnerability

Description of bug:
NOTE: This is purely theoretical I haven't actually tested this because i686 builds aren't available for raiblocks and I don't want to bother building it from source right now.
An assert call can lead to a DoS in the account_decode function which is called in the rai::uint256_union::decode_account function.
The rai::uint256_union::decode_account has an error condition before calling account_decode that checks if a character is in the given range:
https://github.com/nanocurrency/raiblocks/blob/d865ad68e0b45ec65fcf525aeb93b9abe2f95a16/rai/lib/numbers.cpp#L87-L97
It basically says that any character between 0x30 ('0') and 0x80, however the account_decode checks for characters in between 0x30 ('0') and 0x7E ('~').
https://github.com/nanocurrency/raiblocks/blob/d865ad68e0b45ec65fcf525aeb93b9abe2f95a16/rai/lib/numbers.cpp#L30-L36
Knowing that the 0x7F and 0x80 are valid for decode_account but not for account_decode one can pass the following account address to the decode_account function:
"xrb_3\x7"

Note the \x7f scape sequence
And crash because of the assert call.
I don't know how this can be exploited because I haven't explored too much the raiblocks codebase, but there are some calls to the decode_account function increasing the vulnerability exposure.
PD: Sorry if I misspelled something, I'm not a native English speaker.

Addresses on the network are in binary form, not the encoded form.

@cryptocode but this function it's still a problem, if misused.

One important note here is that assert(3) is not used in releases.  In releases, calls to assert(3) are replaced with no-ops.
I'm going to leave this ticket open so we can add some more documentation related to the code above inline as comments.

I created issue #1098 to create a release-assert mechanism.  If it had been created and used here then this would be a DoS vulnerability.

Since #1098 was closed, can this issue one be closed as well?

Closing as the asserts are debug-only.


prefilter/multibuf: test with multiple packets

Ticket: 7326
Ticket
Redmine ticket: https://redmine.openinfosecfoundation.org/issues/7326

Not sure why, but this fails on my openbsd runner.

Force-pushed the pcap resaved with my Wireshark
Before :
input.pcap:  pcap capture file, microsecond ts, extensions (little-endian) - version 2.4 (No link-layer encapsulation, capture length 262144)
After:
input.pcap: pcap capture file, microsecond ts (little-endian) - version 2.4 (No link-layer encapsulation, capture length 262144)

Merged in #2118, thanks!


Request - Rename Error to FieldError for clarity

Title says it all. It would be my preference, have the "Error" rename to "FieldError" as to not overlap the native Error object. Better yet might be to maybe prefix all the Tag like the FormifyForm like FormifyField - FormifyFieldError - FormifyFieldArray
Maybe something similar for the useForm? Just a suggestion

In the first iteration of the package everything had the Formify prefix, but after a while it felt more nicer to my eye and preference to use simple naming (I know it's against the vue style guide because it's prefer multi-word names) so that is why I start to use these naming. I left FormifyForm for easier backwards compatibility for myself. 😄
I will leave it that way now, but later if it requested by more user I will reconsider this decision.


refactor!: simplify network configuration

Tests pass :tada:

Apparently need a new review after resolving merge conflict.


Two support requests are created on each account delete request

Build: master-31099

@cristinamanum , is this still an issue?

Close as NoRepro


What is the use of  YouTubeOverlay?

It's a part of this: https://github.com/vkay94/DoubleTapPlayerView


Adding Dockerfile and instructions to run locally

This PR adds the ability to run Openordex locally using an Apache server using Docker.
This is a precursor to be able to create this as an app in the Umbrel app store.

Thanks!


Issue when I try to input a date manually.

I creating a GUI and need the user can input the date both manually and selecting a date in datepicker.  When I input the date manually de modelview value isn't setted. How do I can make it?

I'm having the same issue. I set the format to be DD/MM/YYYY. When the user enters a date manually using keyboard, say, 03/12/2016 and then I click the date picker it shows the date as march 12th instead of dec 3rd. But when I pick a date on the date picker it displays the properly formatted date in the input box.

In the respective js file provide a format of 'YYYY/MM/DD' because moment.js reads it as the format i mentioned.

$scope.dob = moment($scope.dob, "DD/MM/YYYY").format('YYYY/MM/DD');


Added details w.r.t Diskless Config flag

Description of the change
Added Details for how to use Diskless Config Flag
Benefits

Drawbacks

Applicable issues

fixes #

Additional information
Checklist

[] Chart version bumped in Chart.yaml according to semver.
[] Variables are documented in the README.md
[] Title of the PR starts with chart name (e.g. [charts/gateway])
[] If the chart contains a values-production.yaml apart from values.yaml, ensure that you implement the changes in both files

Changes are already in https://github.com/CAAPIM/apim-charts/pull/318


Setup UI: Testing

We should test the setup UI by sending requests via curl. This will require some HTML "parsing" (well, grepping for some values) but should be easier than driving a virtual browser to click on things.

Do you prefer curl as a tool of choice, or just saying for simplification of the task? I can do this in selenium or webdriver if you want?

It was mostly for simplicity's sake. I guess the challenges will be similar for both approaches, so I don't have a strong preference. If we can get selenium to work in a nix environment I'd be fully on board.
Selenium might have the benefit of being easier to migrate when we switch from our current server-side web app to a API+PWA approach assuming the frontend doesn't change too much.
What do you think @jkitman, @justinmoon, @dpc?

Looks like we could even write the tests in Rust https://github.com/stevepryde/thirtyfour
I'm not sure whether it makes sense to write UI integration tests before some of the changes we want to make to the setup UI such as #1628 and #1893

IMO, this is low prio for now. UI tests are notoriously difficult subject, and I don't think we covered well more fundamental testing aspects. And we are yet to rewrite the whole thing anyway.
But since we're talking about it: If we are to make the setup UI a SPA or something, then we'll probably have to use some headless browser testing approach. I worked with cypress before and hated it. :D

IMO we shouldn't test our current UI since it's going to change heavily. Do https://github.com/fedimint/fedimint/issues/1893 first, test those endpoints, then write a good UI with some tests.

Do https://github.com/fedimint/fedimint/issues/1893 first, test those endpoints

This is the most sane approach we can take

then write a good UI with some tests

Agreed. UI tests for SPA could be atomic functional/accessibility/localization tests on small units of the UI. Test framework TBD based on the framework we write the UI in. When the time comes, I'd vote for React or it's variants so we can do simple jest testing
With both good test coverage on SPA UI and the driving API, I'd expect that we have limited to zero need for UI integration tests needing Selenium or curl grep :) as discussed above

subset of #2466

Let's close this in favor of https://github.com/fedimint/fedimint/issues/2466


PHP 7.4.11 error

Hi,
I encountered an error in the code, which is pretty simple to fix.
Here is the error:
In Credentials.php line 32:

  Unparenthesized `a ? b : c ? d : e` is deprecated. Use either `(a ? b : c) ? d : e` or `a ? b : (c ? d : e)`

It is about the code:
$lwaAccessToken = $useMigrationToken === true ? $this->getMigrationToken() : $useMigrationToken === 'grantless' ? $this->getGrantlessAuthToken() : $this->getLWAToken();
New version of PHP does not appreciate the ambiguity without parenthesis.
Can you fix this line plz ?
Thanks,
Vincent

Yes, sure. Will notify you when it is done


Fix HRMP messages delivery for unregistered channels

This PR aims to fix HRMP messages delivery for unregistered HRMP channels. Previous solution of opening channels on the fly did not really work, as it required at least one of the Parachains with an open EgressChannel.
Now, HRMP channels are expected to be opened in the Relay Chain.
Additionally, there are also a couple of improvements foreseen in upcoming runtime changes:

Fixing slot calculation for genesis with async backing
Producing an extra block for DMP messages in case MessageQueue is used

Thank you for this PR but unfortunately this will break a few things. I made validation data to inject any para id coming from hrmp without relaychain which mean you can open the channel just by submitting and empty message parachain.submitHorizontalMessages(paraId, []). It's very easy implementation. I will like to drop this PR and address this differently.

It's very easy implementation. I will like to drop this PR and address this differently.

Ok, but notice that there are other two changes in this PR that would be still applicable?

Fixing slot calculation for genesis with async backing
Producing an extra block for DMP messages in case MessageQueue is used

Can I revert HRMP changes and keep those two?
Can you share what approach you will follow to address the issue?

Fixing slot calculation for genesis with async backing

can you do another PR for this?

Producing an extra block for DMP messages in case MessageQueue is used

I don't why need to build block

Producing an extra block for DMP messages in case MessageQueue is used

I don't why need to build block

Message is processed in the next block after being received. Without building an extra block I can not assert the expected events.

but we already listen to downwardMessageQueues and trigger block building but if you start the chain with block-build-mode: manual then you have to manually build the block

but we already listen to downwardMessageQueues and trigger block building but if you start the chain with block-build-mode: manual then you have to manually build the block

Not sure if I understand, I think we are talking about different things. Soon, AssetHub runtimes will be using MessageQueue to handle received dmp messages. With that implementation, messages are actually processed in the next block they are included in the queue. If a new block is not automatically produced, the expected events coming from the XCM execution itself will not be emitted, and I wouldn't like to manually build another block. Ideally it should be an automated process.

Thanks, I wasn't aware. In that case we'll need to proper upgrade to MessageQueue with fallback to DMP

if you can upload latest assethub genesis it will help me a lot

Soon, AssetHub runtimes will be using MessageQueue to handle received dmp messages.

I don't think know we need any change yet. We don't use DmpQueue so this doesn't affect us. I will check tomorrow if we need to upgrade anything

@NachoPal do you any example that doesn't work so I can look at?

this is the reason the messageQeueue needs an extra block https://github.com/paritytech/polkadot-sdk/issues/3709
basically messageQueue is broken and we need to deal with the consequences

This is already fixed and can be closed


Publish RFCs as mdbook to GitHub Pages.

Spoke with someone at Rust Belt Rust who mentioned this idea (@mre 👋)
and decided to take a stab at it. I originally implemented this upstream
in the Rust source repository (by adding a git submodule and dynamically
generating the contents of the book there). But then then I spoke with
@steveklabnik and agreed that doing it in CI in this repository would be
a lot easier.

Can see an example of it here: https://frewsxcv.github.io/rfcs/
Assuming this is desired, Travis just needs to be enabled (which I have permissions to turn on) and the GitHub 'personal access token' needs to be registered in the Travis config (which I think I can also do).

cc @rust-lang/docs

I was holding off on suggesting this thinking it'd be a lot more work, so I'm +100 on this.

Oh wow! That is awesome @frewsxcv. Thanks for that. ❤

This is great!
@rust-lang/core what do you think?
I think merging this as-is is fine; but I'd also like to extend it someday; for example, we could map RFCs to section headings corresponding to year, etc.

I'd pull the script out of Travis and into a separate file (could just be generate-book.sh or something like that). That way we'd make it easier for contributors to generate the book locally if they wanted to...

Seems fine by me!

I'd pull the script out of Travis and into a separate file (could just be generate-book.sh or something like that). That way we'd make it easier for contributors to generate the book locally if they wanted to...

Done!

Also, I take back what I said earlier. I do not have permissions to enable Travis CI for this repo, so someone else will have to do that for me.
https://travis-ci.org/profile/rust-lang

Also, I take back what I said earlier. I do not have permissions to enable Travis CI for this repo, so someone else will have to do that for me.

I have the power! I enabled travis, turned on "build only if .travis.yml is present" and turned off branch updates and PR updates, since this really only matters for master. Let me know if anything else should be changed!
Love this btw!!

We tested this out on a branch; deploying is using an ssh key that's added as a deploy key to this repo only and is encrypted in travis. The deploy worked! http://rust-lang.github.io/rfcs/ so this should start auto deploying when this gets merged to master!

 (I was standing beside this conversation and got giddy about it just listening in and now here it is.)


[Zewo] Add SQL

@formbound Let me know if you're willing to take this on (you know the code best). Otherwise, I'll give it a go myself :)

@Danappelxx

Adding SQL to the Zewo repo, or the org?

Taking it out of zewograveyard and as a subrepo of zew/zewo.


Investigate why the model loss curve diverges between train-val and test datasets.

By looking at the model loss across the epochs it seems we are overfitting.

Investigate how to fix that.

Sometimes the loss curve is fine though! Look:

So I guess it might be because there are too little samples in the test dataset.


[Question] Infinite redirect loop on Microsoft Edge and Edge inPrivate window

Hi there,
First of all thank you for the library, I appreciate the time and effort it goes into something like this.
I am having a strange issue with this library and Microsoft Edge browser.
Current version: Microsoft EdgeHTML 18.18363 Edge 44.18362.449.0
The problem in local development: When calling signinRedirectCallback we get the error 'No matching state found in storage'.
The problem in production: When being redirected back to the application, we get stuck in an infinite loop. This is most likely due to the signinRedirectCallback not firing.

Normally I'd have the signinRedirectCallback on another page, however due to client limitations this is not possible.
Do you have any help or advice moving forward? This does not occur in any of the other browsers, not even IE11.
Thank you

Just a hunch, since Edge is now using Chrome's engine, you might be running into the same issue as with Chrome Incognito where '3rd party cookies' (cookies from another domain, your IDS might live on another domain than your app, esp if your app's on localhost) are being blocked?
Other than that, it might help to have a way to reproduce your scenario, if you could post a sample way to get the same behavior?

Thanks for responding 😄. Unfortunately this is an Edge version before being based off Chromium
I wish I could reproduce my scenario but not sure if I can give away the required urls. I will say that I think it is related to Security zones. https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-js-known-issues-ie-edge-browsers
I will say that my auth process is cross-domain, where the authorization is done on a separate domain to the application.
"Specifically, the session storage is cleared in the regular browser navigation, and both the session and local storage are cleared in the InPrivate mode of the browser."

Ah yeah, I understand. But if the situation is super specific to your scenario, and we cannot reproduce the problem, it's a bit hard to help (without resorting to guessing to the cause). You might have to debug yourself then what's going on... (at least, I don't have other ideas what it could be, apart from what you already suggested). Good luck!


Data in Bright and HA not the same

Two images showing similar views in Bright and Home Assistant. Have I missed something. Downloaded earlier this week. Thanks

Hey @smcfadden, this looks like it may be the same issue as #305. Feel free to follow and add to the discussion there!
Duplicate of #305


Changes for pycurl, Requests and service classes to use pycurl by default (do not merge)

Eric,
why the build on jenkins page has a red ball even though
DMWM-PR-unittests is yellow? From the console output I see the following:
Checking console output
/var/lib/jenkins/jobs/DMWM-PR-test/builds/1024/log:
Testing of python code. DMWM-FAIL-UNIT
Run condition [Text Finder] enabling perform for step [BuilderChain]
[DMWM-PR-test] $ /bin/sh -xe /tmp/hudson4038148752549809285.sh
+ echo 'One or more checks gave bad results'
One or more checks gave bad results
+ exit 1

I'm not sure which DMWM-FAIL-UNIT it reads.
On  0, DMWM Jenkins<EMAIL_ADDRESS>wrote:

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/1024/ for details

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-151566483

You have a red ball for two reasons:
One unit test moved from success to error
Score for src/python/WMCore/Services/pycurl_manager.py changed from 9.72 to 9.68
All the yellow ball for DMWM-unittests means is that it was able to run all the tests (no timeouts or other crashes in the script). DMWM-PR-test does the final analysis of all the sub jobs and determines success or failure.

test this please

Sengchan, it seems to me that all tests passed, but I found that two latests builds 1033 and 1034 contradict each other in terms of unit tests. It seems to me that those are unstable tests. Could you please check. Apart from that I think pycurl is ready to go. Please review the changes and let me know if you want to merge them or split them apart. This PR can super seed #5845 because it contains its changes plus additional one I made.

Thank you very much Valentin, yes those tests are unstable test. I will close #5845 then.  Changes looks good to me, yes, please merge commits together.  I will merge this patch after we update the production agents. Thanks again

Seangchan, I squashed the changes, please review and let's wait for unit tests to complete.

test this please

@vkuznet Hi Valentin, I getting some errors with this patch. (*)
Error seems to be happening here.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WebTools/RESTModel.py#L246
I will try to create the unittest which we can reproduce the problem. (Unless you know what where it causes the problem, please don't investigate yet)
(*)
2015-11-17 18:05:15,093 WARNING  Service: The cachefile /data/srv/state/workqueue/cache/.wmcore_cache_100034/requests/reqmgr2-dev.cern.ch/-1062535079034185883_PUT_request does not exist and the service at https://reqmgr2-dev.cern.ch/reqmgr/reqMgr/request is unavailable - it returned 400 because Bad Request
with result: {"exception": 400, "message": "Invalid input: Input arguments failed sanitation. XXXXX", "type": "HTTPError"}
2015-11-17 18:05:15,220 ERROR    WorkQueueReqMgrInterface: Error talking to ReqMgr about request "sryu_MonteCarloFromGEN_pnn_test_151117_061147_4617": Traceback (most recent call last):
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 191, in report
self.reportElement(ele)
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 321, in reportElement
self.reportRequestStatus(element['RequestName'], element['Status'])
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 275, in reportRequestStatus
raise ex
HTTPException: url=https://reqmgr2-dev.cern.ch/reqmgr/reqMgr/request, code=400, reason=Bad Request, headers={'Content-Length': '109', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Connection': 'close', 'Etag': '4604053556711648119', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Tue, 17 Nov 2015 17:05:15 GMT', 'Content-Type': 'application/json', 'CMS-Server-Time': 'D=8306 t=1447779915082738'}

Can you post how to reproduce this problem. The raised exception does not
provide any information about which arguments have failed, so I would suggest
to fix exception message to give us more information.
On  0, ticoann<EMAIL_ADDRESS>wrote:

@vkuznet Hi Valentin, I getting some errors with this patch. (*)
Error seems to be happening here.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WebTools/RESTModel.py#L246
I will try to create the unittest which we can reproduce the problem. (Unless you know what where it causes the problem, please don't investigate yet)
(*)
2015-11-17 18:05:15,093 WARNING  Service: The cachefile /data/srv/state/workqueue/cache/.wmcore_cache_100034/requests/reqmgr2-dev.cern.ch/-1062535079034185883_PUT_request does not exist and the service at https://reqmgr2-dev.cern.ch/reqmgr/reqMgr/request is unavailable - it returned 400 because Bad Request
with result: {"exception": 400, "message": "Invalid input: Input arguments failed sanitation. XXXXX", "type": "HTTPError"}
2015-11-17 18:05:15,220 ERROR    WorkQueueReqMgrInterface: Error talking to ReqMgr about request "sryu_MonteCarloFromGEN_pnn_test_151117_061147_4617": Traceback (most recent call last):
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 191, in report
self.reportElement(ele)
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 321, in reportElement
self.reportRequestStatus(element['RequestName'], element['Status'])
File "/data/srv/HG1511c/sw.pre/slc6_amd64_gcc481/cms/workqueue/1.0.11.pre8/lib/python2.6/site-packages/WMCore/WorkQueue/WorkQueueReqMgrInterface.py", line 275, in reportRequestStatus
raise ex
HTTPException: url=https://reqmgr2-dev.cern.ch/reqmgr/reqMgr/request, code=400, reason=Bad Request, headers={'Content-Length': '109', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Connection': 'close', 'Etag': '4604053556711648119', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Tue, 17 Nov 2015 17:05:15 GMT', 'Content-Type': 'application/json', 'CMS-Server-Time': 'D=8306 t=1447779915082738'}

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-157450061

Actually the code I pointed is where the exception occur. I will try to create the unittest reproduce the problem so it can be investigated. (Please don't spend time until then, thanks)
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WebTools/RESTModel.py#L246

Or, you can fix RESTModel.py#L246 in your local setup to print kwars e.g.
change
if input_kwargs:
raise HTTPError(400, 'Invalid input: Input arguments failed sanitation.')
to
if input_kwargs:
print "Invalid kwargs", input_kwargs
raise HTTPError(400, 'Invalid input: Input arguments failed sanitation.')
and then re-run what you had before.
On  0, ticoann<EMAIL_ADDRESS>wrote:

Actually the code I pointed is where the exception occur. I will try to create the unittest reproduce the problem so it can be investigated. (Please don't spend time until then, thanks)
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WebTools/RESTModel.py#L246

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-157484808

I did already, somehow that reads to 500 error some where else.

Hi Valentin, it seems the content length is needed when make post request (or request body is not empty). (Our webtool requirement) You can run below unittest
https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py

Seangchan,
can you re-run your tests now. It seems tome I fixed 411 error, but I'm facing
another one
{"exception": 400, "message": "Invalid input: Input data failed validation.",
"type": "HTTPError"}
which came from
debug(2): PUT /unittests/rest/version/fake/fake HTTP/1.1
Host: localhost:8899
Cache-Control: no-cache
Content-type: application/json
Accept: application/json
User-agent: WMCore.Services.Requests/v001
Content-Length: 0
WARNING:cherrypy.access:NullAuth called for:
WARNING:cherrypy.access:    role(s): []
group(s): []
site(s): []
DEBUG:cherrypy.error:version raw data: {'args': [], 'kwargs': {}}
DEBUG:cherrypy.error:version sanitised input_data: {'version': 'fake', 'scramArch': 'fake'}
DEBUG:cherrypy.error:'fake' does not match regular expression CMSSW(\d+){3}([a-zA-Z0-9_]+)?$
DEBUG:cherrypy.error:call to PUT with args: ['version', 'fake', 'fake'] kwargs: {} resulted in Invalid input: Input data failed validation.
INFO:cherrypy.access:[18/Nov/2015:11:06:32] lnxcu9.classe.cornell.edu <IP_ADDRESS> "PUT /unittests/rest/version/fake/fake HTTP/1.1" 400 Bad Request [data: - in 98 out 1166 us ] [auth: - "" "" ] [ref: "" "WMCore.Services.Requests/v001" ]
debug(1): HTTP/1.1 400 Bad Request
as you can see this is not HTTP issue per se, and rather validation of PUT request
fake does not match regular expression CMSSW(\d+){3}([a-zA-Z0-9_]+)?$
So, I'm not sure it this is something I need to fix in this PR.
On  0, ticoann<EMAIL_ADDRESS>wrote:

Hi Valentin, it seems the content length is needed when make post request (or request body is not empty). (Our webtool requirement) You can run below unittest
https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-157514972

Thanks Valentin, I will rerun the integration test.

test this please

test this please

test this please

Hi Valentin, I ran some test with this patch. Especially following unittest
https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py
It is successful without this patch but fails if I applied patch.
First problem is  the dbs validation fails. (am not sure why this fails here but not when httplib2 call is used) - it might be some authentication setting problem.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMSpec/WMWorkloadTools.py#L67
Even if I ignore this error (by simply returning without calling DBS). It still failed put request with bad request error. It seems problem lies on encoding and content-type mismatch. I will try to debug more. But if you can look at that, I will appreciate. Thanks,

Seangchan,
If time permit I'll have a look, but I doubt I'll be able to work until next
week. I saw your comments and I think you're right about Content-type.
Valentin.
On  0, ticoann<EMAIL_ADDRESS>wrote:

Hi Valentin, I ran some test with this patch. Especially following unittest
https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py
It is successful without this patch but fails if I applied patch.
First problem is  the dbs validation fails. (am not sure why this fails here but not when httplib2 call is used) - it might be some authentication setting problem.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/WMSpec/WMWorkloadTools.py#L67
Even if I ignore this error (by simply returning without calling DBS). It still failed put request with bad request error. It seems problem lies on encoding and content-type mismatch. I will try to debug more. But if you can look at that, I will appreciate. Thanks,

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-167882960

test this please

Thanks Valentin, take your time. and happy new year.
Here is what I found.

putTeam call is weird. It expects urlencoded data from url string but not body parts.
previous code append the params to url string if content_type is application/x-www-form-urlencoded. which I don't think correct way to do.
There seems to be decoding twice if the pycurl is used.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/RequestManager/RequestManager.py#L61
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/pycurl_manager.py#L143

previous versions default decode function just change to string not json.
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/Requests.py#L294
https://github.com/dmwm/WMCore/blob/master/src/python/WMCore/Services/Requests.py#L280

Seangchan, I modified pycurl_manager to properly calculate Content-length. But during tests I found that RequestManager_t.py fails in setUp call since I don't have scramArch and CMSSWVersion parameters. I can't find where exactly they are setup, may be I'm lack of some config file in my test setup. Here is failed part: https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/RequestManager_t/utils.py#L25 and it reads schema from here https://github.com/dmwm/WMCore/blob/master/test/python/WMCore_t/RequestManager_t/utils.py#L25 I can't find anywhere in WMCore where those parameters are set, so I modified utils.py to setup them manually and at least my RequestManager_t fails further down. The good sign that now it fails identically using pycurl and httplib.

Hmm, seems like my last changes allow to pass all tests, am I right?

To make RequestManager_t unittest to pass we need https://github.com/dmwm/WMCore/pull/6495

test this please

@vkuznet, Hi Valentin, after applying #649, I am getting following errors when I run WMCore_t/Services_t/RequestManager_t/RequestManager_t.py test. Are you getting a same error?
DEBUG:cherrypy.error:PUT to request expects 4 argument(s), got 80
DEBUG:cherrypy.error:call to PUT with args: ['request'] kwargs: {} resulted in Invalid input: Input arguments failed sanitation.
INFO:cherrypy.access:[11/Jan/2016:09:59:37] mac-122010.dhcp.fnal.gov ::1 "PUT /unittests/rest/request HTTP/1.1" 400 Bad Request [data: - in 103 out 958 us ] [auth: - "" "" ] [ref: "" "WMCore.Services.Requests/v001" ]
Exception during set up, reason: Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 66, in setUp
    r = self.jsonSender.put('request', schema)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 130, in put
    encode, decode, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 147, in makeRequest
    encoder, decoder, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 173, in makeRequest_pycurl
    verb=verb, ckey=ckey, cert=cert, capath=capath, decode=decoder)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 201, in request
    raise exc
HTTPException: url=http://localhost:8899/unittests/rest/request, code=400, reason=Bad Request, headers={'Content-Length': '103', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '2211139151384643101', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 15:59:37 GMT', 'Content-Type': 'application/json'}

E
======================================================================
ERROR: testA_RequestManagerService (__main__.RequestManagerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 74, in setUp
    raise ex
HTTPException: url=http://localhost:8899/unittests/rest/request, code=400, reason=Bad Request, headers={'Content-Length': '103', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '2211139151384643101', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 15:59:37 GMT', 'Content-Type': 'application/json'}

----------------------------------------------------------------------
Ran 1 test in 1.236s

FAILED (errors=1)

Seangchan,
my test is clean, I think I forgot to apply this part to RequestManager_t.py
63         try:
64             r = self.jsonSender.put('request/' + schema['RequestName'], schema)
65             try:
66                 self.requestName = r[0]['RequestName']
67             except:
68                 self.requestName = r[0].values()[0]['RequestName']
69         except Exception as ex:
70             print "Exception during set up, reason: %s" % ex
71             raise ex
so, try to modify your line 66 with
r = self.jsonSender.put('request/' + schema['RequestName'], schema)
the PUT request needs the name of request in URI !
Valentin.
On  0, ticoann<EMAIL_ADDRESS>wrote:

@vkuznet, Hi Valentin, after applying #649, I am getting following errors when I run WMCore_t/Services_t/RequestManager_t/RequestManager_t.py test. Are you getting a same error?
DEBUG:cherrypy.error:PUT to request expects 4 argument(s), got 80
DEBUG:cherrypy.error:call to PUT with args: ['request'] kwargs: {} resulted in Invalid input: Input arguments failed sanitation.
INFO:cherrypy.access:[11/Jan/2016:09:59:37] mac-122010.dhcp.fnal.gov ::1 "PUT /unittests/rest/request HTTP/1.1" 400 Bad Request [data: - in 103 out 958 us ] [auth: - "" "" ] [ref: "" "WMCore.Services.Requests/v001" ]
Exception during set up, reason: Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 66, in setUp
    r = self.jsonSender.put('request', schema)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 130, in put
    encode, decode, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 147, in makeRequest
    encoder, decoder, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Requests.py", line 173, in makeRequest_pycurl
    verb=verb, ckey=ckey, cert=cert, capath=capath, decode=decoder)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/pycurl_manager.py", line 201, in request
    raise exc
HTTPException: url=http://localhost:8899/unittests/rest/request, code=400, reason=Bad Request, headers={'Content-Length': '103', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '2211139151384643101', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 15:59:37 GMT', 'Content-Type': 'application/json'}

E
======================================================================
ERROR: testA_RequestManagerService (__main__.RequestManagerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 74, in setUp
    raise ex
HTTPException: url=http://localhost:8899/unittests/rest/request, code=400, reason=Bad Request, headers={'Content-Length': '103', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '2211139151384643101', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 15:59:37 GMT', 'Content-Type': 'application/json'}

----------------------------------------------------------------------
Ran 1 test in 1.236s

FAILED (errors=1)

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-170597531

Hi Valentin,
I am still getting errors like following.
Server side.
DEBUG:cherrypy.error:call to PUT with args: ['message', 'Taizong_160111_151802_3315'] kwargs: {} resulted in cannot parse JSON description: error

                                              stack trace: Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTApi.py", line 103, in _buildResponse
    args, kwargs)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTModel.py", line 117, in handler
    data = methodCall(*params, **kwargs)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTModel.py", line 200, in wrapper
    return func(**input_data)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/HTTPFrontEnd/RequestManager/ReqMgrRESTModel.py", line 618, in putMessage
    obj = JsonWrapper.loads( body )
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Wrappers/JsonWrapper/__init__.py", line 40, in loads
    raise ex
DecodeError: cannot parse JSON description: error

Client code
======================================================================
ERROR: testA_RequestManagerService (__main__.RequestManagerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 127, in testA_RequestManagerService
    self.reqService.sendMessage(requestName,"error")
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/RequestManager/RequestManager.py", line 157, in sendMessage
    contentType = 'application/json')
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/RequestManager/RequestManager.py", line 56, in _getResult
    verb = verb, contentType = contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Service.py", line 212, in refreshCache
    self.getData(cachefile, url, inputdata, incoming_headers, encoder, decoder, verb, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Service.py", line 319, in getData
    raise he
HTTPException: url=http://localhost:8899/unittests/rest/message/Taizong_160111_151802_3315, code=500, reason=Internal Server Error, headers={'Content-Length': '68', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '-1680540008677053678', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 21:18:02 GMT', 'Content-Type': 'application/json'}

----------------------------------------------------------------------
Ran 1 test in 1.707s

FAILED (errors=1)

Seangchan,
my fault, this is because I forgot to update sendMessage API of RequestManager.
The original code sent wrong PUT request, i.e. it uses PUT with
application/json content-type while sends string message. I fixed it with
proper wrapper of messages in args. I updated my PR, see
https://github.com/vkuznet/WMCore/blob/pycurl-fix8/src/python/WMCore/Services/RequestManager/RequestManager.py#L145
With this fix the error you saw should disappear.
Valentin.
On  0, ticoann<EMAIL_ADDRESS>wrote:

Hi Valentin,
I am still getting errors like following.
Server side.
DEBUG:cherrypy.error:call to PUT with args: ['message', 'Taizong_160111_151802_3315'] kwargs: {} resulted in cannot parse JSON description: error

                                              stack trace: Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTApi.py", line 103, in _buildResponse
    args, kwargs)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTModel.py", line 117, in handler
    data = methodCall(*params, **kwargs)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/WebTools/RESTModel.py", line 200, in wrapper
    return func(**input_data)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/HTTPFrontEnd/RequestManager/ReqMgrRESTModel.py", line 618, in putMessage
    obj = JsonWrapper.loads( body )
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Wrappers/JsonWrapper/__init__.py", line 40, in loads
    raise ex
DecodeError: cannot parse JSON description: error

Client code
======================================================================
ERROR: testA_RequestManagerService (__main__.RequestManagerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/sryu/WorkArea2/WMCore/test/python/WMCore_t/Services_t/RequestManager_t/RequestManager_t.py", line 127, in testA_RequestManagerService
    self.reqService.sendMessage(requestName,"error")
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/RequestManager/RequestManager.py", line 157, in sendMessage
    contentType = 'application/json')
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/RequestManager/RequestManager.py", line 56, in _getResult
    verb = verb, contentType = contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Service.py", line 212, in refreshCache
    self.getData(cachefile, url, inputdata, incoming_headers, encoder, decoder, verb, contentType)
  File "/Users/sryu/WorkArea2/WMCore/src/python/WMCore/Services/Service.py", line 319, in getData
    raise he
HTTPException: url=http://localhost:8899/unittests/rest/message/Taizong_160111_151802_3315, code=500, reason=Internal Server Error, headers={'Content-Length': '68', 'Expires': 'Sun, 28 Jan 2007 00:00:00 GMT', 'Vary': 'Accept-Encoding', 'Server': 'CherryPy/3.2.2', 'Etag': '-1680540008677053678', 'Pragma': 'no-cache', 'Cache-Control': 'no-cache, must-revalidate', 'Date': 'Mon, 11 Jan 2016 21:18:02 GMT', 'Content-Type': 'application/json'}

----------------------------------------------------------------------
Ran 1 test in 1.707s

FAILED (errors=1)

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-170693876

Valentin, Seangchan, could you please summarize what the status of this PR is?
If this PR is still valid and we want to keep testing it, we need Valentin to fix the conflicts.

Alan,
my understanding that pycurl patch conflicts with old ReqMgr code and the idea
was to post-pone it until we retire ReqMgr (since its code needs re-factoring
to use new pycurl). Let's wait what Seangchan will say.
Meanwhile, can you elaborate which conflicts are you talking about.
Thanks,
Valentin.
On  0, Alan Malta Rodrigues<EMAIL_ADDRESS>wrote:

Valentin, Seangchan, could you please summarize what the status of this PR is?
If this PR is still valid and we want to keep testing it, we need Valentin to fix the conflicts.

Reply to this email directly or view it on GitHub:
https://github.com/dmwm/WMCore/pull/6360#issuecomment-180335057

Thanks, Valentin! I'm refering to merge conflicts, but if ReqMgr needs to go away first, then we better not spend time on it right now.

Most - if not all - of this implementation has been merged through #9365


Cover unit test for use case GetProductsListUseCase.kt

Currently there is no testing for use case. It would be nice if we can set up one and cover them

I did some in app/src/test/java/com/hieuwu/groceriesstore/domain/usecases/impl/GetProfileUseCaseImplTest.kt, you can start writing test following the convention.

Hi @hieuwu  ,
I would like to work on this issue , could you please assign it to me .
Thanks

Hi @hieuwu ,
I would like to work on this issue , could you please assign it to me .
Thanks

Sure

Hi ,
I can see some test cases already in the GetProductsListUseCaseImplTest , so just wanted to check with you on whether we need to add any more test cases to test some particular functionality .
Thanks

Hi @hieuwu  ,
Just wanted to follow up with you on this , could you please confirm on whether we need to add new test cases to (or) is the existing one sufficient ?
Thanks

Hey it’s completed man. Great work


Skip pylint for ci

closes #25
Skip pylint for ci
Temporary fix for #25 . Skips pylint hook when running pre-commit.ci on PRs.
Changes

Skips pylint hook when running pre-commit.ci on PRs.
Changes pre-commit.ci autoupdate frequency to monthly

Additional Notes
Temporary fix to allow PRs to pass. Will need to fix in future to allow pylint to run on CI.

Yeah, this is a temporary fix just to allow PRs to pass the pre-commit ci check. For some reason it cannot find the pylint command. That's the only hook it is having an issue with.


Add building labels

There's a commented out layer for this, rescue it! Probably from another house style file, though?

this is done


For those who are writing Firefox addon, the solution here helped me, I have to add "*://localhost/*"

For those who are writing Firefox addon, the solution here helped me, I have to add "*://localhost/*"
to permissions key in manifest.json:

  "permissions": [
    "storage",
    "*://localhost/*"
  ],

Chrome just works without the need to add this permission...
Originally posted by @hantatsang in https://github.com/github/fetch/issues/310#issuecomment-454662463

Thank you for sharing your workaround!
Closing since this issue tracker is for reporting bugs with the polyfill only.


Can't Install python extention

VSCode Version:1.46.1
OS Version:10.0.18363

Can't Install python extention. Tried to download menually and when I installed I got the message: "VSIXInstaller.NoApplicableSKUsException: One or more extensions are for Visual Studio Code. Try installing them in Visual Studio Code"

Install your downloaded .vsix file from here.

Please follow these steps

Help -> Toggle Developer Tools -> Network
Then check for any network failed requests while browsing and installing the extensions if found all requests fails then its probably a proxy or DNS issue at your end.
To confirm its a DNS issue check if you can access this page https://ms-python.gallerycdn.vsassets.io/extensions/ms-python/python/2019.8.29288/1565127378610/Microsoft.VisualStudio.Services.Icons.Default, if you cannot please try configuring your DNS like pointing to <IP_ADDRESS> as suggested here? https://<IP_ADDRESS>/dns/. Otherwise please contact your network administrator.
If it is not a network issue, please share the logs from the following locations

Help -> Toggle Developer Tools -> Console
View -> Command Pallette... -> Toggle Shared Process -> Console
View -> Open View… -> Log (Shared)
View -> Open View… -> Log (Window)


ARROW-14071: [R] Try to arrow_eval user-defined functions - implicit approach

Once this PR is merged, users should be able to use their own functions in an {arrow} dplyr-like pipeline (provided they can be translated with the help of existing bindings).
Thoughts around the design were captured in this design doc.
This PR implements an implicit approach, while #13789 implements the explicit one.

explicit: walks through the abstract syntax tree and translates each translatable function call encountered

works with filter(), mutate() and transmute() , i.e. nse_funcs
doesn’t yet work with summarise(), i.e. agg_funcs
the code is a bit more complex and might require more effort to maintain
nested translations work

implicit: modifying the environment of the unknown functions

more difficult to get nested translations right (doing the registration recursively), but they work
works with filter(), mutate() and transmute()
doesn’t yet work with summarise()
less complex in terms of code, but more complex in terms of mental model (getting recursion right when environments are involved is a bit trickier), but potentially easier to maintain.

Examples
library(dplyr)
library(arrow)

nchar2 <- function(x) {
  1 + nchar(x)
}

# simple expression
tibble::tibble(my_string = "1234") %>%
  mutate(
    var1 = nchar(my_string),
    var2 = nchar2(my_string)) %>%
  collect()
#> # A tibble: 1 × 3
#>   my_string  var1  var2
#>   <chr>     <int> <dbl>
#> 1 1234          4     5

# a slightly more complicated expression
tibble::tibble(my_string = "1234") %>%
  mutate(
    var1 = nchar(my_string),
    var2 = 1 + nchar2(my_string)) %>%
  collect()
#> # A tibble: 1 × 3
#>   my_string  var1  var2
#>   <chr>     <int> <dbl>
#> 1 1234          4     6

nchar3 <- function(x) {
  2 + nchar(x)
}

# multiple unknown calls in the same expression (to test the iteration)
tibble::tibble(my_string = "1234") %>%
  mutate(
    var1 = nchar(my_string),
    var2 = nchar2(my_string) + nchar3(my_string)) %>%
  collect()
#> # A tibble: 1 × 3
#>   my_string  var1  var2
#>   <chr>     <int> <dbl>
#> 1 1234          4    11

# user function defined using namespacing
nchar4 <- function(x) {
  2 + base::nchar(x)
}

tibble::tibble(my_string = "1234") %>%
  mutate(
    var1 = nchar(my_string),
    var2 = 1 + nchar4(my_string)) %>%
  collect()
#> # A tibble: 1 × 3
#>   my_string  var1  var2
#>   <chr>     <int> <dbl>
#> 1 1234          4     7

isPowerfulEngine <- function(x) x > 200
mtcars %>%
  arrow_table() %>%
  filter(isPowerfulEngine(hp)) %>%
  collect()

#>    mpg cyl disp  hp drat    wt  qsec vs am gear carb
#> 1 14.3   8  360 245 3.21 3.570 15.84  0  0    3    4
#> 2 10.4   8  472 205 2.93 5.250 17.98  0  0    3    4
#> 3 10.4   8  460 215 3.00 5.424 17.82  0  0    3    4
#> 4 14.7   8  440 230 3.23 5.345 17.42  0  0    3    4
#> 5 13.3   8  350 245 3.73 3.840 15.41  0  0    3    4
#> 6 15.8   8  351 264 4.22 3.170 14.50  0  1    5    4
#> 7 15.0   8  301 335 3.54 3.570 14.60  0  1    5    8

I'm closing this PR for now as it was handed over to me to take a look at, but I don't think I'm going to end up with time to work on this in the near future, and we're just having a bit of a tidy of PRs ahead of the upcoming release.


[BUG] libcamera-apps stopped working

Describe the bug
Libcamera-still just waiting.
Today I just rebooted my raspberry and after that libcamera-apps stopped working. I'm not aware that I would update anything that might break it.
I used unchanged libcamera-apps from "Raspberry Pi OS with desktop, April 4th 2022" . So I tried to build it myself, and it didn't help. Then I updated kernel and it worked for a few minutes, but not anymore. Then I rebuild it on updated kernel and there is report from your tool.
I'm afraid there may be a hardware problem
Bug report
report.txt

This possibly looks like a hardware/connectivity issue.  Can you ensure your camera connector cable is connected securely on both ends?  Alternatively, please try another camera board if you have another available to use.

Don't know why, but today it start working again. I Did nothing from yesterday and yes cable is connected securely. I checked it yesterday. Some ghosts in machine probably. But anyway, thank you for your time.


Mouse wheel scroll to page when bounds reached

This is a (multiple allowed):

[ ] bug
[x] enhancement
[ ] feature-discussion (RFC)

When scrolling using the mousewheel over a swiper slider with mousewheel enabled,
the page should receive the scroll events when the bounds of slider have been reached (with loop disabled).
E.g. a vertical slider is scrolled upwards using the mousewheel, when the first slide is reached,
there is no further scrolling of slides possible, then the page should get the scroll events.

https://swiperjs.com/api/#mousewheel

releaseOnEdges
boolean
false
Set to true and swiper will release mousewheel event and allow page scrolling when swiper is on edge positions (in the beginning or in the end)


[SPARK-34900][TEST] Make sure benchmarks can run using spark-submit cmd described in the guide

What changes were proposed in this pull request?
Some spark-submit  commands used to run benchmarks in the user's guide is wrong, we can't use these commands to run benchmarks successful.
So the major changes of this pr is correct these wrong commands, for example, run a benchmark which inherits from SqlBasedBenchmark, we must specify --jars <spark core test jar>,<spark catalyst test jar> because SqlBasedBenchmark based benchmark extends BenchmarkBase(defined in spark core test jar) and SQLHelper(defined in spark catalyst test jar).
Another change of this pr is removed the scalatest Assertions dependency of Benchmarks because scalatest-*.jar are not in the distribution package, it will be troublesome to use.
Why are the changes needed?
Make sure benchmarks can run using spark-submit cmd described in the guide
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Use the corrected spark-submit commands to run benchmarks successfully.

Test build #136663 has started for PR 31995 at commit c76be67.

Thanks. All of these are tested manually, @LuciferYang ?

Yes, I tested all the benchmarks manually.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136663/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41245/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41245/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41245/

Merged to master.
@LuciferYang can we create backporting PRs?

@LuciferYang can we create backporting PRs?

OK ~


Decide how to migrate from YouTrack

Originally this project was handled in https://aircuryinterns.youtrack.cloud/agiles/141-3/current?tab=general. Decide what and how to import the content to GitHub.

I don't see this YouTrack ticket here: https://aircuryinterns.youtrack.cloud/agiles/141-3/current?tab=general&issue=SH-35. Any reasons for that?
Also what should we do about finished tickets?

The YouTrack themes are already added and here is a little documentation about what was done before taking on the project.
https://docs.google.com/document/d/1K6nOoEC3sIEx_FgOYCTkyBVPNuPR-m2klPgOB374I9g/edit?usp=sharing

It looks good, would you mind to recommend what part of that documentation could be added to the repository or this board?
Maybe some things could more easily accessed in a .md and others could be part of the description/summary of this project.