Create cursors by matching text objects

Just an idea I'm kicking around.
Creating cursors based on matching/splitting with regex is great. What about creating cursors based on text objects?
Splitting the following into arguments by vi( and splitting on ,  is great:
foo(x, y, z)

But what about this?
foo(x, bar(y, z), (a, b))

Text objects are good. Treesitter text objects are super smart. A few uses:

Selecting all (top-level) arguments in an argument list
selecting all quoted strings in a text paragraph
select all sentences in a text paragraph
selecting all list items in a list

I've got an initial implementation here. It splits a single visual selection. For the example above, it creates three selections:
foo(x, bar(y, z), (a, b))
    -  ---------  ------

It is likely naive and fragile. It relies on the search capability of mini.ai so that e.g. vi" will find quoted text to the right.
Thoughts? Am I missing something obvious?

seems useful. i think it would be best if iw, i" and some others were implemented by hand and then everything else automatically falls back to the idea you've come up with. because iw can select non alphabetic words which is very unlikely what the user wants. i" can select the stuff between quotes. these would both be fairly easy to add and would be the most used.

playing around with it, i think it can be done with a filter of (char: string) => boolean.
for iw, we define the filter as iskeyword and while iterating over each char, we only query the iw text object once we have found an iskeyword char. this would avoid the whitespace etc.
then for i", we define the filter as char => char == '"'. once we find a text object, we can record its start/end position. trying to match a i" between two strings will end up having the start position the same as a previous end position and so it will not be added.
then for other text objects, the prefilter is just () => true.
does this make sense?

small feedback, Cursor:feedkeys is fairly expensive: it writes the cursor state (all its registers, marks, position) to vim, feeds the keys & flushes, reads the new cursor state, and updates its extmarks. so it's a good idea to combine Cursor:feedkeys calls when possible.
search_cursor:feedkeys(TERM_CODES.ESC .. "`>")

small feedback, Cursor:feedkeys is fairly expensive

Ah, this is a good tip. I divided it into multiple calls just to visually separate "change to normal mode" vs "move to end," but with as much as I'm creating cursors (simply to act as placeholders), I should aim for efficiency.
As for the filtering, hmm. I agree that the built-in iw and i" text objects are not likely to be useful. For the latter, I use the i" implementation from mini.ai, which is smart about matched pairs. I'm a bit leery of the specialization here:

I hate to over-anticipate. I can't think of a use for splitting on iw, but that doesn't mean no one wants to grab the words in *+*x*+*x*+*.
I kind of like the separation of concerns of  "defining text objects" from "multicursor." I might create an "identifier" text object outside of this. Or match on [%w_]+.
I'm not sure about "it uses your text objects, except these two." Messes with the clean abstraction.

I don't know that these vague concerns stand up against "make the plugin more useful, with no dependencies," though. In any case, I mostly want to use it for treesitter-derived text objects — things where regex/patterns are a pain.
Your implementation concept for these two makes sense to me. We are both probably re-implementing echasnovski's work in mini.ai; I haven't looked closely, but he clearly put a lot of thought into searching for text objects not covered by the cursor.

We can figure out if they have the key remapped using mapcheck('i"', "o"). Then, if it is mapped to iw, we apply the iw special case. If it is mapped to something random, then we use that text object as is.
In the end it should appear like plain text objects are being used to user, but a tiny bit of magic behind the scenes for builtin text objects so they work "correctly".

i am working on this

needs a lot more work but here is my prototype:
local regexes = {
    ["iw"] = "\\k",
    ['i"'] = '"',
    -- ["i'"] = "'",
}

function examples.textObjects()
    local motion = vim.fn.input("> ")
    if not motion or motion == "" then
        return
    end

    mc.action(function(ctx)
        local mainCursor = ctx:mainCursor()
        local visualStartPos, visualEndPos = mainCursor:getVisual()
        local startPos = mainCursor:mode() == "V"
            and { visualStartPos[1], 1 }
            or visualStartPos
        local INT_MAX =<PHONE_NUMBER>
        local endPos = mainCursor:mode() == "V"
            and { visualEndPos[1], INT_MAX }
            or visualEndPos
        local lastPos = { startPos[1], startPos[2] - 1 }
        local lines = mainCursor:getVisualLines()
        local origPos = mainCursor:getPos()
        local found = false

        local mapped = vim.fn.mapcheck(motion)
        if not mapped or mapped == "" then
            mapped = motion
        end
        local regex = regexes[mapped]

        if regex then
            for _, match in ipairs(util.matchlist(lines, regex)) do
                local newPos = {
                    startPos[1] + match.idx,
                    startPos[2] + match.byteidx
                }
                if comparePos(newPos, lastPos) then
                    goto continue
                end
                mainCursor:setPos(newPos)
                mainCursor:setMode("n")
                mainCursor:feedkeys("v" .. motion)
                local vs, ve = mainCursor:getVisual()
                if comparePos(vs, lastPos)
                    or comparePos(ve, lastPos)
                then
                    goto continue
                end
                found = true
                mainCursor:clone()
                lastPos = { ve[1], ve[2] + 1 }
                ::continue::
            end
        else
            local newPos = { startPos[1], startPos[2] }
            local reg = vim.fn.getreginfo("z")
            while newPos[1] - startPos[1] < #lines do
                vim.fn.setpos("'[", { 0, 0, 0, 0 })
                mainCursor:setMode("v")
                mainCursor:setVisual(newPos, newPos)
                mainCursor:feedkeys(motion, { remap = true })
                if not mainCursor:inVisualMode() then
                    newPos[2] = newPos[2] + 1
                else
                    local vs, ve = mainCursor:getVisual()
                    if comparePos(endPos, vs) or comparePos(endPos, ve)
                    then
                        break
                    end
                    if comparePos(vs, lastPos) or comparePos(ve, lastPos)
                    then
                        newPos[2] = newPos[2] + 1
                    else
                        found = true
                        mainCursor:clone()
                        lastPos = { ve[1], ve[2] + 1 }
                    end
                end
                if newPos[2] > #lines[newPos[1] - startPos[1] + 1] then
                    newPos[2] = 1
                    newPos[1] = newPos[1] + 1
                end
            end
            if found then
                mainCursor:delete()
            else
                mainCursor
                    :setMode("n")
                    :setPos(origPos)
            end
            vim.fn.setreg("z", reg)
        end
    end)
end

function examples.matchCursors(pattern)
    mc.action(function(ctx)
        pattern = pattern or vim.fn.input("Match: ")
        if not pattern or pattern == "" then
            return
        end
        --- @type Cursor[]
        local newCursors = {}
        ctx:forEachCursor(function(cursor)
            if cursor:hasSelection() then
                newCursors = tbl.concat(
                    newCursors, cursor:splitVisualLines())
            else
                newCursors[#newCursors + 1] = cursor
                cursor:setMode("v")
            end
        end)
        for _, cursor in ipairs(newCursors) do
            local selection = cursor:getVisualLines()
            local matches = util.matchlist(selection, pattern, {
                userConfig = true,
            })
            local vs = cursor:getVisual()
            for _, match in ipairs(matches) do
                if #match.text > 0 then
                    local newCursor = cursor:clone()
                    newCursor:setVisual(
                        { vs[1], vs[2] + match.byteidx + #match.text - 1 },
                        { vs[1], vs[2] + match.byteidx }
                    )
                    newCursor:setMode("n")
                end
            end
            cursor:delete()
        end
    end)
end

Very nice. Thanks for working with me on this. I got very excited when I found that this plugin worked with non-standard objects and motions :).
In the next few days, I will mess around with and without mini.ai. Definitely a goal that this work as well as possible without additional plugins. However, my primary goal is that it work with any crazy user-defined text objects.
I see you have replaced a bunch of my uses of extra-cursor-and-g`< by directly using the positions. That makes a lot of sense to me; in general I prefer programming via api instead of scripting inputs, because it feels less fragile. Vim doesn't always make that easy.
Part of the reason I did it that way is that the neovim api is annoying with its variety of position indexing; then multicursor must necessarily add its own conventions. It'd be easier if multicursor used named indexing instead of tuples. Before:
local startPos, endPos = cursor:getVisual()
local newPos = vim.deepcopy(startPos)
newPos[2] = newPos[2] + 1

After:
local range = cursor:getVisual()
local newPos = vim.deepcopy(range.start)
newPos.line = newPos.line + 1

On the topic of bounds, three things. First, the extra-cursor-and-g`< trick, while not ideal, does handle both visual and visual-line modes without needing mainCursor:mode() == "V" checks. Perhaps the API could hide this so that start is start, regardless of mode?
Second, I hadn't thought at all about visual-block selections, that's a good point. That makes things trickier.
Third, I was pretty sloppy thinking about bounds in any case. Here's an example using vit:
           selection
           ---------------------
<foo>****</foo><bar>****</bar><baz>****</baz>
     ----           ----
     result         result

So that's:

get objects selectable from the range of the current selection
if the first object is outside the range, keep it
after the first object, exclude objects outside the range

Instead, it should probably be consistent. The three boundary conditions I can think of are "selectable from inside range," "overlaps range," and "is contained in range." I think I prefer the first, which would look like:
           selection
           ---------------------
<foo>****</foo><bar>****</bar><baz>****</baz>
     ----           ----           ----
     result         result         result

At the moment, that doesn't work with my implementation and default mini.ai behavior, which seeks ahead, leading to this:
           selection
           ---------------------
<foo>****</foo><bar>****</bar>     <baz>****</baz>
     ----           ----                ----
     result         result              result

In my opinion, that's a good reason to disable the default searching behavior of mini.ai, which has other methods to opt-in to search.
The reality is that for any i motion, you probably want objects where the corresponding a motion would overlap the initial selection.  So I vote for "selectable from within the range." I think that also simplifies visual-block mode, later.
It's also the intention to select maximal objects, so it should select this for vi(
(  (  )  )((  )(  ))
 --------  --------

and not this:
(  (  )  )((  )(  ))
 --------   --  --

Right now, the maximal behavior is happy result of scanning left-to-right and skipping to the end of found objects. I can't think of any examples where you'd find a larger object on the right than on the left; can you?

I got very excited when I found that this plugin worked with non-standard objects and motions :)

Thanks. I don't find multicursor plugins useful when they don't feel like real vim.

It'd be easier if multicursor used named indexing instead of tuples.

I think this can be done without breaking changes. I would return a { line: int, col: int, offset: int } object with a metatable with __index and __newindex metamethods to handle integer keys. I don't even think this would have a performance impact since metamethods are only called for keys which don't exist. This would need a bit of extra thought for cases like table unpacking but I think it's a good idea.

Perhaps the API could hide this so that start is start, regardless of mode?

Yes, I tried changing it to be that way but something else broke. I plan to have it that way but need enough time to test everything.

On the topic of bounds

I think only objects completely contained within the visual selection should be included. A crude example would be visually selecting the line } foo { bar } baz {, where the braces at the start/end are massive blocks. If we select with i}, we definitely don't want to select the top and bottom brace blocks.
You mentioned selecting maximal objects. I think we should match from top to bottom and matches contained within objects will be skipped since they come after their parent, as you pointed out.

EDIT: It might be a better idea return the [int, int, int]

This makes sense to me. I'm a frequent user of Python's namedtuple for coordinates in other domains. Having both named and indexed fields is handy. It sounds like luajit only supports that in one way, native tuples (more or less) with named metamethods. Works for me. I feel like the overhead of metamethods is insignificant, especially for this.

I think only objects completely contained within the visual selection should be included.

You're right, that makes the most sense. It prevents some astonishing behaviors.

To avoid duplication of effort: I'm thinking of starting a little testing framework for this. I'm tired of reloading and doing the same motions. I like to visually see the test conditions; maybe something where a test case looks like this (where <leader>j is mapped to matchCursors):
foo(a, b(c, d), (f, g))
<   v-----------------
>   v  v------  v-----
= <leader>jia<cr>

I'm not decided on that syntax or anything; just want to see if you're interested or already have something cooking.

Progress is slow for this one since I keep trying different ideas to make it work seamlessly. I managed to use "zy from normal mode and measured the yanked text to tell whether it was successful. Works for text objects like i" but other objects just yank a single char which is not ideal.
I think we may just have to have an API to help define text objects. All it really needs is the motion (w for iw and aw) and a regex (\k).
My progress so far:
--- @param a SimplePos
--- @param b SimplePos
--- @return boolean
local function comparePos(a, b)
    if a[1] == b[1] then
        return a[2] < b[2]
    end
    return a[1] < b[1]
end

--- @class TextObjectInfo
--- @field regex string
--- @field motion fun(char: string): string
--- @field type "i" | "a"

local function identity(x)
    return x
end

--- @type {[string]: TextObjectInfo}
local TEXT_OBJECT_LOOKUP = {}

--- @param key string
--- @param regex string
--- @param motion? string | fun(match: string): string
local function addTextObject(key, regex, motion)
    local motionCallback
    if type(motion) == "string" then
        motionCallback = function() return motion end
    elseif motion then
        motionCallback = motion
    else
        motionCallback = function()
            return key
        end
    end
    for _, type in ipairs({"i", "a"}) do
        TEXT_OBJECT_LOOKUP[type .. key] = {
            type = type,
            regex = regex,
            motion = motionCallback
        }
    end
end

addTextObject("w", "\\k")
addTextObject("W", "\\S")
addTextObject("p", "\\S")
addTextObject("'", "'")
addTextObject('"', '"')
addTextObject('`', '`')
addTextObject("t", "<")
addTextObject("s", "\\S")
addTextObject("(", "(")
addTextObject(")", ")")
addTextObject("{", "{")
addTextObject("}", "}")
addTextObject("[", "[")
addTextObject("]", "]")
addTextObject("<", "<")
addTextObject(">", ">")
addTextObject("q", "[\"'`]", identity)
addTextObject("b", "(", "(")
addTextObject("B", "{", "{")

--- @param vs SimplePos
--- @param startPos SimplePos
--- @param fullLines string[]
--- @return boolean
local function skipLeadingWhitespace(vs, startPos, fullLines)
    while true do
        local line = fullLines[vs[1] - startPos[1] + 1]
        if not line then
            return false
        end
        while true do
            local char = line:sub(vs[2], vs[2])
            local isWhitespace = char == " " or char == "\t"
            if char == "" then
                break
            elseif not isWhitespace then
                return true
            else
                vs[2] = vs[2] + 1
            end
        end
        vs[1] = vs[1] + 1
        vs[2] = 1
    end
end

function examples.textObjects()
    --- @type string | nil
    local motion = vim.fn.input("> ")
    if motion == "" then
        motion = nil
    end
    if not motion then
        return
    end

    --- @type string | nil
    local mapped = vim.fn.mapcheck(motion, "o")
    if mapped == "" then
        mapped = nil
    end

    local info = (mapped and TEXT_OBJECT_LOOKUP[mapped]) or TEXT_OBJECT_LOOKUP[motion]

    if not info and not mapped then
        util.echoerr(
            "multicursor.nvim: unrecognized text object '"
            .. motion .. "'")
        return
    end

    mc.action(function(ctx)
        local mainCursor = ctx:mainCursor()
        local visualStartPos, visualEndPos = mainCursor:getVisual()
        local startPos = mainCursor:mode() == "V"
            and { visualStartPos[1], 1 }
            or visualStartPos
        local endPos = mainCursor:mode() == "V"
            and { visualEndPos[1], INT_MAX }
            or visualEndPos
        local lastPos = { startPos[1], startPos[2] - 1 }
        local lines = mainCursor:getVisualLines()
        local fullLines = mainCursor:getFullVisualLines()
        local origPos = mainCursor:getPos()
        local found = false

        if info then
            for _, match in ipairs(util.matchlist(lines, info.regex)) do
                local newPos = {
                    startPos[1] + match.idx,
                    startPos[2] + match.byteidx
                }
                if comparePos(newPos, lastPos) then
                    goto continue
                end
                mainCursor:setPos(newPos)
                mainCursor:setMode("n")
                mainCursor:feedkeys("v" .. info.type .. info.motion(match.text))
                local vs, ve = mainCursor:getVisual()

                if not mainCursor:inVisualMode()
                    or comparePos(ve, lastPos)
                    or comparePos(endPos, ve)
                    or not skipLeadingWhitespace(vs, startPos, fullLines)
                    or comparePos(vs, lastPos)
                    or comparePos(endPos, vs)
                then
                    goto continue
                end
                found = true
                mainCursor:clone()
                    :setVisual(vs, ve)
                lastPos = { ve[1], ve[2] + 1 }
                ::continue::
            end
        else
            local newPos = { startPos[1], startPos[2] }
            while newPos[1] - startPos[1] < #lines do
                mainCursor:setMode("v")
                mainCursor:setVisual(newPos, newPos)
                mainCursor:feedkeys(motion, { remap = true })
                local vs, ve = mainCursor:getVisual()

                if not mainCursor:inVisualMode()
                    or comparePos(ve, lastPos)
                    or comparePos(endPos, ve)
                    or not skipLeadingWhitespace(vs, startPos, fullLines)
                    or comparePos(vs, lastPos)
                    or comparePos(endPos, vs)
                then
                    newPos[2] = newPos[2] + 1
                else
                    found = true
                    mainCursor:clone()
                    lastPos = { ve[1], ve[2] + 1 }
                    newPos = lastPos
                end
                if newPos[2] > #fullLines[newPos[1] - startPos[1] + 1] then
                    newPos[2] = 1
                    newPos[1] = newPos[1] + 1
                end
            end
        end
        if found then
            mainCursor:delete()
        else
            mainCursor
                :setMode("n")
                :setPos(origPos)
        end
    end)
end

Don't rush on my account; I'm doing alright with my little prototype for now. Also, I'm both using your advice and stealing your ideas from here to improve it.
I will fork the repo, create a branch, and make a PR solely to make discussion easier. Maybe you make a PR too? Easier than pasting code in comments. We can circle around a few ideas for UI and implementation before you pick whatever you like best.
I wonder if you're going down a rabbit hole here. My feeling is that you want this to be 100% agnostic to objects, not be adding more object-related code. If that means that iw doesn't make a lot of sense by default with "match cursors to text objects," so be it.
You could provide some opt-in object mappings like iw that check if a multicursor operation is progress — maybe set/unset a buffer variable during the action — and change behavior accordingly.

I wonder if you're going down a rabbit hole here. My feeling is that you want this to be 100% agnostic to objects, not be adding more object-related code. If that means that iw doesn't make a lot of sense by default with "match cursors to text objects," so be it.

The priority is that the feature works how the user expects. When a user
selects words with iw, they expect words to be selected, not the whitespace
and symbols between. I consider any other behaviour a bug.
Elegance and simplicity come second.

User experience first is a solid philosophy. 👍


METRON-2353: Plugin e2e tests should support multiple brokers

Contributor Comments
This gives us two kafka brokers and an arbitrary number of partitions for our e2e test to improve testing.
Changes

Somewhat cosmetically updates support to librdkafka 1.4.2 (from 1.4.2-RC3) and zeek 3.1.3 (from 3.1.2).
Replaces the metron-bro-plugin-kafka_kafka base image with a more maintained base image
Removes the unused docker_execute_configure_zeek_plugin.sh script, which was a copy of docker_execute_configure_plugin.sh.

testing

Run ./run_end_to_end.sh and see that it works.
Test the new --partition argument to see that it works with an arbitrary number of partitions across the two (hard-coded) brokers.

Pull Request Checklist
Thank you for submitting a contribution to Apache Metron's Bro kafka writer plugin.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[X] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[X] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[X] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
[ ] Have you included steps or a guide to how the change may be verified and tested manually?
[X] Have you ensured that the full suite of tests and checks have been executed via:bro-pkg test $GITHUB_USERNAME/metron-bro-plugin-kafka --version $BRANCH

[ ] Have you written or updated unit tests and or integration tests to verify your changes?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] Have you verified the basic functionality of the build by building and running locally with Apache Metron's Vagrant full-dev environment or the equivalent?

I'll test and merge

@ottobackwards ready for review now.  In troubleshooting some issues running this up locally I found that I was hitting the consumer timeout; hence increasing it from 1000 to 5000 based on the additional resources needed for multiple brokers/partitions.  However, I found many other small bugs along the way which are fixed in this PR.

Great work @JonZeolla, thanks!


[Android] Collection view not working correctly

Cloned repo and ran on an android emulator :

10 videos are retrieved but only 2 are shown in the collection view.
Any ideas?

Not sure what the issue is.
I tried on Android myself on VS4Mac and it works fine. I can see all 10 cells and videos. I am using MAUI version 6.0.424. Which SDK version you are using? Maybe it is a VS for Windows bug. A lot of other people have complained as well. I do not have Windows machine to check; maybe someone can debug on Windows and provide a solution.
Meanwhile, here is how it looks on Android via VS4Mac.

@MentYy you have to share the error logs. Most likely its an issue with the size of the thumbnail image being too big. It's a common android issue, but it could be anything. Could you check your Application Output?

I'm using version 6.0.486. I also seem to experience a bug where it builds and runs fine the 1st time on both a physical device and android emulators but any time after it just crashes. I haven't been able to identify the issue or problem area with the limited testing I've been able to do. Clearing app cache or data doesn't help.

@saamerm I thought so too. I set the size of the thumbnail image to a fixed smaller size which results in a smaller cell but the same issue with only 2 cells visible and the massive space in between. I'll have to check the output when I can test again.

I suspect @saamerm is correct with regards to sizing being an issue.
I removed - HeightRequest="{Binding ItemsHeight, Source={x:Reference RootPage}}" from the video cell control nad got result :

This also fixed the crashing - somehow ItemsHeight is causing the crashes as well as the unexpected appearance.

Strange. Is this only happening on VS4Win for Android? On Mac, there is no problem. And I have been using this approach for years in xamarin as well. Never faced this issue.

Dear @MentYy,
I removed - HeightRequest="{Binding ItemsHeight, Source={x:Reference RootPage}}" from the video cell control and set Image HeightRequest  set 260.

got result :

Yes, but this is not the behavior I wanted. Hence I put in dynamically computing the Row Height.
And as I said, this seems to be happening on VS4Win for Android only. On Mac, there is no problem for both iOS and Android. I have been using this approach for years in xamarin forms without any issues. So must be a Maui bug on VS4Win only.

Hi,
this is a great course and great video 👍
it is actually same behavior on both MAC and PC , I haven't completed the courses yet though maybe later on there is a fix for it once i complete the videos

@amasoumi @MentYy What did you get?

@amasoumi @MentYy What did you get?
Last time I tried still the same behaviour, maybe it will be fixed in .net 8

FYI, it's not fixed in .net 8.rc1

Not fixed on .NET 8. Everything the same


Is it possible to output only color palette?

Hello,
When I execute wal with proper arguments, I get such output:
wal -i futuristic_industry.jpg

When I use the quiet option, I get this output:
wal -q -i futuristic_industry.jpg

Is it possible to get colors only in the output?
wal ? -i futuristic_industry.jpg

I tried greping out lines with square brackets, but that removes colors too.
Thanks in advance :)

You can use wal --preview to display the current palette.

Looks like I wasn't using the newest version of pywal - 3.1.0. I upgraded to 3.3.0 and everything works fine now. Thank you! :)


I added nLogin to my aternos server it gives this error

[14:33:29 INFO]: UUID of player Playerardaberk is d0a299c2-9f6e-4e82-8767-78baf7a05ea0
[14:33:30 INFO]: Playerardaberk[/<IP_ADDRESS>:31106] logged in with entity id 343 at (137.5, 66.0, 88.5)
[14:33:33 WARN]: Can't keep up! Is the server overloaded? Running 2142ms or 42 ticks behind
[14:33:48 INFO] [nLogin]: The user Playerardaberk has successfully registered.
[14:33:51 INFO]: Playerardaberk lost connection: Disconnected
[14:33:53 INFO]: UUID of player Playerardaberk is d0a299c2-9f6e-4e82-8767-78baf7a05ea0
[14:33:54 ERROR] [nLogin]: The unique id of the player being connected (Playerardaberk) is different from the one stored in the nLogin database (d0a299c2-9f6e-4e82-8767-78baf7a05ea0 != 41c04eab-0325-3588-bbe7-86404e334fab)
[14:33:54 INFO]: Disconnecting com.mojang.authlib.GameProfile@24c01377[id=d0a299c2-9f6e-4e82-8767-78baf7a05ea0,name=Playerardaberk,properties={textures=[com.mojang.authlib.properties.Property@2e135c12]},legacy=false] (/<IP_ADDRESS>:31142): [nLogin]
[14:33:54 INFO]: com.mojang.authlib.GameProfile@24c01377[id=d0a299c2-9f6e-4e82-8767-78baf7a05ea0,name=Playerardaberk,properties={textures=[com.mojang.authlib.properties.Property@2e135c12]},legacy=false] (/<IP_ADDRESS>:31142) lost connection: [nLogin]
What is the issue?

pls give me time to response I live in Turkey

Hi, @Playerardaberk
This is usually caused when ProtocolLib is not working properly.
Try updating your ProtocolLib: https://ci.dmulloy2.net/job/ProtocolLib/lastSuccessfulBuild/

It works Thanks for helping me.


Copy over the error callback when reconnecting.

As otherwise, when reconnecting, we end up just using the default callback, which limits the ability to handle errors gracefully.

Good catch! Thanks for the PR!


small wheels

add? please?  smaller wheels to create a motorcycles fnd small cars

github issues are not for feature requests.

github issues are not for feature requests.


Fixed FireTVService crash

NullPointerException in FireTVService$Subscription
https://github.com/ConnectSDK/Connect-SDK-Android-FireTV/issues/2

Thanks. It looks good, I'll add a test for this case.


Expand handling of LIMIT 1, 2 handling to include sqlite

Sqlite itself parses this syntax, so this parser should parse it when operating in sqlite mode.
Tangential fixups:

I also noticed that ClickhouseDialect was missing from all_dialects(), so I rectified that. This required excluding ClickhouseDialect from one existing test, but it works everywhere else.
Added specific_dialects(...) function to more cleanly be able to specify exactly the dialects this LIMIT comma syntax works for, in tests
Added a test for LIMIT 1, 2
(very minor) fix capitalization of is in doc comment for dialect_of! macro

Pull Request Test Coverage Report for Build<PHONE_NUMBER>8
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

23 of 24 (95.83%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 89.315%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

tests/sqlparser_common.rs
8
9
88.89%

Totals

Change from base Build<PHONE_NUMBER>3:
0.003%

Covered Lines:
29790

Relevant Lines:
33354

💛  - Coveralls

Thanks again @joshuawarner32 and @iffyio


Block BlobStore from starting if it was down for X days.

Block BlobStore from starting if it was down for X days.

Codecov Report
Attention: Patch coverage is 0% with 28 lines in your changes are missing coverage. Please review.

Project coverage is 18.71%. Comparing base (52ba813) to head (0c8e05d).
Report is 3 commits behind head on master.

Files
Patch %
Lines

...rc/main/java/com/github/ambry/store/BlobStore.java
0.00%
26 Missing :warning:

...main/java/com/github/ambry/config/StoreConfig.java
0.00%
2 Missing :warning:

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #2777       +/-   ##
=============================================
- Coverage     64.24%   18.71%   -45.54%
+ Complexity    10398     2910     -7488
=============================================
  Files           840      840
  Lines         71755    71814       +59
  Branches       8611     8623       +12
=============================================
- Hits          46099    13439    -32660
- Misses        23004    57139    +34135
+ Partials       2652     1236     -1416

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Modified onnx_to_nnir.py to take input and output dimensions

@LakshmiKumar23 is this ready to be merged? has this passed all the tests?

Yes, I have tried it with a few models and you need to specify input and output dimensions just as in caffe_to_nnir.


fmt: Fscanf fails to read an integer from a special file

What version of Go are you using (go version)?

$ go version
go version go1.19.1 linux/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/xxx/.cache/go-build"
GOENV="/home/xxx/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/xxx/gocode/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/xxx/gocode"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/home/xxx/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/home/xxx/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3607947791=/tmp/go-build -gno-record-gcc-switches"

What did you do?
https://go.dev/play/p/Kodf5klMoFG
What did you expect to see?
cat /proc/sys/fs/pipe-max-size outputs 1048576, so I expected this number.
What did you see instead?
1
When it's a regular file with the same content (the number + a newline), Fscanf works fine (dd if=/proc/sys/fs/pipe-max-size of=pipe-max-size, then use os.Open("pipe-max-size") on line 10.)
ioutil.ReadAll also reads all the data.

Duplicate of #29399


wip: support for eth-duties

https://github.com/TobiWo/eth-duties

LGTM, needs to wait for lighthouse PR which contains overlay and pkgs triggers, which links this package installable.
Can this be added to as a default package to both nodes?

I'm having issues with pyinstaller dependencies, specifically zlib. Additionally, the compiling process seems to be taking an abnormally long time, and I'm getting various warnings during the process. I'll try to fix this in the future, considering trying with the buildPythonPackage module instead of poetry2nix.

This should be added directly to Nixobolus. It can be used in this repository through the 'homestakeros' module.

If you make a new PR on nixobolus side and reference it here, we can close this issue here.

Continuing here: https://github.com/ponkila/nixobolus/pull/7


Transaction function returns error is {}, what is the reason？

GORM Playground Link
err:=db.Transaction(func(tx *gorm.DB) error {
retrun nil
})
err is {}
what is the reason

可以提供一个 playground 嘛 ? https://github.com/go-gorm/playground


vector: 0.21.0 -> 0.21.1

Description of changes
This fixes a small regression. https://vector.dev/releases/
@kira-bruneau noticed you posted something last time about darwin breakage. I couldn't find a way to fix the breakage on darwin and was imagining that most usage was on linux anyway. Sorry about the disturbance!
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

@GrahamcOfBorg eval

@kira-bruneau noticed you posted something last time about darwin breakage. I couldn't find a way to fix the breakage on darwin and was imagining that most usage was on linux anyway. Sorry about the disturbance!

Oh no problem! That's probably still a fair assumption 😅. We use vector as part of our dev flake at work and I just noticed the darwin build broke trying to do an update. It's not exactly a huge deal though, since we can just to pin to the old version for now.
Do you have an idea of what was wrong with using kafka? I can spend a bit of time looking into this at some point 🙂 I kind of let my maintenance slide though, and I have a blacklog of other things I probably want to get through first 😓
I'm just waiting on the Linux build to finish right now, but it looks like you just pushed some changes to disable the tests 😅

right, this new version has 100 tests failing out of 1100.
I'm not too happy about it, but I have to say that the tests were already failing on darwin for a long time.
on darwin the kafka failure was a weird symbol missing. It made me think it was a matter of the sdk used in nix being too old.
Removing the kafka feature another thing broke (If I remember correctly it was during zstd-sys build. Since the failure was not immediately obvious, I just gave up).
no rush on this. If you ever have time, great, however there might be more important stuff.
I'm there if you need.

Result of nixpkgs-review pr 169934 run on x86_64-linux 1

1 package built:

vector

Thanks a lot for the review. It looks like it's working, so I'm going to merge this and revert If I see weird failures when using it on a server.
I'll most probably give the tests another go on the next version.


Vote counter doesn't show zero when vote is retracted and no other votes exist.

Fixed in f465e85360d205d4f6dfdc11b5f353591a765ca1.


Small problema in note of Navigation in Elements

HI
I tried use the note Url in Elements but I found some problem at subscriptions.
I think the code is missing the function composition.

#76

Fixed by #76. Thank you for the report!


"ReferenceError: __dirname is not defined" - Using the library client side is failing because of references to __dirname

Checklist

[x] Read and understood the Code of Conduct.
[x] Searched for existing issues and pull requests.

Description
Using the stdlib dependency in a client side application (using npm install @stdlib/stdlib) results in the following error:
ReferenceError: __dirname is not defined     at Object../node_modules/<EMAIL_ADDRESS>(index.js:54)
Questions
No.
Other
Demo
Reproduction
In order to reproduce this bug, do the following:

create a client side app
add the stdlib dependency
run it

Environments

What environments are affected (e.g., Node v0.4.x, Chrome, IE 11)? If Node.js, include the npm version, operating system, and any other potentially relevant platform information.

The following environments are affected:

Node 12.16.1

@chriszrc How are you trying to load stdlib in a client side application? Are you using a bundler, such as Webpack or browserify, or are you trying to load as an ES module?

Hi,
It's an angular app, so yes, written with es modules, but bundled with
webpack. Interestingly, changing the stdlib dependency to point directly at
github rather than npm @.***/stdlib":
"github:stdlib-js/stdlib#develop",`) allows everything to work just fine
On Mon, Apr 5, 2021, 7:48 PM Athan @.***> wrote:

@chriszrc https://github.com/chriszrc How are you trying to load stdlib
in a client side application? Are you using a bundler, such as Webpack or
browserify, or are you trying to load as an ES module?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stdlib-js/stdlib/issues/390#issuecomment-813693851,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGEBMKPDNXKRL3FVZGIYEPTTHI43TANCNFSM42NQZBCQ
.

@chriszrc There may be an issue with your webpack configuration.
@Planeshifter Perhaps you have some insight here, given your experience with using stdlib with webpack?

@chriszrc Tracked it down. The latest version on GitHub has a bug fix. Will publish a new version tomorrow with the fix. Does that work for you?

@kgryte nice! yes, as I mentioned, pointing directly to github worked fine, but yes, having it up to date on npm is certainly better-

@chriszrc A new version has been published. Let me know if you run into any issues. :)


Can't install powertoys 0.62.1 x64 on windows 10; "The feature you are trying to use is on a network resource that is unavailable"

Microsoft PowerToys version
0.62.1
Running as admin

[X] Yes

Area(s) with issue?
Installer
Steps to reproduce
Just double click the installer for 0.62.1 x64
✔ Expected Behavior
I expect it to install straight away
❌ Actual Behavior

Other Software
No response

Something cleaned out the installed 0.61.0 MSI from your computer. You can extracting it from the 0.61 installer executable following these instructions:
https://learn.microsoft.com/en-us/windows/powertoys/install#extracting-the-msi-from-the-bundle
/needinfo

I had the same problem again when installing 0.63. I extracted the msi and now the installer is looking for 0.61. Why is it asking for 0.61 if it's a 0.63 installer?

Can someone please send me the msi file for the latest powertoys version ? I have literally 0 knownledge on how to use WiX, and it is not straightforward in the documentation on how to install WiX...

Same issue on upgrading 0.76.2


fix(currencyFilter): trim currency string

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug fix
What is the current behavior? (You can also link to an open issue here)
For certain locales, when the currencySymbol is set to an empty string, there is trailing white space. See issue #15018
What is the new behavior (if this is a feature change)?
Any currency string will be trimmed.
Does this PR introduce a breaking change?
yes
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

Other information:
Trim any beginning or trailing white space in a currency string.
Motivation: global fix for issue 15018

Correction: this is a breaking change.

Does anybody know how I would change locale for an automated test?

You mean in a unit test?

Yes
On Sep 5, 2016 3:03 PM, "Martin Staffa"<EMAIL_ADDRESS>wrote:

You mean in a unit test?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/angular/angular.js/pull/15085#issuecomment-244810928,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE1cIPvO8OhTDh_WwTjhwv-03Lfqwb_Vks5qnIOKgaJpZM4JzSB6
.

Why did you remove the fix from the commit?

@Narretz The fix is still there. I was just trying things to get a unit test. Check the "files changed" tab.

Ah I see, sorry. The test code you changed is a docs end 2 end test,  not a unit test though. The filter spec files are in the test folder.

@Narretz Thanks!  Got the unit test in.

This does not fix #15018. In fact it just fixes certain cases (i.e. locales which put the currency symbol after the number), but is not a generic fix (see https://github.com/angular/angular.js/issues/15018#issuecomment-239480197 for more details).
I am tempted to merge this even if it fixes only some of the cases (as a proper fix is quite more involved), but that would be "unfair" to certain locales :/

@gkalpak Would appending a .replace(\-/s*\g, '-') fix the potential issue with having a space between the negative sign and the currency amount?  I don't see any locale delimiting their currency with anything that would make the fix problematic.
I'll try to add some unit tests and fix based off those when I get home.

No, I don't think it would work for all cases (although it would take care of several more than the current implementation). Take into account that people might be overwriting the locals pattern object. E.g. in finance/accounting-related apps people prefer to display negative values wrapped in (...) instead of prefixing them with -.
I have a more general fix. I will submit a PR.


User keywords in parallel

We tried to run user keywords(written in ROBOT) in parallel with multithreading and looks like the internal context or state is messed up.
Ex:
Thread1:
CALL_RO:
CALL_R0_KW1
CALL_R0_KW2
CALL_R0_KW3
Thread2:
CALL_R1:
CALL_R1_KW1
CALL_R1_KW2
CALL_R1_KW3
When the above threads which calls ROBOT nested KW’ are called, the robot framework is messing up execution by calling R1 KW inside R0 KW
Observed call sequence:
CALL_RO:
CALL_R0_KW1
CALL_R1:
CALL_R1_KW1
CALL_R0_KW2
CALL_R1_KW3
We need this feature and blocking us.
Please let me know when can we expect this support?
Thanks,

There are a number of issues with your post:

It doesn't contain enough information to help diagnose the problem. You claim "Robot is messing up" but you are using multi-threading, so by definition there is no guarantee that your keywords will be called in a specific order between the two threads. The CPU is free to schedule processing time between the two threads however it likes, and it may interrupt Thread 1 to execute Thread 2 (or vice versa) in whatever manner it chooses. That's just the nature of multi-threading I'm afraid. If you need to ensure something done by Thread 1 happens strictly before or after something done by Thread 2, then you need to synchronise them yourself. This is a pretty fundamental concept of multi-threaded programs.

Your post comes across as pretty rude, by saying things like "We need this feature and blocking us. Please let me know when can we expect this support?".  This is your issue and nobody on this project (a lot of whom are volunteers giving up their time for free) is obligated to provide you with help when you clearly don't properly understand the concept of threading and cross-thread synchronisation.

I suggest you update the thread with more information about exactly what your code is doing, what keywords it is calling in what order, and how it is invoking them in a multi-threaded environment. Only then will anyone have enough information to begin to help you out.

Yes, Robot isn't thread safe in general and especially running user keywords in parallel is pretty much impossible. It would be a nice feature, but getting the framework into a shape where it would be possible will take time. We are working towards it but it's not too high priority. If this is very important feature for you, you can consider sponsoring development, but unfortunately I cannot immediately give any estimates how big/expensive task this would be. Anyway, if you are interested, a good first step would be joining Robot Framework Foundation.
We already have #1792 tracking this particular enhancement so I close this issue.


improve: show disable guide in peer dep messages

closes #112
TODO

[ ] docs
[x] tests

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@next-major dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Updates links to proper version numbers.

Readme link said 1.3.7, which was a dead link.

Thanks for that, we are behind with the documentation. We will be soon updating the site with new examples and new features.


Feature/flex daemonset also run on masters + change flex directory

Run Daemonset on masters as well by using "tolerations" tag.
In addition change the directory of the flex config + log + certificate into the same directory of the binary file /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ibm~ubiquity-k8s-flex
Some text fixes per code review.

Full details -> #116
The PR passed sanity (and also sanity with certificates \ verify-full)

This change is

Review status: 0 of 16 files reviewed at latest revision, 10 unresolved discussions.

ubiquity-k8s-provisioner.conf, line 1 at r1 (raw file):

# NOTE : This script is obsolete because ubiquity is been installed as docker image.

If it's obsolete then why don't you delete it?

cmd/flex/main/cli.go, line 42 at r1 (raw file):

	"configFile",
	k8sresources.FlexConfPath,
	"config file with ubiquity client configuration params",

If the use see this description, it should be reviewed by Dima. Anyway, I would change to "Flex Volume configuration file, containing Ubiquity connection parameters."

deploy/k8s_deployments/setup_flex.sh, line 16 at r1 (raw file):

if [ ! -d "${MNT_FLEX_DRIVER_DIR}" ]; then
  echo "Creating the flex driver directory [$DRIVER] for the first time."
  echo "***Attention*** : Kubernetes version below 1.8 - the user must restart manually the kubelet service in order to load the new flex driver."

All echo statements should be reviewed by Dima.
"If you are running on a Kubernetes version which is lower then 1.8, a restart to the kubelet service is required to take affect."

deploy/k8s_deployments/setup_flex.sh, line 20 at r1 (raw file):

fi

echo "Copying the flex driver ~/$DRIVER into ${MNT_FLEX_DRIVER_DIR} directory"

be consistent with "." at the end of a sentence.

deploy/k8s_deployments/setup_flex.sh, line 24 at r1 (raw file):

mv -f "${MNT_FLEX_DRIVER_DIR}/.$DRIVER" "${MNT_FLEX_DRIVER_DIR}/$DRIVER"

echo "Copying the flex config file ${FLEX_CONF_PATH} to ${MNT_FLEX_DRIVER_DIR}/${FLEX_CONF}"

same here "."

deploy/k8s_deployments/setup_flex.sh, line 28 at r1 (raw file):

cp ${FLEX_CONF_PATH} ${FLEX_TMP}
if [ -n "$UBIQUITY_USERNAME" ]; then
    echo "Update \"username\" in config file based on environment UBIQUITY_USERNAME"

Updating based on environment variable.

deploy/k8s_deployments/setup_flex.sh, line 62 at r1 (raw file):

   fi
else
       echo "The ubiquity public certificate ENV UBIQUITY_PLUGIN_VERIFY_CA is empty, so cannot copy the certificate to ${MNT_FLEX_DRIVER_DIR}"

You just need to say that it's not set and the server certificate will not be verified. I am not sure that all those Variable names help the user, instead instruct him that if he requires to verify the certificate, then point him to the guide to take the next steps.
Same if verification is set.

deploy/k8s_deployments/ubiquity_flex_daemonset.yml, line 1 at r1 (raw file):

# NOTE : This script is obsolete - go to scripts/installer-for-ibm-storage-enabler-for-containers

obsolete ===> delete it

deploy/k8s_deployments/ubiquity-configmap/ubiquity-k8s-flex.conf, line 1 at r1 (raw file):

# NOTE : This script is obsolete - go to scripts/installer-for-ibm-storage-enabler-for-containers

same here

scripts/ubiquity-k8s-provisioner.env, line 5 at r1 (raw file):

# NOTE : This script is obsolete because ubiquity is been installed as docker image.

delete it

Comments from Reviewable

Reviewed 16 of 16 files at r1.
Review status: all files reviewed at latest revision, 10 unresolved discussions.

Comments from Reviewable

Review status: all files reviewed at latest revision, 10 unresolved discussions.

ubiquity-k8s-provisioner.conf, line 1 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
If it's obsolete then why don't you delete it?

Opened a ticket for this cleanup since we should consider to keep some of them with adjustments -> UB-443

cmd/flex/main/cli.go, line 42 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
If the use see this description, it should be reviewed by Dima. Anyway, I would change to "Flex Volume configuration file, containing Ubiquity connection parameters."

Done.

deploy/k8s_deployments/setup_flex.sh, line 16 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
All echo statements should be reviewed by Dima.
"If you are running on a Kubernetes version which is lower then 1.8, a restart to the kubelet service is required to take affect."

Done.

deploy/k8s_deployments/setup_flex.sh, line 20 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
be consistent with "." at the end of a sentence.

Done.

deploy/k8s_deployments/setup_flex.sh, line 24 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
same here "."

Done.

deploy/k8s_deployments/setup_flex.sh, line 28 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
Updating based on environment variable.

Done.

deploy/k8s_deployments/setup_flex.sh, line 62 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
You just need to say that it's not set and the server certificate will not be verified. I am not sure that all those Variable names help the user, instead instruct him that if he requires to verify the certificate, then point him to the guide to take the next steps.
Same if verification is set.

Done.

deploy/k8s_deployments/ubiquity_flex_daemonset.yml, line 1 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
obsolete ===> delete it

Done.
This is for sure need to be deleted.

deploy/k8s_deployments/ubiquity-configmap/ubiquity-k8s-flex.conf, line 1 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
same here

it with scale params.
Lets scrub it later on please.

scripts/ubiquity-k8s-provisioner.env, line 5 at r1 (raw file):
Previously, ranhrl (Ran Harel) wrote…
delete it

part of UB-443

Comments from Reviewable

scripts/installer-for-ibm-storage-enabler-for-containers/scbe-credentials.yml, line 11 at r2 (raw file):

   username: "SCBE_USERNAME_VALUE"

   # Base64 encoded password defined for SCBE Ubiquity interface.

something i am missing here. are we still calling it ubiquity interface?

Comments from Reviewable

scripts/installer-for-ibm-storage-enabler-for-containers/ubiquity_cli.sh, line 13 at r2 (raw file):

#   3. Create ubiquity-k8s-provisioner deployment
#   4. Create ubiquity-k8s-flex Daemonset
#   5. Create ubiquity-db-deployment deployment  (This step uses PVC, so ubiqiuty must be up and ready for it)

typo?

Comments from Reviewable

Reviewed 13 of 16 files at r1, 3 of 3 files at r2.
Review status: all files reviewed at latest revision, 12 unresolved discussions.

Comments from Reviewable


Change FilterMixin's get_filterset_kwargs

In FilterMixin's get_filterset_kwargs function, here's the code to set queryset to kwargs:
        kwargs.update({
            'queryset': self.get_queryset(),
        })

I wonder if it would be better to change it to:
        kwargs.update({
            'queryset': super(FilterMixin, self).get_queryset(),
        })

I think the change would make this mixin more friendly with other views and mixins.

Sorry. Thinking about it again, seems the original code is better.


FIX: backend validation for images uploading

Also, please do the same in category.rb

Can you also add validation of maximum size of 5MB?

@mkhatib what cause it to fail ? I hope it's not my code addition.


VSZ raised 8M after every control time, is it normal?

Hello.
I ran it on an openwrt, and with cmd top I  found that VSZ raised high, every time it increased 8M,
is this a problem?
Greeting

From what I can gather in Instruments for a short period of time, it seems that controlling accessory doesn't cause any leak. The leak on OSX w/ OpenSSL seems to focus on the session pairing. My OpenWRT routers only seeing 2MB per command (then again, mine is mostly binary switches)
So, I guess the problem maybe be the underlying OpenSSL? I will check once I got my Linux platform back in order

But in the meantime, I don't think it will be a big matter. In order to fill your VM, it would probably takes awhile?

my openwrt meminfo:
cat /proc/meminfo
MemTotal:          60784 kB
MemFree:           16968 kB
Buffers:            6652 kB
Cached:            14496 kB
SwapCached:            0 kB
Active:            19660 kB
Inactive:           9712 kB
Active(anon):       8276 kB
Inactive(anon):       64 kB
Active(file):      11384 kB
Inactive(file):     9648 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          8256 kB
Mapped:             4760 kB
Shmem:               116 kB
Slab:               7668 kB
SReclaimable:       2180 kB
SUnreclaim:         5488 kB
KernelStack:         408 kB
PageTables:          372 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:       30392 kB
Committed_AS:      32680 kB
VmallocTotal:    1048372 kB
VmallocUsed:        5336 kB
VmallocChunk:    1029616 kB
only have small memory

Consider your system not having swap, wouldn't 4 more instruction and your system crashed already?
I think some resource is pre-allocated by pthread, hence high VSZ (virtual address). I will see if different compiler would help. (Like I said, I don't have my Linux box up yet. I think it's still in my suitcase somewhere. 😅)

Can you try putting pthread_detach(pthread_self()); on the line 273 of PHKNetworkIP.cpp?
I wonder if it's just pthread doesn't clean up on openwrt.

it doesn't work,
Mem: 45676K used, 15108K free, 0K shrd, 6708K buff, 14496K cached
CPU:   2% usr   4% sys   0% nic  90% idle   0% io   0% irq   0% sirq
Load average: 0.07 0.12 0.13 2/56 28944
PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
1805  1208 root     S     7952  13%   1% {SHGuarder.sh} /bin/sh /root/bin/SHGu
28796 15832 root     S     697m1171%   1% /home/homebridge/PHK
27246     1 root     S    24160  40%   0% ./teleport
15247 15232 root     R     3648   6%   0% top
1     0 root     S     2812   5%   0% /sbin/procd
110     2 root     SW       0   0%   0% [kworker/0:1]
186     2 root     RW       0   0%   0% [mmcqd/0]
1391     1 root     S     3812   6%   0% /usr/sbin/ntpd -n -p 0.openwrt.pool.n
1171     1 root     S     3648   6%   0% /usr/sbin/telnetd -F -l /bin/login.sh
15419  1171 root     S     3648   6%   0% -ash
15832  1171 root     S     3648   6%   0% -ash
15232  1171 root     S     3648   6%   0% -ash
32302     1 avahi    S     3588   6%   0% avahi-daemon: running [CCUW.local]
1154     1 root     S     3584   6%   0% /usr/sbin/crond -f -c /etc/crontabs -
28736 10410 root     S     3536   6%   0% {exe} ftpd -w /home
10410     1 root     S     3472   6%   0% {exe} tcpsvd -vE <IP_ADDRESS> 21 ftpd -w /
1208     1 root     S     3472   6%   0% {AutoRun.sh} /bin/sh /bin/AutoRun.sh
22608 22503 root     S     3472   6%   0% udhcpc -p /var/run/udhcpc-apcli0.pid
22611 22503 root     S     3472   6%   0% udhcpc -p /var/run/udhcpc-apcli0.pid
28936  1805 root     S     3340   5%   0% sleep 3
I found it increased because the code in PHKAccessory.cpp line 553-568, if i make the updateNotify = true , the VSZ wouldn't raise, but I know it should not do this

Add the pthread_detach() at the beginning of broadcastMessage and connectionLoop, then recompile.
void broadcastMessage(void *sender, char *resultData, size_t resultLen) {
pthread_detach(pthread_self());
...
and also
void *connectionLoop(void *threadInfo) {
pthread_detach(pthread_self());
It works on Ubuntu (basically halves the amount since the program gets memory only if it's needed, but virutal page size = sum of all thread theortical stack size)
I'm going to look for actual memory usage to shrink the size, but it will take a while

I just take a look at the meminfo (really busy this week):
VmallocTotal: 1048372 kB
VmallocUsed: 5336 kB
VmallocUsed is the actual page used, which is 5MB, so the underlining memory used is below 5MB (again, pthread_detach would help)
So, unless the memory space runs out (which is solved by pthread_detach), it should be okay
I have one running in a house in Hong Kong, and under the optimized version it takes about 13MB in 21% of virtual memory space. I know it's not in the KB boundary, but it does not grow, and most of the space is reserved in case of pthread grow (which means the space is not taken), or just dynamic library (which means the space is used by loading the library, which takes about 10MB thanks to OpenSSL etc)

Hello @etwmc .
thank you very much.
i just coded according to your information and i cross-compiled the code.
but my OpenWrt was taken away by my colleague, so this moment i can't check the result.
anyway, thank you !
Greeting


dx [breaking change]: Remove function-as-a-child pattern from  and components...

that implement Money
Description
BREAKING CHANGE:
The following components no longer use the function-as-a-child (aka "render props") pattern. See #589 for more details.

Money
CartLinePrice
ProductPrice
SelectedVariantPrice

Additional context
Part of #589

Before submitting the PR, please make sure you do the following:

[x] Add your change under the Unreleased heading in the package's CHANGELOG.md
[x] Read the Contributing Guidelines
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123)
[x] Update docs in this repository for your change, if needed

Did a visual pass on the demo store, and everything looks the same as far as I could tell.

Broke a couple product pages:

http://localhost:3000/products/the-liquid
http://localhost:3000/products/mail-it-in-freestyle-snowboard
http://localhost:3000/products/the-full-stack
http://localhost:3000/products/the-oxygen
http://localhost:3000/products/shopify-aurora

Weird, I'll take another look; was working when I tested it earlier. Thanks!


Command discovery on systems with symlinks to slow filesystems is slow

Not all commands.  ls takes 20 seconds.  dir is nearly instant.  This is just the simplest example, but the overall experience makes me want to stop using powershell on Linux.  :-(  Tab completion also takes around 20 seconds.
Steps to reproduce

PS > measure-command { ls }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 20
Milliseconds      : 25
Ticks             : 200252179
TotalDays         : 0.000231773355324074
TotalHours        : 0.00556256052777778
TotalMinutes      : 0.333753631666667
TotalSeconds      : 20.0252179
TotalMilliseconds : 20025.2179

PS > measure-command { dir }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 3
Ticks             : 33095
TotalDays         : 3.83043981481481E-08
TotalHours        : 9.19305555555556E-07
TotalMinutes      : 5.51583333333333E-05
TotalSeconds      : 0.0033095
TotalMilliseconds : 3.3095

Expected behavior
ls takes a second or so.

Actual behavior
ls takes 20 seconds.

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      6.1.0-preview.1
PSEdition                      Core
GitCommitId                    v6.1.0-preview.1
OS                             Linux 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02)
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
PSRemotingProtocolVersion      2.3
SerializationVersion           <IP_ADDRESS>
WSManStackVersion              3.0

@SurferJeffAtGoogle Can you run Get-Command ls and Get-Command dir to see what commands are actually being executed? We don't install the "ls -> Get-ChildItem" alias on Linux by default so you should just be running the native /bin/ls (or whatever's in your path). We do ship the "dir->Get-ChildItem" alias but you say that's OK which is confusing.

What's surprising is that calling get-command ls also takes 20 seconds!
PS > get-command ls

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Application     ls                                                 <IP_ADDRESS>    /bin/ls

PS > get-command dir

CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias           dir -> Get-ChildItem

PS > measure-command { get-command ls }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 20
Milliseconds      : 35
Ticks             : 200359003
TotalDays         : 0.000231896994212963
TotalHours        : 0.00556552786111111
TotalMinutes      : 0.333931671666667
TotalSeconds      : 20.0359003
TotalMilliseconds : 20035.9003

But with a bash shell, the equivalent of get-command takes hardly any time:
$ which ls
/bin/ls
$ time which ls
/bin/ls

real	0m0.003s
user	0m0.000s
sys	0m0.000s

I have experiencing the same issue sporadically in both Windows and Linux.  Sometimes is fast and other time slow.
(?)

Launching with the full path runs quick:
PS > measure-command { ls }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 20
Milliseconds      : 62
Ticks             : 200625091
TotalDays         : 0.000232204966435185
TotalHours        : 0.00557291919444444
TotalMinutes      : 0.334375151666667
TotalSeconds      : 20.0625091
TotalMilliseconds : 20062.5091

PS > measure-command { /bin/ls }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 6
Ticks             : 66497
TotalDays         : 7.69641203703704E-08
TotalHours        : 1.84713888888889E-06
TotalMinutes      : 0.000110828333333333
TotalSeconds      : 0.0066497
TotalMilliseconds : 6.6497

PS > measure-command { true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 20
Milliseconds      : 30
Ticks             : 200306653
TotalDays         : 0.000231836403935185
TotalHours        : 0.00556407369444444
TotalMinutes      : 0.333844421666667
TotalSeconds      : 20.0306653
TotalMilliseconds : 20030.6653

PS > measure-command { /bin/true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 15
Ticks             : 155612
TotalDays         : 1.80106481481481E-07
TotalHours        : 4.32255555555556E-06
TotalMinutes      : 0.000259353333333333
TotalSeconds      : 0.0155612
TotalMilliseconds : 15.5612

Here's my path.  It's identical in bash, and bash does not experience the same slowness.
PS /usr/local/google/home/rennie/tmp>   ls
PS /usr/local/google/home/rennie/tmp> /bin/ls
PS /usr/local/google/home/rennie/tmp> $env:PATH
/opt/microsoft/powershell/6.1.0~preview.1:/usr/local/google/home/rennie/Downloads/google-cloud-sdk/bin:/usr/local/google/home/rennie/Downloads/phantomjs-1.9.8-linux-x86_64/bin:/usr/local/google/home/rennie/.composer/vendor/bin:/usr/local/google/home/rennie/bin:/usr/local/google/home/rennie/.phpenv/bin:/usr/local/google/home/rennie/Downloads/dotnet-2.0.5:/usr/local/google/home/rennie/Downloads/dotnet-1.1.2:/usr/local/google/home/rennie/Downloads/apache-maven-3.3.9/bin:/usr/local/google/home/rennie/Downloads/node-v4.2.1-linux-x64/bin:/usr/local/google/home/rennie/Downloads/google-cloud-sdk/bin:/usr/local/google/home/rennie/.composer/vendor/bin:/usr/local/google/home/rennie/bin:/usr/local/google/home/rennie/.phpenv/bin:/usr/local/google/home/rennie/Downloads/dotnet-2.0.5:/usr/local/google/home/rennie/Downloads/dotnet-1.1.2:/usr/local/google/home/rennie/Downloads/apache-maven-3.3.9/bin:/usr/local/google/home/rennie/Downloads/node-v4.2.1-linux-x64/bin:/usr/lib/google-golang/bin:/usr/local/buildtools/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PS /usr/local/google/home/rennie/tmp>

rennie@rennie:~/tmp$
rennie@rennie:~/tmp$ env | grep ^PATH
PATH=/usr/local/google/home/rennie/Downloads/google-cloud-sdk/bin:/usr/local/google/home/rennie/Downloads/phantomjs-1.9.8-linux-x86_64/bin:/usr/local/google/home/rennie/.composer/vendor/bin:/usr/local/google/home/rennie/bin:/usr/local/google/home/rennie/.phpenv/bin:/usr/local/google/home/rennie/Downloads/dotnet-2.0.5:/usr/local/google/home/rennie/Downloads/dotnet-1.1.2:/usr/local/google/home/rennie/Downloads/apache-maven-3.3.9/bin:/usr/local/google/home/rennie/Downloads/node-v4.2.1-linux-x64/bin:/usr/local/google/home/rennie/Downloads/google-cloud-sdk/bin:/usr/local/google/home/rennie/.composer/vendor/bin:/usr/local/google/home/rennie/bin:/usr/local/google/home/rennie/.phpenv/bin:/usr/local/google/home/rennie/Downloads/dotnet-2.0.5:/usr/local/google/home/rennie/Downloads/dotnet-1.1.2:/usr/local/google/home/rennie/Downloads/apache-maven-3.3.9/bin:/usr/local/google/home/rennie/Downloads/node-v4.2.1-linux-x64/bin:/usr/lib/google-golang/bin:/usr/local/buildtools/java/jdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
rennie@rennie:~/tmp$

Even when I reduce my path to almost nothing, the performance is similar:
PS /usr/local/google/home/rennie/tmp> $env:PATH
$env:PATH
/opt/microsoft/powershell/6.1.0~preview.1:/usr/bin
PS /usr/local/google/home/rennie/tmp> measure-command { ls }
ls : The term 'ls' is not recognized as the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
At line:1 char:19
+ measure-command { ls }
+                   ~~
+ CategoryInfo          : ObjectNotFound: (ls:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 40
Milliseconds      : 122
Ticks             : 401220615
TotalDays         : 0.000464375711805556
TotalHours        : 0.0111450170833333
TotalMinutes      : 0.668701025
TotalSeconds      : 40.1220615
TotalMilliseconds : 40122.0615

PS /usr/local/google/home/rennie/tmp> measure-command { /bin/ls }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 14
Ticks             : 141501
TotalDays         : 1.63774305555556E-07
TotalHours        : 3.93058333333333E-06
TotalMinutes      : 0.000235835
TotalSeconds      : 0.0141501
TotalMilliseconds : 14.1501

PS /usr/local/google/home/rennie/tmp>

@SurferJeffAtGoogle Is this still an issue with 6.2?

I haven't tried 6.2 yet:
PS /usr/local/google/home/rennie> $PSVersionTable.PSVersion
Major  Minor  Patch  PreReleaseLabel BuildLabel

6      1      3
PS /usr/local/google/home/rennie> measure-command { clear }
Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 14
Milliseconds      : 249
Ticks             : 142493659
TotalDays         : 0.000164923216435185
TotalHours        : 0.00395815719444444
TotalMinutes      : 0.237489431666667
TotalSeconds      : 14.2493659
TotalMilliseconds : 14249.3659
On Sat, Mar 23, 2019 at 12:06 PM Steve Lee<EMAIL_ADDRESS>wrote:

@SurferJeffAtGoogle https://github.com/SurferJeffAtGoogle Is this still
an issue with 6.2?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PowerShell/issues/6577#issuecomment-475896182,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALO54srnStQZaWiQRpKutZjzkwd1KVeLks5vZns3gaJpZM4TJN34
.

@SurferJeffAtGoogle I setup a new Debian 9 VM on my macBook Pro.
Is there a tool I can run to trace where PowerShell is spending its time?
On Mon, Mar 25, 2019 at 11:27 AM Steve Lee<EMAIL_ADDRESS>wrote:

@SurferJeffAtGoogle https://github.com/SurferJeffAtGoogle I setup a new
Debian 9 VM on my macBook Pro. Installed 6.1.3, and executed:
measure-command { clear }
That finished in 46ms
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PowerShell/issues/6577#issuecomment-476322368,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALO54iKThLfzku3XDHnNqrJg21avv2nWks5vaRUHgaJpZM4TJN34
.

Would you be able to clone my branch from https://github.com/PowerShell/PowerShell/pull/9230 which adds timestamps to the trace command and do this:
trace-command -Name * -Expression { true } -PSHost

And see where it's spending its time?

Thank you.  I'll try that.  But it could take a few days due to my other
commitments.
On Tue, Mar 26, 2019 at 11:22 AM Steve Lee<EMAIL_ADDRESS>wrote:

Would you be able to clone my branch from #9230
https://github.com/PowerShell/PowerShell/pull/9230 which adds
timestamps to the trace command and do this:
trace-command -Name * -Expression { true } -PSHost
And see where it's spending its time?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/PowerShell/PowerShell/issues/6577#issuecomment-476784404,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ALO54t30ivILSl7Yc95tMSSjBCra0SuRks5vamVNgaJpZM4TJN34
.

@SurferJeffAtGoogle Thanks!  No rush, not much we can do for 6.2 right now anyways.

Fellow Googler here, with similar corp setup.
The problem seems to be that we have symlinks in /usr/bin, pointing to files on slow filesystems. I zsh I can see ls --color=never /usr/bin is instant but ls ---color=always /usr/bin takes seconds (presumably by resolving symlinks to print them in them red if invalid). I can confirm by looking at trace-command output that the time is spent "looking for true in /usr/bin" and removing /usr/bin from $env:PATH fixes the issue.

Thank you @bendowski!  That helps.

I see this as a pain point, but this seems to do with the filesystem perf and not something PowerShell can do without someone coming up with a proposal

For the record: other shells (bash, zsh) work fine in this setup. I can only guess they don't check the executable bit (or other file attributes) ahead of time for every file on the PATH, only when needed to actually run a binary.

@bendowski ok, that's something we could potentially do as an optimization!

We look files by paths from PAT environment variable and by executable extentions in https://github.com/PowerShell/PowerShell/blob/48db3de03ed427f10646f1fbb824a2b4eb958378/src/System.Management.Automation/engine/CommandPathSearch.cs#L408
We could try new API  FileSystemEnumerable() in the method where we can use custom predicate to filter files in directory but it is another scenario (Suggestions, or FuzzyMatch).
Also we see possible perf issue in the algorithm. We use search pattern like "ping.exe." and then the same for all extensions - "ping.exe.ps1", "ping.exe.cpl" and so on, but it seems the first pattern overlaps the follow ones: if we do not find first then it makes no sense check rest.
Also file enumeration is perf expensive. So for a file we should check all patterns but we do enumerate for every pattern.

@SteveL-MSFT If I remember correctly PATHEXT is only for Windows. So
there is no need to check extensions on Unix-s, right?

@SteveL-MSFT We have a perf regression on Windows. For Get-Command ping ~3 ms on my system.
Test script:
for ($i=0; $i -lt 2000; $i++) {
    Get-Command ping > $null
}

On Windows PowerShell and up to PowerShell Core 6.1.0-Preview2 I get ~2.5 sec.
Starting with PowerShell Core 6.1.0-Preview2 I get ~8.2 sec.
I compare traces for PowerShell Core 6.1.0-Preview2 and PowerShell Core 6.1.0-Preview3

You can see that there other P/Invokes are used in Preview3.
I guess it was a change in .Net Core. If so I guess they know about this and accepted consciously.
Should we worry about this or can we ignore it?

@iSazonov yes, PATHEXT is a Windows only thing.  On non-Windows, the executable bit is what is interesting although we special case .ps1.  The perf data is different from what this issue is discussing which is specific to traversing symlinks opening files at discovery time rather than at runtime.

@SteveL-MSFT Thanks! Clear about PATHEXT.
My trace is for discovery code. I think just the enumeration is slow, not exec bit check.
Our exec bit check is on last step for single file (after we enumerate all files in a directory and filter by name) so if we remove them it does not help.

@bendowski @SurferJeffAtGoogle Please download compiled artifact from #10815 and look if you see any perf change.

@iSazonov Can you open a corefx issue about Directory.EnumerateFiles()? That perf regression is really bad and not expected.

@0xd4d PowerShell Core repo is not useful for Core team. This requires a C# repo. I didn't have time to check it out (on different .Net Core versions).

@iSazonov I opened a corefx issue about that problem https://github.com/dotnet/corefx/issues/41898

I'm sorry for the delay. I tried the build you linked as well as the latest preview (7.0.0-preview5) and they seem to behave the same. It's much better than before (with ls taking 20 seconds), but still some commands take longer than I would expect. This seems to be somewhat non-deterministic, see this example of running "true" 4 times in a row:
PS /> measure-command { true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 28
Ticks             : 289087
TotalDays         : 3.34591435185185E-07
TotalHours        : 8.03019444444444E-06
TotalMinutes      : 0.000481811666666667
TotalSeconds      : 0.0289087
TotalMilliseconds : 28.9087

PS /> measure-command { true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 443
Ticks             : 4435776
TotalDays         : 5.134E-06
TotalHours        : 0.000123216
TotalMinutes      : 0.00739296
TotalSeconds      : 0.4435776
TotalMilliseconds : 443.5776

PS /> measure-command { true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 34
Ticks             : 346932
TotalDays         : 4.01541666666667E-07
TotalHours        : 9.637E-06
TotalMinutes      : 0.00057822
TotalSeconds      : 0.0346932
TotalMilliseconds : 34.6932

PS /> measure-command { true }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 422
Ticks             : 4221326
TotalDays         : 4.88579398148148E-06
TotalHours        : 0.000117259055555556
TotalMinutes      : 0.00703554333333333
TotalSeconds      : 0.4221326
TotalMilliseconds : 422.1326

Doing the same with trace-command confirms the additional ~400ms is spent in /usr/bin:
DEBUG: 2019-10-24 13:53:38.2773 CommandDiscovery Information: 0 : Looking for true.ps1 in /usr/sbin
DEBUG: 2019-10-24 13:53:38.2784 CommandSearch Information: 0 :  WriteLine   Current directory results are invalid
DEBUG: 2019-10-24 13:53:38.2785 CommandSearch Information: 0 :  WriteLine   Current patterns exhausted in current directory: /usr/sbin
DEBUG: 2019-10-24 13:53:38.2785 CommandDiscovery Information: 0 : Looking for true in /usr/bin
DEBUG: 2019-10-24 13:53:38.6796 CommandSearch Information: 0 :  WriteLine   Current directory results are invalid
DEBUG: 2019-10-24 13:53:38.6798 CommandDiscovery Information: 0 : Looking for true.ps1 in /usr/bin
DEBUG: 2019-10-24 13:53:38.6914 CommandSearch Information: 0 :  WriteLine   Current directory results are invalid
DEBUG: 2019-10-24 13:53:38.6914 CommandSearch Information: 0 :  WriteLine   Current patterns exhausted in current directory: /usr/bin

Would love to see if someone could reproduce this one again on .NET 5 with the latest version of Win10. I know a ton of work has been done around filesystem access and symlinks in both of those products.

Just noting that this issue still seems to be present on the latest PowerShell 7.1.
I tried to install PowerShell 7 on my gLinux host today. Unfortunately, it's too slow at the moment for me to switch to using PowerShell on Linux. I did try modifying path as @bendowski suggested, but still experienced this issue.
Happy to provide more logs as necessary and help troubleshoot, as I imagine others have/will experience the same issues.
The latest PowerShell 7 on my Windows corp workstation (20H2) does not experience this issue.

Just noting that this issue still seems to be present on the latest PowerShell 7.1.
I tried to install PowerShell 7 on my gLinux host today. Unfortunately, it's too slow at the moment for me to switch to using PowerShell on Linux. I did try modifying path as @bendowski suggested, but still experienced this issue.
Happy to provide more logs as necessary and help troubleshoot, as I imagine others have/will experience the same issues.
The latest PowerShell 7 on my Windows corp workstation (20H2) does not experience this issue.

Happy to provide more logs as necessary and help troubleshoot, as I imagine others have/will experience the same issues.

@TsekNet I have already prepared some fixes. There is already open PR (one from some follow). But we have no resources to make this work fast. We need more code reviewers and developers. MSFT team introduce Working Groups (you can read in their PowerShell blog). You could take participation in related WG if you have an interest.

Happy to provide more logs as necessary and help troubleshoot, as I imagine others have/will experience the same issues.

@TsekNet I have already prepared some fixes. There is already open PR (one from some follow). But we have no resources to make this work fast. We need more code reviewers and developers. MSFT team introduce Working Groups (you can read in their PowerShell blog). You could take participation in related WG if you have an interest.


Rename API command file_seed

"seeding a file" generally refers to the act of making a file accessible to peers in a p2p network.
API command file_seed did not actually do that, the command is either for stopping or restarting a download.
I renamed it to file_set_status but open to other suggestions.
Also ran the gen_api_docs.py script , I see that there are some other API changes that needs this.

merged


Yatin: Watch se.com

Yatin requests the watch of the watch_keyword se\.com. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

se\.com has been seen in 1007 true positives, 292 false positives, and 41 NAAs.

Rejected by tripleee in Charcoal HQ. Reason: 'seems too FP-prone (3/5)'

The MS results are actually 3/5, not the thousands of hits reported above.


renamed matadata.yml to metadata.yaml

/lgtm
/approve

@wanghaoran1988 we should card and address this maybe in a wider level?

we should card and address this maybe in a wider level?

I agree, a managedJob should be created regardless of the metadata file extension


Example Scripts/Commands for Creating SFT & Reward Models for PPO/RLOO/Other Trainers

Feature request
Please provide example scripts in https://github.com/huggingface/trl/tree/main/examples/scripts/ppo for how to create corresponding SFT and RM checkpoints to use for PPO
Motivation
TRL helpfully provides an example script of PPOv2Trainer.
However, it does not provide an example script of how the SFT checkpoint (cleanrl/EleutherAI_pythia-1b-deduped__sft__tldr) and Reward Model checkpoint (cleanrl/EleutherAI_pythia-1b-deduped__reward__tldr) were created.
I tried creating my own SFT and RM checkpoints using SFTTrainer and RewardTrainer and then ran PPO using PPOv2Trainer. The SFT and RM training runs looked reasonable (stable, good test losses, good test accuracies for the RM) but the PPO runs were unstable and bad. I now do not know whether my SFT'd model is bad, whether my reward model is bad, or whether I have bad hyperparameters for PPO.
It would be amazing if scripts and/or commands were included for demonstrating how to properly create SFT and RM checkpoints to subsequently use in PPO, RLOO, other methods.
Edit: I know examples exist for SFT and RM training. I want examples that specifically demonstrate how to create SFT and RM checkpoints for PPO because my attempts thus far have not been successful.
Your contribution
I will try today to reproduce the Pythia 1B deduped checkpoints that are used. If that works, I can open a PR demonstrating how I created them. If that doesn't work, I don't know what I can do.

I did open discussions on the cleanrl models but I haven't heard back: https://huggingface.co/cleanrl/EleutherAI_pythia-1b-deduped__reward__tldr/discussions/1

I just discovered that the default RM has no padding token nor chat template:
https://huggingface.co/cleanrl/EleutherAI_pythia-1b-deduped__reward__tldr/blob/main/tokenizer_config.json
This is inconsistent with the corresponding default SFT model:
https://huggingface.co/cleanrl/EleutherAI_pythia-1b-deduped__sft__tldr/blob/main/tokenizer_config.json
which also has no chat template. This makes me think that the reward model was trained differently than the SFT'd equivalent model, and that the SFT'd model is used with a chat template it wasn't trained on in the PPOv2Trainer example.
I really think we need a demonstration of how to make SFT'd models and reward models to use with PPOv2Trainer
cc: @qgallouedec


Make zulip more compatible with mypy 0.4.7.

mypy 0.4.7 gives many errors on zulip. I have fixed some of them.

Automated message from Dropbox CLA bot
@sharmaeklavya2, it looks like you've already signed the Dropbox CLA. Thanks!

Merged the last commit too, thanks @sharmaeklavya2!!


changed input to raw_input

After playing around trying to get this to work with when it wanted strings and when it didnt want strings,
I changed the input type to raw_input to accept strings without throwing a error.
I also added a str() command around the OTP code it sends off to onelogin as its expecting a string. During the interactive input, it didn't reject the otp code and then the server kicked back a 500 response and the script crashed with a json not parsed error.

Now that I think about it, this changes in Python 3.0.. :sigh:

@jmymy Thanks for digging into this.  I've added a note to the README to use python 3.  Lemme know if you have other ideas.


When I run the test case under the command line package, I get an error. I don't know if it is a bug, or does it require some prerequisites?

I'm not sure if this is a bug, please take a look.

@hk-lrzy @lichang-bd @BlockLiu

I think it's IDEA cache problem. Please reimport the maven project or invalidate cache and restart intellij.
https://stackoverflow.com/questions/12132003/getting-cannot-find-symbol-in-java-project-in-intellij

@BlockLiu
Boss, I cleaned the idea's cache and reopened the project, but it didn't work.😂

@lichang-bd

@hk-lrzy @BlockLiu
Good, your suggestion is very helpful to me, it's not an idea cache problem, it's that the enable annotation processing setting is not checked😂
thank you for your help.

The test is running


feat: marshal toml with yaml.MapSlice and 3rdparty toml lib

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[x] N
[ ] Y

feat: marshal toml with yaml.MapSlice and 3rdparty toml lib

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 6 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.02%) to 63.177%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/tools/gen/gentoml.go
0
6
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.02%

Covered Lines:
3615

Relevant Lines:
5722

💛  - Coveralls


a bug  about buying and selling in China A

In China A,  'hundred_each_trade' apply in buying stock,  not in selling.
It's the opposite in you code.

Thanks for reporting it. We will test it.

Both in selling and buying, hundred_each_trade is True by default.

Our price data are already adjusted for (stock) dividends. Thus we don't consider the case of stock dividends in simulation. I think it is OK to restrict buying and selling to 'hundred_each_trade' in the A-shares stock trading environment.


feat: add password security requirements

What kind of change does this PR introduce?
feature
What is the current behavior?
Any password equal to or greater than the minimum length is permitted
What is the new behavior?
Four new config settings that require at least one lowercase, uppercase, numeric and special character in the password. Trying to set a password that doesn't comply with these requirements results in a 422 error (similar to the existing behaviour for passwords that are too short).
Additional context
This could be done in a bunch of other ways (e.g. regular expressions, some way of requiring X different sets but not necessarily caring about which ones, ...). I had a look around for other configurable password policies and the only one I could see that was clearly documented is the one for AWS IAM: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_passwords_account-policy.html#password-policy-details
Relevant to this change is the recent decision by the French GDPR authority in relation to Discord (see https://www.cnil.fr/en/discord-inc-fined-800-000-euros):

At the time of the online investigation, when creating an account on DISCORD, a password of six characters including letters and numbers was accepted.
The restricted committee considered that DISCORD's password management policy was not sufficiently strong and restrictive to ensure the security of users' accounts.

Hi @kangmingtay, thanks for the reply. Do you have a rough timeline for when your alternate solution would be available?
We can't really use supabase in production in good conscience (or in compliance with the GDPR) without a way to prevent horribly insecure passwords.

Hi @kangmingtay, thanks for the reply. Do you have a rough timeline for when your alternate solution would be available?

Webhooks are one of our top next priorities, so ideally (tho this may change) is the first half of 2023.

We can't really use supabase in production in good conscience (or in compliance with the GDPR) without a way to prevent horribly insecure passwords.

You can enforce this on the browser side for the time being. It's actually much better to do it there as you can guide the user towards a secure password. You can also integrate with HaveIBeenPwned or use libraries like zxcvbn. Backend enforcement is good, but as @kangmingtay explained we plan on letting you customize this behavior using custom hooks / functions.

Hey @csmith,
Thanks for patiently waiting. We've recently introduced more configurability around password security checks which should address the issue.
I'm going to close the PR for now but let me know if there are unresolved concerns or additional features needed.
Once again, thank you for contributing to Supabase


fix: pagination as links bug, fullscreen props

This began as a refactor of several components away from classes. While doing so I was able to remove the Delegate library that was throwing react errors because it was very out of date. Additonally I found a bug with the way proptypes were defined for headerComponent in two components, and also a bug with the pagination when trying to use the pager buttons as links.
Was not able to refactor some tests away from enzyme with this as I'd hoped because create does not seem to work with jest mock functions. This may be why Clint used two patterns in his PR.

Description

Which issue(s) does this PR relate to?

Testing
Validate the behavior is the same in prod storybook for DonutChart, Dropdown, FullScreenView, Modal (fullscreen)
Validate that the pagination that uses links actually do something in the browser.
Trade-offs

Screenshots

Checklist

[ ] If any new components were added, there are exported from packages/index.ts
[ ] This PR is associated with a JIRA and mentions in the commit message footer ("Closes …")
[ ] This PR contains breaking changes and states in the commit message body ("BREAKING CHANGE: …")
[ ] I have reviewed the changes and provided detail to the sections above

I think the page reloading on Previous And Next Buttons As Links is the desired behavior... but maybe I should revert it back to what it was. Just didn't make sense as it was written to only show the urls if disabled

I think the page reloading on Previous And Next Buttons As Links is the desired behavior... but maybe I should revert it back to what it was. Just didn't make sense as it was written to only show the urls if disabled

@vacas5 I think you should keep what you've changed it to. Sure, it isn't exactly how we'd expect it to behave, but from what I can tell, it was broken before. At least it's working now, even if the links don't change depending on what page you're on.


Improve Typescript support for as property

I'm struggling on the following use case:
I defined a link component:
const Link = styled.a`
color: pink;
`

And for outbound links I use a custom component. Let say:
const OutboundLink: React.FC<{to: string} & AnchorHTMLAttributes<HTMLAnchorElement>> = ({to, ...props}) => <a {...props} href={to} />

If I'm right, this is a typical use case of the prop as available in styled-components v4.
So when adding an outbound link, I write:
const MyLink = () => <Link as={OutboundLink} to='https://www.styled-components.com/' />

But typescript raises an error: Property 'to' does not exist on type IntrinsicAttributes & Pick<DetailedHTMLProps<AnchorHTMLAttributes<HTMLAnchorElement>, HTMLAnchorElement>>.
For now I am dealing with this case like this:
const SomeLink = styled(Link)<OutboundLinkProps>``

Do you have any recommendation on how to handle this use case? Or any plan to support this use case in the future?
Thanks for your help

Please feel free to open this over at DefinitelyTyped, but we do not maintain the types here.

Your solution works if Link and OutboundLink share the same props.
You'll also lose the custom props needed by the initial component, ex:
const Link = styled.a<{ color: string }>`
  color: ${props => props.color};
`

const SomeLink = styled(Link)<OutboundLinkProps>``

<SomeLink to="/" /> // -> error, color is missing

I use this to solve the issue (works with components and elements)
export function GetComponentAs<T extends keyof JSX.IntrinsicElements | React.ComponentType<any>, P extends object>(
  component: StyledComponent<any, any, P>,
  as: T
) {
  return (component as any) as StyledComponent<T, any, P>;
}

usage
const SomeLink = GetComponentAs(Link, OutboundLink)


Homematic does not work properly

Try normal thermostat as homematic-thermostat is only for Homematic products.

Hi, what exactly is not working? What is mode and power? Please send the RAW of the datapoint.
Von: gujiys<EMAIL_ADDRESS>Gesendet: Montag, 24. Juni 2019 21:51
An: sbormann/ioBroker.iqontrol<EMAIL_ADDRESS>Cc: Sebastian Bormann<EMAIL_ADDRESS>Comment<EMAIL_ADDRESS>Betreff: Re: [sbormann/ioBroker.iqontrol] Homematic does not work properly (#52)
[30]https://user-images.githubusercontent.com/50275040/60047495-70ce3d00-96fc-11e9-9f97-e9a02e360f79.png
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/sbormann/ioBroker.iqontrol/issues/52?email_source=notifications&email_token=ALXEXBB3GPAF4X5JXQXG4S3P4EQSVA5CNFSM4H2XMNU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYOAVXI#issuecomment-505154269, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALXEXBEWVOOBD5EAEUURC7TP4EQSVANCNFSM4H2XMNUQ.

No further feedback - therefore closed. Please reopen, if the issue persists.


Bug: failure handling error of TinuURL API calls

Describe the bug
LD-Cool-P fails to correctly handle exceptions related to creating TinyURL links. This means that if an error happens, subsequent links for that dataset wont get generated
Reproduction Steps
Run get_qualtrics for a dataset that generates a TinyURL error
Outputted Messages
URLs have been redacted
8:01:24 -     INFO: TinyURL link does not exist. Creating!
18:01:25 -  WARNING: Caught an HTTPError: 400 Client Error: Bad Request for url: http://tinyurl.com/api-create.php?url=xxxxxx
--- Logging error ---
Traceback (most recent call last):
  File "/home/curator/codes/LD_Cool_P/ldcoolp/curation/email/urls.py", line 56, in tiny_url
    response.raise_for_status()
  File "/home/curator/.local/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://tinyurl.com/api-create.php?url=xxxxxx

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/anaconda3/envs/curation/lib/python3.7/logging/__init__.py", line 1025, in emit
    msg = self.format(record)
  File "/opt/anaconda3/envs/curation/lib/python3.7/logging/__init__.py", line 869, in format
    return fmt.format(record)
  File "/opt/anaconda3/envs/curation/lib/python3.7/logging/__init__.py", line 608, in format
    record.message = record.getMessage()
  File "/opt/anaconda3/envs/curation/lib/python3.7/logging/__init__.py", line 369, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
Call stack:
  File "/home/curator/codes/LD_Cool_P/ldcoolp/scripts/generate_qualtrics_links", line 93, in <module>
    da_tinyurl = urls.tiny_url(full_url, alias=da_alias)
  File "/home/curator/codes/LD_Cool_P/ldcoolp/curation/email/urls.py", line 60, in tiny_url
    log.warning('Body:\n', response.text)
Message: 'Body:\n'
Arguments: ('Error',)
Traceback (most recent call last):
  File "/home/curator/codes/LD_Cool_P/ldcoolp/curation/email/urls.py", line 56, in tiny_url
    response.raise_for_status()
  File "/home/curator/.local/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://tinyurl.com/api-create.php?url=xxxxxx

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/curator/codes/LD_Cool_P/ldcoolp/scripts/generate_qualtrics_links", line 93, in <module>
    da_tinyurl = urls.tiny_url(full_url, alias=da_alias)
  File "/home/curator/codes/LD_Cool_P/ldcoolp/curation/email/urls.py", line 61, in tiny_url
    raise HTTPError
requests.exceptions.HTTPError

Version information

LD_Cool-P version: 1.1.17

Expected behavior
Execution continues with a warning logged

This bug surfaced due to the bug in #261

fixed in #260