Normalize:  could not find function "sizeFactors"

Hello,
I have a dataset to normalize and when using normalize from scib I get the following error:
RRuntimeError: Error in sizeFactors(computeSumFactors(SingleCellExperiment(list(counts = data_mat)),  :
could not find function "sizeFactors"

Versions:
scib                          1.0.3
scanpy                        1.9.1
anndata                       0.8.0
anndata2ri                    1.0.6
Any tips on where this is coming from are very much apprechiated.

Hi, thanks for pointing that out.
You are missing the R package scran for that function to work properly. We haven't addressed it in the documentation yet. Since R methods can't be managed by python installers, you need to install scran manually to the R instance that rpy2 accesses. Make sure you have Bioconductor installed as well.
We implemented scran as it was recommended by the Current best practices in single-cell RNA-seq analysis.
Depending on your problem, you might also want to consider using one of the simpler normalisation methods implemented in scanpy.


initial kafka-consumer commit

This is the inital submission of the kafka consumer code. More to come

Thank you for the feedback. It has been really busy lately but will work on addressing your feedback in the coming weeks

Any updates on this PR? I'm very interested in this function. It would be useful that BGPalerter can monitor not only RISLive but also other sources (e.g. BMP, etc).

Hi @a16, that is the exact use case we are using. We collect BMP data, feed to a kafka broker and then consume that with BGPAlerter. Its been a while since I last looked but I believe that we are running the code in this PR in our environment without issue.
Regarding the changes suggested by @massimocandela, unfortunately this has fallen into the backlog for me as other high priority items have surfaced. I still plan to go back and make the suggested changes but that may take some time

@massimocandela I wanted to thank you again for the feedback you have provided. I have started implementing some of it and mainly focused on kafka-node -> kafka-js. I will be continuing to address feedback

@massimocandela I had a question regarding monitoring. We are trying to monitor much more than just our own ASN so for now we have made some queries to apis to return prefixes and their ASNs. This then gets parsed into the prefixes.yml for BGPAlerter.
I think there are many challenges to this but do you think there is there a way that one could "Monitor everything"? One large challenge I think would be establishing the source of truth as this is done through the config/prefix files


Use subprocess instead of os.system

closes #1026, closes #1470, closes #1476

Master is now Python 3 only. This needs to be rebased, and compat for shutil.which can be removed.
#1543 reverts the previous shlex.quote pr because of an issue with calling quote on a command that might actually be a command and options, like less -FRSX (and also an issue with quoting on Windows, which is sidestepped here). Unfortunately, just calling shlex.split on the command first will fail on Windows unless slashes in paths are double escaped:
>>> shlex.split("C:\\path\\to\\exe /test other\\path")
['C:pathtoexe', '/test', 'otherpath']
>>> shlex.split("C:\\path\\to\\exe /test other\\path".replace("\\", "\\\\"))
['C:path\\to\\exe', '/test', 'other\\path']
>>> shlex.split("C:\\path\\to\\exe /test other\\path", posix=False)
['C:path\\to\\exe', '/test', 'other\\path']

Technically that last one with posix=False is not correct, as the argument refers to non-Posix shells on Unix, not completely different platforms like cmd on Windows. Either solution would probably be fine for the limited case here, although I'd lean towards replace since it's more clear what's being special-cased.

It seems that when more is piped into, it adds \r\n on Windows. It does not do this when reading from a file using more < .... I tried some workarounds but failed so it's not using piping with more for that reason.
Additionally, which usage is required as recommended by the Popen Python docs. If not used on Windows, it will results in an OSError because of there being a difference on how the resolving works. I've made the pager methods return True if which was able to find the file and False otherwise. This makes it easy to try other pagers we support easily.
It seems to be that newline behaviour is different on GitHub's test runner however. On GitHub \n is not translated into \r\n but it is on my machine. 🤔


Syntax errors in example script to create AG with dtc_support

The script to create an Availability Group with DTC Support contains syntax errors.

Server1 and Server2 need to be enclosed by single quotes.
Missing comma between the specified replica's.
CREATE AVAILABILITY GROUP MyAG
WITH (
DTC_SUPPORT = PER_DB
)
FOR DATABASE DB1, DB2
REPLICA ON
'Server1' WITH (
ENDPOINT_URL = 'TCP://SERVER1.corp.com:5022',
AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,
FAILOVER_MODE = AUTOMATIC
),
'Server2' WITH (
ENDPOINT_URL = 'TCP://SERVER2.corp.com:5022',
AVAILABILITY_MODE = SYNCHRONOUS_COMMIT,
FAILOVER_MODE = AUTOMATIC
)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4525f922-c768-bb9c-fc93-4a3b790c03fb
Version Independent ID: f50f9610-c6a6-0fad-5f53-975a64a53704
Content: Configure distributed transactions for an availability group - SQL Server Always On
Content Source: docs/database-engine/availability-groups/windows/configure-availability-group-for-distributed-transactions.md
Product: sql
Technology: high-availability
GitHub Login: @MashaMSFT
Microsoft Alias: mathoma

@Ventilios -- Ymer, thank you for your feedback.  I am updating the documentation.

@Ventilios -- Ymer, thank you for your feedback.  I have updated the documentation and it should be live by tomorrow.


Open repo

Allow users to open the repo of fossasia gci 15 website with the app. It should show if origin and upstream remotes are configured correctly.

Working on this issue...

@YagoGG Great!
Here are some ideas:

Use Dialog module. Its only available in the main process. (read about electron's renderer and main processes)

To communicate between the main and renderer processes you can use the ipcMain module.

Let me know if you have other ideas.

Dialog was exactly what I was looking for! Thanks!

I'm having some problems trying to run code after all the code Promise has finished running.
I need to use a variable in a switch() case structure, but that variable's content has been set inside the promise.
Chaining a .then() after the Promise won't work.
Here's a synthesis of what I'm trying to do:
Git.Repository.open(path)
        .then(function(repo) {
                // Generate myVar
        })

switch(myVar)
        case ...

If I attempt to run the code above, the switch won't work properly, because the part before is async, and myVar hasn't been set, yet.
Ideas?

As you've said code inside the then block runs asynchronously. So there is no alternative. You have to use the switch case statement inside the then block. Why do you need to keep it outside?
You can make a PR with what you did so far and keep it open and work on it. Others can review and give you feedback that way.

@YagoGG If you are having a problem with understanding asynchronous logic, try out the guides on bluebirdjs.com , it's what nodegit uses internally.

Thanks you both! I finally sorted it out, I'll create my PR soon.

@YagoGG good to hear. :)

closed by #14


Crawl4AI Error: This page is not fully supported.

I was wondering if you could help me with a recurrent issue which I can find no repeatable solution for. Giving this URL as an example: https://www.newcleo.com/. I have tried many combinations of wait_for, and various js_code strategies but cannot access the actual page. I don't see any significant anti-bot measures on chrome. I do notice that for a split second a .gif animation pops up before the page renders. If i try to use crawl4ai without a delay I basically scrape this url. If i add a delay I see the following error.
async def main():
    async with AsyncWebCrawler(always_by_pass_cache=True, verbose=True) as crawler:
        result = await crawler.arun(
            url="https://www.newcleo.com/",
            magic=True,
            headless=True,
            #delay_before_return_html=5.0
            )

        print(result.markdown)
        return None

if __name__ == "__main__":
    asyncio.run(main())

Crawl4AI Error: This page is not fully supported. Possible reasons: 1. The page may have restrictions that prevent crawling. 2. The page might not be fully loaded. Suggestions: - Try calling the crawl function with these parameters: magic=True, - Set headless=False to visualize what's happening on the page. If the issue persists, please check the page's structure and any potential anti-crawling measures.
Thanks for any help!

@Olliejp Please try the following code:
async def main():
    async with AsyncWebCrawler(
            headless=False,  # Set to False to see what is happening
            verbose=True,
    ) as crawler:
        result = await crawler.arun(
            url="https://www.newcleo.com/",
            cache_mode=CacheMode.BYPASS,
            remove_overlay_elements=True,
            wait_for="css:.background-video"
        )
        print(len(result.markdown))

if __name__ == "__main__":
    asyncio.run(main())

Pay attention to this code. Typically when I face an issue, first I set the headless mode to false to see the browser and understand what is going on. In this case, you will notice that in headless mode, the initial GIF animation takes a little bit more time. Then, we will not get anything; we just get the content of that text animation, so that's why I use 'wait_for'. I waited for a specific class, which is important; it's background-video. There's also another way: you can just add a delay to use a delay before getting the HTML, and then pass that delay, and then wait for that one. Let me know if you have any other questions.

Just FYI exactly the same behavior with this url:
https://nanonuclearenergy.com/
With magic = True I get this error:
[ERROR] 🚫 arun(): Failed to crawl https://nanonuclearenergy.com/, error: [ERROR] 🚫 crawl(): Failed to crawl https://nanonuclearenergy.com/: Page.evaluate: Execution context was destroyed, most likely because of a navigation
and with magic=False, I get no error at all.

@Olliejp you are right that you can use the delay before returning the final HTML; absolutely, it's a good way to not be dependent on its specific HTML elements on a website and other things. I just released a new version right now, and when I set magic to true, I don't get the error that you are getting, so I assume that recent changes fixed this issue, whatever it was. Please check that one. Let me know.
async def main():
    async with AsyncWebCrawler(
            headless=False,  # Set to False to see what is happening
            verbose=True,
    ) as crawler:
        result = await crawler.arun(
            # url="https://www.newcleo.com/",
            url="https://nanonuclearenergy.com/",
            cache_mode=CacheMode.BYPASS,
            magic=True,
            remove_overlay_elements=True,
            # wait_for="css:.background-video",
            delay_before_return_html=5.0
        )
        print(len(result.markdown))

if __name__ == "__main__":
    asyncio.run(main())

Output:
[INIT].... → Crawl4AI 0.3.74
[FETCH]... ↓ https://nanonuclearenergy.com/... | Status: True | Time: 7.63s
[SCRAPE].. ◆ Processed https://nanonuclearenergy.com/... | Time: 78ms
[COMPLETE] ● https://nanonuclearenergy.com/... | Status: True | Total: 7.73s
13303


Style for bottom right corner in ScrollView is missing

I encounterd a missing styling in the ScrollView that is inside a TextBox.
The corner rectangle in the bottom right that fills the space for both the vertical and horizontal scrollbar is tinted white in dark mode:

There sort of is a workaround as it is possible to override this style with the following code:
<TextBox>
    <TextBox.Resources>
        <SolidColorBrush x:Key="{x:Static SystemColors.ControlBrushKey}" Color="{DynamicResource {x:Static adonisUi:Colors.Layer2BackgroundColor}}"/>
    </TextBox.Resources>
</TextBox>

Resulting into the following:

But unfortunatley this overridden color does not match the background gradient of the TextBox that is present for the first 100 pixels or so.
(The color difference might be hard to notice in the images)

Thank you for reporting this!
I will have a look at it as soon as I can.

Has this been merged into master? I'm having the issue as well:

Using AdonisUI 1.10.0

It is merged and should be fixed in the latest version on NuGet as well (v1.11.0-preview).
PS:
A stable version is following in a few days I hope. I released it as a preview because the NuGet packaging changed in this release because of .NET Core 3.0 and I wanted to be careful to not break anything. But as everything seems to work fine for everyone and my personal tests passed as well, I will probably just make the current release a stable one without changing anything. So the preview can be used relatively safely.

Glad to hear, I'm not in a hurry with the release so I'll wait for it to go to master. Thanks!


v and ^v

only V works right now. Normal selection and block selection would be awesome.

I'm quite interested in getting these working as well (especially blockwise visual), I might take this on at some point if it's possible.

v (lowercase) should work. Block mode is harder as you can't use the normal selection feature of MD to select multiple unconnected text ranges.

Hm... so they don't expose an API for the built-in block selection? (https://mhut.ch/journal/2011/03/monodevelop_tips_block_selection_mode)

In theory you can either reimplement logic for selecting by copying parts of code from MonoDevelop or open PR against MonoDevelop repository with logic that exposes API that you need...
Blog post that you mentioned... This line changes setting: https://github.com/mono/monodevelop/blob/master/main/src/addins/MonoDevelop.SourceEditor2/MonoDevelop.SourceEditor/SourceEditorView.cs#L3400 look at rest of file where this setting changes behavior of selection logic...


Implement TTN controller/operator

Implement an example controller/operator using the device registry events. Syncing devices from our internal registry to TTN, using the v3 API.
This should:

Sync the app
Sync devices of the app
Set up at least one endpoint (e.g. HTTP)
Work with upstream and downstream events

There may be some loose ends, but a first (working) version of that exists!


Trying to install the extension and this is what I got

Is there a restriction regarding the VS code version ?

Can't install 'reactreedev.reactree' extension because it is not compatible with the current version of Visual Studio Code (version 1.68.1).

Same only with even lower version


Allow AMQP automatic recovery to be enabled

Automatic recovery is disabled by default in AMQP and there was no way to change that flag.
The change keeps default to false, but allows the recovery to be enabled.

Thanks. I merged your change.


USD conversion stoped working

This glTF can't get converted:
$ export PYTHONPATH="/usr/local/USD/lib/python" && python /usr/local/gltf2usd/Source/gltf2usd.py -g /var/app/current/tmp/1575031728-28148742/model_with_local_uris.gltf
 -o /var/app/current/tmp/1575031728-28148742/dist.usdz
Traceback (most recent call last):
  File "/usr/local/gltf2usd/Source/gltf2usd.py", line 923, in <module>
    convert_to_usd(os.path.expanduser(args.gltf_file), os.path.abspath(os.path.expanduser(args.usd_file)), args.fps, args.scale, args.arkit, args.verbose, args.use_euler_rotation, args.optimize_textures, args.generate_texture_transform_texture, args.scale_texture)
  File "/usr/local/gltf2usd/Source/gltf2usd.py", line 832, in convert_to_usd
    usd = GLTF2USD(gltf_file=gltf_file, usd_file=temp_usd_file, fps=fps, scale=scale, verbose=verbose, use_euler_rotation=use_euler_rotation, optimize_textures=optimize_textures, generate_texture_transform_texture=generate_texture_transform_texture, scale_texture=scale_texture)
  File "/usr/local/gltf2usd/Source/gltf2usd.py", line 56, in __init__
    self.gltf_loader = GLTF2Loader(gltf_file, optimize_textures, generate_texture_transform_texture)
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2loader.py", line 117, in __init__
    self._initialize()
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2loader.py", line 125, in _initialize
    self._initialize_meshes()
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2loader.py", line 212, in _initialize_meshes
    mesh = Mesh(mesh_entry, i, self)
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2/Mesh.py", line 124, in __init__
    primitive = Primitive(primitive_entry, i, self, gltf_loader)
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2/Mesh.py", line 67, in __init__
    data = gltf_loader.get_data(accessor, accessor_index)
  File "/usr/local/gltf2usd/Source/_gltf2usd/gltf2loader.py", line 255, in get_data
    bufferview = self.json_data['bufferViews'][accessor['bufferView']]
KeyError: 'bufferView'

It seems this is due to the use of a draco mesh. I'm not sure if this should be fixed on your side. We will for now skip draco compression

Yeah Draco support would be quite heavy for Python. There is another conversion tool hosted by Google that lists support for Draco meshes https://github.com/google/usd_from_gltf/blob/master/README.md


Update onnx-mlir/utils/documentOps.py to include min and max version

We want to correctly capture the minimum and maximum supported operation version for onnx-mlir.

The PR is merged... so we can close this out. Thanks!


[Screen Readers - Power Toys - Power Rename]: In scan/item mode, screen reader focus navigation is not proper in both forward and backward direction.

[Power Toys Utility-Power Rename]
User Impact:
If any application does not expose information and mechanisms necessary to track focus, text insertion point, and selection attributes (e.g. whether text has been selected for copy/paste) of user interface components,  it would be difficult for the platforms that provide Accessibility APIs, and would in turn affect the screen reader dependent users to determine the focus.
Test Environment:
OS Version: 20221.1000
App Name: Power Toy Preview
App Version: v0.23.0
Screen Reader: Narrator
Reproe-Steps:

Launch Narrator.
Open Power Toys Settings App.
Open File explorer and select multiple files and open context menu and select Power rename. 'Power Rename' dialog will open.
Make narrator in scan mode using caps+Space keys.
Navigate in the 'Power Rename' dialog in scan/item mode in both forward and backward direction.
Observe the issue.

Note:
This issue is repro throughout the app.
Actual Result:
'In scan/item mode, screen reader focus navigation is not proper in both forward and backward direction.
Expected Result:
'In scan/item mode, screen reader focus navigation should be logical and proper in both forward and backward direction.
MAS Reference:
https://microsoft.sharepoint.com/:w:/r/teams/msenable/_layouts/15/WopiFrame.aspx?sourcedoc={b24201d1-2276-4471-8183-e1e5353031ad}&cid=b0e66f0d-86ce-4b30-8979-3dbd3189fda1
42_MAS4.2.10_Scan_item_mode_navigation_not_proper.zip
42_UIA_dialog.zip

Closing as fixed on V 0.76.2
https://github.com/microsoft/PowerToys/assets/87321414/175cd396-c2bf-49bb-a27c-bf53b929f453


Class starting with a digit are invalid

Your classes 12HourTimePickerChildDiv and 24HourTimePickerChildDiv are invalid.
To be valid the first character must be an underscore (_), a hyphen (-), or a letter(a–z).
This prevent the popup to be customized a little.

Yes, you are right. I even forgot about these classes. Thanks for raising this bug. I will implement this in the next release and then i will close this issue.

Checkout the v0.2.1 for this bug fix.


[Event Request] Codeunit 5632 FA Jnl . -Post Line "OnFAJnlPostLinePostReverseTypes" &OnFAJnlPostLinePostReverseType

Can we please get the following event?
procedure PostReverseType(FALedgEntry: Record "FA Ledger Entry")
    var
        EntryAmounts: array[4] of Decimal;
        i: Integer;
    begin
     ```
     //Event Request Begin
        OnFAJnlPostLinePostReverseType();
     //Event Request End

    CalculateDisposal.CalcReverseAmounts(FANo, DeprBookCode, EntryAmounts);
    FALedgEntry."FA Posting Category" := FALedgEntry."FA Posting Category"::" ";
    FALedgEntry."Automatic Entry" := true;
 ```
// Event Request Begin
    OnFAJnlPostLinePostReverseTypes();
// Event Request Begin

        for i := 1 to 4 do
            if EntryAmounts[i] <> 0 then begin
                FALedgEntry.Amount := EntryAmounts[i];
                FALedgEntry."FA Posting Type" := "FA Ledger Entry FA Posting Type".FromInteger(CalculateDisposal.SetReverseType(i));
                FAInsertLedgEntry.InsertFA(FALedgEntry);
                if FALedgEntry."G/L Entry No." > 0 then
                    FAInsertLedgEntry.InsertBalAcc(FALedgEntry);
            end;
    end;

 [IntegrationEvent(false, false)]
    local procedure OnFAJnlPostLinePostReverseType()
    begin
    end;

[IntegrationEvent(false, false)]
  local procedure OnFAJnlPostLinePostReverseTypes()
  begin
  end;

Hi,
Requested events do not have parameters. Could you please clarify what is the scenario of using those events?
Rgds,
Oleg.

Hi Oleg,
As per the below comment, I want to confirm you that we don't required any parameters in the publisher. We are using this publisher to call our customized global procedure.
Thanks,
SWGswar.

Hi SWGswar,
Could you please describe the scenario of using those events?
Is it enough to have only first event to call your global procedures?
Rgds,
Oleg.

Hi Oleg,
It's enough to have only the first event to call our global procedure, Could you please proceed further.
Thanks,
SWGswar.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


actions: RunCmd, error: margo has not started after 0.200s

In Sublime Text 3.1.1

I couldn't install GoSublime from Package Control: Install Package because it was absent.

I went to packages folder C:\Users\lanastasov\AppData\Roaming\Sublime Text 3\Packages and did git clone https://margo.sh/GoSublime now the plugin is displayed in Preferences -> Package Settings

When I press ctrl+b and then type go run main.go receive actions: RunCmd, error: margo has not started after 0.200s error

in the console i get

[13:57:57] margo: agent#072: starting
margo: agent#072: ['go', 'install', '-v', 'margo.sh']:
margo.sh/mgpf
# margo.sh/mgpf
........\GoSublime\src\margo.sh\mgpf\mgpf.go:39: p.Round undefined (type time.Duration has no field or method Round)
........\GoSublime\src\margo.sh\mgpf\mgpf.go:41: p.Round undefined (type time.Duration has no field or method Round)
........\GoSublime\src\margo.sh\mgpf\mgpf.go:43: p.Round undefined (type time.Duration has no field or method Round)
[13:58:00] margo: agent#072: ['margo.sh', 'start', 'margo.sublime', '-codec', 'msgpack']
[13:58:00] margo: agent#072: stopping
[13:58:00] margo: agent#072: Cannot start margo: Cannot find command margo.sh

Any Ideas how to fix GoSublime?

question 2, you can install gosublime from github

GoSublime r19.10.22-1 9o: type `help` for help and command documentation

[ `go version` | done ]  go version
  actions: RunCmd, error: margo has not started after 0.200s
[ `go help` | done ] go help
  actions: RunCmd, error: margo has not started after 0.200s
[ E:/WorkSpace/Go/ ] #

?????

GoSublime r19.10.22-1 9o: type `help` for help and command documentation

[ `go version` | done ]  go version
  actions: RunCmd, error: margo has not started after 0.200s
[ `go help` | done ] go help
  actions: RunCmd, error: margo has not started after 0.200s
[ E:/WorkSpace/Go/ ] #

?????

here is log
[14:21:44] margo: agent#024: ['E:\\Programming\\sublime_text_3211\\Data\\Packages\\GoSublime\\bin\\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[14:21:44] margo: agent#024: log: ``` go install -v -tags=margo margo.sh/cmd/margo.sublime ```
[14:21:44] margo: agent#024: log: press ` ctrl+. `,` ctrl+x ` to configure margo or check console for errors
[14:21:44] margo: agent#024: log: *Not* using margo extension: Error: cannot find package "margo" in any of:
[14:21:44] margo: agent#024: log: 	E:\Programming\Cmder\vendor\go\src\margo (from $GOROOT)
[14:21:44] margo: agent#024: log: 	E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo (from $GOPATH)
[14:21:44] margo: agent#024: log: 	E:\Programming\sublime_text_3211\Data\Packages\GoSublime\src\margo
[14:21:44] margo: agent#024: stopping
[14:21:44] margo: agent#024: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
[14:21:48] margo: agent#025: starting
  margo: agent#025: ['go', 'install', '-v', '-x', 'margo.sh']:
  WORK=C:\Users\Admin\AppData\Local\Temp\go-build747012658
[14:21:49] margo: agent#025: ['E:\Programming\sublime_text_3211\Data\Packages\GoSublime\bin\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[14:21:49] margo: agent#025: log: go install -v -tags=margo margo.sh/cmd/margo.sublime
[14:21:50] margo: agent#025: log: press ctrl+.,ctrl+x to configure margo or check console for errors
[14:21:50] margo: agent#025: log: Not using margo extension: Error: cannot find package "margo" in any of:
[14:21:50] margo: agent#025: log: 	E:\Programming\Cmder\vendor\go\src\margo (from $GOROOT)
[14:21:50] margo: agent#025: log: 	E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo (from $GOPATH)
[14:21:50] margo: agent#025: log: 	E:\Programming\sublime_text_3211\Data\Packages\GoSublime\src\margo
[14:21:50] margo: agent#025: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
[14:21:50] margo: agent#025: stopping
[14:21:51] margo: agent#026: starting
margo: agent#026: ['go', 'install', '-v', '-x', 'margo.sh']:
WORK=C:\Users\Admin\AppData\Local\Temp\go-build395460954
[14:21:52] margo: agent#026: ['E:\Programming\sublime_text_3211\Data\Packages\GoSublime\bin\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[14:21:52] margo: agent#026: log: go install -v -tags=margo margo.sh/cmd/margo.sublime
[14:21:52] margo: agent#026: log: press ctrl+.,ctrl+x to configure margo or check console for errors
[14:21:52] margo: agent#026: log: Not using margo extension: Error: cannot find package "margo" in any of:
[14:21:52] margo: agent#026: log: 	E:\Programming\Cmder\vendor\go\src\margo (from $GOROOT)
[14:21:52] margo: agent#026: log: 	E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo (from $GOPATH)
[14:21:52] margo: agent#026: log: 	E:\Programming\sublime_text_3211\Data\Packages\GoSublime\src\margo
[14:21:52] margo: agent#026: stopping
[14:21:52] margo: agent#026: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
<zipfile.ZipFile object at 0x0000023A405C0AC8>
<zipfile.ZipFile object at 0x0000023A405C0C88>
[14:22:48] margo: agent#027: starting
margo: agent#027: ['go', 'install', '-v', '-x', 'margo.sh']:
WORK=C:\Users\Admin\AppData\Local\Temp\go-build141230270
[14:22:49] margo: agent#027: ['E:\Programming\sublime_text_3211\Data\Packages\GoSublime\bin\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[14:22:49] margo: agent#027: log: go install -v -tags=margo margo.sh/cmd/margo.sublime
[14:22:49] margo: agent#027: log: press ctrl+.,ctrl+x to configure margo or check console for errors
[14:22:49] margo: agent#027: log: Not using margo extension: Error: cannot find package "margo" in any of:
[14:22:49] margo: agent#027: log: 	E:\Programming\Cmder\vendor\go\src\margo (from $GOROOT)
[14:22:49] margo: agent#027: log: 	E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo (from $GOPATH)
[14:22:49] margo: agent#027: stopping
[14:22:49] margo: agent#027: log: 	E:\Programming\sublime_text_3211\Data\Packages\GoSublime\src\margo
[14:22:49] margo: agent#027: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
[14:22:52] margo: agent#028: starting
margo: agent#028: ['go', 'install', '-v', '-x', 'margo.sh']:
WORK=C:\Users\Admin\AppData\Local\Temp\go-build685651526
[14:22:52] margo: agent#028: ['E:\Programming\sublime_text_3211\Data\Packages\GoSublime\bin\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[14:22:52] margo: agent#028: log: go install -v -tags=margo margo.sh/cmd/margo.sublime
[14:22:53] margo: agent#028: log: press ctrl+.,ctrl+x to configure margo or check console for errors
[14:22:53] margo: agent#028: log: Not using margo extension: Error: cannot find package "margo" in any of:
[14:22:53] margo: agent#028: log: 	E:\Programming\Cmder\vendor\go\src\margo (from $GOROOT)
[14:22:53] margo: agent#028: log: 	E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo (from $GOPATH)
[14:22:53] margo: agent#028: log: 	E:\Programming\sublime_text_3211\Data\Packages\GoSublime\src\margo
[14:22:53] margo: agent#028: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
[14:22:53] margo: agent#028: stopping

@jingyuexing
Did you follow the installation instructions at http://margo.sh/b/migrate?
Does it work if you press ctrl+.,ctl+x and save the file that it opens?

@jingyuexing
Did you follow the installation instructions at http://margo.sh/b/migrate?
Does it work if you press ctrl+.,ctl+x and save the file that it opens?

the frist question ,yes , i use git clone command,
the second question,  no ,it not work after press

@jingyuexing
What happens when you press those keys?
It should create and open a new file E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo\margo.go where you can configure margo. It should restart+recompile margo after you save that file. You might also need to restart Sublime Text.

@jingyuexing
What happens when you press those keys?
It should create and open a new file E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo\margo.go where you can configure margo. It should restart+recompile margo after you save that file. You might also need to restart Sublime Text.

[19:12:08] margo: agent#167: ['E:\\Programming\\sublime_text_3211\\Data\\Packages\\GoSublime\\bin\\margo.sh.exe', 'start', 'margo.sublime', '-codec', 'msgpack']
[19:12:09] margo: agent#167: log: Using margo extension: E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo
[19:12:09] margo: agent#167: log: ``` go install -v -tags="margo margo_extension" margo.sh/cmd/margo.sublime ```
[19:12:09] margo: agent#167: log: margo.sh/vendor/github.com/rogpeppe/go-internal/semver
[19:12:09] margo: agent#167: log: margo.sh/vendor/golang.org/x/sync/errgroup
[19:12:09] margo: agent#167: log: margo.sh/vendor/golang.org/x/net/context
[19:12:09] margo: agent#167: log: margo.sh/vendor/github.com/rogpeppe/go-internal/module
[19:12:09] margo: agent#167: log: margo.sh/format
[19:12:09] margo: agent#167: log: margo.sh/golang/goutil
[19:12:10] margo: agent#167: log: margo.sh/vendor/golang.org/x/tools/go/internal/gcimporter
[19:12:10] margo: agent#167: log: margo.sh/vendor/kuroku.io/margocode/lookdot
[19:12:10] margo: agent#167: log: margo.sh/vendor/kuroku.io/margocode/suggest
[19:12:10] margo: agent#167: log: margo.sh/vendor/github.com/rogpeppe/go-internal/modfile
[19:12:10] margo: agent#167: log: margo.sh/golang/cursor
[19:12:10] margo: agent#167: log: margo.sh/golang/gopkg
[19:12:10] margo: agent#167: log: margo.sh/golang/snippets
[19:12:10] margo: agent#167: log: margo.sh/vendor/golang.org/x/tools/go/gcexportdata
[19:12:10] margo: agent#167: log: margo.sh/golang/internal/srcimporter
[19:12:10] margo: agent#167: log: margo.sh/golang/internal/pkglst
[19:12:10] margo: agent#167: log: margo.sh/kimporter
[19:12:10] margo: agent#167: log: margo.sh/golang
[19:12:10] margo: agent#167: log: margo
[19:12:10] margo: agent#167: log: margo.sh/cmdpkg/margosublime
[19:12:10] margo: agent#167: log: margo.sh/cmd/margo.sublime
[19:12:12] margo: agent#167: log: ``` E:\Programming\sublime_text_3211\Data\Packages\GoSublime\bin\margo.sublime.exe -codec msgpack ```
[19:12:12] margo: agent#167: log: store.go:137: started
[19:12:12] margo: agent#167: log: margocode.go:332: VFS.Scan GOPATH ( E\P\s\D\P\User\GoSublime ): 0 packages preloaded in 2ms
[19:12:12] margo: agent#167: log: ``` go install margo.sh/vendor/golang.org/x/tools/cmd/guru ```
[19:12:13] margo: agent#167: log: margocode.go:332: VFS.Scan GOPATH ( E\P\s\D\Packages\GoSublime ): 73 packages preloaded in 281ms
[19:12:14] margo: agent#167: log: margocode.go:332: VFS.Scan GOROOT ( E\P\C\vendor\go ): 377 packages preloaded in 1.32s
[19:12:29] margo: agent#167: log: restart.go:153: Mg/Restart: exit status 1
[19:12:29] margo: agent#167: log: ``` go install -v -tags=margo margo.sh/cmd/margo.sublime ```
[19:12:29] margo: agent#167: log: press ` ctrl+. `,` ctrl+x ` to configure margo or check console for errors
[19:12:29] margo: agent#167: log: *Not* using margo extension: Error: E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime\src\margo\margo.go:2:1: expected 'package', found 'func'
[19:12:29] margo: agent#167: log: agent GOPATH is E:\Programming\sublime_text_3211\Data\Packages\User\GoSublime;E:\Programming\sublime_text_3211\Data\Packages\GoSublime
[19:12:29] margo: agent#167: log:

I know what happen now,because i not add GOBIN in environment，hahahaha,thanks your reply text


Volume examples with applications

Application examples should go in the documentation.

Include steps to discover other container dockerfiles and how to choose volumes properly
Find popular containers that are persistent
Show commands for running containers based on Volume points in Dockerfiles
Relevant volume options docker volume create --name test --opt=size=10 --opt=iops=1000 --opt=volumetype=io1

Some examples in https://github.com/emccode/rexray/pull/228


Add option to use sudo when cleaning up old releases

Q
A

Issue Type
Feature Request

Deployer Version
N/A

Local Machine OS
N/A

Remote Machine OS
N/A

Description
Add an option to use sudo when cleaning up old releases.
Clean up fails if the deployment user does not have the appropriate permissions to remove files created, for example, by Apache.

Can you use deploy:writable task for correcting permissions on folder while deploying?

I needed exactly the same feature, because runtime files were written by the webserver and clean as the normal ssh user didn't work because I had not enough rights. So I've rewritten the original clean task and added a sudo option like in the writable task:
task('cleanup:custom_project', function () {
    $releases = get('releases_list');
    $sudo = get('cleanup_use_sudo') ? 'sudo' : '';
    $keep = get('keep_releases');
    if ($keep === -1) {
        // Keep unlimited releases.
        return;
    }
    while ($keep - 1 > 0) {
        array_shift($releases);
        --$keep;
    }
    foreach ($releases as $release) {
        run(sprintf('%s rm -rf {{deploy_path}}/releases/%s', $sudo, $release));
    }
    run('cd {{deploy_path}} && if [ -e release ]; then rm release; fi');
    run('cd {{deploy_path}} && if [ -h release ]; then rm release; fi');
})->desc('Cleanup of older releases with sudo option');

Hope it's of any help. Maybe it can be added to the real cleanup task?

👍 same issue here, a sudo flag may be usefull.

Done!

For those unsure about what "done" means here, you can now specify this option to use sudo during the cleanup task:
set('cleanup_use_sudo', true);


Incorrect documentation for color option

I was trying to figure out how to draw a circle. The Draw documentation says that the dot instance method accepts a Color as an an argument (link). The Color documentation says that a symbol (like :red) or three value array (like [0.2, 0.3, 0.4]) can be passed as a color option. I could not get either of these to work.

expected Array of 4 elements (ArgumentError)

I was able to get it to work with a four value array, where alpha was set to 1.0. I'm not sure if this is a bug which does not allow these other short-cut options, or if this is an error in the documentation.

It appears that the way color is dealt with is inconsistent (in both motion-game and Cocos2d-x).  Sometimes a 3 value array is expected, sometimes a 4 value array is expected.  We could update the places that expect a 4 value array to also accept a 3 value array and default the alpha.  For those that expect a 3 value array, we could either accept a 4 value array and ignore the alpha or raise an error.  Any thoughts on the best approach?
Expects 3 value array
Node.color
Scene.color
Button.text_color
Layout.background_color
Expects 4 value array
Node.draw_rect/draw_dot/draw_triangle
Particle.start_color/end_color
Text.text_color


syzbot: enable UBSAN for linux

Tracking issue for enabling KUBSAN for linux on syzbot.
Moved from the discussion on a patch:
https://groups.google.com/g/syzkaller/c/Ex_s_aokx0A/m/AfzioXHEAwAJ
Things we know we need:

panic_on_warn support would be nice to land, but not critical.

If we want to selectively enable checks, then we need to wait for CONFIG_UBSAN_MISC support. @kees proposed the following config:

CONFIG_UBSAN=y
CONFIG_UBSAN_BOUNDS=y
# CONFIG_UBSAN_MISC is not set

But I've just tested latest linux-next with already supported:
CONFIG_UBSAN=y
CONFIG_UBSAN_SANITIZE_ALL=y
CONFIG_UBSAN_NO_ALIGNMENT=y

and I don't see any massive reports. So perhaps we enable everything (except for alignment?) right away? No need to wait for the patches then.
Update: with a bit more testing I found 3 invalid shift exponents in ext4, nfs and f2fs mount syscall.

Figure out what we do with bug titles.
Current we just say "UBSAN: undefined-behaviour":
https://github.com/google/syzkaller/blob/master/pkg/report/testdata/linux/report/453
UBSAN does not give a sane bug type identity, for bug type it produces lengthy prose with lots of moving parts:

index 6380 is out of range for type 's8 [257]'
index 8 is out of range for type 'char [8]'
signed integer overflow: -1531247276 cannot be represented in type 'int'

which is not suitable as part of bug identity.
We have 2 options: (1) leave it as-is (just say undefined-behaviour) or (2) rework UBSAN bug reporting to provide sane bug type.

UBSAN vs KASAN precedence.
It turns out UBSAN steals bugs from KASAN. E.g. this one:
https://github.com/google/syzkaller/blob/master/pkg/report/testdata/linux/report/454
is already detected and reported by KASAN. This is quite unfortunate b/c we may start reporting lots of dups if we enable UBSAN. I don't have good ideas so far.

FTR, UBSAN improvements from @kees were merged mainline, in particular:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ef065653e526a020d2a71549f413f14a830db799

I think we're ready to move forward, which some discussion on point 4:

panic_on_warn was added
bounds checking has been split out
bug titles are fixed, as you noted above
I'm not sure how to deal with collisions between UBSAN and KASAN, but it seems like UBSAN might trip earlier (which might make things easier to debug), but KASAN includes function names (which can make reports faster to analyze). The referenced example was "index 6380 is out of range for type 's8 [257]'". Was that more or less useful than what KASAN reported? (I can't find what KASAN said about this flaw...)

Regardless, I think the follow Kconfig would be the best first step:
CONFIG_UBSAN=y
# CONFIG_UBSAN_TRAP is not set
CONFIG_UBSAN_BOUNDS=y
# CONFIG_UBSAN_MISC is not set
CONFIG_UBSAN_SANITIZE_ALL=y
# CONFIG_UBSAN_ALIGNMENT is not set

This will get us bounds checking (which from my testing has a low false-positive rate). I'd like to split arithmetic overflow instrumentation out of CONFIG_UBSAN_MISC in the future, but it still has several other problems to solve first:
https://github.com/KSPP/linux/issues/26
https://github.com/KSPP/linux/issues/27

I'm not sure how to deal with collisions between UBSAN and KASAN, but it seems like UBSAN might trip earlier (which might make things easier to debug), but KASAN includes function names (which can make reports faster to analyze). The referenced example was "index 6380 is out of range for type 's8 [257]'". Was that more or less useful than what KASAN reported? (I can't find what KASAN said about this flaw...)

I think they are mostly equivalent. I was more concerned about getting KASAN reports for some bugs, and now start getting UBSAN reports for the same bugs, which will be reported as new bugs. Let's see if it will be a problem or not. Maybe it won't be a problem because we don't seem to detect massive amounts of these UBSAN reports.
Config and report parsing changes are now in (#2120). Let's see what happens :)

We now have 2 UBSAN bugs, everything seems to be working as intended.
@kees thanks for working on the kernel part.
If/when we have any new checks to enable, that can be done as in #2120.

One issue that come up:
syzbot tried to bisect this UBSAN bug:
https://syzkaller.appspot.com/bug?extid=9b64b619f10f19d19a7c
https://syzkaller.appspot.com/x/bisect.txt?x=1697348d900000
and then diverged into another bug as in the past we had more checks enabled and there were no way to selectively enable them, so it started hitting all possible minor ones.
Additionally UBSAN was build-broken for 4 releases v4.11-v4.15.
Potentially we need to restrict bisection of UBSAN bugs down to v5.3 only.


HDDS-7394. OM RPC FairCallQueue decay decision metrics list caller username in the metric

What changes were proposed in this pull request?
On the Prometheus endpoint for the OM, in the DecayRpcScheduler summary for users, the username is exposed in the metric name. It makes almost impossible to monitor these values as every time a new user shows up we need to register a new metrics name.
The metric name from org_apache_hadoop_ipc_decay_rpc_scheduler_volume becomes org_apache_hadoop_ipc_decay_rpc_scheduler_caller_hadoop_volume for a user with hadoop username.
The proposed solution is to filter the logs and remove the username from the metric and add it in a username tag.
This metric comes from hadoop-common-3.3.4.jar/DecayRpcScheduler and more specifically
Metrics2Util.NameValuePair entry = (Metrics2Util.NameValuePair)topNCallers.poll();
String topCaller = "Caller(" + entry.getName() + ")";
String topCallerVolume = topCaller + ".Volume";
String topCallerPriority = topCaller + ".Priority";
rb.addCounter(Interns.info(topCallerVolume, topCallerVolume), entry.getValue());

The name is in the format Caller(username).MetricType eg. Caller(hadoop).Volume. The username might exist in the metric name for a purpose. We don't want to change the way the metric works but the way it's presented. The cleanest way to deal with this seems to filter the metric in PrometheusMetricsSink. We are not making any functional changes but we are only filtering what is presented to the user.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-7394
How was this patch tested?
This patch was tested manually, with a docker cluster and the OM /prom endpoint.
To test it:
in compose/ozone add in docker-config
CORE-SITE.XML_ipc.9862.callqueue.impl=org.apache.hadoop.ipc.FairCallQueue
CORE-SITE.XML_ipc.9862.scheduler.impl=org.apache.hadoop.ipc.DecayRpcScheduler
CORE-SITE.XML_ipc.9862.scheduler.priority.levels=2
CORE-SITE.XML_ipc.9862.backoff.enable=true
CORE-SITE.XML_ipc.9862.faircallqueue.multiplexer.weights=99,1
CORE-SITE.XML_ipc.9862.decay-scheduler.thresholds=90
OZONE-SITE.XML_ozone.om.address=<IP_ADDRESS>:9862

then
$ export COMPOSE_FILE=docker-compose.yaml:monitoring.yaml
$ docker-compose up --scale datanode=3 -d
$ docker exec -it ozone_s3g_1 bash
bash-4.2$ export AWS_ACCESS_KEY=test AWS_SECRET_KEY=pass
bash-4.2$ ozone freon s3bg -t 1 -n 10

on your browser go to http://localhost:9874/prom and you should see
# TYPE org_apache_hadoop_ipc_decay_rpc_scheduler_priority counter
org_apache_hadoop_ipc_decay_rpc_scheduler_priority{context="ipc.9862",hostname="e32f2e3bddb9",username="hadoop"} 1
...
...
...
# TYPE org_apache_hadoop_ipc_decay_rpc_scheduler_volume counter
org_apache_hadoop_ipc_decay_rpc_scheduler_volume{context="ipc.9862",hostname="e32f2e3bddb9",username="hadoop"} 21

CC @duongkame

@kerneltime any updates on this?

Thanks @xBis7 for the patch.

The name is in the format Caller(username).MetricType eg. Caller(hadoop).Volume. The username might exist in the metric name for a purpose. We don't want to change the way the metric works but the way it's presented. The cleanest way to deal with this seems to filter the metric in PrometheusMetricsSink.

I still think this should rather be a change in hadoop-common. To keep it backward compatible, we may just introduce a new monitorable metric without changing the existing one. Capturing and modifying metrics in Ozone PrometheusMetricsSink looks pretty hacky, but if we have to do so, we'd rather copy the metrics to a new one without changing the original metric.

@xBis7 I understand the need, but I think there was intent in the original metric, and this comes from being able to diagnose which application/user is seeing what metric.
From: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/FairCallQueue.html
The implementation of RpcScheduler used with FairCallQueue by default is DecayRpcScheduler, which maintains a count of requests received for each user.

Is this for ofs use cases or s3 based access?

@kerneltime @duongkame Thanks for taking a look at the patch.

I think there was intent in the original metric, and this comes from being able to diagnose which application/user is seeing what metric.

I agree, that's why there are no functional changes. The metric that gets passed back and forth is exactly the same and we are only filtering how the endpoint presents it, like here with the RocksDb metrics.

Is this for ofs use cases or s3 based access?

It was used with S3 but it could be for both use cases.

I still think this should rather be a change in hadoop-common.

That was my first thought but it seems too complex for what we need. We don't want to change the metric or reduce the times it gets registered. We need the name to be consistent, so that it's easier to track with a wildcard search or something like that, but still want to be able to see the user for every registry.
If you think that's the best, I'm happy to move forward with it. We don't have to make the change to hadoop-commons but instead have a class in Ozone, overriding the one from the jar.


Large refactor

Refactor to break out api and check management into sub-packages. Add lots of testing (go test -v).
Note: the configuration and initialization of circonus-gometrics has changed. Most notable are that the configuration parameters for Api, Check, and Broker have their own configuration sections. (see example in readme)

Let's merge this and sort out issues (if there are any) going forward.


ObjectWriteStream sync to server without Close()

What component of google-cloud-cpp is this feature request for?
google/cloud/storage
Is your feature request related to a problem? Please describe.
In fact, I am implementing Google Cloud Storage API for Tensorflow. I would like to have an ObjectWriteStream which can sync to server without close the temporary std::basic_ostream<char>. I know that object of GCS can not be appended but I want to provide a feeling that is the same as on local filesystem to the users.
Describe the solution you'd like
Here is a psuedo-code
ObjectWriteStreamCustom f; // Create a temporary ostream
f.open("gs://bucket/object");
f << "abc"

// Create an object in cloud. Both temp ostream and object have the same content "abc"
f.sync();

// Append to ostream. ostream now is "abcdef"
f << "def"

// Overwrite the old object with a new object whose content is "abcdef"
f.sync();

// Curently, we can only save file to server by calling Close()`
f.close();

Describe alternatives you've considered
Till now, I create a struct with a fstream
typdef struct f {
    std::fstream file;
    gcs::Client cl;
}

Sync() {
    // The code to move data from fstream to basic_stream.
    // This code may cause error when fstream is a super large file (for example, dataset to train AI)
    // Because we need to allocate a large enough buffer to hold fstream and after that
    // we read the buffer into ObjectWriteStream, so we likely have 3 copy of a dataset.
    // if writer support Sync(). I won't even need the struct nor the temporary fstream.
    // In addition, transfering data between streams in c++ is usually not a good pratice
    int size = f.file.tellp();
    char* buffer = malloc(size);
    auto writer = f.cl.WriteObject(bucket, object);
    writer.read(buffer, size);
    writer.Close();
}

I thought TensorFlow already had GCS support?
In any case, what you want to do, sounds hard.  Let me make sure sure I follow:

You want folks to be able to read a section of a GCS object?  That is possible today using the gcs::Range() option. But maybe you want more?

Or you want folks to be able to read a section of a GCS object and then skip or rewind to a different section?  That is not possible today, you could try to implement this using a library like you describe and maybe some sort of cache.  That is, when the application reads bytes [1, 1000] you save them before returning them to the rest of the application.  When they ask for bytes [500, 1500] you can just read the bytes that are missing ([1001, 1500]) and return the union.

If you want or need to offer an iostream API then I would start by looking at the Boost.IOStream library, it may have functionality to compose iostream objects (not sure, but worth looking before reinventing that wheel)

Do you want to be able to write in the middle of a GCS object?  That will be very hard. You basically would be trying to implement POSIX semantics over an  immutable object store.

You could avoid storing large files and just store "chunks" that then you reassemble in whatever API you expose, so rewriting in the middle does not change that much data.
Or implement some kind of log-based filesystem.
Or stream all the bytes before the change to a temporary object, then stream the changed bytes to a second temporary object, and then stream the trailing bytes to a third temporary object and finally use gcs::Client::Compose to join them.  That would be very slow too.

Sorry, I have misexpressed myself about what I needs.
https://github.com/googleapis/google-cloud-cpp/blob/1b8f971101ee7547c9346bca78694d9176844c06/google/cloud/storage/object_stream.h#L240-L254
Here is how Close() works with ObjectWriteStream. I would like to have a Sync() which works extractly the same as Close()works except close the stream. Instead, it sets a bool variable, let's say bool need_sync to false. Each call to Close() and Sync() will check that variable to see that if it needs sync or not. In case it need sync, we will overwrite the object on cloud with a new object, not append to the object on cloud. Each function that modify the stream will set need_sync to true

DownloadToFile and UploadFile seem to be what I need. I'm sorry for not reading carefully the documents. Thank you for your help

No problems.  You may want to take a look at storage::ParallelUploadFile() if you are uploading particularly large objects:
https://googleapis.dev/cpp/google-cloud-storage/latest/namespacegoogle_1_1cloud_1_1storage_1_1v1.html#aa7c3ff7d43779f608cb2e814cb4386f1

Thank you. I have to deal with particularly large objects so ObjectReadStream and ObjectWriteStream which are stored in memory aren't what I need. I will take a look at storage::ParallelUploadFile() as well.


Slabs are rendered black in the generated map

Black parts in the picture above are actually stone slabs and stone brick slabs, but the texture seems missing in the generated map.
Some logs may have something to do with the bug:
Missing Texture(2): 15 80 15 minecraft:stone_block_slab:System.Collections.Generic.Dictionary`2[System.String,System.Object] (8)

Papyrus CS ver 0.5.2
Minecraft ver 1.19.41


Exploration for Beamline Rendering

We will build a prototype using Dear ImGui, GLFW, and Vulkan to test the potential of rendering beamlines for design work.

Update on what needs to be done:

[ ] Reading in the the beamline and creating triangles from a plane mirror works, but only one is being rendered correctly atm (likely due to the other facing the wrong side)
[ ] Rendering rays instead of just random lines
[ ] Fixing swapchain recreation bug
[ ] It would be nice to have better controls for looking at the scene (mouse/keyboard interaction)


Trying to run a Discord bot with Sanic at the same time

I got this code:
bota = my_bot_class().run(token)
botb = asyncio.ensure_future(bota)

sanica = app.create_server(host="<IP_ADDRESS>", port=8765)
sanicb = asyncio.ensure_future(sanica)

loop.run_forever()

But when I run it, it runs the Discord bot first, and Sanic doesn't start. Is there a way to run Sanic and a bot at the same time?

It is hard to know without seeing your bit code.
Just curious, why are you trying to run them on the same loop? Generally this is not advisable to have two services in the same process using the same loop.

It is hard to know without seeing your bot code.
Here it is:

import discord, os, asyncio
from sanic import Sanic
from discord.ext import commands
from sanic.response import json
from dotenv import load_dotenv
load_dotenv()
token = os.getenv("TOKEN")
loop = asyncio.get_event_loop()
app = Sanic(__name__)

@app.route("/bot/get_data")
async def get_tdb_data(request):
    return json({
        "online": True
    })

class my_bot_class(commands.Bot):
    def __init__(self):
        super().__init__(command_prefix=".")

    async def on_ready(self):
        print("Logged in.")

# the code i mentioned before is here...

Just curious, why are you trying to run them on the same loop?
Because I want to make something for the bot status, online or offline

Can you start your bot in a listener?
https://sanic.readthedocs.io/en/latest/sanic/middleware.html#listeners

Okay, I tried with listeners, but, when I run the code, it starts up Sanic, but it, instead of "Starting worker" that starts server, it shows the "Logged in" thing that says the bot is logged in. CTRL+C just exits without starting server.
This is the code I used:
@app.listener('before_server_start')
async def bot_startup(app, loop):
    print("Starting up bot...")
    await my_bot_class().start(token)

@app.listener('after_server_start')
async def notify_server_started(app, loop):
    print('Bot successfully started!')

@app.listener('before_server_stop')
async def notify_server_stopping(app, loop):
    print('Shutting down!')

@app.listener('after_server_stop')
async def bot_logout(app, loop):
    await my_bot_class().logout()

Your bot doesn't look like it needs access to the loop. In that case, I'd suggest running it from a listener and using app.add_task.

@Vincysuper07 you'd likely be better off completely decoupling these two functions and using a k/v store that is read by Sanic and written by the bot in its own loop. The way you have it now, they're tightly coupled - which means that if the bot is offline, the Sanic server probably is too; though not necessarily guaranteed. It's going to add lots of complexity to manage the state of both your web service and your bot service in the same loop.

I agree with @sjsadowski. If your plan is to put this anywhere near pofuctonw, they should be on separate processes.

I managed to fix with this:
@app.listener('before_server_start')
async def bot_startup(app, loop):
    loop.create_task(my_bot_class().start(token))

@app.listener('before_server_stop')
async def notify_server_stopping(app, loop):
    await my_bot_class().logout()

This is perfectly working as I wanted. Thanks everyone for help!

Just be sure not to run Sanic with multiple workers or you'll get multiple bots too. Autoreloader should work but you might not want your bot restarting and losing connection every time the code is changed, so you probably don't want to use that either.

Also, there is a convenience wrapper app.add_task for where the loop is not available.


add transaction support

Evaluate to what extent MongoDB's transaction support can be implemented in Django.

I did a little investigation in https://github.com/timgraham/django-mongodb/tree/transactions. I ran into a database error: "Transaction numbers are only allowed on a replica set member or mongos". At this point, I think there are higher priority things to work on.


Upgrading to 3.4 breaks ref behaviour in highcharts-vue

Vue version
3.4.5
Link to minimal reproduction
https://stackblitz.com/edit/vitejs-vite-k8kyr7?file=src%2FApp.vue
Steps to reproduce

Launch the app and open the devtools
The error will be visible in the console
Downgrade to Vue 3.3 to confirm the code works there

What is expected?
The graph should be displayed
What is actually happening?
The app throws errors and warnings:
 [Vue warn]: Missing ref owner context. ref cannot be used on hoisted vnodes. A vnode with ref must be created inside the render function.

[Vue warn]: Unhandled error during execution of mounted hook

Uncaught TypeError: can't access property "nodeName", t3 is null

System Info
No response
Any additional comments?
In Vue 3.4, the ref here is in its initial state (null) and does not contain a ref to the container: https://github.com/highcharts/highcharts-vue/blob/master/src/component.js#L80
In Vue 3.3 the ref correctly points to the <div>. When trying with a production build, the app still doesn't work, but a log there will  eventually show the ref pointing to a <div>. This means somehow the order of execution of things has changed between the call to render() and the call to the onMounted() callback.
Also reported on the highchart-vue repository: https://github.com/highcharts/highcharts-vue/issues/256

Downgrading to v3.4.4 works fine.

this is related to oruga, not vue.
oruga not ready

Oruga is not in the dependencies, how can it be related to that?

This was reported by a jsPlumb Toolkit licensee also, in relation to one of our demonstration apps:
<jsplumb-toolkit ref="toolkitComponent"
                             surface-id="surfaceId"
                             :render-params="this.renderParams()"
                             :view="this.viewParams()"
                             :toolkit-params="this.toolkitParams()"
                             url="copyright.json">
            </jsplumb-toolkit>

The ref here causes the error "Missing ref owner context. ref cannot be used on hoisted vnodes. A vnode with ref must be created inside the render function" to be logged.
https://github.com/jsplumb/jsplumbtoolkit-applications/blob/main/apps/flowchart-builder/vue3/src/components/Flowchart.vue#L246


javascript, stylesheet and schema Liquid tag syntax highlighting

The schema tag, javascript tag and stylesheet tag could get the appropriate highlighting. Would this be possible? I think it should just piggy-back on how <script> and <style> highlighting works.
{% javascript %}
  // highlight this code as Javascript
{% endjavascript %}

{% stylesheet %}
  // highlight this code as CSS (or SCSS if you can't differentiate the next one)
{% endstylesheet %}

{% stylesheet 'scss' %}
  // highlight this code as SCSS
{% endstylesheet %}

{% schema %}
  // highlight this code as JSON or Javascript
{% endschema %}

@ADTC Solved this in https://github.com/GingerBear/vscode-liquid/pull/15


fix: remove param 'chapterName'; view quran type(reading or tr…

…anslation) changed to param

:tada: This PR is included in version 1.2.2-beta.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.3.0-beta.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


can.List should implement a "reduce" method

Instead of adding them all individually we should probably discuss adding Underscore/Lodash collection methods more.

Well I added a simple implementation and some tests.  If we decided to go a different route, that is completely fine. Just happen to need in a project so I made an implementation since we had to use lodash instead.

Closing for: https://github.com/canjs/can-define/issues/13


[TS] Add types for "req" and "res"

This PR adds types for req and req objects.
Usage:
const ApiGateway = require("moleculer-web");
const { IncomingRequest, GatewayResponse } = require("moleculer-web");

Related issue: moleculerjs/moleculer-template-project-typescript#20

Pull Request Test Coverage Report for Build 482

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.84%

Totals

Change from base Build 471:
0.0%

Covered Lines:
665

Relevant Lines:
687

💛  - Coveralls

Pull Request Test Coverage Report for Build 482

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.84%

Totals

Change from base Build 471:
0.0%

Covered Lines:
665

Relevant Lines:
687

💛  - Coveralls


Please make sure the workflow.connect statements are left out by black. Otherwise, old-school nipype devs will cry (I'm the first of those).

Done.


WIP fixing invoker ignoring failure in project build.

PR job #1 was: UNSTABLE
Possible explanation: This should be test failures

Reproducer
build-chain build full_downstream  -f 'https://raw.githubusercontent.com/${AUTHOR:apache}/incubator-kie-kogito-pipelines/${BRANCH:main}/.ci/buildchain-config-pr-cdb.yaml' -o 'bc' -p apache/incubator-kie-kogito-runtimes -u https://github.com/apache/incubator-kie-kogito-runtimes/pull/3257 --skipParallelCheckout
NOTE: To install the build-chain tool, please refer to https://github.com/kiegroup/github-action-build-chain#local-execution

Please look here: https://ci-builds.apache.org/job/KIE/job/kogito/job/main/job/pullrequest_jobs/job/kogito-runtimes-pr/job/PR-3257/1/display/redirect
Test results:

PASSED: 3382
FAILED: 1

Those are the test failures:

maven.invoker.it.integration-tests-springboot-kafka-it.integration-tests-springboot-kafka-it
The build exited with code 1. See /home/jenkins/workspace/_jobs_kogito-runtimes-pr_PR-3257/kogito-pipelines/bc/apache_incubator-kie-kogito-runtimes/springboot/integration-tests/target/it/integration-tests-springboot-kafka-it/build.log for details.

Closing as replaced by #3258


Password confirmation not working

Hi, we are trying to add the password confirmation to the schema, but it doesn't seem like the two password fields talk to each other. We currently have it like following:
"validations" : [{
"type" : "isEmail"
}]
},{
"questionId" : "password",
"question" : "Password",
"input" : {
"type" : "passwordInput",
"placeholder" : "8 characters minimum",
"required" : true
},
// Make sure password is entered
"validations" : [{
"type" : "isLength",
"params" : [1]
}]
},{
"questionId" : "passwordConfirm",
"question" : "Password Confirm",
"input" : {
"type" : "passwordInput",
"placeholder" : "confirm password"
},
"validations" :[{
"type" : "equals",
"params" : ["12345678"]
//"matches" : "({passwordConfirm}, {password} )"
//"message" : "Confirm password must match password field"
}]
Thanks!

Hi there.
This may be due to the Winterfell version you have installed via NPM. If your version is up to date (currently 1.0.4) it most likely does not have this feature yet. However the feature is in the master branch. I will be pushing out a new update sometime soon, meanwhile you could download a copy of the master branch and use that version of Winterfell?
Thanks.

New version published.

This worked, thank you!


Rewrote the contributors page. Improved sorting.

I refactored how the main contributor institutes are listed on the About page. Now uses YAML and looks a lot nicer. Includes colour logos as well as white (will update later to cycle the logos on the homepage).
Sorting pipelines can now be done with stars. Double click a sorting type to get the reverse.

Do you want us to have a look once you#re done or are you intending to work on this more?


updating file gives wrong size

When updating a file of approximately 20 MB or more, the size shown for the v2 after update is wrong.

Before update:

After update:

Tested downloading the v2 file. It only downloads a partial file that is the incorrect size shown in v2.


remving region input variable mentions from readme

what

Updatind docs

why

Removing input variable region mention on the docs

/rebuild-readme

/test all

/rebuild readme

/test all


BZ-1270690 - Task list: Active Tab shows completed tasks

Reduce the number of executed queries while the tabs are regenerated

@nmirasch  The fact that this line of code had to be added to five different places is a bit disconcerting. Not sure that I missed something, but the implementation of the method restoreTabs() looks identical in all five DataSet*ViewImpl classes. Wouldn't it be good to pull it up to a superclass? What do you think?
Also it would be nice to have some tests :-)

You are right the same code in every section that manages tabs. We had a cleaning task of classes that are not exposed and sure includes code optimization.
We also, planned add the UI test after this period of BZ resolving.

Neus, I built jbpm-console-ng with this PR locally and the issue seems to be gone. I'm no longer able to reproduce the issue. I also executed existing task list tests and this doesn't seem to break anything. I vote +1 for merging this PR to 6.3.x

merged in 6.3.x


Add autoware_lint_common for each package's dependencies

Use autoware_lint_common instead of amant_lint_common to format in clang-format/Black.

Actually, amant_lint_common contains a few linters such as copyright, .cmake, etc.
https://github.com/tier4/tier4_ament_lint/blob/cae03f86e3795e17a225a4f94331acffdfee4587/autoware_lint_common/package.xml
This is because we use pre-commit for formatting.

Merged https://github.com/autowarefoundation/autoware_common/pull/2


Do our openssl related docs need updating?

https://github.com/rusoto/rusoto/blob/master/CONTRIBUTING.md has steps for setting up openssl.  This was when we depended on hyper's transitive dependency on https://github.com/sfackler/rust-openssl .  Since we're using hyper-native-tls and reqwest that has been abstracted to https://github.com/sfackler/rust-native-tls .  Work was done in https://github.com/rusoto/rusoto/pull/524 .
We should see if we can remove those steps if they aren't required.

Data point: on Windows, the latest release of Rusoto compiles and Just Works.  The previous release did not, failing with being unable to find openssl.

Been quiet here, closing soon unless someone has input.


Feature request: Allow apps access to device location

It would be cool to be able to selectively grant access to device APIs. Obviously it would be disabled by default. The obvious one is surely location.
This would be useful for map sites but also others, which are increasingly providing buttons to "Find one near me" and so on. These work in a browser once the location permission has been given. It would be great for the WebApps sandbox to be able to do this too.
Thanks for your work on this excellent piece of software.

Arrgh, just saw someone else proposed exactly the the same. Delete this or merge it or whatever. Sorry.


update readme

improvment for readme

Please @ngdream , it's better to use another branch than the main one to make a PR. I want to test if the Greetings Action works, please update your main branch with the recent changes I just made on osscameroon main branch.

Keep the table of content, great work 💯 , I am a real fan of Built with love & Built by developers, we'll keep that. Once everything is ready, we'll merge this week end.

Use "CONTRIBUTING " in capital letters, if not we get a 404 as you can see here, just test by yourself:
https://github.com/ngdream/js-generator/blob/main/CONTRIBUTING.md
https://github.com/ngdream/js-generator/blob/main/contributing.md
We are happy with every contribution,please have a look to our [contribution guidelines](contributing.md)


ci: add release new version action

Description

Add release new version action

General Checklist

[ ] Added new tests to cover change, if needed
[ ] Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
[ ] Documentation update for the change if required
[x] PR title conforms to conventional commit style
[ ] If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report
Patch and project coverage have no change.

Comparison is base (6b3b527) 99.65% compared to head (6762a45) 99.65%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #12   +/-   ##
=======================================
  Coverage   99.65%   99.65%
=======================================
  Files          24       24
  Lines         882      882
  Branches      151      151
=======================================
  Hits          879      879
  Misses          3        3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Unable to find a specification for MetalPerformanceShadersProxy

When running pod install with pod 'MetalBender', :git => 'https://github.com/xmartlabs/Bender.git' I get the following error in the terminal.

[!] Unable to find a specification for MetalPerformanceShadersProxy

Looking at the cocoa pods specs it seams the MetalPerformanceShadersProxy were removed a few hours ago. Did I miss something?

Cocoa Pods 1.3.1, Xcode 8.3.3, ios 10 with iphone 6

Yeah, we deleted yesterday to upload it as MetalProxy, as it involves not only MetalPerformanceShaders :sweat_smile:
Today we will fix it, but as a workaround, you can just fork it, make MetalPerformanceShadersProxy in the podspec file point to the github repo (https://github.com/xmartlabs/MetalPerformanceShadersProxy) and then use that as a pod.

Sounds good! Tried to simply override the location of MetalPerformanceShadersProxy now which worked:
  pod 'MetalPerformanceShadersProxy', :git => 'https://github.com/xmartlabs/MetalPerformanceShadersProxy.git'
  pod 'MetalBender', :git => 'https://github.com/xmartlabs/Bender.git'

Good to know! Btw, we are planning to release a version of Bender to CocoaPods today :smile:

Ok, fixed by #66
Btw, MetalPerformanceShadersProxy still keeps its name.


Calendar improvment

This Pull request contains three calendar improvements:

Support for multiple calendars
Make update of table smoother and update only rows which change
Show events currently running with endtime

Very nice update! Just one thing: since I will be using just one calendar, I don't want an icon. If I leave the icon blank, the calendar isn't aligned anymore. Could you modify this? - Thanks!

There is an option I introduced to disable the icon. It's false if unset
and in the config.js I pushed it's set to true.
The behavior of an 'unaligned' empty icon is on purpose, because an
other calendars might have an Icon set.
Hope that's ok with you.
Am Di, 26. Jan 2016, um 15:43, schrieb Michael Teeuw:

Very nice update! Just one thing: since I will be using just one
calendar, I don't want an icon. If I leave the icon blank, the calendar
isn't aligned anymore. Could you modify this? - Thanks!

Reply to this email directly or view it on GitHub:
https://github.com/MichMich/MagicMirror/pull/56#issuecomment-175052054

My bad! I totally missed that disable function. Nice work!


Improve heatmap plotting

Extra parameter added to heatmap generation to allow for handling of
values below cbar_min

The NaN checking requires an extra import (of numpy) and this assumes Z is a numpy array, but it should otherwise do as discussed!


Draft: Liquid Etchings

This is a draft PR. Although the code builds, it is not yet functional. Tests are failing and some stuff was just commented out to make the build work. More fancy variable names need to be brainstormed. The purpose of this PR is to provide an idea of how a lightweight AMM and liquidity bootstrapping implementation could look for runes.
The Idea
We cannot swap on-chain against native BTC (yet). Therefore, this PR only allows swapping runes against other runes. However, it is possible to create some WBTC rune with a centralized treasury. This PR also assumes that the most "entertaining" aspect for degens is the ability to etch runes in a mode where liquidity is bootstrapped, and the etched rune can be traded on-chain immediately. To achieve this, the etcher of that rune must specify a base rune to mint the new rune. The base rune can be any rune, such as UNCOMMON•GOODS or DOG•GO•TO•THE•MOON (random examples), assuming these have good off-chain liquidity in the long term. Each mint then requires a certain amount of base runes, all of which are locked in a liquidity pool. Once the rune is fully minted, a seed (a certain amount of the etched rune) is added to the pool along with the base runes.
Problem: MEV
This PR does not yet include MEV prevention, but here are some ideas that might eliminate MEV:

The output of a swap may not be used to swap again in the same block to prevent sandwiching.
Commit/reveal-based swaps for large trades to prevent frontrunning. Miners could still censor reveals but setting a narrow slippage can make that unprofitable.

Why?
It would be beneficial if runes could contribute more to miner fees as the subsidy decreases. Rune mints have slowed down since launch. Bootstrapped on-chain liquidity could bring additional transaction volume to the network and could also make us more independent from CEX-listings for liquidity.
@casey @raphjaph If you are open to integrating this feature, my friends and I are happy to fully implement it.

Sorry to close this, but anything that introduces MEV is a non-starter.

Also, I would prefer to see swaps implemented with existing bitcoin primitives, which should eventually be sufficient, not rune-specific primitives.

Sorry to close this, but anything that introduces MEV is a non-starter.

@casey I opened the PR because we can make swaps work without introducing MEV. Without generic programmability prevention of MEV will be much easier than on Ethereum.

Also, I would prefer to see swaps implemented with existing bitcoin primitives, which should eventually be sufficient, not rune-specific primitives.

I also don't view this PR as a competition to concepts like light pools but more as a fun addition to rune etchings. If runes have a proper AMM, the chances are actually higher that no other meta protocol will introduce an AMM with less MEV protection.

What MEV were you referring to when you wrote "This PR does not yet include MEV prevention"?

Can you also describe what this PR does so I can understand it without needing to refer to the code?

What MEV were you referring to when you wrote "This PR does not yet include MEV prevention"?

Biggest MEV potential would be sandwiching. User submits a swap with a 3% slippage, miner puts a buy right before the swap and a sell right after to extract the max potential slippage. No risk for the miner because they can buy and sell in the same block. To avoid that (not included in the code yet), we can design it so that runes on a swap output can not be swapped within the same pool again in the same block (maybe even 2 blocks). This way, if a miner wants to extract the slippage, they would need to hold different amounts of runes on different outputs. However, this comes with the risk of holding certain runes and is not really MEV but market making.

Can you also describe what this PR does so I can understand it without needing to refer to the code?

The PR adds a new etch mode to runes where you can mint runes by spending existing runes. Once the mint cap is reached, both the spent runes plus a configured amount of the newly etched rune (seed) is put into a pool which people can trade against. Therefore a new swap flag is introduced.
Some stuff you would find in an Ethereum style AMM has been left out to keep the upgrade simple. For example, you can’t add or withdraw liquidity (both runes) from pools. The liquidity is not owned by anyone.

I have a few thoughts:

Most wrapped BTC implementations are fully centralized. I don't want to encourage their development or use.
I don't think using runes on both sides of a trading pair is useful. BTC should be on one side of every trading pair to avoid fragmentation.
There is MEV introduced by this implementation, and avoiding it requires a bunch of weird restrictions.

I think the proper direction for experimenting with improving trading is with base-layer bitcoin, which, although limited, is programmable.


Document ROS Iron discovery options

Documents new discovery options in ROS Iron
https://github.com/ros2/ros2/issues/1359

https://github.com/ros2/rcl/issues/1070 as follow-up to remove localhost_only for J-turtle.

OK, with the work that @fujitatomoya did, this is now ready to get in.  I've rebased this and also revamped it to be bullet points to be easier to read.  I also fixed up all of the highlighted areas.
I'm going to approve this, since I think this is now ready to go.  But since I've done substantial work here, I'd like to get another approval before I merge.


add esp-now support: add bindings for esp_now

The main PR is in esp-idf-svc - this just adds header to the bindings so that they get generated
related: https://github.com/esp-rs/esp-idf-sys/issues/70

Hmm, come your PR in esp-idf-svc compiles without these changes?

@ivmarkov pushed a change so that esp-now is part of wifi component without additional flags

Thanks, merged!


Change babel strategy (low priority: for HUGE production apps only)

Right now babel is used to write a function call that will create a normalization schema. This is useful for avoiding the large load of the clientSchema, but these function calls can be 10x bigger than the fragments they replace.
A very basic app has a clientSchema that is 1.5Kb gzipped (removed description, etc. fields from the introspectionQuery). Even if a production app were 10x bigger, it'd still be fine to ignore.
HOWEVER, if it's facebook-sized, a 100Kb clientSchema could exist. In this scenario, I could use babel + webpack.
Step 1: for each type in the clientSchema, write it to a .json where the name is type.name.value.
Step 2: In babel, parse the queryString into an AST. Make a list of all the types used. Map those to require statements where each one add itself to a prop of a types constant.
Step 3: Reference the types constant directly.
This is superior to the current (relay) strategy in a few ways:

it eliminates bloat of each babelfied TaggedTemplateString
it eventually loads the entire clientSchema onto the client, loading each type only once
by loading per-require, if the client needs the whole clientSchema (eg graphiql), it can request all requires and only receive the diff, reducing transmission
webpack can handle where to split the requires per page

One thing that facebook does internally is replace the queryString with a short UID on the client. this eliminates over-the-wire transfer of a huge, declarative string. However, Cashay needs that string so it can parse the AST.
We could still have the queryString on the client & just send a UID for frequent queries, like how we do for subscriptions.
However, if we wanted to completely remove the queryString from the client, then we'd need the AST, which is what relay does.
More thought needed....

Another possibility is combining the requestAST with the schema in babel.
Doing so provides, a single object to traverse instead of walking 2 at the same time. Computationally, this is a good thing.
The bad part is that if done naively, it'll be a larger payload, since we'll be including more info than necessary.
Ideally, we follow the original plan of sending the useful parts of the queries down to the client via webpack.
Then, when replacing the queryString with a reqAST, add in requires that will grab the necessary info from those useful schema parts.
eg Babel compilation step:
reqAST = parse(queryString);
reqAST.selectionSet.selections[0]._expectedArgs = require('./schemaParts/getPosts.json').args
reqAST.selectionSet.selections[0]._expectedFields = require('./schemaParts/getPosts.json').fields

@mattkrick I think I am bumping against this logic.  Getting Cannot read property 'selections' of null.  I'm not sure where to start digging here; can you point me in the right direction?

Created #131


Improvement: -latomic detection

some systems need to link the atomic lib in order to use std::atomic functionality. I did some sort of detection for simpleai at https://github.com/mgerhardy/simpleai/commit/f811dd50179d4b7b1a62228a02968844eaacbae4
You might want to consider adding this to fips, too - as you are using c++11 and threads for it anyway.

if (FIPS_LINUX)
    check_function_exists(__atomic_fetch_add_4 HAVE___ATOMIC_FETCH_ADD_4)
    if (NOT HAVE___ATOMIC_FETCH_ADD_4)
        check_library_exists(atomic __atomic_fetch_add_4 "" HAVE_LIBATOMIC)
        if (HAVE_LIBATOMIC)
            fips_deps(atomic)
        endif()
    endif()
endif()

Hmm interesting, didn't know that. What system(s) requires this?

At least my system which is a ubuntu 14.10. I will check the relevant lib
versions once i am on the machine again - if needed.
Am 12.02.2015 17:40 schrieb "Andre Weissflog"<EMAIL_ADDRESS>
Hmm interesting, didn't know that. What system(s) requires this?
—
Reply to this email directly or view it on GitHub
https://github.com/floooh/fips/issues/54#issuecomment-74104198.

Just digged a little bit deeper: They've moved atomics implementation since gcc 4.8 into libatomic.a - and this must be linked whenever you use atomics.
Do you think this patch can make it into fips? Or should I change something?

Well, in fact gcc < 4.8 didn't support atomics at all... - just to clarify.

I'd like to figure out first why Oryol is linking at all on Linux with gcc4.8 without explicitly linking libatomic... :) For instance I'm using std::atomic_exchange_explicit() and some operators (++, ...).
I'm on Linux Mint 17.1, based on Ubuntu 14.04.1 LTS
gcc -- version: gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2

So... it looks like it worked in older Ubuntu and/or gcc versions but no longer in newer ones? What's your exact gcc version?

gcc (Ubuntu 4.9.1-16ubuntu6) 4.9.1
lsb_release  -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.10
Release:	14.10
Codename:	utopic

btw. oryol links for me, too - which is very weird. Maybe not all symbols are put into the atomic lib.

in simpleai this works:
cd /home/mgerhardy/dev/simpleai/src/run && /usr/bin/cmake -E cmake_link_script CMakeFiles/simpleai-run.dir/link.txt --verbose=1
/usr/bin/c++    -std=c++11 -Wall -Wextra -fno-exceptions   CMakeFiles/simpleai-run.dir/GameEntity.cpp.o CMakeFiles/simpleai-run.dir/Main.cpp.o  -o simpleai-run  -L/home/mgerhardy/dev/simpleai/src/ai -rdynamic ../ai/libsimpleai.a -lpthread -latomic ../libs/lua/liblua.a -ldl -Wl,-rpath,/home/mgerhardy/dev/simpleai/src/ai
while this doesn't:
cd /home/mgerhardy/dev/simpleai/src/run && /usr/bin/cmake -E cmake_link_script CMakeFiles/simpleai-run.dir/link.txt --verbose=1
/usr/bin/c++    -std=c++11 -Wall -Wextra -fno-exceptions   CMakeFiles/simpleai-run.dir/GameEntity.cpp.o CMakeFiles/simpleai-run.dir/Main.cpp.o  -o simpleai-run  -L/home/mgerhardy/dev/simpleai/src/ai -rdynamic ../ai/libsimpleai.a -lpthread -latomic ../libs/lua/liblua.a -ldl -Wl,-rpath,/home/mgerhardy/dev/simpleai/src/ai
the only difference is -latomic
CMakeFiles/simpleai-run.dir/GameEntity.cpp.o: In Funktion std::atomic<ai::Vector3f>::store(ai::Vector3f, std::memory_order)': GameEntity.cpp:(.text._ZNSt6atomicIN2ai8Vector3fEE5storeES1_St12memory_order[_ZNSt6atomicIN2ai8Vector3fEE5storeES1_St12memory_order]+0x27): Nicht definierter Verweis auf __atomic_store'
CMakeFiles/simpleai-run.dir/GameEntity.cpp.o: In Funktion std::atomic<ai::Vector3f>::load(std::memory_order) const': GameEntity.cpp:(.text._ZNKSt6atomicIN2ai8Vector3fEE4loadESt12memory_order[_ZNKSt6atomicIN2ai8Vector3fEE4loadESt12memory_order]+0x42): Nicht definierter Verweis auf __atomic_load'
collect2: error: ld returned 1 exit status

Strange. Would you mind providing a pull request for fips? I think this should go into cmake-toolchains/linux.toolchain.cmake. There's already a couple of standard libs defined (-lpthread and -lrt),
additionally to your detection whether atomic exists, you could also limit this to gcc with an if (FIPS_GCC).

created a pull request.
while we are on it - there are systems that don't have librt - they have clock_gettime somewhere else (afair in libc directly - but my memories might be wrong, fixed that for ufoai a felt decade ago)
I will try to fix it in a proper cmake way and provide another pull request.

Cleaning up leftover tickets...


UX: Use a service worker

Using a service worker (https://developers.google.com/web/fundamentals/primers/service-workers/) will enable 3 major user experience issues
Issue 1: offline experience
The local webui can't be loaded if ipfs is not started
Solution 1: service worker caching
Being able to load the webui and see "not connected" even if ipfs is not started, by using a service worker
Issue 2: notifications
We need to use an extra extension (ipfs-companion) to receive notifications when ipfs in online or offline
Solution 2: service worker communication & background sync
The ipfs-webui could push notifications like "connected to ipfs" or "your file is now available on ipfs.io gateway":

even if the browser tab is not visible (using service worker communication)
even if the browser is actually closed (using background sync inside the service worker)

Issue 3: desktop installable
Make the webui installable on the desktop
Solution 3: Progressive Web App
Using a service worker would make the webui recognized as a Progressive Web App (https://developers.google.com/web/progressive-web-apps/)
As a result, it could be installed on the desktop with any browser that supports PWAs (e.g Brave browser)

I'm generally in favour of this. I enabled the standard create-react-app service worker on ipld-explorer... it just means you need to add some UI to inform the user when there is a newer version and nudge them to refresh.
I'm less sure about browser notifications... I generally turn them off on any site that asks for them.
Being able to install webui as a PWA would be cool, but we'd generally recommend installing ipfs-desktop as it can manage updating your ipfs commandline tools too.


Resources Estimator throws InvalidOperationException when Q# contains no intrinsic gates

Describe the bug
When running the resources estimator on a Q# program that does not contain calls to intrinsically defined gates, an InvalidOperationException is thrown.
The error is being thrown from here: https://github.com/microsoft/qsharp-runtime/blob/main/src/Simulation/EntryPointDriver/Simulation.cs#L41
It is caused by there not being a "Microsoft.Quantum.QSharp.Core" assembly being referenced, so the LINQ First fails with this error.
To Reproduce
open Microsoft.Quantum.Canon;
open Microsoft.Quantum.Intrinsic;
@EntryPoint()
    operation Main () : Unit {
        Message("Hello World!");
}

Expected behavior
The resources estimator should run without error, producing a result that shows no resources are used.
Alternatively, a more appropriate error message could be thrown.
Actual behavior
Unhandled exception: System.InvalidOperationException: Sequence contains no elements
   at System.Linq.ThrowHelper.ThrowNoElementsException()
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at Microsoft.Quantum.EntryPointDriver.Simulation`3.Simulate(IEntryPoint entryPoint, TIn input, DriverSettings settings, String simulator) in C:\Repos\qsharp-runtime\src\Simulation\EntryPointDriver\Simulation.cs:line 37
   at System.CommandLine.Invocation.CommandHandler.GetResultCodeAsync(Object value, InvocationContext context)
   at System.CommandLine.Invocation.ModelBindingCommandHandler.InvokeAsync(InvocationContext context)
   at System.CommandLine.Invocation.InvocationPipeline.<>c__DisplayClass4_0.<<BuildInvocationChain>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseParseErrorReporting>b__20_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass15_0.<<UseHelp>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass24_0.<<UseVersionOption>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass22_0.<<UseTypoCorrections>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseSuggestDirective>b__21_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseParseDirective>b__19_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<UseDebugDirective>b__11_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c.<<RegisterWithDotnetSuggest>b__10_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.CommandLine.Builder.CommandLineBuilderExtensions.<>c__DisplayClass13_0.<<UseExceptionHandler>b__0>d.MoveNext()

System information

OS: Windows 10

.NET Core Version: .NET 5.0.103

IQ# Version N/A

Python Version (if applicable) N/A

Additional context
N/A

Closely related but not entirely the same: https://github.com/microsoft/qsharp-runtime/issues/542.
@ScottCarda-MS Are you already looking into this one as part of adapting the entry point driver to process multiple entry points? Would you mind also taking a look at the related one I linked?

@bettinaheim I posted this as after a discussion with @swernli about this issue. Based on that I had thought this was an issue that @swernli would pick up when he had capacity, as it sounded, at the time, like he had some ideas on how to address this. @swernli can I assign this item to you?

@bettinaheim While this issue did surface during my work on the entry point driver, it doesn't seem to be related. In a conversation with @swernli, he showed me how to avoid the issue during my work on the entry point driver, but a genuine fix to the issue, as I remember him describing it, would take some refactoring and is outside the scope of my work with the entry point driver. I think @swernli would be a better assignee for this issue as he is more familiar with the code it touches and seemed to have an idea of what needed to be done to genuinely fix the underlying issue, which may also fix #542, although I haven't looked into the code on that one.

@ScottCarda-MS Capturing what we concluded in our offline chat: The root of the issue is indeed in a different place than I was suspecting based on the description. I'll hence unassign it for now, and then we will add it to our normal triage process.


[WIP] Update TypedData in line with SNIP-12

Describe your changes

Linked issues

Closes #423
Breaking changes

[ ] This issue contains breaking changes

Codecov Report
Attention: Patch coverage is 50.00000% with 24 lines in your changes are missing coverage. Please review.

Project coverage is 69.87%. Comparing base (706f6a6) to head (dd59869).
Report is 2 commits behind head on main.

Files
Patch %
Lines

...in/kotlin/com/swmansion/starknet/data/TypedData.kt
43.58%
12 Missing and 10 partials :warning:

...et/data/serializers/TypedDataTypeBaseSerializer.kt
0.00%
0 Missing and 1 partial :warning:

...in/com/swmansion/starknet/data/types/MerkleTree.kt
87.50%
0 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #428      +/-   ##
==========================================
+ Coverage   69.23%   69.87%   +0.64%
==========================================
  Files          73       72       -1
  Lines        3130     3187      +57
  Branches      308      321      +13
==========================================
+ Hits         2167     2227      +60
+ Misses        813      803      -10
- Partials      150      157       +7

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Homebrew does not have aws-sam-cli

Description
Homebrew does not have aws-sam-cli formula
Steps to reproduce
run `brew install aws-sam-cli
Observed result
Error: No available formula with the name "aws-sam-cli"
Expected result
Install aws-sam-cli on macOS
Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS: MacOS Catalina 10.15.4
Homebrew Version:  2.2.13

@challapradyumna I have the same issue.  Is this resolved?

first add the brew tap
brew tap aws/tap

Have same issue (after running brew tap aws/tap), output ..
$ brew upgrade aws-sam-cli
Error: aws/tap/aws-sam-cli not installed

@spuds51 - I had the same issue
after that I just wrote brew install aws-sam-cli although I though it was already installed and it worked.


figure out how to compute KL divergence effectively

resid acts are
*batch seq model
logits are
*batch seq vocab
so logits are vocab/model larger than activations
which mean we can't precompute them for all layers and positions

we could save SAEExamples to persistent storage and then do the rest of experiments for a single layer at a time


I am too dumb for this

I cannot for the life of me figure out how to install this mod I have tried running the gradle bat, and it says something about succeeding, but I can't find a .Jar or anything anywhere. I am in need of education in this, so please enlighten me.

The built jar is produced in the build/libs/ folder. You can also get a build from Github Actions.


#141 - Support Microsoft SQL Server

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.

Coverage decreased (-0.4%) to 80.92% when pulling c646a18607eef9accdcc69643324d81af8179c32 on issue-141-support-microsoft-sql into 405df9b55ae995fddf3d2bc9745cf54a2c7f945b on master.