Logger and initialization refinement and overhaul

1) File logger and the loggers in general:
Usage:

Any entity that likes to use the logger has to include mastercore_log.h
To add a new category, an entry in mastercore_log.h and mastercore_log.cpp should be created
To enable or disable logging of a specific category, mastercore_log.cpp is the place to go

Notes:

It provides a central place for anything log related
With _log.h included, there is no need for external ... anymore
mp_log() and mp_error() were never used, so I don't see any need to carry them around for now
The categories might be enabled and disabled via RPC or config file at some point later

2) Console or status logger:
As subsitute for printf() another logger function PrintToConsole() was added:

PrintToConsole() has a similar behavior like file_log(), but instead of printing to mastercore.log, the logger prints directly to the console
PrintToConsole() is superior to printf(), as it comes with smarter tinyformat based formatting, and for example .c_str() can be omitted, when printing strings
The main motivation though, was to gain control over all those printf(), which were sort of annoying and difficult to tame.. ;)
In particular, encapsulating the console output allows to apply transformations, like using the option -logtimestamps to prepend the output with timestamps
It is thinkable to use categories in the future, or to add more configuration options, such as -quiet, to silence all regular console output

3) More useful and readable initializtion messages:
Instead of printing line numbers and file names, the status messages were updated.
Further, the dumping of balances after an initial scan was removed completely for a few reasons:

It was only printed after a scan, but not on every start
The information is not really relevant
The information is still available via other channels, e.g. via RPC mscrpc, getallbalances..MP

4) Progress reporting during the intial transaction scanning:
Every 30 seconds the progress of the scanning is printed to the console, written to the debug log file, and the RPC status, as well as the splash screen progress label, are updated.
It was important for me to show that Omni Core hasn't stalled, or is broken and doesn't do anything  during the scanning, but in fact has already processed n of m blocks.
5) Shutdown request handling during the initial scanning:
Until now there was no way to shutdown Omni Core during the initial scanning, whether the shutdown request was intentionally, or triggered by some other event, for example a restart of the OS. The only way was to forcefully kill the process, which is neither user friendly, nor does it properly close the databases, and in the worst case it could result in some corruption.
Closing the QT splash screen window, or killing the process, triggers a global (Bitcoin Core based) shutdown request, which is honored by msc_initial_scan().
6) Low-hanging fruits on the way:
After @zathras-crypto gave green light for the initial merge, I started to wonder how to address any potential merge conflicts, and I finally came to the conclusion that it's probably better to bundle somewhat related changes, to reduce the impact on other pull requests.
While I generally think changes should come in smaller packets, the gain seemed to outweight the cost in this case, and in particular I'm referring to the last three commits 79749e9, f07a23d and 93eae99, which are hopefully easy to review and basically without any real impact.
7) No merge conflicts:
I actually got away with creating none for the pending pull requests.
The UI PR #5 can be merged directly, and #13 can be merged without conflicts, after merging https://github.com/zathras-crypto/mastercore/pull/63.
8) Let's see some results of the updated initialization and status reports... :)
$ ./src/test/test_bitcoin
Initializing Omni Core v<IP_ADDRESS>-dev [unittest]
Loading trades database: OK
Loading send-to-owners database: OK
Loading transactions database: OK
Loading smart property database: OK
Exodus balance: 0.00000000 MSC
Omni Core initialization completed
Running 150 test cases...
Omni Core shutdown completed

*** No errors detected

$ ./src/qt/bitcoin-qt -logtimestamps # or bitcoind
2015-04-15 22:04:10 Initializing Omni Core v<IP_ADDRESS>-dev [main]
2015-04-15 22:04:10 Loading trades database: OK
2015-04-15 22:04:10 Loading send-to-owners database: OK
2015-04-15 22:04:10 Loading transactions database: OK
2015-04-15 22:04:10 Loading smart property database: OK
2015-04-15 22:04:10 Scanning for transactions in block 249498 to block 352066..
2015-04-15 22:04:25 Still scanning.. at block 253221 of 352066. Progress: 3.63 %
2015-04-15 22:04:40 Still scanning.. at block 256863 of 352066. Progress: 7.18 %
2015-04-15 22:04:55 Still scanning.. at block 258254 of 352066. Progress: 8.54 %
2015-04-15 22:05:10 Still scanning.. at block 259397 of 352066. Progress: 9.65 %
2015-04-15 22:05:25 Still scanning.. at block 260657 of 352066. Progress: 10.88 %
2015-04-15 22:05:40 Still scanning.. at block 261969 of 352066. Progress: 12.16 %
# ctrl+c, kill bitcoind, kill bitcoin-qt, or close bitcoin-qt
2015-04-15 22:05:44 Scan stopped early at block 262346 of block 352066
2015-04-15 22:05:44 3690943 transactions processed, 121 meta transactions found
2015-04-15 22:05:44 Exodus balance: 3845.77658936 MSC
2015-04-15 22:05:44 Omni Core initialization completed
2015-04-15 22:05:44 Omni Core shutdown completed

$ ./src/bitcoind -logtimestamps -startclean # or bitcoin-qt
2015-04-15 22:18:37 Initializing Omni Core v<IP_ADDRESS>-dev [regtest]
2015-04-15 22:18:37 Loading trades database: OK
2015-04-15 22:18:37 Loading send-to-owners database: OK
2015-04-15 22:18:37 Loading transactions database: OK
2015-04-15 22:18:37 Loading smart property database: OK
2015-04-15 22:18:37 Scanning for transactions in block 5 to block 749..
2015-04-15 22:18:43 1591 transactions processed, 432 meta transactions found
2015-04-15 22:18:43 Exodus balance: 38002.65700426 MSC
2015-04-15 22:18:43 Omni Core initialization completed

Reviewed & supported.  OK to merge from my end :)

Alright. This pull request was updated significantly.
Sorry @zathras-crypto, and please don't get scared by the wall of text, hehe.. the overall changes are actually rather small.
I figured it would probably make sense to bundle several related changes, to reduce the overall time, because otherwise I'd probably would have pushed the logger PR, then the follow up, then another follow up, and another follow up.
I suggest to step through commit to commit, and they were created to be easily reviewed one by one.
Please let me know, if there is anything that should changed, removed, added, ... and I'm curious, which work flow you would prefer for the future: smaller chucks or bundled pull requests like this?
Anyway, feedback is very welcomed.. :)

@dexX7 there are some excellent suggestions in here - starting to review now.
Quick nitpick - "LogStatus" instead of printf?  It's just the name that seems off to me - to "log" something is to place an entry in a log; if we are not doing that (just printing to console does not log anything) is "log" the right term?

Yeah, I have a mixed feeling about LogStatus, too.
I'd be happy to replace this part, or the status messages, if you have any other idea.
Maybe StatusPrint(), ConsolePrint(), or just Print()?

Perhaps I'm 'old skool' hehe but I've always referred to it as 'echoing to console' (too much time spent on batch files in the 90s maybe?!?!? haha).
ConsolePrint(), PrintToConsole(), EchoToConsole() whatever you're comfortable with mate - I'm not that fussed, just thought that we might forget and assume LogStatus actually logs later down the track.
2 commits reviewed - so far so good - sitting in the waiting room at the doc's - if it wasn't for a lappy and tethering I think daytime TV (one big infomercial) would drive me nuts!!!

In this case I tend to use PrintToConsole(), because I could imagine to rename file_log() to PrintToLog() at some point, which would match well.

In this case I tend to use PrintToConsole()

Agreed.

Ready for review

Reviewed, all fine with me mate - few minor niggles I've commented about but other than that OK with me.  Great work mate, this is really good stuff :)

Nice. So to summarize:

LogStatus() -> PrintToConsole()
Scan progress reporting time from 15 seconds to 30 seconds

What do you think about the messages?

Also a note about the logging altogther: we might adopt a similar approach like Bitcoin Core, where no categories are hardcoded, but provided as argument, for example:
LogPrint("selectcoins", "total %s\n", FormatMoney(nBest));

... instead of something like we use currently:
if (selectcoins) { LogPrintf("selectcoins", "total %s\n", FormatMoney(nBest)); }

The categories are then activated via configuration or startup options:
./src/bitcoind -logtimestamps -debug=selectcoins -debug=bench -debug=net

So to summarize:
LogStatus() -> PrintToConsole()
Scan progress reporting time from 15 seconds to 30 seconds

Yep :)

What do you think about the messages?

Messages are clear - all good :)

Also a note about the logging altogther: we might adopt a similar approach like Bitcoin Core, where no categories are hardcoded, but provided as argument

Yeah this is a nice model - improving logging had been on my todo for a while so I really appreciate you cleaning it up.  My main objection to our current model is the requirement to recompile to enable debug flags (which means we can't easily ask integrators/users to enable them for support purposes).  I'd been planning to go RPC, but it makes sense to adopt the Bitcoin Core methodology and use startup params so there is a standard logging methodology right across the software.

requirement to recompile ... planning to go RPC

The debug flags are no longer constant and could now be updated at runtime. :)
One more point: the Exodus balance is printed at the end of the initialization, but the intend wasn't really clear to me, so I'm unsure how to handle it, or if there might be another way.
I guess this is sort of a verification/consensus hint? If so, it might make sense to print to block height as well, to make it comparable.

I guess this is sort of a verification/consensus hint? If so, it might make sense to print to block height as well, to make it comparable.

Indeed.  But it's irrelevant information for most.  I think perhaps a better way to go would be to have an --informativestartup or similar option, and only echo stuff a developer would want if that param is set.  What do you think?

and only echo stuff a developer would want if that param is set

It's reasonable, but what would you consider as informative, in this context? Printing all balances brings us back to the beginning, and there is actually not much I could think of, which I would consider as helpful. Some general numbers maybe, like the number of transactions, properties, etc. maybe, but all this can also be retrieved via explicit commands.

It's interesting (for me at least) - I'm quite a visual person.  The brain is essentially just one large pattern recognition engine and you'd be surprised at the things you spot simply because they look out of place compared to prior reference - even if you're not looking for them.
I figure when you start the client as many times as we do (numerous times per day) dumping some info would give us an easy spot on some things without directly looking for them - eg if we echo totaltrades and for whatever reason that number goes down, we'd likely pick up on the fact we'd broken MetaDEx without actually looking for it.
Perhaps that's just my perspective on things though - perhaps a bit 'fringe' hehe - tl:dr; if we had a --verbosestartup that printed extra info at startup I'd use it all the time for the above reason.  As to what that extra info would contain I'm not sure - examples would be Exodus balance, number of transactions, number of trades, totaltokens for MSC (and perhaps a couple other high activity properties that aren't grant/revoke-able) so we would pick up if any transactions had incorrectly credited without debiting or vice-versa.
All just thinking out loud here mate :)

Don't get me wrong, I like the idea, and a clean summary would be awesome. I have just some trouble pinning down specifics, at least when I think about it at the moment.
It's easier for me to see something, and use that as basis, so to speak. :) But when I further think about it: I'd be interested in protocol usage information, e.g. the number (in total and "recently") of properties created, transactions or trades. Also important in my opinion would be to know, whether the client is fully synchronized and up to date, given that "initialized" doesn't equal "up to date", but only "started".

Re my comments on splash above, I tested it - works well - this is a very nice change here mate thanks :)
I just dropped the following into msc_initial_scan (adapted from your PR as a quick test) - seems to work well :)
 if (GetTime() >= nNow + 15) {
        double dProgress = 100.0 * (blockNum - nHeight) / (max_block - nHeight);
        std::string updateText = strprintf("Still scanning.. at block %d of %d. Progress: %.2f %%\n", blockNum, max_block, dProgress);
        uiInterface.InitMessage(_(updateText.c_str()));
        nNow = GetTime();
    }

Ha! Nicely done. I was actually thinking about the progress signal, but this seems to be easier than anticipated.
I'll finish the branch quickly, and merge, to get a basis for the UI integration. :)
Quick side note re: _(updateText.c_str()):
/**
 * Translation function: Call Translate signal on UI interface, which returns a boost::optional result.
 * If no translation slot is registered, nothing is returned, and simply return the input.
 */
inline std::string _(const char* psz)
{
    boost::optional<std::string> rv = uiInterface.Translate(psz);
    return rv ? (*rv) : psz;
}

In this case there would be no sentence or word to translate, but a bunch of words with numbers, which are probably difficult to translate. :)

Updated as discussed. Diff: https://github.com/OmniLayer/omnicore/compare/93eae991967d1313411ff00c435cc33a2c6c31e9...dexX7:oc-0.10-extract-logger

Did you want to throw a UI splash update in there too mate?
static void ReportScanProgress(int nFirst, int nCurrent, int nLast)
{
    double dProgress = 100.0 * (nCurrent - nFirst) / (nLast - nFirst);
    std::string progressText = strprintf("Still scanning.. at block %d of %d. Progress: %.2f %%\n", nCurrent, nLast, dProgress);
    PrintToConsole(progressText);
    uiInterface.InitMessage(progressText.c_str());
}

@zathras-crypto: what are the up or downsides over uiInterface.ShowProgress(title, nProgress?
I'll give it a try: as far as I can see:

When using bitcoind, InitMessage() writes to debug.log:
2015-04-17 14:49:36 No coin database inconsistencies in last 7 blocks (74 transactions)
2015-04-17 14:49:36  block index            1865ms
2015-04-17 14:49:36 init message: Parsing Omni Layer transactions...
2015-04-17 14:49:41 init message: Still scanning.. at block 273003 of 349450. Progress: 11.57 %
2015-04-17 14:49:46 init message: Still scanning.. at block 288789 of 349450. Progress: 29.83 %

When using bitcoind, InitMessage() updates the RPC response:
$ ./src/bitcoin-cli getinfo -wait
error: {"code":-28,"message":"Parsing Omni Layer transactions..."}
$ ./src/bitcoin-cli getinfo -wait
error: {"code":-28,"message":"Still scanning.. at block 273003 of 349450. Progress: 11.57 %"}
$ ./src/bitcoin-cli getinfo -wait
error: {"code":-28,"message":"Still scanning.. at block 288789 of 349450. Progress: 29.83 %"}

When using bitcoin-qt, InitMessage() also updates the message on the splash screen:

When using bitcoind, ShowProgress() does not write to debug.log.
When using bitcoind, ShowProgress() does not update the RPC response.
When using bitcoin-qt, ShowProgress() updates the message on the splash screen:

When mixing this with our PrintToConsole()...

debug.log
splash
rpc
console

bitcoind, InitMessage
x
-
x
-

bitcoind, ShowProgress
-
-
-
-

bitcoind, InitMessage, PrintToConsole
x
-
x
x

bitcoind, ShowProgress, PrintToConsole
-
-
-
x

bitcoin-qt, InitMessage
x
x
x
-

bitcoin-qt, ShowProgress
-
x
-
-

bitcoin-qt, InitMessage, PrintToConsole
x
x
x
x

bitcoin-qt, ShowProgress, PrintToConsole
-
x
-
x

I'd like:

frequent updates of the splash screen

frequent updates of the RPC status response

infrequent, or no output to the debug.log

infrequent output to the console, ideally only if not in UI mode

So far, I'd say combining InitMessage() with PrintToConsole() would be the way to go, with a time of 30 seconds. I tested additionally firing ShowProgress() more frequent, but this just didn't seem clean.

@zathras-crypto: what are the up or downsides

I favour InitMessage() due to the RPC updates (though I had not realized it pumped into debug.log too - I spend all my time looking at mastercore.log I guess I should pop over to the bitcoin logs every now and then!!!)
Logging these "still scanning..." lines seems excessive, but updating the RPC response during startup is very beneficial and it sounds like from your testing it's both or neither.

Shutdown request handling during the initial scanning

Figured I'd just drop a note here - I'm still looking into it, but I've added your fix in msc_initial_scan() as per the following:
if (ShutdownRequested()) {
        file_log("Shutdown requested, stop scan at block %d of %d\n", blockNum, max_block);
        break;
    }

But am still getting a "fatal internal error" when I call a shutdown during scan - output:
Auditor was notified of a new property creation with ID<PHONE_NUMBER>
Auditor was notified of an increase of 500 tokens for property<PHONE_NUMBER> due to Fixed issuance (txid: 972ca2b99a62475313212db4083bfa6ff5150c47b74db9572fd1e875d0aeefa6)
Auditor was notified of a new property creation with ID<PHONE_NUMBER>
Auditor was notified of an increase of<PHONE_NUMBER>00000 tokens for property<PHONE_NUMBER> due to Fixed issuance (txid: 46e8b47764cc13309ec7b19cee6e39932c7f44a7d2cd82a9d2a9ba5eadd0c539)
Auditor has detected an invalid trade (txid: 75b687ca67f580d2ab1f7b88afc30fb1893d6eae4d40e83e6a596e49d21801a5) present in the MetaDEx following block 307607
Error: Error: A fatal internal error occured, see debug.log for details

2MuiZNuQR2vEmCq81XRTGRSHNETUGxEWR3C =>            +0.00000000 [SO_RESERVE=            +0.00000000 , ACCEPT_RESERVE=            +0.00000000 ]            +0.00000000
2Mw1R9LibSx7bLvrrXqDFj6dVFDnEdLuQsV =>          +151.53431577 [SO_RESERVE=            +0.00000000 , ACCEPT_RESERVE=            +0.00000000 ]            +0.00000000
<<<<<blah blah cut>>>>>
n4ckUx8KQP7QMtpDT9adpVf7RFZw8J2VRy =>            +2.22334400 [SO_RESERVE=            +0.00000000 , ACCEPT_RESERVE=            +0.00000000 ]            +0.00000000
n4gWyudHNTKwfXcwcmUhUJ87LYxfGk9dvx =>            +2.00000000 [SO_RESERVE=            +0.00000000 , ACCEPT_RESERVE=            +0.00000000 ]            +0.00000000
starting block= 263000, max_block= 349410
n_total= 947365, n_found= 637
[Initialized] Exodus balance: 27841.05142034
mastercore_shutdown(), line 2659, file: mastercore.cpp

I'm using if (!GetBoolArg("-overrideforcedshutdown", false)) AbortNode("Shutting down due to audit failure"); as the requester.
As I say just FYI, I'll have a nose around & see what I can find :)

Logging these "still scanning..." lines seems excessive

Given that this progress blocks the whole program, frequent updates aren't that bad, especially when almost every second something is written to the debug.log. FWIW: users scan a whole lot of transactions usually only once.
I don't really like it either though, and I also don't like the infrequent UI updates, but I think it's a middleground, and the alternative, a smarter status reporter, with different time settings and more targeted reporting, seems like an overkill - at least at the very moment.
This could be something for a follow up, and it sort of fits into the category of more targeted signaling in general.

But am still getting a "fatal internal error" when I call a shutdown during scan - output

Let me quickly build and test your branch. Can you describe how I can reproduce the issue?

but I think it's a middleground

Agreed - it's a fantastic improvement over what we had previously (an apparent hung process for 60 mins) :)

Can you describe how I can reproduce the issue?

If you just build and run 0.0.10-Z-Auditor the auditor is enabled by default - you can play with the omni_debug_auditor and omni_debug_auditor_verbose settings in omnicore_auditor.cpp but I don't think you'll need to, you should just see it attempt to shutdown and fail when it detects that bad trade in the MetaDEx on testnet.
Oh, and run with --startclean so it reparses everything to pick up the issue :)
Thanks again bud

Note, if you want to play around with it just introduce a math error somewhere (for example in STO calculation or something) that results in tokens disappearing/appearing unexpectedly to see it in action :)

But am still getting a "fatal internal error" when I call a shutdown during scan

Haha oh well. This was unexpected.. :) Check this out: src/main.cpp#L2838

Haha oh well. This was unexpected.. :) Check this out: src/main.cpp#L2838

Oh!  That wasn't there in 0.9 - hmm - that will affect shutdown's due to alerts also - nice spot :)

Yeah. :)
Is there anything you'd like to change in this PR before a merge?

Nope, it all looks good mate
On Apr 18, 2015 10:03 AM, "dexX7"<EMAIL_ADDRESS>wrote:

Yeah. :)
Is there anything you'd like to change in this PR before a merge?
—
Reply to this email directly or view it on GitHub
https://github.com/OmniLayer/omnicore/pull/17#issuecomment-94103102.

Awesome.
Here is a tip for your current branches (except the class C branch, which unfortunally requires a lot, but no different than the following, manual tweaking):
# update the local copy of the main branch
git fetch upstream omnicore-0.0.10
git checkout omnicore-0.0.10
git pull upstream omnicore-0.0.10

# then checkout your feature branch
git checkout 0.0.10-Z-UI

# and apply all commits upon the new "base"
git rebase omnicore-0.0.10

This should work without any issue for the UI branch, and I prefer this route over merging some main branch into a feature branch, every time there is an update, but maybe that's just personal taste. :)

When doing this with the auditor branch, at some point git will complain due to conflicts in mastercore.cpp:
Auto-merging src/mastercore_tx.cpp
Applying: Auditor : omnicore_auditor.cpp/h
Applying: Auditor : makefile changes
Applying: Auditor : remove manual debug line
Applying: Auditor : block sequence check
Applying: Auditor : Add MetaDEx bad trades check
Applying: Auditor : include DexX's fix for breaking out of scan (note still fatal error on shutdown)
Using index info to reconstruct a base tree...
M	src/mastercore.cpp
Falling back to patching base and 3-way merge...
Auto-merging src/mastercore.cpp
CONFLICT (content): Merge conflict in src/mastercore.cpp
Failed to merge in the changes.
Patch failed at 0008 Auditor : include DexX's fix for breaking out of scan (note still fatal error on shutdown)
The copy of the patch that failed is found in:
   ~/Projects/Cpp/omnicore/.git/rebase-apply/patch

When you have resolved this problem, run "git rebase --continue".
If you prefer to skip this patch, run "git rebase --skip" instead.
To check out the original branch and stop rebasing, run "git rebase --abort".

Here, you could manually resolve the conflict by removing the red lines:
@@ -1575,10 +1575,23 @@ static int msc_initial_scan(int nFirstBlock)
             break;
         }

-<<<<<<< HEAD
         if (GetTime() >= nNow + 30) { // seconds
             ReportScanProgress(nFirstBlock, nBlock, nLastBlock);
             nNow = GetTime();
         }
-=======
-  CBlock block;
-  for (int blockNum = nHeight;blockNum<=max_block;blockNum++)
-  {
-    if (ShutdownRequested()) {
-        file_log("Shutdown requested, stop scan at block %d of %d\n", blockNum, max_block);
-        break;
-    }
-
-    CBlockIndex* pblockindex = chainActive[blockNum];
-    string strBlockHash = pblockindex->GetBlockHash().GetHex();
->>>>>>> Auditor : include DexX's fix for breaking out of scan (note still fatal error on shutdown)

         CBlockIndex* pblockindex = chainActive[nBlock];
         if (NULL == pblockindex) break;

And after mastercore.cpp was saved, the file can be tagged as updated, and it can continue:
git add src/mastercore.cpp
git rebase --continue

Next, git will complain, because this specific commit is now empty, and this can be resolved by:
git rebase --skip

... and you're done! :)
This is probably followed by overwriting the remote version of that branch:
git push origin 0.0.10-Z-Auditor -f

See here - it shows "commited by dexx", because I'm not you obviously.. :)

Hmm, I actually just did this before seeing your comment - I just needed to pull from origin, resolve that merge conflict in mastercore.cpp then just use git add/git commit followed by a push - I didn't need to do the rebase stuff (https://github.com/zathras-crypto/mastercore/commit/6c7d919de09af334afd0a9c19074104e98ea564d) - did I miss something?

... did I miss something?

Haha no.. rebasing is just an alternative to merging, but it's perfectly fine. :)
But unfortunally I noticed one of the later commits of this PR actually did cause a conflict for the class C branch.. :/ (~two lines change though)

But unfortunally I noticed one of the later commits of this PR actually did cause a conflict for the class C branch.. :/ (~two lines change though)

Two lines :exclamation:
No problems at all mate, it's to be expected when we're bouncing around various different things :)

Two lines

Maybe I'm overly concerned, when it comes to those things, but let's just say:
I'm trying to avoid to interrupt the workflow of other pull requests, which have a higher priority in my opinion.. at least as best as I can. :)


Update loki log docs for helm

Not sure the best way to communicate helm values, the path.with.dots version or a little YAML snippet like you added in that other doc.


parametric: support otel config tests in ruby

Motivation
Enables Test_Otel_Env_Vars for ruby tracers.
Changes

Workflow

⚠ Create your PR as draft ⚠
Work on you PR until the CI passes (if something not related to your task is failing, you can ignore it)
Mark it as ready for review

Test logic is modified? -> Get a review from RFC owner. We're working on refining the codeowners file quickly.
Framework is modified, or non obvious usage of it -> get a review from R&P team

:rocket: Once your PR is reviewed, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist

[ ] Relevant label (run-parametric-scenario, run-profiling-scenario...) are presents
[ ] If PR title starts with [<language>], double-check that only <language> is impacted by the change
[ ] No system-tests internal is modified. Otherwise, I have the approval from R&P team
[ ] CI is green, or failing jobs are not related to this change (and you are 100% sure about this statement)
[ ] A docker base image is modified?

[ ] the relevant build-XXX-image label is present

[ ] A scenario is added (or removed)?

[ ] Get a review from R&P team

Error in parametric runs in CI

I generated the protos using an incompatible version of grpcio-tools. I will re-generate the protos with the correct version.

The failing ruby parametric tests are unrelated and will be resolved in a future PR: Details


Add scenario test for container module

This checklist is used to make sure that common guidelines for a pull request are followed.
General Guidelines

[ ] The PR has modified HISTORY.rst with an appropriate description of the change (see Modifying change log).

Command Guidelines

[ ] Each command and parameter has a meaningful description.
[ ] Each new command has a test.

(see Authoring Command Modules)

@yolo3301,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

Codecov Report

Merging #4118 into master will increase coverage by 0.26%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4118      +/-   ##
==========================================
+ Coverage   70.32%   70.59%   +0.26%
==========================================
  Files         437      437
  Lines       28147    28147
  Branches     4318     4318
==========================================
+ Hits        19794    19869      +75
+ Misses       7062     6988      -74
+ Partials     1291     1290       -1

Impacted Files
Coverage Δ

...nent/azure/cli/command_modules/component/custom.py
16.23% <0%> (ø)
:arrow_up:

src/azure-cli-core/azure/cli/core/prompting.py
25.3% <0%> (ø)
:arrow_up:

src/azure-cli-core/azure/cli/core/util.py
70.66% <0%> (ø)
:arrow_up:

...dback/azure/cli/command_modules/feedback/custom.py
31.25% <0%> (ø)
:arrow_up:

...ice/azure/cli/command_modules/appservice/custom.py
73.44% <0%> (+0.23%)
:arrow_up:

...zure-cli-vm/azure/cli/command_modules/vm/custom.py
75.52% <0%> (+0.28%)
:arrow_up:

...ource/azure/cli/command_modules/resource/custom.py
65.49% <0%> (+0.29%)
:arrow_up:

.../azure/cli/command_modules/vm/_template_builder.py
86.89% <0%> (+0.43%)
:arrow_up:

...azure-cli-core/azure/cli/core/commands/__init__.py
70.28% <0%> (+0.44%)
:arrow_up:

...cli-vm/azure/cli/command_modules/vm/_validators.py
78.02% <0%> (+0.47%)
:arrow_up:

... and 14 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 088adea...befb76f. Read the comment docs.

Test code looks good. As long as the docstring are converted, it is good to go. Thanks.


Draft: Add Message and Enum support for Storage Write API

This PR is based on #112, so I'll leave it as a draft until that is merged (and I assume the same test failure there will show up here). I also want to add unit tests for messages and enums before this gets merged.
I went ahead and added some convenience constructors to help reduce breaking changes for people that are using simple, flat protos. Even though I'm pretty sure this change gives us full support all types BigQuery has documented, I imagine as this gets used we'll run into some quirks in BigQuery that may require further changes to accommodate.

@fore5fire #112 is merged. Thanks


No device detected when trying hip samples

Hi,
With a system using AMD Threadripper 1950X,
I am trying to use my GPU, Radeon VII, on ubuntu 20.04.1 server as a VM under Proxmox. I was able to pass the GPU into the VM. To verify:
running
/opt/rocm/bin/rocminfo
ROCk module is loaded
Able to open /dev/kfd read-write
=====================
HSA System Attributes
=====================
Runtime Version:         1.1
System Timestamp Freq.:  1000.000000MHz
Sig. Max Wait Duration:<PHONE_NUMBER>3709551615 (0xFFFFFFFFFFFFFFFF) (timestamp count)
Machine Model:           LARGE
System Endianness:       LITTLE

==========
HSA Agents
==========
*******
Agent 1
*******
  Name:                    AMD Ryzen Threadripper 1950X 16-Core Processor
  Uuid:                    CPU-XX
  Marketing Name:          AMD Ryzen Threadripper 1950X 16-Core Processor
  Vendor Name:             CPU
  Feature:                 None specified
  Profile:                 FULL_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        0(0x0)
  Queue Min Size:          0(0x0)
  Queue Max Size:          0(0x0)
  Queue Type:              MULTI
  Node:                    0
  Device Type:             CPU
  Cache Info:
    L1:                      65536(0x10000) KB
  Chip ID:                 0(0x0)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   0
  BDFID:                   0
  Internal Node ID:        0
  Compute Unit:            32
  SIMDs per CU:            0
  Shader Engines:          0
  Shader Arrs. per Eng.:   0
  WatchPts on Addr. Ranges:1
  Features:                None
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: KERNARG, FINE GRAINED
      Size:                    49342116(0x2f0e6a4) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
    Pool 2
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    49342116(0x2f0e6a4) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       TRUE
  ISA Info:
    N/A
*******
Agent 2
*******
  Name:                    gfx906
  Uuid:                    GPU-ec0678e172fd5d72
  Marketing Name:          Vega 20 [Radeon VII]
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  Profile:                 BASE_PROFILE
  Float Round Mode:        NEAR
  Max Queue Number:        128(0x80)
  Queue Min Size:          4096(0x1000)
  Queue Max Size:          131072(0x20000)
  Queue Type:              MULTI
  Node:                    1
  Device Type:             GPU
  Cache Info:
    L1:                      16(0x10) KB
  Chip ID:                 26287(0x66af)
  Cacheline Size:          64(0x40)
  Max Clock Freq. (MHz):   1801
  BDFID:                   256
  Internal Node ID:        1
  Compute Unit:            60
  SIMDs per CU:            4
  Shader Engines:          4
  Shader Arrs. per Eng.:   1
  WatchPts on Addr. Ranges:4
  Features:                KERNEL_DISPATCH
  Fast F16 Operation:      FALSE
  Wavefront Size:          64(0x40)
  Workgroup Max Size:      1024(0x400)
  Workgroup Max Size per Dimension:
    x                        1024(0x400)
    y                        1024(0x400)
    z                        1024(0x400)
  Max Waves Per CU:        40(0x28)
  Max Work-item Per CU:    2560(0xa00)
  Grid Max Size:<PHONE_NUMBER>(0xffffffff)
  Grid Max Size per Dimension:
    x<PHONE_NUMBER>(0xffffffff)
    y<PHONE_NUMBER>(0xffffffff)
    z<PHONE_NUMBER>(0xffffffff)
  Max fbarriers/Workgrp:   32
  Pool Info:
    Pool 1
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED
      Size:                    16760832(0xffc000) KB
      Allocatable:             TRUE
      Alloc Granule:           4KB
      Alloc Alignment:         4KB
      Accessible by all:       FALSE
    Pool 2
      Segment:                 GROUP
      Size:                    64(0x40) KB
      Allocatable:             FALSE
      Alloc Granule:           0KB
      Alloc Alignment:         0KB
      Accessible by all:       FALSE
  ISA Info:
    ISA 1
      Name:                    amdgcn-amd-amdhsa--gfx906
      Machine Models:          HSA_MACHINE_MODEL_LARGE
      Profiles:                HSA_PROFILE_BASE
      Default Rounding Mode:   NEAR
      Default Rounding Mode:   NEAR
      Fast f16:                TRUE
      Workgroup Max Size:      1024(0x400)
      Workgroup Max Size per Dimension:
        x                        1024(0x400)
        y                        1024(0x400)
        z                        1024(0x400)
      Grid Max Size:<PHONE_NUMBER>(0xffffffff)
      Grid Max Size per Dimension:
        x<PHONE_NUMBER>(0xffffffff)
        y<PHONE_NUMBER>(0xffffffff)
        z<PHONE_NUMBER>(0xffffffff)
      FBarrier Max Size:       32
*** Done ***

running
/opt/rocm/bin/hipconfig --full
HIP version  : 3.7.20315-077bcfa0

== hipconfig
HIP_PATH     : /opt/rocm-3.7.0/hip
ROCM_PATH    : /opt/rocm-3.7.0
HIP_COMPILER : clang
HIP_PLATFORM : hcc
HIP_RUNTIME  : ROCclr
CPP_CONFIG   :  -D__HIP_PLATFORM_HCC__=  -I/opt/rocm-3.7.0/hip/include -I/opt/rocm-3.7.0/llvm/bin/../lib/clang/11.0.0 -I/opt/rocm-3.7.0/hsa/include -D__HIP_ROCclr__

== hip-clang
HSA_PATH         : /opt/rocm-3.7.0/hsa
HIP_CLANG_PATH   : /opt/rocm-3.7.0/llvm/bin
clang version 11.0.0 (/src/external/llvm-project/clang ee4e4ebbadcc8ea14ce99e34ed31ab31e94827ac)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm-3.7.0/llvm/bin
LLVM (http://llvm.org/):
  LLVM version 11.0.0git
  Optimized build.
  Default target: x86_64-unknown-linux-gnu
  Host CPU: znver1

  Registered Targets:
    amdgcn - AMD GCN GPUs
    r600   - AMD GPUs HD2XXX-HD6XXX
    x86    - 32-bit X86: Pentium-Pro and above
    x86-64 - 64-bit X86: EM64T and AMD64
hip-clang-cxxflags : -D__HIP_ROCclr__ -std=c++11 -isystem /opt/rocm-3.7.0/llvm/lib/clang/11.0.0/include/.. -isystem /opt/rocm-3.7.0/hsa/include -D__HIP_ROCclr__ -isystem /opt/rocm-3.7.0/hip/include -D__HIP_ARCH_GFX906__=1  -O3
hip-clang-ldflags  :  -L/opt/rocm-3.7.0/hip/lib -O3 -lgcc_s -lgcc -lpthread -lm

=== Environment Variables
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

== Linux Kernel
Hostname     : csns
Linux csns 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.1 LTS
Release:	20.04
Codename:	focal

However, then with my account under the group video,
when trying the bit_extract sample by executing ./bit_extract

without sudo
error: 'hipErrorNoDevice'(100) at bit_extract.cpp:58
with sudo

/src/external/hip-on-vdi/rocclr/hip_code_object.cpp:92: guarantee(false && "hipErrorNoBinaryForGpu: Coudn't find binary for current devices!")
Aborted

Finally, I checked the path
echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/opt/rocm/bin:/opt/rocm/profiler/bin:/opt/rocm/opencl/bin

Is VM compatible with ROCm? Or may be I am missing something from the installation?
Any ideas are appreciated.

did you resolve this issue ? I am also getting the hipErrorNoDevice though rocminfo shows both cpu and gpu

Me, too. Is there a solution?

Unfortunately not really, I actually just setup another Proxmox system with the latest software version, plus using ROCm 4.0.1.
I am being hit with the same error as I am trying to use cupy with RVII.

I noticed that I was missing some rights. In my case, I was not part of a Linux "group" which was the only group allowed to operate on the device. After being added to the group, I also had to recompile all sources compiled which hipcc, etc. now it works nicely.

I noticed that I was missing some rights. In my case, I was not part of a Linux "group" which was the only group allowed to operate on the device. After being added to the group, I also had to recompile all sources compiled which hipcc, etc. now it works nicely.

May I know which group was it? I have already added myself into video, but it did not help.

Before Ubuntu-20.04 (not included 20.04), we need group 'video'.
After Ubuntu-20.04 (include 20.04), we need group 'render'.
You can check kfd group by ls -l /dev/kfd.
work@fc00e52d30c9:~$ ls -l /dev/kfd
crw-rw-rw- 1 root render 239, 0 Mar 15 18:09 /dev/kfd

Before Ubuntu-20.04 (not included 20.04), we need group 'video'.
After Ubuntu-20.04 (include 20.04), we need group 'render'.
You can check kfd group by ls -l /dev/kfd.
work@fc00e52d30c9:~$ ls -l /dev/kfd
crw-rw-rw- 1 root render 239, 0 Mar 15 18:09 /dev/kfd

Thanks a lot!
I haven't tried but this is likely why.
I didn't read the wiki in full I guess, as I kind of remember the steps lol

I noticed that I was missing some rights. In my case, I was not part of a Linux "group" which was the only group allowed to operate on the device. After being added to the group, I also had to recompile all sources compiled which hipcc, etc. now it works nicely.

May I know which group was it? I have already added myself into video, but it did not help.

video, yes

Solved by adding the user to the group "video" and "render", the latter is required in new ROCm implementations.


optimize: optimize the insert logic of null expr & default expr

[ ] I have registered the PR changes.

Ⅰ. Describe what this PR did
support use default as primary key && allow to insert more than one rows with primary key is null or default
支持在插入语句中使用default关键字作为主键 && 支持通过Statement(非PreparedStatement)插入超过1条的主键是null/default的行
Ⅱ. Does this pull request fix one issue?

Ⅲ. Why don't you add test cases (unit test/integration test)?
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

Codecov Report

Merging #4254 (e5d4151) into develop (631ed7f) will decrease coverage by 0.11%.
The diff coverage is 14.28%.

@@              Coverage Diff              @@
##             develop    #4254      +/-   ##
=============================================
- Coverage      48.87%   48.76%   -0.12%
+ Complexity      3799     3796       -3
=============================================
  Files            724      724
  Lines          24201    24204       +3
  Branches        2957     2928      -29
=============================================
- Hits           11828    11802      -26
- Misses         11149    11153       +4
- Partials        1224     1249      +25

Impacted Files
Coverage Δ

...o/seata/rm/datasource/exec/BaseInsertExecutor.java
71.12% <0.00%> (ø)

...a/sqlparser/druid/mysql/MySQLInsertRecognizer.java
55.38% <0.00%> (-1.76%)
:arrow_down:

.../rm/datasource/exec/mysql/MySQLInsertExecutor.java
68.47% <33.33%> (-1.53%)
:arrow_down:

...obuf/convertor/BranchRegisterRequestConvertor.java
90.47% <0.00%> (-9.53%)
:arrow_down:

...otobuf/convertor/GlobalBeginResponseConvertor.java
92.59% <0.00%> (-7.41%)
:arrow_down:

...n/src/main/java/io/seata/common/util/IdWorker.java
77.08% <0.00%> (-6.25%)
:arrow_down:

...otobuf/convertor/GlobalCommitRequestConvertor.java
93.75% <0.00%> (-6.25%)
:arrow_down:

...otobuf/convertor/GlobalStatusRequestConvertor.java
93.75% <0.00%> (-6.25%)
:arrow_down:

...obuf/convertor/GlobalRollbackRequestConvertor.java
93.75% <0.00%> (-6.25%)
:arrow_down:

...protobuf/convertor/RegisterTMRequestConvertor.java
94.73% <0.00%> (-5.27%)
:arrow_down:

... and 20 more

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: caohdgege:x: a364176773You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.0 out of 2 committers have signed the CLA.:x: a364176773:x: caohdgegeYou have signed the CLA already but the status is still pending? Let us recheck it.


Example app doesn't run in Chrome

The app in the example folder is not made for flutter web. It doesn't run in Chrome. Throws an unimplemented error.
======== Exception caught by scheduler library =====================================================
The following UnimplementedError was thrown during a scheduler callback:
UnimplementedError

When the exception was thrown, this was the stack:
dart-sdk/lib/_internal/js_dev_runtime/private/ddc_runtime/errors.dart 251:49  throw_
lib/_engine/engine/canvaskit/layer_scene_builder.dart 35:5                    addPerformanceOverlay
packages/flutter/src/rendering/layer.dart 901:12                              addToScene
packages/flutter/src/rendering/layer.dart 546:5                               [_addToSceneWithRetainedRendering]
packages/flutter/src/rendering/layer.dart 1106:12                             addChildrenToScene
packages/flutter/src/rendering/layer.dart 1681:5                              addToScene
packages/flutter/src/rendering/layer.dart 938:5                               buildScene
packages/flutter/src/rendering/view.dart 228:30                               compositeFrame
packages/flutter/src/rendering/binding.dart 508:18                            drawFrame
packages/flutter/src/widgets/binding.dart 892:13                              drawFrame
packages/flutter/src/rendering/binding.dart 370:5                             [_handlePersistentFrameCallback]
packages/flutter/src/scheduler/binding.dart 1146:15                           [_invokeFrameCallback]
packages/flutter/src/scheduler/binding.dart 1083:9                            handleDrawFrame
packages/flutter/src/scheduler/binding.dart 864:7                             <fn>
dart-sdk/lib/_internal/js_dev_runtime/private/isolate_helper.dart 48:19       internalCallback
====================================================================================================

Found the problem - showPerformanceOverlay() in main. Will send a PR.


Page transition not working with anchor tag

Environment

Operating System: Linux
Node Version:     v16.20.0
Nuxt Version:     3.6.2
Nitro Version:    2.5.2
Package Manager<EMAIL_ADDRESS>Builder:          vite
User Config:      devtools, app
Runtime Modules:  -
Build Modules:    -

Reproduction
https://stackblitz.com/edit/github-qjv6tr-vvemyr?file=app.vue
Describe the bug
Hi! 💚
I'd tried to use page transition with a vue3 component library that has an anchor tag as a link but it seems not working.
Page transition works only with NuxtLink, the anchor tag does not trigger page transition.
Additional context
No response
Logs
No response

According to the docs, using NuxtLink component necessary for using page transitions since it uses vue-router under the hood and that prevents full page refresh. Anchor tag however fully refreshes page and that prevents page transition to work.

I'm developing a component library in vue3 and I would like to use the nuxt3 page transition
How can I achieve that?


Bugfix start time on gantt diagram is invalid when job is retried

https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2022-11-28#get-a-workflow-run
    "run_started_at": {
      "type": "string",
      "format": "date-time",
      "description": "The start time of the latest run. Resets on re-run."
    },

created_at does not reset on re-run.

I rewrite code to use run_attempt aware APIs. However, @actions/core does not provides attempt number, so it got by environment variables.
ref: https://github.com/actions/toolkit/pull/1387


Expand client-server interaction in submitting posts/events

This is the first issue that we'll resolve before we can move forward. When submitting the form, the javascript client currently submits no information to the event API, and it submits only these three fields for the basic page API:
{
    content, name: 'page',
    published: false
}

Tasks for this Issue

[ ] The client should give a real name, rather than 'page' for each page .
[ ] Our API should also use optional fields (ie, slug, title, headline, tags).
[ ] The event API should be used by the client on each "event" submission.

@michaellavery-grp, this added use of the event API is where I could use the most help. For now, I would like feedback on the basic structure of the existing code: the model event.py, its use in api.py lines 132 - 159, and its use in mockup.py lines 86 - 118.  Importantly, Instead of a database, the Nationbuilder mock-up uses state.py to write all stored data to JSON on the local file system. These files are ignored by git.
The event API is implemented in both the python api.py and mockup.py. The full nation builder event API is defined here, of which we use a subset, defined in event.py. Also, this is the  full basic pages API. We also only use a subset, defined in basic_page.py.
Current client-server interactions

The Nationbuilder mockup server is ready to send/receive/store data
The api server is ready to communicate between the client and the mockup

Now, it's up to the JS client to use these resources effectively.
This is done on "Submit" in a limited way already:

The submitPage in actions.js is called when submitting
The sendBasicPage api in api.js is called by submitPage
Then submitPage returns the new complete list of pages

@tvquizphd Sorry for the delay . I have had more time tonight to look at the code. My main issue was getting the pydantic Python module to work . They must have done some updates recently because I had to change some pydantic modules to pydantic_settings in the import section. I would push the code back to the repo, but I don't have permissions. Can you add me to the members? I forked a new branch , so I won't overwrite anything... Thanks, Mike L.


Log configuration dictionary when a spine is reset

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

4 of 8 (50.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.09%) to 87.798%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

vulp/spine/Spine.cpp
4
8
50.0%

Files with Coverage Reduction
New Missed Lines
%

vulp/spine/Spine.cpp
1
69.5%

Totals

Change from base Build<PHONE_NUMBER>:
-0.09%

Covered Lines:
1655

Relevant Lines:
1885

💛  - Coveralls


FancyZones: Alignment/Snapping/Ruler

When you create an additional layout, it would be useful to align it to the first one, so if both have the same positions but different sizes, you could cycle through them without the window moving x/y positions. Hope that makes sense.
You could also select several layouts and choose align centre or left, right etc.
It would also be useful if there were on screen coordinates / ruler of you could perfectly align things up nicely.

For displaying the size of the zone while editing, we have this issue to track it https://github.com/microsoft/PowerToys/issues/524

@mikeparkie
we haven't added the ruler yet, but we have magnetic snapping and alignment for zones.
The alignment occurs when they are touching but also when they don't.
It will be available in 0.16.

Nice one @enricogior I'm looking forward to testing them out. Thanks for all the efforts thus far.

For ruler, i say we create a new issue for it.  Multi-part requests become hard to verify complete.

@mikeparkie, as @crutkas suggested, when 0.16 will be released we will close this one. Feel free to open a new issue just for the ruler.
Thank you.

The fix is now available in 0.16 https://github.com/microsoft/PowerToys/releases


Helm Plugin - New CMD

Add the init helm plugin.
The call will be done in a follow-up.

Hi @estroz and @joelanford,
I applied some suggestions and did a few improvements. Let's me know what do you think now and if you see anything else that should be changed before we merge this one.


WIP:  Added F# Client

There are a couple of commits here, which I can squash but preferred to leave them distinct for now in case one of them in particular needs to be tossed out.
I had to make a change to get the code to compile out of source control, but I am uncertain whether that change is correct.
I moved the F# code over and here have thus far done just the minimum to get the solution to build.

Also, may I ask the overarching purpose of this (nblockchain) organization and this repo?  I'm guessing it is to make some of these lower-level components reusable by the public at-large, but I'll still ask to confirm.
The C# code has come from the Nethereum repo?  What is its intended future state?

There are a couple of commits here, which I can squash but preferred to leave them distinct for now in case one of them in particular needs to be tossed out.

No need to be squashed because each serves its own purpose and single unit of work, so it's perfect now.

I had to make a change to get the code to compile out of source control, but I am uncertain whether that change is correct.

Seems correct.

The C# code has come from the Nethereum repo? What is its intended future state?

The idea is that by adding cancelation token support to the RPC libs of Nethereum, we make a case for Nethereum to rather adopt JsonRpcSharp (and maybe even NBitcoin at some point).

Added another commit to support an Option, but have a look at it and see if it matches your intentions.

Can you remove any cancellationToken-related work from this PR and park it somewhere to propose it later as a second PR? I want this PR only to add the F# code (as it is in geewallet) and the build fix.

Is this to say you would like for me to begin making these changes to the C# code? If so, I might use something like CancellationToken.None, as CancellationToken is a struct and can't really be null.

Yes sounds good.

Code review bits to address, only two:

Don't include the LegacyTcpClient please (as a consequence of this, no need to include Config.fs either).
Rename FsClient to TcpClient please (in project name and file name).

I don't completely understand why this PR shows me pushing 8 commits, only 1 of which is from me.
I can guess:  If I do N commits, then to squash them I run git rebase -i HEAD~Y where Y is some number much larger than N, so that I don't have to count N commits, nor do I have to worry about off-by-one errors.
Then in the interactive rebase file, I use s to squash all of my commits, leaving other people's previous commits with the default pick option.  Perhaps doing this causes new SHAs to be assigned to the commits that precede my own?
If you can confirm, that would be an annoying behavior on Git's part, but at least my mental model would be accurate.
But also if that is the case, I do not know how to resolve the issue with this PR aside from just deleting and recreating my fork, which would likely cause all of this conversation to be lost.

Hey. When squashing 2 commits into 1, you don't have to mark 2 commits with
's' but just one (the last one), so when squashing 3 into 1, mark 2 and so
on. Also if you made let's say 3 commits on top of master, when rebasing
you have to use HEAD~3, if you use more than 3 you may destroy previous
history and make github confused, showing more commits than it should be.
Last thing, this conversation will never be lost. You could create a new
fresh clone, then create a branch with same name as the one you used for
this PR, recreate the work, and then push with force (-f) to overwrite
previous work in the PR.
Let me know if you have more doubts.
On Tue, 25 Sep 2018, 8:11 am David Falkner<EMAIL_ADDRESS>wrote:

I don't completely understand why this PR shows me pushing 8 commits, only
1 of which is from me.
I can guess: If I do N commits, then to squash them I run git rebase -i
HEAD~Y where Y is some number much larger than N, so that I don't have to
count N commits, nor do I have to worry about off-by-one errors.
Then in the interactive rebase file, I use s to squash all of my
commits, leaving other people's previous commits with the default pick
option. Perhaps doing this causes new SHAs to be assigned to the commits
that precede my own?
If you can confirm, that would be an annoying behavior on Git's part, but
at least my mental model would be accurate.
But also if that is the case, I do not know how to resolve the issue with
this PR aside from just deleting and recreating my fork, which would likely
cause all of this conversation to be lost.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/nblockchain/JsonRpcSharp/pull/1#issuecomment-424165482,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAUOJwC0XpwxZrLLtLB1gvnfYrQlMNvIks5ueXSrgaJpZM4W06g1
.


Introduce new attribute connection.mtls

What this PR does / why we need it: Introduce new mixer attribute "connection.mtls" which indicates whether connection is mutual TLS enabled. Add virtual function bool IsMutualTlsEnabledConnection() into http/check_data.h and tcp/check_data.h, and call this function inside AttributesBuilder::ExtractCheckAttributes(). Add another virtual function bool GetDestinationIpPort(std::string* ip, int* port) to report destination IP and port from HTTP filter.
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #426
Special notes for your reviewer:
Release note:

Please take a look. Thanks.

@JimmyCYJ: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/mixerclient-presubmit.sh
7ca20a135e77a7542ddb35fc75761e005a326779
link
/test mixerclient-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test mixerclient-presubmit

@JimmyCYJ: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/mixerclient-presubmit.sh
7ca20a135e77a7542ddb35fc75761e005a326779
link
/test mixerclient-presubmit

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/lgtm
/approve


Question about onehot contact prediction model

Hi,
I was able able to get the onehot model working for contact prediction (after making a few changes, like setting defaults in config class), however the performance is much lower (~0.06 precision@L/5) than reported in the paper.
If I'm understanding the code correctly, the pytorch version of the contact prediction head uses a simple linear classifier, whereas the original implementation  also included convolutional layers, as described in the paper.
Is this understanding correct?
Thanks!

Yes, I did not add the additional convolutional model for contact prediction to the pytorch implementation.


Automated Package Upgrade

:tada: This PR is included in version 9.0.32 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Trigger the release pipeline of the custom Apollo Router when a new version is available

To keep it up to date

#2665


Noise(?) in PWM signal

Hello all.
I am sorry if this 'issue' board is not appropriate for question.
Device : PixhawkMini
Firmware : px4fmu-v2_lpe
I am changing the firmware to apply my own controller(model-based) which outputs (thrust(N), 3-axis moment(N.m)).
Before changing the controller, I am checking the PWM signal flow to find out where I can change PWM values directly.
I found that the file "fmu.cpp" calls functions "mix" and "pwm_limit_calc".
In the last line of the function "pwm_limit_calc", I added a line effective_pwm[0]=1400 to rotate one motor with desired velocity.
It works well since the rostopic "rc/out" shows same value as 1400.
However, the value does not always maintain 1400, showing 1395, 1386, or other values.
The following script shows the phenomenon.
`header:
seq: 2564
stamp:
secs:<PHONE_NUMBER>
nsecs: 102663376
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
header:
seq: 2565
stamp:
secs:<PHONE_NUMBER>
nsecs: 166788376
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
header:
seq: 2566
stamp:
secs:<PHONE_NUMBER>
nsecs: 211253936
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
header:
seq: 2567
stamp:
secs:<PHONE_NUMBER>
nsecs: 259376936
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
header:
seq: 2568
stamp:
secs:<PHONE_NUMBER>
nsecs: 326525336
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1178, 1061, 1240, 1000, 1500, 1500, 0, 0]
header:
seq: 2569
stamp:
secs:<PHONE_NUMBER>
nsecs: 371479336
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
header:
seq: 2570
stamp:
secs:<PHONE_NUMBER>
nsecs: 415743272
frame_id: ''
channels: [0, 0, 0, 0, 0, 0, 0, 0, 1300, 1000, 1000, 1000, 1500, 1500, 0, 0]
At the seq:2568, the value is changed to 1178.
Please let me know why this weird thing happens.

Is there any other functions which have authority to designate PWM values?

Moreover, whenever I upload my modified firmware, the channels are changed. For now, you can see first eight channels are empty with zero values, and the last eight channels are designated to send PWM values.

How can I assign first eight channels to show PWM values in mavros?

Thank you for your time.

@Clark-Youngdong-Son can you use http://discuss.px4.io/ for questions? I think you'll need to share some of your code for someone to be able to help you.


remove ns-list workflows

Describe your changes
resolve the integration test fail. Remove the ns-list action for the workflows directory. It does not exist.
Issue ticket number and link
Closes #xxxx (Replace xxxx with the GitHub issue number)
Checklist before requesting a review

[x] I have performed a self-review of my code

Check the correct box. Does this PR contain:

[ ] Breaking changes
[ ] New functionality
[ ] Major changes
[ ] Minor changes
[x] Bug fixes

[x] Proposed changes are described in the CHANGELOG.md

[x] CI Tests succeed and look good!

@KaiWaldrant, could you make a commit which contains the partial commit message 'ci force'? This way we can check if this workflow works with the proposed changes.

the test fails are mitigated by PR #206


Complete RAD index.html as replication of www.ihe.net

current skeleton index.html is likely sufficient given RAD has only one IG today. Mary will work with RAD to understand their interest.


Longterm caching

Use webpack-md5-hash to address issues in Webpack 3 hash generation
Separate webpack runtime into separate chunk

I am closing this. This issue was created in Webpack 1 era.


Simplify construction of resolvers not attached to a project

This allows us to move DependencyMetaDataProvider and Module into dependency-management.
We further restrict detached resolvers from creating consumable configurations, to ensure they only represent resolvable root components and do not expose variants.
Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

Pull Request Feedback (Trigger) (Check)

@bot-gradle test this

I've triggered the following builds for you. Click here to see all build failures.

Pull Request Feedback (Trigger) (Check)


stuck while shutting down

2023-09-18 22:48:07.754 - voice - ovos_dinkum_listener.voice_loop.voice_loop:run:261 - INFO - Loop stopped running
2023-09-18 22:48:07.758 - voice - ovos_dinkum_listener.service:on_stopping:68 - INFO - DinkumVoiceService is shutting down...
2023-09-18 22:48:07.761 - voice - ovos_dinkum_listener.service:run:279 - INFO - Service stopping
2023-09-18 22:48:37.672 - voice - ovos_dinkum_listener.service:stop:346 - ERROR - voice_loop didn't call _shutdown
-----------Keyboard Interrupt--------
Traceback (most recent call last):
  File "/home/ovos/.local/bin/ovos-dinkum-listener", line 8, in <module>
2023-09-18 22:50:22.220 - voice - ovos_dinkum_listener.service:run:281 - DEBUG - shutdown done
2023-09-18 22:50:22.225 - voice - ovos_dinkum_listener.service:run:283 - DEBUG - stopped
    sys.exit(main())
  File "/home/ovos/.local/lib/python3.10/site-packages/ovos_dinkum_listener/__main__.py", line 29, in main
    service.stop()
  File "/home/ovos/.local/lib/python3.10/site-packages/ovos_dinkum_listener/service.py", line 347, in stop
    self._shutdown()
  File "/home/ovos/.local/lib/python3.10/site-packages/ovos_dinkum_listener/service.py", line 369, in _shutdown
    self.mic.stop()
  File "/home/ovos/.local/lib/python3.10/site-packages/ovos_microphone_plugin_alsa/__init__.py", line 55, in stop
    self._thread.join()
  File "/usr/lib/python3.10/threading.py", line 1096, in join
    self._wait_for_tstate_lock()
  File "/usr/lib/python3.10/threading.py", line 1116, in _wait_for_tstate_lock
    if lock.acquire(block, timeout):
KeyboardInterrupt

Haven't seen this one before; how did you start/stop the service (CLI entrypoint, Python script, something else)?

this came up during testing running only messagebus and listener service (most likely during a systemctl stop)
But i just saw that it is misplaced as it originates from ovos-microphone-plugin-alsa


[MAJOR UPDATE] Use (and not modify) standard robot controller

Now, all users have to do is modify a few build files and create a custom Vision-enabled OpMode!

@homu r+

:pushpin: Commit 3ffeda2 has been approved by smo-key

:zap: Test exempted - status


DEV-146 | add codebox ide

details: https://issues.teracy.org/browse/DEV-146

Pr review and tested ok. Rebased code pushed here https://github.com/phuonglm/dev/tree/tasks/146-add-codebox-ide-to-vagrant

closed, refer to https://github.com/teracyhq/dev/pull/122 for following updates and rebase


replication requesting blocks it should not

The replication is requesting blocks that it shouldn't. When you request the last content block it starts to request all of them from 0. This is happening by default with _selectLatest and causing all our issues I think.
For example, we have an archive with 2 blocks. A single file has been updated twice, so we only want block 2. It requests block 2 but then requests 0 and 1:
hyperdrive-archive clone downloadAll loop self.download { name: 'test.txt',
  blocks: 1,
  content: { blockOffset: 2, bytesOffset: 16 },
  type: 'file' } +0ms
hypercore-selection add() clone:content { start: 2, end: 3 } +1ms
hypercore-replicate clone:content chan=05519e..97 update() requesting block=2 +1ms
hypercore-replicate clone:content chan=05519e..97 update() requesting block=0 +0ms
hypercore-replicate clone:content chan=05519e..97 update() requesting block=1 +0ms

These requests are coming from:
feed._updatePeers = function () {}
// which calls peers.update()
this.peers[i].update()
// the message is created:
var message = this.feed.selection.next(this.remoteBitfield, this._blocks)

It is in feed.selection.next() that things are going wrong. It is returning three requests: 2,0,1 when it should be returning 2 then finishing.
It shouldn't be requesting blocks 0,1 at all: {sparse: true} and the only block prioritized is 2.
The range is set to:
Range {
    start: 2,
    end: 3,
    strategy: 0,
    bytes: 0,
    priority: 2,
    callback: undefined,
    _downloaded: 2,
    _index: 0 }

These tests get to the root of the issue. Its definitely an issue with sparse downloads and the prioritization API.
This test fails. It should only download block 2 but all the blocks get downloaded (2 then 0 - 1 - 2)
tape('replicate live sparse with prioritization', function (t) {
  var core1 = hypercore()
  var core2 = hypercore()

  var feed = core1.createFeed()

  feed.append('hello')
  feed.append('world')
  feed.append('again')
  feed.finalize(function () {
    var clone = core2.createFeed(feed.key, {
      sparse: true
    })

    replicate(clone, feed)

    clone.on('download', function (block) {
      t.same(clone.blocks, 3, 'should be 3 blocks')
      console.log(block)
      if (block !== 2) return t.fail('unknown block')
      t.pass('correct block')
      t.end()
    })
    clone.prioritize({start: 2, end: 3})
  })
})

This test passes, and only the block we want is downloaded:
tape('replicate live sparse with download', function (t) {
  var core1 = hypercore()
  var core2 = hypercore()

  var feed = core1.createFeed()

  feed.append('hello')
  feed.append('world')
  feed.append('again')
  feed.finalize(function () {
    var clone = core2.createFeed(feed.key, {
      sparse: true
    })

    replicate(clone, feed)

    clone.on('download', function (block) {
      t.same(clone.blocks, 3, 'should be 3 blocks')
      console.log(block)
      if (block !== 2) return t.fail('unknown block')
      t.pass('correct block')
      t.end()
    })
    clone.get(2, function (_, data) {
      t.same(data.toString(), 'again', 'correct string')
    })
  })
})

Did a sanity check on bugs Im seeing and tested starting by prioritizing block 1 and it still gets all of them in the order: 1, 0, 2.

I had an embarrassing bug but this should be fixed in 4.5.2

If this is not fixed please re-open

republished in 4.19.5 (i'm bad at git)

@mafintosh git better

@juliangruber nono it's "git gud"
On Fri, Feb 17, 2017 at 4:48 AM, Julian Gruber<EMAIL_ADDRESS>wrote:

@mafintosh https://github.com/mafintosh git better
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mafintosh/hypercore/issues/69#issuecomment-280618130,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABNhUx_neeoWFEkFlzLWhWIRCTkmmKbtks5rdXr1gaJpZM4L_so0
.


Enhanced Version Of titleCase

Here's an enhanced version of the titleCase function with additional features:

Excludes short words from capitalization: Words like "and", "or", "the" won't be capitalized unless they're the first or last word.
Handles acronyms: Words in all caps will remain in all caps.
Deals with hyphenated words: Both parts of hyphenated words will be capitalized.

String titleCase(String text) {
  final List<String> excludeWords = ['and', 'or', 'the', 'to', 'of', 'for', 'in', 'on', 'at', 'by', 'with', 'as'];
  final List<String> words = text.trim().split(RegExp(r'\s+'));

  String capitalize(String word) {
    if (word.length <= 2 || word.toUpperCase() == word) return word;
    return word[0].toUpperCase() + word.substring(1).toLowerCase();
  }

  for (int i = 0; i < words.length; i++) {
    if (words[i].contains('-')) {
      words[i] = words[i]
          .split('-')
          .map((part) => capitalize(part))
          .join('-');
    } else if (i == 0 || i == words.length - 1 || !excludeWords.contains(words[i].toLowerCase())) {
      words[i] = capitalize(words[i]);
    } else {
      words[i] = words[i].toLowerCase();
    }
  }

  return words.join(' ');
}

This enhanced version of titleCase takes care of more specific capitalization rules, making the output more refined and accurate for a variety of text inputs.

Function Definition
String titleCase(String text) {

The function titleCase takes a String parameter called text and returns a String.
The purpose of this function is to convert a given string into title case, where the first letter of each word is capitalized, except for certain small words unless they appear at the beginning or end of the string.

List of Words to Exclude
  final List<String> excludeWords = ['and', 'or', 'the', 'to', 'of', 'for', 'in', 'on', 'at', 'by', 'with', 'as'];

excludeWords is a list of small words that should not be capitalized unless they are the first or last word in the input string.

Splitting the Text into Words
  final List<String> words = text.trim().split(RegExp(r'\s+'));

The text string is trimmed to remove any leading or trailing whitespace.
The split method is used with a regular expression RegExp(r'\s+') to divide the string into a list of words based on one or more whitespace characters.

Capitalize Function
  String capitalize(String word) {
    if (word.length <= 2 || word.toUpperCase() == word) return word;
    return word[0].toUpperCase() + word.substring(1).toLowerCase();
  }

The capitalize function takes a String parameter called word.
If the word length is 2 or less, or if the word is already in uppercase, it returns the word as it is.
Otherwise, it capitalizes the first letter and makes the rest of the word lowercase.

Loop Through Words and Capitalize as Needed
  for (int i = 0; i < words.length; i++) {
    if (words[i].contains('-')) {
      words[i] = words[i]
          .split('-')
          .map((part) => capitalize(part))
          .join('-');
    } else if (i == 0 || i == words.length - 1 || !excludeWords.contains(words[i].toLowerCase())) {
      words[i] = capitalize(words[i]);
    } else {
      words[i] = words[i].toLowerCase();
    }
  }

A for loop iterates over each word in the words list.
If a word contains a hyphen (-), it splits the word into parts, capitalizes each part, and then joins them back with hyphens.
If the word is the first or last word in the list, or if it is not in the excludeWords list, it is capitalized.
If the word is in the excludeWords list, it is converted to lowercase.

Joining Words Back into a String
  return words.join(' ');
}

The words are joined back into a single string with a space separating each word, and this string is returned as the result of the function.

Summary
This function converts a given string into title case while handling special cases such as hyphenated words and small words that should not be capitalized unless they are the first or last word in the string.


Feature: Improved damage chat in D&D 5e

Suggestion: It would be interesting if it only showed "TokenName HP Updated: 11-6(6)=5".
Thanks a lot for the module! I loved.

As someone who doesn't play 5e, I need steps to reproduce this message. Please provide clear steps, as well as the ChatMessage object created. This is done in the console using game.messages.get("messageID");. Foundry logs created ChatMessage IDs in the console, so you can find the ChatMessage ID there. If determined to be easily re-formatted, your feature request will be fulfilled. If this turns out to be a module-specific chatmessage, I'll close and mark this as wontfix.

The message is specific to the Midi module, so I think it can close.

If you want to assist in helping create support for midi, as it is a widely-used module, feel free to contact me in Discord and we can talk about it. I don't know the specifics of the stylings midi uses, so having someone that does would help. I would work on it on my own if I wanted to, but since midi is a feature-rich module, I wouldn't know where to start.

It would be awesome! Midi is responsible for automation in D&D 5e (from the simplest to the most complex) and is widely used. I can't tomorrow, but I'll get in touch as soon as I have time.


Give a way to specify static images folder

I've discovered Daux.io few days ago thanks to wallabag and I'm currently moving my documentation on it.
Actually I've a little problem: documentation is opened to contributions in its self repository but only the ./docs folder is visible. Since images have to be added in ./img to be accessible and it is not visible in the repo, contributors have no easy way to add images.
I would like to specify another folder (e.g. in ./docs/lang/img where lang could be en, fr, etc.) to get static images.
If you have no time to develop that, I may try to look at the source code to add this feature and then do a PR if you're interested.

this is now changed and the images can be in the same folder as the documentation, relative paths work correctly.
It is in the develop branch for now but will be in master very soon

Thanks but I moved to another system since I opened this ticket. But still, it's good to know ;).

@marienfressinaud Out of curiosity what system did you move to? I'm looking for a good docs editor and I haven't found anything.

@meghart I moved to MkDocs (http://www.mkdocs.org/). It's not totally the same usecase since it is a generator of static doc websites but it fits with my needs.


Support previewing asset types other than video and images

Is your feature request related to a problem? Please describe.
There's currently no way to preview an asset type that isn't a video or image. This has come up when adding raw asset types. They aren't caught by the first condition in src/components/AssetPreview.tsx and so a preview is attempted with an img tag, but there are other resource_types available that won't preview properly with the default return value

(this particular asset is .doc type, but this happens with other formats like pdfs)
Describe the solution you'd like
Add another condition to render resource types other than images or videos, or allow the consumer to define their own preview config
Describe alternatives you've considered
I'm considering recreating the cloudinary asset schema type for the instances where files or other raw assets types are often used, so that we can define our own preview config, but this is not an ideal solution

It would be nice to be able to pass in/use a customized AssetPreview component that had access to the Cloudinary schema, so one could customize the preview with either text or perhaps a transformed version of the asset if it is a convertible but not  tag supported image type.

I solved this by creating a component:
/components/cloudinaryPreviewImage.tsx

export default (url?: string, alt?: string) => () => (<img src={url?.replace('image/upload', 'image/upload/c_thumb,w_32,g_face,q_auto,f_auto')} alt={alt} />)

And then when I require a preview image:
/schemaTypes/my-new-type.ts

import previewCloudinaryImage from "../components/previewCloudinaryImage";

export const myNewType = defineType({
  name: "myNewType",
  type: "document",
  fields: [
    defineField({
      name: "title",
      type: "string,
    }),
    defineField({
      name: "image",
      type: "cloudinary.asset",
    }),
  ],
  preview: {
    select: {
      title: 'title',
      media: 'image.url',
    },
    prepare: ({ title, media }) => (
        title,
        media: previewCloudinaryImage(media, title)
    })
  },
}


Failure on client.follow() and client.unfollow()

Expected behaviour
Join or Leave a space by calling function
Actual behavior
Throws error:
TypeError: can't access property "length", str is undefined
Steps to reproduce the behavior
Call async function await client.follow(web3, account, {  "name":space}) or await client.unfollow(web3, account, {  "name":space})

Hi @vswee Follow and unfollow expect a alias wallet to sign the message, refer to https://github.com/snapshot-labs/snapshot/blob/master/src/composables/useFollowSpace.ts#L63-L100


ERROR: *** no cameras available ***

I'm trying to get the raspberry pi camera module v1 working on my raspberry pi 4 but I get this error whenever I run the command below
pi@raspberrypi:~ $ libcamera-hello -t 0
libEGL warning: DRI2: failed to authenticate
[0:06:37.643508933] [1155]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
ERROR: *** no cameras available ***

Things i tried:
Checking the camera and camera cable, I know the camera works because it used to work with buster and my other setup (raspberry pi 4, an HDMI capture card, and the same cable I'm using with the camera ) works flawlessly, so it can't be the hardware I guess.
It is a clean install so I don't think I messed up the settings.
after following this  guide,
My config.txt looks like this:
# Enable audio (loads snd_bcm2835)
dtparam=audio=on
disable_touchscreen=1

# Enable DRM VC4 V3D drive
dtoverlay=ov5647
max_framebuffers=2
arm_64bit=1
start_x=1
gpu_mem=128

At first, I was doing ssh but later I switched to HDMI display. when I run " libcamera-hello -t 0" a little black window pops up for less than a second and vanishes away.
I also tried adding disable_touchscreen=1 in /boot/config.txt but no luck.
output of "sudo vcgencmd version":
pi@raspberrypi:~ $ sudo vcgencmd version
Oct 29 2021 10:48:24
Copyright (c) 2012 Broadcom
version b8a114e5a9877e91ca8f26d1a5ce904b2ad3cf13 (clean) (release) (start_x)

It's most probably an official camera module. I am using raspberry pi 4 4Gb model with "2021-05-07-raspios-buster-arm64" OS
Can you tell me what am I doing wrong?
When I use the same SD card with my RPI3 B+ the red light of the camera lights up at boot but I see nothing with Pi 4.

UPDATE
I flashed my other sd card with bullseye. The behavior of RPI4 is the same but with RPI3 the camera light flashes once while booting then it stays on, what am I missing?
My config file now:
# Enable audio (loads snd_bcm2835)
dtparam=audio=on

# Automatically load overlays for detected cameras
camera_auto_detect=1

# Automatically load overlays for detected DSI displays
display_auto_detect=1

# Enable DRM VC4 V3D driver
#dtoverlay=vc4-kms-v3d
dtoverlay=ov5647
max_framebuffers=2

# Run in 64-bit mode
arm_64bit=1

# Disable compensation for displays with overscan
disable_overscan=0

[cm4]
# Enable host mode on the 2711 built-in XHCI USB controller.
# This line should be removed if the legacy DWC2 controller is required
# (e.g. for USB device mode) or if USB support is not required.
otg_mode=1

ok so now there's a new issue:
pi@raspberrypi:~ $ libcamera-still -e png -o test.png
libEGL warning: DRI2: failed to authenticate
[0:00:29.413902869] [1076]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
[0:00:29.477313332] [1076]  INFO Camera camera.cpp:937 configuring streams: (0) 1296x972-YUV420
[0:00:29.477469666] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt BG10 Score: 2067 (best 2067)
[0:00:29.477522814] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt BG10 Score: 1849.67 (best 1849.67)
[0:00:29.477550554] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt BG10 Score: 1500 (best 1500)
[0:00:29.477575943] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt BG10 Score: 3796 (best 1500)
[0:00:29.477603943] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt BA81 Score: 4796 (best 1500)
[0:00:29.477628888] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 2592x1944 fmt pBAA Score: 1567 (best 1500)
[0:00:29.477653610] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 1920x1080 fmt pBAA Score: 1349.67 (best 1349.67)
[0:00:29.477678591] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 1296x972 fmt pBAA Score: 1000 (best 1000)
[0:00:29.477702999] [1090]  INFO RPI raspberrypi.cpp:122 Mode: 640x480 fmt pBAA Score: 3296 (best 1000)
[0:00:29.477800462] [1090]  INFO RPI raspberrypi.cpp:620 Sensor: /base/soc/i2c0mux/i2c@1/ov5647@36 - Selected mode: 1296x972-pBAA
[0:00:29.502436462] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: Saturation = 1.000000
[0:00:29.502533443] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: Contrast = 1.000000
[0:00:29.502559869] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: Brightness = 0.000000
[0:00:29.502583980] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: AwbMode = 0
[0:00:29.502607869] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: Sharpness = 1.000000
[0:00:29.502631795] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: ExposureValue = 0.000000
[0:00:29.502656277] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: AeExposureMode = 0
[0:00:29.502678443] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: FrameDurationLimits = [ 33333, 33333 ]
[0:00:29.502705703] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: AeMeteringMode = 0
[0:00:29.502727184] [1094]  INFO IPARPI raspberrypi.cpp:626 Request ctrl: NoiseReductionMode = 3
[0:00:29.504377925] [1090]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:00:29.504420684] [1090]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:00:29.504439128] [1090]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:00:29.504456536] [1090]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:00:29.576412184] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (7 left)
[0:00:29.621891888] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (6 left)
[0:00:29.655425295] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (5 left)
[0:00:29.689159814] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (4 left)
[0:00:29.721822758] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (3 left)
[0:00:29.755262258] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (2 left)
[0:00:29.788660425] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (1 left)
[0:00:29.822052795] [1090]  INFO RPI raspberrypi.cpp:1636 Dropping frame at the request of the IPA (0 left)
terminate called after throwing an instance of 'std::runtime_error'
  what():  failed to import fd 19
Aborted
pi@raspberrypi:~ $

Hi, let's start with the first problem.
From what you've said I've understood that that you're using a 64-bit Buster image. The date seems quite old, but maybe there isn't a newer one. Is it fully up to date?
The firmware seems to date from October which is much newer. Have you done a sudo rpi-update? Except I expect that would have updated the kernel too, so I'm not really understanding what you have.
Could you also please report the output of libcamera-hello --version. Where did you get libcamera and libcamera-apps from, the apt repositories or have you built them for yourself? That actually makes quite a big difference - the latest versions of the libcamera source requires a more up to date kernel that you have, and vice versa.
Now a couple of other things that might (or might not) help:

You don't seem to specify a display driver. Usually you need once of dtoverlay=vc4-fkms-v3d or dtoverlay=vc4-kms-v3d. I'm not sure what happens if you don't specify one. Maybe you get the full legacy driver which won't work.
When using ssh with X-forwarding (ssh -X) the preview windows don't work because they use hardware acceleration which isn't supported across a network. But you can add the --qt-preview to get an unaccelerated preview which does work. You can also always use the -n option which suppresses the preview window altogether.
On the Pi 3 you need to re-enable Glamor. Please check here: https://forums.raspberrypi.com/viewtopic.php?t=323547

ok so I switched to Bullseye OS, and yes I tried sudo rpi-update both in Buster and Bullseye
To keep it clear I'm using Raspberry pi 4 with Bullseye OS
The libcamera was already there in Bullseye so this time I didn't did anything, previously I used this link to build it
Here is the output:
pi@raspberrypi:~ $ libcamera-hello --version
libcamera-apps build: 2a38ae93f143 21-10-2021 (14:53:32)
libcamera build: v0.0.0+3156-f4070274

Things I did:
added dtoverlay=vc4-kms-v3d in config.txt
ran sudo rpi-update again
Other details that might help:
pi@raspberrypi:~ $ cat /etc/debian_version
11.1

pi@raspberrypi:~ $ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

pi@raspberrypi:~ $ uname -a
Linux raspberrypi 5.10.82-v8+ #1497 SMP PREEMPT Fri Dec 3 16:30:35 GMT 2021 aarch64 GNU/Linux

The error is still there:
pi@raspberrypi:~ $ libcamera-still -e png -o test.png
Preview window unavailable
[0:02:00.257211009] [1558]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
ERROR: *** no cameras available ***

pi@raspberrypi:~ $ libcamera-still -e png -n -o test.png
[0:02:18.830181788] [1563]  INFO Camera camera_manager.cpp:293 libcamera v0.0.0+3156-f4070274
ERROR: *** no cameras available ***

So one thing you need to be aware of is that, rather unfortunately, the latest kernel (which you get when you do sudo rpi-update) has moved on from the libcamera and libcamera-apps apt packages. So if you do an rpi-update then you must checkout out the latest libcamera code and build it from source. We're working hard on an update the last release which will bring all this stuff back together again, but does it sound like this is what has "bitten" you?

Yes. I have the same problem. I am using camera module v 1.3. and up to date version bullseye and the same message i get from the libcamera which is ERROR: *** no cameras available ***

@Shraimsami Thanks for reporting this. Could you perhaps file a new bug report as there can be many causes of this sort of error, and it could quite easily be different in your case. Please be sure to include all the helpful information that you can:

The exact model of Pi that you are using.
I see that you're using a v1.3 camera. Is it an official module, or do you have a link where it came from?
The output of uname -a.
The output of vcgencmd version.
The output of libcamera-hello --version.
Any modifications you have made to your /boot/config.txt file.
The exact command line that you are running.
The console output produced by that command line.
Have you done a sudo rpi-update at any point?
Have you checked out and rebuilt either libcamera or libcamera-apps?
Are there any error messages relating to the camera in dmesg?
Can you say if you're logged in directly to the Pi with a monitor and keyboard, or are you using ssh?

Thanks!

HiThank you for your fast respond. I find the problem. I was controlling raspberry pi through ethernet and using putty. I fixed the problem by downloading buster OS into my raspberry pi. I have a question. What is an alternative way to controll the camera module beside picamera library. Is there another library  that I can use Which can be compatible with libcamera.I want to make server and clint to stream video can you give a link to an example. I am using python.Thanks very much again.
Sent from Yahoo Mail on Android
On Mon., 20 Dec. 2021 at 10:25, @.***> wrote:
@Shraimsami Thanks for reporting this. Could you perhaps file a new bug report as there can be many causes of this sort of error, and it could quite easily be different in your case. Please be sure to include all the helpful information that you can:

The exact model of Pi that you are using.
I see that you're using a v1.3 camera. Is it an official module, or do you have a link where it came from?
The output of uname -a.
The output of vcgencmd version.
The output of libcamera-hello --version.
Any modifications you have made to your /boot/config.txt file.
The exact command line that you are running.
The console output produced by that command line.
Have you done a sudo rpi-update at any point?
Have you checked out and rebuilt either libcamera or libcamera-apps?
Are there any error messages relating to the camera in dmesg?
Can you say if you're logged in directly to the Pi with a monitor and keyboard, or are you using ssh?

Thanks!
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you were mentioned.Message ID: @.***>

@Shraimsami There is a Picamera2 library in development, unfortunately it is not yet available.

HiThank you for your fast respond. I find the problem. I was controlling raspberry pi through ethernet and using putty. I fixed the problem by downloading buster OS into my raspberry pi. I have a question. What is an alternative way to controll the camera module beside picamera library. Is there another library  that I can use Which can be compatible with libcamera.I want to make server and clint to stream video can you give a link to an example. I am using python.

If you are looking to be able to stream the output of the camera, you probably want to use features from something like gstreamer as well to provide your network streaming. If that's the case, then gstreamer already has python bindings, and an element that can use libcamera -- so you should be able to construct a pipeline that uses a libcamerasrc element, and connect it to your network streaming pipeline, all controlled from python, using gstreamer already.
There are still some missing features in the gstreamer element, so you likely have to manually specify the framerate and colorimetry caps, with a caps filter.

HelloYes thanks for the tips. I have spent alot of time trying  to download gstreamer to windows. I have managed to download it  to raspberry  pi.  Can you help me installing gstreamer on python  on windows. I have downloaded gstreamer from the official website but couldnot import gi to python. couldnot put the path for  environmental variables. I have python 3.9. And I want to  capture the stream from raspberry pi to a PC windows OS.i want to use h246 format because of high framerate. Best regards Sami shraim
Sent from Yahoo Mail on Android
On Tue., 21 Dec. 2021 at 11:58, Kieran @.***> wrote:
HiThank you for your fast respond. I find the problem. I was controlling raspberry pi through ethernet and using putty. I fixed the problem by downloading buster OS into my raspberry pi. I have a question. What is an alternative way to controll the camera module beside picamera library. Is there another library that I can use Which can be compatible with libcamera.I want to make server and clint to stream video can you give a link to an example. I am using python.
If you are looking to be able to stream the output of the camera, you probably want to use features from something like gstreamer as well to provide your network streaming. If that's the case, then gstreamer already has python bindings, and an element that can use libcamera -- so you should be able to construct a pipeline that uses a libcamerasrc element, and connect it to your network streaming pipeline, all controlled from python, using gstreamer already.
There are still some missing features in the gstreamer element, so you likely have to manually specify the framerate and colorimetry caps, with a caps filter.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you were mentioned.Message ID: @.***>

There are quite a lot of different questions in this issue now, which I think have mostly been answered? So I'm going to close this one, but if there's still anything outstanding please feel free to open a new "clean" issue instead. Thanks!


Gz bridge add mag air pressure

add magnetometer forwarding from gz sim topic to uorb topic
add barometer forwarding from gz sim topic to uorb topic (from upstream PX4)
add airspeed forwarding from gz sim topic to uorb topic (from upstream PX4)

closed due to invalid. use sensor from px4 sim instead


[Question]  problem porting to other LLM

thanks to the author. When porting to other LLM，such as baichuan 13B or qwen, there are a lot of warnings when doing finetuning, but the dataset LLaVA-Instruct-150K seem to right, What could be the problem？ thank you ！
WARNING: tokenization mismatch: 406 vs. 410. (ignored)
WARNING: tokenization mismatch: 504 vs. 508. (ignored)
WARNING: tokenization mismatch: 296 vs. 300. (ignored)
WARNING: tokenization mismatch: 257 vs. 261. (ignored)
WARNING: tokenization mismatch: 215 vs. 219. (ignored)
WARNING: tokenization mismatch: 676 vs. 680. (ignored)
WARNING: tokenization mismatch: 145 vs. 148. (ignored)
Related Issue：https://github.com/haotian-liu/LLaVA/issues/346

Hi, can you share a repo that allows me to easily reproduce this? Because we have supported MPT which has a different prompt format, and it is unclear where the issue stems from. Thanks.

Same problem, stage1 is fine, stage2 will report this warning, only the model(baichuan2-6b) is replaced, and the dataset is not modified

@zjh908491372 Could you share what all changes are to be made to change the llm in LLaVA? Thanks,

Hi, can you share a repo that allows me to easily reproduce this? Because we have supported MPT which has a different prompt format, and it is unclear where the issue stems from. Thanks.

we are training a model based on Yi-6b in pretrain stage and received the same WARNING like
WARNING: tokenization mismatch: 924 vs. 928. (ignored)
WARNING: tokenization mismatch: 922 vs. 926. (ignored)
WARNING: tokenization mismatch: 462 vs. 464. (ignored)
WARNING: tokenization mismatch: 246 vs. 248. (ignored)
WARNING: tokenization mismatch: 333 vs. 337. (ignored)
WARNING: tokenization mismatch: 980 vs. 990. (ignored)
WARNING: tokenization mismatch: 169 vs. 171. (ignored)
WARNING: tokenization mismatch: 1241 vs. 1243. (ignored)
WARNING: tokenization mismatch: 335 vs. 337. (ignored)
WARNING: tokenization mismatch: 230 vs. 232. (ignored)
WARNING: tokenization mismatch: 181 vs. 183. (ignored)
WARNING: tokenization mismatch: 761 vs. 765. (ignored)
WARNING: tokenization mismatch: 1876 vs. 1882. (ignored)
WARNING: tokenization mismatch: 160 vs. 162. (ignored)
WARNING: tokenization mismatch: 86 vs. 88. (ignored)
WARNING: tokenization mismatch: 90 vs. 92. (ignored)
WARNING: tokenization mismatch: 328 vs. 330. (ignored)
WARNING: tokenization mismatch: 242 vs. 246. (ignored)
WARNING: tokenization mismatch: 1188 vs. 1192. (ignored)

"I want to replace LLM with Qwen-7B. How can I make this modification?Thanks!


fix: 1. vpc, cloudprovider should not violate the sharing limit of cl…

…oud account 2.
sharing mode of host and attached local storage should be consistent. 3.
cannot change owner of shared resource
这个 PR 实现什么功能/修复什么问题:
修正：1、vpc和cloudprovider的共享需要遵守对应云账号的共享 2. 共享的资源不允许change owner 3. host和本地存储的共享需要一致

是否需要 backport 到之前的 release 分支:

release/3.2

/cc @zexi  @ioito
/area region
/hold

/hold cancel

/hold

Codecov Report

Merging #6219 into master will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #6219      +/-   ##
==========================================
- Coverage    7.04%    7.04%   -0.01%
==========================================
  Files         647      647
  Lines      110913   110926      +13
==========================================
  Hits         7815     7815
- Misses     102497   102510      +13
  Partials      601      601

Flag
Coverage Δ

#aFlag
7.04% <0.00%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/cloudcommon/db/domainresource.go
0.00% <0.00%> (ø)

pkg/cloudcommon/db/infraresource.go
0.00% <ø> (ø)

pkg/cloudcommon/db/sharablevirtual.go
0.00% <ø> (ø)

pkg/cloudcommon/db/virtualresource.go
0.00% <0.00%> (ø)

pkg/compute/guestdrivers/managedvirtual.go
0.00% <0.00%> (ø)

pkg/compute/models/cloudproviders.go
0.94% <0.00%> (-0.02%)
:arrow_down:

pkg/compute/models/disks.go
0.54% <0.00%> (ø)

pkg/compute/models/hosts.go
0.30% <0.00%> (+<0.01%)
:arrow_up:

pkg/compute/models/networks.go
0.54% <0.00%> (ø)

pkg/compute/models/purge.go
0.00% <0.00%> (ø)

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0c45818...46ef98f. Read the comment docs.

/hold