Added tests for src\client\common\installer\moduleInstaller.ts

For #9231

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[x] Title summarizes what is changing
[ ] Has a news entry file (remember to thank yourself!)
[ ] Appropriate comments and documentation strings in the code
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[x] Unit tests & system/integration tests are added/updated
[ ] Test plan is updated as appropriate
[ ] package-lock.json has been regenerated by running npm install (if dependencies have changed)
[ ] The wiki is updated with any design decisions/details.

Codecov Report

Merging #9266 into master will decrease coverage by 0.55%.
The diff coverage is 94.44%.

@@            Coverage Diff             @@
##           master    #9266      +/-   ##
==========================================
- Coverage   57.78%   57.22%   -0.56%
==========================================
  Files         557      557
  Lines       29590    29218     -372
  Branches     4474     4474
==========================================
- Hits        17098    16720     -378
- Misses      11548    11557       +9
+ Partials      944      941       -3

Impacted Files
Coverage Δ

src/client/common/installer/moduleInstaller.ts
94.52% <94.44%> (+38.96%)
:arrow_up:

...nt/datascience/jupyter/invalidNotebookFileError.ts
0% <0%> (-60%)
:arrow_down:

src/client/formatters/serviceRegistry.ts
50% <0%> (-50%)
:arrow_down:

src/datascience-ui/native-editor/redux/store.ts
50% <0%> (-50%)
:arrow_down:

src/datascience-ui/history-react/redux/store.ts
50% <0%> (-50%)
:arrow_down:

src/datascience-ui/react-common/logger.ts
0% <0%> (-40%)
:arrow_down:

...cience/jupyter/jupyterDebuggerNotInstalledError.ts
33.33% <0%> (-33.34%)
:arrow_down:

src/datascience-ui/react-common/locReactSide.ts
16.66% <0%> (-33.34%)
:arrow_down:

...jupyter/kernels/jupyterKernelPromiseFailedError.ts
0% <0%> (-33.34%)
:arrow_down:

...ience/jupyter/jupyterDebuggerRemoteNotSupported.ts
50% <0%> (-30%)
:arrow_down:

... and 42 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c2ca78...88b9f94. Read the comment docs.


Device parameter

in the main train.py script you can pass the device param. but there is a lot of stuff cuda Is used for and can't use the CPU to do anything? is this meant to do stuff like cuda:4 or so?

Okay, I'm having trouble getting it up and running due to the fact all those datasets seem to be broken / not available for download

Please check README. I have updated the download link. You can download VL-CMU-CD-binary255 at here.


nixos/users-groups: avoid top level with lib; use lib before builtins

#208242

Motivation
Top-level with turns off any sort of typo detection at parse time; the file must be actually evaluated to find out whether there's an error.
Description of changes

[x] Pulled in every symbol used in this file from lib in an inherit statement
[x] If a symbol could come from lib but was used from builtins, switch to lib

Things done

[x] Tested with nix-build -A nixosTests.installer.simple
[x] Fits CONTRIBUTING.md.

I'm sure you inspected this file and found that it's not worth doing that right here right now.

Quite so. This is one of the larger and more complicated NixOS modules, and other than switching all the way to lib. uses, which ~40% of the file uses, nothing felt right. So I went with the inherit route.
When this file along with the rest of nixpkgs gets nixfmt'd, I think it'll make the choice to switch to lib.thing really obvious and straightforward.


Implement add command

Implement AddCommand, which takes name as a necessary parameter and all other arguments as optional

Codecov Report

Merging #60 (cb99288) into master (9a98437) will increase coverage by 0.49%.
The diff coverage is 77.41%.

@@             Coverage Diff              @@
##             master      #60      +/-   ##
============================================
+ Coverage     66.72%   67.21%   +0.49%
- Complexity      503      507       +4
============================================
  Files            84       84
  Lines          1704     1690      -14
  Branches        190      194       +4
============================================
- Hits           1137     1136       -1
+ Misses          518      505      -13
  Partials         49       49

Impacted Files
Coverage Δ

...ain/java/seedu/address/ui/ContactDetailsPanel.java
0.00% <0.00%> (ø)

src/main/java/seedu/address/ui/MainWindow.java
0.00% <0.00%> (ø)

...c/main/java/seedu/address/model/person/Person.java
91.30% <80.00%> (-2.73%)
:arrow_down:

.../java/seedu/address/logic/commands/AddCommand.java
92.30% <100.00%> (ø)

...java/seedu/address/logic/commands/EditCommand.java
72.32% <100.00%> (ø)

...a/seedu/address/logic/parser/AddCommandParser.java
100.00% <100.00%> (ø)

.../java/seedu/address/storage/JsonAdaptedPerson.java
97.59% <100.00%> (+2.24%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9a98437...cb99288. Read the comment docs.

#60 #51


Add % of deviation when comparing two values and tolerance for this column is set to a percentage

When we apply tolerance - In the testresult it is giving the failure results (if the results are in tolerance range ) but how get the difference value between two numbers.

Could you please clarify what you are asking? Are you asking how to calculate a relative deviation?
=(new_value/original_value)-1
If I'm not mistaken it's +356% in your case.
For me personally, it's sufficient to have the result presented with its base values.

Thank you for the quick response. Yes, I am asking for relative difference in the result file. It is easy to know how much percentage difference is there b/w two numbers (if the tolerance in %).

It's really something easy to add but it adds really little value. If someone is motivated to do it, feel free, I'll merge this.

I will like to take it

@riktimmondal great, feel free to ask questions if needed.

Thank you @riktimmondal.
To add more context on the same, if we are comparing two numbers with no tolerance then the difference value should be displayed.

Could you explain why you need this? What are you doing with this information?

On 22 Aug 2017, at 20:27, aniketh02<EMAIL_ADDRESS>wrote:
Thank you @riktimmondal.
To add more context on the same, if we are comparing two numbers with no tolerance then the difference value should be displayed.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

My personal opinion: I like the base values to track down the reason easier. That a deviation is out of bounds tells the error message already.
However, it's a minor thing, I won't argue about it :)

@FuegoArtificial I also make use of base values to understand the root cause of the red test. On the other hand, I don't see a case where it would have been easier to have a % (or a difference) to have a better understanding of the issue. I'm really curious to have a case where it helps.

The Case:- Because of changes to the code, We are aware the expected  difference is  a few hundred thousands here the difference seen is -26,454,611 and to Highlight the percentage of  deviation, this helps Prioritize the Fix in  a release.
So displaying the difference or Percentage helps the developers and testers flag the importance and rank the issues by the percentage of deviation.

@riktimmondal could you please update on the status of this request.

@aniketh02 I see your point. However, I believe, that a test framework is not the right tool to figure priorities. The test is a pass or no pass decision.
If it fails, a process of your choice begins where people decide on whichever reasons how to handle the failed test.
Your way is a very specific one to handle a failed test (a number of prioritized tasks). For the next failed test, the way could be different and the deviation would be clutter information for you, irrelevant and harder to find the reason of the failed test for you.
Each case how to specifically handle a failed test can be different. With your test you find the problems, what the test framework is for.
Perhaps, it could be some sort of property of the test definition how to display the error message? So it optionally shows percentage instead of the base values? :-)

Thanks Fuego,
I think, I mixed both of them (Failure's and Prioritization), let me take a step back on Prioritization and detail on Failure.

When a developer is Unit testing and he knows that there is an issue, It a Qualitative fact, that it is a pass or a fail
If the developer knows the size of the issue, that is quantitative, he can justify that the code is not causing a great harm, or causing a Big Harm to the numbers, by providing the difference of values and percentages, it helps quantify an issue in his code.
The Idea "Perhaps, it could be some sort of property of the test definition how to display the error message? So it optionally shows percentage instead of the base values? " is great

@aniketh02 I agree with your point 1 but not the 2. As @FuegoArtificial explained, you've many ways to define the priorities: it could be the count of rows with a failure, the delta between actuals and expectations, the percentage, the variance of the deltas, ... Too many options to rely on a test framework to perform this analysis for you.
IMO, if you want to automatically sets priorities on your bugs (or even manually review the priorities assisted by a software creating a report) then you should rely on a tool processing your test-results and not on the test framework. That's just separation of concerns.
Post-processing of test-results can be facilitated by #307, there you'll have the option to setup a second tool (not aware of any tool doing such a thing) parsing the test-results and applying your own calculations, enriching the results and facilitating your own prioritizations. If you want to specify in the test what kind of calculations or prioritizations should be apply by this other tool, feel free to use the traits ... that's what they are intended to do: specify parameters that should be used by other tools.
The more we're speaking about it, the less I think it's something for NBi. I'm thinking to close this issue by out-of-scope, if we've no new arguments in favor to do such things without impacting all users.
@riktimmondal have you already started to work on this?


Add IRepos with CRUD methods for base enteties

User dapper  https://github.com/StackExchange/dapper-dot-net

Review: https://github.com/MorphineForOdin/GoT2_LCG/commit/f2fd52d4ae0804e9e958c4852cded4473d307c93?diff=split

Remove 'Base' sufix from Repo and Entity names


Update navigation in index.md

navigation section 2

Merging into my own branch for further changes

https://github.com/mendix/docs/pull/6829


SnapIt Horizontal bounds cleanup and settings

What did you fix? (provide a description or issue closes statement)
Horizontal pad for detecting item name overlap in SnapIt removed by default, but can be modified in settings. This is in response to an issue reported by Corgo#6755 on Discord, where certain items (Inaros Prime Chassis Blueprint, Ivara Prime Chassis Blueprint) being next to eachother would cause a bounds overlap on 3440x1440 resolution, which also caused a crash on attempted item count save in said scenario. Also updated the number detection box to adjust for this change, and added settings to enable manual adjustment to said box width.
All new settings are exclusively in file, as they should only be needed to adjust for other issues

Reproduction steps

Use SnapIt, with item counting enabled if you want to check the adjustment to the number detection box.
Check the resulting SnapItImageBounds image in the debug folder to see the new bounds
(optional) change the new settings and repeat from step 1 to see how that affects the outcome.
"SnapItHorizontalNameMargin" (double) for what multiple of the text height should be used as horizontal pad for text bounds checking
"DoCustomNumberBoxWidth" (boolean) to enable the use of custom number detection box width (next setting)
"SnapItNumberBoxWidth" (double) for what multiple Number Box Left Edge - Column Right Edge to use as number box width

Evidence/screenshot/link to line
Just check all changed lines, it's very short
Considerations

Does this contain a new dependency? [No]
Does this introduce opinionated data formatting or manual data entry? [In settings file, but rarely needed]
Does this pr include updated data files in a separate commit that can be reverted for a clean code-only pr? [No]
Have I run the linter? [No]
Is is a bug fix, feature request, or enhancement? [Bug Fix]

yolo for now


Doesn't find any devices

Hey devs!
I'm struggling to understand ReactiveX, and I was wondering why this was the chosen as a technology.
Regardless of that, I'm wondering why the demo app doesn't seem to proceed past the initial search screen. It seems to not find any groups... for example, in LoadingViewController when getAllGroups is called, the array of groups within filter is empty. Not sure why... I have a fairly simple topology: a playbar and two play 1s. All are in different rooms and all are grouped together.

Hi Antonino,
Thank you for your feedback!

The reason for choosing RX is rather easy, I wanted to become better in using RX and as you can see in my commit history I have learned a lot.
So the demo app, first things first, it's just a quick and really dirty implementation. What happens is that the moment the Loading view is presented it waits until their is at least one group. The first time the subscription returns an empty array because the SSDP (local network search for sonos devices) is not finished yet. The second time, which should be executed after 4 seconds it should return at least one group. in the latest release (0.10.0) I did change some stuff, so it could be possible that their is a bug. Please help me out by debugging buildInteractorObservable in GetGroupsInteractor.swift.
But the issue could also be SSDP related, to test this, run the demo app provided in this repo: https://github.com/stefanrenne/RxSSDP

Fixed! Thanks @stefanrenne!


Reorg repo structure

Synchronizing all names:

Gradle project names
Project directory names
Project artifact ID

hrmm... do we expect to build anything else except first-party? Perhaps that should be the root directory for building? I mean base build.gradle should be in first-party?

Ah, that's a good idea. Haven't thought about it. Let's reorg it.

yeah it would really help clean up the settings.gradle


__lzcnt64 fix

NativeJIT/inc/NativeJIT/BitOperations.h:137:35: error: use of undeclared identifier       '__lzcnt64'             *highestBitSetIndex = __lzcnt64(value);
Fixed adding
#define __LZCNT__
in BitOperations.h.
[  PASSED  ] 108 tests
OSX Yosemite.

Thanks for the bug report and fix!
Also, could you please run the following command and paste the output back into this issue?
sysctl -n machdep.cpu.brand_string

I'd like to try to figure out if this issue is "because" of Yosemite or maybe because of a hardware difference.

Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz

Great! Thanks again!
I believe this is an Ivy Bridge CPU, which doesn't support lzcnt. Just for future reference for myself, I think that defining LZCNT should still produce a correct result as long as we don't pass in -mlzcnt or something else that forces the compiler to think that the CPU supports lzcnt.
Again, just for future reference for myself, we have a test that calls GetHighestBitSet with a 0 operand. If that test passed, that seems to indicate that this fix doesn't cause the (potential) issue where older CPUs will interpret lzcnt as bsr (bsr is the same as lzcnt, except when the input operand is 0) instead of throwing a #ud

If you need some more tests just tell me.

@mikucionisaau Hi! Did you try the fix suggested above, and if so, did tests still pass for you afterwards?

with __LZCNT__ defined at the top of BitOperations.h I am getting something else:
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/immintrin.h:73:0,
                 from /usr/lib/gcc/x86_64-linux-gnu/6/include/x86intrin.h:48,
                 from /tmp/NativeJIT/inc/NativeJIT/BitOperations.h:37,
                 from /tmp/NativeJIT/inc/NativeJIT/CodeGen/X64CodeGenerator.h:32,
                 from /tmp/NativeJIT/inc/NativeJIT/CodeGen/FunctionBuffer.h:40,
                 from /tmp/NativeJIT/src/CodeGen/FunctionBuffer.cpp:26:
/usr/lib/gcc/x86_64-linux-gnu/6/include/lzcntintrin.h: In function ‘short unsigned int __lzcnt16(short unsigned int)’:
/usr/lib/gcc/x86_64-linux-gnu/6/include/lzcntintrin.h:41:29: error: ‘__builtin_clzs’ was not declared in this scope
   return __builtin_clzs (__X);
                             ^
src/CodeGen/CMakeFiles/CodeGen.dir/build.make:158: recipe for target 'src/CodeGen/CMakeFiles/CodeGen.dir/FunctionBuffer.cpp.o' failed

I am actually confused what this option is supposed to give.
I grepped the GCC headers for __LZCNT__ and found only the following:
#ifndef __LZCNT__
#pragma GCC push_options
#pragma GCC target("lzcnt")
#define __DISABLE_LZCNT__
#endif /* __LZCNT__ */

And then it does not seem to be sensitive to __DISABLE_LZCNT__ and it gets undefined at the end.

clang-3.9 is also upset (even with __LZCNT__ defined):
/tmp/NativeJIT/inc/NativeJIT/BitOperations.h:139:35: error: always_inline function '__lzcnt64'
      requires target feature 'lzcnt', but would be inlined into function 'GetHighestBitSet' that is compiled without support
      for 'lzcnt'
            *highestBitSetIndex = __lzcnt64(value);

Thus I replaced the __lzcnt64(value) call in BitOperations.h with the following binary search and then make test passes:
           *highestBitSetIndex = 64;
           if (value) {
               if (value & 0xFFFFFFFF00000000) { *highestBitSetIndex -=32; value >>= 32; }
               if (value & 0x00000000FFFF0000) { *highestBitSetIndex -=16; value >>= 16; }
               if (value & 0x000000000000FF00) { *highestBitSetIndex -= 8; value >>=  8; }
               if (value & 0x00000000000000F0) { *highestBitSetIndex -= 4; value >>=  4; }
               if (value & 0b0000000000001100) { *highestBitSetIndex -= 2; value >>=  2; }
               if (value & 0b0000000000000010) { *highestBitSetIndex -= 1; value >>=  1; }
               if (value) { *highestBitSetIndex -= 1; }
           }

Perhaps this could be surrounded with CPU checks to be more efficient on modern CPUs. Anyway, I don't know what I am doing :-)

Great! Thanks for the suggested workaround.
Yeah, I agree that we should have CPU checks that fall back to something that works on older CPUs and do something "fast" on modern CPUs. Unfortunately, I don't think that either @MikeHopcroft or I have easy access to a machine with an older CPU, so it's hard for us to check that we're doing something reasonable.
Let me dig around to see if I can find someone with an old atom or P4 or something, since I think we'll have all of our (x86) bases covered if we work on an atom and a P4 (although maybe we should also check an old AMD or Centaur chip or something).
That might take a while, though.

I'm woefully under qualified to be weighing in on this, but it's piqued my interest watching it develop over the past few days, so I've been looking into it, and I think I might have a small bit to add.
In short, it looks like what is happening is that GCC and Clang both define __lzcnt64 in a way such that they assume lzcnt will be supported (target("lzcnt")), or in other words, they both essentially implicitly assume lzcnt support unless unless __LZCNT__ is defined (which seems a bit backwards, but... whatever). This means that somehow __LZCNT__ needs to be defined on architectures where it's not supported.
Interestingly, this would make static binaries non portable since the static binary might use the non-existent lzcnt instruction if compiled on a Haswell and run on something older. Or, in other words, it would actually need to be a one time check at startup or something.
What I'm a bit confused on though, and I think I'm probably missing something here, is why even use lzcnt? The advantage of lzcnt over bsr is that lzcnt has more pleasant behavior if the argument is 0. GetHighestBitSet already has special logic for 0 though, so why not just use bsr? Or rather, why not just use __builtin_clzll (which is just 64 - bsr)? gcc always provides it, and Clang appears to always provide it on x64 (and this can easily be checked on Clang with __has_builtin(__builtin_clzll) if there was a desire to fall back to a C version if necessary).
To make what has turned into a long post short, I think the following would work and be both performant and portable (at least across x64):
    // Stores the index of the highest 1 bit and returns true. Returns
        // false and leaves highestBitSetIndex in an underterminate state
        // if the value has no bits set.
        // Uses BSR instruction. See http://felixcloutier.com/x86/BSR.html
        inline
        bool GetHighestBitSet(uint64_t value, unsigned* highestBitSetIndex)
        {
#ifdef _MSC_VER
        // VC++ provides _BitScanReverse64()
            return _BitScanReverse64(SameTargetSizeCast<BitScanType32>(highestBitSetIndex),
                                     value)
                   ? true
                   : false;
#else
            if (value == 0) {
                return false;
            }
            *highestBitSetIndex = 63 - static_cast<unsigned>(__builtin_clzll(value));
            return true;
#endif
        }

Strictly speaking, this is not actually as efficient as raw bsr would be since the subtraction would be unnecessary with the real bsr instruction (which means bsr is actually more efficient than lzcnt here, now that I think about it), but it's equally efficient to the existing code.

Thanks for commenting. It's great that we're getting so much feedback!

What I'm a bit confused on though, and I think I'm probably missing something here, is why even use lzcnt ... GetHighestBitSet already has special logic for 0 though

I think that we shouldn't use GetHighestBitSet and should just use lzcnt directly, which is issue #50 . But yes, given that we haven't fixed that (I believe that would require changing ExpressionTree.h and CallNode.cpp), it seems like we shouldn't use lzcnt here.
Before 684246d0, the code was doing basically what you suggest. I believe it was changed mostly because this seemed simpler and we (as in Bing) run entirely on newer hardware. It's probably worth changing back if we don't make the change suggested in #50 since it lets people run on older hardware without having to patch the code, but we're likely to eventually make the change in #50, which will put is back into the boat we're in right now.

Ah, it appears I should have looked into more history around this. I didn't realize the code at one point actually had used __builtin_clz.
Is the desire for GetHighestBitSet to return the operand size (i.e. 64) if no bits are set, or what is the goal of #50? I'm afraid I'm probably missing something obvious. Is lzcnt innately faster than bsr? Or is the goal to avoid having to get out two pieces of data from GetHighestBit? Or is the goal to eliminate the extra conditional for the bsr 0 special case?
Is the end goal to turn this
    unsigned r = 0;
    while (BitOp::GetHighestBitSet(xmmVolatiles, &r))

into something more like this?
unsigned r = BitOp::GetHighestBitSet(xmmVolatiles);
while (r != std::numeric_limits<decltype(xmmVolatiles)>::digits) {
     // Do stuff to xmmVolatiles
    r = BitOp::GetHighestBitSet(xmmVolatiles);
}

I hope I'm not causing a distraction or annoyance with what are probably stupid questions (please tell me if I am). I'm just intrigued by this since it seems like the code is currently using lzcnt + subtraction to essentially emulate bsr. I also have a bit of a vested interest in this since I'm wanting to use NativeJIT, but am currently weighing different work arounds since my machine doesn't support lzcnt :).

The minimal expectation of #50 is that GetHighestBitSet would return 64 and that every caller of GetHighestBitSet would, instead of checking the boolean result, would check for == 64 or  != 64, which would eliminate the extra level of indirection we have now, where if the result is 0, we set a boolean, and then we may look at that boolean later.

 I'm happy that people are interested in this project.
If you (or someone else) wants to submit a PR that replaces lzcnt with __builtin_clzll, that's fine and we would take that PR.
One day, we're going to hook up the open source version of BitFunnel to NativeJIT. If, when we do that, we find that the funny indirection described above makes a significant difference for BitFunnel's performance, then we'd try to figure out how to eliminate the extra indirection without causing problems for anyone with an older CPU who's relying on NativeJIT. If there's a PR that makes the change suggested in #50, keeps lzcnt here, and correctly falls back to other code on CPUs that don't support lzcnt, that would be even better.
If no one submits a PR, I think we'll eventually fix this, but we have some changes to BitFunnel that are higher priority for us at the moment, so it will take a while.

The short version of this post is that I'm going to put together a pull request to use __builtin_clzll in a few minutes. As long as -march=native is set, and the machine has a Haswell processor, __builtin_clzll will use lzcnt under the hood.  Otherwise, it will fall back to bsr, so the desired behavior is actually already all there.
Now that a clear path forward has been established, I don't think it's particularly necessary, but I'm still intrigued by all of this, so there's more thoughts below :).

It's strange to me that lzcnt is though to be more efficient than bsr in this situation. After all, we're wanting to find the index of the highest bit set which is what bsr does. lzcnt requires arithmetic in the form of 63 - lzcnt to get the index of the highest bit set. Also, both require special logic: bsr requires a check for zero, and lzcnt requires a check for lzcnt(value) == 64 (since otherwise, we'd do 63 - lzcnt(64) and get a -1, although... for efficiency's sake, that could be specified as the return value of GetHighestBitSet in the case no bits are set).
Unless lzcnt is way faster than bsr, I think the fastest loop possible is actually based on bsr and taking advantage that it sets the zero flag. I'm not sure it's possible to guarantee that will happen without inline assembly, but Clang actually appears to optimize the hell out of __builtin_clzll when it's using bsr under the hood:
inline
bool GetHighestBitSet(unsigned long long value, unsigned* highestBitSetIndex)
{
  if (value == 0) {
    return false;
  }
  *highestBitSetIndex = 63 - static_cast<unsigned>(__builtin_clzll(value));
  return true;
}

int use(unsigned long long val) {
	unsigned bit;
  	while (GetHighestBitSet(val, &bit)) {
      val = val & ~(1 << bit);
  	}
  	return 0;
}

use(unsigned long long):                                # @use(unsigned long long)
        test    rdi, rdi
        je      .LBB0_2
.LBB0_1:                                # =>This Inner Loop Header: Depth=1
        bsr     rcx, rdi
        mov     eax, -2
        rol     eax, cl
        cdqe
        and     rdi, rax
        jne     .LBB0_1
.LBB0_2:
        xor     eax, eax
        ret

Note that the subtraction goes away. I also think with clever restructuring of the assembly, the initial je could maybe be eliminated by simply checking the zero flag after bsr (as is already being done).
Compared to lzcnt (using -1 as the sentinel, rather than 64 -- returning 64 is even worse):
#include <x86intrin.h>
#include <limits>

inline
unsigned GetHighestBitSet(unsigned long long value)
{
  unsigned leading_zeros = __lzcnt64(value);
  return leading_zeros == 64 ? 64 : 63 - leading_zeros;
}

int use(unsigned long long val) {
	unsigned bit = GetHighestBitSet(val);
  	while (bit != std::numeric_limits<decltype(val)>::digits) {
      val = val & ~(1 << bit);
      bit = GetHighestBitSet(val);
  	}
  	return val;
}

use(unsigned long long):                                # @use(unsigned long long)
        jmp     .LBB0_2
.LBB0_1:                                #   in Loop: Header=BB0_2 Depth=1
        mov     eax, -2
        rol     eax, cl
        cdqe
        and     rdi, rax
.LBB0_2:                                # =>This Inner Loop Header: Depth=1
        lzcnt   rax, rdi
        mov     ecx, 63
        sub     ecx, eax
        cmp     ecx, -1
        jne     .LBB0_1
        xor     eax, eax
        ret

Note that the lzcnt version has to do an extra subtraction and an extra comparison, though the bsr version does have an extra test and je before entering the loop.
This is obviously an incredibly contrived example, and Clang is optimizing the loops quite cleverly in some places other than just jsr/lzcnt. For example, the bsr version checks if val is zero instead of trying to use the zero flag of bsr like I thought it would, but the lzcnt loop doesn't get a clever optimization like that -- it just does a subtract and compare. I do think it shows that bsr can be very, very efficient in this circumstance though.
Either way, it's probably not worth thinking about it all this much. My understanding of it is that these methods are only used to figure out register allocation when JITing code, so a few extra cycles is going to most likely be unnoticeable compared to if the generated code were less efficient. I just think the relation between BSR and LZCNT and BSR's weird

Thanks for all of the comments and help, everyone! 👍

Just to confirm, compiling fine now in a machine without native lzcnt. (Intel Xeon E5620).
All tests passed.

Works for me too (Intel Core i7-2600), thanks!

bsr and lzcnt are totally different instructions. For a byte 0x01, lzcnt returns 7 while bst returns 1


Upsert operation not working and job is running longer while using "Record level index" in Apache Hudi 0.14 in EMR 6.15

Describe the problem you faced
I am facing issues for upsert operation in hudi 0.14 RLI in EMR 6.15 Spark 3.4.1 using "Record level Index".
i see insert mode working as expected but upsert operation is not working with existing hudi table(lower version) and having several issues.  In short, I am getting error for hoodieException config conflict(key current value  existing value)
Hoodie.database.name : database name
I am giving the same database name(hudi db)  where data is stored as target and same db providing in source data ( hudi config) and trying to upsert but throwing this hoodie.database.name issues.
Can you please have a look and help me to fix this issues.
To Reproduce
Steps to reproduce the behavior (For me):

Spin up new EMR cluster (emr-6.15.0 with spark 3.4.1
placed the jar  hudi-spark3.4-bundle_2.12-0.14.0.jar  and spark-avro_2.13-3.5.0.jar on S3 jar location
trying to run the exact script which is running on emr-6.5.0 with Spark 3.1.2 and hudi-spark3.1.2-bundle_2.12-0.10.1.jar

Expected behavior
My expectations would be to have the same script run successfully and insert and update data into Hudi table.
Environment Description
Hudi version : 0.14.0
Spark version : 3.4.1
Hive version : 3.1.3
Hadoop version : 3.3.6
Storage (HDFS/S3/GCS..) : s3
Running on Docker? (yes/no) : no
Additional context
config I am using for upsert operation:
Upsert operation config:
hudi_operation = "upsert"
hudi_write_mode = "append"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.keygenerator.class"] = "org.apache.hudi.keygen.ComplexKeyGenerator"
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.database.name"] = tgt_db
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
hudi_config["hoodie.index.type"] = "RECORD_INDEX"
hudi_config["hoodie.datasource.write.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.precombine.field"] = "eff_fm_cent_tz"
hudi_config["hoodie.metadata.record.index.enable"] = "true"
below is code snippet :-
hudi_config = {
"className":"org.apache.hudi",
"hoodie.table.name": tgt_tbl,
"hoodie.datasource.write.recordkey.field": "id",
"hoodie.datasource.write.precombine.field": "eff_fm_cent_tz",
"hoodie.datasource.write.operation": "upsert",
"hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.ComplexKeyGenerator",
"hoodie.datasource.write.partitionpath.field": "year,month",
"hoodie.datasource.hive_sync.support_timestamp": "true",
"hoodie.datasource.hive_sync.enable": "true",
"hoodie.datasource.hive_sync.assume_date_partitioning": "false",
"hoodie.datasource.hive_sync.table": tgt_tbl,
"hoodie.datasource.hive_sync.use_jdbc": "false",
"hoodie.datasource.hive_sync.mode": "hms",
"hoodie.datasource.hive_sync.partition_extractor_class": "org.apache.hudi.hive.MultiPartKeysValueExtractor",
"hoodie.datasource.write.hive_style_partitioning": "true",
"hoodie.upsert.shuffle.parallelism": hudi_upsert_parallelism,
"hoodie.delete.shuffle.parallelism": hudi_delete_parallelism,
"hoodie.bulkinsert.sort.mode": "GLOBAL_SORT",
"hoodie.metadata.enable": "true",
"hoodie.metadata.record.index.enable": "true",
"hoodie.index.type": "RECORD_INDEX",
"hoodie.metadata.index.column.stats.column.list": "res_sys_id,pnr_rec_loc_id,pnr_cre_dt",
"hoodie.enable.data.skipping": "true"
}
hudi_delete_config = {
"hoodie.datasource.write.payload.class":
"org.apache.hudi.common.model.EmptyHoodieRecordPayload"
}
# Initialize local s3
if (files_exist == True):
print("upsert started")
hudi_operation = "upsert"
hudi_write_mode = "append"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.keygenerator.class"] = "org.apache.hudi.keygen.ComplexKeyGenerator"
hudi_config["hoodie.database.name"] = tgt_db
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
hudi_config["hoodie.index.type"] = "RECORD_INDEX"
hudi_config["hoodie.metadata.enable"] = "true"
hudi_config["hoodie.datasource.write.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.precombine.field"] = "eff_fm_cent_tz"
hudi_config["hoodie.metadata.record.index.enable"] = "true"
print("Upserting records into " + tgt_tbl + " Hudi table")
res_pnr_concat.write.format("org.apache.hudi").options(*hudi_config).mode("Append").save(hudi_tbl_path + tgt_tbl)
print("Successfully process records for " + tgt_tbl + " Hudi table")
print("res_pnr hudi loading for upsert ended ----",datetime.now())
if len(target_records) > 0:
print("delete logic started ----",datetime.now())
res_pnr_concat.createOrReplaceTempView("res_pnr_concat")
res_pnr_del_df=spark.sql("select distinct A. from res_pnr_cached A where not exists(select 1 from res_pnr_concat B where A.RES_SYS_ID = B.RES_SYS_ID AND A.PNR_REC_LOC_ID = B.PNR_REC_LOC_ID AND A.PNR_CRE_DT = B.PNR_CRE_DT AND A.EFF_FM_CENT_TZ = B.eff_fm_cent_tz and A.year = B.year and A.month = B.month)").drop("src_hoodie_record_key").drop("_hoodie_commit_seqno").drop("_hoodie_commit_time").drop("_hoodie_file_name").drop("_hoodie_partition_path").drop("_hoodie_record_key")
if len(res_pnr_del_df.take(1)) > 0:
common_config = {**hudi_config, **hudi_delete_config}
spark.sql("uncache table if exists res_pnr_src_df")
print("Deleting records from RES_PNR Hudi table")
res_pnr_del_df.write.format("org.apache.hudi").options(**common_config).mode("append").save(hudi_tbl_path + tgt_tbl)
else:
print("Delete eligible records are NOT identified")
print("delete logic Ended ----",datetime.now())
else:
pass
print("res_pnr hudi upsert logic ended ----",datetime.now())
else:
hudi_operation = "bulk_insert"
hudi_write_mode = "overwrite"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.database.name"] = tgt_db
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
hudi_config["hoodie.index.type"] = "RECORD_INDEX"
hudi_config["hoodie.metadata.enable"] = "true"
hudi_config["hoodie.datasource.write.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.precombine.field"] = "eff_fm_cent_tz"
hudi_config["hoodie.metadata.record.index.enable"] = "true"
print("bulk insert " + tgt_tbl + " Hudi table")
res_pnr_concat.write.format("org.apache.hudi").options(**hudi_config).mode(hudi_write_mode).save(hudi_tbl_path + tgt_tbl)
print("Successfully bulk upserted records into " + tgt_tbl + " Hudi table")
print("res_pnr hudi loading for upsert ended ----",datetime.now())
Upsert operation job is throwing an error:
An error was encountered:
An error occurred while calling o2149.save.
: org.apache.hudi.exception.HoodieException: Config conflict(key	current value	existing value):
hoodie.database.name:	datalake_dev1_entp_cds
at org.apache.hudi.HoodieWriterUtils$.validateTableConfig(HoodieWriterUtils.scala:211)
at org.apache.hudi.HoodieSparkSqlWriter$.writeInternal(HoodieSparkSqlWriter.scala:177)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:132)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:150)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:47)
: org.apache.hudi.exception.HoodieException: Config conflict(key	current value	existing value):
hoodie.database.name:	datalake_dev1_entp_cds
at org.apache.hudi.HoodieWriterUtils$.validateTableConfig(HoodieWriterUtils.scala:211)
at org.apache.hudi.HoodieSparkSqlWriter$.writeInternal(HoodieSparkSqlWriter.scala:177)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:132)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:150)

@SudhirSaxena As the error suggests, you are writing on existing hudi table for which database name is different. Can you confirm  value for tgt_db.
org.apache.hudi.exception.HoodieException: Config conflict(key current value existing value):
hoodie.database.name: datalake_dev1_entp_cds

tgt_db value is same datalake_dev1_entp_cds, which i am using for insert operation and upsert operation.
Currently i am facing another issue " Caused by: java.lang.StackOverflowError "  so i have increased driver stack spark.driver.extraJavaOptions=-Xss512m but still job is running longer and getting failed after 1-2 hours,
when i am running same job first time(as insert mode) so working fine and completing within 30 min.
do we need to add any additional configuration to handle upsert operation with append mode.
spark-submit --deploy-mode client --jars s3:///jars/Jars_Latest/hudi-spark3.4-bundle_2.12-0.14.0.jar,s3:///jars/Jars_Latest/spark-avro_2.13-3.5.0.jar --conf spark.sql.autoBroadcastJoinThreshold=-1 --conf spark.executors.memory=25G --conf spark.driver.memory=25G --conf spark.executors.cores=5 --conf spark.driver.cores=5 --conf spark.executor.instances=17 --conf spark.driver.maxResultSize=10G --conf spark.yarn.executor.memoryOverhead=5G --conf spark.sql.inMemoryColumnarStorage.compressed=true --conf spark.sql.files.maxPartitionBytes=2097152 --conf spark.executor.memoryOverhead=5G --conf spark.executor.extraJavaOptions=-Xss128m --conf spark.driver.extraJavaOptions=-Xss512m s3:///s3_to_hudi/res/RES_PNR_H_RLI.py --config s3:///utilities/configs/hudi_config_14.ini --env dev
error message at calling main dataframe : An error occurred while calling o1777.showString.
: java.util.concurrent.ExecutionException: Boxed Error
at scala.concurrent.impl.Promise$.resolver(Promise.scala:87)
at scala.concurrent.impl.Promise$.scala$concurrent$impl$Promise$$resolveTry(Promise.scala:79)
at scala.concurrent.impl.Promise$KeptPromise$.apply(Promise.scala:406)
at scala.concurrent.Promise$.fromTry(Promise.scala:142)
at scala.concurrent.Promise$.failed(Promise.scala:128)
at scala.concurrent.Future$.failed(Future.scala:624)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeLike.$anonfun$materializeFuture$1(ShuffleExchangeExec.scala:104)
at org.apache.spark.sql.util.LazyValue.getOrInit(LazyValue.scala:41)
at org.apache.spark.sql.execution.exchange.Exchange.getOrInitMaterializeFuture(Exchange.scala:69)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeLike.materializeFuture(ShuffleExchangeExec.scala:100)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeLike.materialize(ShuffleExchangeExec.scala:88)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeLike.materialize$(ShuffleExchangeExec.scala:87)
at org.apache.spark.sql.execution.exchange.ShuffleExchangeExec.materialize(ShuffleExchangeExec.scala:140)
at org.apache.spark.sql.execution.adaptive.ShuffleQueryStageExec.doMaterialize(QueryStageExec.scala:182)
at org.apache.spark.sql.execution.adaptive.QueryStageExec.$anonfun$materialize$1(QueryStageExec.scala:77)
at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:265)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:262)
at org.apache.spark.sql.execution.adaptive.QueryStageExec.materialize(QueryStageExec.scala:75)
at org.apache.spark.sql.execution.adaptive.MaterializeExecutable.tryStart(AdaptiveExecutable.scala:370)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutorRuntime.startChild(AdaptiveExecutor.scala:232)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper.doStart(ExecutionHelper.scala:94)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper.start(ExecutionHelper.scala:50)
at org.apache.spark.sql.execution.adaptive.QueryStageExecutable$$anon$2.$anonfun$new$1(AdaptiveExecutable.scala:251)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper.$anonfun$startAll$2(ExecutionHelper.scala:81)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper.$anonfun$startAll$2$adapted(ExecutionHelper.scala:78)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper$Listener.$anonfun$onChildSuccess$2(ExecutionHelper.scala:102)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper$Listener.$anonfun$onChildSuccess$2$adapted(ExecutionHelper.scala:101)
at scala.Option.foreach(Option.scala:407)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper$Listener.$anonfun$onChildSuccess$1(ExecutionHelper.scala:101)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper$Listener.$anonfun$onChildSuccess$1$adapted(ExecutionHelper.scala:100)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.sql.execution.adaptive.ExecutionHelper$Listener.onChildSuccess(ExecutionHelper.scala:100)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutorRuntime.$anonfun$onActiveChildSuccess$2(AdaptiveExecutor.scala:321)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutorRuntime.$anonfun$onActiveChildSuccess$2$adapted(AdaptiveExecutor.scala:321)
at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutorRuntime.onActiveChildSuccess(AdaptiveExecutor.scala:321)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutorRuntime.onChildSuccess(AdaptiveExecutor.scala:291)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutor.$anonfun$doRun$1(AdaptiveExecutor.scala:92)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutor.$anonfun$doRun$1$adapted(AdaptiveExecutor.scala:91)
at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutor.doRun(AdaptiveExecutor.scala:91)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutor.tryRunningAndGetFuture(AdaptiveExecutor.scala:66)
at org.apache.spark.sql.execution.adaptive.AdaptiveExecutor.execute(AdaptiveExecutor.scala:57)
at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.$anonfun$getFinalPhysicalPlan$1(AdaptiveSparkPlanExec.scala:275)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.getFinalPhysicalPlan(AdaptiveSparkPlanExec.scala:274)
at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.withFinalPlanUpdate(AdaptiveSparkPlanExec.scala:556)
at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.executeCollect(AdaptiveSparkPlanExec.scala:518)
at org.apache.spark.sql.Dataset.collectFromPlan(Dataset.scala:4244)
at org.apache.spark.sql.Dataset.$anonfun$head$1(Dataset.scala:3205)
at org.apache.spark.sql.Dataset.$anonfun$withAction$2(Dataset.scala:4234)
at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:570)
at org.apache.spark.sql.Dataset.$anonfun$withAction$1(Dataset.scala:4232)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:250)
at org.apache.spark.sql.execution.SQLExecution$.executeQuery$1(SQLExecution.scala:123)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$9(SQLExecution.scala:160)
at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:107)
at org.apache.spark.sql.execution.SQLExecution$.withTracker(SQLExecution.scala:250)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$8(SQLExecution.scala:160)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:271)
at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:159)
at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:827)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:69)
at org.apache.spark.sql.Dataset.withAction(Dataset.scala:4232)
at org.apache.spark.sql.Dataset.head(Dataset.scala:3205)
at org.apache.spark.sql.Dataset.take(Dataset.scala:3426)
at org.apache.spark.sql.Dataset.getRows(Dataset.scala:286)
at org.apache.spark.sql.Dataset.showString(Dataset.scala:325)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:182)
at py4j.ClientServerConnection.run(ClientServerConnection.java:106)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.StackOverflowError
at org.apache.spark.sql.catalyst.expressions.CodeGeneratorWithInterpretedFallback.createObject(CodeGeneratorWithInterpretedFallback.scala:53)

I hope this message finds you well. I have a question regarding the Bloom index in Hudi. Were you using the Bloom index previously, and have you recently changed the index type to a record-level index (RLI)? I am curious about the scenario where a Hudi table already exists, and you are attempting to modify the index type. My understanding is that this action could result in an error unless it is a completely new table with an RLI. Could you kindly confirm if my interpretation is accurate or provide any additional insights on this matter? Your assistance is greatly appreciated

Hi @soumilshah1995 ,
yes, we are currently using bloom index (below hudi configuration given) hudi lower version(0.10) and it's working but somewhat slow performance, so we are trying to use RLI to improve performance. as of now, we are not integrating with existing Hudi table, and trying to load in new table to see how it works, if it will work as expected then will see later how to integrate or use for new tables.
spark command:
spark-submit --deploy-mode client --jars s3:///jars/hudi-spark3.1.2-bundle_2.12-0.10.1.jar,s3:///jars/spark-avro_2.12-3.1.2.jar --conf spark.sql.autoBroadcastJoinThreshold=-1 --conf spark.executors.memory=25G --conf spark.driver.memory=25G --conf spark.executors.cores=5 --conf spark.driver.cores=5 --conf spark.executor.instances=21 --conf spark.driver.maxResultSize=5G --conf spark.yarn.executor.memoryOverhead=5G --conf spark.sql.inMemoryColumnarStorage.compressed=true --conf spark.sql.files.maxPartitionBytes=2097152 --conf spark.executor.memoryOverhead=5G --conf spark.executor.defaultJavaOptions=-XX:+UseG1GC --conf spark.driver.defaultJavaOptions=-XX:+UseG1GC s3:///RES_PAX_AIR.py --config s3:///utilities/configs/hudi_config.ini --env dev
hudi_config = {
"className":"org.apache.hudi",
"hoodie.table.name": tgt_tbl,
"hoodie.datasource.write.recordkey.field": "id",
"hoodie.datasource.write.precombine.field": "eff_fm_cent_tz",
"hoodie.datasource.write.operation": "upsert",
"hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.ComplexKeyGenerator",
"hoodie.datasource.write.partitionpath.field": "year,month",
"hoodie.datasource.hive_sync.support_timestamp": "true",
"hoodie.datasource.hive_sync.enable": "true",
"hoodie.datasource.hive_sync.assume_date_partitioning": "false",
"hoodie.datasource.hive_sync.table": tgt_tbl,
"hoodie.datasource.hive_sync.use_jdbc": "false",
"hoodie.datasource.hive_sync.mode": "hms",
"hoodie.datasource.hive_sync.partition_extractor_class": "org.apache.hudi.hive.MultiPartKeysValueExtractor",
"hoodie.datasource.write.hive_style_partitioning": "true",
"hoodie.upsert.shuffle.parallelism": hudi_upsert_parallelism,
"hoodie.delete.shuffle.parallelism": hudi_delete_parallelism,
"hoodie.bulkinsert.sort.mode": "GLOBAL_SORT",
"hoodie.index.type": "BLOOM",
"hoodie.metadata.enable": "true",
"hoodie.metadata.index.bloom.filter.enable": "true",
"hoodie.metadata.index.bloom.filter.parallelism": hudi_filter_parallelism,
"hoodie.metadata.index.bloom.filter.column.list": "id",
"hoodie.bloom.index.use.metadata": "true",
"hoodie.metadata.index.column.stats.enable": "true",
"hoodie.metadata.index.column.stats.column.list": "res_sys_id,pnr_rec_loc_id,pnr_cre_dt",
"hoodie.enable.data.skipping": "true"
}
upsert:
hudi_operation = "upsert"
hudi_write_mode = "append"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.keygenerator.class"]="org.apache.hudi.keygen.ComplexKeyGenerator"
hudi_config["hoodie.database.name"]=tgt_db
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
print("Upserting records into " + tgt_tbl + " Hudi table")
bulk_insert:
hudi_operation = "bulk_insert"
hudi_write_mode = "overwrite"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db

I would like to bring to your attention a potential issue that may have arisen during the transition from the Bloom index to the Record Level index. It seems that the Record Level index was not built on a fresh table, which could be the reason for the error you are encountering. I kindly suggest creating a new table with the Index Type set to Record Level index to address this issue. Thank you for your attention to this matter.

didn't you mentioned you have changed the index type Bloom -> RLI
and your job was failing in order to use RLI I am saying you need to use fresh table

Hi @soumilshah1995 ,
i am not using any existing hudi table to do transition from Bloom index to the Record Level index. since it's new version so have to test first, hence i am trying to implement with fresh new table.

i am able to insert the data into Hudi table(fresh new table) using Record level index (hudi 0.14 with EMR 6.15). it takes max 20-25 min to complete the job.
when i am running 2nd time to perform upsert operation, it's keep running and not at all completing, looks like job is not able to fetch the data from Hudi target table, and keep running, i have cancelled the job after 2-3 hours, it's stuck the same df more then 2 hours.

Hudi properties for Record level index :
hudi_config = {
"className":"org.apache.hudi",
"hoodie.table.name": tgt_tbl,
"hoodie.datasource.write.recordkey.field": "id",
"hoodie.datasource.write.precombine.field": "eff_fm_cent_tz",
"hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.ComplexKeyGenerator",
"hoodie.datasource.write.partitionpath.field": "year,month",
"hoodie.datasource.hive_sync.support_timestamp": "true",
"hoodie.datasource.hive_sync.enable": "true",
"hoodie.datasource.hive_sync.assume_date_partitioning": "false",
"hoodie.datasource.hive_sync.table": tgt_tbl,
"hoodie.datasource.hive_sync.use_jdbc": "false",
"hoodie.datasource.hive_sync.mode": "hms",
"hoodie.datasource.hive_sync.partition_extractor_class": "org.apache.hudi.hive.MultiPartKeysValueExtractor",
"hoodie.datasource.write.hive_style_partitioning": "true",
"hoodie.upsert.shuffle.parallelism": hudi_upsert_parallelism,
"hoodie.delete.shuffle.parallelism": hudi_delete_parallelism,
"hoodie.bulkinsert.sort.mode": "GLOBAL_SORT",
"hoodie.metadata.enable": "true",
"hoodie.metadata.record.index.enable": "true",
"hoodie.index.type": "RECORD_INDEX",
"hoodie.metadata.index.column.stats.column.list": "res_sys_id,pnr_rec_loc_id,pnr_cre_dt",
"hoodie.enable.data.skipping": "true"
}
insert operation:
hudi_operation = "bulk_insert"
hudi_write_mode = "overwrite"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.database.name"] = tgt_db
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
hudi_config["hoodie.index.type"] = "RECORD_INDEX"
hudi_config["hoodie.metadata.enable"] = "true"
hudi_config["hoodie.datasource.write.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.precombine.field"] = "eff_fm_cent_tz"
hudi_config["hoodie.metadata.record.index.enable"] = "true"
res_pnr_df.write.format("org.apache.hudi").options(**hudi_config).mode(hudi_write_mode).save(hudi_tbl_path + tgt_tbl)
Upsert operation config:
hudi_operation = "upsert"
hudi_write_mode = "append"
hudi_config["className"] = "org.apache.hudi"
hudi_config["hoodie.table.keygenerator.class"] = "org.apache.hudi.keygen.ComplexKeyGenerator"
hudi_config["hoodie.table.name"] = tgt_tbl
hudi_config["hoodie.database.name"] = tgt_db
hudi_config["hoodie.datasource.write.recordkey.field"] = "id"
hudi_config["hoodie.datasource.write.operation"] = hudi_operation
hudi_config["hoodie.datasource.hive_sync.table"] = tgt_tbl
hudi_config["hoodie.datasource.hive_sync.database"] = tgt_db
hudi_config["hoodie.index.type"] = "RECORD_INDEX"
hudi_config["hoodie.metadata.enable"] = "true"
hudi_config["hoodie.datasource.write.table.name"] = tgt_tbl
hudi_config["hoodie.datasource.write.precombine.field"] = "eff_fm_cent_tz"
hudi_config["hoodie.metadata.record.index.enable"] = "true"
res_pnr_df.write.format("org.apache.hudi").options(**hudi_config).mode("append").save(hudi_tbl_path + tgt_tbl)

Got it got it
Can we do a small test here
Can you run job in UPSERT mode and not INSERT at start and then see if it makes any difference
I know UPSERT fresh table RLI index would be build for sure
could we do this test ?

so you mean i shouldn't give bulk_insert operation option in job, just run to check for fresh table.
hudi_operation = "bulk_insert"
one question : for testing upsert only for new table, first time i will run as upsert hudi operation and if job completes then again i will run as upsert operation . is that correct ?

yes without bulk_insert
use UPSERT remove settings bulk_insert settings

@soumilshah1995  @ad1happy2go
i have run this job as upsert operation first time to load the data into fresh Hudi table and it's completed in 7 minutes(attached screenshot). I have verified data into Hudi table and it's 2003 records.
now, when i am running again as upsert operation to do actual upsert so job is keep running past 40 mins and not finishing.not sure why upsert operation is creating an issues and how to fix this issues.
Can you please help me on this to fix ?

@soumilshah1995  @ad1happy2go  @nsivabalan
just sharing hoodie.properties and metadata/hoodie.properties. I see bydefault it's taking hoodie.table.type=MERGE_ON_READ in .hoodie/metadata/.hoodie/hoodie.properties and hoodie.table.type=COPY_ON_WRITE in .hoodie/hoodie.properties . please have a look if there is any changes required.
s3://*/res_pnr_upsert_test/.hoodie/hoodie.properties
#Updated at 2024-01-31T00:02:47.700Z
#Wed Jan 31 00:02:47 UTC 2024
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.table.precombine.field=eff_fm_cent_tz
hoodie.table.version=6
hoodie.database.name=datalake_dev1_entp_cds
hoodie.datasource.write.hive_style_partitioning=true
hoodie.table.metadata.partitions.inflight=
hoodie.table.checksum=966479762
hoodie.partition.metafile.use.base.format=false
hoodie.table.cdc.enabled=false
hoodie.archivelog.folder=archived
hoodie.table.name=res_pnr_upsert_test
hoodie.populate.meta.fields=true
hoodie.table.type=COPY_ON_WRITE
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=false
hoodie.table.metadata.partitions=files,record_index
hoodie.timeline.layout.version=1
hoodie.table.partition.fields=year,month
hoodie.table.recordkey.fields=id
s3://*/res_pnr_upsert_test/.hoodie/metadata/.hoodie/hoodie.properties
#Properties saved on 2024-01-31T00:02:33.236Z
#Wed Jan 31 00:02:33 UTC 2024
hoodie.datasource.write.drop.partition.columns=false
hoodie.table.type=MERGE_ON_READ
hoodie.archivelog.folder=archived
hoodie.populate.meta.fields=false
hoodie.compaction.payload.class=org.apache.hudi.metadata.HoodieMetadataPayload
hoodie.timeline.layout.version=1
hoodie.table.version=6
hoodie.table.base.file.format=HFILE
hoodie.table.recordkey.fields=key
hoodie.table.name=res_pnr_upsert_test_metadata
hoodie.table.keygenerator.class=org.apache.hudi.metadata.HoodieTableMetadataKeyGenerator
hoodie.compaction.record.merger.strategy=eeb8d96f-b1e4-49fd-bbf8-28ac514178e5
hoodie.table.checksum=2433971754

@SudhirSaxena Metadata table is always MOR, so that's expected. Can you identify from spark UI where exactly the job is taking time.

Hi @ad1happy2go  , @soumilshah1995  ,@nsivabalan
I am trying to see where job is getting stuck . i see driver which is in Executor id summary (below screenshot) is running more than 1 hour and not moving. I am not sure what could be the reason and why it's happening. any idea why it's happening and how to resolve this to run the job for upsert operation using Record level index in hudi 0.14 in EMR 6.15.
appreciate your help on this.

Executor ID
Address
Status
RDD Blocks
Storage Memory
Disk Used
Cores
Active Tasks
Failed Tasks
Complete Tasks
Total Tasks
Task Time (GC Time)
Input
Shuffle Read
Shuffle Write
Logs

driver
ip-10-156-17-51.ec2.internal:41585
Active
0
0.0 B / 15.8 GiB
0.0 B
0
0
0
0
0
1.2 h (0.0 ms)
0.0 B
0.0 B
0.0 B

That's strange! looks like it has stalled on driver. Can you check driver logs during this time.

let me check now

@SudhirSaxena The above are driver logs only. But can you see what logs it is printing during that one hour period. We can sync up on slack and connect on call also to understand it more.

@ad1happy2go , it's printing the same which i have posted above even for 2-3 hours, creating task, finishing task but there is no actual execution happening, even there is no executor_id active in spark-UI ,
sure, we can connect over call. Can you please schedule time for call, i will connect or let me know how to connect so that i can do working session with you/your team. thanks

Hi @ad1happy2go , @soumilshah1995 ,@nsivabalan,
thought of checking for Hudi upsert operation issues, did you get a chance to check on this. i am still facing the same issues, insert is running fine, when i am running 2nd time same job to perform upsert operation, it's running for forever and not completing. looking like driver is running, creating task and finishing but executor side no task is getting creating so executor is idle it seems as per Spark-ui.
will you be able to help on this. appreciate for your help.

@ad1happy2go as discussed for this issue, please find the driver logs,
24/02/06 05:04:07 INFO MemoryStore: 2 blocks selected for dropping (468.0 KiB bytes)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_12999_piece0 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_12999_piece0 to disk
24/02/06 05:04:07 INFO BlockManagerInfo: Updated broadcast_12999_piece0 on disk on ip-10-156-17-116.ec2.internal:35559 (current size: 47.4 KiB, original size: 0.0 B)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13000 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13000 to disk
24/02/06 05:04:07 INFO MemoryStore: After dropping 2 blocks, free memory is 1342.0 KiB
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48457 stored as values in memory (estimated size 420.5 KiB, free 921.5 KiB)
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48457_piece0 stored as bytes in memory (estimated size 47.4 KiB, free 874.0 KiB)
24/02/06 05:04:07 INFO BlockManagerInfo: Added broadcast_48457_piece0 in memory on ip-10-156-17-116.ec2.internal:35559 (size: 47.4 KiB, free: 14.2 GiB)
24/02/06 05:04:07 INFO SparkContext: Created broadcast 48457 from take at /mnt/tmp/spark-7f0d7e5b-8199-405d-aadf-6da6fd1c5cd0/RES_PNR_prod_HRLI.py:1191
24/02/06 05:04:07 INFO MemoryStore: 2 blocks selected for dropping (468.0 KiB bytes)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13000_piece0 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13000_piece0 to disk
24/02/06 05:04:07 INFO BlockManagerInfo: Updated broadcast_13000_piece0 on disk on ip-10-156-17-116.ec2.internal:35559 (current size: 47.4 KiB, original size: 0.0 B)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13001 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13001 to disk
24/02/06 05:04:07 INFO MemoryStore: After dropping 2 blocks, free memory is 1342.0 KiB
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48458 stored as values in memory (estimated size 420.5 KiB, free 921.5 KiB)
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48458_piece0 stored as bytes in memory (estimated size 47.4 KiB, free 874.0 KiB)
24/02/06 05:04:07 INFO BlockManagerInfo: Added broadcast_48458_piece0 in memory on ip-10-156-17-116.ec2.internal:35559 (size: 47.4 KiB, free: 14.2 GiB)
24/02/06 05:04:07 INFO SparkContext: Created broadcast 48458 from take at /mnt/tmp/spark-7f0d7e5b-8199-405d-aadf-6da6fd1c5cd0/RES_PNR_prod_HRLI.py:1191
24/02/06 05:04:07 INFO MemoryStore: 2 blocks selected for dropping (468.0 KiB bytes)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13001_piece0 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13001_piece0 to disk
24/02/06 05:04:07 INFO BlockManagerInfo: Updated broadcast_13001_piece0 on disk on ip-10-156-17-116.ec2.internal:35559 (current size: 47.4 KiB, original size: 0.0 B)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13002 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13002 to disk
24/02/06 05:04:07 INFO MemoryStore: After dropping 2 blocks, free memory is 1342.0 KiB
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48459 stored as values in memory (estimated size 420.5 KiB, free 921.5 KiB)
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48459_piece0 stored as bytes in memory (estimated size 47.4 KiB, free 874.0 KiB)
24/02/06 05:04:07 INFO BlockManagerInfo: Added broadcast_48459_piece0 in memory on ip-10-156-17-116.ec2.internal:35559 (size: 47.4 KiB, free: 14.2 GiB)
24/02/06 05:04:07 INFO SparkContext: Created broadcast 48459 from take at /mnt/tmp/spark-7f0d7e5b-8199-405d-aadf-6da6fd1c5cd0/RES_PNR_prod_HRLI.py:1191
24/02/06 05:04:07 INFO MemoryStore: 2 blocks selected for dropping (468.0 KiB bytes)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13002_piece0 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13002_piece0 to disk
24/02/06 05:04:07 INFO BlockManagerInfo: Updated broadcast_13002_piece0 on disk on ip-10-156-17-116.ec2.internal:35559 (current size: 47.4 KiB, original size: 0.0 B)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13003 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13003 to disk
24/02/06 05:04:07 INFO MemoryStore: After dropping 2 blocks, free memory is 1342.0 KiB
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48460 stored as values in memory (estimated size 420.5 KiB, free 921.5 KiB)
24/02/06 05:04:07 INFO MemoryStore: Block broadcast_48460_piece0 stored as bytes in memory (estimated size 47.4 KiB, free 874.0 KiB)
24/02/06 05:04:07 INFO BlockManagerInfo: Added broadcast_48460_piece0 in memory on ip-10-156-17-116.ec2.internal:35559 (size: 47.4 KiB, free: 14.2 GiB)
24/02/06 05:04:07 INFO SparkContext: Created broadcast 48460 from take at /mnt/tmp/spark-7f0d7e5b-8199-405d-aadf-6da6fd1c5cd0/RES_PNR_prod_HRLI.py:1191
24/02/06 05:04:07 INFO MemoryStore: 2 blocks selected for dropping (468.0 KiB bytes)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13003_piece0 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13003_piece0 to disk
24/02/06 05:04:07 INFO BlockManagerInfo: Updated broadcast_13003_piece0 on disk on ip-10-156-17-116.ec2.internal:35559 (current size: 47.4 KiB, original size: 0.0 B)
24/02/06 05:04:07 INFO BlockManager: Dropping block broadcast_13004 from memory
24/02/06 05:04:07 INFO BlockManager: Writing block broadcast_13004 to disk

Had a conversation with @SudhirSaxena on this and looked at his setup. He is using emr-6.15 with OSS hudi 0.14.0.

With RLI enabled, the upsert job is getting stuck for hours, no progress. Also no useful logs. No running stage in UI.
driver Logs - Above comment.
We tried with RLI disabled keeping everything else same, But similar behaviour. So, RLI may not have any issue.

Next steps -

Try creating a test script which do bulk insert and insert from quickstart and see if its working.
Try same setup with 0.14.1 version.
Try degrading EMR version which supports spark 3.3. Try running hudi 0.14.0 if we see same behaviour.
Then just degrade hudi version to 0..12.3 which was used before and confirm if that works fine.

Thanks @ad1happy2go  , i will follow these steps, and let you know..

@SudhirSaxena Did you get a chance to try this out. Thanks in advance for all the contribution on this.

hey @ad1happy2go : do let me know if we find any data consistency issues w/ MDT or RLI. thanks.


Bug with wires in 1.12.2

Link to the bug IC2's issue tracker
https://bt.industrial-craft.net/view.php?id=1959
https://bt.industrial-craft.net/view.php?id=2027
these reports are from 1.10 but I have this problem on 1.12.2
Describe the bug
When you re-enter the server, the machines and wires lose connection with each other and can no longer make it.
To Reproduce
Minecraft: 1.12.2
IC2: 2.8.222-ex112
Place a couple of machines on the server.
Connect them with wires.
Logout from the server.
Login to the server.
The wires are disconnected and until the server is restarted / chunk data is uploaded they will not be fixed.
Expected behavior
The wires lose connections with each other, if you remove them and put them back in, everything is ok. BUT then they don’t connect to the machines and they also have to be removed and put back in again (Or unload the chunk and enter it again)
Additional context

There is also some spam from the server console
[10:22:32] [ic2-poolthread-6/WARN] [ic2.EnergyNet]: Tile ic2.core.block.wiring.TileEntityCable@31466589 (dim 0: -927/64/58), sub tile ic2.core.block.wiring.TileEntityCable@31466589 (dim 0: -927/64/58) addition is conflicting with a previous registration at the same location: ic2.core.block.wiring.TileEntityCable@5c838cb8.
[10:22:32] [ic2-poolthread-6/WARN] [ic2.EnergyNet]: Tile ic2.core.block.wiring.TileEntityCable@490bd5aa (dim 0: -926/64/58), sub tile ic2.core.block.wiring.TileEntityCable@490bd5aa (dim 0: -926/64/58) addition is conflicting with a previous registration at the same location: ic2.core.block.wiring.TileEntityCable@31737fee.
[10:22:32] [ic2-poolthread-6/WARN] [ic2.EnergyNet]: Tile ic2.core.block.wiring.TileEntityCable@4cc4200d (dim 0: -924/64/58), sub tile ic2.core.block.wiring.TileEntityCable@4cc4200d (dim 0: -924/64/58) addition is conflicting with a previous registration at the same location: ic2.core.block.wiring.TileEntityCable@67c2793c.

I am unable to reproduce this issue on plain IC2 2.8.222 server + client >.>
Does this still happen with IC2 Patcher installed? We have a patch for world data issues like this when you reload a world on singleplayer. And as I see Thermal there, do you have AdvancedRocketry installed and any rocket entities on the server?

I apologize for not providing a list of mods earlier.
AdvancedRocketry - Not installed

Does this still happen with IC2 Patcher installed?

Yes

Mod list: (I can upload the archive for you, but only after 10 hours, unfortunately)
!mixinbooter-8.8.jar
additionalpipes-<IP_ADDRESS>.jar
Advanced Machines-61.0.1.jar
Advanced Solar Panels-4.3.0.jar
AdvMachinesPatch-1.1.jar
AdvSolarPatch-1.2.1.jar
ae2stuff-<IP_ADDRESS>-mc1.12.2.jar
AE2WTLib-1.12.2-1.0.34.jar
AppleSkin-mc1.12-1.0.14.jar
appliedenergistics2-rv6-stable-7.jar
AsmodeusCore-1.12.2-1.0.0.jar
bdlib-<IP_ADDRESS>-mc1.12.2.jar
BetterBuildersWands-1.12-<IP_ADDRESS>+69d0d70.jar
blockdrops-1.12.2-1.4.0.jar
buildcraft-all-<IP_ADDRESS>.jar
Chisel-MC1.12.2-<IP_ADDRESS>.jar
ClientTweaks_1.12.2-3.1.11.jar
CodeChickenLib-1.12.2-<IP_ADDRESS>8-universal.jar
CoFHCore-1.12.2-<IP_ADDRESS>-universal.jar
CoFHWorld-1.12.2-<IP_ADDRESS>-universal.jar
ComboArmors-1.12.2-0.0.9.jar
Crops++[1.12.2]1.12.2-0.0.1-1.12.2-0.0.1.jar
CTM-MC1.12.2-<IP_ADDRESS>.jar
EnergyControl-1.12.2-0.3.1.jar
energyconverters_1.12.2-<IP_ADDRESS>.jar
extrautils2-1.12-1.9.9.jar
FastFurnace-1.12.2-1.3.1.jar
FastWorkbench-1.12.2-1.7.4.jar
forestry_1.12.2-<IP_ADDRESS>7.jar
ForgeMultipart-1.12.2-<IP_ADDRESS>-universal.jar
ftb-ultimine-1202.3.5.jar
FTBLib-<IP_ADDRESS>.jar
FTBUtilities-<IP_ADDRESS>.jar
Galacticraft-1.12.2-4.0.6.jar
GalaxySpace-1.12.2-2.1.1.jar
GraviSuite-Patcher-1.0.0.jar
Gravitation Suite-3.1.1.jar
Hwyla-1.8.26-B41_1.12.2.jar
IC2CropBreeding Plugin V1.12-1.1.jar
ic2magma-1.0.jar
ic2patcher-2.0.7.jar
ImmersiveCables-1.12.2-1.3.2.jar
ImmersiveEngineering-0.12-98.jar
immersivepetroleum-1.12.2-1.1.10.jar
ImmersivePosts-0.2.1.jar
industrialcraft-2-2.8.222-ex112.jar
InventoryTweaks-1.63.jar
ironchest-1.12.2-<IP_ADDRESS>7.jar
jei_1.12.2-<IP_ADDRESS>1.jar
jeibees-<IP_ADDRESS>-mc1.12.2.jar
jeiintegration_1.12.2-1.6.0.jar
jeivillagers-1.12-1.0.2.jar
jepb-1.12-1.2.1.jar
journeymap-1.12.2-5.7.1.jar
JustEnoughCalculation-1.12.2-3.2.7.jar
JustEnoughEnergistics-1.12.2-1.0.8.jar
JustEnoughPetroleum-0.1.jar
JustEnoughResources-1.12.2-<IP_ADDRESS>.jar
logisticspipes-<IP_ADDRESS>.jar
malisiscore-1.12.2-6.5.1.jar
malisisdoors-1.12.2-7.3.0.jar
Mantle-1.12-<IP_ADDRESS>.jar
More-Planets-1.12.2-2.3.2-GC4.0.5.jar
MouseTweaks-2.10-mc1.12.2.jar
MrTJPCore-1.12.2-<IP_ADDRESS>-universal.jar
p455w0rdslib-1.12.2-2.3.161.jar
Placebo-1.12.2-1.6.1.jar
ProjectRed-1.12.2-<IP_ADDRESS>-Base.jar
ProjectRed-1.12.2-<IP_ADDRESS>-compat.jar
ProjectRed-1.12.2-<IP_ADDRESS>-fabrication.jar
ProjectRed-1.12.2-<IP_ADDRESS>-integration.jar
ProjectRed-1.12.2-<IP_ADDRESS>-lighting.jar
ProjectRed-1.12.2-<IP_ADDRESS>-mechanical.jar
ProjectRed-1.12.2-<IP_ADDRESS>-world.jar
railcraft-12.0.0.jar
RedstoneFlux-1.12-<IP_ADDRESS>-universal.jar
secretroomsmod-1.12.2-5.6.4.jar
SteveKunG's-Lib-mc1.12.2-v1.3.0.jar
TConstruct-1.12.2-<IP_ADDRESS>.jar
ThermalCultivation-1.12.2-<IP_ADDRESS>-universal.jar
ThermalDynamics-1.12.2-<IP_ADDRESS>-universal.jar
ThermalExpansion-1.12.2-<IP_ADDRESS>-universal.jar
ThermalFoundation-1.12.2-<IP_ADDRESS>-universal.jar
ThermalInnovation-1.12.2-<IP_ADDRESS>-universal.jar
TinkersComplement-1.12.2-0.4.3.jar
tinkersjei-1.2.jar
TinkerToolLeveling-1.12.2-1.1.0.jar
TravelersBackpack-1.12.2-1.0.35.jar
UniDict-1.12.2-3.0.10.jar
WanionLib-1.12.2-2.91.jar
Waystones_1.12.2-4.1.0.jar
WirelessFluidTerminal-1.12.2-1.0.4.jar
WirelessInterfaceTerminal-1.12.2-1.0.2.jar
YARCF-0.14(1.12.2).jar

I might have an idea of what is happening... Mind sending over the modpack + save, or trying out the dev jar of IC2 Patcher 😅
If you don't want to send it over here, feel free to write to me on discrord, @kanzaji. Gonna send the dev jar with potential fix when ya reply 😄

@Kanzaji I sent a request on discord. I'll send the mods as an archive and download the world now

I recall @Speiger recently (and finally) resolving this on 1.12.2's IC2 Classic, so he might have an idea as well.

I will ask him about it 👍 Let's hope he will give me some clues on how to fix it in IC2 Exp - Decompilling Classic and trying to compare it to exp would be a pain otherwise >.>


Open Dev Terminal is a no op

When the menu is selected, nothing is happening.
I have this in the Podman Desktop log:
Error: Unknown command: crc.dev.terminal
    at W6.executeCommand (C:\Users\demo\AppData\local\Programs\podman-desktop\resources\app.asar\packages\main\dist\index.cjs:14:41998)
    at IpcMainImpl.<anonymous> (C:\Users\demo\AppData\local\Programs\podman-desktop\resources\app.asar\packages\main\dist\index.cjs:91:38747)
    at IpcMainImpl.emit (node:events:513:28)
    at i.menuItem.click (C:\Users\demo\AppData\local\Programs\podman-desktop\resources\app.asar\packages\main\dist\index.cjs:14:33918)
    at MenuItem.click (node:electron/js2c/browser_init:2:30795)
    at a._executeCommand (node:electron/js2c/browser_init:2:36191)

Hmmmm... interesting...

I have able to open the terminal:
https://github.com/crc-org/crc-extension/blob/3acf5090000385204a388d20023ab12013586951/src/dev-terminal.ts#L34

Need more information

Works for me with Podman Desktop 1.1.0 and extension 1.0.3


tests/raftstore: add a retry argument for call_command_on_leader

call_command_on_leader actually has two step:

get leader
2.call command on leader
with retry_cnt we can control actions of step1

I think this change is unnecessary except replacing Timeout error.


Using glightbox stops the default search box appearing

If I add the following to my mkdocs.yml:
plugins:
 - glightbox

...then glightbox works as expected - I can click on images to zoom into them, etc. However, the Search field at the top-right of the page disappears. If I remove those lines, the Search field is back. How can I have glightbox and Search at the same time?

Thanks!


Roll engine 0c9c293b56cc..9bc5e0bea5d2 (2 commits)

https://github.com/flutter/engine/compare/0c9c293b56cc...9bc5e0bea5d2
git log 0c9c293b56cc588b28ea985ccecc3935ffcf9a4a..9bc5e0bea5d2af40a1f488f5e8bb2bb8450a5c4c --no-merges --oneline
9bc5e0bea Roll src/third_party/dart e70273c306..9a07ad88f4 (5 commits)
1502300cc Log the sticky error during isolate shutdown (flutter/engine#8742)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (liyuqian@google.com), and stop
the roller if necessary.

Trybots failed. These were the failed builds: https://cirrus-ci.com/task/5431612119449600

Commit queue failed; closing this roll.


Improve local search method

Use the approximated derivative from APF to direct the evaluations.
Probably useful only for camera position.

Added a randomly placed center point

Now using APF for position optimisation.
APF is used only to direct hill climbing towards better solutions


Option to clean between each run

I'm attempting to use cargo hack test --feature-powerset on a project, but given the sheer number of combinations this results in exceeding the available disk space on Github Actions with all the resulting test binaries.
If it were possible to run cargo clean -p <project> between each test run, that would delete the test binaries but leave the precompiled dependencies behind. It's probably not wanted as the default since it would slow down rerunning cargo hack --feature-powerset locally when you are able to cache all the results, but being an option would be very useful for CI.

(Though TBH I think given the 512 feature combinations in this project I won't be able to use test --feature-powerset in CI anyway because of the time it takes, but maybe this would be useful for other projects that are closer to being able to run the full set except for disk space issues).

It may not always be enough in a project where features fine control over dependencies' features, but I think this option is useful to some extent for this use case.
Also, I think this might help as a workaround for https://github.com/rust-lang/rust-clippy/issues/4612. (Probably makes sense to run cargo clean -p "before each run" rather than "between each run".)

(Though TBH I think given the 512 feature combinations in this project I won't be able to use test --feature-powerset in CI anyway because of the time it takes, but maybe this would be useful for other projects that are closer to being able to run the full set except for disk space issues).

AFAIK, if features are additive, then the test only needs to run with --all-features once.
Also, If check if the test compile under all --feature combinations, it doesn't need to actually run the test and  I think check --all-targets or test --no-run is fine. (This probably needs to be separated from checking of the library/binary itself, as dev-dependencies may enable features.)
If features are non-additive, ideally it is preferable to run the test for each feature, but test --feature-powerset takes too long, so there is probably no way yet to do this easily. (However, I don't think such features are good for libraries in many cases, as other dependencies may enable it.)
Making --skip works for --all-features is probably one of the reasonable solutions for this. If one feature is non-additive and other features are additive (e.g., using different implementations between std and no-std), I think this is enough.

This probably needs to be separated from checking of the library/binary itself, as dev-dependencies may enable features.

Ah, that's a good point. I had reduced my usage to running check --all-targets for the feature-sets since that only takes 18 minutes, and only running the tests once with --all-features, but didn't think about having to still have a separate check --all-targets --no-dev-deps.

Published in 0.3.11 (#49)


Fix sha256, bump build number, update maintainers, rerender

Last merged version to master was not published. Update to fix that.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

bemeurer is now lovesegfault, as indicated by the linter in https://github.com/conda-forge/beautysh-feedstock/pull/3

Ok. Builds now. Ready to merge :)

Only @bollwyvl can merge this, @dbast. I left the conda-forge org because of the tremendous amount of noise it created on my Travis panel.

Sorry, not at my keyboard at the moment. Will address this evening. Thanks
for the ping!
On Thu, May 23, 2019, 16:30 Bernardo Meurer<EMAIL_ADDRESS>wrote:

Only @bollwyvl https://github.com/bollwyvl can merge this, @dbast
https://github.com/dbast. I left the conda-forge org because of the
tremendous amount of noise it created on my Travis panel.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/conda-forge/beautysh-feedstock/pull/4?email_source=notifications&email_token=AAALCRG4BCRFQXCMPD5NVQLPW35GBA5CNFSM4HPGKTEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWDMZ7I#issuecomment-495373565,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAALCRHHMWNG35KCF5O7BX3PW35GBANCNFSM4HPGKTEA
.

Just to be a tiny bit picky, setuptools is a dependency of pip, so not
strictly necessary in host, but otherwise looks good!
On Thu, May 23, 2019, 17:07 Nicholas Bollweg<EMAIL_ADDRESS>wrote:

Sorry, not at my keyboard at the moment. Will address this evening. Thanks
for the ping!
On Thu, May 23, 2019, 16:30 Bernardo Meurer<EMAIL_ADDRESS>wrote:

Only @bollwyvl https://github.com/bollwyvl can merge this, @dbast
https://github.com/dbast. I left the conda-forge org because of the
tremendous amount of noise it created on my Travis panel.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/conda-forge/beautysh-feedstock/pull/4?email_source=notifications&email_token=AAALCRG4BCRFQXCMPD5NVQLPW35GBA5CNFSM4HPGKTEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODWDMZ7I#issuecomment-495373565,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAALCRHHMWNG35KCF5O7BX3PW35GBANCNFSM4HPGKTEA
.

Ok, right. Updated once again :)

Thanks @dbast! Will monitor the build and do... something if it doesn't work again.

Everything looks on the up-and-up!

Thanks!


Sensor Disconnected On Connection Beaglebone Black Ubuntu

Hi Sandeep,
I tried to run the test.js. It is discovering the sensor and immediately shutting down.
I have a beaglebone black, with a kinivo btd 400 usb ble 4.0. I have installed ubuntu
I have installed bluez
Linux arm 4.1.5-ti-r10 #1 SMP PREEMPT Thu Aug 13 20:43:10 UTC 2015 armv7l armv7l armv7l GNU/Linux

ubuntu@arm:~/sandytest/node-sensortag$ sudo node test.js
discovered: {"id":"68c90b17cd0c","type":"cc2650"}
connectAndSetUp
readDeviceName
	device name = SensorTag 2.0
readSystemId
	system id = 68:c9:0b:00:00:17:cd:0c
readSerialNumber
	serial number = N.A.
readFirmwareRevision
	firmware revision = 1.12 (Jun 23 2015)
readHardwareRevision
	hardware revision = PCB 1.2/1.3
readSoftwareRevision
	software revision = PCB 1.2/1.3
readManufacturerName
	manufacturer name = Texas Instruments
enableIrTemperature
disconnected!

Another thing I noticed is that it doesnt run without sudo
noble warning: adapter state unauthorized, please run as root or with sudo
or see README for information on running without root/sudo:
https://github.com/sandeepmistry/noble#running-on-linux

@snowbreeze please provide debug logs and hcidump capture:
sudo DEBUG=* node test.js

sudo hcidump -t -x

How close is the SensorTag? I believe I was having issues with kernel 4.1 as well, kernel 4.0 worked better.

I had a similar issue after upgrading the Raspberry Pi firmware to version 4.1. The connection failed also using gatttool.
After downgrading to version 3.18 again fixed the problem.

@snowbreeze any updates?

I shifted to old kernel and it works fine...@krishnandu is continuing the research on the new kernel.

@snowbreeze @martin-doyle if you still have access to something running the newer kernel, can you please check the output of dmesg, I think it's related to this change: https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git/commit/net/bluetooth?id=25ba265390c09b0a2b2f3fd9ba82e37248b7a371
So if you see: "SMP security requested but not available that's the issue.

Hi ... I'm having Kernel 4.1.6 on RaspberryPi and get this message in dmesg ... so the only solution for now is to go back to an old kernel?

@tjandt yes, downgrading the kernel is my recommended short term work around for now, see: https://github.com/sandeepmistry/noble/issues/242

A v1.2.1 of noble has been published with a fix for https://github.com/sandeepmistry/noble/issues/242 a fresh install of node-sensortag will pull that version down.


Removed beginning underscore from some setters/getters

Just for consistency sake, as they seemed like the only ones that had them.
Also renamed OptionButton's "_select_int" to "set_selected", to match the getter.

The underscore before the function name makes it unavailable from GDscript (thus it make the function  editor-only). Thus it may have been intended.

As I commented on IRC, I'm not sure if the setgets should be made visible or if it's the properties that should be hidden. But I don't see the point of making a visible property with a hidden setter/getter pair.

Unless I'm missing something, I believe it was done that way because that's how properties typically work in other languages. It's essentially two methods (get and set) that are hidden by something that looks like a field, but really isn't.
IE: node.visible is actually two methods that are something like: node._set_visible(bool visible) and bool node._get_visible().
In C# you can make a property:
public bool visible { get; set; }

Which internally looks something like: get_visible() and set_visible() but is simply called by whatever.visible.
In other words, I'm 99% sure the methods are intentionally hidden so that the person doing the scripting only has to worry about a single property as opposed to two methods. Making these visible now makes it a property and two methods instead of single property which does the same thing as the two methods :)

@NathanWarden not really, because setters and getters are shown with the property in editor help:

They aren't shown in the regular list of methods.
OTOH, for those underscore setters and getters, they are still shown along the properties, even though non-setget methods are hidden if they start with an underscore. For instance, the MenuButton items property:

My guess is that those setgets were supposed to be hidden but ended up in the surface when properties were added. That's why I'm wondering if those properties were meant to be visible or even if the should be bound at all. E.g. instead of using MenuButton.items, you should use MenuButton.get_popup().add_item(), so the proper structure is generated.

From the documentation instructions at: https://hackmd.io/s/H1es51Oeqb#
DO NOT EDIT SET/GET methods! They’ll be removed from the docs in Godot 3.0. Instead, document the member variables (anything in the  XML nodes)
So, they're definitely supposed to be hidden.

Those should not be visible in the doc, it's more of a doc bug
On Nov 25, 2017 2:38 PM, "Nathan"<EMAIL_ADDRESS>wrote:

From the documentation instructions at: https://hackmd.io/s/H1es51Oeqb#
DO NOT EDIT SET/GET methods! They’ll be removed from the docs in Godot
3.0. Instead, document the member variables (anything in the XML nodes)
So, they're definitely supposed to be hidden.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/pull/13262#issuecomment-346954795,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF-Z272Vdntm9Uvz6zPGJLsjxi4P7Zn8ks5s6FCugaJpZM4QqTSZ
.

@NathanWarden No, those doc instructions are outdated.

@reduz should those specific properties in this PR be available to GDScript? Either the properties should be made hidden or the setters should be made visible.

@groud They can be used in GDScript.

For the methods that make sense to have a property, they should either be all exposed properties or all exposed methods, but not both. Obviously if we're keeping methods we just simply shouldn't use properties. If we're using properties, then the related methods should be hidden.
@akien-mga @NathanLovato Are you sure it's outdated? If that part of it is outdated then it should probably be removed, right?

@NathanWarden Yes it was changed... yesterday? The warning was because you would have a lot of setters and getters that were bound to a member variable and ignored while building the docs - but docs writers still filled them. The thing is that was not all of the methods starting with set/get, just the majority.
All setters and getters are gone since... yesterday? Now the remaining methods that start with set/get are different, they're not bound to variables.

@NathanLovato Ah, okay, I think that makes sense if I'm understanding correctly... All of the setters/getters for properties are gone, but the rest are still there. Am I understanding that correctly? Thanks :)

I think the proper way to solve this problem would be to:

add a PROPERTY_USAGE_INTERNAL mask in object.h
add it to the usage of the properties that we dont want visible
make sure they are not used in doc_data.cpp and not used on bindings.

I've opened #13334 to keep track of the issue, closing this PR as it's not the right fix.


Put your ideas on what more can be added to this !!

Any new feature or enhancement for this !!

Can you first explain the project?

Wanted to make a search box which have all types of filter and also improve the UI of the website overall ...and also it can be linked with google so that search appears @aniketsharma00411

There are several enhancement possible-
1.Currently It is showing only video searches . So  Tag line could be- Woogle- Video Searching Platform.
2. Size of search button could be enhanced.
3.If the result cold be wrapped inside a card and click to watch button could be added.
Can i work on above suggested points.

Yes sure please work on the ideas specified by you @Himanshu-Modi

Yes and if you want to add that please go ahead @dev-abir

are u sure, it's legal?

no

well, so how should we add filters?


Subtitle is hidden behind icon when title is empty.

When I set subtitle, but leave title to "", the subtitle is hidden behind the icon circle., which makes it impossible to read.
After a quick look at the code I found out that the problem might be in computing vertical position of subviews.

@matoushybl Could you try this PR? - https://github.com/vikmeup/SCLAlertView-Swift/pull/332


Added support for custom mqtt server port

Hey, would be great to have an option to configure a custom mqtt server port.
Here a little contribution to this great project!... more comming :)
All the best

thanks!


Don't share testhosts

Disables sharing testshosts, and puts it under feature flag, because it is causing problems when projects with different dependencies are running under the same process. This only affects .NET Framework test projects that don't enable parallelization, or don't disable app domains.
Adds option to skip default adapters (the extensions that are place in Extensions folder in our VisualStudio folder). This allows projects to run even if external extensions (such as Chutzpah) add dependencies that are incompatible with the projects. Without this option all extensions from the built in folder are always included in the run.

/azp run


Port of DataLoader for Java and Vert.x now available

I have created a port of DataLoader for use with Java 8 and Vert.x.
Its still a work-in-progress, but you can check it out at:
https://github.com/engagingspaces/vertx-dataloader

Very awesome!

Also available in a pure Java 8 form as https://github.com/graphql-java/java-dataloader/

Yes, the fork @bbakerman made from vertx-dataloader to make it independent of Vert.x should be considered the official version and the way to go.
I added a note of this nature at the top of the README some time ago. Though you could still use the Vert.x version, I do not actively maintain it anymore.


change np_datetime.c references to be to the library

Made changes to setup.py to remove np_datetime.c references as sources and refer back to the library.

Awaiting @lithomas1 on https://github.com/pandas-dev/pandas/pull/47081


updated readme

updated readme to include changes I made by adding search engines
What is this Pull Request About?

What will this Pull Request Affect?

Any additional information?

Hey, I appreciate this PR but we change all of the additions and modified etc in a new update seperately. Thanks for the PR though!


improve config.sh, fix ncurses detection

(tested with dash, the most restrictive/posixly_correct shell)

just ftr, the ncurses detection before this commit was broken when using netbsd-curses, also the transparency and wget_wch checks could never complete successfully because the required C/LDFLAGS for ncurses were not used for those checks.

@X-Ryl669 : would you mind testing this and provide your feedback? thanks!

This patch breaks everything and generates a non-working mp:
tiny@hell:~/src/mp-5.x$ ./mp-5
mp_core.mpsl: file not found
Didn't investigate further but from the make output I see it's
generating mp.tar but expecting mp.zip.
11/11/20 14:40, rofl0r:

    You can view, comment on, or merge this pull request online at:

https://github.com/ttcdt/mp-5.x/pull/9
https://github.com/ttcdt/mp-5.x/pull/9
    Commit Summary

config.sh: introduce chk_pkgconfig

config.sh: introduce chk_compiles

config.sh: fix ncurses detection
File Changes

M config.sh
https://github.com/ttcdt/mp-5.x/pull/9/files#diff-f664e29ba13a6135bb7bf8f7e4c51e380b0cd6266be561c046467e0accdcdaa5
(243)
Patch Links:

https://github.com/ttcdt/mp-5.x/pull/9.patch
https://github.com/ttcdt/mp-5.x/pull/9.patch

https://github.com/ttcdt/mp-5.x/pull/9.diff
https://github.com/ttcdt/mp-5.x/pull/9.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ttcdt/mp-5.x/pull/9, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AP4PJ2PQYY3Z6Y2VR2RUYSDSPKH3DANCNFSM4TSADFOQ.


Fix HTTP Response kwargs

Replaces the mimetype with content_type in all HttpReponses.

ping @ldko @somexpert

:+1:

:+1:


typings bundle: file already exists

typings bundle . -o bundle.d.ts

typings ERR! message EEXIST: file already exists, mkdir '/Users/hwong/github/typed-typings/npm-through2/bundle.d.ts'

@unional It now creates a directory, not a file. Should document that change.

I see. Updating scripts

The --out is a required field now? Not default to bundle or dist folder?

It is a required field, yes. There's no default, but there could be.

updating help text to:
typings bundle --out <directory>

Options:
  [--name] <string> Target module name
  [--ambient]           Bundle as ambient/global module

Awesome, sounds good. Tiny nit, can we align the options? This format already seems more useful :+1:

Sure.

Maybe we can also create a task that'll build a commands.md from this too! I'll make an issue, it'll save having to update the README all the time. So if there's text and it's easily parsable, I'll do that.

This issue fixed. Closing. Will update each repo. :sob:


cherry-pick(cleaner): add /dev mount to cleanup job (#649)

mount /dev directory from host inside the container, so that wipefs
performed on the disk generates a corresponding udev change event and
the change is reflected on the host also.
Signed-off-by: Akhil Mohan<EMAIL_ADDRESS>cherry-pick #649

Codecov Report

Merging #650 (b4ab471) into v1.7.x (6745dcf) will decrease coverage by 0.47%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           v1.7.x     #650      +/-   ##
==========================================
- Coverage   46.93%   46.46%   -0.48%
==========================================
  Files          78       78
  Lines        3814     3814
==========================================
- Hits         1790     1772      -18
- Misses       1866     1882      +16
- Partials      158      160       +2

Impacted Files
Coverage Δ

cmd/ndm_daemonset/probe/eventhandler.go
30.55% <0.00%> (-8.34%)
:arrow_down:

cmd/ndm_daemonset/probe/addhandler.go
74.40% <0.00%> (-3.08%)
:arrow_down:

cmd/ndm_daemonset/probe/udevprobe.go
49.01% <0.00%> (-1.19%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6745dcf...b4ab471. Read the comment docs.


Carousel in Vanilla JS

Is your feature request related to a problem? Please describe.
A carousel is a slideshow for cycling through series of contents. This can be used for testimonials, reviews section etc.
Describe the solution you'd like
It is built using HTML, CSS, and JS only (no front-end frameworks like Bootstrap, Tailwind CSS, etc).

@Ayush7614 I would like to work on this as a part of HCSSoC'21


Vivek5200 patch 1

Implement Linked List Data Structure

Added linked_list.c files
Defined struct Node to represent list nodes
Implemented functions: create_list, insert_node, delete_node, get_node_data, display_list
Added appropriate comments and documentation

@HarshwardhanPatil07 , Please Merge this time I done What you are Said.

@vivek5200 take input from user not inbuilt


Enable hot reload for development server

It was inconvenient because file updates were not automatically reflected when Flask was running.
Therefore, we enabled hot reloading of Flask.

Updated server.py to pass the debug parameter to app.run() based on the DEBUG environment variable.
Added a new DEBUG field to the ServerSettings class in settings.py with a default value of True.

By setting debug=True,  The server automatically restarts whenever a file is modified.

Genius. I kept running the server, seeing an error, stopping the server, changing a file and then starting the server again to see if the bug was fixed. That was getting... tedious 😂
Good call, merging

@ryansurf
Thanks!
Another benefit of setting the debug mode to True is that when an error occurs, Flask will display the error's stack trace on the screen 🥴

Woah, that is useful. Nice!


Add weakly compressible version of MSFR pronghorn

refs #46
@dzenun
This is a closed loop. I ll update to an inlet-outlet soon

Job VTB Documentation on f2e7a26 wanted to post the following:
View the site here
This comment will be updated on new commits.

Closing this. I ll update MSFR with the Physics syntax, maybe with the segregated solver too