Arbitrary sized FFTs using modified kissFFT as default backend and MKL otherwise

This PR adds a C++ port of kissFFT as the default backend of dlib's fft routines. All the kiss code is inserted into the dlib namespace to avoid conflicts with user code that may or may not be using original (C) kissFFT. All MKL fft code is put into it's own separate header file. Now dlib/matrix_fft.h simply calls the right wrappers depending on a pre-processor macro (as before). I've removed the FFTW wrapper code as per @davisking's request as the fftw planner isn't thread safe. I've also removed the original fft backend code, again, as per @davisking's request.
Note that this PR isn't quite ready yet. Need more unit tests. Not quite sure why MKL wrappers only worked for matrix<complex<double>> before. There's no reason why it couldn't work for matrix<complex<float>> if using DFTI_SINGLE instead of DFTI_DOUBLE. (maybe user code had to do a cast?) So need more units tests for MKL wrappers.
Not quite happy with the number of copies at the moment. When passing const matrix_exp<EXP>& data as input, there are 2 copies: one to evaluate the matrix expression to matrix<typename EXP::type> and one to do either the in-place fft (copy is done in the implementation details of the fft functions) or out-of-place fft (copy is done explicitly in  dlib/matrix_fft.h). For example, if you want to do the fft of std::vector<std::complex>, you have to use dlib::mat which returns a matrix expression then dlib::fft will evaluate that to dlib::matrix<std::complex<float>>. So there is a copy of std::vector to dlib::matrix that is unnecessary. Could add some function overloads to support std::vector. Don't know what's the best thing to do without dirtying the API. Could benefit from @davisking's advice.
I think if we've gone this far, we should support real FFTs  so the requirement that input should be complex vanishes. Output will be complex though. Both KISS and MKL support this so there shouldn't be a lot of leg work. More unit tests.

Also would be a good idea to SIMD the kissFFT wrappers using dlib's SIMD wrappers.

Do we want to cache and copy MKL descriptors using DftiCopyDescriptor?

kissFFT backend could also be optionally parallelised if a dlib::thread_pool is provided

kissFFT backend could also be optionally parallelised if a dlib::thread_pool is provided, using dlib::parallel_for

Actually that's not a good idea.

For unit tests, it looks like a lot of libraries simply do something like :
assert(abs(ifft(fft(x)) - x) < tol)

assuming ifft performs scaling and tol is some small value.
Is this sufficient? Doesn't feel like it. Maybe should test FFTs of sinusoids, complex exponentials, standard window functions, etc. Would be great if there was some open source dataset for unit testing FFTs...

https://dsp.stackexchange.com/a/635 gives some ideas.

Real FFTs are fixed now. Still need to implement fftr and ifftr for MKL

@davisking It would be great if you could do a review. I don't expect this PR to be complete. I still have to finish MKL stuff but there is enough here i think to have an intermediate review.

For unit tests, it looks like a lot of libraries simply do something like :
assert(abs(ifft(fft(x)) - x) < tol)

assuming ifft performs scaling and tol is some small value.
Is this sufficient? Doesn't feel like it. Maybe should test FFTs of sinusoids, complex exponentials, standard window functions, etc. Would be great if there was some open source dataset for unit testing FFTs...

This test should be done.  But adding a single additional test that runs known data with known outputs through each codepath is also a good idea.  There is some of this already in the dlib code.  I don't think it matters what's in the test data so long as it's not trivial in some way.  I would just add in random gaussian noise as I have a hard time imagining code that somehow computes correct ffts for random noise but doesn't for other inputs.

@davisking Silly question. I don't know much about travis and appveyor, but is there a way to build the tests with MKL? At the moment all the tests pass even though i haven't done the MKL wrappers for fftr and ifftr

@davisking Silly question. I don't know much about travis and appveyor, but is there a way to build the tests with MKL? At the moment all the tests pass even though i haven't done the MKL wrappers for fftr and ifftr

Not that I know of.  I assume you are testing it locally with the Intel MKL though?

@davisking Silly question. I don't know much about travis and appveyor, but is there a way to build the tests with MKL? At the moment all the tests pass even though i haven't done the MKL wrappers for fftr and ifftr

Not that I know of. I assume you are testing it locally with the Intel MKL though?

I haven't done the MKL stuff yet. I wanted to test the MKL stuff standalone in a Netbeans project (i'm not that good with cmake yet) but was struggling to link to all the MKL libraries manually. So will go back to just testing with the dlib build. For real FFTs, I'm a bit confused by the CCE and CCS format + all the other configurations that MKL supports for real FFTs. Will get there when i have time to work on this again.

Hmm, test failed with :
The job exceeded the maximum time limit for jobs, and has been terminated.

@davisking If you have time, do you mind reviewing the overloads I added ? Also i amended find_blas.cmake to check for the oneAPI install directory. Hope that's ok.

@davisking I think i'm gonna have to recommit that change to the rand tests. The continuous-integration tests fail with error:
[100%] Linking CXX executable dtest
[100%] Built target dtest
+./dtest --runall
Running test_rand -
The job exceeded the maximum time limit for jobs, and has been terminated.

@davisking I think i'm gonna have to recommit that change to the rand tests. The continuous-integration tests fail with error:
[100%] Linking CXX executable dtest
[100%] Built target dtest
+./dtest --runall
Running test_rand -
The job exceeded the maximum time limit for jobs, and has been terminated.

Huh, yeah I didn't realize how long these tests were taking.  I'm going to push a change that makes them run much faster.

@davisking Ok, so i've replaced std::vector with a new type fft_size. Do you mind doing a quick review and checking you're happy with that? It wraps std::array so there aren't any dynamic memory allocations. Hopefully it's a tad faster.

@davisking I think this is nearly finished. Not sure if we want to do caching with MKL descriptors? In theory why not, but i think that could be in a different PR. The purpose of this PR is to generalize the default backend.

Weird, an SVM test failed.

@davisking I think this is nearly finished. Not sure if we want to do caching with MKL descriptors? In theory why not, but i think that could be in a different PR. The purpose of this PR is to generalize the default backend.

Yeah, don't worry about that for now.

@davisking sorry for the delay. I think this is ready for another round of light roasting. Also, happy holidays

This all looks pretty good overall.  Just some relatively minor stuff.  Run a little benchmark too to check that this is faster or at least the same as the current code too.  Looks like it ought to be though.

I added some unit tests that compared the kiss backend to mkl backend, provided mkl is available. So naming the functions the same won't allow for this test

I wanted a test that checked that all backends agreed on outputs. That's how i found the bug with mkl_fftr and mkl_fftri having incorrect strides. So this kind of test is important i think.

I added some unit tests that compared the kiss backend to mkl backend, provided mkl is available. So naming the functions the same won't allow for this test

Oh right.  Yeah, well, just don't use #define :) .  Put #ifdef blocks inside dlib::fft() instead or something like that then.

I wanted a test that checked that all backends agreed on outputs. That's how i found the bug with mkl_fftr and mkl_fftri having incorrect strides. So this kind of test is important i think.

Yeah, definitely.

Right, will correct everything tomorrow. If i spend more time on this today, my partner won't be in a pleasant state.

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

I might put the code from squeeze_ones inside the constructor of fft_size, and update the contract so users know exactly what's going on. I.e. the constructor will remove values equal to 1 such that the product of all dimensions stays unchanged. If the initializer_list contains a single element and it is equal to 1, then nothing is removed.

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

I just added the functionality of squeeze_ones in the constructor

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

I just added the functionality of squeeze_ones in the constructor

Isn’t that kinda weird though?  I need to look at the code again, maybe it’s not. But it seems kinda funky.

MSVC complaining about casting const long* to const_iterator. Argh

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

I just added the functionality of squeeze_ones in the constructor

Isn’t that kinda weird though? I need to look at the code again, maybe it’s not. But it seems kinda funky.

it is a bit, but the contract stipulates that it does that. And fft_size doesn't even entertain the fact that it resembles an STL container. So it doesn't have to behave like one

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

I just added the functionality of squeeze_ones in the constructor

Isn’t that kinda weird though? I need to look at the code again, maybe it’s not. But it seems kinda funky.

it is a bit, but the contract stipulates that it does that. (or at least it guarantees that num_elements is not affected) And fft_size doesn't even entertain the fact that it resembles an STL container. So it doesn't have to behave like one

Right. But having such a constructor makes it possible to implement things like this squeeze function. This suggests that the interface isn’t really complete as there is something you think is important but couldn’t implement without such an constructor.
Anyway. If it really just doesn’t make sense in the code to ever have a size {1,1} then it’s cool and to make it not a possible state. I’m on my phone and haven’t looked at the code yet.

Struggling to re-implement squeeze_ones without it being friend specified, and therefore without the non-const versions of begin() and end() available. I've removed them otherwise fft_size wouldn't be truly immutable.

Add a constructor that takes a std::array. Then you can make whatever array and pass that.

I just added the functionality of squeeze_ones in the constructor

Isn’t that kinda weird though? I need to look at the code again, maybe it’s not. But it seems kinda funky.

it is a bit, but the contract stipulates that it does that. (or at least it guarantees that num_elements is not affected) And fft_size doesn't even entertain the fact that it resembles an STL container. So it doesn't have to behave like one

Right. But having such a constructor makes it possible to implement things like this squeeze function. This suggests that the interface isn’t really complete as there is something you think is important but couldn’t implement without such an constructor.
Anyway. If it really just doesn’t make sense in the code to ever have a size {1,1} then it’s cool and to make it not a possible state. I’m on my phone and haven’t looked at the code yet.

I see your point. Having {1,1} is definitely a valid state. So i will re-implement squeeze_ones. Then it is clear that it's kind of like an optimisation step used in the backend FFT code, whereby we remove "dummy" dimensions.

Really don't like the fact visual studio compiler doesn't allow a cast from std::initializer_list::const_iterator to std::array<long,5>::const_iterator. Oh well

@davisking So i think the business with squeeze_ones is resolved. i.e. constructors don't implement squeeze_ones. Backend FFT code calls that explicitly. So there shouldn't be any confusions as to what is valid state inside fft_size.

@davisking this is ready for another (hopefully final) review

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ?

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ?

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

How about doing fewer sizes then.  Like do sizes in the range 1 to 12 instead of 1 to 128.  It's fairly important that these tests not take a long time.

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

How about doing fewer sizes then.  Like do sizes in the range 1 to 12 instead of 1 to 128.  It's fairly important that these tests not take a long time.

@davisking Presumably, the contents of get_decoded_string() was machine generated, correct ? If so, how did you print it out? I can compress and base64 encode it easily enough. But when printing the contents to stdout or to file, lets say 80 characters per line, i get something like this:
const std::string str = "gREgHkVPSRDT5z8rk175NS624DOfWDQcPg2MOBWP6zKp5jxHZcD2U1VxamzdjaOSqfLISu"
"O5XxmI
wNbs2oCQ05Jnba2XSMe0/fI4D3HwzE2Py5EXfucAyO7xzaN+5OQO/vFwu9K7ocE"
"hrFnCIIrELaOE
F3Ocpy9CMf1bs5fhz2LHnfXO/hYboIhIiT7tDmny1OI/diOkdfSvEOBA"
"RrBx72j+beMHFoAuH4/L
/V76B7hRdlR2O9G6w7NcgpqkAMav7L6NshODMCnm3a+oCMOb9"
"k+x20CupDgjh5GrUk4TS98a+6vF
n3jhOLiW08ydsHJN7L15AKc1IFXITHLlHJzReNVI/v"
"lsJ/bfpahh/gJ1ToHm8OVsSQnBX2SBqv9Z
IxDea8Gbkjsxo2Ptvewv5MTm7d9K13T8ACo"
"IP8rhyhd/J7d1n7/qb7dwIquGMnd4ugbX2UdXGzhA
lQjtEbulmCpTfGvR2owFYhcViTmz"
"h1Q4nMYfT1JFAZ9bWIVMi//QdBKASkhcepD/FS8eCKcNDqzJ
4oVpYHvJNauFx2visid1/"
"uiRVfuI3hzOkVFQtxzNywqU/nJ4TgY6cGk+Fc74pXezjNQt653LDEQz
0m2Tzq+IjGymYR"
"KU+PqcQbxzO+7A47wg2wRO5bHCxI1DZ9rU45jrxkyzgYrmx6B8R4wXLJlqB2Jc
4LmVQXi"
"xuu7Y6ria6Dmdq91/P1QW5F/GPirFEhV+1k1FBg3gj+wDdWwVaEnFP/sSWdNfsu8ysBlS
"
"sroFquFysayDxtmxu4SA5u17bOhpRvWIkEm8MciQaTYNYvQcgfShlFSBrmdnbGGZyO4vTC"
...

Did you have to use dlib::ustring or something?

@davisking Presumably, the contents of get_decoded_string() was machine generated, correct ? If so, how did you print it out? I can compress and base64 encode it easily enough. But when printing the contents to stdout or to file, lets say 80 characters per line, i get something like this:
const std::string str = "gREgHkVPSRDT5z8rk175NS624DOfWDQcPg2MOBWP6zKp5jxHZcD2U1VxamzdjaOSqfLISu"
"O5XxmI
wNbs2oCQ05Jnba2XSMe0/fI4D3HwzE2Py5EXfucAyO7xzaN+5OQO/vFwu9K7ocE"
"hrFnCIIrELaOE
F3Ocpy9CMf1bs5fhz2LHnfXO/hYboIhIiT7tDmny1OI/diOkdfSvEOBA"
"RrBx72j+beMHFoAuH4/L
/V76B7hRdlR2O9G6w7NcgpqkAMav7L6NshODMCnm3a+oCMOb9"
"k+x20CupDgjh5GrUk4TS98a+6vF
n3jhOLiW08ydsHJN7L15AKc1IFXITHLlHJzReNVI/v"
"lsJ/bfpahh/gJ1ToHm8OVsSQnBX2SBqv9Z
IxDea8Gbkjsxo2Ptvewv5MTm7d9K13T8ACo"
"IP8rhyhd/J7d1n7/qb7dwIquGMnd4ugbX2UdXGzhA
lQjtEbulmCpTfGvR2owFYhcViTmz"
"h1Q4nMYfT1JFAZ9bWIVMi//QdBKASkhcepD/FS8eCKcNDqzJ
4oVpYHvJNauFx2visid1/"
"uiRVfuI3hzOkVFQtxzNywqU/nJ4TgY6cGk+Fc74pXezjNQt653LDEQz
0m2Tzq+IjGymYR"
"KU+PqcQbxzO+7A47wg2wRO5bHCxI1DZ9rU45jrxkyzgYrmx6B8R4wXLJlqB2Jc
4LmVQXi"
"xuu7Y6ria6Dmdq91/P1QW5F/GPirFEhV+1k1FBg3gj+wDdWwVaEnFP/sSWdNfsu8ysBlS
"
"sroFquFysayDxtmxu4SA5u17bOhpRvWIkEm8MciQaTYNYvQcgfShlFSBrmdnbGGZyO4vTC"
...

Did you have to use dlib::ustring or something?

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

Cool, i just used numpy in the end. It was an absolute PAIN, getting python data, to dlib::matrix types, compressed and base64 encoded (those 2 bits were simple actually), then printed to a C++ file that can be included in fft.cpp.

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

Cool, i just used numpy in the end. It was an absolute PAIN, getting python data, to dlib::matrix types, compressed and base64 encoded (those 2 bits were simple actually), then printed to a C++ file that can be included in fft.cpp.

Python code was:
def save_good_fftr():
    import numpy as np
    import msgpack
    np.random.seed(10000)

    ret = []

    x = np.random.randn(64,64)
    y = np.fft.rfftn(x)
    y2 = np.stack((y.real,y.imag),-1)
    ret.append([x.tolist(),y2.tolist()])

    x = np.random.randn(131,48)
    y = np.fft.rfftn(x)
    y2 = np.stack((y.real, y.imag), -1)
    ret.append([x.tolist(),y2.tolist()])

    with open("good_real_ffts.msgpack", "wb") as f:
        msgpack.pack(ret,f)

And C++ code to generate a header file was:
   dlib::matrix<double> from_msgpack(const std::vector<std::vector<double>>& x_v)
   {
    dlib::matrix<double> x(x_v.size(),x_v[0].size());

    int offset = 0;
    for (const auto& v : x_v)
    {
        assert(v.size() == x_v[0].size());
        std::copy(v.begin(), v.end(), x.begin() + offset);
        offset += v.size();
    }

    return x;
}

dlib::matrix<complex<double>> from_msgpack(const std::vector<std::vector<std::vector<double>>>& y_v)
{
    dlib::matrix<complex<double>> y(y_v.size(), y_v[0].size());

    int offset = 0;
    for (const auto& v : y_v)
    {
        assert(v.size() == y_v[0].size());
        for (const auto& x : v)
        {
            assert(x.size() == 2);
            *(y.begin() + offset++) = std::complex<double>(x[0],x[1]);
        }
    }

    return y;
}

void save_good_fftr()
{
    std::ifstream in("good_real_ffts.msgpack", std::ios::binary);
    assert(in.good());
    nlohmann::json j = nlohmann::json::from_msgpack(in);

    std::vector<std::vector<double>> x_v;
    std::vector<std::vector<std::vector<double>>> y_v;

    dlib::matrix<double> x1,x2;
    dlib::matrix<complex<double>> y1,y2;

    j[0][0].get_to(x_v);
    j[0][1].get_to(y_v);
    x1 = from_msgpack(x_v);
    y1 = from_msgpack(y_v);

    j[1][0].get_to(x_v);
    j[1][1].get_to(y_v);
    x2 = from_msgpack(x_v);
    y2 = from_msgpack(y_v);

    std::stringstream compress_in, compress_out, base64_out;
    dlib::serialize(compress_in) << x1 << y1 << x2 << y2;

    dlib::compress_stream::kernel_1ea compressor;
    compressor.compress(compress_in, compress_out);

    dlib::base64 base64_coder;
    base64_coder.encode(compress_out, base64_out);

    string out1 = base64_out.str().insert(0, "\"");
    out1.push_back('\"');
    const string out = std::regex_replace(out1, std::regex("\n"), "\"\"");

    std::ofstream fout("compressed_fftrs.txt");
    fout << "static const std::string GOOD_FFTRs_COMPRESSED_BASE64_ENCODED =";

    const int step = 78;
    for (int i = 0 ; i < out.size() ; i += step)
    {
        fout << endl;
        const int end = min<int>(i + step, out.size());
        for (int j = i ; j < end ; j++)
            fout << out[j];
    }
    fout << ";" << endl;
}

there probably was a much simpler way of doing all this...

Python code was:
def save_good_fftr():
    import numpy as np
    import msgpack
    np.random.seed(10000)

    ret = []

    x = np.random.randn(64,64)
    y = np.fft.rfftn(x)
    y2 = np.stack((y.real,y.imag),-1)
    ret.append([x.tolist(),y2.tolist()])

    x = np.random.randn(131,48)
    y = np.fft.rfftn(x)
    y2 = np.stack((y.real, y.imag), -1)
    ret.append([x.tolist(),y2.tolist()])

    with open("good_real_ffts.msgpack", "wb") as f:
        msgpack.pack(ret,f)

And C++ code to generate a header file was:
   dlib::matrix<double> from_msgpack(const std::vector<std::vector<double>>& x_v)
   {
    dlib::matrix<double> x(x_v.size(),x_v[0].size());

    int offset = 0;
    for (const auto& v : x_v)
    {
        assert(v.size() == x_v[0].size());
        std::copy(v.begin(), v.end(), x.begin() + offset);
        offset += v.size();
    }

    return x;
}

dlib::matrix<complex<double>> from_msgpack(const std::vector<std::vector<std::vector<double>>>& y_v)
{
    dlib::matrix<complex<double>> y(y_v.size(), y_v[0].size());

    int offset = 0;
    for (const auto& v : y_v)
    {
        assert(v.size() == y_v[0].size());
        for (const auto& x : v)
        {
            assert(x.size() == 2);
            *(y.begin() + offset++) = std::complex<double>(x[0],x[1]);
        }
    }

    return y;
}

void save_good_fftr()
{
    std::ifstream in("good_real_ffts.msgpack", std::ios::binary);
    assert(in.good());
    nlohmann::json j = nlohmann::json::from_msgpack(in);

    std::vector<std::vector<double>> x_v;
    std::vector<std::vector<std::vector<double>>> y_v;

    dlib::matrix<double> x1,x2;
    dlib::matrix<complex<double>> y1,y2;

    j[0][0].get_to(x_v);
    j[0][1].get_to(y_v);
    x1 = from_msgpack(x_v);
    y1 = from_msgpack(y_v);

    j[1][0].get_to(x_v);
    j[1][1].get_to(y_v);
    x2 = from_msgpack(x_v);
    y2 = from_msgpack(y_v);

    std::stringstream compress_in, compress_out, base64_out;
    dlib::serialize(compress_in) << x1 << y1 << x2 << y2;

    dlib::compress_stream::kernel_1ea compressor;
    compressor.compress(compress_in, compress_out);

    dlib::base64 base64_coder;
    base64_coder.encode(compress_out, base64_out);

    string out1 = base64_out.str().insert(0, "\"");
    out1.push_back('\"');
    const string out = std::regex_replace(out1, std::regex("\n"), "\"\"");

    std::ofstream fout("compressed_fftrs.txt");
    fout << "static const std::string GOOD_FFTRs_COMPRESSED_BASE64_ENCODED =";

    const int step = 78;
    for (int i = 0 ; i < out.size() ; i += step)
    {
        fout << endl;
        const int end = min<int>(i + step, out.size());
        for (int j = i ; j < end ; j++)
            fout << out[j];
    }
    fout << ";" << endl;
}

there probably was a much simpler way of doing all this...

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

How about doing fewer sizes then. Like do sizes in the range 1 to 12 instead of 1 to 128. It's fairly important that these tests not take a long time.

If you look at the last commit, i've replaced large primes like 1021 and 1031 to smaller primes like 131. That helped a lot with speed. And i now check for all states up to 64, not 128. that helps a lot. And test_against_saved_good_fftrs() works as expected.

Hmm, replacing nr = fft_next_fast_size(nr + 1) with nr++, and same for nc in the tests, has dramatically increased the time it takes to run the tests, even when removing the iter for-loops. Oh well

How about doing fewer sizes then. Like do sizes in the range 1 to 12 instead of 1 to 128. It's fairly important that these tests not take a long time.

If you look at the last commit, i've replaced large primes like 1021 and 1031 to smaller primes like 131. That helped a lot with speed. And i now check for all states up to 64, not 128. that helps a lot. And test_against_saved_good_fftrs() works as expected.

@davisking Sorry for the last commits being such a mess. But the tests all run under 30s on my machine now, and all successful. So hopefully we are all good now.

@davisking Sorry for the last commits being such a mess. But the tests all run under 30s on my machine now, and all successful.

Ok and i've added another test for fftr. It's only for 1D stuff though. But still good to have.

Ok and i've added another test for fftr. It's only for 1D stuff though. But still good to have.

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

Cool, i just used numpy in the end. It was an absolute PAIN, getting python data, to dlib::matrix types, compressed and base64 encoded (those 2 bits were simple actually), then printed to a C++ file that can be included in fft.cpp.

Ah, sorry, I thought you knew about the http://dlib.net/file_to_code_ex.cpp.html example.  It does all that for you :|

@davisking For test_against_saved_good_ffts(), what did you use as an authority? numpy ? I could use MKL, but the dlib MKL wrappers are subject to testing too, so by definition, they cannot be used as an authority

I just grabbed it out of octave IIRC.

Cool, i just used numpy in the end. It was an absolute PAIN, getting python data, to dlib::matrix types, compressed and base64 encoded (those 2 bits were simple actually), then printed to a C++ file that can be included in fft.cpp.

Ah, sorry, I thought you knew about the http://dlib.net/file_to_code_ex.cpp.html example.

Ha, yeah.

Ha, yeah.  Use http://dlib.net/file_to_code_ex.cpp.html.  it will format it all so it works on any compiler (where any means works around MSVC nonsense like this)

Yeah got there in the end. Hopefully all ready now

Yeah got there in the end. Hopefully all ready now

Morning @davisking. Please let me know if there are any additional changes required. Otherwise, I'm looking forward to this getting merged.

Morning @davisking. Please let me know if there are any additional changes required. Otherwise, I'm looking forward to this getting merged.

urg, wtf.  I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

urg, wtf.  I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

Sweet, looks good.  Thanks for the PR :)

Sweet, looks good.  Thanks for the PR :)

urg, wtf. I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

I don't quite understand what the problem was. Maybe i should have rebased this PR's branch? Please let me know what the issue was so i know not to do it next time. Thanks again @davisking

urg, wtf. I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

I don't quite understand what the problem was. Maybe i should have rebased this PR's branch? Please let me know what the issue was so i know not to do it next time. Thanks again @davisking

urg, wtf. I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

I don't quite understand what the problem was. I never pulled master into my branch, so when you pulled and added a commit, i don't see where you would have got conflicts. Maybe i should have rebased this PR's branch yesterday just to be sure. Please let me know what the issue was so i know not to do it next time. Thanks again @davisking

It wasn't rebased.  I'm surprised github itself doesn't complain that it fails to rebase.  It must fall back on some more aggressive merge/squash when it checks if a PR applies cleanly.

urg, wtf. I just pulled this PR, added a commit, and pushed it back but somehow this PR's branch had these other commits in it :|

I don't quite understand what the problem was. I never pulled master into my branch, so when you pulled and added a commit, i don't see where you would have got conflicts. Maybe i should have rebased this PR's branch yesterday just to be sure. Please let me know what the issue was so i know not to do it next time. Thanks again @davisking

It wasn't rebased.  I'm surprised github itself doesn't complain that it fails to rebase.  It must fall back on some more aggressive merge/squash when it checks if a PR applies cleanly.


Improve boot time

Improve boot time for:

[ ] Firmware (~5.5s)
[ ] Kernel (~4.0s)

Related info

https://elinux.org/Boot-up_Time_Reduction_Howto
https://wiki.archlinux.org/title/Improving_performance/Boot_process

moved to private task list


fix dev_proxy example when using account auth

Fixes a bug in the dev_proxy example preventing auth from working on various websites.
Fixes #28

Thank you!


Bug report: basictex uninstall deletes wrong folder

When performing brew cask uninstall basictex, the folder /usr/local/texlive/2016gets deleted (this is where the regular mactex would be installed).
The correct folder would be /usr/local/texlive/2016basic as stated in http://tug.org/mactex/morepackages.html
I think the same applies for ~/Library/texlive/2016vs ~/Library/texlive/2016basic.

https://github.com/caskroom/homebrew-cask/pull/27479


Add netp check for subscription

Task/Issue URL: https://app.asana.com/0/0/1206140936022689/f
Description
See attached task description
Steps to test this PR
https://app.asana.com/0/0/1206170013724657/f

Current dependencies on/for this PR:

develop

PR #4000

PR #4003   👈

This stack of pull requests is managed by Graphite.


Retry rate limits on chunk uploading

I signed it!

This looks okay to me. @jonparrott, any thoughts?
How might this change get test coverage?

Change looks fine to me. @lichray can you update any relevant tests or let us know if they do not need to be updated (unlikely)?

Tests added.

LGTM, merging. Thank you for your contribution. :)


clarify meaning of canvas({...}) syntax

This rewords a bit the description of the canvas call syntax.
I was confused when I first read the doc, especially the part "call the function like any other except place a pair of curly braces {} inside the brackets". I was not sure what was happening when, and what I could do or could not do. I understand that this should be described in details in the upcoming "internals" section, but I think making things a bit less magical and more concrete here can help the reader get the right idea...

Thanks! that does make it sound much better


Track constraint checking status in the database

We need to reflect the status of constraint checking the database. This should be similar to how we store simulation and scheduling statuses.

I think we need to do both of these things at the same time, each constraint_run record should contain the status of the result, and probably some other properties as well. I'll take a look at the db to see how scheduling and simulation handle this.


Add Session in the app

Is it a ?

[ ] Bug Report

[x] Feature Request

[ ] Chore

Description
This is aimed at implementing all the functionalities related to session shown here in the API.
Would you like to work on the issue?
Yes

@iamareebjamal Is this feature in scope of the app? If yes, then please provide the necessary details.

For sessions, tracks would also have to be implemented and more. This is in the original road map, and surely we will look forward to implementing it this summer, but for now let's focus on fixing current bugs and completing the missing functionality as this issue is a big undertaking

Yes that's most important

The issue is completed. Closing this.


empty search string causes internal server error

When navigating to the search page and just hitting enter, without any value entered before it results in a 500 Internal Server Error Response.

Specifically, the user is directed to https://www.vutuv.de/search_queries:

Pardon us! Something went wrong. If you think this is a bug, please submit a bug report.

...whose HTML markup is invalid (see the )?


Russian

Hello Afshine Amidi and Shervine Amidi.
You did a great job with your short notes! Making material in my language was also a big job.
I don't know your algorithms for creating pdf and website, so if you have questions, I will help.

Thank you @AlexandrParkhomenko for your incredible work, this is fantastic!
To be able to get the review process going, could you please:

Split the current pull request into the following ones:

[ru] cs-221-logic-models
[ru] cs-221-reflex-models
[ru] cs-221-states-models
[ru] cs-221-variables-models
[ru] cs-229-deep-learning
[ru] cs-229-linear-algebra
[ru] cs-229-machine-learning-tips-and-tricks
[ru] cs-229-probability
[ru] cs-229-supervised-learning
[ru] cs-229-unsupervised-learning
[ru] cs-230-convolutional-neural-networks
[ru] cs-230-deep-learning-tips-and-tricks
[ru] cs-230-recurrent-neural-networks

Invite anyone you would know to review your work in these each of these pull requests.

Thank you so much for all your help and work!


postgresql litters $out/share directory

Issue description
postgresql is littered with files in $out/share. They should go in $out/share/postgresql.
Steps to reproduce
$ ls -1 /nix/store/rgnwfbhybj5zykg4fy38yd8f229ga44f-postgresql-9.6.12/share
conversion_create.sql
extension
information_schema.sql
pg_hba.conf.sample
pg_ident.conf.sample
pg_service.conf.sample
postgres.bki
postgres.description
postgres.shdescription
postgresql.conf.sample
psqlrc.sample
recovery.conf.sample
snowball_create.sql
sql_features.txt
system_views.sql
timezonesets
tsearch_data

Technical details
Please run nix-shell -p nix-info --run "nix-info -m" and paste the
results.

Weird. Not sure why GH did not close this.


Will this plugin support ACL in the future?

really wish it can support acl feature. thanks.

@weizengquan. ACL is supported in 1.1 release

excellent! thank you.


Allow setting the max_bit_rate and min_bit_rate in codec context

These variables are documented as part of ffmpeg.
It might be useful for others to read and write them.

OK could we have a test for this please?

I'm actually no longer sure what should happen when these things change. I had to specificy the codec context options dictionary to have a visible change in my output quality.

I'm guessing this PR hasn't been rebased yet as the new tests are against pypy3.9 and not pypy3?

I've rebased the patch, but I'm not sure what effect "writing" to these two properties has.

This PR is too old.


feat: add support for release assets with multiple spaces within the name

This PR improves the Action's ability to handle release assets with names containing two or more spaces. It enhances the functionality for updating existing releases and uploading new assets by correctly identifying and managing all existing release assets, regardless of the number of spaces in their original names.

lgtm, thanks @dukhine for your first contribution!


'BERTPreprocessor' object has no attribute 'preprocess_train_called'

I've successfully loaded my preprocessor object with ktrain.load_predictor(/path/to/file), however, when it comes time to make a prediction on the text and I call .predict(text), I get this error:
AttributeError: 'BERTPreprocessor' object has no attribute 'preprocess_train_called'
I followed the traceback and looked into the scripts, but I still can't figure out why I'm getting this. It looks like the only argument required for this to work is the texts argument, which is definitely provided in my case.
Some context in case it helps:
I had originally created the .preproc file and the associated classifier file in Google Colab a couple months back. I just downloaded those two files and saved them onto my own machine today to replicate the predictor I had built in Google Colab (which used to work completely fine, but is now also giving me the same exact error). I upgraded both ktrain and tensorflow today before running and both upgrades were successful. Could it be that in between the time I created the original files and the time I downloaded them today, the scripts were updated to include the preprocess_train_called attribute? That's my only thought. Any ideas?
Thanks!

Yes, I think you're probably right.  There have been several new releases between the time your originally created your files and now.  There a few things you can try:

Install the same version of ktrain you used originally on Google colab
Using the current version of krain, re-create the preproc object from a scratch and replace the old preproc with the new one.  To do this, you just re-call the data-loading function (e.g., texts_from_folder, texts_from_array, etc.).

If all else fails, you can just retrain and re-create the predictor files (model and preproc), which is the ML equivalent of rebooting your computer.

Today, I encountered this same problem in one of the unit tests.  I solved it by re-training the model and re-creating the predictor objects.  But, downgrading ktrain or re-generating and replacing just the preproc object should also work.
Will close this issue. Thanks.


Support unambiguous arbitrarily ordered flags

Is it possible to support flags in any order? For example:
clitool subcommand arg
clitool subcommand --help arg
clitool subcommand arg --help
Reasoning; with complex commands that have many or long arguments it's fairly annoying to have to seek back to before the arguments and add a flag. Especially if that flag is --help.
I believe flags can be parsed unambiguously so they could technically be mixed with args, but I know this is probably a complex topic.

--help should currently be special cased to work in most locations, but it's possible there are some cases where it doesn't.
That said, I agree. What I would really like to do is be able to specify any flag for any command/subcommand at any point after the command. That is, if "foo" is a flag for command "foo" (ie. foo --foo) then --foo can also be specified to any sub-command of foo. eg. foo bar --foo. Additionally, --help for "bar" should display all ancestor commands and flags.

Agree

I'm not 100% sure this is possible in full.
Imagine this definition:
flag := kingpin.Flag("flag", "").String()
cmd := kingpin.Command("cmd" ,"")
arg := cmd.Arg("arg", "").String()
cmdFlag := cmd.Flag("cmdflag", "").String()

Given this command line:
cli --flag cmd

It's not really possible to determine whether the user meant cmd to be an argument to --flag, or the command cmd.
That said, this commit in the v2-unstable branch lets you specify flags at any point after their definition.
So in the example above one could do:
cli cmd --flag=foo --cmdflag=bar

I think that is probably sufficient, so I'm going to close this issue. Let me know what you think.

While this definition is certainly challenging to program a parser for I don't know if it's ambiguous. If the Flag isn't a boolean flag that means it requires an argument so the next token on the stream would be the argument to the flag. This would consume "cmd" in your first example.
If the flag was a boolean it would not consume the next token on the stack and cmd would be interpreted as the subcommand.


Emissions Import: Add Long Term Fuel Data

As an ECMPS user, I want to be able to import long term fuel data into the emissions module in order to support program goals

[ ] Modify emissions import logic to accept long term fuel data

To test, you should be able to successfully import the following json:
{
  "orisCode": 1913,
  "year": 2017,
  "quarter": 2,
  "longTermFuelFlowData": [
    {
      "unitId": "4",
      "monitoringSystemId": "LM2",
      "fuelFlowPeriodCode":"A",
      "longTermFuelFlowValue":1,
      "longTermFuelFlowUomCode":"SCF",
      "grossCalorificValue":2,
      "gcvUnitsOfMeasureCode":"BTUSCF",
      "totalHeatInput":3
    }
  ]
}


Update develop.md

simulation jobs fail because setup script doesn't set iamRole arn in the roboMakerSettings.json file with a parameter validation error for the empty iamRole.

There are two issues here:

the install_deps.sh script does not correctly escape forward slashes in the role ARN when using sed to replace it in roboMakerSettings.json
the initial bootstrap CloudFormation template does not export ROLE_ARN

See the the following PR I proposed to fix this: codingphun/aws_robomaker_jetbot_reinvent_workshop#3


ARROW-11322: [Rust] Re-opening memory module as public

I am really sorry about that. It was not intended.

I am really sorry about that. It was not intended.

On an related note, allocate_aligned no longer initializes zeroed regions. The region is uninitialized. Use allocate_aligned_zeroed for zeroed, initialized regions.

On an related note, allocate_aligned no longer initializes zeroed regions. The region is uninitialized. Use allocate_aligned_zeroed for zeroed, initialized regions.

Great to know; thanks Jorge!

Great to know; thanks Jorge!


Time to support qt 6.2.x

Please add support for qt 6.2.x

sorry - this will take some time.
I just ported my projects to qt 5.15
now have to do many work to refactor and prepare for qt6
all this must be done for all my customer projects, too
for me I have decided to start with qt 6.4 to have enough time . also there are many things fixed / added for 6.3 and 6.4


🌱 Add v1.2 CAPI release contract to e2e metadata file

/test-ubuntu-integration-release-1-2

/test-ubuntu-e2e-release-1-2

/test-ubuntu-e2e-release-1-2

/test-ubuntu-integration-release-1-2

/retest

/retest


fix sshd_config sample link.

the previous link is broken.

@ebibibi : Thanks for your contribution! The author, @msangapu, has been notified to review your proposed change.

Thank you for your feedback @ebibibi .
#sign-off


Tools Plugin not showing in top menu

I wanted to use this plugin and I have placed this plugin at path :
~/Library/Application\ Support/Hopper/PlugIns/v4/Tools/Swift\ Demangle.hopperTool
But  when i restarted hopper, Tools plugin was not showing in top menu as shown in image here.

Xcode console:  Plugin was not loaded. Please check any error message in the Console application.
Using Hopper Disassembler V4 - Demo
ERROR:
2018-07-15 17:00:45.814732+0500 Hopper Disassembler v4[4702:159347]   Plugin was not loaded. Please check any error message in the Console application.
2018-07-15 17:01:02.135631+0500 Hopper Disassembler v4[4702:159352] Month 13 is out of bounds
objc[4702]: Class FIFinderSyncExtensionHost is implemented in both /System/Library/PrivateFrameworks/FinderKit.framework/Versions/A/FinderKit (0x7fff87014a70) and /System/Library/PrivateFrameworks/FileProvider.framework/OverrideBundles/FinderSyncCollaborationFileProviderOverride.bundle/Contents/MacOS/FinderSyncCollaborationFileProviderOverride (0x11a662cd8). One of the two will be used. Which one is undefined.
2018-07-15 17:01:14.250398+0500 Hopper Disassembler v4[4702:159377] errors encountered while discovering extensions: Error Domain=PlugInKit Code=13 "query cancelled" UserInfo={NSLocalizedDescription=query cancelled}

Thanks I'll try and look into this. Feel free to submit a fix if you get to the bottom of this

default	22:06:13.915024 +0200	Hopper Disassembler v4	  ERROR: Cannot load /Users/___/Library/Application Support/Hopper/PlugIns/v4/Tools/Swift Demangle.hopperTool, plugin makes use of external function(s), and may be too slow for a plugin. Also, make sure that plugin is compiled for OS X 10.9, or higher.
default	22:06:13.915059 +0200	Hopper Disassembler v4	  error: Plugin imports dylib @rpath/Demangler.framework/Versions/A/Demangler

It seems Hopper has mysteriously stopped allowing external frameworks to load in tools. Additionally, if one were to merge Demangler.framework and build all in one, you'd get the same error but over libswiftCore.dylib.

same to me

same to me


Mono doesn't provide a way to define custom signals

To mirror this issue here, at some point this was possible through add_user_signal but that function has been removed, and AFAIK there is no alternative way to define a signal.

IINW add_user_signal is still there. I have in mind an idea for declaring signals in a more friendly way, I'll try to implement it after alpha2.

Ah, sorry, I missed that because grepping add_user_signal on a freshly cloned repo doesn't give anything, but that's apparently because this is generated.

I'll re-open this issue and rename it.

@akien-mga we can close this. C# uses [Signal] annotation now.


⚠ Beamable Stats (Object) has degraded performance

In e67c4f0, Beamable Stats (Object) (https://api.beamable.com/object/stats/platform/docs) experienced degraded performance:

HTTP code: 504
Response time: 10023 ms

Resolved: Beamable Stats (Object) performance has improved in e3443ad after 8 minutes.


Another bug in Star Transformer

#140 fix it .

Codecov Report

Merging #141 into master will increase coverage by 11.94%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           master     #141       +/-   ##
===========================================
+ Coverage   64.32%   76.26%   +11.94%
===========================================
  Files         112       94       -18
  Lines        9390     7918     -1472
===========================================
- Hits         6040     6039        -1
+ Misses       3350     1879     -1471

Impacted Files
Coverage Δ

fastNLP/modules/encoder/star_transformer.py
98.8% <100%> (ø)
:arrow_up:

fastNLP/models/enas_model.py
94.31% <0%> (-1.71%)
:arrow_down:

reproduction/Chinese_word_segmentation/utils.py

reproduction/HAN-document_classification/train.py

reproduction/CNN-sentence_classification/model.py

reproduction/CNN-sentence_classification/train.py

...oduction/HAN-document_classification/preprocess.py

...production/HAN-document_classification/evaluate.py

reproduction/Char-aware_NLM/test.py

... and 12 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 56410c9...28d3f50. Read the comment docs.


set expectation that az aks create ... will take up to N minutes

$ az aks create -n $NAME -g $NAME -c 2 -k 1.7.7 --generate-ssh-keys

Running ..

This takes a while to run...

Agreed. Create a PR.

Updated


add dirty data to job information class

Problem:
Jenkins and Bamboo pass data between handlers, and do some dirty hacks to attempt to determine if data has changed and should issue a save request to the server.
Solution:
Simple solution by adding a hash to track dirty data so we know what to issue an update for.
Reviewers:
@moizjv @mehmetg @mehmetg-sauce
More changes are coming to the various plugins to use this new updated code.

Current coverage is 38.00%

Merging #15 into master will increase coverage by +3.69% as of cb40ee4

@@            master    #15   diff @@
=====================================
  Files           12     12
  Stmts         1052   1063    +11
  Branches       174    178     +4
  Methods          0      0
=====================================
+ Hit            361    404    +43
- Partial         52     56     +4
+ Missed         639    603    -36

Review entire Coverage Diff as of cb40ee4

Powered by Codecov. Updated on successful CI builds.

Cool looks good :+1:

Looks good to me! :+1:
Off topic: This is sharing some functionality with sauce_rest doesn't it?

yep, eventually I think most, or all of the non sauceconnect stuff should
move to saucerest
On Wed, Feb 10, 2016 at 5:22 PM, Mehmet<EMAIL_ADDRESS>wrote:

Looks good to me! [image: :+1:]
Off topic: This is sharing some functionality with sauce_rest doesn't it?
—
Reply to this email directly or view it on GitHub
https://github.com/saucelabs/ci-sauce/pull/15#issuecomment-182666756.

--
Gavin MoganSoftware Developer
Sauce Labs http://saucelabs.com/ | Sauce blog http://sauceio.com/
Talk to us anytime:  https://twitter.com/saucelabs
https://www.linkedin.com/company/sauce-labs
https://plus.google.com/+SauceLabs_Official/posts
https://www.facebook.com/saucelabs


Provide macOS Binaries with Releases

I'm a maintainer of the Ansible degoss role, and I use Goss heavily for testing Ansible roles across distributions and operating systems.
Due to unforeseen circumstances, I now need to port almost all of my roles to run on macOS. Unfortunately so far, Goss has only shipped amd64, arm, and 386 architectures. As Goss is written in Go, creating a binary for macOS shouldn't be too difficult unless Goss relies on specific syscalls and doesn't just use the Go standard library.
Would it be possible to ship macOS binaries in future releases of Goss?

There's some initial work on this:
https://github.com/aelsabbahy/goss/issues/192
https://github.com/aelsabbahy/goss/pull/380
But I don't think anyone compiled goss for OSX and made a list of all the things that work and don't work on there. I know there was some research into freebsd: https://github.com/aelsabbahy/goss/issues/26#issuecomment-265962007
I'm open to pull requests to add support for OSX, so long as there's a way to verify it through CI to ensure it's working correctly. That said, I don't think goss can support OSX without some changes.

I'll also need macOS (and Windows) support within goss, and so am intending to contribute where necessary.
I noticed https://docs.travis-ci.com/user/reference/osx/ so that's promising, but I don't think travis supports Windows so far (https://github.com/travis-ci/travis-ci/issues/2104). AppVeyor can probably be pressed into service for this; it's also free for OSS and supports Windows. @aelsabbahy I've configured it before; would you mind signing up to that service and creating a skeleton appveyor.yml in the repo?

@aelsabbahy would you have any objection to my setting up Windows CI via appveyor and then granting you admin rights into what I create?

Turns out travis started to support Windows. I'm trying that out, and macOS in #403.
@aelsabbahy I have some bandwidth, and some need, to work on this and the macOS and Windows support. Would you be open to adding some collaborators to this repo to aid with pull-request throughput?

We're now publishing Windows and Mac OS binaries at https://github.com/improbable-eng/goss/releases (but we have not ensured all features work on all platforms). We're gradually making changes to get to feature parity with Linux. You're welcome to use and give feedback.

This was really nice to see that there is some project to build MacOs binaries. And I see migrate tag was added. Is there any ETA when this can be migrated to this repo.

The migrate label is about re-writing the ticket in the new format:

I have a new ticket/discussion that will be posted in a few days to discuss all the requirements needed for multi-os support to be added to Goss.

https://github.com/aelsabbahy/goss/issues/551
The pre-req discussion for multi-os support has moved to the above ticket. New tickets will be created based on that discussion.

Community supported macOS alpha binaries are now part of goss releases.


[bug] OpenFunctions-v2: how to continue conversation?

This is more like a question rather than a bug, but what to do after the model returns a function call?
I followed this example:
To help with quick prototyping, we provide a hosted Gorilla Openfunctions-v2 model for inference. Or you can run it locally, or self-host it by accessing the model from [HuggingFace](https://huggingface.co/gorilla-llm/gorilla-openfunctions-v2). The example below, demonstrates how to invoke the hosted Gorilla Openfunctions-v2 model:
import openai

def get_gorilla_response(prompt="", model="gorilla-openfunctions-v2", functions=[]):
    openai.api_key = "EMPTY"  # Hosted for free with ❤ from UC Berkeley
    openai.api_base = "http://luigi.millennium.berkeley.edu:8000/v1"
    try:
        completion = openai.ChatCompletion.create(
            model="gorilla-openfunctions-v2",
            temperature=0.0,
            messages=[{"role": "user", "content": prompt}],
            functions=functions,
        )
        # completion.choices[0].message.content, string format of the function call
        # completion.choices[0].message.functions, Json format of the function call
        return completion.choices[0]
Prompt the model:
What's the weather like in the two cities of Boston and San Francisco?
Format your function call: The model will return the function call based on your request.
query = "What's the weather like in the two cities of Boston and San Francisco?"
functions = [
    {
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA",
                },
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
            },
            "required": ["location"],
        },
    }
]
Get Your Function Call: The model will return a Python function call based on your request.
This opens up possibilities for developers and non-developers alike, allowing them to leverage complex functionalities without writing extensive code.
Input:

get_gorilla_response(prompt=query, functions=[functions])
Output:

[get_current_weather(location='Boston, MA'), get_current_weather(location='San Francisco, CA')]

and assume I have function to check the weather of Boston and SF, how do I append those results back and continue the conversation?

Thanks for trying it out @tybalex
Let me preface it by mentioning that, OpenFunctions-v2 is currently tested for single-turn only, so multi-turn behavior is ill-defined. Anyways if you want to try it out. You would do as such:
completion = openai.ChatCompletion.create(
           model="gorilla-openfunctions-v2",
           temperature=0.0,
           messages=[{"role": "user", "content": prompt_1},
                                {"role": "assistant", "content": response_1},
                                {"role": "user", "content": prompt_2}],
           functions=functions,
       )

where you would replace prompt_1 is what you input the first time, response_1 is the LLM function call returned by the LLM, and prompt_2 is what you want to continue the convsersation.. Hope this helps!


'Unable to open logs' error.

I've tried to follow your instructions, but when I start the docker container, I get the following messages:
Initialize empty Git repository in /var/lib/git/dummy.git
Initialized empty Git repository in /var/lib/git/dummy.git/
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using <IP_ADDRESS>. Set the 'ServerName' directive globally to suppress this message
(13)Permission denied: AH00091: httpd: could not open error log file /etc/httpd/logs/error_log.
AH00015: Unable to open logs

Thanks
Mark.

Can you please provide an exact command line that you user to start the container?
I was not able to reproduce your issue using the commands:
$ docker build -t gitweb_gitweb .
$ docker run -it --name gitweb_test -p 8080:8080 gitweb_gitweb

and tested with:
$ git clone http://localhost:8080/git/dummy
Cloning into 'dummy'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.

Closing this issues as no feedback was provided


Port Docker (GitHub Packages) CI Recipe

Description
Since I have this recipe in the past, I should port it in.
Expected Behavior
Packaging and releasing docker to GitHub Packages is seamless.
Current Behavior
Packaging and releasing docker to GitHub Packages requires manual efforts.
Associated Data Files
No response

Woah, this is great!

Yeap. Implemented.


invalid tag format: tag ending with \\

influxdb version: 1.8.6
influxdb client version: 5.3.1
data = 'check_invalid_tag,mount_point=d:\\\\,physical_filename=d:\\\\MicrosoftSQLServer\\\\MSSQL14.X3V12PRE\\\\MSSQL\\\\Data\\\\tempdb.mdf size=50<PHONE_NUMBER>'

influx_client.write_points([data], time_precision='s', database='arindam', protocol='line')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/influxdb/client.py", line 608, in write_points
    consistency=consistency)
  File "/usr/local/lib/python3.5/dist-packages/influxdb/client.py", line 685, in _write_points
    protocol=protocol
  File "/usr/local/lib/python3.5/dist-packages/influxdb/client.py", line 419, in write
    headers=headers
  File "/usr/local/lib/python3.5/dist-packages/influxdb/client.py", line 378, in request
    raise InfluxDBClientError(err_msg, response.status_code)
influxdb.exceptions.InfluxDBClientError: 400: {"error":"unable to parse 'check_invalid_tag,mount_point=d:\\\\,physical_filename=d:\\\\MicrosoftSQLServer\\\\MSSQL14.X3V12PRE\\\\MSSQL\\\\Data\\\\tempdb.mdf size=50<PHONE_NUMBER>': invalid tag format"}

looks like it's a influxdb-python bug


training videos in mva not available.

Hey I am a new for the UWP programming and found out that MVA was shutting down, so the Template 10 training videos are no longer available. anyone know where I can download them and watch?

Cool, we're talking about what to do now.

MVA is gone. Does Template10 and/or PRISM have any future ? 5 months after johnbnstx comment and nearly 3 months after "we're talking about what to do now" ! TIA


include MIT license in build artifacts

MIT license should be embedded into the distributed artifacts/builds.
Fixes #1352

thanks @rchande


Make a snazzy homepage

It's gonna be snazzy!

Pretty much done - needs more lifecycle/matrix diagrams etc. to make it more eye-catching but not a necessity at this stage


There's no code!

Consider using godot and/or blender...

I installed godot on my machine, but I'm having trouble running it with Visual Studio
I had to install miniforge, then scons, and then I added scons.exe to my Window PATH
C:\Users\mattd\miniforge3\envs\godot\Scripts\

I also can't figure out how to copy my godot/ folder into the gg/ repo without it turning into a submodule..

ok, I used git subrepo on a linux machine to add thirdparty/godot and games/blockbot to this repo. I had created both as other repos on my git account, but I wanted them to be combined into gg. The subrepo command makes that easy.

Since there is now some code, I am closing this ticket.


Add spotless

Enforce style

@vlsi I'd like to do something like https://github.com/apache/avro/blob/master/.github/workflows/spotless.yml but I don't know how to apply it for all modules
❯ ./gradlew spotlessCheck
Type-safe project accessors is an incubating feature.

> Configure project :
Building testng 7.5-SNAPSHOT

FAILURE: Build failed with an exception.

* What went wrong:
Task 'spotlessCheck' not found in root project 'testng-root'.

Any idea?

I'm biased to https://github.com/autostyle/autostyle

not found in root project 'testng-root'.

That is the project defined by /build.gradle.kts

I'm biased to autostyle/autostyle

I supposed you are ;)
No offense but spotless looks to have a bigger community and that's why I choose it.

That is the project defined by /build.gradle.kts

Sure! I've already tried:
plugins {
    id("testng.repositories")
    id("com.github.vlsi.stage-vote-release")
    id("com.diffplug.spotless")
}

But my gradle skills are too low and I don't understand the complaint:
* What went wrong:
Plugin [id: 'com.diffplug.spotless'] was not found in any of the following sources:

- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Included Builds (None of the included builds contain this plugin)
- Plugin Repositories (plugin dependency must include a version number for this source)

I tried to follow what you did with Sonarqube, have I added the good files at the good places?

About Sonarqube, why did you apply it only to the core module?

I'm biased to autostyle/autostyle

Do you plan to add the ratchetFrom feature?
TBH, I don't want to "fix" all the base code but only check the diff from PR.

@juherr , I think it is worth fixing all the issues in a single go because:

The codebase is really small, and there's no issue with just fixing it
You have a clear view on what is changed, and you can tweak the settings
If someone has merge-conflicts, they just run gradlew ...Apply and that is it
I would recommend to avoid adding rules that require manual intervention. In other words, if something is autofixable, then it is fine. If something is not autofixable, it would be better to avoid adding it to the code style.


feat: wallet client & wallet admin

Related issues 🔗
Closes #44
Description ℹ

rename wallet-client to wallet-admin
create wallet-client, template it for the client wallet API specs with built in type support and configurable fetching
add some smart features to the jsonrpc generator:

allow template document URLs in the codegen config so that the json rpc specs come from the environment instead of having it hardcoded
allow glob patterns on for selecting methods to use as the base of the generate

set up a webhook to be able to auto-generate the wallet-client and wallet-admin when the wallet API specs change - this will be have to called from the vega repo's workflow upon tagging - the webhook will create a PR to prepare the release of the two frontend libs.

Details
The wallet client can be used the following way, as an example for connecting, requesting permissions, listing the public keys and sending a transaction request:
import { WalletClient } from '@vegaprotocol/wallet-client'

const token = retrieveToken()
const walletClient =  new WalletClient({ hostname: 'http://localhost:1789', token })

{
  const { result: { chainId } } = await walletClient.GetChainId()
  ...
  const { result: { token } } = await walletClient.ConnectWallet()
  saveToken(token)
  ...
  await walletClient.RequestPermissions()
  ...
  const { result: { keys } } = await walletClient.ListKeys()
  await walletClient.SendTransaction({
    publicKey: keys[0].publicKey,
    sendingMode: 'TYPE_SYNC',
    transaction: {...}
  })
}

NOTE: retrieveToken and saveToken are just pseudo code for any sort of way you want to store a token received from the wallet API
The instance will automatically store the token in runtime memory once you connect to a wallet, you'll have to implement the persistence if you need it by digging it out from the response tho.
It also exposes a mock server running on express, you can interact with it on the same endpoint as with the wallet service, e.g. curl -H "Origin: http://dapp.xyz" -H "Content-Type: application/json" --data '{...}' http://localhost:1789/api/v2/requests

@edd the wallet-ui is using the wallet-admin as a prop which it expects to be passed in. the wallet-admin is indeed not useful anywhere outside the wallet's scope, as it's not exposed. but it is used internally in the desktop-wallet and in the browser-wallet.

@notbot00 did this quick sketch that helped me understand where wallet-admin fits:


Incorrect rendering of  and  combination. Missing global styles?

Detailed description
I'm experimenting with the UI shell components at the moment and I noticed some differences in rendering to the "official" components (e.g. react, Vanilla variants). I'm not sure if I'm doing something wrong.
My HTML is minimal:
<!DOCTYPE html>

    <title>m14r</title>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />

    <m14r-main></m14r-main>

My Component looks like this:
import {
  css,
  customElement,
  html,
  LitElement,
  TemplateResult,
} from "lit-element";
import "carbon-web-components/es/components/ui-shell/header";
import "carbon-web-components/es/components/ui-shell/header-name";
import "carbon-web-components/es/components/ui-shell/header-menu-button";
import "carbon-web-components/es/components/ui-shell/side-nav";
import "carbon-web-components/es/components/ui-shell/side-nav-link";

@customElement("m14r-main")
export class Main extends LitElement {
  static styles = css`
    * {
      box-sizing: border-box;
    }

    bx-header ~ bx-side-nav {
      margin-top: 3rem;
      height: calc(100% - 3rem);
    }

    bx-header ~ .bx--content {
      margin-left: 16rem;
      margin-top: 3rem;
      height: calc(100% - 3rem);
      width: calc(100% - 16rem);
    }
  `;

  render(): TemplateResult {
    return html`
      <bx-header>
        <bx-header-menu-button></bx-header-menu-button>
        <bx-header-name prefix="">M14R</bx-header-name>
      </bx-header>
      <bx-side-nav>
        <bx-side-nav-link href="javascript:void(0)" active
          >Item</bx-side-nav-link
        >
        <bx-side-nav-link href="javascript:void(0)"
          >Another Item</bx-side-nav-link
        >
      </bx-side-nav>

UI Shell Content

          Pellentesque habitant morbi tristique senectus et
          netus et malesuada fames ac turpis egestas. Vestibulum tortor quam,
          feugiat vitae, ultricies eget, tempor sit amet, ante. Donec eu libero
          sit amet quam egestas semper.
          Aenean ultricies mi vitae est. Mauris placerat eleifend leo.
          Quisque sit amet est et sapien ullamcorper pharetra. Vestibulum erat
          wisi, condimentum sed, commodo vitae, ornare sit amet,
          wisi. Aenean fermentum, elit eget tincidunt condimentum, eros ipsum
          rutrum orci, sagittis tempus lacus enim ac dui.
          Donec non enim in turpis pulvinar facilisis. Ut felis.

    `;
  }
}

Without the added styles, I get incorrect rendering like this:

Specifically:

The spacing rule for header (and many similar) seems to be globally defined in the Carbon stylesheets. However, they were missing from my environment. Am I supposed to wrap the custom element invocations in a specific way to get them? Including the carbon css globally didn't work because these styles don't affect m14r-main's shadow dom. Also I understood it would be incorrect to do so anyways, as the web components already carry the relevant rules in their constructed style sheets. What did work, but leads to styles being loaded twice, is to return a <link rel=stylesheet> from render(). Defining this myself feels wrong:

    bx-header ~ bx-side-nav {
      margin-top: 3rem;
      height: calc(100% - 3rem);
    }

box-sizing: border-box; seems to come from css-reset in the storybook. Missing this rule even with the bx-header ~ bx-side-nav  above, I would get a 1px error where the side-nav would overlap the header (as the header has a bottom border of 1px). As css-reset has many other rules I'm not sure if I'm missing some additional ones that just didn't appear yet in my testing.

Is there a way to somehow grab just the "global" style bits into my components? Including the sibling styles (for spacing, etc.), reset styles and other stuff that doesn't really belong to a specific element like , and I might be missing. Then I could include them in each of my elements like <m14r-main> above (assuming they don't weigh a ton because they will be duplicated in all of my components).

I wondered if defining  myself is correct. I noticed this is a Carbon concept, and there are rules and mixins (carbon-content ?) defined, but couldn't find an equivalent wrapper in carbon-web-components. I looked for something like <bx-content> which would encapsulate the layout and typography, etc.

Hi @pschyska thank you for those questions!

Created #549 to clearly document that the style needs to be added in the application code.
It can be done by importing _css--reset.scss into your Sass code, or by generating CSS from that Sass file.
The  should be replaced with your application content.  is there solely for demonstration purpose.

Hope above helps - Let us know if you have any follow-up questions.

I think bx-header ~ bx-side-nav is not enough in general. There are more things to consider: Any type of UI shell content must be pushed down (let's say a rule bx-header ~ bx--content), and when the side nav is supposed to be fixed, which it is IIUC by default when using the header+responsive-mode side-nav (and screen is above the breakpoint), the content should have a left margin of some sort. I think it's unfortunate to not being able to use the default Carbon rules (I'm completely new to the design system as well, so I don't know if this issue would pop up often, or if it's isolated with the ui shell components).
I realize it's not trivial, because the upstream rules - i.e. .#{$prefix}--header ~ .#{$prefix}--side-nav - wouldn't work due to isolation and the fact that in the web components case bx--header and bx--side-nav aren't direct siblings in the first place (for that reason it's also probably impossible to have the derived components adopt a shared style sheet containing all the cross-cutting rules like that).
I had hoped I could get around pulling a sass processor into my build. So far I could get away with a simple esbuild-based solution :-) I think it would be great if carbon-web-components could provide a CSSResult somewhere as importable ES module. Then users could:

import {reset} from "carbon-web-components/es/something/something/css-reset";

@customElement("m14r-main")
export class Main extends LitElement {
  static styles = [reset, css`
    :host {
      something...
    }
  `]
}

There is a way to make this efficient in the browser, so that all the components using reset adopt the same constructed style sheet and don't incur the costs each time it's included in a component (which could be dozens of time if you build small components). I don't know if the simplistic solution above (importing CSSResult and refering) would do that though.

I understand that the  is just a placeholder here. I was talking about the ruleset associated to bx-content. I would have to pull it into each of my components because there is no component in carbon-web-components that bundles that. So bx--content is defined for UI shell here, but I have to know and look it up and define it myself. I'd rather have a way to write something like <bx-content><mainorwhatever>.. and get these rules automatically.

Anyhow, I was wondering if I'm doing anything wrong in principle and it appears I'm not, so I'm gonna close this issue.


Info Window tap changes;

This pull request adds in a callback for clicking on the map info window. It also updates the google play services to 15.+ to avoid runtime issues with other plugins using a later version of google play.
To use the callback just do something like this in your flutter code:
controller.onInfoWindowTapped.add((Marker marker) {
});
Note that this feature was mention in this issue: #17730
The google play issue was mentioned in this issue: #17926

I signed it


fix(core): assign baseUrl property

#721 did declare baseUrl property, but did not assign that.
This PR fixes it to fix problems something like:
failed to handle: base_dir() must be absolute. Please ensure that jsc.baseUrl is specified correctly.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@Brooooooklyn
Would you please review this PR again?

thanks!


Compared with appgyver and more

Hello,
Today I noticed this project and thought it might be interesting for us to use and contribute to. I do have a couple of questions I hope you could answer...

How much of this is actually native?
When is android support coming?
How does this compare to AppGyver?
How does this compare to nativescript?
How does one use native plugins for stuff like local push notifications, facebook login (heh :)) etc using react-native?
How much of the stuff being created by react-native is actually native?
Finally, perhaps a bit more react specific, is it possible to write the markup elsewhere? So not inside of the javascript code?

I must admit I didn't fully research all of these questions, but I was hoping they'd inspire whoever answers this to feed me with even more information.
Thanks!

Hi @RWOverdijk! Your enthusiasm is very welcome here, that said the issues are probably not the best place to ask these questions. Some quick answers for you below, hope it helps, please direct further questions to the irc channel #reactnative on Freenode or StackOverflow.

As much as you want it to be
http://www.reactnativeandroid.com/
React powered (huge win), Facebook backed, already deployed on large apps. Never used AppGyver.
React powered (huge win), Facebook backed, already deployed on large apps. Never used NativeScript, similar idea though (same goes for Titanium)
See other issues for push notifications, local notifications work but the API isn't great at the moment, we'll be improving soon. Remote push works fine at the moment. Facebook login: https://github.com/brentvatne/react-native-login or https://github.com/magus/react-native-facebook-login
Everything you see on the screen is native backed. Whether it is an actual native component or one built in JS by using native primitives depends.
Nope ;)

Best,
Brent

@brentvatne Thanks! I'll direct further questions (which I already have) to IRC tomorrow.

@RWOverdijk there is additionally https://discuss.reactjs.org/ which may be of some help.

And previous issues of http://brentvatne.ca/react-native-newsletter
And https://github.com/ericvicenti/react-native-community (although not entirely kept up to date)
And https://www.youtube.com/results?search_query=react+native


Please add Login error code 50199

The Login error Code 50199 is not documented

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fa732de6-2cab-d780-aa21-dfec80e0a8fe
Version Independent ID: 1a11b9b6-cf4f-3581-0864-0d5046943b6e
Content: Error codes in the Azure Active Directory portal
Content Source: articles/active-directory/reports-monitoring/reference-sign-ins-error-codes.md
Service: active-directory
Sub-service: report-monitor
GitHub Login: @MarkusVi
Microsoft Alias: markvi

@chino0913, Thanks for the feedback! We are taking a look into this and will get back to you soon.

@chino0913, Can you please share some more details around this Error Code, Like what steps got you this error and what does the description of this error states.

@chino0913, Closing this thread as got no further updates.

I'd like to reopen that error as I've seen this error code in my logins in combination with iOS Accounts App login into Exchange Online.  (See screenshot below for details.) The error occurs occasionally an is not directly reproducible but happens on different users throughout the organization.

Additional vote to reopen, I also have this error code similar to above.

Dear MS, this error still appears. Please document this error.


Units tests

Currently Units tests are a mess and somehow not deterministic, at least they work in a Windows machine but when been executed by GitHub actions they always crash. Goals:

[x] Cleaner code
[x] Deterministic results
[x] Concurrent execution
[x] Remove any sleep action

The tool itself works perfectly but it will be great to ensure these tests also pass in google actions:

[ ] TestSocks5BindAcceptFilter

This should be fixed with new V4


add support for Android

Since iOS support was asked (#233), I was wondering if anyone asked for Android support and was surprised to find nothing about it.
So here I am, I'd be happy if there was an Android support - and that would definitely make me want to do the switch from Tauri to Floem.

That's awesome feedback to hear! iOS and android support would are desired features but we don't have the current bandwidth to work on them now. We will eventually get there but we would also accept any help on adding that support in the meantime


fix(client): Export metrics types

Is there an issue we can close so this gets added to the list of close issues in the release notes?

Closed via #14016 13971

For changelog: #14016


MAPREDUCE-7267 Merge paths with multi threads during commit job in FileOutputCommitter

cc @steveloughran

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 33s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
1m  7s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m  3s
trunk passed

+1 :green_heart:
compile
16m 58s
trunk passed

+1 :green_heart:
checkstyle
2m 39s
trunk passed

+1 :green_heart:
mvnsite
2m 30s
trunk passed

+1 :green_heart:
shadedclient
20m 43s
branch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 52s
trunk passed

+0 :ok:
spotbugs
1m 10s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 23s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 25s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 36s
the patch passed

+1 :green_heart:
compile
16m 18s
the patch passed

+1 :green_heart:
javac
16m 18s
the patch passed

-0 :warning:
checkstyle
2m 42s
root: The patch generated 10 new + 66 unchanged - 0 fixed = 76 total (was 66)

+1 :green_heart:
mvnsite
2m 29s
the patch passed

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  2s
The patch has no ill-formed XML file.

+1 :green_heart:
shadedclient
14m 21s
patch has no errors when building and testing our client artifacts.

+1 :green_heart:
javadoc
1m 41s
the patch passed

+1 :green_heart:
findbugs
3m 45s
the patch passed

_ Other Tests _

+1 :green_heart:
unit
5m 39s
hadoop-mapreduce-client-core in the patch passed.

-1 :x:
unit
33m  0s
hadoop-mapreduce-client-jobclient in the patch passed.

-1 :x:
unit
0m 43s
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
1m 12s
The patch does not generate ASF License warnings.

152m 48s

Reason
Tests

Failed junit tests
hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp

Subsystem
Report/Notes

Docker
Client=19.03.8 Server=19.03.8 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1916

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml

uname
Linux 4d206f580620 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / 745a6c1

Default Java
1.8.0_242

checkstyle
https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/artifact/out/diff-checkstyle-root.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt

unit
https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt

Test Results
https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/testReport/

Max. process+thread count
1605 (vs. ulimit of 5500)

modules
C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-aws U: .

Console output
https://builds.apache.org/job/hadoop-multibranch/job/PR-1916/1/console

versions
git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1

Powered by
Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Comment

+0 :ok:
reexec
0m 33s
Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  1s
No case conflicting files found.

+1 :green_heart:
@author
0m  0s
The patch does not contain any @author tags.

+1 :green_heart:
test4tests
0m  0s
The patch appears to include 4 new or modified test files.

_ trunk Compile Tests _

+0 :ok:
mvndep
0m 27s
Maven dependency ordering for branch

+1 :green_heart:
mvninstall
19m 59s
trunk passed

+1 :green_heart:
compile
23m 31s
trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
compile
20m 46s
trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09

+1 :green_heart:
checkstyle
3m  8s
trunk passed

+1 :green_heart:
mvnsite
2m 29s
trunk passed

+1 :green_heart:
shadedclient
22m 40s
branch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 35s
hadoop-mapreduce-client-jobclient in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

-1 :x:
javadoc
0m 43s
hadoop-aws in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

+1 :green_heart:
javadoc
1m 48s
trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09

+0 :ok:
spotbugs
1m 24s
Used deprecated FindBugs config; considering switching to SpotBugs.

+1 :green_heart:
findbugs
3m 48s
trunk passed

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 28s
Maven dependency ordering for patch

+1 :green_heart:
mvninstall
1m 49s
the patch passed

+1 :green_heart:
compile
23m 33s
the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1

+1 :green_heart:
javac
23m 33s
the patch passed

+1 :green_heart:
compile
18m 16s
the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09

+1 :green_heart:
javac
18m 16s
the patch passed

-0 :warning:
checkstyle
2m 45s
root: The patch generated 10 new + 66 unchanged - 0 fixed = 76 total (was 66)

-1 :x:
mvnsite
0m 43s
hadoop-mapreduce-client-jobclient in the patch failed.

-1 :x:
mvnsite
0m 46s
hadoop-aws in the patch failed.

+1 :green_heart:
whitespace
0m  0s
The patch has no whitespace issues.

+1 :green_heart:
xml
0m  2s
The patch has no ill-formed XML file.

+1 :green_heart:
shadedclient
14m 12s
patch has no errors when building and testing our client artifacts.

-1 :x:
javadoc
0m 33s
hadoop-mapreduce-client-jobclient in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

-1 :x:
javadoc
0m 38s
hadoop-aws in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.

+1 :green_heart:
javadoc
1m 45s
the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09

-1 :x:
findbugs
0m 40s
hadoop-mapreduce-client-jobclient in the patch failed.

-1 :x:
findbugs
0m 46s
hadoop-aws in the patch failed.

_ Other Tests _

+1 :green_heart:
unit
7m 10s
hadoop-mapreduce-client-core in the patch passed.

-1 :x:
unit
0m 40s
hadoop-mapreduce-client-jobclient in the patch failed.

-1 :x:
unit
0m 44s
hadoop-aws in the patch failed.

+1 :green_heart:
asflicense
0m 46s
The patch does not generate ASF License warnings.

179m 20s

Subsystem
Report/Notes

Docker
ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/1916

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle xml

uname
Linux 58553411e528 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
personality/hadoop.sh

git revision
trunk / e756fe35909

Default Java
Private Build-1.8.0_252-8u252-b09-1~18.04-b09

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/branch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

checkstyle
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/diff-checkstyle-root.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-mvnsite-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt

mvnsite
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-mvnsite-hadoop-tools_hadoop-aws.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-javadoc-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

javadoc
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt

findbugs
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-findbugs-hadoop-tools_hadoop-aws.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt

unit
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/testReport/

Max. process+thread count
1646 (vs. ulimit of 5500)

modules
C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient hadoop-tools/hadoop-aws U: .

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1916/3/console

versions
git=2.17.1 maven=3.6.0 findbugs=4.0.6

Powered by
Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.


Feature request: Data Layer capability

User Story
As a Analytics consultant I would like to be able to track the whole commerce flow in a standardised way without the need of code changes on component/template level.
Description & Motivation
Is it planned to add Data Layer capability to the CIF Core Components? If so, when will this be available?
Deliverables
tbd
Acceptance Criteria
tbd
Verification Steps
tbd

@retozigerlig yes that is planed and we will add Data Layer capabilities as well. I let @aditidc comment on the roadmap.
You are also free to add it and create a PR :-)

Thanks for your reply @mhaack . Where can I find the mentioned roadmap?

Can I take this issue if no one is working on it?

@creatorpiyush feel free to submit a PR :)

This was added by https://github.com/adobe/aem-core-cif-components/pull/415


Initial support for importing address diversified pointer auth qualified field function pointers

This does not yet include code generation of copy functions needed for such pointers.

@swift-ci test

@swift-ci test windows platform


sedutil-passhasher.py: Add sha512 option

Sedutil forks such as the one at sedutil.com have switched to SHA512 since SHA1 has been broken for a few years. This patch adds an option to hash the password using SHA512 to match the algorithm used when setting up the drive using one of these forks.

Thanks for the PR. I have no spare drive to test, but it looks simple enough to just review and say "look good to me" and merge as is. Good to know some active forks of sedutil exists...
I'll update the readme and tag a release late this or next weekend.


Payconiq transaction visible in Payconiq portal, not in OCA dashboard

Search Mobicage google drive for "Logs - Payconiq missing orders for payments".
Looks like both iPhones didn't receive the kick to update the payment step. I suggest to do a getTransaction call or so. 5 seconds after the app returns to the foreground and no updateTransaction is received.

all apps need to be updated for the workaround to take effect


GST API for Master Data

IT is discussed with Vinodji .. about API cost and Operation.
Some API are PUBLIC API which does not required OTP verification..
One of API is

GST Search - It will give Master data of company as under

We will use this data to prefill all fileds on creation of new company

Process will be as under
We will ask for GST Number to get details via API call.
Cost will be X/API Call .. the above data will cost as

API for Authorisation
API for Data Call

please confirm this,

Design
Multiple Import Design
Pricing
New On Boarding Company - changes / usability for importing Company / Contact
any other use cases other than this
Background Task / duplication issue handling

URL: [GET] https://app.mastergst.com/publicsearch1?gstin=24ADOFS0200R1ZV&ipAddress=<IP_ADDRESS>
Header: "Referer" Value "https://mastergst.com/"
Discussed with Vinod sir... to use above link to get details.. and Approved by Sir.

Alternate URL
https://blog-backend.mastersindia.co/api/v1/custom/search/gstin/?keyword=24APWPC5810K1ZP&unique_id=L9dPokJ0DqVv4pcPblVRULZTqqtWR9


Fixed Authority Logs Typo

Codecov Report

Merging #5430 (697ab76) into dev (b6e7656) will decrease coverage by 0.31%.
The diff coverage is 100.00%.

Flag
Coverage Δ

*Carryforward flag

msal-angular
96.50% <ø> (ø)

Carriedforward from 8ee7fc6

msal-browser
86.47% <ø> (ø)

Carriedforward from 8ee7fc6

msal-common
85.37% <100.00%> (ø)

msal-core
82.84% <ø> (ø)

Carriedforward from 8ee7fc6

msal-node
83.33% <ø> (ø)

Carriedforward from 8ee7fc6

msal-node-extensions
76.03% <ø> (?)

Carriedforward from 8ee7fc6

msal-react
94.68% <ø> (ø)

Carriedforward from 8ee7fc6

node-token-validation
88.88% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

Impacted Files
Coverage Δ

lib/msal-common/src/authority/Authority.ts
91.39% <100.00%> (ø)

...-extensions/src/persistence/KeychainPersistence.ts
88.23% <0.00%> (ø)

...msal-node-extensions/src/error/PersistenceError.ts
74.07% <0.00%> (ø)

extensions/msal-node-extensions/src/Dpapi.ts
100.00% <0.00%> (ø)

...ions/msal-node-extensions/src/utils/Environment.ts
42.59% <0.00%> (ø)

...nsions/msal-node-extensions/src/utils/Constants.ts
100.00% <0.00%> (ø)

extensions/msal-node-extensions/src/index.ts
100.00% <0.00%> (ø)

...tensions/src/persistence/PersistenceCachePlugin.ts
75.67% <0.00%> (ø)

...node-extensions/src/persistence/FilePersistence.ts
73.77% <0.00%> (ø)

...c/persistence/FilePersistenceWithDataProtection.ts
81.08% <0.00%> (ø)

... and 5 more


/bin/bash permission denied

Hey folks
I'm running docker 1.8.2 on Ubuntu 14.04
It used to work great but now nothing works. Each time I want to run or build something I got the same error /bin/bash permission denied.
Step to reproduce (on my machine only)

Run docker run -ti ubuntu:trusty /bin/bash

Results received:
exec: "/bin/bash": permission denied
Error response from daemon: Cannot start container 1db470dc718cc5110404bd4b04e88eb21ed6539e30e0042dc5915e19526376e5: [8] System error: exec: "/bin/bash": permission denied

Results expected:
Open a running container with bash open
The same happens if I write only bash or sh or /bin/sh. On this machine, both bash and sh are working well. I tried as root. Same result
How can I debug that? I found nothing on google.
docker version:
Client:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Thu Sep 10 19:19:00 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.2
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   0a8c2e3
 Built:        Thu Sep 10 19:19:00 UTC 2015
 OS/Arch:      linux/amd64

docker info:
Containers: 27
Images: 9
Storage Driver: aufs
 Root Dir: /media/alexis/Windows8_OS/docker/aufs
 Backing Filesystem: <unknown>
 Dirs: 63
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-39-generic
Operating System: Ubuntu 14.04.1 LTS
CPUs: 4
Total Memory: 3.559 GiB
Name: pegase
ID: FUUM:LIMJ:GXUM:CDOY:P6OJ:JLB2:Y7N6:O3GX:6GIX:EDF2:7LAX:GZ4H
Username: alexislg
Registry: https://index.docker.io/v1/
WARNING: No swap limit support

uname -a:
Linux pegase 3.13.0-39-generic #66-Ubuntu SMP Tue Oct 28 13:30:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Just a long shot; is there something special with /media/alexis/Windows8_OS/docker/aufs? (I see the output shows Backing Filesystem: <unknown>)

Oh you were right : /media/alexis/Windows8_OS/docker/aufs is a NTFS mounted volume. I put #DOCKER_OPTS="-g /media/alexis/Windows8_OS/docker" in my /etc/default/docker
If I remove this line and restart docker it's working. Can't we store docker data on external volume?

I've never tested its but I suspect that aufs (or the other layered file systems) doesn't work properly on NTFS. It certainly isn't a setup we support at this moment.

@alexislg2 Storage drivers that might work are devicemapper as a loopback, or vfs. Neither are considered production-grade (or particularly fast), but if you're using a Windows 8 mount you're probably OK with that.
For more info:
https://docs.docker.com/reference/commandline/daemon/#daemon-storage-driver-option

I'm going to close this issue because this is not something we support, but feel free to continue the discussion here.
Thanks for the extra hints, @tjdett


Update the supported currencies

This PR will update the currently, and officially, supported currencies:
http://help.2checkout.com/articles/Knowledge_Article/2Checkout-Offers-87-Currency-Options
The code only contains the list of currencies supported in the US since there is no check of which country that the WooCommerce store is operating in.
It has been confirmed and working for the addition of IDR as of commit 9accc73

Looks good. Thanks for taking care of this dude!


feat: add paginator for opensearch

With this change, we can use paginator for the OpenSearch APIs with Max Results limit, instead of looping with NextToken manually.
# a trivial example
instances = []
opensearch = boto3.client('opensearch')
paginator = opensearch.get_paginator('describe_reserved_instances')
for page in paginator.paginate():
    instances.extend(page['ReservedInstances'])

fixes: #3245

Thanks for creating this PR, but we can't accept these changes for reasons described in https://github.com/boto/botocore/issues/3245#issuecomment-2332598532


mwc-button highlight fill color doesn't scroll under while the rest of the button does

** PLEASE READ THIS BEFORE FILING AN ISSUE **
I'm submitting a:

[ X] bug report
[ ] feature request

What OS are you using?
Mac OS
What browser(s) is this bug affecting:
Chrome 79.0.3945.88
Current behavior:
Unselected button with no fill not highlighted scrolls under the top header and disappears as expected.
But when selected the button fill color does not dissappear.
Expected behavior:
Fill color of selected button should have z index adjusted appropriately.
Steps to reproduce:

Related code:
// insert any relevant code here or add a include a codepen

Other information:

I'm not sure I understand your use case. Can you provide a reproduction?

Closing for inactivity, please re-open with a reproduction if you're still experiencing issues!


Error when Importing to my Project

Hello suragch.
I tested your final project and it works and no error at all.
But unfortunately when I imported your project into my project, there are errors everywhere especially the one with value.
Like this file: audio_handler.dart lines 39 (Error on value)
playbackState.add(playbackState.value.copyWith(  <--- value is error here, very strange
        controls: [
          MediaControl.skipToPrevious,
          if (playing) MediaControl.pause else MediaControl.play,
          MediaControl.stop,
          MediaControl.skipToNext,
        ],

The error says:
"The getter 'value' isn't defined for the type 'BehaviorSubject'.
Try importing the library that defines 'value', correcting the name to the name of an existing getter, or defining a getter or field named 'value'."
I had done "Project Clean", but nothing happen.
It still gives same error.
I don't change any of your codes yet.
Any clue why this happen?

BehaviorSubject is part of rxdart, which audio_service uses. Are you using rxdart in your project already? There might be a conflict (that could be solved by using the same version of rxdart).

No, I haven't used any rxdart at all.
Maybe I will try to add one version of rxdart.

Okay, now it works all of them.
I just need to add null check (!) after value.
It seems I need to import rxdart.dart first in audio_handler.dart.
Thank you once again for your help, suragch.
God bless you.
May you always be healthy and wealthy.

What version of audio_service are you using in pubspec.yaml? I had to had the ! In previous version but not in the most recent version.

audio_service: ^0.18.0-beta.0
The same one as written in your project.yaml

Ok, that's strange then that value is nullable in your project but not in mine. My last guess would be to double check pubspec.lock.

I think some of them were lost when cleaning the project, especially image.
Probably that was the cause?

I'm not sure. Let me know if you ever figure it out.

I used to leave it as empty for GetX version in pubspec.yaml. But today, I felt I had to add one in GetX.
After I updated the GetX to the latest in pubspec.yaml, the errors show up in all null checks I put earlier. It forced me to remove all of them. After I removed them, there are no errors. Strange.