cmd/compile: illegal combination 02044

What version of Go are you using (go version)?

$ go version
go version go1.19.3 linux/arm

Can also repro on 1.19.5.
Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="arm"
GOBIN=""
GOCACHE="/home/pi/.cache/go-build"
GOENV="/home/pi/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/pi/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/pi/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm"
GOVCS=""
GOVERSION="go1.19.3"
GCCGO="gccgo"
GOARM="6"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/path/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -marm -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1078720696=/tmp/go-build -gno-record-gcc-switches"

What did you do?

It's a large type switch:
func Decode[T any](b []byte) (T, int) {
    switch any(*new(T)).(type) {
    case int:
        v, n := DecodeInt(b)
        return any(v).(T), n
    case int8:
        v, n := DecodeInt8(b)
        return any(v).(T), n
    case int16:
        v, n := DecodeUint16(b)
        return any(v).(T), n
   // Elided for brevity
   case uint:
        v, n := DecodeUint(b)
        return any(v).(T), n
   case uint8:
        v, n := DecodeUint8(b)
        return any(v).(T), n
   case uint16:
        v, n := DecodeUint16(b)
        return any(v).(T), n
   // More cases, about 30 total.
}

The source code is not public (yet) and I haven't been able to create a minimal repro. I can provide the source code privately for debugging purposes, though.
When I comment out both the int16 and uint16 cases it works.
What did you expect to see?
It compile.
What did you see instead?
illegal combination 02044 (/path/decode.go:817) WORD package-path..autotmp_151-286(SP); NONE NONE HAUTO; from 0 0; to 3 3

I can repro with 1.20rc3. Same error.

Weird, this is basically an assembler failure. It would really help to get a reproducer.

@golang/arm

@randall77 I can e-mail you the code if you'd like. Or I believe I can invite you to the repo. It's not secret, we just haven't decided on a license yet.

There is no hurry. If it's going to be available soon (before May 1, say) then we can wait. If not, then try to get a standalone reproducer. Even if we could look at private code, it makes it hard to, e.g., write a test case that we can submit into the public repo.
In any case, I don't think I'm the one to look at this. I don't know much about the arm assembler.

It looks to me that this switch https://cs.opensource.google/go/go/+/master:src/cmd/internal/obj/arm/asm5.go;l=691 may miss a case for HAUTO (and a few others). I'll send a CL for you to try. A reproducer may still be good. Thanks.

@elagergren-spideroak could you try if CL https://golang.org/cl/463138 helps? (You'll need to build the Go toolchain with that patch applied.) Thanks.

@elagergren-spideroak could you share a more complete repro so we can have a test case? Thanks.

@cherrymui I am working on a minimal repro. It might not be perfectly minimal, though. Sorry in advance.

Thanks. Added to the CL.


Added osde2e setup target and tests

Also added a new Containerfile for OSDE2E testing. this image will be used only for OSDE2E.

/hold
Want https://github.com/openshift/release/pull/35327 to be merged first, so the osde2e image will also be created.

/unhold

/lgtm


merge ota client v2 to main

To merge ota-client v2 to master as is.

I think it is also a good idea to leave a copy of legacy code in the repo,

We can retrieve by commit hash or if we can set tag, we can retrieve old ota-client easier.


Indent lines change color when using :PackerCompile

It's hard to know why this happens, but when I'm configuring my plugins.lua configuration file, and run :PackerCompile (or :PackerSync, which also does the compilation step), the indentation lines all change to a white colour, and remain that way until I restart Nvim.
The white indentation lines also show up if I switch to a different file afterwards.
I haven't seen it happen in any other cases.

PackerCompile probably reloads your color scheme, which clears all highlight groups.
You can do something like this to reset the colors again.
autocmd User PackerComplete,PackerCompileDone lua require("indent_blankline.utils").reset_highlights()

This should normally happen on ColorScheme anyway. But packer does some magic I don't fully understand.

Thanks @lukas-reineke, that works brilliantly.


Add support for serving multiple versions

/cc @zendesk/bunyip-numbat
Description
Serve all the versions of models available on the model_base_path
Modify FileSystemStoragePathSource to emit all versions rather than just the latest version if the following environment variable is set TF_SERVING_MULTIPLE_VERSIONS is set to true.
Related Links:
This is discussed in the following Github issue :
https://github.com/tensorflow/serving/issues/197
Upstream pull request totensorflow/serving:
https://github.com/tensorflow/serving/pull/217

not required anymore. its already accepted and merged into upstream main repo


Running Kotlin server locally does not return output in text field

I simply clone this repo, compile & run it locally on localhost:8080.
When sending the same request as on your documentation page:
curl -X POST \
  http://localhost:8080/api/compiler/run \
  -H 'Content-Type: application/json' \
  -d '{
    "args": "1 2 3",
    "files": [
        {
            "name": "File.kt",
            "text": "fun main(args: Array<String>) {\n    println(\"123\")\n}"
        }
    ]
}'

The result does not contain the STDOUT which should be printed on the console. It just returns this:
{"errors":{"File.kt":[{"interval":{"start":{"line":0,"ch":9},"end":{"line":0,"ch":13}},"message":"Parameter 'args' is never used","severity":"WARNING","className":"WARNING"}]},"exception":null,"text":""}
The log of the Spring application does not show any errors.
Can somebody help?

Import add-on: this happens on my Mac. I just tried on an Ubuntu Linux where it works fine. Why is there a difference ONLY in the text returned which should contain the STDOUT?

Hi,  sorry, I missed the issue =(
I think the problem is due to java security policy. If you commented line it will work as expected.
btw it works on my osx


feat: add getPerspectiveTransform and warpPerspective

Adding a couple of extra functions (and also a missed overload for toJSValue for RotatedRect (c++ implementation already existed)

Great job! Thanks a lot


feature(plugin-board): created plugin-board

In creating the plugin-board, other modules were affected:

created test-helper-mock-socket
created test-helper-mock-web-socket
moved page from ciscospark module to spark-core

that has to be there because of this.timeout()


Updated my member name

That's strange.  I didn't file the original PR to get my name on the site to start, so I don't know why it's referencing that account which isn't mine.  Good catch, I missed that was a GH account reference.

Hi, based on the record of whimsy tool, Andrea Gray is not an ASF member. If you believe this name is wrong, would you please to contact with our Secretarial Team for update? Thanks.


TODOs in HordeAd.Core.DualNumber about defining more derivatives of common functions

Fixes https://github.com/Mikolaj/horde-ad/issues/19

Thank you very much, Staszek!
CI fails. In most of the cases, that's because your formula for tanh, while beautiful (no doubt, though I can't tell if it's better behaved numerically or if it's faster or slower and on what architectures), gives subtly different results than the old one. In some cases the whole computation is not numerically stable wrt to the tanh computation and then the difference becomes huge.
However, not all failures are clear, so let's try to understand them before deciding on a fix. A couple of failures are probably due to my sparse matrix experiment not coping well with matrices of undetermined size (as your (recip 2)). It's possible than changing the general multiplication by (recip 2) to scaling by a constant (dScale) would lessen the strain.
The last and embarrassing case is where a test says only "wrong result". It would be helpful to make it print any details.

CI fails. In most of the cases, that's because your formula for tanh, while beautiful (no doubt, though I can't tell if it's better behaved numerically or if it's faster or slower and on what architectures), gives subtly different results than the old one. In some cases the whole computation is not numerically stable wrt to the tanh computation and then the difference becomes huge.

Right, and it also contains division (/). Is this the reason you're assuming your original formula is more accurate than mine?

However, not all failures are clear, so let's try to understand them before deciding on a fix. A couple of failures are probably due to my sparse matrix experiment not coping well with matrices of undetermined size (as your (recip 2)). It's possible than changing the general multiplication by (recip 2) to scaling by a constant (dScale) would lessen the strain.

Not that I understand, but I just eliminated (recip 2) from atanh, sinh and cosh, instead applying those functions to the primal component explicitly. Does this make sense? Why? :)

The last and embarrassing case is where a test says only "wrong result". It would be helpful to make it print any details.

Fixed the test to print out more info, on my machine it now produces:
Running 1 test suites...
Test suite shortTestForCI: RUNNING...
Short tests for CI
  MNIST RNN short tests
    randomLL2 140 0 3 5 54282 5.0e-2: FAIL (2.25s)
      test/common/TestMnistRNN.hs:173:
      wrong result: 30.061871495725264 is expected to be a member of [30.061871495723956,30.06187089990927]

which, again, is a subtle numerical error. But in which function and which number is closer to the truth?...
BTW why do those tests here on Github take 40 minutes to complete? Can we improve that?

Not sure what to do with tan: all the forms I could find contain the division somewhere. For example, the first derivative: 1/cos^2.

Right, and it also contains division (/). Is this the reason you're assuming your original formula is more accurate than mine?

I'm not assuming that. However, if I knew one of them is more accurate (or faster) and if I had to guess which, I'd assume mine, just because it's repeated on ML blogs, while yours is derived from calculus. Ultimately, checking the quality of our pre-defined functions is a separate task, so I wouldn't worry and settle on something uniform and cheap (e.g., not requiring a change of tests). Probably, avoiding division by zero is a good idea, too, but not at a too high complexity cost (I guess tan is fine; if users don't like it, let them propose a change). I'm learning this as I go, so I'm afraid I don't have good answers yet.

Not that I understand, but I just eliminated (recip 2) from atanh, sinh and cosh, instead applying those functions to the primal component explicitly. Does this make sense? Why? :)

Actually, what I had in mind was to change
cosh z = (recip 2) * ((exp z) + (exp (-z)))

to
cosh z = scale (recip 2) ((exp z) + (exp (-z)))

and this looks better to me, because it underlines that we are not using a bilinear function * that has a complex derivative, but we use a scaling by a constant that has a simple derivative. But your new version
cosh (D u u') = D (cosh u) (dScale (sinh u) u')
is even clearer to me, because it tell me what is the primal component and what is the dual component of the dual number (the derivative that, indeed, is a small expression tree). Please ask more, if what I said requires further clarification.

Fixed the test to print out more info, on my machine it now produces:
Running 1 test suites...
Test suite shortTestForCI: RUNNING...
Short tests for CI
  MNIST RNN short tests
    randomLL2 140 0 3 5 54282 5.0e-2: FAIL (2.25s)
      test/common/TestMnistRNN.hs:173:
      wrong result: 30.061871495725264 is expected to be a member of [30.061871495723956,30.06187089990927]

which, again, is a subtle numerical error. But in which function and which number is closer to the truth?...

Thank you for the fix. What is truth? I'd recommend to change the test to expect one of two results: the result you get at home and the result CI obtains with the same test (if it differs). Ignore the old results; they are close enough, so we probably haven't made an error.

BTW why do those tests here on Github take 40 minutes to complete? Can we improve that?

How much do they take on your machine? We can surely solve #14 and #20, but they are really hard (and #14 is almost finished).

BTW, I've just merged a big PR to master. Apologies for the conflicts and for, probably, some extra boilerplate TODOs to do in DualNumber.hs.
The code on master is now hlint-clean except for a dozen of cases. You can use hlint to tell you which parens are unnecessary. But feel free to ignore or to silence via .hlint.yaml any hints that are not useful.

Thank you for the boilerplate. I guess that's all the relevant TODOs? Could you rebase your branch instead of merge, to keep git history clean? We will then merge this PR with --no-ff to preserve PR number in git history.

What is truth?

The real tanh result which should be there, if our floating point arithmetic was error-free. BTW, did you consider symbolic calculations and/or using your own arithmetic with infinite precision?

I'd recommend to change the test to expect one of two results: the result you get at home and the result CI obtains with the same test (if it differs).

No longer relevant since I rolled back my tanh implementation (the test is now green).

BTW why do those tests here on Github take 40 minutes to complete? Can we improve that?

How much do they take on your machine?

I had to stop them, but maybe I will run them again on some cold winter day.

Ups sorry, I just merged, then read this. Shall I make a new branch and replay my commits there?

Oh, no, please don't bother, that's not a big deal. Also, if you rebase now, the merge commits are probably going to vanish (though you might to solver a conflict of two manually, as often with rebasing). Then just git push -f to this branch.

BTW, did you consider symbolic calculations and/or using your own arithmetic with infinite precision?

No really because, as I'm slowly learnings, ML is fine with lack of precision and sometimes it's even good (introduces a fuzz that helps break away from local minima and not to overfit) and definitely not worth a big performance price to eliminate. I think ML applications even prefer smaller floats than 64 bits in order to lower peak memory.

I'd recommend to change the test to expect one of two results: the result you get at home and the result CI obtains with the same test (if it differs).

No longer relevant since I rolled back my tanh implementation (the test is now green).

Great. Thanks.

I had to stop them, but maybe I will run them again on some cold winter day.

I suggested in that ticket that running just the short test, the one that CI runs, may be a better idea than running both and concurrently. Did this suggestion make sense?

Ups sorry, I just merged, then read this. Shall I make a new branch and replay my commits there?

Oh, no, please don't bother, that's not a big deal. Also, if you rebase now, the merge commits are probably going to vanish (though you might have to solve a conflict or two manually, as is common with rebasing). Then just git push -f to this branch.

Done.

BTW, did you consider symbolic calculations and/or using your own arithmetic with infinite precision?

Not really because, as I'm slowly learnings, ML is fine with lack of precision and sometimes it's even good (introduces a fuzz that helps break away from local minima and not to overfit) and definitely not worth a big performance price to eliminate. I think ML applications even prefer smaller floats than 64 bits in order to lower peak memory. [Edit: it's only serious numerical instability that becomes a problem and for which, indeed, exact arithmetic would be a solution.]

I was expecting that. Now the question is, do we ever experience those serious errors.
Anything else to do in this ticket?

Anything else to do in this ticket?

Let me check the comments...
I think only this comment remains, if it makes sense to you (please ask if anything is unclear):

The code on master is now hlint-clean except for a dozen of cases. You can use hlint to tell you which parens are unnecessary. But feel free to ignore or to silence via .hlint.yaml any hints that are not useful.


Navmesh not working correctly

TODO: research more on how to implement a 2D navmesh since it's not supported out of the box.

 Fix navmesh

f173700efe3d62b01498b9759eb8058d05ca1b11


Update the REAME File

Update the correct procedure of installing the AssemblyLine
Update the Full Example, it is now an actual and working Full Example
Update an instruction of error shooting(ldconfig issue).

Done.


checking changes

Well done but there is a bug in your PR
Kindly check the attachment for the javascript error shown


ros-one on Ubuntu 23.10

Upgrading from 23.04
https://github.com/lucasw/ros_from_src/issues/30#issuecomment-1817897053

In file included from /usr/include/gtest/gtest-message.h:57,
                 from /usr/include/gtest/gtest-assertion-result.h:46,
                 from /usr/include/gtest/gtest.h:64,
                 from /home/lucasw/base_catkin_ws/src/extra/geometry/tf/test/transform_listener_unittest.cpp:30:
/usr/include/gtest/internal/gtest-port.h:270:2: error: #error C++ versions less than C++14 are not supported.
  270 | #error C++ versions less than C++14 are not supported.
      |  ^~~~~

Errors     << grid_map_filters:check /home/lucasw/install_base_catkin_ws/logs/grid_map_filters/build.check.005.log
CMake Warning (dev) at /usr/share/catkin/cmake/python.cmake:7 (find_package):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

Call Stack (most recent call first):
  /usr/share/catkin/cmake/all.cmake:164 (include)
  /usr/share/catkin/cmake/catkinConfig.cmake:20 (include)
  CMakeLists.txt:30 (find_package)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at CMakeLists.txt:35 (find_package):
  Could not find a configuration file for package "TBB" that exactly matches
  requested version "2021.8.0".

  The following configuration files were considered but not accepted:

    /usr/lib/x86_64-linux-gnu/cmake/TBB/TBBConfig.cmake, version: 2021.9.0
    /lib/x86_64-linux-gnu/cmake/TBB/TBBConfig.cmake, version: 2021.9.0

relax requirement of tbb - added new commit to git@github.com:lucasw/grid_map tbb_2021

Errors     << rviz:cmake /home/lucasw/install_base_catkin_ws/logs/rviz/build.cmake.002.log
CMake Error at /usr/share/cmake-3.27/Modules/FindPkgConfig.cmake:607 (message):
  A required package was not found
Call Stack (most recent call first):
  /usr/share/cmake-3.27/Modules/FindPkgConfig.cmake:829 (_pkg_check_modules_internal)
  CMakeLists.txt:41 (pkg_check_modules)

cd /home/lucasw/install_base_catkin_ws/build/rviz; catkin build --get-env rviz | catkin env -si  /usr/bin/cmake /home/lucasw/base_catkin_ws/src/ros/rviz --no-warn-unused-cli -DCATKIN_DEVEL_PREFIX=/home/lucasw/install_base_catkin_ws/devel/.private/rviz -DCMAKE_INSTALL_PREFIX=/home/lucasw/install_base_catkin_ws/install -DCMAKE_BUILD_TYPE=Release -Wno-deprecated; cd -

............................................................................................................................................................................................................................
Failed     << rviz:cmake                          [ Exited with code 1 ]
Failed    <<< rviz ```

sudo apt install libogrenextoverlay2.2.5```

Looks to be fully building with repos in https://github.com/lucasw/ros_from_src/commit/6b1323c17ab0a5baafa6478351c1b4517f4b6c4a (minus all the ignored ones)- but see if it builds in github action

Need to update https://github.com/lucasw/image_transport_plugins enable_pub branch

Errors     << lvr2:make /home/lucasw/install_base_catkin_ws/logs/lvr2/build.make.001.log
/usr/bin/ld: CMakeFiles/lvr2_reconstruct.dir/Main.cpp.o: undefined reference to symbol 'LZ4_decompress_safe_continue'
/usr/bin/ld: /lib/x86_64-linux-gnu/liblz4.so.1: error adding symbols: DSO missing from command line


Fix: #7. Add tests

This PR
refactor main adding a smoke test
Note
A comparison test can be added in another PRs

This PR looks great :rocket:

Thank you very much!!


Fix spelling mistakes, incorrect dates, and more

This PR adds the following:
File: wip/flight-hub/index.md:

Changed 2/12/2020 to 2/11/2020 to match the date of when build 19041.84 was published to the Slow ring
Changed 2/25/2020 to 2/26/2020 to match the date of when build 19041.84 was published to WSUS.
Removed en-us locale from build 19041.84 ISO link

File: wip/at-home/Active-Dev-Branch-Beta.md:

Changed Developmemnt to Development
Changed RS_PRELEASE to RS_PRERELEASE
Added hyperlinks to Microsoft Design posts from Medium to match the blog post of build 19569

File: wip/at-home/Whats-new-wip-at-home-1809.md

Added spaces between characters to make it more readable
Changed keyboard shortcut from Ctrl + B to Ctrl + E to match the one shown in the image

File: wip/at-home/feedback.md

Removed unneeded space to match other bullet points

File: wip/at-home/whats-new-apps.md

Added bullet points to a content under the Phone screen now supports pen input sub-header, which is under the Your Phone header

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks.

@bleblancMSFT - PR has been copy edited and is ready for final review.
Thank you.
cc: @kenwith

There is one item we need to revert. The date for Build 19041.84 going to WSUS needs to reflect 2/26 which is the date we officially made the announcement blog post and not 2/25. This was something we did intentionally in Flight Hub to match the announcement.

@bleblancMSFT The changes look fine here: https://github.com/MicrosoftDocs/windows-insider/blob/656c0696a4cee83178d33fe20bc95e99680c9b92/wip/flight-hub/index.md#upcoming-windows-10-feature-update-20h1. Is there anything I'm missing?

@ItzLevvie you are right, I mis-red! I merged this!


[web] Standardized way to preload emoji font using CanvasKit

Use case
When using CanvasKit and trying to display emojis, issue https://github.com/flutter/flutter/issues/76248 arises, i.e. "Emojis take a few seconds to render on canvaskit".
This means that when first displaying emojis, tofu boxes appear for a few seconds.
Proposal
It would be great to have an exposed function that allows loading the Noto font on app start, so that when the first emojis are displayed, the Noto font is already preloaded.
Workaround
The way we solve this currently is the following:
@override
Widget build(BuildContext build) {
  return Stack(
    children: [
      // w/e
      const Opacity(
        opacity: 0,
        // Insert invisible emoji in order to load the emoji font in CanvasKit
        // on startup. Note that this is a *workaround solution* - we want to
        // find a better one :)
        child: Text('✨'),
      ),
    ],
  );
}

This is inserted on startup. As soon as we actually display emojis, the Noto font has already been downloaded. However, this really feels like a hack.

Thank you for the suggestion. We should allow customizing fallback fonts, although we also should pick a more reasonably-sized font for emojis (we currently use a 9MB file).
/cc @hterkelsen assigning to you for triage. WDYT about providing an API for custom fallbacks? This may be needed for offline support.

Thanks for your response 🙏 @yjbanov
That sounds like a good idea (as well). As well because it is of course not exactly the issue.

we also should pick a more reasonably-sized font for emojis (we currently use a 9MB file).

My take on it is that this is fine probably or rather in our case I would say it is worth the trade-off. Actually, the Noto font is still missing some compound emojis (not sure if that is the proper technical term). So a bigger font that supports all common emojis is something that I would be fine with.

@yjbanov I checked the network logs and it seems like https://fonts.googleapis.com/css2?family=Noto+Color+Emoji+Compat is downloaded.
If you take a look at this screenshot, you can see that a few emojis are not supported at all (these should be single emojis):

Then, taking a look at the latest Noto Color Emoji font at https://www.google.com/get/noto/help/emoji/smileys-people/, these emojis are all supported:

We would definitely be fine to have a slightly larger emoji font if those were all supported in Flutter.

The Google fonts team is working on the COLRv1 color font format, which will significantly reduce the font file size. Is the COLR format supported in flutter?
https://github.com/googlefonts/color-fonts

We can add an API to preload the fallback fonts or allow for customizing them. Since this can be worked around by adding the fonts as assets, assigning P4.

@yisibl AFAIK Flutter can't render COLR/CPAL nor SVG Tables, so the only option for colored fonts is CBDT.

Here is another more elegant hack:
class WebEmojiLoaderHack extends StatelessWidget {
  const WebEmojiLoaderHack({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return const Offstage(
      // Insert invisible emoji in order to load the emoji font in CanvasKit
      // on startup.
      child: Text('✨'),
    );
  }
}

Just put this widget in any place on program startup (inside Column or Row for example) and Flutter will load emojis.  If it is offstage, it should not be rendered by default, which should mean that the font is also not loaded (my assumption).

@rodion-m Are you sure that this works? If it is offstage, it should not be rendered by default, which should mean that the font is also not loaded (my assumption).

It works, I checked. I think it happens because Text('✨') is called anyway.

Hi @creativecreatorormaybenot, I'm starting out on Flutter. Where do you suggest to call the widget (the solution you posted in your workaround)? I am currently working on a web project that renders emoji right in the first screen. If I'm not wrong, the first method that executes is the main method and it is not meant for widgets.
Perhaps stackoverflow is the right place for my query, but I'm asking it here anyway.

Someone posted this on another thread and it seems to work, albeit janky:
import 'dart:ui';

ParagraphBuilder pb = ParagraphBuilder(ParagraphStyle(locale: window.locale));
pb.addText('\ud83d\ude01'); // smiley face emoji
pb.build().layout(const ParagraphConstraints(width: 100));

Any update on this? emojis are a vital part of web development. Needs to be sorted


Viewing selected classifiers of all DES method

Hi all,
How to know the classifiers that were selected by the DES method.

@EverAPC Hello,
In the classification pipeline, the selected classifiers are computed when calling the select method which returns a boolean matrix selected_classifiers with shape  [n_samples, n_classifiers] containing (1) when the base model is selected and (0) otherwise.
Unfortunately, this information is not returned to the user because we couldn't find a good way to return this information without having any problems with the scikit-learn interface and side-effects. There is a current discussion about this point in the original DESlib repository (this one is just a fork): https://github.com/scikit-learn-contrib/DESlib/issues/130
If you have any suggestions about how this feature should work or can contribute with adding this feature please let me know!


Match Brackets

Is it possible to set brackets match like this? if yes how to do it anyone please advise, thanks.

you may try atom-material-syntax

@zhaolinlau thanks but it's not what i asked :(


improved documentation in the getting started section and development…

Summary
Improved the getting started and developing sections of the documentation.
Getting started section

Developing section

These are some of the stuff I had a hard time understanding when I set up my local wagtail codebase for the first time.

Changes made as requested


Where clause doesn't work with checksum addresses

:wave: I was playing with your subgraph and I noticed that the where clause (both for tokens and tokenContracts) is case sensitive. Not sure if this is something that should be fixed on this repo or on the "The Graph" repo.
Given that the address is a number (wrapped in a string) I expected it to work in any case.
Query using a checksum address (sad :panda_face:)
{
  tokenContracts (where:{id: "0xDCe09254dD3592381b6A5b7a848B29890b656e01"}) {
    id
    supportsEIP721Metadata
    numTokens
  }
}

No results:
{
  "data": {
    "tokenContracts": []
  }
}

Query using a lowercase address (works!)
{
  tokenContracts (where:{id: "0xdce09254dd3592381b6a5b7a848b29890b656e01"}) {
    id
    supportsEIP721Metadata
    numTokens
  }
}

I can has tokens!
{
  "data": {
    "tokenContracts": [
      {
        "id": "0xdce09254dd3592381b6a5b7a848b29890b656e01",
        "numTokens": "571",
        "supportsEIP721Metadata": true
      }
    ]
  }
}

This is how thegraph works, ID are string
So while we could use checksum address, this would fails for lower case query

Sorry for bumping these after so much time, but is there any way of storing addresses with checksum on a subgraph ? Whenever you store it as String via .toHex() or as Bytes both are lowercasing the address.

@enzoferey yes we could add extra field for these, but the id would still need to be a string and we need to choose a format (checksum, uppercase, or lowercase)

I mean, you should be able to convert a given address to whatever format you want to use (one of the 3 you mentioned) and then I would of course expect that if I called .toChecksumString() then I would need to pass a checksum address to the where clause.
As of right now, it's an implicit .toLowercaseString() and that forces clients to to convert the data going in and parse the data coming out.


User warning about sequence matcher

When I grabbed the latest Elara Docker image and ran the CLI from inside the container I saw the following warning:
$ docker run -it --entrypoint /bin/bash<PHONE_NUMBER>94.dkr.ecr.eu-west-1.amazonaws.com/elara

root@92d61ece6463:/# elara --help
/usr/local/lib/python3.8/site-packages/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning
  warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')
Usage: elara [OPTIONS] COMMAND [ARGS]...

  Command line tool for processing a MATSim scenario events output.

Options:
  --help  Show this message and exit.

Commands:
  run              Run Elara using a config.
  event-handlers   Access event handler output group.
  plan-handlers    Access plan handler output group.
  post-processors  Access post processing output group.

Some detail on a fix can be found here. However, pip install python-Levenshtein fails inside the container, apparently due to the lack of a gcc installation:
root@92d61ece6463:/# pip install python-Levenshtein
Collecting python-Levenshtein
  Using cached python-Levenshtein-0.12.2.tar.gz (50 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from python-Levenshtein) (57.5.0)
Building wheels for collected packages: python-Levenshtein
  Building wheel for python-Levenshtein (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [32 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.8
      creating build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/StringMatcher.py -> build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/__init__.py -> build/lib.linux-x86_64-3.8/Levenshtein
      running egg_info
      writing python_Levenshtein.egg-info/PKG-INFO
      writing dependency_links to python_Levenshtein.egg-info/dependency_links.txt
      writing entry points to python_Levenshtein.egg-info/entry_points.txt
      writing namespace_packages to python_Levenshtein.egg-info/namespace_packages.txt
      writing requirements to python_Levenshtein.egg-info/requires.txt
      writing top-level names to python_Levenshtein.egg-info/top_level.txt
      reading manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no previously-included files matching '*pyc' found anywhere in distribution
      warning: no previously-included files matching '*so' found anywhere in distribution
      warning: no previously-included files matching '.project' found anywhere in distribution
      warning: no previously-included files matching '.pydevproject' found anywhere in distribution
      adding license file 'COPYING'
      writing manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
      copying Levenshtein/_levenshtein.c -> build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/_levenshtein.h -> build/lib.linux-x86_64-3.8/Levenshtein
      running build_ext
      building 'Levenshtein._levenshtein' extension
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/Levenshtein
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.8 -c Levenshtein/_levenshtein.c -o build/temp.linux-x86_64-3.8/Levenshtein/_levenshtein.o
      unable to execute 'gcc': No such file or directory
      error: command 'gcc' failed with exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for python-Levenshtein
  Running setup.py clean for python-Levenshtein
Failed to build python-Levenshtein
Installing collected packages: python-Levenshtein
  Running setup.py install for python-Levenshtein ... error
  error: subprocess-exited-with-error

  × Running setup.py install for python-Levenshtein did not run successfully.
  │ exit code: 1
  ╰─> [32 lines of output]
      running install
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.8
      creating build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/StringMatcher.py -> build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/__init__.py -> build/lib.linux-x86_64-3.8/Levenshtein
      running egg_info
      writing python_Levenshtein.egg-info/PKG-INFO
      writing dependency_links to python_Levenshtein.egg-info/dependency_links.txt
      writing entry points to python_Levenshtein.egg-info/entry_points.txt
      writing namespace_packages to python_Levenshtein.egg-info/namespace_packages.txt
      writing requirements to python_Levenshtein.egg-info/requires.txt
      writing top-level names to python_Levenshtein.egg-info/top_level.txt
      reading manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no previously-included files matching '*pyc' found anywhere in distribution
      warning: no previously-included files matching '*so' found anywhere in distribution
      warning: no previously-included files matching '.project' found anywhere in distribution
      warning: no previously-included files matching '.pydevproject' found anywhere in distribution
      adding license file 'COPYING'
      writing manifest file 'python_Levenshtein.egg-info/SOURCES.txt'
      copying Levenshtein/_levenshtein.c -> build/lib.linux-x86_64-3.8/Levenshtein
      copying Levenshtein/_levenshtein.h -> build/lib.linux-x86_64-3.8/Levenshtein
      running build_ext
      building 'Levenshtein._levenshtein' extension
      creating build/temp.linux-x86_64-3.8
      creating build/temp.linux-x86_64-3.8/Levenshtein
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.8 -c Levenshtein/_levenshtein.c -o build/temp.linux-x86_64-3.8/Levenshtein/_levenshtein.o
      unable to execute 'gcc': No such file or directory
      error: command 'gcc' failed with exit status 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> python-Levenshtein

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

The solution is probably to install gcc and pip install python-Levenshtein in the Docker image . This may also improve the performance of any fuzzy matching operations in Elara by having them use faster distancing algorithm implementations.

how can i download the latest image to work on the problem?
i've tried the cmd above but it returned authentication error
docker run -it --entrypoint /bin/bash<PHONE_NUMBER>94.dkr.ecr.eu-west-1.amazonaws.com/elara

Unable to find image '758645626094.dkr.ecr.eu-west-1.amazonaws.com/elara:latest' locally
docker: Error response from daemon: Head "https://758645626094.dkr.ecr.eu-west-1.amazonaws.com/v2/elara/manifests/latest": no basic auth credentials.
See 'docker run --help'.

how i can download the latest image to work on the problem?
i've tried the cmd above but it returned authentication error
docker run -it --entrypoint /bin/bash<PHONE_NUMBER>94.dkr.ecr.eu-west-1.amazonaws.com/elara

Unable to find image '758645626094.dkr.ecr.eu-west-1.amazonaws.com/elara:latest' locally
docker: Error response from daemon: Head "https://758645626094.dkr.ecr.eu-west-1.amazonaws.com/v2/elara/manifests/latest": no basic auth credentials.
See 'docker run --help'.

Hi @ahmednreldin
Sorry for the late reply. You won't be able to pull that image because it's currently private inside our ECR repo, but you can build the image locally directly from the Dockerfile (docker build -t elara-local . should do it).

The warning has been fixed by this commit


Added flowLabel, includeLabelInRouter flow fields + relabel output

Fixes #1115

@pepov This is probably logic, that you requested https://github.com/kube-logging/logging-operator/pull/1294/commits/d6adc622cbdf09d071ad42197d61cd8e1ae35898 ,
if FlowLabel is empty, automatically include flow in router unless explicitly defined to exclude
if FlowLabel is not empty, automatically exclude flow from router unless explicitly defined to include

Thank you as well, hope it is OK now

@gvizdpeter thanks for the nice contribution, merging this now!

Created a follow up issue for the docs: https://github.com/kube-logging/kube-logging.github.io/issues/131
@gvizdpeter could you please help with the documentation part? No need to figure out the right place or the proper wording, just creating a sketch with a general description and the most important use-cases.

Hello @pepov I will try to add something to opened issues.

Does someone happen to have a nice example of this with the concat plugin using timeouit_label and then the relabel in action?

If concat produces messages with new labels, then you don't need relabel, just a flow with the label you want to redirect timeouted messages:
apiVersion: logging.banzaicloud.io/v1beta1
kind: Flow
metadata:
  name: concat-timeout-flow
spec:
  flowLabel: '@concat-timeout'
  includeLabelInRouter: false
  localOuputs: ...

Maybe the question is what do you want to do with those messages ran into timeout?


No internet inside container?

Steps to produce:
git clone .../mkosi-files
cd mkosi-files
sudo ./update-nspawn-images buildspawn/
sudo systemd-nspawn -UM buildspawn
passwd
exit

# start to container
sudo machinectl start buildspawn
# will failed to start due to # Bind=/ramdisk/scratch/:/scratch/
# I have to comment out Bind=/ramdisk/scratch/:/scratch/ in /etc/systemd/nspawn/buildspawn.nspawn to make it work

# login to container
sudo machinectl login buildspawn
root
<password.

ip ad
ping google.com
# failed

on host I have ve-* interface, and inside container I have another interface without IP address, I would expect the non-lo interface should have and ip address assigned to it and I'll be able to ping google.com from container.
all setup are mostly default and I haven't changed anything.
I use archlinux latest lts-linux, systemd-networkd to manage my wifi connection via wpa_supplicant profile.

@tmpm697 Argh, my approgies, I've not documented about veth...
Try below rules for iptables:
-A INPUT -i ve-buildspawn -j ACCEPT

From my experience (sorry not to show you any sources), you must add rule INPUT for veth with -j ACCEPT to communitcate outside container with veth.
(i.e. VirtualEthernet=yes in /etc/systemd/nspawn/*.nspawn)
I added above to README.md at https://github.com/nosada/mkosi-files/commit/18bed45f29f3b0cf744446072d2db92b1ad386f7.
Thanks for your reporting :relaxed:

This is weird. I don't have iptables or nftables services running.
How to setup that container will have static IP address?
My setup:
00-mynet.network in /etc/systemd/network that match against a wifi nic wlxxx. (On host).
I use archlinux latest with linux-lts, systemd-networkd to manage network for both host and container. I have<EMAIL_ADDRESS>to start wifi profile and the profile in /etc/wpa_supplicant on host.
Everything else has default config.
I want build and start buildspawn with a specific ipadress but failed.
I followed this article: https://wiki.archlinux.org/index.php/Systemd-networkd#Usage_with_containers
Basically:
On host:
Add Bridge=br0 to 00-mynet.network
Create br0.netdev:
#/etc/systemd/network/MyBridge.netdev
[NetDev]
Name=br0
Kind=bridge
Create bridge network:
br0.network
/etc/systemd/network/MyBridge.network
[Match]
Name=br0
[Network]
DNS=<IP_ADDRESS>
Address=<IP_ADDRESS>/24
Gateway=<IP_ADDRESS>
#<IP_ADDRESS> is also my host's gateway, <IP_ADDRESS>/24 is my host's LAN network.
I build and start buildspawn container as default, the add static ip adress as in article:
On container:
#/etc/systemd/network/80-container-host0.network
[Match]
Name=host0
[Network]
DNS=<IP_ADDRESS>
Address=<IP_ADDRESS>/24
Gateway=<IP_ADDRESS>
Ip route on host:
wlxxx (container 192.168.1.xx)
ve-buidspawn@f13 status DOWN, no ipadress
Ip route on container:
lo
host0 DOWN, no ipadress
I think there's something thing wrong with my setup, do you have same issue before?
P/s I'll put code block when in desktop.
Thanks.

I see.
I haven't ever used bridge interface with nspawn + networkd, so there's no tips I can show you...
I'll try to use bridge + nspawn + networkd later when I have time.

I haven't tried bridge + nspawn + networkd yet, but found my mistaken (or forgot) point for using of veth.
If you use veth and then go outside from container, you must set up NAT with iptables (or nftables), as described in https://wiki.archlinux.org/index.php/systemd-nspawn#Use_a_virtual_Ethernet_link.
Below's an example for iptables I used before (cf. https://wiki.archlinux.org/index.php/Simple_stateful_firewall) :
Install this to /etc/iptables/iptables.rules

*filter
:INPUT DROP [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:TCP - [0:0]
:UDP - [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

-A INPUT -i lo -j ACCEPT
-A INPUT -i ve-+ -j ACCEPT

-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p icmp -m icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -p udp -m conntrack --ctstate NEW -j UDP
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A INPUT -p tcp -m recent --set --name TCP-PORTSCAN --mask <IP_ADDRESS> --rsource -j REJECT --reject-with tcp-reset
-A INPUT -p udp -m recent --set --name UDP-PORTSCAN --mask <IP_ADDRESS> --rsource -j REJECT --reject-with icmp-port-unreachable
-A TCP -p tcp -m recent --update --seconds 60 --name TCP-PORTSCAN --mask <IP_ADDRESS> --rsource -j REJECT --reject-with tcp-reset
-A UDP -p udp -m recent --update --seconds 60 --name UDP-PORTSCAN --mask <IP_ADDRESS> --rsource -j REJECT --reject-with icmp-port-unreachable
COMMIT

and below's for nftables I'm using now (cf. https://wiki.archlinux.org/index.php/Nftables#Simple_stateful_firewall):
Install this to /etc/nftables.conf

table inet filter {
        chain input {
                type filter hook input priority filter; policy drop;
                ct state established,related accept
                iifname "lo" accept
                iifname "ve-*" accept
                ct state invalid drop
                ip6 nexthdr ipv6-icmp icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, mld-listener-query, mld-listener-report, mld-listener-done, nd-router-solicit, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, ind-neighbor-solicit, ind-neighbor-advert, mld2-listener-report } accept
                ip protocol icmp icmp type { destination-unreachable, router-advertisement, router-solicitation, time-exceeded, parameter-problem } accept
                ip protocol igmp accept
                ip protocol udp ct state new jump UDP
                ip protocol tcp tcp flags & (fin | syn | rst | ack) == syn ct state new jump TCP
                ip protocol udp reject
                ip protocol tcp reject with tcp reset
                meta nfproto ipv4 counter packets 0 bytes 0 reject with icmp type prot-unreachable
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
        }

        chain output {
                type filter hook output priority filter; policy accept;
        }

        chain TCP {
        }

        chain UDP {
        }
}

I don't use iptables or nftables in my system.
I have ve-xxxx on host and host0 in container but as the archwiki:
"When you start the container, an IP address has to be assigned to both interfaces (on the host and in the container). If you use systemd-networkd on the host as well as in the container, this is done out-of-the-box"
This is not true in my case, both use systemd-networkd but there's no ip under container.
ve-xxx and host0 both in DOWN state.
I use wifi profile with wpa-supplicant but I think it doesn't relate to this case.

Ah, sorry, I misread your comment.
I could maybe reproduce your issue (no IP address on ve-* in host, host0 in container) by doing below (seems to be identical inside systemd) in host:

remove /usr/lib/systemd/network/80-container-ve.network
Create symlink /etc/systemd/network/80-container-ve.network to /dev/null

"networkctl" result

# In Host

$ networkctl
IDX LINK          TYPE     OPERATIONAL SETUP
  1 lo            loopback carrier     unmanaged
  2 enp0s25       ether    no-carrier  configuring
  4 wlan0         wlan     routable    configured
  8 ve-buildspawn ether    off         unmanaged     <- veth for nspawn container

4 links listed.

$ networkctl status ve-buildspawn
● 8: ve-buildspawn
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: n/a
                          Type: ether
                         State: off (unmanaged)
                        Driver: veth
                    HW Address: 8a:41:5e:XX:XX:XX
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noop
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp

Oct 13 00:07:30 HOST systemd-networkd[4744]: ve-buildspawn: Link DOWN
Oct 13 00:07:30 HOST systemd-networkd[4744]: ve-buildspawn: Lost carrier
Oct 13 00:07:30 HOST systemd-networkd[4744]: ve-buildspawn: DHCPv6 lease lost
Oct 13 00:12:10 HOST systemd-networkd[23786]: ve-buildspawn: IPv6 successfully enabled
Oct 13 00:12:10 HOST systemd-networkd[23786]: ve-buildspawn: Link UP
Oct 13 00:12:12 HOST systemd-networkd[23786]: ve-buildspawn: Gained carrier
Oct 13 00:12:14 HOST systemd-networkd[23786]: ve-buildspawn: Gained IPv6LL
Oct 13 00:12:24 HOST systemd-networkd[23786]: ve-buildspawn: Link DOWN
Oct 13 00:12:24 HOST systemd-networkd[23786]: ve-buildspawn: Lost carrier
Oct 13 00:12:24 HOST systemd-networkd[23786]: ve-buildspawn: DHCPv6 lease lost

# In container

$ networkctl
IDX LINK  TYPE     OPERATIONAL SETUP
  1 lo    loopback carrier     unmanaged
  2 host0 ether    no-carrier  configuring

2 links listed.

$ networkctl status host0
● 2: host0
                     Link File: n/a
                  Network File: /usr/lib/systemd/network/80-container-host0.network
                          Type: ether
                         State: no-carrier (configuring)
                    HW Address: 7a:4d:b2:YY:YY:YY
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11fe986dfda77474780000

Oct 13 00:12:51 buildspawn systemd-networkd[19]: host0: IPv6 successfully enabled
Oct 13 00:12:51 buildspawn systemd-networkd[19]: host0: Link UP

"ip address show" result

# In host

$ ip address show dev ve-buildspawn
8: ve-buildspawn@if2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 8a:41:5e:XX:XX:XX brd ff:ff:ff:ff:ff:ff link-netnsid 0

# In container

$ ip address show dev host0
2: host0@if8: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
    link/ether 7a:4d:b2:YY:YY:YY brd ff:ff:ff:ff:ff:ff link-netnsid 0

Both could do them.
After reproduced, I restored 80-container-ve.network then restart systemd-networkd.service and things recovered.
Here's /etc/systemd/nspawn/buildspawn.nspawn I used:
[Exec]
PrivateUsers=true
NotifyReady=true

[Files]
# Commented out below also works fine
Bind=/ramdisk/scratch/:/scratch/

[Network]
Private=yes
VirtualEthernet=yes

Maybe you couldn't treat veth by systemd-networkd properly.
Could you try checking your *.network in /etc/systemd/network/ and /usr/lib/systemd/network/?
P.S. I use iwd to manage wifi profile instead of wpa_supplicant, I've used before. But when wpa_supplicant I haven't experience issue like you reported.
So I think so too that wpa_supplicant doesn't relate to this issue.

ALL of my configs:
ON HOST:
cat /etc/systemd/network/00-wireless.network
# NOTE: I use static ip address.
# /etc/systemd/network/00-wireless.network
[Match]
Name=wlp1sX
SSID=<my wifi ssid>

[Network]
Address=192.168.Y.XXX/24
Gateway=192.168.E.F
DNS=<IP_ADDRESS> <IP_ADDRESS>

cat /etc/wpa_supplicant/wpa_supplicant-wlp1sX.conf
network={
        ssid="<my wifi ssid>"
        scan_ssid=1
        psk=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
}

networkctl
IDX LINK          TYPE     OPERATIONAL SETUP
  1 lo            loopback carrier     unmanaged
  # ...
  8 ve-buildspawn ether    no-carrier  configuring

ip address show dev ve-buildspawn
8: ve-buildspawn@if2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default qlen 1000
    link/ether ee:bc:06:a5:c1:fc brd ff:ff:ff:ff:ff:ff link-netnsid 0

cat /etc/systemd/nspawn/buildspawn.nspawn
[Exec]
PrivateUsers=true
NotifyReady=true

[Files]
# Commented out below also works fine
# Bind=/ramdisk/scratch/:/scratch/

[Network]
Private=yes
VirtualEthernet=yes

networkctl status ve-buildspawn
● 8: ve-buildspawn
                     Link File: /usr/lib/systemd/network/99-default.link
                  Network File: /usr/lib/systemd/network/80-container-ve.network
                          Type: ether
                         State: no-carrier (configuring)
                        Driver: veth
                    HW Address: ee:bc:06:a5:c1:fc
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noqueue
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp
             DHCP6 Client DUID: DUID-EN/Vendor:0000ab11127bcb58b9aa75fa0000
           Offered DHCP leases: none

Oct 13 14:11:43 localhost systemd-networkd[389]: ve-buildspawn: IPv6 successfully enabled
Oct 13 14:11:43 localhost systemd-networkd[389]: ve-buildspawn: Link UP

cat /usr/lib/systemd/network/99-default.link
#  SPDX-License-Identifier: LGPL-2.1+
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Match]
OriginalName=*

[Link]
NamePolicy=keep kernel database onboard slot path
AlternativeNamesPolicy=database onboard slot path
MACAddressPolicy=persistent

ON CONTAINER:
networkctl
WARNING: systemd-networkd is not running, output will be incomplete.

IDX LINK  TYPE     OPERATIONAL SETUP
  1 lo    loopback n/a         unmanaged
  2 host0 ether    n/a         unmanaged

cat /usr/lib/systemd/network/80-container-ve.network
#  SPDX-License-Identifier: LGPL-2.1+
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# This network file matches the host-side of the virtual Ethernet link
# created by systemd-nspawn's --network-veth switch. See systemd-nspawn(1) for
# details.

[Match]
Name=ve-*
Driver=veth

[Network]
# Default to using a /28 prefix, giving up to 13 addresses per container.
Address=<IP_ADDRESS>/28
LinkLocalAddressing=yes
DHCPServer=yes
IPMasquerade=yes
LLDP=yes
EmitLLDP=customer-bridge

cat /usr/lib/systemd/network/80-container-host0.network
#  SPDX-License-Identifier: LGPL-2.1+
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# This network file matches the container-side of the virtual Ethernet link
# created by systemd-nspawn's --network-veth switch. See systemd-nspawn(1) for
# details.

[Match]
Virtualization=container
Name=host0

[Network]
DHCP=yes
LinkLocalAddressing=yes
LLDP=yes
EmitLLDP=customer-bridge

[DHCP]
UseTimezone=yes

ip address show host0
ip address show host0
2: host0@if9: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a2:32:d8:4f:17:cf brd ff:ff:ff:ff:ff:ff link-netnsid 0

networkctl status host0
WARNING: systemd-networkd is not running, output will be incomplete.

Failed to query link bit rates: The name org.freedesktop.network1 was not provided by any .service files
Failed to query link DHCP leases: The name org.freedesktop.network1 was not provided by any .service files
● 2: host0
                     Link File: n/a
                  Network File: n/a
                          Type: ether
                         State: n/a (unmanaged)
                    HW Address: a2:32:d8:4f:17:cf
                           MTU: 1500 (min: 68, max: 65535)
                         QDisc: noop
  IPv6 Address Generation Mode: eui64
          Queue Length (Tx/Rx): 1/1
              Auto negotiation: no
                         Speed: 10Gbps
                        Duplex: full
                          Port: tp

systemctl status systemd-networkd
● systemd-networkd.service - Network Service
     Loaded: loaded (/usr/lib/systemd/system/systemd-networkd.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Tue 2020-10-13 14:25:08 UTC; 14s ago
TriggeredBy: ● systemd-networkd.socket
       Docs: man:systemd-networkd.service(8)
    Process: 488 ExecStart=/usr/lib/systemd/systemd-networkd (code=exited, status=200/CHDIR)
   Main PID: 488 (code=exited, status=200/CHDIR)

Oct 13 14:25:08 buildspawn systemd[1]: systemd-networkd.service: Scheduled restart job, restart counter is at 5.
Oct 13 14:25:08 buildspawn systemd[1]: Stopped Network Service.
Oct 13 14:25:08 buildspawn systemd[1]: systemd-networkd.service: Start request repeated too quickly.
Oct 13 14:25:08 buildspawn systemd[1]: systemd-networkd.service: Failed with result 'exit-code'.
Oct 13 14:25:08 buildspawn systemd[1]: Failed to start Network Service.

cat /etc/systemd/network/all-ethernet.network
[Match]
Type=ether

[Network]
DHCP=yes

It's oposite of yours as my host0 is completely in down state and ve-* is in configuring state. I have no idea why it just don't work.

How about /usr/lib/systemd/network/80-container-ve.network in host? You showed it in container, not host.
And your log shows systemd-networkd not running in container. Would you show me your journalctl -u systemd-nspawn@buildspawn after starting container?
Also, any changes when you use kernel providedlinux instead of linux-lts?

on HOST:
cat /usr/lib/systemd/network/80-container-ve.network
#  SPDX-License-Identifier: LGPL-2.1+
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

# This network file matches the host-side of the virtual Ethernet link
# created by systemd-nspawn's --network-veth switch. See systemd-nspawn(1) for
# details.

[Match]
Name=ve-*
Driver=veth

[Network]
# Default to using a /28 prefix, giving up to 13 addresses per container.
Address=<IP_ADDRESS>/28
LinkLocalAddressing=yes
DHCPServer=yes
IPMasquerade=yes
LLDP=yes
EmitLLDP=customer-bridge

vi /etc/systemd/nspawn/buildspawn.nspawn
[Exec]
PrivateUsers=true
NotifyReady=true

[Files]
# Commented out below also works fine
# Bind=/ramdisk/scratch/:/scratch/

[Network]
Private=yes
VirtualEthernet=yes

journalctl -u systemd-nspawn@buildspawn log here
I use linux-lts, not tried with linux yet.

Thank you.
Your 80-container-ve.network seems to be fine, so there's no fault for your configuration.
It seems there are some problems in networkd & resolved in nspawn container with error: Failed to connect stdout to the journal socket, ignoring: Permission denied. I googled the error message, but there's no result for nspawn.
I don't have any answer / solution for you now...
This may not be caused by mkosi or this repository, I think.
You might want to consider asking for your issue in ArchWiki or https://github.com/systemd/systemd/issues.


Add Aprilaire Programming Guide?

Can you add the programming guide to this repo or is the guide to big for that? Aprilaire hasn't replied to any of my emails so I don't have it.

Unfortunately since it is Aprilaire's IP, I can't include it in the repo due to licensing. If you shoot me a DM on https://community.home-assistant.io/ I can send you a copy. I got it from them very easily via their contact form at https://www.aprilaire.com/contact, where I requested the Wifi Programmer Guide and received it in under 24 hours.


Text formatting on the homepage

I tried a variety of categories and they are all formatting this way

I enabled the excerpt field...

It'll appear at the bottom of the page, you can enter a small summary sentence or two here, which fixes this.


New option: Allow running the app in the background without tray icon

What feature or improvement do you think would benefit Files?
Some people like me don't like a try area that is too crowded, but would still like to benefit from the advantages of letting Files run in the background
Requirements
Maybe give the user an additional option to manage if there is a try icon independently from the settings that lets Files run in the background
Files Version
<IP_ADDRESS>
Windows Version
10.0.26100.1
Comments
No response

Thanks for the feedback, there isn't any plans on adding a option in Files. You can put it in the overflow via Windows Settings > Personalization > Taskbar > Other tray icons and then disable the overflow if you want.

Ok, unfortunate but fair

To be fair, this is something we're considering doing but we haven't committed to it just yet.

Yeah, all good, it's not a huge problem anyways :P

We're working on a built-in setting for this in #15947.

Great, thanks


Basic error handling

Create error pages:

application error page (replace the default /error fallback page),
404 page,
403 page (possibly differentiate between unauthenticated session and lack of permissions)

What is left to do @apardyl? 403 page and "application error page"?

What is left to do @apardyl? 403 page and "application error page"?

404 and 403. Default fallback page/application error has been implemented in 34bca3daef49282ba43f040fb77e9a19c0abeb8a


remove the layer-breaking getLobCreator() methods from Hibernate class

This is in principle a breaking change, but one I consider pretty innocuous, since I can't imagine why anyone was calling Hibernate.getLobCreator(session) instead of session.getLobHelper().
It's important that we not leak internal APIs like SessionImplementor into the signature of public APIs in org.hibernate.

P.S. if someone is very concerned about the breakage, we can put back the version which accepts Session. But I can't quite see it's purpose.


RAS-638 Investigate how "Ready for live" button works, when replacing sample file (track time)

What and why?
This PR reorders the generation of the collection exercise page so that when a sample is loaded in response operations, and all other values are loaded, the Set Ready for live button appears.
How to test?

Deploy this PR to your environment
Deploy https://github.com/ONSdigital/rm-sample-service/pull/248 to your environment.

Must be deployed last

Deploy https://github.com/ONSdigital/response-operations-ui/pull/888 to your environment
Set up a collection exercise in this order

Required event dates
A collection instrument
A sample file

Ensure the sample has loaded
There is no Refresh link at the top of the page
The Set ready for live button appears and the collection exercise

Jira
RAS-638

/deploy scorfs

/deploy harrisonj


Deep relationship loading fails with morph relation

Reproducible Example: https://jsfiddle.net/51vrmud1/37/
When trying to load relation through morphMany relation, it some how retrieves the record with default values. Not sure why this is happening but it seems like caused by morph relationship...
/cc @phortx

This might not be a defect since withAll will only fetch one level deep relation. You need to use withAllRecursive. However, if you create morphRelation through create method, fetched data with withAll will contain record with default value instead of null and this is a defect. Looking into it.


Document authentication

How to pass the password to the PSQL connection is not obvious from the current documentation.
Add a basic example with links to the more documentation.

This is a libpq feature and there are many different ways to do this.
I'd rather have this in the FAQ part and would prefer something like :

pg_activity uses libpq to access to PostgreSQL. All the traditional methods are available.
You can pass the password for the database connection in a password file. Information can also be given
via PostgreSQL's environment variables (PGPASSFILE or PGPASSWORD) or via the connection string parameters.
The password file is preferred since it's more secure (security is deferred to the OS). Please avoid password in connection string at all cost.

Also : @miohtama, thank you for your contribution :)

Are you guys ok with this ? If so I can do the PR.

Are you guys ok with this ? If so I can do the PR.

Yes.

Pushed it that way.
@miohtama, thanks for the patch
@dlax, thanks for reviewing


PLUGIN-380 Filter Support

added: Filter support for Spanner source
Jira Ticket: https://issues.cask.co/browse/PLUGIN-380

lgtm. please squash the commits

squashed

Please resolve the conflicts

Please resolve the conflicts

resovled


Kibana dashboard sustainability

The Kibana dashboards draw from the metrics-conversion-prod index pattern. The dashboards are used to support DE decision making but they also produce the stats for the quarterly Departmental meeting and slide pack for the Board of Governors.
Each visualisation uses a different set of criteria based on the values of certain fields. If the values change, I need to be told so that I can change the visualisation definitions and produce accurate stats.

The fields that are currently used for the dashboards are:
page.name
page.pathname
page.query.id
page.query.query
page.query.workdId
properties.event
properties.imageId
source

The definitions are held in Notion.

As of 8 February only one of the dashboards has been updated to reflect item and works page (or equivalent) view for both library catalogue and images. Such as they are, here are the definitions on 8 February:

@harrisonpim Inspiration struck. I've started building a checking dashboard as a quick visual check on changes to individual fields over time.

This ticket has been replaced by #10555


Ignore ngInclude directive

It's possible to disable specific $http request create by ngInclude directive?

Hmm, no.  Happy to accept a PR!


Restructure intro, getting started and tutorial

Description
This PR:

Adds more dlt topics coverage to the tutorial.
Restructures "Getting started" and "Introduction" pages.

@AstrakhantsevaAA thanks for the comment!

Now I don’t really see the differences between the tutorial and how-to guides. if we combine them, nothing will change.
There should be a difference in the level of specificity: tutorial is a more general guide that teaches the core concepts, where how-to guides are more specific steps of instructions.

The Getting Started page has also lost its relevance. they are now similar to the Introduction page. maybe makes sense to merge them too?

I don't think Getting Started lost its relevance: I shortened the page so it's really a quick start: Install – Load - View.
Introduction is an overview of dlt: what that is + a brief showcase of different sources

update: at least the title Load data with API do not reflect the content. I got the idea of the tutorial section: it is something between How-to and using-dlt. mb rename it to "Load data with append, replace, merge mode"

I believe an uninitiated user won't know what is append, replace and merge: the point of the first part is to teach them what that actually is and why they may need to use different loading modes.
I'm open to suggestion of a better naming for the section. My goal is to summarize the content of the first part.

@AstrakhantsevaAA thanks for the feedback. The content from the getting started is still in the docs, I've moved it to the first part of the tutorial. My hypothesis is that it would not affect the understanding of the material. Another is that having a cohesive storyline also aids people in grasping dlt's key concepts fast.


Apply detect() on readable PDF files

Hi there,
from the docs I infere that detect() operates, for example, on PIL.Image objects. Is there way to directly operate on already readable PDF files (which obviates the need applying OCR as well).
Greetings

Thanks for pointing out. Providing support for PDF is among one of our future plans. And in the meantime, you could try to use the pdf2image library to firstly convert the PDF to some image scans and perform layout detection on them. You might want to go to their Github homepage for a detailed installation instruction: https://github.com/Belval/pdf2image .

@lolipopshock thanks for the response! Thought about this workflow as well, however, I assumed this approach would ignore the fact that PDF files are already run through OCR beforehand (i.e. readable). I find it somewhat cumbersome to first convert a readable PDF to an image, only to then re-apply OCR...

You might want to check the pdfplumber library and here is some starter code for you:

import pdfplumber
from typing import List, Union, Dict, Any, Tuple

def obtain_word_tokens(cur_page: pdfplumber.page.Page) -> List:
    words = cur_page.extract_words(
                x_tolerance=1.5,
                y_tolerance=3,
                keep_blank_chars=False,
                use_text_flow=True,
                horizontal_ltr=True,
                vertical_ttb=True,
                extra_attrs=["fontname", "size"],
    )

    return lp.Layout([
                lp.TextBlock(
                    lp.Rectangle(float(ele['x0']), float(ele['top']),
                                 float(ele['x1']), float(ele['bottom'])),
                    text=ele['text'],
                    id = idx
                ) for idx, ele in enumerate(words)]
            )

plumber_pdf_object = pdfplumber.open(pdf_path)

all_pages_tokens = []
for page_id in range(len(plumber_pdf_object.pages)):
    cur_page = plumber_pdf_object.pages[page_id]
    tokens = obtain_word_tokens(cur_page)
    all_pages_tokens.append(tokens)

I am thinking of combining Poppler and this layout parser to extract structured paragraph from readable PDFs, such that both advantages from the two can be leveraged.
Has anyone done that?

You might want to check the pdfplumber library and here is some starter code for you:
import pdfplumber
from typing import List, Union, Dict, Any, Tuple

def obtain_word_tokens(cur_page: pdfplumber.page.Page) -> List:
    words = cur_page.extract_words(
                x_tolerance=1.5,
                y_tolerance=3,
                keep_blank_chars=False,
                use_text_flow=True,
                horizontal_ltr=True,
                vertical_ttb=True,
                extra_attrs=["fontname", "size"],
    )

    return lp.Layout([
                lp.TextBlock(
                    lp.Rectangle(float(ele['x0']), float(ele['top']),
                                 float(ele['x1']), float(ele['bottom'])),
                    text=ele['text'],
                    id = idx
                ) for idx, ele in enumerate(words)]
            )

plumber_pdf_object = pdfplumber.open(pdf_path)

all_pages_tokens = []
for page_id in range(len(plumber_pdf_object.pages)):
    cur_page = plumber_pdf_object.pages[page_id]
    tokens = obtain_word_tokens(cur_page)
    all_pages_tokens.append(tokens)

You might want to check the pdfplumber library and here is some starter code for you:
import pdfplumber
from typing import List, Union, Dict, Any, Tuple

def obtain_word_tokens(cur_page: pdfplumber.page.Page) -> List:
    words = cur_page.extract_words(
                x_tolerance=1.5,
                y_tolerance=3,
                keep_blank_chars=False,
                use_text_flow=True,
                horizontal_ltr=True,
                vertical_ttb=True,
                extra_attrs=["fontname", "size"],
    )

    return lp.Layout([
                lp.TextBlock(
                    lp.Rectangle(float(ele['x0']), float(ele['top']),
                                 float(ele['x1']), float(ele['bottom'])),
                    text=ele['text'],
                    id = idx
                ) for idx, ele in enumerate(words)]
            )

plumber_pdf_object = pdfplumber.open(pdf_path)

all_pages_tokens = []
for page_id in range(len(plumber_pdf_object.pages)):
    cur_page = plumber_pdf_object.pages[page_id]
    tokens = obtain_word_tokens(cur_page)
    all_pages_tokens.append(tokens)

Hm, indeed that would help to extract the actual text, token-by-token. However, if I got it correctly, this approach would still require to use something like the pdf2image library to convert the PDF to images to detect actual contiguous text blocks and then match the word-level text blocks to the Layout object respectively the page-level TextBlocks detected by one of the layout detection models (to not incorporate words/tokens that correspond to figures or tables). I am still not quite sure with respect to the optimal pipeline. Do you expect the PDF-functionality in the near future? Otherwise, I would proceed with one of the "dirtier" workflows proposed in this thread.

Hi there,
you probably should add an additional step between 2. and 3. in order to check if there is any text embedded inside the boundaries of the text_block. Because you may have to fall back to use OCR when there is no text embedded.
I'm currently working on a project where I'll need a structured output of a handful of PDFs in different qualities which some of them will consist of text and others just are filled with images from scanned documents.

Something like tabula https://github.com/tabulapdf/tabula might be helpful? Although this is a bit applied to tables, it does leverage the structure of the pdf to pull tables.

This raised a new issue, however. That is, the coordinates (x1, x2, y1, y2) assigned by pdfplumber are on a different scale compared to the coordinates detected in 2. which are in turn reliant on the dpi when using convert_from_path in the pdf2image library I assume...

May I ask how you solved this scale issue. I also have different x,y coordinates for the text of pdfplumber and the layout boxes of LayoutParser.

Hi @gevezex

May I ask how you solved this scale issue.
I must admit that I simply dropped the project for now. I found it way too tedious worfklow, hoping the team will add some support for already readable PDF-files in the future

Solved it like this with PyMuPdf (pip install pymupdf)
Check also the pymupdf utility for retrieving text out of certain box coordinate

# function for rescaling xy coordinates
def scale_xy(textblock, scale=72/200):
    x1 = textblock.block.x_1 * scale
    y1 = textblock.block.y_1 * scale
    x2 = textblock.block.x_2 * scale
    y2 = textblock.block.y_2 * scale
    return (x1,y1,x2,y2)

# Using PyMuPdf for retrieving text in a bounding box
import fitz  # this is pymupdf

# Function for retrieving the tokens (words). See pymupdf utilities
def make_text(words):
    """Return textstring output of get_text("words").
    Word items are sorted for reading sequence left to right,
    top to bottom.
    """
    line_dict = {}  # key: vertical coordinate, value: list of words
    words.sort(key=lambda w: w[0])  # sort by horizontal coordinate
    for w in words:  # fill the line dictionary
        y1 = round(w[3], 1)  # bottom of a word: don't be too picky!
        word = w[4]  # the text of the word
        line = line_dict.get(y1, [])  # read current line content
        line.append(word)  # append new word
        line_dict[y1] = line  # write back to dict
    lines = list(line_dict.items())
    lines.sort()  # sort vertically
    return "\n".join([" ".join(line[1]) for line in lines])

# Open your pdf in pymupdf
pdf_doc = fitz.open(filename)
pdf_page4 = pdf_doc[3]  # this wil retrieve for example page 4
words = pdf_page4.get_text("words")

# Get one of your inferenced TextBlocks what is detected with your model (model LayoutParser)
# In the doc it was called "layout". So will use that one
# first recognized bounding box:  layout[0]
# When I print my pdf version the output of layout[0] is like this:
>>> TextBlock(block=Rectangle(x_1=104.882, y_1=133.696, x_2=124.79, y_2=147.696), text=Het, id=0, type=None, parent=None, next=None, score=None)

# Rescale the coordinates
new_coordinates = scale_xy(layout[0])

# Create a Rect object for fitz (similar to TextBlock for the bounding box coordinates)
rect = fitz.Rect(*new_coordinates)

# Now we can print find and print all the tokens in the bounding box:
mywords = [w for w in words if fitz.Rect(w[:4]).intersects(rect)]

print("\nSelect the words intersecting the rectangle")
print("-------------------------------------------")
print(make_text(mywords))

Sorry for the confusion of terminologies. I am still learning pdf related stuff.

You might also want to refer to the PDF Parsers that I've implemented in another project recently -- https://github.com/allenai/VILA/blob/master/src/vila/pdftools/pdfplumber_extractor.py @ https://github.com/allenai/VILA/pull/6 . It should provide similar functionalities and readily applicable to the layout-parser library as well. I will merge the PDF parsers in the layout-parser library soon.

See #71  and #72


typescript @types for @GoogleCloud/Logging?

When trying to import @GoogleCloud/Logging I get -

Could not find a declaration file for module "google-cloud/logging"

Note that I removed the "at" as it would try to tag people, and the text disappeared.
Is there a fix to this?
Environment details

OS: Mac OS High Sierra 10.13.1
Node.js version: 6.11.1
npm version: 3.10.10
@google-cloud/logging version: 1.1.1

Steps to reproduce

Apply typescript to your project and import the logging library with the import syntax.

Kind regards Chris

I've posted a response in your SO OP with a draft for typedefs. You're free to reuse it of course !

Thanks for getting back to me! I'll have a go at it in a few hours. It looks good though :)
Kind regards Chris

Hey @sveinburne
I've read your stack overflow post, and I'm having a bit of trouble understanding the steps. I've followed a guide for setting cloud functions up with typescript.
The part that confuses me is that you write I need to have @types/node installed - which I have, and then declare target as node which I also have (I believe). I have a tsconfig.json looking like this.
{     "compilerOptions": {         "target": "es6",         "module": "commonjs",         "outDir": "./",         "noImplicitAny": true     },     "include": [         "src/**/*.ts"     ],     "exclude": [         "node_modules"     ] }
and a web pack file looking like this:
`'use strict';
var nodeExternals = require('webpack-node-externals');
module.exports = {
entry: './src/index.ts',
output: {
filename: 'index.js',
libraryTarget: 'this'
},
target: 'node',
module: {
rules: [
{
test: /.tsx?$/,
loader: 'ts-loader',
options: {
transpileOnly: true
}
}
]
},
resolve: {
extensions: ['.ts', '.tsx', '.js']
},
externals: [nodeExternals()]
};`
Should the const Logging: any = require('@google-cloud/logging') work out of the box then?
I haven't been able to test it yet - still working on implementing a class which uses the library.
Kind regards Chris

@chriseenberg
If you use the typedefs I provided, you don't need to use require nor to add @types/node! Just write import * as Logging from '@google-cloud/logging' and add a file logging.d.ts as a sibling or a parent of the source consuming this API containing the definitions I wrote.

Thanks, it works perfectly now! closing this issue.


python310Packages.pytensor: fix src hash

Description of changes
Follow up of https://github.com/NixOS/nixpkgs/pull/255882

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg build python310Packages.pytensor python311Packages.pytensor


Update the MemoryManager API so it can be used in conjunction with NIO to provide efficient buffer management

If MemoryManager implemented something like a  ByteBuffer allocate(int size) then I could be used to directly transfer data to/from Files and Sockets without having to first copy them to a byte[].

First of all it should be allocateDirect() and then
ByteBuffer.allocateDirect() is a performance bottleneck - it is called just
once during the initialization phase right to address this issue.
sun.Unsafe could offer benefits for this approach instead
On Mon, Nov 7, 2011 at 6:01 PM, Hiram Chirino <
<EMAIL_ADDRESS>
If MemoryManager implemented something like a  ByteBuffer allocate(int size) then I could be used to directly transfer data to/from Files and
Sockets without having to first copy them to a byte[].

Reply to this email directly or view it on GitHub:
https://github.com/raffaeleguidi/DirectMemory/issues/49

I think you mis-understand me.  I don't want to allocation new direct byte buffers.  I want ByteBuffer allocate(int size) to allocate a slice of the existing OffHeapMemoryBuffers that have been allocated.

Uhm but allocate() is a static method that (quote from [1]) "Allocates a
new direct byte buffer"... I'm sure I am missing something (sorry, I'm
getting old :P) - but maybe you could write a small gist to better explain
what you mean
Ciao,
R
[1]
http://download.oracle.com/javase/1.4.2/docs/api/java/nio/ByteBuffer.html#allocate(int)
On Mon, Nov 7, 2011 at 7:29 PM, Hiram Chirino <
<EMAIL_ADDRESS>
I think you mis-understand me.  I don't want to allocation new direct byte
buffers.  I want ByteBuffer allocate(int size) to allocate a slice of the
existing OffHeapMemoryBuffers that have been allocated.

Reply to this email directly or view it on GitHub:
https://github.com/raffaeleguidi/DirectMemory/issues/49#issuecomment-2657322

I'm not talking about using the ByteBuffer.allocate call.  I'm talking about adding a method to MemoryManager:
ByteBuffer bb = MemoryManager.allocate(1024*1024);

So that I can directly modify the ByteBuffer using Channels to avoid a memory copy..  Compare this to using the MemoryManager store, update, and retrieve calls.  They all force you to work with byte[]s so you end up doing a memory copies into and out of direct memory.

Ok, I finally got it. Basically you would like to create a new ByteBuffer
that calls the slice() method on the allocated buffer and returns it to the
user - this could of course be useful, but a pointer should be created in
any case and added to the list to "reserve" space for it - otherwise it
could be overwritten by other operations. Let's say the method should
return a DirectBufferWithPointer (ok, let's find a better name...)
encapsulating both the pointer and the ByteBuffer, otherwise handling two
return values (the bytebuffer and the pointer) would be problematic
On Mon, Nov 7, 2011 at 11:27 PM, Hiram Chirino <
<EMAIL_ADDRESS>
I'm not talking about using the ByteBuffer.allocate call.  I'm talking
about adding a method to MemoryManager:
ByteBuffer bb = MemoryManager.allocate(1024*1024);
So that I can directly modify the ByteBuffer using Channels to avoid a
memory copy..  Compare this to using the MemoryManager store, update, and
retrieve calls.  They all force you to work with byte[]s so you end up
doing a memory copies into and out of direct memory.

Reply to this email directly or view it on GitHub:
https://github.com/raffaeleguidi/DirectMemory/issues/49#issuecomment-2660872


Main build failure due to missing Microsoft.Taef.10.58.210222006-develop

Externals don't have access to Taef-develop package
C:\Sources\ProjectReunion\test\CppTest\CppTest.vcxproj(342,5):
error : This project references NuGet package(s) that are missing on this computer.
Use NuGet Package Restore to download them.  For more information, see http://go.microsoft.com/fwlink/?LinkID=322105.
The missing file is ..\..\packages\Microsoft.Taef.10.58.210222006-develop\build\Microsoft.Taef.targets.

https://github.com/microsoft/ProjectReunion/blob/93e786dc7461e41ddd4afc2476a4d1cf765a6bdd/test/CppTest/CppTest.vcxproj#L342
cc: @kythant

This should be visible on our public-facing ADO feed.
Can you see this link? I switched to my personal ADO account, and it seems to be working, so you may just need to add that feed to your VS configuration (which also means I may need to fix instructions somewhere).
https://dev.azure.com/ms/ProjectReunion/_packaging?_a=package&feed=ProjectReunion-Dependencies&package=Microsoft.Taef&protocolType=NuGet&version=10.58.210222006-develop

This was not because of PR-dependencies feed. This was due to the ExternalES feed I was using. I didn't realize it was not public when I originally did the change. So I pulled the Taef package from there and pushed it to PR-Dependencies. The pull request is out for the fix.
https://github.com/microsoft/ProjectReunion/pull/494


Fix objects jumping when manipulating with VR controllers in editor.

Overview
This is a very simple change that disables camera controls when an XR device is present. Without this change, the camera position, returned by the camera cache, would jump upwards when the VR controller trigger was pulled. You can see this in the gifs below, as the profiling panel jumps up and down with the head position as the trigger is pulled.
When the head position jumped upwards, it was suddenly closer or further from the controller. This distance is used in TwoHandMoveLogic to scale the distance between the controller and the object. This was then causing the object to jump.
In editor before change:

Built for x64 running on local machine:

In editor after change:

Changes

Disabled camera controls when an XR device is present (equivalent to disabling IsCameraControlEnabled in the input simulation profile)

Fixes: #6108

/azp run


[ML-9741] Use atexit handler to delete cached folder

What changes are proposed in this PR?
When make_spark_converter(...) in a python process write a df cache into a folder, a delete function is registered to the atexit handler to delete the folder. When the python process exits (normally with SIGTERM), the cache written by this process would be deleted.
Why it is needed?
To protect data security.
How is this PR tested?
Unit test.
User-facing API changes?
No.
Note: This PR include the changes in #2, when #2 is merged to master, this PR would include minimum changes.

Codecov Report

Merging #4 into master will decrease coverage by 0.3%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master       #4      +/-   ##
==========================================
- Coverage   85.56%   85.26%   -0.31%
==========================================
  Files          80       80
  Lines        4233     4235       +2
  Branches      668      668
==========================================
- Hits         3622     3611      -11
- Misses        503      511       +8
- Partials      108      113       +5

Impacted Files
Coverage Δ

petastorm/spark/spark_dataset_converter.py
100% <100%> (+2.32%)
:arrow_up:

petastorm/compat.py
63.63% <0%> (-36.37%)
:arrow_down:

petastorm/arrow_reader_worker.py
90.4% <0%> (-1.6%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f9235f4...d6db354. Read the comment docs.


Support for snappy codec

Stack trace:
Traceback (most recent call last):
  File "get_sig_set.py", line 34, in <module>
    records = get_records_fastavro("avro-files/" + fname)
  File "get_sig_set.py", line 11, in get_records_fastavro
    records = [record for record in reader]
  File "fastavro/_reader.py", line 432, in _iter_avro (fastavro/_reader.c:9237)
ValueError: Unrecognized codec: u'snappy'

Version: 0.12.2
Python: 2.7.13

Do you have python-snappy installed? snappy will only be available as a codec if you have that installed: https://github.com/tebeka/fastavro/blob/master/fastavro/reader.py#L411-L422
What's your current pip list?

No, I did not. I installed that module and everything worked fine. Please feel free to close.

Thanks @scottbelden !


[Feature]: Single Gsheet as a datasource

Is there an existing issue for this?

[X] I have searched the existing issues

Summary
follow-up from the PoC, relevant documents:
https://github.com/appsmithorg/appsmith/issues/18337
https://www.notion.so/appsmith/POC-for-Google-Sheet-Datasource-for-Single-Sheet-8ca6c3dfeaba47509acc35282887b0a8
I should be able to use a single google sheet as a datasource, so that I don't have to give appsmith the access to my entire google drive or all my google sheets
I should be able to revoke the authorization for the single sheet, so that appsmith can no longer access it, and queries on this datasource will fail with an appropriate authorization error
Why should this be worked on?
When creating a google sheets datasource today, appsmith requests access to the entire google drive - (what can be perceived as) too much access. This may make users reluctant to give the necessary access, and therefore they are not able to execute the queries. This directly affects the trial conversion.
Very often, I come to appsmith with the idea to build an app that takes data from a single data source. This feature will lower the barrier for such use cases, and help users to build apps more readily.

closing this in favour of #21055


Add moleculerjs and nextjs boilerplate to users

Add moleculerjs and nextjs boilerplate to users.json

Deploy preview for docusaurus-preview failed.
Built with commit 1a4a53463bbd3cad79ef1163dd41bf03c3ba7adf
https://app.netlify.com/sites/docusaurus-preview/deploys/5d2f18200b9a2d0008245711

Deploy preview for docusaurus-preview failed.
Built with commit 168beee89b7acd9b3b79e5e62dde24528c8991d4
https://app.netlify.com/sites/docusaurus-preview/deploys/5d2f18cc3e30830007b5e292


Feature/django 1.8.x

Please do not merge this yet.
We need to work out how to deploy migrations with --fake-initial.
The index for date_created on table reversion_revision has been created in all environments.

:+1:
Had a conversation with @leenagupte and @timmow in meatspace about to manage the migration issue.


Broken in atom 1.7.0

atom 1.7.0 changes the behaviour of tab switching and this plugin breaks-it does display me a list of files, but it is not focused-the underlying editor still has focus

I have this problem too.

Facing same issue on atom v1.7.1

same here

+1

+1

Same.

Same for me

Sorry for late response, issue was fixed, please update to the latest version, also take into consideration that this plugin will override new atom MRU tab switching behavior.

It fails to update now.

@masics , can you give me more details please, what OS, atom version


Fix crash for cli commands

Good catch, thanks! 👍

This patch is now available in v3.0.1
Thanks again for submitting it! 🍺