Capturing logging output within the report

Hey,
I would like the ability include python stdlib logging output in the report under test/phase they were produced. This would be useful regardless of its success or failure.
I could add this functionality myself if you added some more granular hooks for modifying the report around each test run/phase. Alternatively I could technically implement it with the current API by managing my own logging output tree and merging it with the report at the end within the pytest_json_modifyreport hook, is this how you intended it to be used?
My end goal here would be to essentially integrate the structlog and its JSON log output format with your report structure for my own uses.
Cheers,
Rob

Thanks for reporting!
The intended way to add metadata to a test item is via the json_metadata fixture.
E.g., you could use an autouse fixture which automatically surrounds every test call:
import pytest

@pytest.fixture(autouse=True)
def my_fixture(json_metadata, request):
    json_metadata['started'] = request.node.nodeid
    yield
    json_metadata['finished'] = request.node.nodeid

def test_foo():
    assert 1 + 1 == 2

This would give you a test item like:
{
    "nodeid": "test_foo.py::test_foo",
    "path": "test_foo.py",
    "outcome": "passed",
    ...
    "setup": {
        "duration": 0.001293182373046875,
        "outcome": "passed",
    },
    "call": {
        "duration": 0.00014352798461914062,
        "outcome": "passed"
    },
    "teardown": {
        "duration": 0.0002033710479736328,
        "outcome": "passed"
    },
    "metadata": {
        "started": "test_foo.py::test_foo",
        "finished": "test_foo.py::test_foo"
    }
}

The same way you could setup and tear down a logger. Would that work for you?

Thanks for the quick response,
I'm also going to want to record the logging output produced by setup/teardown of all fixtures used by a test. I don't think you can control fixture execution order, so I'm probably going to have to do the work at the plugin level.
My first approximation would be something like this:
import pytest
from contextlib import contextmanager
import logging
from collections import defaultdict

class LoggingHandler(logging.Handler):
    def __init__(self):
        super().__init__()
        self.captured_logs = []

    def emit(self, record):
        self.captured_logs.append(self.format(record))

class CapturePlugin(object):
    def __init__(self):
        self.captured_logs = defaultdict(dict)

    @pytest.mark.hookwrapper
    def pytest_runtest_setup(self, item):
        with self._capture_logs(item, 'setup'):
            yield

    @pytest.mark.hookwrapper
    def pytest_runtest_call(self, item):
        with self._capture_logs(item, 'call'):
            yield

    @pytest.mark.hookwrapper
    def pytest_runtest_teardown(self, item):
        with self._capture_logs(item, 'teardown'):
            yield

    @contextmanager
    def _capture_logs(self, item, phase):
        handler = LoggingHandler()
        logging.getLogger().addHandler(handler)
        try:
            yield
        finally:
            logging.getLogger().removeHandler(handler)
        self.captured_logs[item.nodeid][phase] = handler.captured_logs

    def pytest_json_modifyreport(self, json_report):
        # merge the logs into the report here...
        json_report["captured_logs"] = self.captured_logs

def pytest_configure(config):
    plugin = CapturePlugin()
    config._log_capture = plugin
    config.pluginmanager.register(plugin)

As you can see it could be slightly nicer to have a hook for extending your reports test nodes as they are created.
Honestly I always find myself requiring all the log output saved when dealing with test systems (I mainly deal with tests which drive external hardware so may remote things can go wrong), would be something you would consider supporting natively?

Thanks for the elaboration.

As you can see it could be slightly nicer to have a hook for extending your reports test nodes as they are created.

Would a simple pytest_json_modifytest / _modifystage hook be sufficient here?

would be something you would consider supporting natively?

Generally, yes. Log captures are definitely a common use case. That said, I'm not entirely sure about the implementation. Pytest uses its own logging handlers which I'd want to take advantage of, if possible. I'd happily take PRs or suggestions.

Since this is really the only extra feature I'd want before using your package more widely, I'd be glad to help.
Pytest has recently integrated pytest-catchlog into its builtin plugins (https://docs.pytest.org/en/latest/logging.html) so pretty much all logging configuration is now handled pretty nicely by the framework.
I'm not sure why we would need to mess with any of their logging handlers, any handlers attached to the root logger will receive the same information. There is a question of assigning a formatter through and if you want to store the logs a opaque strings or as more structured JSON objects.
In regards to the additional modify test item hook, maybe I'm being overly pedantic. There's something to be said for keeping the interface simple and its not really that challenging to merge within the  pytest_json_modifyreport hook. I think you would probably need to think of a few more use cases before implementing such a thing.

I implemented log capturing based on your approach.
There's now a log list on each test stage in which logs were captured. You can also call logging.makeLogRecord() on an entry to get back a proper LogRecord object.
Let me know what you think.

I pretty much copied my approach from the internals of the pytest-catchlog package, so I can't take all the credit.
I'm had play with your branch, looks great overall, I just have a few questions.
I like that you have preserved all the information about the log capture in a structured form. However this is going to massively bloat and effect the human readability of your produced reports. I've seem tests that can produce hundreds of logs in just their setup phase, this is going to dominate everything else.
I think a hook where you can transform a LogRecord into a JSON object of your choice could be quite useful with your current implementation being the default.
Something like pytest_json_logtojson or pytest_json_logjsonserializer maybe.
I appreciate this would cause you loose the ability to reconstruct the LogRecord Object.
Actually it seems using a custom formatter to make the stdlib logging output JSON instead of opaque strings is not a new idea:
https://github.com/madzak/python-json-logger
However format is still expected to return string (even if that contains encoded JSON), so its probably not appropriate here.
You only capture stdout and logging in the case a failed test phase. I would like an option to capture everything regardless of outcome. There is still useful information in the output of a successful test, such as any form of subsystem profiling.

Thanks for taking the time to review!

I think a hook where you can transform a LogRecord into a JSON object of your choice could be quite useful with your current implementation being the default.

Alternatively, I was thinking about an option to switch between the dict or a specified formatted string representation. But if you need more control, I'm fine with adding a hook, too.
I was also thinking if there is an elegant way to specify a generalized hook for modifying different sections of the report along the lines of pytest_json_modify(section, object) where the passed section can be "testitem", "teststage", "logitem", etc. Not sure if that's bad design though.

You only capture stdout and logging in the case a failed test phase.

I didn't plan to exclude logging or stdout from successful tests. Could give me a quick test case? I'm currently unable to reproduce that.

Turns out I can't reproduce it either, sorry about that, not sure what led me to that conclusion. Seems to capture everything I would want.
Its difficult to get a more granular hook system right. Multiplexing all the sections onto a single hook is fine. However you want to make sure the user has access all the relevant context about the section object's position within report as well as object itself. I think this should probably be discussed in a separate issue, where we can outline the potential use cases for such hooks.
I think the current logging branch is ok to merge with the appropriate update in documentation. A command line argument to override to the JSON log output with a logging format string could be useful, but I'd to wait for someone to actually request it before implementing it.
The only situation where I'd want to remove structured information in favor of a opaque string is one where a human was viewing the JSON reports. In such a case you should probably be using pytest-html anyway.

Hello !
Nice work that’s exactly what I need !
But I can’t seem to get this work. I don’t have the "log" attribute in my report.
All I’m doing is a thing like:
import logging

def test_foo():
    logging.info("This is a message.")

Do I need something more ?


mysql cdc job faild com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool

Describe the bug(Please use English)
A clear and concise description of what the bug is.
Environment :

Flink version :  1.14.2
Flink CDC version: 2.2
Database and version: mysql 5.7

To Reproduce
Steps to reproduce the behavior:
start  mysql cdc job repeatedly with different source, there will some connection connect to mysql not closed.

flink host is <IP_ADDRESS>

I think this because the HikariDataSource was not closed properly.

There isn't any code to close the HikariDataSource.

Closing this issue because it was created before version 2.3.0 (2022-11-10). Please try the latest version of Flink CDC to see if the issue has been resolved. If the issue is still valid, kindly report it on Apache Jira under project Flink with component tag Flink CDC. Thank you!


Dart Format

Since most of my PRs look like a lot has changed (because this project doesn't follow the idiomatic Dart format), this PR is the result of running:
dart format .

From this PR forward, all the changes should be in the idiomatic Dart format - so that the changes won't end up looking like a huge mess.

Sure thing. Let me do that!
This is indeed the first PR that needs to be merged - otherwise you'll see in the other PRs (unless you check commit by commit) huge, unrelated changes


Respect additional ways to identify s (name="...", is="...")

[X] CHANGELOG.md has been updated

@azakus @sorvell

@mgiuffrida informed me that <dom-module is="..."> also works (source) so i've added that as well.

LGTM


No SSL connection to mediathekview backend on Android 8

This is a bug I already fixed a few years ago... I have to reproduce it on an emulartor first.

I cannot reproduce this on any emulator. Closed until confirmed.


[Bug]:  Full screen mode lyrics

🔍 Have you checked that you are using an auto updating method or latest version of the extension?

[X] I have checked that I am on the latest version.

🔍 Is there already an issue for your problem?

[X] I have checked older issues, open and closed

Choose the extension for bug report.
Full Screen Mode
ℹ Environment / Computer Info
- Spotify version: Spotify for Windows <IP_ADDRESS>2.g24733a46
- Spicetify version: 2.14.1

📝 Description
No matter what, i can not show the song lyrics in the full screen mode
📸 Screenshots

i have the same issue :(

Install lyrics plus
spicetify config custom_apps lyrics-plus


add UUID assertion more verbose

Thank you for contributing! 🎉

For future reference, please submit PRs from topic or feature branches rather than your master branch. See here, for more information: https://guides.github.com/introduction/flow/


First Fit fixed algo for fixed size memory blocks in c, c++, java, python

added first fit fixed algo for fixed size memory blocks in c, c++, java, python

🛠 Issue #1494

Issue no #1494
👨💻 Changes proposed

✔ Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

📄 Note to reviewers

📷 Screenshots
C++

Java

C

Python

@ranjanmangla1 Add output screenshots

Done @Arun9739.
Please review it, @Kumar-laxmi, @072arushi


imv: 4.3.1 -> 4.4.0

Description of changes
Update to latest release

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Can you update the homepage to https://sr.ht/~exec64/imv/?

Can you update the homepage to https://sr.ht/~exec64/imv/?

Sure, thanks adjusted ✌🏻

The build is failing on darwin, may you fix it or mark it broken?

@ofborg build imv

The build is failing on darwin, may you fix it or mark it broken?

> meson.build:94:2: ERROR: Dependency "wayland-client" not found, tried pkgconfig and framework
I've had a look but I'm not sure why it would depend on this on darwin, so I marked it broken on darwin and aarch64

The aarch64-linux build is fine, is just darwin that's failing, likely because wayland doesn't support it.
Try with this patch:
diff --git a/pkgs/applications/graphics/imv/default.nix b/pkgs/applications/graphics/imv/default.nix
index b824b435670..22efd5e3d36 100644
--- a/pkgs/applications/graphics/imv/default.nix
+++ b/pkgs/applications/graphics/imv/default.nix
@@ -11,7 +11,7 @@
 , icu
 , pango
 , inih
-, withWindowSystem ? "all"
+, withWindowSystem ? null
 , xorg
 , libxkbcommon
 , libGLU
@@ -27,6 +27,12 @@
 }:

 let
+  # default value of withWindowSystem
+  withWindowSystem' =
+         if withWindowSystem != null then withWindowSystem
+    else if stdenv.isLinux then "all"
+    else "x11";
+
   windowSystems = {
     all = windowSystems.x11 ++ windowSystems.wayland;
     x11 = [ libGLU xorg.libxcb xorg.libX11 ];
@@ -47,7 +53,7 @@ let
 in

 # check that given window system is valid
-assert lib.assertOneOf "withWindowSystem" withWindowSystem
+assert lib.assertOneOf "withWindowSystem" withWindowSystem'
   (builtins.attrNames windowSystems);
 # check that every given backend is valid
 assert builtins.all
@@ -67,7 +73,7 @@ stdenv.mkDerivation rec {
   };

   mesonFlags = [
-    "-Dwindows=${withWindowSystem}"
+    "-Dwindows=${withWindowSystem'}"
     "-Dtest=enabled"
     "-Dman=enabled"
   ] ++ backendFlags;
@@ -87,7 +93,7 @@ stdenv.mkDerivation rec {
     libxkbcommon
     pango
     inih
-  ] ++ windowSystems."${withWindowSystem}"
+  ] ++ windowSystems."${withWindowSystem'}"
     ++ builtins.map (b: backends."${b}") withBackends;

   postInstall = ''
@@ -96,7 +102,7 @@ stdenv.mkDerivation rec {
     install -Dm644 ../files/imv.desktop $out/share/applications/
   '';

-  postFixup = lib.optionalString (withWindowSystem == "all") ''
+  postFixup = lib.optionalString (withWindowSystem' == "all") ''
     # The `bin/imv` script assumes imv-wayland or imv-x11 in PATH,
     # so we have to fix those to the binaries we installed into the /nix/store

@@ -113,6 +119,5 @@ stdenv.mkDerivation rec {
     license = licenses.mit;
     maintainers = with maintainers; [ rnhmjoj markus1189 ];
     platforms = platforms.all;
-    broken = (stdenv.isDarwin || stdenv.isAarch64);
   };
 }

Thanks for helping out with a patch, I've updated the PR - let's see what ofborg says :)✌🏻

It's still broken for another reason: can you add a meta.badPlatforms = platforms.darwin?

It's still broken for another reason: can you add a meta.badPlatforms = platforms.darwin?

np, or should I also revert the changes from the patch as well? not sure

np, or should I also revert the changes from the patch as well? not sure

No, that's fine: it more correct this way. Thank you.


Add internal memoMap file to fix layer and managed runtime circular imports

Type

[X] Refactor
[ ] Feature
[ ] Bug Fix
[ ] Optimization
[ ] Documentation Update

Description
Right now, there is two typeId definitions for effect/ManagedRuntime, one in layer and one in managedRuntime. If you change one of them then everything still type-checks, but it would not be correct. Would this be an appropriate solution? Didn't make a public MemoMap file because I didn't think it warranted changing the public Layer file

Related

Related Issue #
Closes #

/rebase

I think in this case we could just move the ManagedRuntime.TypeId to a internal/managedRuntime/circular.ts file.


Add another measure for conflicts of interest

This helps clarify another situation which would be appropriate for recusal.

I can't approve my own PR, but consider my approval given :).

@ellisonbg - very good point about context. How about:
An issue where the person privately gains an advantage from project resources, but The Project has no gain or suffers a disadvantage.

+1
On Thu, Feb 8, 2018 at 4:55 PM, Jason Grout<EMAIL_ADDRESS>wrote:

@ellisonbg https://github.com/ellisonbg - very good point about
context. How about:
An issue where the person privately gains an advantage from project
resources, but The Project has no gain or suffers a disadvantage.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jupyter/governance/pull/48#issuecomment-364300701,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABr0GZFRw-fQcithWNXsGmYfa8T94zPks5tS5ecgaJpZM4R-mDW
.

--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

@ellisonbg - I updated with that change.

This came out of a thread on the Steering council in January-February of this year, and there was wide agreement there about being more explicit about what constitutes a conflict of interest. Merging it now.


Signup html

HTML good enough for the IRL demo.
Broke the DB though, Neal need to bring it back online and add some more columns

fixed the db (well, if you have the new Keys.py)


docs(operators): Add documentation for fromEvent

This adds documentation for the fromEvent operator. The documentation has been partially rewritten
to clarify certain parts a bit more. This change will close issue #78. Also added an extra innerHTML property binding for the parameter.description to be able to use a hyperlink.

Codecov Report

Merging #286 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #286   +/-   ##
=======================================
  Coverage   79.62%   79.62%
=======================================
  Files          17       17
  Lines         216      216
  Branches        9        9
=======================================
  Hits          172      172
  Misses         42       42
  Partials        2        2

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 692d790...f6579c9. Read the comment docs.


Sheets functionality and usage description

What are the relevant tickets?
Related to https://github.com/mitodl/mitxonline/pull/1717
Description (What does it do?)
Adds documentation on how google sheets work and how to use it
How can this be tested?
Read README

@cachob can you review this, in addition to an software engineer?


Add stylelint

https://github.com/ntwb/awesome-stylelint
stylelint is to CSS as ESLint is to JS
Lots of plugins from a broad community, the aim of this awesome list is to have a central collection of all things stylelint.
By submitting this pull request I confirm I've read and complied with the below requirements.
Failure to properly do so will just result in the pull request being closed and everyone's time wasted. Please read it twice. Most people miss many things.

I have read and understood the contribution guidelines and the instructions for creating a list.
This pull request has a descriptive title. For example, Add Name of List, not Update readme.md or Add awesome list.
The entry in the Awesome list should include a short description about the list project/theme (not the list itself), and should be added at the bottom of the appropriate category.
The list I added

has been around for at least 30 days,
is the result of hard work and is the best I could possibly produce,
is a non-generated Markdown file in a GitHub repo,
is not a duplicate,
includes a succinct description of the project/theme at the top (example),
includes a project logo/illustration (whenever possible) at the top right of the readme (example). (The logo should link to the project website or any relevant website. The logo should be high-DPI, so set it to maximum half the width of the original image.),
only has awesome items with descriptions (Awesome lists are curations of the best, not of everything),
contains the awesome badge on the right side of the list heading,
has a Table of Contents section named Contents as the first section,
has an appropriate license (Meaning something like CC0, NOT a code licence like MIT, BSD, Apache, etc) (If you use a license badge, it should be SVG, not PNG),
has contribution guidelines,
doesn't include a Travis badge,
has been checked for inconsistent formatting, spelling, and grammar:

Each link description starts with an uppercase character and ends with a period.
Drop all the A / An prefixes in the descriptions.
Consistent naming. For example, Node.js, not NodeJS or node.js.

Please read it twice. Most people miss many things.

I would read it a couple of more times. Many things not adhering.


File reading seek with UInt64 number crash

Error:
#<Errno:0x9c7640 @message="Unable to seek: Invalid argument", @cause=nil, @callstack=CallStack(@callstack=[Pointer(Void)@0x471d47, Pointer(Void)@0x471cda, Pointer(Void)@0x471caa, Pointer(Void)@0x462cbe, Pointer(Void)@0x4a56d3, Pointer(Void)@0x4da54a, Pointer(Void)@0x469179, Pointer(Void)@0x4859e4, Pointer(Void)@0x4684d6, Pointer(Void).null], @backtrace=nil), @errno=22>

Code for crash:
file = File.new "./data/20gb.txt"
file.seek<PHONE_NUMBER>3709551615_u64

slice = Bytes.new 50000
file.read slice

puts String.new slice

How can I fix that?

Related to #3209

But...
# 20 gb
21474836480 == 20 * 1024 * 1024 * 1024

# you are passing
18446744073709551615

The number<PHONE_NUMBER>3709551615 is huge and it's way more than 20 gb so I think it's natural that the seek will fail. Or am I missing something?

The offset is passed as is to lseek, so it's unrelated to the size overflow. Errno is EINVAL which means that the resulting offset (depending on whence) is negative or beyond the file size, which appears to the case, here.

@asterite @ysbaddaden Right. Thanks for the work. Really need to calculate more the next time.


issue-827 expired artifacts handler [to review]

Fixes #827.

@carlspring , @sbespalov please review this pull. I'll resolve failing tests tomorrow. Functionality is done.

@fuss86 can you please also make a test for group and proxy repositories, which will check that maven-metadata.xml expired and updated?
It can probably be something similar to mocked RestArtifactResolver that we have in TestCaseWithMavenArtifactGenerationAndIndexing

@carlspring , @sbespalov please review this PR .
Couple of notes:

Old ChecksumCacheManager has been renamed to CalculatedChecksumCacheManager (feel free to propose better name)
I've introduced my own ChecksumCacheManager which I benefit from
I haven't done anything else in the previous  ChecksumCacheManager other than renaiming it (the same goes to the test class)
I've added 3 test cases: MavenMetadataExpirationMultipleGroupCaseTest, MavenMetadataExpirationProxyCaseTest, MavenMetadataExpirationSingleGroupCaseTest
I've rolled back all previous changes related with ParallelDownloadRemoteArtifactTest
Group repositories expired artifacts work as agreed (aggregation)

@carlspring , @sbespalov once again request for your review here ;-)

@sbespalov, cache related things are now rolled back

Thanks for all the hard work and the thorough discussions! :)


Use Into for &str vs. String

I heard there's a technique to accept both &str and String as an argument. With this we can avoid unneeded .to_string()s.

Now it is used by names::class_fullname.


Add balancer members after configs have been written

Fixes a regression in #14007 that affects the initial start of the
appliance which caused a 503 error when trying to access the UI.
https://bugzilla.redhat.com/show_bug.cgi?id=1422988
Because adding balancer members does a validation of the configuration
files and these files try to load the redirect files among others,
we need to add the balancers members after all configuration files have
been written by install_apache_proxy_config.
I wish I knew how to test this outside of an appliance.  ❓

cc @fbladilo (hopefully this fixes your container builds)

@jrafanie :+1:

Euwe backport details:
$ git log -1
commit f91838277dab716effa5a4fe8c46571076c7474f
Author: Nick Carboni<EMAIL_ADDRESS>Date:   Tue Mar 14 09:23:01 2017 -0400

    Merge pull request #14311 from jrafanie/add_balancer_members_after_all_configs_written

    Add balancer members after configs have been written
    (cherry picked from commit 5b167ab4748c17ff5c937bb2eba1aaa37fdba8c7)

    https://bugzilla.redhat.com/show_bug.cgi?id=1432463


Prepare release

still working through a very messy concepts file. I will resume work on this tomorrow.

I worked on the conceptlist because it was very much unmapped. If this is fit, I'll update the concept list in conception and then here.


setup.py should not attempt to install argparse on python >= 2.7

The argparse module is part of the python standard library beginning with python 2.7.  When run on a system with that version, setup.py still wants to download and install argparse despite that.  The install_requires list should probably be defined in an if statement earlier in the file to prevent that from happening.

I'm looking to correct this using environment markers such as below.  This works on 2.7 (in a virtualenv) but I see it still installs argparse due to another package requiring it apparently.  (it installs argparse 1.3.0 whereas below I ask for 1.2.2).
diff --git a/setup.py b/setup.py
index 39146ef..a145ff7 100644
--- a/setup.py
+++ b/setup.py
@@ -5,6 +5,8 @@ from distutils.command.sdist import sdist
import os.path
import subprocess
import glob
+import multiprocessing
+import sys
from setuptools import setup, find_packages
@@ -87,8 +89,9 @@ setup(
packages=find_packages(),
test_suite='nose.collector',
tests_require=['nose'],

install_requires=['fabric', 'PyYaml', 'argparse', 'stevedore', 'sphinx',

                 'pbr >= 0.10.7', 'six >= 1.9.0'],

setup_requires=['pbr==0.11.0'] + (['argparse<=1.2.2'] if sys.version_info[:2] == (2, 6) else []),
install_requires=['pbr==0.11.0', 'fabric', 'PyYaml', 'stevedore<1.4.0', 'sphinx']

                + (['argparse<=1.2.2'] if sys.version_info[:2] == (2, 6) else []),

scripts=['bin/calyptos'],
classifiers=[
'Development Status :: 5 - Production/Stable',

Fixed here:
e4b95c646bcd9f85338192aab86d735fa52d19e1


WIP: reduce use of xalan

Description
https://bz.apache.org/bugzilla/show_bug.cgi?id=66171

This is a POC - if the changes look like a valid way forward, I can put more effort in.
Some tests are failing and will be investigated
Probably need some new tests too
The ant build.xml needs changes - is Ant still used?

Motivation and Context

Xalan releases are infrequent.
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-34169 is not fixed

How Has This Been Tested?
unit tests
Screenshots (if appropriate):
Types of changes

Breaking change (fix or feature that would cause existing functionality to not work as expected)

Probably not a huge deal. Main worry is that some XPath expressions may not behave exactly the same as before. Another issue is that PropertiesBasedPrefixResolver and PropertiesBasedPrefixResolverForXpath2 are public classes that extend a xalan interface (PrefixResolver) - this POC removes the use of that xalan interface.
Checklist:

[x] My code follows the code style of this project.
[ ] I have updated the documentation accordingly.

Ant is still used to generate our docs. I think all xalan related stuff can be deleted safely.

Codecov Report

Merging #721 (9501013) into master (cbcf41f) will decrease coverage by 0.05%.
The diff coverage is 53.41%.

@@             Coverage Diff              @@
##             master     #721      +/-   ##
============================================
- Coverage     55.24%   55.18%   -0.06%
+ Complexity    10386    10377       -9
============================================
  Files          1062     1063       +1
  Lines         65777    65811      +34
  Branches       7536     7538       +2
============================================
- Hits          36337    36320      -17
- Misses        26839    26888      +49
- Partials       2601     2603       +2

Impacted Files
Coverage Δ

...a/org/apache/jmeter/extractor/XPath2Extractor.java
89.15% <ø> (ø)

...rg/apache/jmeter/functions/XPathFileContainer.java
87.09% <ø> (ø)

.../org/apache/jmeter/util/PrefixResolverDefault.java
10.71% <10.71%> (ø)

...r/util/PropertiesBasedPrefixResolverForXpath2.java
40.00% <11.11%> (-60.00%)
:arrow_down:

...che/jmeter/util/PropertiesBasedPrefixResolver.java
18.60% <11.76%> (-11.03%)
:arrow_down:

...rc/main/java/org/apache/jmeter/util/XPathUtil.java
76.19% <74.76%> (-0.85%)
:arrow_down:

...jmeter/protocol/http/control/HttpMirrorServer.java
41.41% <0.00%> (-1.02%)
:arrow_down:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cbcf41f...9501013. Read the comment docs.

@FSchumacher thanks for reviewing the changes. The build is now working. I've had to make a few changes to get a few existing tests to pass. Would you be able to review the test changes to see if I've made a change that is dangerous?

I raised https://github.com/JetBrains/lets-plot/issues/576 - I'm not sure if that blocks us from making progress here.

Thanks for your work so far on this. Hopefully I will get time to look more into this. So far it looks good to me.
In my tests, I added code to look at the actual used TransformerFactory and set the used spaces and line-breaks for xalan-like and saxon respectively. But that might really be a bit too much.

I raised JetBrains/lets-plot#576 - I'm not sure if that blocks us from making progress here.

That issue has been resolved and we could use the newer releases to get rid of xalan, when we update our dependencies to the next major version of lets-plot, which has changed the API locations. The changes on our side are a few lines of code.

I see Xalan Java team is not really welcoming contributions. For example, see single-line PRs that say without review, without merge, and even with "not wanted" replies: https://github.com/apache/xalan-java/pulls
So I think we should indeed drop the use of Xalan as they just do not care about the users.


move feature from %sqlrender  -> %sqlcmd snippets

we're going to deprecate %sqlrender, we should move the logic to %sqlcmd snippets
currently, this is how it works:
%%sql --save penguins
select * from penguins.csv limit 10

%%sql --save subset
select * from penguins where island is not null

cte = %sqlrender subset
print(cte)

prints:
WITH penguins AS (
select * from penguins.csv limit 10
)
select * from penguins where island is not null

Now, with the change, we should get the same when executing: %sqlcmd snippets subset
i.e.,
cte = %sqlcmd snippets subset
print(cte)

should print:
WITH penguins AS (
select * from penguins.csv limit 10
)
select * from penguins where island is not null

Acceptance Criteria:

Make %sqlcmd snippets {name} gives the same output as %sqlrender {name}
Add FutureWarning to %sqlrender {name} that includes a link to %sqlcmd snippets documentation
Update %sqlcmd snippets documentation under API Reference tab to include explanation and behavior of %sqlcmd snippets {name}
Add test functions for testing %sqlcmd snippets {name}

Add FutureWarning to %sqlrender {name} that includes a link to %sqlcmd snippets documentation

should also include the equivalent command. e.g., if user ran %sqlrender stuff, then the warning should tell the user to migrate to %sqlcmd snippets stuff

Okay. I will include that in the error message that the FutureWarning shows.


Missing Tags

Issue description
Missing c:storage_blocks and c:storage_blocks/{material} on all your storage blocks.
Steps to reproduce
No response
Minecraft version
1.21.1 (Latest)
Platform
Neoforge
Loader version
21.1.83
Fabric API version
NONE
Oritech version
0.12.1
Other relevant versions
No response
If a (crash)log is relevant for this issue, link it here: (It's almost always relevant)
No response

This goes for ingots, dusts, tiny dusts, etc. All of these should be using the common tags even if the materials are unique to your mod.

Huh, didn't even know about those tags. But I'll make the appropriate changes and include the tags in the next version. Ingots and dusts should already be using the correct tags I think. Are tiny dusts from other mods usually 2x2 or 3x3 recipes? If there's  a mismatch this could be an issue when both dust variants use the same tag.
Also oritech gems are pretty specific variants during ore pressing, does it make sense to include them in the gems tag? (No idea where this tag is usually used). Or would this tag be for something like fluxite?

Hi, just pushed tags for all of these. If I missed any please let me know. I skipped the tiniy dusts for now as I was unsure if they usually are 2x2 or 3x3 variants. Oritech uses 3x3 variants and I couldn't directly find any reference mods


Failed to build osg [Mac OS]

Hi all, i tried to install open scene graph library on my Mac machine, but i get this error:
./vcpkg install osg
The following packages will be built and installed:
osg[core]:x64-osx
Starting package 1/1: osg:x64-osx
Building package osg[core]:x64-osx...
-- Note: osg only supports dynamic library linkage. Building dynamic library.
-- Using cached /Users/roberto/vcpkg/downloads/openscenegraph-OpenSceneGraph-OpenSceneGraph-3.6.3.tar.gz
-- Using source at /Users/roberto/vcpkg/buildtrees/osg/src/raph-3.6.3-e8b171e32a
-- Configuring x64-osx-dbg
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:56 (message):
Command failed: /Users/roberto/vcpkg/downloads/tools/cmake-3.14.0-osx/cmake-3.14.0-Darwin-x86_64/CMake.app/Contents/bin/cmake;/Users/roberto/vcpkg/buildtrees/osg/src/raph-3.6.3-e8b171e32a;-DOSG_USE_UTF8_FILENAME=ON;-DCMAKE_MAKE_PROGRAM=/Users/roberto/vcpkg/downloads/tools/ninja-1.8.2-osx/ninja;-DCMAKE_SYSTEM_NAME=Darwin;-DBUILD_SHARED_LIBS=ON;-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=/Users/roberto/vcpkg/scripts/toolchains/osx.cmake;-DVCPKG_TARGET_TRIPLET=x64-osx;-DVCPKG_PLATFORM_TOOLSET=external;-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON;-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON;-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON;-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE;-DCMAKE_VERBOSE_MAKEFILE=ON;-DVCPKG_APPLOCAL_DEPS=OFF;-DCMAKE_TOOLCHAIN_FILE=/Users/roberto/vcpkg/scripts/buildsystems/vcpkg.cmake;-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON;-DVCPKG_CXX_FLAGS=;-DVCPKG_CXX_FLAGS_RELEASE=;-DVCPKG_CXX_FLAGS_DEBUG=;-DVCPKG_C_FLAGS=;-DVCPKG_C_FLAGS_RELEASE=;-DVCPKG_C_FLAGS_DEBUG=;-DVCPKG_CRT_LINKAGE=dynamic;-DVCPKG_LINKER_FLAGS=;-DCMAKE_INSTALL_LIBDIR:STRING=lib;-DCMAKE_INSTALL_BINDIR:STRING=bin;-G;Ninja;-DCMAKE_BUILD_TYPE=Debug;-DCMAKE_INSTALL_PREFIX=/Users/roberto/vcpkg/packages/osg_x64-osx/debug
Working Directory: /Users/roberto/vcpkg/buildtrees/osg/x64-osx-dbg
Error code: 1
See logs for more information:
/Users/roberto/vcpkg/buildtrees/osg/config-x64-osx-dbg-out.log
/Users/roberto/vcpkg/buildtrees/osg/config-x64-osx-dbg-err.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_configure_cmake.cmake:282 (vcpkg_execute_required_process)
ports/osg/portfile.cmake:21 (vcpkg_configure_cmake)
scripts/ports.cmake:71 (include)
Error: Building package osg:x64-osx failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: osg:x64-osx
Vcpkg version: 2018.11.23-unknownhash
Additionally, attach any relevant sections from the log files above.
Contents of  /Users/roberto/vcpkg/buildtrees/osg/config-x64-osx-dbg-out.log file :
-- The C compiler identification is AppleClang <IP_ADDRESS>10046
-- The CXX compiler identification is AppleClang <IP_ADDRESS>10046
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- broken
-- Configuring incomplete, errors occurred!
See also "/Users/roberto/vcpkg/buildtrees/osg/x64-osx-dbg/CMakeFiles/CMakeOutput.log".
See also "/Users/roberto/vcpkg/buildtrees/osg/x64-osx-dbg/CMakeFiles/CMakeError.log".

The same issue appened in Mint 19.1 fresh install
Building package osg[core]:x64-linux...
-- Note: osg only supports dynamic library linkage. Building dynamic library.
-- Downloading https://github.com/openscenegraph/OpenSceneGraph/archive/OpenSceneGraph-3.6.3.tar.gz...
-- Extracting source /home/roberto/vcpkg/downloads/openscenegraph-OpenSceneGraph-OpenSceneGraph-3.6.3.tar.gz
-- Applying patch collada.patch
-- Using source at /home/roberto/vcpkg/buildtrees/osg/src/raph-3.6.3-e8b171e32a
-- Configuring x64-linux-dbg
-- Configuring x64-linux-rel
-- Building x64-linux-dbg
CMake Error at scripts/cmake/vcpkg_build_cmake.cmake:202 (message):
Command failed: /home/roberto/vcpkg/downloads/tools/cmake-3.14.0-linux/cmake-3.14.0-Linux-x86_64/bin/cmake;--build;.;--config;Debug;--target;install;--;-v
Working Directory: /home/roberto/vcpkg/buildtrees/osg/x64-linux-dbg
See logs for more information:
/home/roberto/vcpkg/buildtrees/osg/install-x64-linux-dbg-out.log
Call Stack (most recent call first):
scripts/cmake/vcpkg_install_cmake.cmake:24 (vcpkg_build_cmake)
ports/osg/portfile.cmake:27 (vcpkg_install_cmake)
scripts/ports.cmake:71 (include)
Error: Building package osg:x64-linux failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with .\vcpkg update, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
Package: osg:x64-linux
Vcpkg version: 2018.11.23-unknownhash

Hi @coderc264, thanks for reporting this issue!
This issue should be fixed in some PRs. Please update vcpkg and rebuild osg.
Thanks.


Cursor can go "below" end of filtered file tree

In the file tree on the right, the cursor can go past the last item and down onto items that have been filtered using Ctrl+A, Ctrl+R, Ctrl+M, etc.
It probably shouldn't do this. Interestingly, there seems to be a maximum to how far it will let you down-cursor when things in the tree are hidden.

Thanks for the gif, this was very helpful. The problem is due to the fact that there are "hidden" elements that are not accounted for when the cursor is allowed to move. This means that the cursor can move as far as the size of the visible part of the tree. That is, if there are 10 elements, and 3 are hidden, the cursor should not be able to move beyond the 7th line... however, since the fact that hidden elements are ignored, the cursor will be allowed to move an additional 3 lines.

Thanks for the gif, this was very helpful.

No problem, @wagoodman. I'm assuming this will get triaged as a bug?

I did neglect to mention that in my explanation... Yup, definitely a bug, good find.


imBack button press appears as message from another user

The button press for imBack (and possibly postBack also) sends the user's id rather than user's name in the message.from.id field. Unsure which one is correct. Screen capture:

I'm not sure what you mean by "user's name"... the id is the only way we have to identify the user.

When we send a text message, we set the from.id field to 'user':
from: {id: 'user'},
But when we send the button ImBack message, we set it to the user's id:
from: {id: this.state.userId},
I believe this is causing the imBack message to be incorrectly identified as not from the user. Here's the code that makes this decision (from History.tsx):
<div className={ 'wc-message wc-message-from-' + (activity.from.id === 'user' ? 'me' : 'bot') }>

In this case the imBack message is actually doing the right thing. I introduced a regression when I did the big Activity refactoring. Fixing now.


Re-add a deprecated builder method to GetWalks operation

In version 1.2.0-RC1 we deleted a 'operation' method from the GetWalks Builder. This should be re-added and marked as deprecated.

Merged into develop.


add missing word "valid" in attestation gossip conditions

nitpick -- make sure only preventing duplicate senders if you've seen a valid attestation from that validator

+1 for nitpicks 😂

got me 🤣


Error after changing something on files

Repro steps:1.2.Action: azureFunctions.deployError type: EPERMError Message: EPERM: operation not permitted, scandir 'c:\Users\Ivan\Desktop\projects\importexternalsource\node_modules\@types'Version: 0.16.0OS: win32VS Code Version: 1.33.1

Likely a duplicate of https://github.com/Microsoft/vscode-azurefunctions/issues/1105


Move take_screenshot logic to runner

See https://flutter-review.googlesource.com/c/recipes/+/14400
We should ideally do the screenshot in the runner, but it's not clear if we can do that reliably with the way LUCI may or may not terminate our process.
We could listen for sigkill, but it's not clear whether that will give us enough time to launch additional processes to take/save a screenshot.
/cc @godofredoc @zanderso

Closing because the bug has not been updated for more than one year.
Feel free to reopen any bugs that you believe are important but please provide updates on how to resource the fix of the bug and what can be done to prevent the issue from not getting updates for another year.

@dnfield Is this issue still valid?

It looks like the recipe no longer has any screenshotting logic in it, so either this was fixed or is obsolete AFAICT.


IT-2616: Update documentation and update to LaTeX

The documentation was on RST, and now it's in LaTeX.
All sites are up to date; disclaimer added to Tucson Site.

Looks good.


Add themeSelector for shadow root use cases

If variables are attached to :root, they don't seem to correctly transcend a shadow root's boundaries. I hacked my local daisyUI installation to attach the variables to * (like Tailwind itself does), and that makes the variables accessible for all elements.

Any update on this? Can this be merged?

It will be merged in the next major version, alongside some improvements.
Estimated release time is 2~3 weeks from now.


Choose appropriate default algorithms for the given key types

The tools should not require that algorithms are explicity mentioned. Instead sign and encrypt should select suitable defaults if alg or enc are missing.

The tools should be able to select appropriate keys based on their use and the appropriate alg based on the related key's  alg property.

This may not always be possible, depending on the token and the key-store.


Update description of StatefulSet behavior on delete

StatefulSet deletion does not have any guarantees of orderly termination of pods as is suggested by the documentation. Prior to 1.11 kubectl did scale down StatefulSets before deleting them, but this went away after the removal of reapers. Ref discussion in #68627
This change updates the documentation to reflect this.

@janetkuo

/lgtm
/approve


Proposal: Create a website enabling users to connect to people all around the world to communicate on Klaytn

Project: Connect to stranger
Target:

Create an online space enable users to anonymous and fetterless communicate with other people.
Encourage cultural exchange and the exchange of ideas between individuals from many different countries.
Provides a safe and friendly platform to discuss diverse topics.
Main feature:
Friendly user interface: Simple design and easy to use, suitable for all ages.
Incognito mode: Users can choose a nickname and do not need to provide personal information.
Blockchain & Klaytn integration

Looks good. Approved to proceed.


Add the ability to draw text to a canvas

Adds two methods to the Canvas widget called write_text and measure_text in order to add text to a canvas with a stroke or fill.

Ready to merge

Dang, that wasn't the prettiest merge - I was on the struggle train for some reason and my commit history will live on forever on that one.
I think that is a great point you brought up about Pango, so I added a check similar to the cairo one.
For your point about measuring text, I think if we were requesting the font size (in pixels) then that would definitely be something for the Font widget. But, in this case we are directly measuring the text height and width so that we can draw things around it. I think that is consistent with Font/Text APIs for Gtk and Cocoa. Did you have another use case for someone wanting to measure the text size?

@danyeaw The most obvious use case I can think of is "how wide do I need to make this column to accomodate the text label I want to put in it".  Fundamentally, I don't care if I'm drawing on a canvas or putting a text label somewhere - I want to know how big it's going to be so it won't overlap with other content on the page.

@freakboy3742 OK, that sounds like a good use case. I'll start implementing this, and probably rename the method to measure_font. Next on my list after that is to complete the Canvas implementation for Cocoa that I started.

@danyeaw something like http://doc.qt.io/qt-5/qfontmetrics.html ?

@freakboy3742 Thanks so much for once again pushing me to make a better implementation, I really appreciate it! I have updated the API to move the measure method to the Font module.


Enable usage of nullptr for MSVC 16 and newer (MSVS 2010)

Support for nullptr was added with MS Visual Studio 2010 which has _MSC_VER == 1600.
This Microsoft document states that nullptr support was in fact introduced with 2010, the mapping to _MSC_VER was taken from here.

We are upgrading to pugixml 1.12.1 and I was surprised when I found out that this is missing.
Sadly branching on _MSC_VER is still necessary, on the one hand because older compilers supporting nullptr will continue to report __cpluplus == 199711L and alas, newer ones by default still do so.

This check was missing because PUGIXML_NULL was introduced for the benefit of compilers that warn on use of 0 as a null pointer, which MSVC thankfully doesn't. That said, this change seems perfectly good to have as well.


when you edit ranks in organization table, sorting the table will undo the changes

Describe the bug

Go to org table
Edit ordering
ctrl-enter to save changes
then try to sort ordering column by new rank - table resets and loses all of user's changes

To Reproduce
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
Please include your Water Quality Portal data query inputs from the Load
data tab:
If applicable, include code to reproduce the behavior:
library(TADAShiny)
TADAShiny::run_app()

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Session Info
Please include your session info:
sessionInfo()
#OR preferred:
devtools::session_info()

Additional context
Add any other context about the problem here.
Reminders for TADA contributors addressing this issue
New features should include all of the following work:

[ ] Create the function/code.

[ ] Document all code using comments to describe what is does.

[ ] Create tests in tests folder.

[ ] Create help file using roxygen2 above code.

[ ] Create working examples in help file (via roxygen2).

[ ] Add to appropriate vignette (or create new one).

This is the issue in the mod_overview.R page:
observeEvent(input$overview_orgtable_cell_edit, {
org_rank = data.frame(OrganizationIdentifier = mapdat$orgs$OrganizationIdentifier, Rank = as.numeric(input$overview_orgtable_cell_edit$value)) %>% dplyr::arrange(Rank)
tadat$orgs = org_rank$OrganizationIdentifier
})
The input$overview_orgtable_cell_edit needs to refresh the table: mapdat$orgs


Update libpng cmake wrapper for dynamic libraries

This is fairly straightforward. This adds the dynamic library as an option to the CMake wrapper for libpng. This makes building dynamic libraries on Linux and macOS (using a custom triplet) work better.

Thanks for the PR and sorry for the delay!


⚠ Samsung has degraded performance

In 7849fcf, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2925 ms

Resolved: Samsung performance has improved in 3ea246a.


Allow the developer to organize the schema of fancy pages, by using groupFields

In apostrophe-schema it states and I quote
"One lonnnnng scrolling list of fields is usually not user-friendly."
It seems that in the case of apostrophe-fancy-pages this remains an issue. Furthermore apostrophe-fancy-pages can get really long and complex given their extensive functionality based on apostrophe-schema.
Take the following module description as an example:
'branch':{
		          extend: 'apostrophe-fancy-page',
		          name: 'branch',
		          label: 'Branch',

		          addFields: [
		            {
		              name: 'branch_name',
		              type: 'string',
		              label: 'Branch Name',
		              required: true
		            },
		            {
		              name: 'phone',
		              type: 'string',
		              label: 'Phone',
		              required: true
		            },
		            {
		              name: 'address',
		              label: 'Address',
		              type: 'string',
		              required: true
		            },
		            {
		              name: 'city',
		              label: 'City',
		              type: 'string',
		              required: true
		            },
		            {
		              name: 'service',
		              label: 'Service',
		              type: 'boolean',
		              required: true
		            },
		            {
		              name: 'testDrive',
		              label: 'Test Drive',
		              type: 'boolean',
		              required: true
		            },
		            {
		              name: 'zip',
		              label: 'Zip',
		              type: 'string',
		              required: true
		            }
		            ,
		            {
		              name: 'lat',
		              label: 'Latitude',
		              type: 'string'
		            },
		            {
		              name: 'lng',
		              label: 'Longitude',
		              type: 'string'
		            },
		            {
		              name: 'email',
		              label: 'Contact Email',
		              type: 'string'
		            },
		            {
		              name: 'serviceEmail',
		              label: 'Service Email',
		              type: 'string'
		            },

		            {
		              name: 'testDriveEmail',
		              label: 'Test Drive Email',
		              type: 'string'
		            },

		            {
		              name: 'usedCarEmail',
		              label: 'Used Car Email',
		              type: 'string'
		            }
		          ],

		          groupFields:[
		            {
		              name:'settings',
		              label: 'Settings',
		              icon: 'settings',
		              fields:[
		                'published','notOrphan','seoDescription','tags','slug'
		              ]
		            },
		            {
		              name:'general',
		              label: 'General',
		              icon: 'general',
		              fields:['type','branch_name','phone','address','city','state','zip','lat','lng','service','testDrive']
		            },
		            {
		              name:'emails',
		              label: 'E-Mails',
		              icon:'emails',
		              fields:['email','serviceEmail','testDriveEmail','usedCarEmail']
		            }

		          ]
		        },

We get the following result

Which is a double tabbed grouping.

 We should figure this out for 0.6 at least.
The tricky bit is when people switch page types. Instead of just switching
out one div that contains all of the custom settings for the page type,
we'll have to rejigger everything, without losing any data.
On Tue, Mar 10, 2015 at 3:46 PM, Fotis Paraskevopoulos <
<EMAIL_ADDRESS>wrote:

In apostrophe-schema it states and I quote
"One lonnnnng scrolling list of fields is usually not user-friendly."
It seems that in the case of apostrophe-fancy-pages this remains an issue.
Furthermore apostrophe-fancy-pages can get really long and complex given
their extensive functionality based on apostrophe-schema.
Take the following module description as an example:
'branch':{
extend: 'apostrophe-fancy-page',
name: 'branch',
label: 'Branch',
              addFields: [
                {
                  name: 'branch_name',
                  type: 'string',
                  label: 'Branch Name',
                  required: true
                },
                {
                  name: 'phone',
                  type: 'string',
                  label: 'Phone',
                  required: true
                },
                {
                  name: 'address',
                  label: 'Address',
                  type: 'string',
                  required: true
                },
                {
                  name: 'city',
                  label: 'City',
                  type: 'string',
                  required: true
                },
                {
                  name: 'service',
                  label: 'Service',
                  type: 'boolean',
                  required: true
                },
                {
                  name: 'testDrive',
                  label: 'Test Drive',
                  type: 'boolean',
                  required: true
                },
                {
                  name: 'zip',
                  label: 'Zip',
                  type: 'string',
                  required: true
                }
                ,
                {
                  name: 'lat',
                  label: 'Latitude',
                  type: 'string'
                },
                {
                  name: 'lng',
                  label: 'Longitude',
                  type: 'string'
                },
                {
                  name: 'email',
                  label: 'Contact Email',
                  type: 'string'
                },
                {
                  name: 'serviceEmail',
                  label: 'Service Email',
                  type: 'string'
                },

                {
                  name: 'testDriveEmail',
                  label: 'Test Drive Email',
                  type: 'string'
                },

                {
                  name: 'usedCarEmail',
                  label: 'Used Car Email',
                  type: 'string'
                }
              ],

              groupFields:[
                {
                  name:'settings',
                  label: 'Settings',
                  icon: 'settings',
                  fields:[
                    'published','notOrphan','seoDescription','tags','slug'
                  ]
                },
                {
                  name:'general',
                  label: 'General',
                  icon: 'general',
                  fields:['type','branch_name','phone','address','city','state','zip','lat','lng','service','testDrive']
                },
                {
                  name:'emails',
                  label: 'E-Mails',
                  icon:'emails',
                  fields:['email','serviceEmail','testDriveEmail','usedCarEmail']
                }

              ]
            },

We get the following result
[image: apostrophe_sandbox_ _home]
https://cloud.githubusercontent.com/assets/210779/6572370/41e32d9a-c71a-11e4-8fcc-dd97a792dc28.png
Which is a double tabbed grouping.
—
Reply to this email directly or view it on GitHub
https://github.com/punkave/apostrophe-fancy-page/issues/1.

--
*THOMAS BOUTELL, *DEV & OPS
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

(It would probably be much less extreme a sport if we continued to insist
that the custom fields be in a distinct set of tabs from the generic stuff.
I think that would be acceptable and wise.)
 We should figure this out for 0.6 at least.
The tricky bit is when people switch page types. Instead of just switching
out one div that contains all of the custom settings for the page type,
we'll have to rejigger everything, without losing any data.
On Tue, Mar 10, 2015 at 3:46 PM, Fotis Paraskevopoulos <
<EMAIL_ADDRESS>wrote:

In apostrophe-schema it states and I quote
"One lonnnnng scrolling list of fields is usually not user-friendly."
It seems that in the case of apostrophe-fancy-pages this remains an
issue. Furthermore apostrophe-fancy-pages can get really long and complex
given their extensive functionality based on apostrophe-schema.
Take the following module description as an example:
'branch':{
extend: 'apostrophe-fancy-page',
name: 'branch',
label: 'Branch',
              addFields: [
                {
                  name: 'branch_name',
                  type: 'string',
                  label: 'Branch Name',
                  required: true
                },
                {
                  name: 'phone',
                  type: 'string',
                  label: 'Phone',
                  required: true
                },
                {
                  name: 'address',
                  label: 'Address',
                  type: 'string',
                  required: true
                },
                {
                  name: 'city',
                  label: 'City',
                  type: 'string',
                  required: true
                },
                {
                  name: 'service',
                  label: 'Service',
                  type: 'boolean',
                  required: true
                },
                {
                  name: 'testDrive',
                  label: 'Test Drive',
                  type: 'boolean',
                  required: true
                },
                {
                  name: 'zip',
                  label: 'Zip',
                  type: 'string',
                  required: true
                }
                ,
                {
                  name: 'lat',
                  label: 'Latitude',
                  type: 'string'
                },
                {
                  name: 'lng',
                  label: 'Longitude',
                  type: 'string'
                },
                {
                  name: 'email',
                  label: 'Contact Email',
                  type: 'string'
                },
                {
                  name: 'serviceEmail',
                  label: 'Service Email',
                  type: 'string'
                },

                {
                  name: 'testDriveEmail',
                  label: 'Test Drive Email',
                  type: 'string'
                },

                {
                  name: 'usedCarEmail',
                  label: 'Used Car Email',
                  type: 'string'
                }
              ],

              groupFields:[
                {
                  name:'settings',
                  label: 'Settings',
                  icon: 'settings',
                  fields:[
                    'published','notOrphan','seoDescription','tags','slug'
                  ]
                },
                {
                  name:'general',
                  label: 'General',
                  icon: 'general',
                  fields:['type','branch_name','phone','address','city','state','zip','lat','lng','service','testDrive']
                },
                {
                  name:'emails',
                  label: 'E-Mails',
                  icon:'emails',
                  fields:['email','serviceEmail','testDriveEmail','usedCarEmail']
                }

              ]
            },

We get the following result
[image: apostrophe_sandbox_ _home]
https://cloud.githubusercontent.com/assets/210779/6572370/41e32d9a-c71a-11e4-8fcc-dd97a792dc28.png
Which is a double tabbed grouping.
—
Reply to this email directly or view it on GitHub
https://github.com/punkave/apostrophe-fancy-page/issues/1.

--
*THOMAS BOUTELL, *DEV & OPS
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

--
*THOMAS BOUTELL, *DEV & OPS
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com

I understand why it would be a hassle, honestly I feel your pain (I saw the code :) ).
At least consider some functionality to organize better the fields within that "Settings" tab.  Like I said "apostrophe-fancy-pages" are a wonder, and this could be discouraging.

Thanks for the encouragement Fotis!
On Tue, Mar 10, 2015 at 4:06 PM, Fotis Paraskevopoulos <
<EMAIL_ADDRESS>wrote:

I understand why it would be a hassle, honestly I feel your pain (I saw
the code :) ).
At least consider some functionality to organize better the fields within
that "Settings" tab. Like I said "apostrophe-fancy-pages" are a wonder, and
this could be discouraging.
—
Reply to this email directly or view it on GitHub
https://github.com/punkave/apostrophe-fancy-page/issues/1#issuecomment-78136558
.

--
*THOMAS BOUTELL, *DEV & OPS
P'UNK AVENUE |<PHONE_NUMBER>  |  punkave.com


improve hello-minikube page for dashboard

fix #28639
clarify dashboard command to avoid confusing beginners.

@jimangel @zparnold Could you please review this PR? Thanks a lot!

@celestehorgan Thanks for the review. I've updated the doc with your comment.

/lgtm
/approve


Fix Rx memory leak in StorIOContentResolver

Closes #359, and little refactoring (of course :).
@nikitin-da PTAL, my fault :(

Oh, no. Delete emission =(

retest this please

Looks like it's emulator problem, I'll restart it later.
4 июня 2015 г. 8:03 AM пользователь "Dmitrii Nikitin" <
<EMAIL_ADDRESS>написал:

retest this please
—
Reply to this email directly or view it on GitHub
https://github.com/pushtorefresh/storio/pull/360#issuecomment-108723111.

Hm, no it isn't. Delete emission test was failed first time

retest this please

Ok)

Cool! Did LeakCanary help you?

Nope, just reviewed code and saw epic fail.
4 июня 2015 г. 8:43 AM пользователь "Dmitrii Nikitin" <
<EMAIL_ADDRESS>написал:

Cool! Did LeakCanary help you?
—
Reply to this email directly or view it on GitHub
https://github.com/pushtorefresh/storio/pull/360#issuecomment-108735026.


How to implement lazy loading / infinite scrolling in Builder's recordList component?

I'm using Builder Plugin and it's recordList component to display the data list. While pagination works fine, I wanted to implement load more to implement pagination which would fetch data and partial using october's ajax handler and append below the list.

Hello,
I could do this with some javascript :
function formPaginate() {
if($('form[data-request]').length) {

	$('.pagination a').off().on('click change', function(e) {
		e.preventDefault();
		var form = $('form[data-request!=""]');
		var request = form.attr('data-request');
		var partials = jQuery.parseJSON('{ '+form.attr('data-request-update')+' }');
		var page = $(this).attr('href').split('/').pop().trim();

		// On effectue la requête ajax
		form.request(request, {
			data: { 'page': page },
			update: partials,
		});

	});

}
}

The idea behind this code is that when you have a form with data-request attribute, it means it's an ajax form, so, when you click on your pagination link, you prevent the url redirection, you extract the page number from that URL, and you trigger the ajax request by passing the page argument.
Hope that help


Add GitLeaks Github Action

Add GitLeaks Github Action
https://github.com/zricethezav/gitleaks/wiki/Scanning

I removed the stalebot late, we probably still want this


Floating number

Is there any reason not to use:
givenNumber = parseFloat(this.element.innerHTML);
instead of:
givenNumber = parseInt(this.element.innerHTML);
The parseInt just seems to limit the usability of the plugin.

Hi @sorenbryder,
It's been a while since I worked on this.. Or used it myself.
But yes, I guess it could use parseFloat instead of parseInt. Updated it


APPS-1215 Split Storage into LocalStorage and S3Storage

And save s3's bucket in the S3Storage (not as part of path) to simplify business logic

Codecov Report
Attention: Patch coverage is 16.94915% with 49 lines in your changes missing coverage. Please review.

Project coverage is 37.89%. Comparing base (5079b62) to head (2a6a1be).

Files with missing lines
Patch %
Lines

pkg/dto/storage.go
14.63%
35 Missing :warning:

pkg/dto/backup_routine.go
0.00%
7 Missing :warning:

pkg/dto/base_dto.go
0.00%
6 Missing :warning:

pkg/service/restore_data.go
75.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##               v2     #233      +/-   ##
==========================================
- Coverage   39.00%   37.89%   -1.12%
==========================================
  Files          30       30
  Lines        1564     1594      +30
==========================================
- Hits          610      604       -6
- Misses        876      912      +36
  Partials       78       78

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Sunstone: batch chmod operation is not available.

Author Name: Carlos Martín (Carlos Martín)
Original Redmine Issue: 1885, https://dev.opennebula.org/issues/1885
Original Date: 2013-04-08

None

It should be done now


Specifying exact output resolutions may cause distortion and incorrect bounds

how does one specify exact output resolutions?
image size 1860x900
the percentages don't always match up
16.67% Wide310x150Logo.scale-100.png8  (outputs desired resolution)
66.67% Wide310x150Logo.scale-400.png8 (outputs 1240 x 599 resolution)
how can i use something like this
310px x 150px Wide310x150Logo.scale-100.png8
1240px x 600px Wide310x150Logo.scale-100.png8

@AVOlight,

It would be helpful if you could provide a sample PSD that includes an 1860x900 layer that does not output correctly at 66.67%.
As a test, try nudging the layer up/down so that it lines up with a whole pixel (if doesn't currently)
The syntax you described for exact pixels should work.  Are you suggesting that those last two layer names you listed are exporting at full scale?

@mcilroyc Thank you for checking on this
Example.zip

@AVOlight, thanks for your example file.  When I run it, see:

The "a" file output correctly at 1240x600
The "b" file however looks very bad; it is 1180 × 512 and misshapen.

Can you confirm your same results?  If so, I'd like to rename this bug to track the "b" scenario (explicit height/width scaling) here.

@mcilroyc yes
i would very much like the ability to specify exact image size in pixels
this will save me so much time compared to using "Save for Web" and setting image size individually
Thank You

The resolution of this file is set to 118.11, I'm guessing that we handle this incorrectly and assume 72 ppi

sorry i don't know, i thought ppi only matters for printing
tried changing it to check, but still same result

@AVOlight please try with latest 2015.5 release.  This should be resolved.  Please confirm.  Thanks.

Closing as fixed.  @AVOlight Please re-open if you are still seeing this.


Admob not working for iOS

I am trying to create interstitial ads in unity for iOS platform. I have followed coding sample provided in GoogleMobileAdsDemoScript file, but unfortunately ads are not produced. I just see a message on console:
 To get test ads on this device, call: request.testDevices = @[@"b9ade3b76dd9d16ccc6c61b78bf7d427"];
I have used following code in unity:
        private void RequestInterstitial()
      {
	print ("Requesting AD");

	#if UNITY_EDITOR
	string adUnitId = "unused";
	#elif UNITY_ANDROID
	string adUnitId = "INSERT_ANDROID_INTERSTITIAL_AD_UNIT_ID_HERE";
	#elif UNITY_IPHONE
	string adUnitId = "ca-app-pub-3113890509031570/1543899246";
	#else
	string adUnitId = "unexpected_platform";
	#endif

	// Create an interstitial.
	interstitial = new InterstitialAd(adUnitId);
	// Register for ad events.
	interstitial.AdLoaded += HandleInterstitialLoaded;
	interstitial.AdFailedToLoad += HandleInterstitialFailedToLoad;
	interstitial.AdOpened += HandleInterstitialOpened;
	interstitial.AdClosing += HandleInterstitialClosing;
	interstitial.AdClosed += HandleInterstitialClosed;
	interstitial.AdLeftApplication += HandleInterstitialLeftApplication;
	// Load an interstitial ad.
	interstitial.LoadAd(createAdRequest());
}

// Returns an ad request with custom ad targeting.
private AdRequest createAdRequest()
{
	return new AdRequest.Builder ()
		.AddTestDevice(AdRequest.TestDeviceSimulator)
		.AddTestDevice("my device ID")
			.AddKeyword("game")
			.AddExtra("color_bg", "454545")
			.Build();

}

#region Interstitial callback handlers

public void HandleInterstitialLoaded(object sender, EventArgs args)
{
	print("HandleInterstitialLoaded event received.");
	interstitial.Show();
}

public void HandleInterstitialFailedToLoad(object sender, AdFailedToLoadEventArgs args)
{
	print("*********** HandleInterstitialFailedToLoad event received with message: " + args.Message);
}

public void HandleInterstitialOpened(object sender, EventArgs args)
{
	print("HandleInterstitialOpened event received");
}

void HandleInterstitialClosing(object sender, EventArgs args)
{
	print("HandleInterstitialClosing event received");
}

public void HandleInterstitialClosed(object sender, EventArgs args)
{
	print("HandleInterstitialClosed event received");
}

public void HandleInterstitialLeftApplication(object sender, EventArgs args)
{
	print("HandleInterstitialLeftApplication event received");
}

#endregion

Please help me in resolving this issue. Thanks.

Closing due to non-response. If this is still an issue feel free to reach to the support group at https://groups.google.com/forum/#!forum/google-admob-ads-sdk.

Ram


made changes related to ois v1.3

operation is optional in OIS v1.3.0 See here.

@metobom So we have a little issue here. OIS v1.3 is not the latest release of OIS since v0.10 of Airnode has not been released. In fact OIS v1.3 cannot be published until then.
I will fix all this for you after I merge this PR today or tomorrow. In the future (because no one told you yet), when a new version of OIS or Airnode is needed please let me know and I will set it up for you. After it gets set up, then you can update it. Sorry no one informed you of this process.


Laravel 5.5.* Support

Laravel 5.5.* Support.
System details

Operating System
PHP Version: 7.0.0+
Laravel Version: 5.5.*
Laravel-DataTables Version: ~8.0

Having same issue

It's supported on v3.x. You can also use use a single installer package if you are interested https://github.com/yajra/datatables to lessen the entries on your composer. Thanks!


Adding AstroPT & Chat to Reseach

for astrollama & astrollama-chat let's use astrollama-1 and astrollama-1-chat

What specifically did you have in mind?

@Smith42 @pranavktrpl how does horizon feel guys? what's the best way to show our research?

[Bumping this one]

Won't have much time for this until Sept, so maybe could just do the papers for now? Once we have some HF/etc project playgrounds on the go we can add them in too


Is this project still maintained?

It seems the last actual code changes were some years ago and Travis CI is not setup.
Is anyone still maintaining it?

i am trying to maintain it here https://github.com/RamyRais/multi-nprogress

i am trying to maintain it here RamyRais/multi-nprogress

Sounds good. Are you open for new maintainers and are there plans to move this to an org account?

yes your are free to join. Currently i don't have any plan or any organisation account. I am just willing to solve issue found by others.

@RamyRais Is the library published npm under a new name?

@hozefaj yes it is, this is the link.
if you find issues or you want to solve some with a pull request, feel free.

@RamyRais if it's being maintained, why not make it into a standalone project, with its own branding README.md and everything?

The project is still active, check out https://github.com/rstacruz/nprogress/pull/218


[Error]: While selfhosting i am getting this error

Unhandled Rejection at: Promise  Promise {
  <rejected> TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
      at GatewayClient.exports.default (./dist/src/utils/commandHandler.js:43:33)
      at runMicrotasks (<anonymous>)
      at processTicksAndRejections (node:internal/process/task_queues:93:5)
}  reason:  TypeError: object null is not iterable (cannot read property Symbol(Symbol.iterator))
    at GatewayClient.exports.default (./dist/src/utils/commandHandler.js:43:33)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)

Any way to resolve it? I got the priviliged intents but idk why this is happening

Please, join the support server: https://discord.gg/g4MfV6N

Please, join the support server: https://discord.gg/g4MfV6N


Renaming in docs

Applies renaming for new branding in docs. It is suggested to review this PR commit-by-commit.

@jcarpanelli thanks so much for reviewing! Ready for another round.


fix(youtube): premiere badge is too low contrast

🔧 What does this fix? 🔧
the premiere badge to indicate a video is a premiere was colored with light text color and light accent color it's now a dark text color
🗒 Checklist 🗒

[x] I have read and followed Catppuccin's contributing guidelines.
[x] I have updated the version appropriately in the ==UserStyle== header of the catppuccin.user.css file.

Unable to reproduce? Where are you seeing this?

@uncenter i saw it on a premiere video in my recommended
before:

after:

Pretty sure I fixed this in #897 but I can't find a premiere video on my home page to test with. Are you sure you were up to date before making this PR?

@uncenter dang i think i might have been slightly out of date the version i was on was before that commit

No worries ahah.


Swashbuckle assets need compression and minification

Javascript and css assets, that are part of SwaggerUI need minification and compression (as in Swashbuckle.Core.5.2.1).
E.g.

GET /swagger/ui/swagger-ui-js   - 1.5Mb Unminified uncompressed

Downloading 1.5 Mb of javascript everytime when opening Swagger UI is not the best experience.
Also as it is extension-less asset - it not being cached properly by client browsers.
I tried to add GZip compression using Global filter in webapi pipeline, but seems that Swashbuckle somehow ignores global setup.
It'd be nice to enable some way to configure filters for /swagger route so we can protect this UI it with basic auth, add compression and modify some behaviour if needed (main idea is to protect UI with auth and add compression to outputs)

I too have this problem, it makes this version of swashbuckle nearly unusable and should be a pretty minor change to use minified version.

I wrote a middleware that minifies everything what comes from /swagger url but it needs some extra work to function properly - somehow it messes up with api responses in SwaggerUI

@centur - I was going to suggest a DelegatingHandler (see http://www.asp.net/web-api/overview/advanced/http-message-handlers) for intercepting the swagger request and applying additional customization (authorization, compression etc.). Of course if you're using OWIN, you could achieve the same result with middleware.
@daward - For the next release, I'll update the swagger-ui index.html served up by SB to reference the minified version of swagger-ui.js. In the meantime though, you could achieve the same thing by injecting your own version (copied from https://github.com/domaindrivendev/Swashbuckle/blob/v5.2.1/Swashbuckle.Core/SwaggerUi/CustomAssets/index.html) with the modified reference. See the readme for details - https://github.com/domaindrivendev/Swashbuckle#provide-your-own-index-file.
On a final note, I switched to extension-less URL's to avoid the need for setting runAllManagedModulesForAllRequests and hence the associated perf. concerns. I hadn't anticipated that browsers would not cache assets as a result and that's an unfortunate side effect.
It's worth noting that extensionless URL's will not be used in the next major version, targeting AspNet 6.0.

Just in case anyone wants it now - here is the simple gzip middleware that compresses js and css on the fly (OWIN based)

Using minified swagger-ui in v5.2.2 - now available


Optimizing scheduling of build pods with a custom scheduler

This issue is about how to optimize the scheduling of BinderHub specific Build Pods (BPs).
Out of scope of this issue is the discussion on how to schedule the user pods, which could be done with image locality (ImageLocalityPriority configuration) in mind.
Scheduling goals

Historic repo build node locality (Historic locality)
It is believed that often a builder pod rebuilding a repo would work faster if it had previously built a repository even though the repo has changed, for example in the README.md file of the repo. So, we want to schedule build pods on nodes where it has previously been scheduled if possible.
Not blocking scale down in autoscaling clusters
We would also like to avoid schedule pods on nodes that may want to scale down.

What is our desired scheduling practice?
We need to answer how we actually want the build-pods to schedule, it is not a obvious way to do it is typically hard to optimize both for performance and auto-scaling viability for example.
Boilerplate desired scheduling practice
I'll now provide a boilerplate idea to start out from on how to schedule the build pods.

We make the scheduler informed about if this is an autoscaling cluster or not. If it is, we greatly reduce the score of almost empty nodes that we could scale down.
We schedule the build pod on nodes with history of building this.

Technical solution implementation details
We must utilize a non-default scheduler. We could use the default kube-scheduler binary and customize its behavior through configuration, or we could make our own. I think we add too much complexity if we are to make our own though, even though making your own scheduler is certainly possible.

We utilize a custom scheduler, but like z2jh's scheduler we deploy a official kube-scheduler binary with a customized configuration that we can reference from the build pods specification using the spec.schedulerName field.

[spec.schedulerName] If specified, the pod will be dispatched by specified scheduler. If not specified, the pod will be dispatched by default scheduler. --- Kubernetes PodSpec documentation.

We customize the kube-scheduler binary, just like in z2jh through a provided config, but we try to use node annotations somehow. For example, we could make the build pod annotate the node it runs on with the repo it attempts to build, and then later the scheduler can attempt to schedule on this repo.

We make the BinderHub builder pod annotate the node by communicating with the k8s API. To allow the BinderHub to communicate like this, it will require some RBAC details setup, for example like the z2jh's user-schedulers RBAC. It will need a ServiceAccount, a ClusterRole, and a ClusterRoleBinding, where the ClusterRole will define it is should be allowed to read and write annotations on nodes. For an example of a pod communicating with the k8s API, we can learn relevant parts from the z2jh's image-awaiter which also communicates with the k8s API.

We make the BinderHub image-cleaner binary also cleanup associated node annotations along with it cleaning nodes, which would also require associated RBAC permissions like the build pods would need to annotate in the first place.

Knowledge reference
kube-scheduler configuration
https://kubernetes.io/docs/concepts/scheduling/kube-scheduler/#kube-scheduler-implementation
kube-scheduler source code
https://github.com/kubernetes/kubernetes/tree/master/pkg/scheduler
Kubernetes scheduling basics
From this video, you should pick up the role of a scheduler, and that the default binary that can act as a scheduler can consider predicates (aka. filtering) and priorities (aka. scoring).

Kubernetes scheduling deep dives

May 2018
Dec 2018
May 2019:

Past related discussion
https://github.com/jupyterhub/binderhub/issues/854

@betatim and I were speaking a lot about this, and these are my notes that can help us implement something.
Notes
Implementation idea (v1)

Tie a (Cluster)Role using a (Cluster)RoleBinding to the existing ServiceAccount mounted by the BinderHub pod in order to be allowed to use the k8s api to inspect either the nodes or DinD DaemonSet pods or both to figure out the kubernetes.io/hostname label of all the nodes that DinD pods have scheduled to.

Use these k8s api permissions to figure out the available kubernetes.io/hostname label values on the nodes where dind daemonset pods reside and save them to a in-memory list. This needs to be done when a build pod is to be created.
Throttle these kinds of request to the k8s api with about 30-60 seconds and fallback to a cache for throttled requests. The throttle timescale should relate to the time it takes to autoscale up and down.

Implement a Rendezvous hashing algorithm that makes a deterministic allocation of repo names to nodes where a build pod should be recommended to schedule through a preferred node affinity.

Verify its behavior, and unit test it. Hopefully it can be done so that we have X nodes, and adding one would take a fraction of the pods and let the other remain on without switching to a new node.

Add a step in the process to create the build pod where two different preferred node label affinities are added weighted to favor one primary and having the other as a backup.

Algorithm pro / cons

Stateless!!!
Distributes build pods of different repositories at random across nodes, which is good and bad
It cannot guarantee historic locality if it autoscales up, but it does a good job without keeping state about where repos has been built in the past.

Future improvements

It could keep build state in memory and fallback to rendezvouz hashing.
It could do above, but also save it in the node labels (which means to use k8s etcd key value store indirectly) or somewhere else persistent. While this may be a slight improvement in result, it also adds complexity to our codebase and may not be considered an improvement.

Standalone points

Container's requested CPU will be available for other containers on a node if they managed to be scheduled there, it is not blocking. In the k8s docs we can read about that they correspond to a docker command line flag --cpu-shares.

We want to make the DinD pods utilize as much CPU as possible without taking it all from the user pods, this can be done in two ways as I see it. Either with a low resource request and no limit, then it could end up grabbing as much CPU as another user for example but could be ending up squeezed down to little CPU if it competes with for example 20 other users for 100% CPU. It could also be done with a high cpu request and limited limit, then it would force away CPU from the users but leave some for them by limiting its CPU request to a portion of the node's CPU. I think the best option is to request about the same as a user or two and let it be unlimited. This way, it will at worst steal CPU power from users with peaking CPU load, but at best allow for full utilization of the CPU cores while not hogging all of the CPU for the users. Overall, I think best practice should be to have dind pods with similar requests to the user pods for CPU, low and umlimited.

build pods scheduling

historic locality has very high priority
LeastUsed
anti build pod affinity
Questions:

Blocking CPU or not?

BP always 0 req
DinD low req if blocking, and like 20% if not

Can we use labels and affinity straight up?

It was great discussing this and seeing how we started with a fairly complicated idea like "write a custom scheduler" and now have a much simpler solution!
There is another implementation of rendezvous hashing here.
To get the possible values of the kubernetes.io/hostname label it seems we can describe each of the DIND pods in the daemonset. Their Node field contains a value that is (on our GKE and OVH clusters) the same as the one used in the label. This means that to compute the list of possible node names we get describe the daemonset to get the Selector (e.g. name=ovh-dind), select pods with that (kubectl get pods -l name=ovh-dind) then inspect the Node field of each pod we found. This is nice because we don't need to inspect the nodes themselves so we don't need a cluster level role.
The following would be my suggestion for how to split this into several steps that can be tackled individually:

create a utility class that can be used to get a ranked list of nodes for a given repository URL. This utility class could use a dummy algorithm (random sorting) to figure out all the mechanics of delivering the right information to the class (for example: is the URL of the repo available when we schedule the build pod?)
(depends on 1.) create an implementation of the utility class that uses rendezvous hashing to compute the ranked list
create a function that obtains the list of possible node names and uses the at_most_every decorator from health.py to throttle API calls
combine all previous steps together to assign a preferred node affinity to each build pod. This behaviour should be configurable via a config option and off by default. The current behaviour should be the default.

What do you think? And do you want to tackle one of these already?

@betatim I think this may be quite fun to implement, but I have a long list of things to work on already so I figure I'll leave this to you :)
I'd be very happy to review whenever work is done and continue the discussion on implementation aspects.

I've started work on 1 (and a little bit of 2). PR coming soon.

I think #949 and follow ups implemented this so I'll close this. Maybe we can make new issues with some of the possible improvements/ideas we had beyond what is implemented.


Improve reusability of DependencyLoader and DependencyFinder

DependencyLoader and DependencyFinder should allow django to reuse templates and static files of third party apps that are not loaded in settings.INSTALLED_APPS.
Because the original code was tied to openwisp-controller, the two classes now don't make much sense because they will go to look into openwisp_utils.__dependencies__.
What about creating a new setting like EXTENDED_APPS?
In the test suite you could add a dependency to django-netjsonconfig, then set in the test project:
EXTENDED_APPS = ['django-netjsonconfig']

Then you could write 2 tests in which you just instantiate the 2 classes and check the list they produce is correct.

#2 Fixes this


HDDS-4223. Optimize the construction method of OzoneVolume and OzoneBucket.

What changes were proposed in this pull request?
Optimize the construction method of OzoneVolume and OzoneBucket.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4223

@sumitagrawl can you please take a look as well?


Support for hdbs files

I've been building Zendesk apps, and they use hdbs as the extension for handlebars. I would be willing to add it if you don't see any issue.

I'm not aware of any conflicts this would create. :)

Well, there is a pull request for it when you got the time.


Only run the CHANGELOG bot against master

Given our workflow, we never use it for anything other than PRs
that merge into master. Given that, running on pushes to other
branches is wasteful.
This commit turns off for anything that isn't master.

Thanks for the review @philix!


mcap merge changed channel schema from foxglove.RawImage to foxglove.SceneUpdate

Description

Version: Mcap cli version: brew info mcap: ==> mcap: stable 0.0.26 (bottled), HEAD. The mcap file was recorded with python mcap 1.0.1
Platform:  mcap cli run on macOS 12.6.1, recording done on Ubuntu 24

mcap merge is changing the schema of a particular channel in my mcap files.
Steps To Reproduce
I've taken a small mcap recording (attached at the bottom) - just one message long so that the attachment will easily fit in a GitHub issue attachment. Here I use mcap info and we see channel depth_1 has schema foxglove.RawImage:
❯ mcap info recording000001.mcap
library: python mcap 1.0.1
profile:
messages: 7
duration: 0s
start: 54.911
end: 54.911
compression:
	lz4: [3/3 chunks] [2.12 MiB/519.75 KiB (76.10%)]
channels:
  	(1) bot_entity     1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
  	(2) origin_entity  1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
  	(3) depth_1        1 msgs (+Inf Hz)   : foxglove.RawImage [jsonschema]
  	(4) depth_grid_1   1 msgs (+Inf Hz)   : foxglove.Grid [jsonschema]
  	(5) at_level_og_1  1 msgs (+Inf Hz)   : foxglove.PointCloud [jsonschema]
  	(6) overhead_og_1  1 msgs (+Inf Hz)   : foxglove.PointCloud [jsonschema]
  	(7) frame_debug    1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
attachments: 0
metadata: 0

For simplicity in these repro steps, I'm just merging a single file: (The problem occurs too though if I merge two different mcap files though)
mcap merge recording000001.mcap -o recording000001_merged.mcap

Now inspecting the result, we see the schema on channel depth_1 has changed:
❯ mcap info recording000001_merged.mcap
library: mcap go v1.0.0
profile:
messages: 7
duration: 0s
start: 54.911
end: 54.911
compression:
	zstd: [1/1 chunks] [2.16 MiB/168.80 KiB (92.38%)]
channels:
  	(1) bot_entity     1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
  	(2) origin_entity  1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
  	(3) depth_1        1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema] # <-- Schema changed from foxglove.RawImage
  	(4) depth_grid_1   1 msgs (+Inf Hz)   : foxglove.Grid [jsonschema]
  	(5) at_level_og_1  1 msgs (+Inf Hz)   : foxglove.PointCloud [jsonschema]
  	(6) overhead_og_1  1 msgs (+Inf Hz)   : foxglove.PointCloud [jsonschema]
  	(7) frame_debug    1 msgs (+Inf Hz)   : foxglove.SceneUpdate [jsonschema]
attachments: 0
metadata: 0

Expected Behavior
I would expect the schema for the channel depth_1 would remain foxglove.RawImage and not change to foxglove.SceneUpdate
Repro data: repro_data.tar.gz

Thanks for the report! This has been fixed in v0.0.27.


Issue running setup-cluster.sh

I've followed the steps to setup the environment, but am running into the following problem when I run ./bin/000/setup-cluster.sh
○ ./bin/000/setup-cluster.sh
Failed to remove container (fluent_restify): Error response from daemon: No such container: fluent_restify
Failed to remove container (fluent_bastion): Error response from daemon: No such container: fluent_bastion
fluent_sinopia
2fc13906fde6ff15632e980669da310d781a6c618ac20a7138508189ef010145
docker: Error response from daemon: Requested CPUs are not available - requested 1, available: 0..
See 'docker run --help'.
docker: Error response from daemon: Could not get container for fluent_restify.
See 'docker run --help'.
Set up the cluster.

Here is the result of my docker images command:
○ docker images
REPOSITORY          TAG                     IMAGE ID            CREATED             SIZE
fluent              service-static-server   a583e79a80cb        33 minutes ago      571 MB
fluent              service-tcp             b32f49b97513        34 minutes ago      549.1 MB
fluent              service-restify         45b49c64713b        34 minutes ago      549.6 MB
fluent              service-load-balancer   4a8d7de525e7        35 minutes ago      608 MB
fluent              service-http            0c431835cabd        35 minutes ago      549.5 MB
fluent              service-demo            26e1d3dbc14f        36 minutes ago      593.8 MB
fluent              service-compute         985920ddb117        36 minutes ago      615.1 MB
fluent              bastion                 65e52e634ee8        37 minutes ago      457.9 MB
fluent              service-app             a97f8c6a893c        37 minutes ago      610.5 MB
redis               latest                  4f5f397d4b7c        4 days ago          177.6 MB
rabbitmq            3-management            f35ba29f7d29        4 days ago          300.8 MB
rabbitmq            latest                  f5635e902611        4 days ago          300.8 MB
sameersbn/bind      latest                  48403cf5eeab        9 days ago          321.4 MB
rnbwd/sinopia       latest                  5aecc332e079        4 months ago        338.8 MB

and my docker info:
○ docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 14
Server Version: 1.10.2
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 86
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: host bridge null
Kernel Version: 4.1.18-boot2docker
Operating System: Boot2Docker 1.10.2 (TCL 6.4.1); master : 611be10 - Mon Feb 22 22:47:06 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 1.956 GiB
Name: default
ID: N4EC:XYFE:L7BT:CW6P:WPEC:RC7D:GZX3:B7EF:NO3T:NDQI:JBUY:YVMS
Debug mode (server): true
 File Descriptors: 17
 Goroutines: 30
 System Time: 2016-03-07T01:55:32.482217566Z
 EventsListeners: 0
 Init SHA1:
 Init Path: /usr/local/bin/docker
 Docker Root Dir: /mnt/sda1/var/lib/docker
Labels:
 provider=virtualbox

I am still pretty new to docker so maybe I didn't start the container correctly. Any help would be appreciated. Thanks.

Unfortunately I ran into the exact same problem, did you find a solution to this?

Thanks for the feedback guys.
Looks like a better option should have been to use a proper DockerFile.
I’ll close this issue, and I’ll be using a dockerfile in the upcoming talks. — It will be much easier.
This error:
docker: Error response from daemon: Requested CPUs are not available - requested 1, available: 0..

is related to a mis-configuration on my side (the container is expecting more than one CPU whereas Docker Virtualbox is set up to have one only. — I don’t know if there’s an easy way to fix it without directly changing Virtualbox’s docker VM’s "assigned CPU core" counts. https://www.virtualbox.org/manual/ch03.html
This
Failed to remove container (fluent_restify): Error response from daemon: No such container: fluent_restify

should be a harmless warning, and should not effect the demos

Sorry for the late response btw.
My email client appears to have filtered this issue notification into a special folder that I look "only when I have time" — and I’m generally busy as hell :)
Closing this issue.
Feel free to comment on it if you need further feedback.


Update README.md

@SRvSaha  Please check comments for DateToSearch fied of appsetting.json.

Issue Resolved with #17, so closing this PR. @priyanka1111


Batch update does not work for keyless tables

Description
Attempting to call .Update() on a query for a table with .HasNoKey() set on its model builder throws an exception here.
I have a keyless table for holding app config values that has only one row. I would like to run a statement like:
UPDATE "Configuration" SET "SomeConfigValue" = 123;

I've tried several variations of EF queries:
await _context.Configuration
  .UpdateAsync(c => new Configuration { SomeConfigValue = 9999 });
await _context.Configuration.Where(x => true)
  .UpdateAsync(c => new Configuration { SomeConfigValue = 9999 });

Exception
System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')
   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at Z.EntityFramework.Plus.BatchUpdate.CreateCommand(IQueryable query, IEntityType entity, List`1 values)
   at Z.EntityFramework.Plus.BatchUpdate.Execute[T](IQueryable`1 query, Expression`1 updateFactory)
   at Z.EntityFramework.Plus.BatchUpdateExtensions.Update[T](IQueryable`1 query, Expression`1 updateFactory, Action`1 batchUpdateBuilder)
   at Z.EntityFramework.Plus.BatchUpdateExtensions.<>c__DisplayClass2_0`1.<UpdateAsync>b__0()

Further technical details

EF version: EF Core 3.1
EF Plus version: 3.0.24
Database Provider: Postgres

Thank you for reporting,
We are supposed to review this feature this month, so we will certainly look at it very soon.
Best Regards,
Jonathan

Performance Libraries
context.BulkInsert(list, options => options.BatchSize = 1000);
Entity Framework Extensions • Entity Framework Classic • Bulk Operations •  Dapper Plus
Runtime Evaluation
Eval.Execute("x + y", new {x = 1, y = 2}); // return 3
C# Eval Function • SQL Eval Function

Hi, Jonathan,
It seems that something else got broken - Update doesn't work at all - no props are updated.
From my test project:
 [Fact]
        public void TestInMemoryBatchUpdate()
        {
            //this is fine - for straightforward update for entities that don''t have navigation props
            _context.Addresses.AsQueryable().Update(x => new Address { AddressLine = "dummy" });

            Assert.True(_context.Addresses.ToList().All(x => x.AddressLine == "dummy"));
        }

This fails.
Also note that comparing to my initial version of test proj, I had to call ToList(). It seems that after
_context.Addresses.AsQueryable().Update(  something is off with context entities, because regular All (__context.Addresses.All(x => x.AddressLine == "dummy")_) would return true.