Broken usage of capsys for inspecting logging output

What happened:

We have one unit test that uses capsys to make assertions about emitted logs: dask/dataframe/io/tests/test_sql.py
Add a second unit test that does the same
You can't have more than one test using that pattern, the first one works and anything after does not

add new test after old test: old test passes, new test fails
skip old test: new test passes
add new test before old test: new test passes, old test fails

What you expected to happen:
We can have as many tests as we want making assertions about logs.
Minimal Complete Verifiable Example:
# repro.py
import sqlalchemy as sa
from tempfile import NamedTemporaryFile

def do_something():
    with NamedTemporaryFile() as f:
        engine = sa.create_engine(f'sqlite:///{f.name}', echo=True)
        engine.execute("CREATE TABLE t (id INTEGER);")

def test1(capsys):
    do_something()
    out, err = capsys.readouterr()
    assert 'CREATE' in out

def test2(capsys):
    do_something()
    out, err = capsys.readouterr()
    assert 'CREATE' in out

Usage:
$ pytest -v repro.py -k test1 # passes
$ pytest -v repro.py -k test2 # passes
$ pytest -v repro.py # test1 passes, test2 fails

  inspected stderr of failed test case

Traceback (most recent call last):
  File "/usr/lib/python3.8/logging/__init__.py", line 1088, in emit
    stream.write(msg + self.terminator)
ValueError: I/O operation on closed file.
Call stack:
  File "/dask/venv/bin/pytest", line 8, in
    sys.exit(console_main())
  File "/dask/venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/dask/venv/lib/python3.8/site-packages/_pytest/config/__init__.py", line 162, in main
    ret: Union[ExitCode, int] = config.hook.pytest_cmdline_main(
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/main.py", line 316, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/main.py", line 269, in wrap_session
    session.exitstatus = doit(config, session) or 0
  File "/dask/venv/lib/python3.8/site-packages/_pytest/main.py", line 323, in _main
    config.hook.pytest_runtestloop(session=session)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/main.py", line 348, in pytest_runtestloop
    item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 109, in pytest_runtest_protocol
    runtestprotocol(item, nextitem=nextitem)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 126, in runtestprotocol
    reports.append(call_and_report(item, "call", log))
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 215, in call_and_report
    call = call_runtest_hook(item, when, **kwds)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 254, in call_runtest_hook
    return CallInfo.from_call(
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 311, in from_call
    result: Optional[TResult] = func()
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 255, in
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/runner.py", line 162, in pytest_runtest_call
    item.runtest()
  File "/dask/venv/lib/python3.8/site-packages/_pytest/python.py", line 1641, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/dask/venv/lib/python3.8/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/dask/venv/lib/python3.8/site-packages/_pytest/python.py", line 183, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/dask/repro.py", line 23, in test2
    do_something()
  File "/dask/repro.py", line 13, in do_something
    engine.execute("CREATE TABLE t (id INTEGER);")
  File "", line 2, in execute
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 401, in warned
    return fn(*args, **kwargs)
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3146, in execute
    return connection.execute(statement, *multiparams, **params)
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1274, in execute
    return self._exec_driver_sql(
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1578, in _exec_driver_sql
    ret = self._execute_context(
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1755, in _execute_context
    self._log_info(statement)
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 136, in _log_info
    self.engine.logger.info(message, *arg, **kw)
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/log.py", line 117, in info
    self.log(logging.INFO, msg, *args, **kwargs)
  File "/dask/venv/lib/python3.8/site-packages/sqlalchemy/log.py", line 163, in log
    self.logger._log(level, msg, args, **kwargs)
Message: 'CREATE TABLE t (id INTEGER);'

Anything else we need to know?: the issue manifested in #8609.
Environment:

SQLAlchemy verison: 1.4.31
pytest version: 6.2.5
Dask version: main
Python version: 3.8.10
Operating System: Ubuntu 20.04
Install method (conda, pip, source): source

Thank you @amirkdv for writing this up so clearly and including the reproducible example!

Right, good point! I think the funkiness happens after we exit the test case. On pytest's end, every fixture has a lifecycle like this:
def capsys(...):
    # ... fixture setup, runs before entering test case
    yield capture_fixture # fed into test function
    # ... fixture teardown, runs after test case has finished

I think it's the fixture teardown logic for capsys that messes with io streams in a way that breaks log handlers.
The way people seem to be working around this in pytest-dev/pytest#5502 is to clear their registered log handlers during test teardown (which happens before pytest's teardown of fixtures used in a test, aye so confusing). This is fine if you're ok to ignore logging, but in our case here we're actually testing log output.

Have you tried using capsys as a module fixture, i.e. calling it once per module instead of once per test? To avoid capsys shutdown to interfere with the logger. AFAIK it should set a buffer for stderr and stdout that all the tests should use, WDYT?


Update build.gradle

Thanks for the report. I'll send a commit to update all affected TF Lite samples so I'll close this pull request for now.


Android error: Cannot resolve symbol 'CardChannel'

When I try to follow the IProvider sample in Android Studio (3.5.2, JRE 1.8.0_202, JVM OpenJDK, on Linux), the following error messages appear:
Cannot resolve symbol 'CardChannel'
It cannot find smartcardio. The import.smartcardio fails with message: "Cannot resolve symbol 'smartcardio' "
Am I missing a dependency other than the "implementation 'com.github.devnied.emvnfccard:library:3.0.0' ?"
Android does not seem to have javax.smartcardio. How did you solve it in the sample app on Google Play?
The sample application from Google Play works, but I stuck with these error messages when trying to follow the samples and use the library. I really like how this library is build, but could not figure out how to use it in my project. Spinning around this for 2 weeks now. Any help is appreciated!

Same problem here !
Did you find any solution ?

This sample is implemented in Java. In Android you have to use NFC Tag Reader

For Android you just need to use the library:
<dependency>
  <groupId>com.github.devnied.emvnfccard</groupId>
  <artifactId>library</artifactId>
  <version>xxx</version>
</dependency>

The PcscProvider is only for desktop applications. I have added a provider sample for Android


UWP/WASDK Tests won't run upgrading to 3.1.1

Describe the bug
Was trying to update from 2.2.10 to 3.1.1 for MSTest.TestAdapter/Framework as we're not seeing Logger.LogMessage/TextContext.WriteLine output appear in Visual Studio (no Output/Additional Output link appears next to test), not sure what's going on there, but figured updating may help.
However, after cleaning our environment and reopening Visual Studio, only some tests are discovered and I saw this in the test log:
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()
========== Starting test discovery ==========
========== Test discovery finished: 44 Tests found in 4.8 sec ==========
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()
Building Test Projects
Aborting test run due to build failures. Please see the build output for more details.
Building Test Projects
Starting test discovery for requested test run
========== Starting test discovery ==========
An exception occurred while test discoverer 'MSTestDiscoverer' was loading tests. Exception: Method not found: 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestAdapterSettings Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestSettingsProvider.get_Settings()'.
No test is available in entrypoint\CommunityToolkit.Tests.Uwp.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 21.5 sec ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

Using VS 2022 17.6.5.
WASDK 1.3.230331000 and UWP
TFMS: net6.0-windows10.0.19041.0/uap10.0.17763
Steps To Reproduce

Fully clone: https://github.com/CommunityToolkit/Labs-Windows (it has a submodule)
Update following files in the tooling directory to latest version of MSTest:

Try and run tests.

Expected behavior
All tests discovered and tests run
Actual behavior
Tests don't run/Not all discovered

You can see none of our source generated tests are even listed after a build (works ok on 2.2.10 after initial build):

Additional context

Same here.

Hi @michael-hawker,
For the various
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()

showing in the output, please open an issue on developer community this is linked to some bug in Test Explorer and it's not handled by us.
For the other part of the problem. you haven't given me much information as to what is missing... I have done a build, open the sln on VS and I see 6 tests under ExampleCanbasLayoutTestClass which seems to be 3 more than you. I can also run them correctly.

I'm seeing same issue here on a non UWP project

I also have this problem. I hope it will get fixed.
The Test output windows shows that there is an exception in the Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestSettingsProvider.get_Settings().
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
Starting test discovery for requested test run
========== Starting test discovery ==========
An exception occurred while test discoverer 'MSTestDiscoverer' was loading tests. Exception: Method not found: 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestAdapterSettings Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestSettingsProvider.get_Settings()'.
No test is available in entrypoint\UnitTestProject1.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 820 ms ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========
Starting test discovery for requested test run
========== Starting test discovery ==========
========== Test discovery finished: 1 Tests found in 2.6 sec ==========
========== Starting test run ==========
========== Test run finished: 1 Tests (1 Passed, 0 Failed, 0 Skipped) run in 125 ms ==========
Starting test discovery for requested test run
========== Starting test discovery ==========
An exception occurred while test discoverer 'MSTestDiscoverer' was loading tests. Exception: Method not found: 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestAdapterSettings Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.MSTestSettingsProvider.get_Settings()'.
No test is available in entrypoint\UnitTestProject1.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery finished: 0 Tests found in 2.3 sec ==========
========== Starting test run ==========
========== Test run finished: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

Same here

Me too


Use with create-react-app

Steps to reproduce
Using create-react-app with this module breaks the build.
I'm using @feathers/client rather than @feathers/feathers and, when I exclude this module, everything builds. With it, the build breaks.
Expected behavior
The create-react-app build should not break.
Actual behavior
It does break. :-P

It seems to have the same error as using @feathers/feathers does, ie something in @feathersjs/errors.
Very similar to https://github.com/feathersjs-ecosystem/feathers-localstorage

You will need to provide information about what causes the issue.

Sorry, I meant to say it was very similar to https://github.com/feathersjs-ecosystem/feathers-localstorage/issues/66
I'm using create-react-app for a frontend webapp app and it needs it's dependencies to be transpiled to be included properly.
With that other issue @daffl figured out that some unbuilt files were making it through. Maybe that's similar.

I don't use create-react-app. A PR would be welcome.

It really just boils down to there being no build step. Is this supposed to be isomorphic?

I think this problem can be fixed with this:
https://github.com/final-form/react-final-form/blob/7dacf6392f44c8c31ec57d4b3103637ffd5f0f2d/package.json#L6-L8

FeathersJS repos all had a build step in Auk. That was removed for server-only packages in Buzzard. The build step was removed from this repo by accident.
I'll fix this next time I address the repo. You can run a webpack step yourself in the meantime.

CRA does not like you messing with webpack.
hope we get this fixed sooner rather than later :crossed_fingers:

feathers-hooks-common is upgraded on a periodic basis.

is there a fix for this?

"CRA does not like you messing with webpack." I am not aware of any repo which could work either server or clinet side which does not use either Webpack or Rollup to produce the client side. The component repos of @feathersjs/feathers-client do so. So I don't understand your comment.
If you want to use this repo client side, just run it through Webpack or Rollup. Feel free to submit a PR.

Closing this old issue because no new activity. Please feel free to comment, if the problem still exists with new version v6.0.0


Update gqlgen to latest

Routine updates to verify no incompatibilities have crept in
Signed-off-by: Steve Coffman<EMAIL_ADDRESS>

Sounds good, thanks!


Display the component and Create the Service

The ‘Display the component’ section  refers to a template which was already created.
The ‘Create the service’ section differs from my code, where hero.component and hero.service seem mixed up.
Running the sample from VSCode produces an error, but ng serve at least displays a webpage, albeit without any Here information displayed, only the Hero header.
I’m using WSL and the current version of node and angular, so many all these errors are a result of changes in the platform and tools.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 435b849c-e5df-bcdf-44df-ec053cf1368f
Version Independent ID: 918ebbe6-ec72-bca2-f514-8a1d1c60f906
Content: MongoDB, Angular, and Node tutorial for Azure - Part 3
Content Source: articles/cosmos-db/tutorial-develop-mongodb-nodejs-part3.md
Service: cosmos-db
Product: unspecified
GitHub Login: @SnehaGunda
Microsoft Alias: sngun

@GrantMeStrength thanks for pointing the issues, looks like the doc was written a while ago and --minimal option didn't generate certain files and code blocks. I fixed them with latest version in this PR MicrosoftDocs/azure-docs-pr#42723
I am working with the source code owner to get the code fixed in samples repo.

@GrantMeStrength agree that there are issues in this doc, I made some changes to fix them. When I tested the code, I saw an error at   selectedHero: Hero; and private heroService: HeroService statements and that error will be resolved when you follow next steps where the Hero and HeroService components are added and imported.

#please-close

@GrantMeStrength We will now proceed to close this thread. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.


Reconciliation: how to improve the "guess column type" functionality ?

As already pointed out in the documentation, the "guess column type" function of reconciliation services is unsatisfactory. The choices are often too specific. As a result, it's rare that a user choose one of the propositions.
Example: I have a column containing a mix of Belgian municipalities and sections of municipalities. Here are the propositions of the reconciliation service with Wikidata:

You can see that none of them really corresponds to what I need, that is to say a sort of "territorial administrative entity (of Belgium)".
I think it's very difficult (if not impossible) to find a system that works well with any reconciliation service. But for Wikidata, the default one, why not look for Least common subsumers (the Lowest common superclasses in the ontology) of the first two or three propositions ? This can be done with a Sparql Query.
In the case of Municipalities of Belgium and Municipality Sections, the LCS would be "political territorial entity (Q1048835)" and "administrative territorial entity of Belgium (Q15623264)", which would be in both cases a much better definition of my column.

Very interesting question! I guess the current heuristics were appropriate for Freebase, but not for Wikidata. What it currently does is count the number of times each type appears in the suggestions for the first few rows, and order the types by decreasing frequency.
The queries made to guess the types look pretty much like any normal query so reconciliation services are not expected to return something tailored for the type detection algorithm. That is usually quite nice because as a reconciliation service developer you do not have to worry about this part. But it makes it harder to tweak the type detection results without changing the global behavior of the interface.
One thing that could potentially solve the issue without changing anything in OpenRefine would be that the Wikidata service returns not only the direct types, but also the indirect types via the "subclass of" (P279) property, up to a certain level. In this way, more general types would emerge. The problem is that fetching these more general types is expensive, so we probably don't want to do it for all queries - ideally it should happen only during type detection (and even so - we want this part to be quick too!).

@wetneb The Sparql query I'm talking about seems fast. I'd rather wait five seconds for relevant proposals than get bad ones in two seconds.

oh sorry I overlooked your proposal, just dumped my own ideas >_<
So, given a list of names in the column, how would you find the qids to feed to that query?

You put your finger on the difficult part. That's why I thought we could use the QID of the proposals made by the standard reconciliation service. For example, the QID of the first two props (or all five, that would work just as well) in my screenshot. But of course, it's always easy to make suggestions when you do not program them yourself.

Ah right, I see, that would make sense. It would probably be quite accurate and efficient. The only issue is that this would be specific to Wikidata, so we would hard-code something for Wikidata in OpenRefine (which has been done for Freebase, but it's better if we can avoid that).
I guess the "clean" solution to implement that is to add a new (optional) endpoint to the reconciliation API, that takes a list of strings and returns type suggestions. If the reconciliation service does not provide it we would resort to the current heuristics. That would enable us to use a custom algorithm for Wikidata. Potentially we could reuse that for the data extension feature too (so that we get more property suggestions, for instance in the case where the column was reconciled against no particular type).

The only issue is that this would be specific to Wikidata

This is obviously a problem. I personally think that Wikidata (Open Refine's standard reconciliation service) might deserve special treatment.
That said, the Least common subsumer method can be transposed to any RDF graph with an ontology (ie, a system of entities and classes linked by rdf:type or instanceOf relationships).

If someone wants to test the idea on a list of QID, here is a little Python3 script (remember that i'm not a developer...)
`# -- coding: utf-8 --
import requests #need to be installed : pip install requests
import json
from bs4 import BeautifulSoup #need to be installed : pip install bs4
def getLCS(list_of_qids):
"""
Get the Least common subsumers between several Wikidata qids in a list
Return a Python list
"""
array_string = ", ".join(["wd:" + x for x in list_of_qids])
print(array_string)
query = {"query": """
SELECT ?lcs ?lcsLabel WHERE {
?lcs ^wdt:P279* %s .
FILTER not exists {
?sublcs ^wdt:P279* %s ;
      wdt:P279 ?lcs .
  }
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en " . } }""" % (array_string, array_string)
         }
url = "https://query.wikidata.org/sparql"
r = requests.get(url, params=query)
soup = BeautifulSoup(r.text, "lxml")

return [x.text for x in soup.find_all("literal")]

if name == 'main':
list_of_qids = ['Q2785216', 'Q493522' ]

print(getLCS(list_of_qids))

`

One thing that could potentially solve the issue without changing anything in OpenRefine would be that the Wikidata service returns not only the direct types, but also the indirect types via the "subclass of" (P279) property, up to a certain level. In this way, more general types would emerge.

This sounds like a good solution and would mimic the behavior of Freebase's much flatter type hierarchy.

The problem is that fetching these more general types is expensive, so we probably don't want to do it for all queries - ideally it should happen only during type detection (and even so - we want this part to be quick too!).

That sounds like an implementation issue. How often does the type hierarchy change? Not very often, I bet, so it probably can be cached effectively. As with the Freebase practice of blacklisting Topic, you probably want to exclude any top level Thing equivalent types.

facet of P1269 as well as part of P361 and maybe opposite of P461 could also be very useful to put into type guessing.
As a good example:  materialism Q7081
Just one simple concept I've thought of in the past of improving the UI for type guessing.  Have 2 columns.  Direct types, and Indirect types (which might just be labeled simply as "Other" and that are inclusive of P1269, P361, P461, maybe P31?).  And where type guessing is a step in a workflow (Next/Previous buttons).  And the boxing could be stretched horizontally for space savings and quick eyeballing and quick clicking like so:
Click a link to choose a type

Direct
Other

materialism (Q7081)
monism (Q178801), idealism (Q33442)

racism (Q8461)
prejudice (Q179742), crime (Q83267), soci(et)al entirety (Q96108364), identity politics (Q2914650), ideology (Q7257)

antisemitism (Q22649)
racism (Q8461) , bigotry (Q859781)


Do not create the charm state in init

Overview

Fixes ISD-1380.
As the moment, the charm state is generated in the __main__ method in the charm.
This creates several problems:

Currently, when the integration with the postgresql is created, the file homeserver.yaml is not updated. This is because of the internal working of the data library for postgresql (see this and this). The password of the relation is not yet available when the charm state is created in the __init__ method, but will be available after the on_relation_changed_event is processed by the data library. As this happens, the only way to update the homeserver.yaml currently is to receive another event (like a config changed) that updates the homeserver.yaml file.
If when the nginx pebble ready event arrives the charm configuration is in an invalid state, the charm will return from the __main__ method before registering the event handlers, and nginx will not be provisioned. As nginx pebble layer is only planned in that event, the charm will be in a state that is unrecoverable without killing the pod/container.
If we want to implement the secret-changed events, we will have problems similar to the first case.

The charm state is now built in the method build_charm_state that is called in every hook method that requires it, and injected into it.
As part of the refactor, the PebbleService class disappears as all its functions that require the charm state will include the it as an argument. Future refactors maybe needed to improve the current code so functions that do not need the full charm_state only receive the required parameters.
Rationale

Fix all the problems/bugs exposed in the overview.
Another alternative is to update the charm_state in certain parts of the code. This would result in several code paths that update the charm_state, which will introduce more complexity, as in the current solution we can think of the charm_state as an immutable structure from the event observer point of view.
Juju Events Changes

Module Changes

Library Changes

Checklist

[x] The charm style guide was applied
[x] The contributing guide was applied
[x] The changes are compliant with ISD054 - Manging Charm Complexity
[x] The documentation is generated using src-docs
[x] The documentation for charmhub is updated.
[x] The PR is tagged with appropriate label (urgent, trivial, complex)

@gregory-schiano @jdkandersson I wonder if we are not getting to the point of going back and using stored state  due complexity of managing the state in the charm.

Thank for the input @amandahla. Can you expand on why you think there is an increase in the complexity related to the stored state? The current PR does not change how the CharmState instance is built, only when it is built (instead of the __init__ method, in every observer that needs it).

@gregory-schiano @jdkandersson I wonder if we are not getting to the point of going back and using stored state  due complexity of managing the state in the charm.

Thank for the input @amandahla. Can you expand on why you think there is an increase in the complexity related/similar to the stored state? The current PR does not change how the CharmState instance is built, only when it is built (instead of the __init__ method, in every observer that needs it).

Indeed I agree with Javier, also I don't see how using a stored state will help as the concept of CharmState (a meaningful object representing the workload state composed of relation data, configuration data, and workload information) will still be needed.
This PR aims at fixing the design issues we having with the state being instantiated in the constructor before the events are observed leading to issues if the state fails to load

@gregory-schiano @jdkandersson I wonder if we are not getting to the point of going back and using stored state  due complexity of managing the state in the charm.

Thank for the input @amandahla. Can you expand on why you think there is an increase in the complexity related/similar to the stored state? The current PR does not change how the CharmState instance is built, only when it is built (instead of the __init__ method, in every observer that needs it).

Indeed I agree with Javier, also I don't see how using a stored state will help as the concept of CharmState (a meaningful object representing the workload state composed of relation data, configuration data, and workload information) will still be needed. The stored state will become another source of data (mostly replacing workload information)
This PR aims at fixing the design issues we having with the state being instantiated in the constructor before the events are observed leading to issues if the state fails to load

Actually this PR just made me think more about state and how is turning out to be more like a layer in front of ops since we are using it instead of getting information directly from relations/unit/configuration.
I remember StoredState because it was easier to understand that we are getting information from somewhere but now having the state concept is a little bit tricky if you are not familiar with event lifecycle/relations.
I wonder if instead of passing state around wouldn't be easier to just use ops and get what we need in the time we need it.

@javierdelapuente approving the PR but just using it to discuss the theory ;)

I believe we've had this discussion in GitHub runner as well, it would help the team greatly if we have a post written on the reasons to do this (if we don't already have it) :D

Do you mean creating the CharmState outside of __main__ or more about the reasons for the CharmState?

I believe we've had this discussion in GitHub runner as well, it would help the team greatly if we have a post written on the reasons to do this (if we don't already have it) :D

Agree we might want to expand and structure that idea in a post, that's good!

@javierdelapuente I meant having initialization of CharmState outside of init, most likely for each function just before they get used. Here's a reference to setup_state outside of charm __init__ on github-runner-operator charm. The reason for this here was to fully complete initialization and register the handlers without failure. The bug we encountered was that the actions wouldn't get registered due to CharmState.from_charm raising exceptions and thus not completing the handler registration (framework.observes).

I will have a look and update ISD014, as it recommends to initialize the state in the init method. Besides the issue of the handlers not registered (number 2 in the overview), we also had another problem (data library needs the event to refresh data in the relation and so the data is not yet there in init, number 1), and I believe using secrets-changed will be difficult to manage this way (number 3).


How can I get my kits to choose which subdirectory has CMakeLists.txt?

Brief Issue Summary
As someone who works in a monorepo, it'd be great to be able to have my kits determine which directory contains the root CMakeLists.txt, and which directory I'm going to be building in. For example, sometimes I'm building all of LLVM and need -S ${workspaceFolder}/llvm -B build, while other times I want to directly build a subproject (e.g. -S ${workspaceFolder}/subproject -B build-subproject).
Being able to set these in cmake-kits.json as opposed to my workspace settings would be fantastic. Is there a way to do this?

Unfortunately, the only way to specify multiple root CMakeLists.txt right now is to open multiple folders in the same workspace.  The feature we would do to support this is tracked by #1374.


Electronic life skeleton

Additionally added random critter, that moves randomly and kills other critters if there is any nearby
closed #435

Added smarter behaviour to critters
closed #436


Switch from net2 to socket2

Closes https://github.com/hyperium/hyper/issues/2205

The doc failure doesn't seem related to this change, maybe it was broken in a recent nightly?


[13.0] ISPN-14853 ManagedConnectionFactory DataSource should be lazily initi…

https://issues.redhat.com/browse/ISPN-14853

Thanks for backporting @tristantarrant

I'll release 13.0.16.Final then


decoder should be a global, not everywhere reference encoding / json

Please use the following questions as a guideline to help me answer
your issue/question without further inquiry. Thank you.
Which version of Elastic are you using?
elastic.v5 (for Elasticsearch 5.x)
Please describe the expected behavior
Please describe the actual behavior
Any steps to reproduce the behavior?

I don't understand. Can you explain with an example? Did you notice that you can configure your own decoder?

Closing until I get more details.


Question: Cutting Line

Hello,
I'm trying to implement CSG into a Three.js project in which a line on the surface of a mesh is used to slice it into two parts. The line is defined as two arrays, one with the positions of each vertex of the polyline, and one with the normal vectors of the mesh at those points (which can be interpolated to "stick" the line to the surface of the mesh between vertices).
So far I haven't had much luck using distance-from-plane slice methods since these are not planes, more like rectangles certainly with discrete edges at the line segment end points, and also likely "height" above and below the surface of the mesh.
So my idea is to sweep this line into a very very skinny rectangle and then use CSG to cut into the mesh. Then, toss out the new "interior" faces (which would be highlighted in blue in the CSG examples) leaving only two red meshes on either side of the line.
I'll be trying to implement this throughout the day but I thought I would post this first to see if anyone has:

Tried this before, and knows it's possible
Knows its not possible
Knows how it should be done (I'm a bit unclear on CSG still... the comments in the code make it clear what it does but not so much how it does it.)

Answering the above could really speed up my experiments.
Thanks in advance!

Try asking this at OpenJSCAD.org community.

@ianpaschal do you mean creating a cross-section of the mesh? Or are the section lines not planar?
If I understand you correctly I build something similar for a Python version of this code. Let me know if this is what you are looking for, then I'll send you the code.

@themmes Unfortunately it's been about a year so I came up with something else. I dove hardcore into computational geometry and wrote my own boolean methods that used lines in 3D space like a sort of virtual scalpel to slice apart polygons.

@ianpaschal  Sir I am also searching cross section - which is like cutting mesh from the centre with plane (box ) and get the centre area - That is  working with Substact method
But I need this in border - like subtracted part in border where the mesh is collided
If you could guide me - I am very weak in 3d geometry stuff :(
https://github.com/nicklockwood/Euclid/issues/76


Ruby - Support Detections and file/mime type

will be done with https://github.com/GDATASoftwareAG/vaas/issues/440


The test result

I change the image size here:

But I get a completely different result?
128*128

512*512

Why is that? I just resized the image.
Looking forward to your answer!

okay! I know it, that is your net's input...


Implement aten::_unique for XPU

🚀 The feature, motivation and pitch
Test TestCompositeComplianceXPU.test_backward_index_fill_xpu_float32 requires the operator.
Alternatives
PT 2.6
Additional context
No response

The operator is implemented, and the case is enabled.
https://github.com/intel/torch-xpu-ops/pull/574


DiscountPercent data type to json.Number

When using this library we found that Amazon sometimes sends back a Number value for DiscountPercent and sometimes a String.
This change allows the Parser to handle both

Thanks !


Example behaves unexpectedly, adds entire input on each key stroke

Using
menu = "0.3.2"
pancurses = "0.16.1"

win 10, cargo run run in Git BASH.
I copied the example and it behaves like so
cmd
>

I press a
>> a

I press b
>> > ab

I press c
>> > a> abc

It appears your terminal doesn't understand "\r" as "go back to start of line"

Is this one now ok to close given that it should be reasonable for a terminal to process return?


Edit Add and Edit feature to include remarks, remove Remarks portion

Can just merge into main instead of upstream branch-UG


Fixed required field name when setting a name with @JsonProperty

Informations

Type
Migration

Fix
No

Description
closes #396
Todos

[ ] Tests
[ ] Coverage
[ ] Example
[ ] Documentation

sorry for indentation changes... I can fix them if you want 😅

No problem but your PR is in conflict, apparently :)

Hi @Romakita
What conflict? github shows "This branch has no conflicts with the base branch"
I'll be happy to fix

Ok Because I use the rebase options. It's merged ;) Thanks @TomerAmirV
The release is 4.23.2
See you

Thanks @Romakita !
Happy to help 😄


Race condition in prometheus metrics: fatal error: concurrent map writes

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
I get a crashing pod with the following error:
 fatal error: concurrent map writes
 fatal error: concurrent map writes

 goroutine 2470 [running]:
 github.com/cilium/cilium/pkg/maps/metricsmap.promMetrics[...].upsert(0x3ecf190?, {{0x3920462, 0x7}, {0x396254b, 0x17}}, 0xc0011ae4c8)
     /go/src/github.com/cilium/cilium/pkg/maps/metricsmap/metricsmap.go:222 +0x117
 github.com/cilium/cilium/pkg/maps/metricsmap.(*metricsmapCollector).Collect.func1(0xc00704b350, 0x0?)
     /go/src/github.com/cilium/cilium/pkg/maps/metricsmap/metricsmap.go:235 +0xd5
 github.com/cilium/cilium/pkg/maps/metricsmap.metricsMap.IterateWithCallback.func1({0x3303760?, 0xc00704b350?}, {0x32417e0?, 0xc0011ae4c8?})
     /go/src/github.com/cilium/cilium/pkg/maps/metricsmap/metricsmap.go:105 +0x49
 github.com/cilium/cilium/pkg/ebpf.(*Map).IterateWithCallback(0x6046380, {0x3303760, 0xc00704b350}, {0x32417e0, 0xc0011ae4c8}, 0xc005a08c50)
     /go/src/github.com/cilium/cilium/pkg/ebpf/map.go:190 +0x191
 github.com/cilium/cilium/pkg/maps/metricsmap.metricsMap.IterateWithCallback({0xc00008f400?}, 0xc005a08d10)
     /go/src/github.com/cilium/cilium/pkg/maps/metricsmap/metricsmap.go:102 +0x85
 github.com/cilium/cilium/pkg/maps/metricsmap.(*metricsmapCollector).Collect(0xc0006c84e0, 0xc005a08f60?)
     /go/src/github.com/cilium/cilium/pkg/maps/metricsmap/metricsmap.go:229 +0x65
 github.com/prometheus/client_golang/prometheus.(*Registry).Gather.func1()
     /go/src/github.com/cilium/cilium/vendor/github.com/prometheus/client_golang/prometheus/registry.go:455 +0x102
 created by github.com/prometheus/client_golang/prometheus.(*Registry).Gather in goroutine 1929
     /go/src/github.com/cilium/cilium/vendor/github.com/prometheus/client_golang/prometheus/registry.go:547 +0xbab

Looks like this is the commit that introduced it: https://github.com/cilium/cilium/commit/f8e94721fcebe4f9e0d45d4d914949ffb7467ad1
Cilium Version
Cilium 1.15.0-rc.0
Kernel Version
Linux 6.1.69
Kubernetes Version
K8s v1.29.0
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Related: https://github.com/cilium/cilium/issues/20651, https://github.com/cilium/cilium/issues/12460


Add ome-files-cpp staging URL to CI docs

As suggested on https://github.com/ome/ome-cmake-superbuild/pull/94 this PR adds the staging URL to the OME Files C++ CI docs.
Staged at https://www.openmicroscopy.org/site/support/contributing-staging/ci-ome-files.html
@rleigh-codelibre not sure how this deployment happens, can expand if needed

Good to merge, thank you.
(Though, now you've got me wondering if the job should be on https://www.openmicroscopy.org/site/support/contributing-staging/ci-docs.html which is where I might have actually looked!)


Proposal: provide solution so the secrets don't end up in source control

Expected Behaviour
Add support for adding secrets, without having to commit them to GitHub
Current Behaviour
Currently, the only way to add secrets is within a stack YAML file.
I cannot just add it to .gitignore, as I need this to build/deploy/...
This usually implies inadvertent commits of secrets in GitHub (slack tokens, mino credentials...)
Possible Solution

support secrets or vault
or
support a way to refactor the secrets into another YAML files which I can include in the stack YAML file

How does this look that I did on my lunch break? https://github.com/alexellis/faas-cli/pull/110
It's available for testing if you can pull and rebuild the env_file branch

Hello @alexellis this works.
One question though, do we have to add the environment: part in each file?
login.yml
environment:
  account: alex.ellis
  secret_key: dogfight

I think it's explained in the PR.. did it make sense?

Derek add label: proposal

@jmkhael yes every external file will look like that:
environment:
  account: alex.ellis
  secret_key: dogfight

Implemented in https://github.com/alexellis/faas-cli/pull/110


Flops and CPU inferencing

@Greak-1124 Hi there,

What is the number of Flops to inference MFFLNet-S and MFFLNet-L?
Can it inference using cpu-only?

1. When the input resolution is 512x1024, the LMFFNet-S is 14.7G and the LMFFNet-L is 16.7G.
2. Our LMFFNet can also use CPU for reasoning.
------------------ 原始邮件 ------------------
发件人: @.>;
发送时间: 2021年6月16日(星期三) 下午3:09
收件人: @.>;
抄送: @.>; @.>;
主题: [Greak-1124/LMFFNet] Flops and CPU inferencing (#1)
@Greak-1124 Hi there,
What is the number of Flops to inference MFFLNet-S and MFFLNet-L?
Can it inference using cpu-only?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

thanks,
Note that i would suggest pruning and quantization, to reduce the flops without loosing much accuracy.
good luck

Thank you for your advice.


have a video , voice recording functions?

Still don't have it. But you don't need a concrete implementation in Royale for it. You can use some existing JS library and integrate in Royale. Royale team expect that others like you can join us and contribute this kind of libraries so other could use it directly or improve it over time.

how to integrate existing JS?

Right, you can follow this tutorial to do something quick: https://royale.apache.org/using-external-javascript-libraries-in-apache-royale/
Or if you want to try to make a Royale library with components and other kind of things that could be integrated and reused, you can mix that tutorial and check MDL lib here:
https://github.com/apache/royale-asjs/tree/develop/frameworks/projects/MaterialDesignLite
just some thoughts.

Thank you @carlosrovira 👍


Webclient is not mentioned in comments

Describe the bug
This annotation is also required with WebClient but the comments are misleading
Sample
The application throws UKNOWN Host with Eureka server if the LoadBalanced annotation is not used but the comments are only mentioning rest template

can you please provide the error you are referring to?


Broken external x-badge reference

Until 9.3.4 the badges from external file is fine.
but since 9.3.5 it does not render at all and after 9.3.7 it completely broken. (failed to load)
Here is the spec I test

badges.yml
Test:
  color: red
  label: external-badge

openapi.yml
openapi: 3.1.0
info:
  version: 0.1.0
  title: Rapidoc Badge Test

paths:
  /badge-test:
    post:
      tags:
        - Test
      summary: Test Badge Ref
      operationId: testBadgeRef
      x-badges:
        - $ref: ./badges.yml#/Test
      requestBody:
        content:
          application/json:
            schema:
              type: object
      responses:
        '202':
          description: Accepted
          content:
            application/json:
              schema:
                type: object

Currently, I use redocly/cli for merge openapi spec fragments.
and this tools does not properly merge all the x- prefixed component.
So, I can't bundle the seperated x- prefixed file to openapi.yml file directly.
Here is rendered result

9.8.4

9.8.5

Since 9.3.5 we are using a more strict OpenAPI parser. When using refs in spec ensure that they are conforming to the rules of OpenAPI standard. First try without using refs  . Refer to #1027 for more details
if it is still not working then re-open a new issue.
As a test case this example uses barges and works


"Dynamic requires" not at all working.

I have a simple wrapper around require() but it seems to have broken my deployment.
const reaction = name => {
	const res = require(name);
	res.name = name.split('/').pop();
	return res;
};

const reactions = {
	repository: {
		publicized: reaction('../reactions/publicize-repository')
	},
	issues: {
		opened: [reaction('../reactions/check-triage')],
		label: [reaction('../reactions/check-triage')]
	}
};

Worked just fine before I added the wrapper. Isn't this what "dynamic require" means in this repository's description?

Dynamic requires that are statically analyzable. We might need to add a *
Not sure if we can improve the static analyzer for this case, though. cc @guybedford

We could add some simple analysis to check for require wrapper functions, certainly, it would just be a little bit of work to fold that into the analysis.

@Qix- we are on it.

This has been implemented in https://github.com/zeit/webpack-asset-relocator-loader/pull/34, although will only land with the next ncc major.

I think this might solve: https://github.com/zeit/now-builders/issues/508
Is there any way to test this with now dev?

Fixed in #378


context: cancelCtx exclusive lock causes extreme contention

What version of Go are you using (go version)?

$ go version
go version go1.15.3 linux/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/niaow/.cache/go-build"
GOENV="/home/niaow/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/niaow/go/pkg/mod"
GONOPROXY="github.com/molecula"
GONOSUMDB="github.com/molecula"
GOOS="linux"
GOPATH="/home/niaow/go"
GOPRIVATE="github.com/molecula"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build497931594=/tmp/go-build -gno-record-gcc-switches"

What did you do?
Passed a single context with cancellation to a bunch of goroutines.
These goroutines had a cold-path compute task, interlaced with calls to context.Err() to detect cancellation.
The loop looks something like:
var out []Thing
for iterator.Next() {
	if err := ctx.Err(); err != nil {
		// caller doesn't need a result anymore.
		return nil, err
	}

	// Fetch thing from iterator, apply some filtering functions, and append it to out.
}

What did you expect to see?
A bit of a slowdown from the context check maybe?
What did you see instead?
Slightly over 50% of the CPU time was spent in runtime.findrunnable. The cancelContext struct uses a sync.Mutex, and due to extreme lock contention (64 CPU threads spamming it), this was triggering lockSlow. From poking at pprof, it appears that about 86% of CPU time was spent in functions related to this lock acquire.
I was able to work around this by adding a counter and checking it less frequently. However, I do not think that this is an intended performance degradation path. Theoretically this could be made more efficient with sync/atomic, although I think a sync.RWMutex would still be more than sufficient.

Thank you for raising this issue. Can you please provide more information, a sample program which demonstrates this, or if that is not possible, the pprof trace you used to identify the lock contention.
Thank you

Theoretically this could be made more efficient with sync/atomic, although I think a sync.RWMutex would still be more than sufficient.

If you have 64 hardward threads, sync.RWMutex is unlikely to do any better than sync.Mutex: you'll just transform the lock contention to cache contention. (See #17973.)

As a workaround, you can guard slow Err() call with a non-blocking Done() read:
done := ctx.Done()
for something {
  select {
  case <-done:
    return ctx.Err()
  default:
  }
}

FWIW, this just showed up in some of our profiles recently. It was only 1% of our overall CPU time, but I expected 0.
Here's a benchmark:
func BenchmarkContextCancelDone(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-ctx.Done():
                        default:
                        }
                }
        })
}

Profile:
dev:issue42564_cancelctx $ go tool pprof prof.cpu
File: issue42564_cancelctx.test
Type: cpu
Time: Jan 15, 2021 at 8:41pm (PST)
Duration: 1.51s, Total samples = 3.40s (225.88%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2630ms, 77.35% of 3400ms total
Dropped 12 nodes (cum <= 17ms)
Showing top 10 nodes out of 67
      flat  flat%   sum%        cum   cum%
     890ms 26.18% 26.18%     1280ms 37.65%  sync.(*Mutex).lockSlow
     590ms 17.35% 43.53%     2500ms 73.53%  context.(*cancelCtx).Done
     250ms  7.35% 50.88%      620ms 18.24%  sync.(*Mutex).Unlock (inline)
     220ms  6.47% 57.35%      220ms  6.47%  runtime.usleep
     150ms  4.41% 61.76%      150ms  4.41%  runtime.cansemacquire (inline)
     140ms  4.12% 65.88%      280ms  8.24%  runtime.lock2
     120ms  3.53% 69.41%      120ms  3.53%  runtime.futex
      90ms  2.65% 72.06%       90ms  2.65%  runtime.osyield
      90ms  2.65% 74.71%       90ms  2.65%  runtime.procyield
      90ms  2.65% 77.35%      120ms  3.53%  runtime.unlock2

If you cache the ctx.Done() value, though:
func BenchmarkContextCancelDone_Workaround(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        done := ctx.Done()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-done:
                        default:
                        }
                }
        })
}

Then it's much faster:
BenchmarkContextCancelDone
BenchmarkContextCancelDone-8                     8253436               164.1 ns/op
BenchmarkContextCancelDone_Workaround
BenchmarkContextCancelDone_Workaround-8         504514413                2.365 ns/op

Looks like this is due to @josharian's 986768de7fc (context: lazily initialize cancelCtx done channel) 😄 which had good numbers, but didn't consider mutex contention.
@josharian elsewhere suggested we could switch cancelCtx to use an atomic.Value.
/cc @crawshaw

FWIW, this just showed up in some of our profiles recently. It was only 1% of our overall CPU time, but I expected 0.
Here's a benchmark:
func BenchmarkContextCancelDone(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-ctx.Done():
                        default:
                        }
                }
        })
}

Profile:
dev:issue42564_cancelctx $ go tool pprof prof.cpu
File: issue42564_cancelctx.test
Type: cpu
Time: Jan 15, 2021 at 8:41pm (PST)
Duration: 1.51s, Total samples = 3.40s (225.88%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 2630ms, 77.35% of 3400ms total
Dropped 12 nodes (cum <= 17ms)
Showing top 10 nodes out of 67
      flat  flat%   sum%        cum   cum%
     890ms 26.18% 26.18%     1280ms 37.65%  sync.(*Mutex).lockSlow
     590ms 17.35% 43.53%     2500ms 73.53%  context.(*cancelCtx).Done
     250ms  7.35% 50.88%      620ms 18.24%  sync.(*Mutex).Unlock (inline)
     220ms  6.47% 57.35%      220ms  6.47%  runtime.usleep
     150ms  4.41% 61.76%      150ms  4.41%  runtime.cansemacquire (inline)
     140ms  4.12% 65.88%      280ms  8.24%  runtime.lock2
     120ms  3.53% 69.41%      120ms  3.53%  runtime.futex
      90ms  2.65% 72.06%       90ms  2.65%  runtime.osyield
      90ms  2.65% 74.71%       90ms  2.65%  runtime.procyield
      90ms  2.65% 77.35%      120ms  3.53%  runtime.unlock2

If you cache the ctx.Done() value, though:
func BenchmarkContextCancelDone_Workaround(b *testing.B) {
        ctx, cancel := context.WithCancel(context.Background())
        defer cancel()
        done := ctx.Done()
        b.RunParallel(func(pb *testing.PB) {
                for pb.Next() {
                        select {
                        case <-done:
                        default:
                        }
                }
        })
}

Then it's much faster:
BenchmarkContextCancelDone
BenchmarkContextCancelDone-8                     8253436               164.1 ns/op
BenchmarkContextCancelDone_Workaround
BenchmarkContextCancelDone_Workaround-8         504514413                2.365 ns/op

Looks like this is due to @josharian's 986768de7fc (context: lazily initialize cancelCtx done channel) 😄 which had good numbers, but didn't consider mutex contention.
@josharian elsewhere suggested we could switch cancelCtx to use an atomic.Value.
/cc @crawshaw

And fortunately #39351 (sync/atomic: add (*Value).Swap and (*Value).CompareAndSwap) was finally accepted after 5 years of requests (and has open CL https://go-review.googlesource.com/c/go/+/241678/), so this could use that.
I'll mark this as Go 1.17, with the assumption that that'll work out.

And fortunately #39351 (sync/atomic: add (*Value).Swap and (*Value).CompareAndSwap) was finally accepted after 5 years of requests (and has open CL https://go-review.googlesource.com/c/go/+/241678/), so this could use that.
I'll mark this as Go 1.17, with the assumption that that'll work out.

(It'd probably need to CompareAndSwap with an old value of an package-unexported sentinel channel that means "uninitialized", as CompareAndSwap per that proposal/CL doesn't accept a nil value for old)

(It'd probably need to CompareAndSwap with an old value of an package-unexported sentinel channel that means "uninitialized", as CompareAndSwap per that proposal/CL doesn't accept a nil value for old)

This doesn't need compare and swap; we already have a mutex available for other reasons to guard writes. I took a first stab in CL 288193, but apparently I screwed something up. Will aim to fix next week...

This doesn't need compare and swap; we already have a mutex available for other reasons to guard writes. I took a first stab in CL 288193, but apparently I screwed something up. Will aim to fix next week...


fix: produce tombstone records

Description
Adds support for tombstone records production.
Fixes https://github.com/Farfetch/kafkaflow/issues/516
How Has This Been Tested?
This was tested via the provided unit and integration tests.
Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have added tests to cover my changes
[x] I have made corresponding changes to the documentation

Disclaimer
By sending us your contributions, you are agreeing that your contribution is made subject to the terms of our Contributor Ownership Statement

How is the fix supposed to work?
Commit 082baf3
adds this

-        byte[] messageValue;
+        byte[] messageValue = Array.Empty<byte>();

-       using (var buffer = s_memoryStreamManager.GetStream())
+        if (context.Message.Value is not null)
-        {
-            await _serializer
-                .SerializeAsync(
-                    context.Message.Value,
-                    buffer,
-                    new SerializerContext(context.ProducerContext.Topic))
-                .ConfigureAwait(false);
-
-            messageValue = buffer.ToArray();
+            using (var buffer = s_memoryStreamManager.GetStream())
+            {
+                await _serializer
+                    .SerializeAsync(
+                        context.Message.Value,
+                        buffer,
+                        new SerializerContext(context.ProducerContext.Topic))
+                    .ConfigureAwait(false);
+
+                messageValue = buffer.ToArray();
+            }

The [SerializerProducerMiddleware](https://github.com/Farfetch/kafkaflow/commit/082baf349c03f513bcfeb95979efbfd615e86546#diff-00dcad3a637cfe5d6ad0ec93d5c61d5685a6a188ad69f21a22d1f1c7c7239838) sets the value to an empty byte array and if the value is not null calls any provided serializers, otherwise it sends the emoty byte array.

So even if you had a custom one that handles null it would never be called.

It even updates the docs [serializer-middleware.md](https://github.com/Farfetch/kafkaflow/commit/082baf349c03f513bcfeb95979efbfd615e86546#diff-5698cd31e603260c9e44f106e2cc0c44da52b958338cd91c9240dd40504ec696) to say that to create a tombstone you must send a null value not an empty byte array?!


releases-tab sometimes doesn't flow into overflow hamburger menu

This is due to a race condition

Be on any (responsive) page
Resize the window

Sometimes it disappears, sometimes it doesn't:

This might be due to the tab being added after GitHub looks for the tabs, so the DOM element should probably be added before the API call, hidden, and then unhidden/removed as necessary.

This might be the code responsible for this:
p(".js-responsive-underlinenav", {
	constructor: HTMLElement,
	subscribe: e=>{
		const t = e.querySelectorAll(".js-responsive-underlinenav-item");
		return async function(e, t) {
			await ge,
			Os(e, t)
		}(e, t),
		ae(window, "resize", ()=>Os(e, t))
	}
})

p is probably selector-observer and as soon as it finds the menu, it selects the items in it. releases-tab might not be in there yet.


Have a question about the source code in @floating-ui/core/computePosition.ts

Thank floating-ui, I have implemented a tooltip by using floating-ui.  it's very useful and wonderful!
And I'm interested in the source code. When I look at the core/src/computePosition.ts, found below code in line59:
    if (i === 0) {
      ({x, y} = computeCoordsFromPlacement({
        ...rects,
        placement: statefulPlacement,
      }));
    }

The computeCoordsFromPlacement  is executed already in line39.  so why need to execute computeCoordsFromPlacement again when the first middleware is applied?
Looking forward to your reply, thank you very much！

When the placement changes the coords need to restart from that point. Actually, that code should live here instead.
https://github.com/floating-ui/floating-ui/blob/489b659ee072344c4719ddf3925101af65f0feb2/packages/core/src/computePosition.ts#L89-L92

Thank you for your reply.
When the i===0, there is no param changed. Execute computeCoordsFromPlacement again just for placement change, I think it is unnecessary.  If put it in the if (reset) ..., I can understand it.
By the way, I think floating-ui is a very useful base-library. Do you plan to support multi-language about Doc? And I'm a Chinese developer, if you need I'm happy to translate it to Chinese.

It's necessary because of reset restarting the loop with i = -1, the base placement coords must be recomputed for the new placement
Anyway I've actually moved it to a better spot:
https://github.com/floating-ui/floating-ui/blob/188efd7b71318a7e800206d08ec8c276dff63b13/packages/core/src/computePosition.ts#L82-L95

By the way, I think floating-ui is a very useful base-library. Do you plan to support multi-language about Doc? And I'm a Chinese developer, if you need I'm happy to translate it to Chinese.

Sure, why not as long as you will maintain it :) Probably need to add a translations UI to the website too.

OK, I'm happy to and will do it in my spare time, the process may not be very fast, but I will do it as possible as I can. If I accomplish some, I will make a PR let you know.


[SPARK-34147][SQL][TEST] Keep table partitioning in TPCDSQueryBenchmak when CBO is enabled

What changes were proposed in this pull request?
This PR keeps partitioning of input tables in TPCDSQueryBenchmark when --cbo option is enabled.
https://github.com/apache/spark/pull/31011 introduced the --cbo option but unfortunately in that mode the table partitioning of the input data is lost. This means that the results of CBO mode is very different to non CBO mode, one example is that Dynamic Partition Pruning doesn't kick in in CBO mode.
Why are the changes needed?
To monitor performance changed with CBO enabled.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Manually checked.

cc @maropu, @dongjoon-hyun

cc @maropu, @dongjoon-hyun

Test build #134163 has started for PR 31218 at commit 5058a8e.

Test build #134163 has started for PR 31218 at commit 5058a8e.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38746/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38746/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38746/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38746/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38746/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38746/

Thank you for pinging me, @peter-toth .

Thank you for pinging me, @peter-toth .

Test build #134163 has finished for PR 31218 at commit 5058a8e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134163 has finished for PR 31218 at commit 5058a8e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134163/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134163/

@peter-toth, looks like we should better have a config that's disabled by default to prevent affecting benchmark results generated so far.

@peter-toth, looks like we should better have a config that's disabled by default to prevent affecting benchmark results generated so far.

Test build #134191 has started for PR 31218 at commit 976b19b.

Test build #134191 has started for PR 31218 at commit 976b19b.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38777/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38777/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38777/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38777/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38777/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38777/

Test build #134191 has finished for PR 31218 at commit 976b19b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134191 has finished for PR 31218 at commit 976b19b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134191/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134191/

cc @wangyum

cc @wangyum

@maropu can you take a quick look for doubly sure when you find some time?

@maropu can you take a quick look for doubly sure when you find some time?

Test build #134410 has started for PR 31218 at commit 7eda469.

Test build #134410 has started for PR 31218 at commit 7eda469.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38997/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38997/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38997/

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38997/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38997/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38997/

Test build #134410 has finished for PR 31218 at commit 7eda469.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #134410 has finished for PR 31218 at commit 7eda469.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134410/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134410/

Merged to master.

Merged to master.

Thanks all for the review.

Thanks all for the review.


Subscription lifecycle

I'm trying to verify the user's subscription lifecycle but I can't. Has anyone been able to see if the subscription is valid every time the user enters the app? Can you share the example with me?

I'm trying to verify the user's subscription lifecycle but I can't. Has anyone been able to see if the subscription is valid every time the user enters the app? Can you share the example with me?

It depends on the platform.
Google Play
Not beautiful, but simple. Since Google recommends restoring purchases every time you launch and resume the app, and the plugin only returns your currently active subscriptions, you just need to check that your subscription is in the list.
App Store
1st way - Apple recommended
Every time you want to check if your subscription is active, you'll need to validate receipt with Apple. This way you will get all the necessary information. There is no support in the plugin for this way and to get it right and safe without revealing your secrets you need to use a proxy server.
2nd way - without validating receipt
It may be easier to implement and works ok as long as subscription status is not a life changer. You need to remember user subscriptions yourself (you can get the list restoring purchases, it returns all active and not active subscriptions, but there is no info about current auto-renewal status). From the token you can decode subscription expiration date and compare it with the system date to check if it is active. Important thing (Apple's advice), you are not spouse of restoring purchases on every start because it may show the store login popup and disturb the user, so it's a better idea to check the current status Restore purchases) only if the user has expired subscription to see if there is a new renewal.
To decode data from token you may use this code:
using Claunia.PropertyList;
public static IDictionary<string, object> DecodeTokenAppStore(string token)
        {
            if (Device.RuntimePlatform == Device.Android)
            {
                return null;
            }

            if (String.IsNullOrWhiteSpace(token))
            {
                return null;
            }

            var data1 = System.Convert.FromBase64String(token);
            if (!data1.Any())
            {
                return null;
            }

            var receiptDic = (NSDictionary)PropertyListParser.Parse(data1);
            if (receiptDic == null)
            {
                return null;
            }

            var purchaseInfoContent = receiptDic.ObjectForKey("purchase-info")?.ToString();
            if (string.IsNullOrEmpty(purchaseInfoContent))
            {
                return null;
            }

            byte[] data2 = System.Convert.FromBase64String(purchaseInfoContent);
            var purchaseInfoDic = (NSDictionary)PropertyListParser.Parse(data2);

            if (purchaseInfoDic == null)
            {
                return null;
            }

            var result = new Dictionary<string, object>();

            foreach (var item in purchaseInfoDic)
            {
                result.Add(item.Key, item.Value.ToObject());
            }

            return result;
        }

I managed to solve this problem using this link:
https://www.programmersought.com/article/5211123151/
But I got into another problem, Google asks to confirm the purchase within 3 days
Reading the documentation I found this:
await billing.AcknowledgePurchaseAsync(PurchaseToken);
but every time I run this section it creates an order on the new console the purchase token remains the same but the order number varies, is that correct?
Will he charge the user every time he performs this stretch?
await billing.AcknowledgePurchaseAsync(PurchaseToken);

2nd way - without validating receipt
..... From the token you can decode subscription expiration date and compare it with the system date to check if it is active. Important thing (Apple's advice), you are not spouse of restoring purchases on every start because it may show the store login popup and disturb the user, so it's a better idea to check the current status (restore purchases) only if the user has expired subscription to see if there is a new renewal.

Hi I've managed to get the expiry date by taking "expires-date-formatted-pst" from the token but it is in this format:

"2021-06-30 12:07:44 America/Los_Angeles"

How did you convert this to a DateTime?
I want to convert it into UTC so that i can compare with DateTime.UtcNow

@ssbssb-cmd something like that:
var token = ShopService.DecodeTokenAppStore(PurchaseToken);

if (token != null && token.TryGetValue("expires-date", out var expiresDateAsStr))
{
 ExpireDate = new DateTime(1970, 1, 1).AddMilliseconds(Convert.ToInt64(expiresDateAsStr));
}

Thanks!!
So what I am planning to do with my app is get the expiry date, store it in the SecureStorage. Then each time the app loads, I will check if the expiryDate is greater than today's date. If it is not, call Restore Purchases and see if there is a newer expiry date which indicates the subscription is still active.
However - it sounds like couple of edge cases:

Grace Period where payment has failed.
If the user's payment has failed, and there is a grace period, will the token returned show the expiry date as the end of the Grace Period (e.g. 5 days)? or as the end of the subscription had they renewed it (e.g. 3 mths)?

Refunds
Once the expiry date has been stored, if the user cancels the subscription and requests a refund which Apple grants, there won't be any way to know that until I check restore purchases again at the end of the billing period.

Also you mention in the earlier post Option 1) to validate receipt with Apple - is the receipt the Purchase Token? I would need to send that to my server and then my server to get the membership status from Apple if going that route?

As I mentioned before, option 1 works when the subscription is not a "life and death" issue.
@1 I did something like that, when app starts:
1.Check locally that the subscription is active, if so continue as usual
2.if not, try to restore purchases (if possible without any user information)
3.If there is a new subscription for the next period, just write down the new expiration date and continue as usual
4.if it was not possible to restore (internet connection) or it was possible to restore, but there is no active subscription I show the user that the subscription is about to expire and if he did not renew it before its expiry + grace date the bonus will not be available (but will retain premium features), the next time the app is launched I will try the same steps again
5. If the grace period + 1 day (1 day just in case) has arrived and there is still no renewal, I inform the user that the premium is deactivated and that to continue, he must re-subscribe (and this time not bother user again with any more messages, maybe he just decided to cancel premium)
@2 I believe first of all, the best way is not to care about it. If I remember correctly, even Apple advices in some guidelines to not cancel user service if he asked refund. Apple doesn't cancel user payment easily, so probably it is not going to be a real problem, and for sure it is not the reason for bother paying users with any additional testing.

Hi,
Came here to find out how to check if an active subscription is present.
Reading above, can I do this on iOS if I don't care about refunds? And assuming there's only one subscription for simplicity of the example:
var purchases = await billing.GetPurchasesAsync(ItemType.Subscription);
var latestSub = new InAppBillingPurchase();

foreach (var item in purchases)
                {
                    if (latestSub.TransactionDateUtc.CompareTo(item.TransactionDateUtc) == -1)
                    {
                        //newer
                        latestSub = item;
                    }
                }

//add 1 month of 31 days. Plus three trial days.

Console.WriteLine($"{latestSub.ProductId} {latestSub.State}: {latestSub.TransactionDateUtc} Expires: {latestSub.TransactionDateUtc.AddDays(34)}");

//compare with system date here.

Thanks for any clarification.

See https://montemagno.com/ios-android-subscription-implemenation-strategies/


Updated Docker

This is already submitted in https://github.com/koding/stacks/pull/1.

This is an updated version after @gokmen  and @rjeczalik suggestions.

https://github.com/koding/stacks/pull/4


Missing requirements

Compare to ovos-listener,  ovos-dinkum-listener is missing some libraries.
Here is the list that I'm using to have a functional ovos-dinkum-listener with wake word and VAD detection.
ovos-stt-plugin-server
ovos-stt-plugin-vosk
ovos-vad-plugin-silero
ovos-ww-plugin-pocketsphinx
ovos-ww-plugin-precise
ovos-ww-plugin-precise-lite
ovos-ww-plugin-vosk

Since those are all plugins they should not be dragged by default, they are large dependencies that might not be used and would not allow this to be installed in some platforms, an extra for default suggested plugins makes sense

Since those are all plugins they should not be dragged by default, they are large dependencies that might not be used and would not allow this to be installed in some platforms, an extra for default suggested plugins makes sense

By default VAD and wake words are enabled in the configuration which means that a new user, will automatically get errors when starting Dinkum.

the user should choose a plugin before launching dinkum, it doesnt work without one we just shouldnt be selecting one for the user
silero wont install in some platforms and is the recommended plugin, but cant be added to requirements
wake words similarly try to use the best plugin whatever is available, we should not bundle one
maybe we can add a [recommended] extras requirement?

the user should choose a plugin before launching dinkum, it doesnt work without one we just shouldnt be selecting one for the user

Yeah but people starts the service directly without changing any configuration because they want to test the solution. Maybe we could prevent the service to start with an INFO/ERROR if no configuration is provided.

silero wont install in some platforms and is the recommended plugin, but cant be added to requirements

Not required by default, I made a mistake, its webrtcvad :p

maybe we can add a [recommended] extras requirement?

Agree.

As a new user, I would be happy with a block flowchart  of how the system worked, with mutually exclusive plugin options associated with, or printed in, each block.
For example, I'm getting a bit lost in the plugins as I try to build an ovos instance with dinkum-listener. I'm using raspbian on a 3B+ and there is a dependency on glibc 2.33 somewhere under ovos-ww-plugin-precise-lite. Rasbpian is using glibc 2.31 and so I need another solution. I'm busy reading the docs and figuring it out.  Other people might not be so interested in spending their time this way, and I worry we will lose them as potential users.
It would be great if there was a table or flowchart that would list the plugins required to make the different options work.


fetchurl: accept SRI in hex and base32

Proof of concept stemming from https://github.com/NixOS/rfcs/pull/131
Problem statement:

single attribute name for fetch* hashes would be nice; the currently most common are sha256 and hash
copy&paste of hash from upstream seems attractive (e.g. https-secured release announcements)
SRI hashes (hash attribute on nixpkgs level) only support base64 encoding
upstreams don't use base64 encoding commonly
implementing on nixpkgs level allows immediate usage; with nix we'd have to wait at least a year, I think

base16 hashes seem way more widespread in practice than base64/SRI, so my point of view is more like moving to a "less standard" approach.
But I'm glad I got at least some feedback on this PR.

The tooling argument: as the original value is base16 typically, you either need to complicate the tools that generate nix expressions (conversion to base64; including humans copy&pasting so far) or those that consume the nix expressions (accept more than base64).  I don't see why the first would be clearly superior.
I thought it's better to have an option to keep the same format as what is published elsewhere (typically).  That option has always been here through attributes like sha256; this PR allows to exercise it additionally with the hash attribute by trivial textual transformation, as I understand that uniformity in attribute names can be useful (we had issues with overriding).
Of course it's about opinions, to some extent 🤷🏽

nix-prefetch-url and others emit base32 by default.  An the nix command is officially still experimental.  IMO it's good to start with such places, as base32 is certainly the worst part here.  (Ideally making the nix command officially stable, at least some parts of it.)

Yes, but I suspect that stabilized implies that changing the default output format won't be accepted.


Macros needed for MinGW for compilation

It wont compile because of conflicting function names between winuser.h and its own function.
Unable to compile on MinGW/32, I added these lines to one of your C files around #include <windows.h> to get it to compile:
#define CloseWindow CloseWindowDontUse
#define ShowCursor ShowCursorDontUse
#define UnhideWindow UnHideWindowDontUse
#include <windows.h>
#undef UnhideWindow
#undef ShowCursor
#undef CloseWindow
For the record, I tried this on revision: ec09fea29aa404d578486a91ddc3663b61d95939

Closing the issue. Already discussing it in the PR.


ASP.NET Core Webapp on Net Framework template fails to build

Problem
The ASP.NET core web app on framework  template does not build out of the box.
Details
The ASP.NET core web app on framework  template references NETStandard.Library.NETFramework version 2.0.0-preview2-25405-01. But the package does not exist on nuget.org or dotnet-core feed
.
As a result you cant build a new project.
csproj file
<Project Sdk="Microsoft.NET.Sdk.Web">

  <PropertyGroup>
    <TargetFramework>net461</TargetFramework>
    <MvcRazorCompileOnPublish>true</MvcRazorCompileOnPublish>
    <AssetTargetFallback>$(AssetTargetFallback);portable-net45+win8+wp8+wpa81;</AssetTargetFallback>
    <UserSecretsId>aspnet-WebApplication3-2DA14627-6636-44B0-B49E-AC1545A2C132</UserSecretsId>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="NETStandard.Library.NETFramework" Version="2.0.0-preview2-25405-01" />
    <PackageReference Include="Microsoft.AspNetCore" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Razor.ViewCompilation" Version="2.0.0-preview2-final" PrivateAssets="All" />
    <PackageReference Include="Microsoft.AspNetCore.StaticFiles" Version="2.0.0-preview2-final" />
    <PackageReference Include="Microsoft.VisualStudio.Web.BrowserLink" Version="2.0.0-preview2-final" />
  </ItemGroup>

  <ItemGroup>
    <DotNetCliToolReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Tools" Version="2.0.0-preview2-final" />
  </ItemGroup>

</Project>

Data
VS: d15Rel/26618.0
NuGet: <IP_ADDRESS>68
netcore sdk: 2.0.0-preview2
Please feel free to move this to another repo if this is not the best place for this.

This issue is being closed because it has not been updated in 3 months.
We apologize if this causes any inconvenience. We ask that if you are still encountering this issue, please log a new issue with updated information and we will investigate.


Allow creating KafkaTopic CR for topic that already present on the Kafka cluster

Q
A

Bug fix?
no

New feature?
yes

API breaks?
no

Deprecations?
no

License
Apache 2.0

What's in this PR?
It allows to create a KafkaTopic CR when the referenced topic is already present on the Kafka cluster.
Why?

Let the user create KafkaTopic resources for that topic which is already on the Kafka cluster
Users can modify the topic configuration and partition number for that topic through the created KafkaTopic CR
Checklist

[x] Implementation tested
[x] Error handling code meets the guideline
[x] Logging code meets the guideline
[x] User guide and development docs updated (if needed)

What's the use case behind this?

@hi-im-aren  One example use case is for the replicated/migrated Kafka topics: for example, if users want to migrate their legacy Kafka clusters to under Koperator's management, they will need to create the KafkaTopic CRs after the replication/migration is done so Koperator can start managing the actual underline Kafka resources

Not sure that the correct way of doing this is removing the validation from the webhook. We are removing a feature because it gets in the way 1% of the time while it's useful in 99% of other use cases.

If the topic was created without a CR then it's either an internal topic, which means there is probably an other system managing this topic already

It will not try to create I checked it. It checks its configuration and when it differs from the actual topic then the Koperator will sync the new configuration to the actual one.
https://github.com/banzaicloud/koperator/blob/635bab34ad604caf9f5b8a6afbab251196fb13d4/controllers/kafkatopic_controller.go#L123

Not sure that the correct way of doing this is removing the validation from the webhook. We are removing a feature because it gets in the way 1% of the time while it's useful in 99% of other use cases:

Im not sure about this why so useful is. For me It is much better to get all of the topic into KafkaTopic CR so somebody who doesn't like dashboards can check all of the presented topic through kubernetes.
I also can change partition count and topic configuration for those topics.
Can you collect your points against this modification so we can argue on those points in parking lot or on a meeting to find what would be the best. (I know this is some kind of protection against noob users)

What's the use case behind this?

If the topic was created without a CR then it's either an internal topic, which means there is probably an other system managing this topic already
or the user created it manually inside kafka which means they know what they are doing and skipped the CR creation on purpose

Also I think just because we remove the validation from the webhook, kafka will still come back with an error that the topic already exists because our reconciler will try to create the topic in kafka.

This is also my case:

I started using koperator in 2018 or so
There was no KafkaTopic cr or maybe I overlooked it
I'm managing topics with a custom scripts
However want to migrate to KafkaTopic cr without deletion anything

The problem is when the topic is internal or it is managed by 3rd party application and we let the user to create KafkaTopic CR for it

then the user can modify topic configs (partition number ...).
when it is modified and the kafka or 3rd party application change its configuration then corresponding KafkaTopic will be out of sync.

Solution A

We should add some kind of confirmation protection when the user want to create KafkaTopic cr for a present topic. When a KafkaTopic has a specific label like: unmanaged=true,  then the webhook checks is the present topic differs from the KafkaTopic CR and if yes, then the webhook reject that KafkaTopic with error message.
When there is an annotation allowconfigmodify=true then the user can modify the topic values.
We cant guarantee that the settings what are in the KafkaTopic is the same as the present topic's settings

Solution B

We can make a controller which purpose to create KafkaTopic for the kafka topics and do a sync time by time.
There would be unmanaged=true label by default for the internal topics. Users can only change those KafkaTopic CR settings when they specify allowmodifyconfig=true.
There would be a timestamp when the configuration is synced by the controller.
the unmanaged=true label cannot be removed (webhook check) from those KafkaTopics that reflect to those topics that are not created by the user.

This PR has been closed because it is re-created here: https://github.com/banzaicloud/koperator/pull/934


This relative module was not found:  * ./components/common/Loading

Runtime error:
This relative module was not found:  * ./components/common/Loading
Author forget  capitalize the first letter to "loading.vue"
@daoshengfu

已解决，上传的时候粗心了

已解决，上传的时候粗心了
强👍


Update eval for Scala 3.3.1

Well actually it seems that eval is not used and that there is a local copy of it in https://github.com/sbt/sbt/blob/develop/buildfile/src/main/scala/sbt/internal/Eval.scala.
@eed3si9n what's the plan here? Should we keep the local copy of Eval or should we use the library instead?

If the behavior is stable maybe we should switch to the library?


[SPARK-25009][CORE]Standalone Cluster mode application submit is not working

What changes were proposed in this pull request?
It seems 'doRunMain()' has been removed accidentally by other PR and due to that the application submission is not happening, this PR adds back the 'doRunMain()' for standalone cluster submission.
How was this patch tested?
I verified it manually by submitting application in standalone cluster mode, all the applications are submitting to the Master with the change.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@vanzin @squito, it's probably deleted by mistake in this commit: https://github.com/devaraj-kavali/spark/commit/3cb82047f2f51af553df09b9323796af507d36f8

LGTM. Wait for the build.

thanks for catching this.  lgtm

Test build #94082 has finished for PR 21979 at commit e753ff8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94082/
Test PASSed.

Merged into master. Thanks.


MOFHY Lite isnt safe

please check my fork
https://github.com/ImLoadingUuU/MOFHY-Lite/security/code-scanning
I scanned it using SonarCloud.it has a lot of security Problem

K https://sonarcloud.io/summary/overall?id=ImLoadingUuU_MOFHY-Lite

I 100% agree that the 44 vulnerabilities need to be fixed. The ones labeled "Bug" are just the program complaint about put-of-date (But still valid) HTML tags. They should be changed, but no rush. The ones labeled "Code Smell" just seem to be dev suggestions that can be ignored.
But those 44 vulnerabilities need to be fixed ASAP.
Direct link for @mahtab2003: https://sonarcloud.io/project/issues?resolved=false&types=VULNERABILITY&id=ImLoadingUuU_MOFHY-Lite

Ok let me fix

@mahtab2003 u can scan it yourself, its free but only for Open Source :>

Ok


Safely exiting in case of error

In case of incomplete download of the firmware file, erasing the flash may start before the reboot

@cronyx
Hello
Look what an interesting situation, I think we need to consider it the same way.
Some process prevented the device from rebooting and the process of executing script commands continued.
I believe that the fuse proposed by the guys from Faceter should be used, there are no questions about it.
However, maybe we should make the “reboot” command more strict, for example with the -f switch?