Add NRT c++ functions

This PR adds device side implementations of some of the NRT c-api. Combined with the change to the CUDATargetContext object, this allows launching of kernels like this:
from numba import cuda
import numpy as np

@cuda.jit
def f(x):
    return x[:5]

@cuda.jit('void()', link=['nrt.cu'])
def g():
    x = cuda.shared.array(10, dtype=np.int32)
    f(x)

g[1,1]()

Several things are as of the time of writing missing:

Currently, no memsys is used and it remains to be discussed how we'd like to expose it, knowing that it may have to be used outside of numba to free objects that persist after the kernel finishes executing.
No tests are currently included
We should discuss how and when we'd like to include these functions in the link. Perhaps some of its compilation might be cached and also it might not be necessary to add it to every link, only the ones that require nrt.

Some additional changes needed for adding the nrt test module: https://github.com/gmarkall/numba-cuda/commit/9bb662d7ca8e5319bc1507f1aa01133cad24c532

WIth the changes from my branch above, and syncing before the original example exits:
from numba import cuda
import numpy as np

@cuda.jit
def f(x):
    return x[:5]

@cuda.jit
def g():
    x = cuda.shared.array(10, dtype=np.int32)
    f(x)

g[1,1]()
cuda.synchronize()

we get:
numba.cuda.cudadrv.driver.CudaAPIError: [700] Call to cuCtxSynchronize results in UNKNOWN_CUDA_ERROR

I presume this also happens for you, but without the synchronize at the end, Python exits before the error is detected?

I see, it looks like the meminfo is null:
========= Invalid __global__ atomic of size 8 bytes
=========     at NRT_incref+0xb0
=========     by thread (0,0,0) in block (0,0,0)
=========     Address 0x0 is out of bounds
=========     and is 8,703,180,800 bytes before the nearest allocation at 0x206c00000 of size 8,388,864 bytes
=========     Device Frame:cudapy::__main__::g[abi:v1,cw51cXTLSUwv1sCUt9Ww0FGw1NRRwGKFLSgtAWVgaQPKRwoMW3A4A4MbLeahKhGJDEkANeHAJJA8YlutDgtedSsFaDSpAyWBPONaTQA_3d]+0x40 in /home/gmarkall/numbadev/issues/numba-cuda-17/original_repro.py:10

/ok to test

From CI:
numba.cuda.cudadrv.error.NvrtcError: NVRTC Compilation failure whilst compiling nrt.cu:

nrt.cu(4): catastrophic error: cannot open source file "cuda/atomic"

I wonder if there are additional CUDA toolkit packages on which this introduces a depdencdency. Perhaps we need to install more in the CI environment (or perhaps the NVRTC include path is just set incorrectly).

From CI:
numba.cuda.cudadrv.error.NvrtcError: NVRTC Compilation failure whilst compiling nrt.cu:

nrt.cu(4): catastrophic error: cannot open source file "cuda/atomic"

I wonder if there are additional CUDA toolkit packages on which this introduces a depdencdency. Perhaps we need to install more in the CI environment (or perhaps the NVRTC include path is just set incorrectly).

I think cuda/atomic is meant to come in from libcudacxx. I'm not sure numba has this dependency yet. I will follow up here

I think Numba is still not finding the headers because it's not necessarily looking in the right place for them - we have an environment variable to set the CUDA include path, but we probably need to find a way for Numba to find these headers when they're installed elsewhere.

I think Numba is still not finding the headers because it's not necessarily looking in the right place for them - we have an environment variable to set the CUDA include path, but we probably need to find a way for Numba to find these headers when they're installed elsewhere.

Looks like when the cccl package is installed, there's a number of places the files can end up. I have a hunch that the real way of fixing this is to rework things in terms of cmake so that cccl (as well as other pieces of cuda that are relied on) are discoverable in any environment seamlessly. Then we can embed the paths from cmake into a file and numba can pick them up from there. I will seek some feedback on this idea.
Without that, we might be stuck with a separate environment variable in the short term.

/ok to test

/ok-to-test

/ok to test

/ok to test


Return dev.py to running all tests for test or tests.

Travis tests have failed
Hey @bakert,
Please read the following log in order to understand the failure reason.
 If this is happening on user time that's undesirable.
CONFIG: slow_page=10.0
----------------------------- Captured stderr call -----------------------------
[2019-11-17 11:36:01,834] ERROR in app: Exception on / [GET]
Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 52, in execute_anything
    return self.execute_with_reconnect(sql, args, fetch_rows)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 68, in execute_with_reconnect
    n = self.cursor.execute(sql, args)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/connections.py", line 231, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.ProgrammingError: (1146, "Table 'decksite._card_stats' doesn't exist")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/decorators.py", line 11, in wrapper
    return decorated_func(*args, **kwargs)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 250, in load_cards
    cs = [Container(r) for r in db().select(sql)]
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 41, in select
    [_, rows] = self.execute_anything(sql, args)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 60, in execute_anything
    raise DatabaseException('Failed to execute `{sql}` with `{args}` because of `{e}`'.format(sql=sql, args=args, e=e))
shared.pd_exception.DatabaseException: Failed to execute `
        SELECT
            name,
            SUM(num_decks) AS num_decks,
            SUM(wins) AS wins,
            SUM(losses) AS losses,
            SUM(draws) AS draws,
            SUM(wins - losses) AS record,
            SUM(perfect_runs) AS perfect_runs,
            SUM(tournament_wins) AS tournament_wins,
            SUM(tournament_top8s) AS tournament_top8s,
            IFNULL(ROUND((SUM(wins) / NULLIF(SUM(wins + losses), 0)) * 100, 1), '') AS win_percent
        FROM
            _card_stats AS cs
        WHERE
            (TRUE) AND (season_id = 14)
        GROUP BY
            name
        ORDER BY
            num_decks DESC,
            record,
            name
    ` with `[]` because of `(1146, "Table 'decksite._card_stats' doesn't exist")`

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 52, in execute_anything
    return self.execute_with_reconnect(sql, args, fetch_rows)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 82, in execute_with_reconnect
    raise e
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 68, in execute_with_reconnect
    n = self.cursor.execute(sql, args)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/connections.py", line 231, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1055, "Expression #11 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'decksite.ct.name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/cache.py", line 80, in decorated_function
    response = make_response(f(*args, **kwargs))
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/main.py", line 49, in home
    view = Home(ns.all_news(max_items=10), ds.latest_decks(), cs.load_cards(season_id=get_season_id()), ms.stats())
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/decorators.py", line 14, in wrapper
    retry_func()
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 35, in preaggregate
    preaggregate_card_person()
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 133, in preaggregate_card_person
    preaggregation.preaggregate(table, sql)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/preaggregation.py", line 13, in preaggregate
    db().execute(sql)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 45, in execute
    [n, _] = self.execute_anything(sql, args, False)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 60, in execute_anything
    raise DatabaseException('Failed to execute `{sql}` with `{args}` because of `{e}`'.format(sql=sql, args=args, e=e))
shared.pd_exception.DatabaseException: Failed to execute `
        CREATE TABLE IF NOT EXISTS _new_card_person_stats (
            name VARCHAR(190) NOT NULL,
            season_id INT NOT NULL,
            person_id INT NOT NULL,
            wins INT NOT NULL,
            losses INT NOT NULL,
            draws INT NOT NULL,
            perfect_runs INT NOT NULL,
            tournament_wins INT NOT NULL,
            tournament_top8s INT NOT NULL,
            deck_type ENUM('league', 'tournament', 'other') NOT NULL,
            PRIMARY KEY (season_id, person_id, name, deck_type),
            FOREIGN KEY (season_id) REFERENCES season (id) ON UPDATE CASCADE ON DELETE CASCADE,
            FOREIGN KEY (person_id) REFERENCES person (id)  ON UPDATE CASCADE ON DELETE CASCADE,
            INDEX idx_person_id_name (person_id, name)
        ) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci AS
        SELECT
            card AS name,
            season.id AS season_id,
            d.person_id,
            SUM(CASE WHEN d.id IS NOT NULL THEN 1 ELSE 0 END) AS num_decks,
            IFNULL(SUM(wins), 0) AS wins,
            IFNULL(SUM(losses), 0) AS losses,
            IFNULL(SUM(draws), 0) AS draws,
            SUM(CASE WHEN wins >= 5 AND losses = 0 AND d.source_id IN (SELECT id FROM source WHERE name = 'League') THEN 1 ELSE 0 END) AS perfect_runs,
            SUM(CASE WHEN dsum.finish = 1 THEN 1 ELSE 0 END) AS tournament_wins,
            SUM(CASE WHEN dsum.finish <= 8 THEN 1 ELSE 0 END) AS tournament_top8s,
            (CASE WHEN ct.name = 'League' THEN 'league' WHEN ct.name = 'Gatherling' THEN 'tournament' ELSE 'other' END) AS deck_type
        FROM
            deck AS d
        INNER JOIN
            -- Eiliminate maindeck/sideboard double-counting with DISTINCT. See #5493.
            (SELECT DISTINCT card, deck_id FROM deck_card) AS dc ON d.id = dc.deck_id

        LEFT JOIN
            competition AS c ON d.competition_id = c.id
        LEFT JOIN
            competition_series AS cs ON cs.id = c.competition_series_id
        LEFT JOIN
            competition_type AS ct ON ct.id = cs.competition_type_id

        LEFT JOIN
            (
                SELECT
                    `start`.id,
                    `start`.code,
                    `start`.start_date AS start_date,
                    `end`.start_date AS end_date
                FROM
                    season AS `start`
                LEFT JOIN
                    season AS `end` ON `end`.id = `start`.id + 1
            ) AS season ON season.start_date <= d.created_date AND (season.end_date IS NULL OR season.end_date > d.created_date)

        LEFT JOIN
            (
                SELECT
                    d.id,
                    d.created_date,
                    d.finish,
                    SUM(CASE WHEN dm.games > IFNULL(odm.games, 0) THEN 1 ELSE 0 END) AS wins, -- IFNULL so we still count byes as wins.
                    SUM(CASE WHEN dm.games < odm.games THEN 1 ELSE 0 END) AS losses,
                    SUM(CASE WHEN dm.games = odm.games THEN 1 ELSE 0 END) AS draws
                FROM
                    deck_match AS dm
                INNER JOIN
                    deck_match AS odm ON dm.match_id = odm.match_id AND dm.deck_id <> odm.deck_id
                INNER JOIN
                    deck AS d ON d.id = dm.deck_id
                GROUP BY
                    d.id
            ) AS dsum ON d.id = dsum.id

        GROUP BY
            card,
            d.person_id,
            season.id
    ` with `[]` because of `(1055, "Expression #11 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'decksite.ct.name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by")`
[2019-11-17 11:36:01,837] ERROR in logger: At request path: /
werkzeug.exceptions.InternalServerError: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

500 error at /
 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Reported on decksite by logged_out

InternalServerError
500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.

Stack Trace:

Python traceback
File "dev.py", line 310, in
run()
File "dev.py", line 23, in run
run_dangerously()
File "dev.py", line 51, in run_dangerously
pytest(args, '')
File "dev.py", line 160, in pytest
code = pytest.main(args)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/config/init.py", line 90, in main
return config.hook.pytest_cmdline_main(config=config)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/main.py", line 230, in pytest_cmdline_main
return wrap_session(config, _main)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/main.py", line 193, in wrap_session
session.exitstatus = doit(config, session) or 0
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/main.py", line 237, in _main
config.hook.pytest_runtestloop(session=session)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/main.py", line 258, in pytest_runtestloop
item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 80, in pytest_runtest_protocol
runtestprotocol(item, nextitem=nextitem)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 95, in runtestprotocol
reports.append(call_and_report(item, "call", log))
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 176, in call_and_report
call = call_runtest_hook(item, when, **kwds)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 201, in call_runtest_hook
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 229, in from_call
result = func()
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 201, in
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in call
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/manager.py", line 86, in
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/runner.py", line 125, in pytest_runtest_call
item.runtest()
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/unittest.py", line 207, in runtest
self._testcase(result=self)
File "/opt/python/3.6.7/lib/python3.6/unittest/case.py", line 653, in call
return self.run(*args, **kwds)
File "/opt/python/3.6.7/lib/python3.6/unittest/case.py", line 601, in run
self.setUp()
File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/smoke_test.py", line 11, in setUp
self.tester: SmokeTester = SmokeTester(APP)
File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared_web/smoke.py", line 9, in init
self.base_tests()
File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared_web/smoke.py", line 12, in base_tests
self.response_test('/', 200)
File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared_web/smoke.py", line 20, in response_test
result = self.test_client.get(path) # type: ignore
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/werkzeug/test.py", line 1029, in get
return self.open(*args, **kw)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/testing.py", line 227, in open
follow_redirects=follow_redirects,
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/werkzeug/test.py", line 993, in open
response = self.run_wsgi_app(environ.copy(), buffered=buffered)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/werkzeug/test.py", line 884, in run_wsgi_app
rv = run_wsgi_app(self.application, environ, buffered=buffered)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/werkzeug/test.py", line 1119, in run_wsgi_app
app_rv = app(environ, start_response)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 2463, in call
return self.wsgi_app(environ, start_response)
File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app
response = self.handle_exception(e)

Exception_hash: c70a8a7ffd03715a878b6595150576ef2cb89cc6
Request Data

Request Method: GET
Path: /?
Cookies: {}
Endpoint: home
View Args: {}
Person: logged_out
Referrer: None
Request Data: {}
User-Agent: werkzeug/0.16.0
Host: localhost

------------------------------ Captured log call -------------------------------
ERROR    decksite:app.py:1891 Exception on / [GET]
Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 52, in execute_anything
    return self.execute_with_reconnect(sql, args, fetch_rows)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 68, in execute_with_reconnect
    n = self.cursor.execute(sql, args)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/connections.py", line 231, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.ProgrammingError: (1146, "Table 'decksite._card_stats' doesn't exist")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/decorators.py", line 11, in wrapper
    return decorated_func(*args, **kwargs)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 250, in load_cards
    cs = [Container(r) for r in db().select(sql)]
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 41, in select
    [_, rows] = self.execute_anything(sql, args)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 60, in execute_anything
    raise DatabaseException('Failed to execute `{sql}` with `{args}` because of `{e}`'.format(sql=sql, args=args, e=e))
shared.pd_exception.DatabaseException: Failed to execute `
        SELECT
            name,
            SUM(num_decks) AS num_decks,
            SUM(wins) AS wins,
            SUM(losses) AS losses,
            SUM(draws) AS draws,
            SUM(wins - losses) AS record,
            SUM(perfect_runs) AS perfect_runs,
            SUM(tournament_wins) AS tournament_wins,
            SUM(tournament_top8s) AS tournament_top8s,
            IFNULL(ROUND((SUM(wins) / NULLIF(SUM(wins + losses), 0)) * 100, 1), '') AS win_percent
        FROM
            _card_stats AS cs
        WHERE
            (TRUE) AND (season_id = 14)
        GROUP BY
            name
        ORDER BY
            num_decks DESC,
            record,
            name
    ` with `[]` because of `(1146, "Table 'decksite._card_stats' doesn't exist")`

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 52, in execute_anything
    return self.execute_with_reconnect(sql, args, fetch_rows)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 82, in execute_with_reconnect
    raise e
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 68, in execute_with_reconnect
    n = self.cursor.execute(sql, args)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
    res = self._query(query)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
    db.query(q)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/MySQLdb/connections.py", line 231, in query
    _mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1055, "Expression #11 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'decksite.ct.name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by")

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/cache.py", line 80, in decorated_function
    response = make_response(f(*args, **kwargs))
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/main.py", line 49, in home
    view = Home(ns.all_news(max_items=10), ds.latest_decks(), cs.load_cards(season_id=get_season_id()), ms.stats())
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/decorators.py", line 14, in wrapper
    retry_func()
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 35, in preaggregate
    preaggregate_card_person()
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/card.py", line 133, in preaggregate_card_person
    preaggregation.preaggregate(table, sql)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/data/preaggregation.py", line 13, in preaggregate
    db().execute(sql)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 45, in execute
    [n, _] = self.execute_anything(sql, args, False)
  File "/home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/shared/database.py", line 60, in execute_anything
    raise DatabaseException('Failed to execute `{sql}` with `{args}` because of `{e}`'.format(sql=sql, args=args, e=e))
shared.pd_exception.DatabaseException: Failed to execute `
        CREATE TABLE IF NOT EXISTS _new_card_person_stats (
            name VARCHAR(190) NOT NULL,
            season_id INT NOT NULL,
            person_id INT NOT NULL,
            wins INT NOT NULL,
            losses INT NOT NULL,
            draws INT NOT NULL,
            perfect_runs INT NOT NULL,
            tournament_wins INT NOT NULL,
            tournament_top8s INT NOT NULL,
            deck_type ENUM('league', 'tournament', 'other') NOT NULL,
            PRIMARY KEY (season_id, person_id, name, deck_type),
            FOREIGN KEY (season_id) REFERENCES season (id) ON UPDATE CASCADE ON DELETE CASCADE,
            FOREIGN KEY (person_id) REFERENCES person (id)  ON UPDATE CASCADE ON DELETE CASCADE,
            INDEX idx_person_id_name (person_id, name)
        ) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci AS
        SELECT
            card AS name,
            season.id AS season_id,
            d.person_id,
            SUM(CASE WHEN d.id IS NOT NULL THEN 1 ELSE 0 END) AS num_decks,
            IFNULL(SUM(wins), 0) AS wins,
            IFNULL(SUM(losses), 0) AS losses,
            IFNULL(SUM(draws), 0) AS draws,
            SUM(CASE WHEN wins >= 5 AND losses = 0 AND d.source_id IN (SELECT id FROM source WHERE name = 'League') THEN 1 ELSE 0 END) AS perfect_runs,
            SUM(CASE WHEN dsum.finish = 1 THEN 1 ELSE 0 END) AS tournament_wins,
            SUM(CASE WHEN dsum.finish <= 8 THEN 1 ELSE 0 END) AS tournament_top8s,
            (CASE WHEN ct.name = 'League' THEN 'league' WHEN ct.name = 'Gatherling' THEN 'tournament' ELSE 'other' END) AS deck_type
        FROM
            deck AS d
        INNER JOIN
            -- Eiliminate maindeck/sideboard double-counting with DISTINCT. See #5493.
            (SELECT DISTINCT card, deck_id FROM deck_card) AS dc ON d.id = dc.deck_id

        LEFT JOIN
            competition AS c ON d.competition_id = c.id
        LEFT JOIN
            competition_series AS cs ON cs.id = c.competition_series_id
        LEFT JOIN
            competition_type AS ct ON ct.id = cs.competition_type_id

        LEFT JOIN
            (
                SELECT
                    `start`.id,
                    `start`.code,
                    `start`.start_date AS start_date,
                    `end`.start_date AS end_date
                FROM
                    season AS `start`
                LEFT JOIN
                    season AS `end` ON `end`.id = `start`.id + 1
            ) AS season ON season.start_date <= d.created_date AND (season.end_date IS NULL OR season.end_date > d.created_date)

        LEFT JOIN
            (
                SELECT
                    d.id,
                    d.created_date,
                    d.finish,
                    SUM(CASE WHEN dm.games > IFNULL(odm.games, 0) THEN 1 ELSE 0 END) AS wins, -- IFNULL so we still count byes as wins.
                    SUM(CASE WHEN dm.games < odm.games THEN 1 ELSE 0 END) AS losses,
                    SUM(CASE WHEN dm.games = odm.games THEN 1 ELSE 0 END) AS draws
                FROM
                    deck_match AS dm
                INNER JOIN
                    deck_match AS odm ON dm.match_id = odm.match_id AND dm.deck_id <> odm.deck_id
                INNER JOIN
                    deck AS d ON d.id = dm.deck_id
                GROUP BY
                    d.id
            ) AS dsum ON d.id = dsum.id

        GROUP BY
            card,
            d.person_id,
            season.id
    ` with `[]` because of `(1055, "Expression #11 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'decksite.ct.name' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by")`
ERROR    decksite:logger.py:12 At request path: /
                               werkzeug.exceptions.InternalServerError: 500 Internal Server Error: The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
=============================== warnings summary ===============================
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask_babel/__init__.py:19
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/flask_babel/__init__.py:19: DeprecationWarning: The import 'werkzeug.ImmutableDict' is deprecated and will be removed in Werkzeug 1.0. Use 'from werkzeug.datastructures import ImmutableDict' instead.
    from werkzeug import ImmutableDict

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.functional - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

decksite/cache.py:8
  /home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/decksite/cache.py:8: DeprecationWarning: 'werkzeug.contrib.cache' is deprecated as of version 0.15 and will be removed in version 1.0. It has moved to https://github.com/pallets/cachelib.
    from werkzeug.contrib.cache import SimpleCache

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.perf - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.tappedout - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.external - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.gatherling - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

/home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325
  /home/travis/virtualenv/python3.6.7/lib/python3.6/site-packages/_pytest/mark/structures.py:325: PytestUnknownMarkWarning: Unknown pytest.mark.goldfish - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/latest/mark.html
    PytestUnknownMarkWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
 generated xml file: /home/travis/build/PennyDreadfulMTG/Penny-Dreadful-Tools/TestResults.xml

=========================== slowest 2 test durations ===========================
0.59s call     decksite/smoke_test.py::DecksiteSmokeTest::test_some_pages
0.04s call     decksite/deck_name_test.py::test_normalize[Dimir Control-Dimir Control-colors0-Control]
=========================== short test summary info ============================
FAILED decksite/smoke_test.py::DecksiteSmokeTest::test_some_pages - Assertion...
================== 1 failed, 90 passed, 10 warnings in 22.26s ==================
>>>> Upload coverage
Storing https://codecov.io/bash in codecov.sh

  _____          _
 / ____|        | |
| |     ___   __| | ___  ___ _____   __
| |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
| |___| (_) | (_| |  __/ (_| (_) \ V /
 \_____\___/ \__,_|\___|\___\___/ \_/
                              Bash-20191111-ecb75bd

==> Travis CI detected.
    project root: .
    Fixing merge commit SHA
    Yaml found at: .codecov.yml
==> Running gcov in . (disable via -X gcov)
==> Python coveragepy exists disable via -X coveragepy
    -> Running coverage xml
==> Searching for coverage reports in:
    + .
    -> Found 1 reports
==> Detecting git/mercurial file structure
==> Appending build variables
    + TRAVIS_OS_NAME
    + 3.6
==> Reading reports
    + ./coverage.xml bytes=684449
==> Appending adjustments
    http://docs.codecov.io/docs/fixing-reports
    -> No adjustments found
==> Gzipping contents
==> Uploading reports
    url: https://codecov.io
    query: branch=master&commit=c557ae9833cd8714e6b29f0fadf251c0a49b75c9&build=7669.1&build_url=&name=&tag=&slug=PennyDreadfulMTG%2FPenny-Dreadful-Tools&service=travis&flags=&pr=6851&job=613065298
    -> Pinging Codecov
https://codecov.io/upload/v4?package=bash-20191111-ecb75bd&token=secret&branch=master&commit=c557ae9833cd8714e6b29f0fadf251c0a49b75c9&build=7669.1&build_url=&name=&tag=&slug=PennyDreadfulMTG%2FPenny-Dreadful-Tools&service=travis&flags=&pr=6851&job=613065298
    -> Uploading
    -> View reports at https://codecov.io/github/PennyDreadfulMTG/Penny-Dreadful-Tools/commit/c557ae9833cd8714e6b29f0fadf251c0a49b75c9
TestFailedException running ['dev.py', 'tests']:  [(<ExitCode.TESTS_FAILED: 1>,)] ExitCode.TESTS_FAILED

2nd Build
View build log

     echo $CMD

python dev.py mypy

     $CMD

$ $CMD
CONFIG: web_cache=.web_cache
CONFIG: redis_enabled=True
CONFIG: redis_host=localhost
CONFIG: redis_port=6379
CONFIG: redis_db=0
CONFIG: whoosh_index_dir=whoosh_index
CONFIG: image_dir=./images
CONFIG: production=False
>>>> Typechecking
dev.py:142: error: Function is missing a return type annotation
Found 1 error in 1 file (checked 278 source files)

Exit status: 1 (Failure)
TestFailedException running ['dev.py', 'mypy']:  [(1,)] 1

3rd Build
View build log

     echo $CMD

python dev.py lint

     $CMD

$ $CMD
CONFIG: web_cache=.web_cache
CONFIG: redis_enabled=True
CONFIG: redis_host=localhost
CONFIG: redis_port=6379
CONFIG: redis_db=0
CONFIG: whoosh_index_dir=whoosh_index
CONFIG: image_dir=./images
CONFIG: production=False
>>>> Running pylint
************* Module dev
dev.py:86: [C0325(superfluous-parens), ] Unnecessary parens after 'in' keyword
dev.py:151: [W0621(redefined-outer-name), pytest] Redefining name 'pytest' from outer scope (line 145)

------------------------------------
Your code has been rated at 10.00/10

TestFailedException running ['dev.py', 'lint']:  [(20,)] 20

TravisBuddy Request Identifier: d64f0570-092e-11ea-875b-c9c491752ea4

Why does this not surprise me 😛

Landed this (eventually) on another branch as a cherry-pick. Closing.


Remote.Push pushes corrupted objects to the server

Related to https://github.com/src-d/borges/issues/264
Remote.Push pushes some corrupted objects to the server, so the repo becomes unreadable. This happens at least with https://github.com/zfcampus/zf-oauth2. We're aware of a lot more repos with this issue, but this was the first I could get my hands on.
The error:
[2858] error: delta replay has gone wild
[2858] fatal: pack has bad object at offset 470810: failed to apply delta

Reproduction
Setup the server:
mkdir /tmp/server-repo
cd /tmp/server-repo
git --bare init --shared
git daemon --verbose --base-path=/tmp --export-all --reuseaddr --enable=receive-pack

Push with go-git:
package main

import (
	"log"

	git "gopkg.in/src-d/go-git.v4"
	"gopkg.in/src-d/go-git.v4/config"
)

func main() {
	r, err := git.PlainClone("/tmp/client-repo", false, &git.CloneOptions{
		URL: "https://github.com/zfcampus/zf-oauth2",
	})

	assert(err)

	err = r.Fetch(&git.FetchOptions{
		RefSpecs: []config.RefSpec{
			"refs/*:refs/*",
			"HEAD:refs/heads/HEAD",
		},
	})

	assert(err)

	remote, err := r.CreateRemote(&config.RemoteConfig{
		Name: "external",
		URLs: []string{"git://localhost/server-repo"},
	})

	assert(err)

	err = remote.Push(&git.PushOptions{
		RemoteName: "external",
		RefSpecs: []config.RefSpec{
			"refs/*:refs/*",
			"HEAD:refs/heads/HEAD",
		},
	})
	assert(err)
}

func assert(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

Doing the same operation with git push does not result in any corrupted object.
I've checked and the repo is not corrupted neither before pushing nor afterwards, the problem only occurs pushing.
I'm currently trying to trace where the issue is.
UPDATE: setting config.Pack.Window to 0 makes the issue go away, but that turns off delta compression, though.

On the given repo, this just happens when object 72bf04bda1466acfd65bf09143f0372457f95136 is deltified with 8f751eef719ef101181bfd18aa8333546a917d19 as its base.
It can either be a problem in the packfile encoder, which is not likely, or in the delta selection.

I was able to reproduce the error using that two files and creating a packfile just with them:
package main

import (
	"log"

	"os"

	git "gopkg.in/src-d/go-git.v4"
	"gopkg.in/src-d/go-git.v4/config"
	"gopkg.in/src-d/go-git.v4/plumbing"
	"gopkg.in/src-d/go-git.v4/plumbing/format/packfile"
	"gopkg.in/src-d/go-git.v4/storage/memory"
)

func main() {
	//r, err := git.PlainClone("/tmp/client-repo", false, &git.CloneOptions{
	//	URL: "https://github.com/zfcampus/zf-oauth2",
	//})

	r, err := git.PlainOpen("/tmp/client-repo")
	assert(err)

	err = r.Fetch(&git.FetchOptions{
		RefSpecs: []config.RefSpec{
			"refs/*:refs/*",
			"HEAD:refs/heads/HEAD",
		},
	})
	h1 := plumbing.NewHash("72bf04bda1466acfd65bf09143f0372457f95136")
	obj1, err := r.Storer.EncodedObject(plumbing.AnyObject, h1)
	assert(err)

	h2 := plumbing.NewHash("8f751eef719ef101181bfd18aa8333546a917d19")
	obj2, err := r.Storer.EncodedObject(plumbing.AnyObject, h2)
	assert(err)

	// Delta generated by the encoder will be this one
	delta, err := packfile.GetDelta(obj2, obj1)
	println(delta.Size())

	mem := memory.NewStorage()
	_, err = mem.ObjectStorage.SetEncodedObject(obj2)
	assert(err)
	_, err = mem.ObjectStorage.SetEncodedObject(obj1)
	assert(err)

	f, err := os.Create("packfile.pack")
	assert(err)

	enc := packfile.NewEncoder(f, mem, false)

	_, err = enc.Encode([]plumbing.Hash{h1, h2}, 2)
	assert(err)
}

func assert(err error) {
	if err != nil {
		log.Fatal(err)
	}
}

After that, we have a packfile.pack file with two objects. Trying to create the index:
17:47 $ git index-pack packfile.pack
error: delta replay has gone wild
fatal: pack has bad object at offset 1884: failed to apply delta

Changing the window from 2 to 0 (so no delta is generated):
17:48 $ git index-pack packfile.pack
8d1e8b9058eb4b716778d379de039d5df87a9268

I think the problem is on diff_delta logic.

@ajnavarro - following fix in delta_diff.go:
			for rl > 0 {
				if rl < maxCopySize {
					buf.Write(encodeCopyOperation(aOffset, rl))
					break
				}

				buf.Write(encodeCopyOperation(aOffset, maxCopySize))
				rl -= maxCopySize
				aOffset += maxCopySize
			}

PR. with fix: https://github.com/src-d/go-git/pull/856


k8s enhancements

More functions for https://github.com/gruntwork-io/terratest/issues/194

Expose some functions that can be used without testing.T to not take in testing.T so that they can be used outside testing context.

GetPublicIpsOfEc2InstancesFromClientE
GetKubernetesClientFromFileE
GetNodesByClientE

Migrate ContinuouslyCheckUrl from module-ecs

Expose StoreConfigToFile from the k8s module tests, as that can be used for storing a kubernetes resource config to disk to be used with kubectl apply

Introduce wrapper functions for calling kubectl:

RunKubectl, RunKubectlE, RunKubectlAndGetOutputE
KubectlApplyFromString, KubectlApplyFromStringE
KubectlDeleteFromString, KubectlDeleteFromStringE

Introduce functions for interacting with Kubernetes Service resources.

Expose some functions that can be used without testing.T to not take in testing.T so that they can be used outside testing context.

What's the use case? Do we really want production code depending on a test library?

Migrate ContinuouslyCheckUrl from module-ecs

Nice 👍

What's the use case? Do we really want production code depending on a test library?

While I agree this is a bit awkward, the wrappers we built around the clouds end up being useful abstractions for writing any code that works against the underlying systems.
For example, in the rollout command I ended up doing a lot of the same things I would do in a test context: waiting for nodes to be ready, initializing clients by config files, manipulating config files, calling kubectl. And I really wanted to avoid copy pasting the code and duplicating the tests.
I guess the most ideal situation would be to break out the cloud verification code into its own repo that then prod code and terratest can import, but I wasn't quite ready to shave the yak of figuring out the right abstraction, and also cause a lot of disruption.

For example, in the rollout command I ended up doing a lot of the same things I would do in a test context: waiting for nodes to be ready, initializing clients by config files, manipulating config files, calling kubectl. And I really wanted to avoid copy pasting the code and duplicating the tests.

Sanity check: is this code really the same or just somewhat the same. I worry that Terratest may update a lot of these functions over time to be more and more suited for testing, with little regard for potential prod uses of this code... And if it's not too much code, sometimes, trying to keep things perfectly DRY can be counter-productive.

is this code really the same or just somewhat the same.

The kubectl wrappers (RunKubectl) are exactly the same right now. Getting and waiting for nodes are also exactly the same. The AWS stuff is more questionable.

I worry that Terratest may update a lot of these functions over time to be more and more suited for testing, with little regard for potential prod uses of this code...

Those are valid points, although as mentioned, it would be nice if they were isolated in a separate library to avoid that... but then there is overhead ⬇

And if it's not too much code, sometimes, trying to keep things perfectly DRY can be counter-productive.

Ok I will revert those and copy paste and see how much duplication it ends up being, and if overtime it really does get out of hand, we can discuss merging then.

Reverted the "FromClient" versions
Addressed all suggested changes:

Added more unique error types
Use the E suffix convention

UPDATES:

c7853c2 - Made ContinuouslyCheckUrl use Errorf. It will also now return a sync.WaitGroup that can be used to wait for the continuous check to shutdown.
ad2c714 - Found and fixed a bug in DummyServer, where we try to register multiple handlers to route / on the DefaultServeMux. Caught this when I wrote a test for ContinuouslyCheckUrl.
4f285bf - Added more k8s functionalities, related to dealing with namespaces. Also changed the function APIs to take in a KubectlOptions object so that you can control which context to use as well as which namespace.
8d4160d - Added a basic k8s example and test

Currently, I am waiting to get a green build.

UPDATE:

0091cc6 - Introduce GetRandomStableRegion for aws, which only selects from regions that have been around for more than a year. The list is hardcoded for now, but added the reference so that it can be evaluated on a yearly basis.

Ok build passed, and review checked off. Going to merge, release, and start drafting an entry in the newsletter!


[PE-D][Tester D] find return message is ungrammatical when there's 1 match

When using the find command, when there's only 1 match, the message returned makes a grammatical error in plurality. A very minor fix!

Labels: severity.VeryLow type.FeatureFlaw
original: trpmy/ped#12

no hindrance to user -> not in scope


No double scan

This should fix the double scan issue in parse_line pointed out during review of the previous PR. Also tries to avoid allocation for link titles and destinations.

This reduces time on the rendering the crdt document to html from ~729µs to ~622µs on my machine.


Two times same numbers on one day?

I used the same tags on one day. But the counters did not go up

Sorry, my fault. Only recognized now. The counter shows the total. Each time and everywhere, not the order. Sorry, my fault ;)

Sorry, my fault. Only recognized now. The counter shows the total. Each time and everywhere, not the order. Sorry, my fault ;)


ci-e2e-upgrade: Remove setting CLI vsn

It's controlled by set-env-variables.

/test

Blocked by https://github.com/cilium/cilium-cli/pull/2129.

/test

/ci-e2e-upgrade

/ci-e2e-upgrade


Default dark mode

Describe the feature request
It would be nice to be able to have the dark mode activated by default ?
Someting like:
ui.darkmode = true
Thanks! :)
Why do you personally want this feature to be implemented?
No response
How long have you been using this project?
No response
Additional information
No response

i like dark theme !!!   Couldn't be better as default option

This change was made 3 years ago https://github.com/TwiN/gatus/commit/4ab5724fc16e3d42c5a8360fd5a9872150136a15#diff-c37cd5e977ef60f10519116a6be209492ae686c52088d4911e812880ac82bfacR75


separate tasks for fetching/extract seed data? location?

As of now, dbLoad always tries to run fetch/extract of seed data unless those tasks "think" they've already been run.

I think it's confusing that the data are kind of buried in the java area, I liked the seed-data directory in database, it was a relatively obvious place for the data to live.

How does it know whether we want the files that are there vs. what's in the remote tgz? I'm uncomfortable letting the dbLoad task make the decision because that's beyond the scope of loading what I've prepeared. I think it would be better to have dbLoad fail if the seed-data directory isn't populated with something that resembles samplestack data.

Perhaps an uber task can run all of the tasks (basically what cleanstart.sh does), and the individual tasks won't all "force" themselves on the developer. I don't necessarily think this should apply universaly, but at the least I don't think loading the data should be hardwired to preparing it to be loaded, because that is a moment where people are likely to diverge, especially once we have more than one data set prepared for them.

This shouldn't really be assigned to me. It's a gradle thing...


Fixes About Task lists article

Why:
Closes #9317

What's being changed:

Changes suggested by felicitymay applied.

Versions "Navigating Issues" section to GitHub.com only
Added a new GIF for GHES wherein issue and pull request URLs aren't auto expanded.

Check off the following:

[x] I have reviewed my changes in staging (look for the latest deployment event in your pull request's timeline, then click View deployment).
[X] For content changes, I have completed the self-review checklist.

@thispsj 👋🏻 - these changes LGTM, thanks for improving our docs ✨ I made some minor suggestions relating to our content model (ALT text) and about simplifying a versioning Liquid statement that I'm going to apply. There are two outstanding issues re: the GIF file that I haven't been able to help with:

The new GHES image needs to be placed in a different directory as it's GHES-only: assets/images/help/writing/task-list-reordered-ghes.gif -> assets/images/enterprise/writing/task-list-reordered-ghes.gif. Can you do that? I wasn't able to get your branch and make these changes myself. You'll need to create a new writing subdirectory in assets/images/enterprise/, place your GIF there, and remove the image from the current directory it's in.
The GIF file looks blurred in the preview. Compare your GIF with the GitHub.com version. We have our own optimization algorithn, so perhaps you should try adding your unoptimized GIF, and we'll see how things look.

Hope this helps.

Yeah I will do the changes you suggested.
@mchammer01 What about the suggested changes ? Should I commit them ?

@thispsj Thank you for enabling the option ✨  Enabling maintainer edits will quicken the PR review and merge process greatly 🐇  Could you please double check for all other PRs created now and in the future before submitting them ✅?
Thanks so much for your passion in contributing to our docs 💖

@thispsj 👋🏻 - thanks for returning to this PR and completing the work. I'm currently working to a tight deadline, so I've unassigned myself as I won't have time to review your latest updates. I'm sure someone else from the team will review this for you 🙂

@mchammer01 no problem

Thanks for moving the GIF 💖
I've fixed a small typo and updated from main. I'll check the preview and then 🤞🏻we'll be ready to merge your change once the repo is unfrozen 🙂


test - Selected location not appearing on map

When I select a location from the list it should be highlighted on the map - as a large black pin
It shouldn't be possible to see a location detail page without that location being highlighted on the map as with SACSS below
This may be an issue with a delay in returning results, and me moving the map before the location was shown.

@adambard1 it is done


Change acr-sp-init.sh to adhere to naming convention

This PR includes a new script, ./setup/acr-setup-test.sh to automate testing the functionality of ./setup/acr-setup.sh.  To test the new functionality, run the following commands from a bash prompt:
# Authenticate to Azure
az login

cd ./tests
./acr-sp-init-test.sh

@rickrain: I was reviewing this PR and trying to execute the script but got the following error:
$ bash --version
GNU bash, version 5.0.3(1)-release (x86_64-apple-darwin18.2.0)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ acr-sp-init.sh -a test -l eastus -s test123
./acr-sp-init.sh: line 34: ${appname,,}: bad substitution

@xtophs The title of the PR is the same as the issue #37.  The scope of the changes in this PR are exactly what was defined in the AC with one exception, and that is the logic I added to retry role assignments instead of delaying for 20s which we talked about.  So, added that enhancement while I was making the other changes.
Title has been updated to reference #37.  The individual commits referenced #37 already but I also tagged it in the title.


README.md

Op

The PR wasn't related to this project. Also, if I found such PRs next time then you will be blocked from doing any kind of contributions!


Got DropboxSDK.h file not found on Xcode7.1

I install ensembles using cocoapods and got DropboxSDK.h file not found compile error In CDEDropboxCloudFileSystem.h line 13 #import "DropboxSDK.h"

I had to go back to cocoapods 0.37.2 for it to work.
sudo gem install cocoapods -v 0.37.2

@drewmccormack is aware of the issue.

If anyone has an idea how this can be fixed, please let me know. It is clearly a Cocoapods issue. I'm no expert, and all the variations of imports I tried seemed to fail.

No problem here. CocoaPods version 0.39.0
Podfile:
source 'https://github.com/CocoaPods/Specs.git'

platform :ios, '8.0'

pod "Ensembles", "~> 1.0"
pod "Ensembles/Dropbox", "~> 1.0"

Podfile.lock:
PODS:
  - Dropbox-iOS-SDK (1.3.13)
  - Ensembles (1.4.1):
    - Ensembles/Core (= 1.4.1)
  - Ensembles/Core (1.4.1)
  - Ensembles/Dropbox (1.4.1):
    - Dropbox-iOS-SDK
    - Dropbox-OSX-SDK
    - Ensembles/Core

DEPENDENCIES:
  - Ensembles (~> 1.0)
  - Ensembles/Dropbox (~> 1.0)

SPEC CHECKSUMS:
  Dropbox-iOS-SDK: e0c9b08d1604b39cb78c1d0333251de0f6bdec60
  Ensembles: 90013c435ff874a7a9ccb7a334b74b93ac53dc12

COCOAPODS: 0.39.0

Are you using use_frameworks!?

Same here.  The latest version of ensembles and cocoa pods works for me now.

Seems may have been a glitch in Cocoapods. Closing.

This keeps popping up for me. It's really weird I've tried reinstalling pods, downgrading pods (0.37.2) upgrading pods (1.0 beta). 0.37.2 crashed and after switching back to 0.39 it worked exactly once. After I stopped the app and clicked 'run' again, the error was back without me having made any changes in the project whatsoever. Really weird.


Pattern: Content group - with pictograms QA testing - Production level version

User Story

As a [user role below]:
developer using the ibm.com Library Content group - with pictograms

I need to:
have a version of the pattern or component that has been tested for accessibility compliance as well as on multiple browsers and platforms

so that I can:
be confident that my ibm.com web site users will have a good experience

Additional information

Browser Stack link
Browser Standard
Browser versions to be tested:  Tier 1 browsers will be tested with defects created as Sev 1 or Sev 2.  Tier 2 browser defects will be created as Sev 3 defects.
Platforms to be tested, by priority:  1)  Desktop  2)  Mobile  3)  Tablet
Mobile & Tablet iOS versions: 13.1 and 13.3
Mobile & Tablet Android versions: 9.0 Pie and 8.1 Oreo
Browsers to be tested:  Desktop: Chrome, Firefox, Safari, Edge, IE, Mobile: Chrome, Safari, Samsung Internet, UC Browser, Tablet: Safari, Chrome, Android
Accessibility Checklist
Creating a QA bug
design folder is on Box
Dev link (#1165 )
Functional specs- Link to the functional specs

Acceptance criteria

[ ] Accessibility testing is complete.  Pattern is compliant.
[ ] All browser versions are tested
[ ] All operating systems are tested
[ ] All devices are tested
[ ] Defects are recorded and retested when fixed

@Praveen-sr @chetankan The design specs are available through the Box folder added in the above description.

Testing Status on 2/5/2020:  functional testing is complete.  Design review in progress.  Accessibility testing in progress.

Testing status 2/6/2020:  Screen reader accessibility testing is the only testing remaining.

Testing Status:- Finished , Design and Accessibility(DAP, Keyboard, CCA) is done, and screen reader is in progress.

@chetankan , @Praveen-sr ,
The updated version was merged in, and it's ready for QA,
Thanks!


[CDE] Change 'YES'/'NO' String to NCit codes

https://www.ebi.ac.uk/ols/search?q=no&groupField=iri&start=0

Fixed


RFC: Add eslint-plugin-compat

This eslint plugin lints against APIs that are only supported by target browsers 🎉 Would a PR for adding this plugin be welcome?
Here's a demo of it in action

Absolutely :)

As an aside, from a quick peek, @amilajack have you thought about using browserlist?
https://github.com/ai/browserslist

@ntwb What does browserlist do?

@amilajack It also uses the caniuse database, though you can have much more detailed queries of what browsers to test against, check the examples at https://github.com/ai/browserslist#queries

@ntwb I see. So are you suggesting that eslint-plugin-compat use browserslist to query caniuse? eslint-plugin-compat is aimed at linting the cross-browser compatibility of apis being used. browserslist 'Shares browsers list between different front-end tools'. Two very different things


feat: add PorterJs transform module type

Description:
This pr adds a new transform module type called PorterJs, which is the bundler I've been working on and off for years. The reason to add this particular module type in SWC, rather than relying purely on SWC plugins, is that the default ones aren't quite the fit for  porter, specifically:

both import and require dependencies need to be collected for porter,
condition requires need to be processed early on, no matter minify is activated or not

more details are discussed in this porter documentation https://porterhq.github.io/porter/swc, with the new module type porter can transform JavaScript and TypeScript modules in one pass, and concatenating them into bundles accordingly. I've done a few benchmarks on compiling the source of https://chuangyi.taobao.com, which builds roughly 50% faster for now.
BREAKING CHANGE:
This pr doesn't break existing features, just adds the transform module type
Related issue (if exists):
n/a

understood, I'll post an RFC first. The bundler uses several plugins already, such as glob import, which is quite handy. But I do need the new module type to properly handle the transform and minify process.


Repo is Abandoned - 代码库已被废弃

The description of problem
Since it's been 8 months since any activity by the owner, and 11 months since the last release (which is broken and cannot be installed) I think it's fair to call this repo abandoned (for now). I hope George Zhao is doing alright.
For the sake of those landing here trying to install this HACS add-on and control their Midea devices, it's worth calling this out and giving advice for how to get their Midea devices working.
After looking around myself and even briefly starting my own fork, I've found this fork to be the most actively maintained: https://github.com/wuwentao/midea_ac_lan
Maybe one day this one will come back, but as the install is broken, it's not a great default anymore.
My opinion for anyone who lands here trying to get Midea AC Lan to work, is to add https://github.com/wuwentao/midea_ac_lan as a Custom repo to HACS instead of using this repo.
The logs
Downloading georgezhao2010/midea_ac_lan with version v0.3.22 failed with (Could not download, see log for details)
(MainThread) [custom_components.hacs] Failed to download https://github.com/georgezhao2010/midea_ac_lan/releases/download/tags/v0.3.22/mieda_ac_lan.zip

thanks for your support,  we will continue to maintain it via https://github.com/wuwentao/midea_ac_lan
we also need more contributions to help us, PR is welcome.
Thanks

Is it possible to brin Midea gas water units in to this?


Add support to use the pipeline with NGS generated using zymo research kit

Description of feature
https://www.zymoresearch.de/products/zymo-seq-switchfree-3-mrna-library-kit

I presume you can analyze the data already with the current pipeline version. After all, the parameter suggestions/UMI settings are just examples and not an exhaustive list. So if your favorite kit is not named, you could still make it work by choosing the pipeline parameters accordingly.
All the listed settings and recommendations in the docs have been worked out by users of the pipeline and kindly shared with everyone else.
I appreciate, that the complexity of the pipeline can be overwhelming, and we are happy to help, if you can't figure something out, but please make an own effort first. Instead of just opening an issue with the URL to the kit's manual for others to read it for you, please summarize the key characteristics of the kit and propose some parameter settings that seem appropriate to you. Then we are happy to chime in and double-check for your peace of mind. Thanks.

No worries, I didn't take offense. Becoming familiar with git and GitHub takes a while for everyone, and it is great to see new members in the nf-core community. Welcome!
If you like, you can join the Slack workspace as well, which is our community's main discussion platform. Here, we have multiple dedicated channels for topics ranging from #rnaseq to starting as a contributor, and you can ask as many questions as needed. Since documentation updates are a great first pipeline contribution (see e.g. pull request #1314) it would be great if you could suggest something to help other users of that kit!


storage: add APIVersion as generic parameter

What is changed and how it works?

Issue Number: ref #11965 #12207
What's Changed:
Add APIVersion as generic parameter of Storage, to prepare for modifying raw interfaces for API V2.
Refer to RFC:

https://github.com/pingyu/tikv-rfcs/blob/rfc_api_v2_pd/text/0069-api-v2.md (https://github.com/tikv/rfcs/pull/79)
https://github.com/pingyu/tikv-rfcs/blob/rawkv-cdc/text/0083-rawkv-cross-cluster-replication.md (https://github.com/tikv/rfcs/pull/86)

Add APIVersion as generic parameter of `Storage`.

Related changes
No.
Check List
Tests

Unit test
Integration test

Side effects
No.
Release note
Add APIVersion as generic parameter of `Storage`.

/cc @BusyJay @Connor1996 @andylokandy

/cc @BusyJay @Connor1996 @andylokandy

PTAL, thanks~

/run-all-tests

/merge

/run-all-tests

/run-all-tests

/run-all-tests

/run-clippy-darwin-amd64

/run-clippy-linux-arm64

/run-all-tests


Objective C code

Hi,
I am having issue with integration of swift file in objective c project.
Please help me with objective c .h and .m files.

I was trying to compile swift code in xcode 7.3.1, then I tried with xcode 8.3.1 and worked.


feat (toolbar):Debug toolbar

Checked out toolbar stuff into this new clean branch as many many messy changes of mind.

Also, it seems you had an example usage of the toolbar in web-wa-sqlite example but you removed it?
Would be good to have an example where the toolbar is being used such that users can see the usage.

@kevin-dp I've pretty done everything as you suggest. A few notes:
I had to add export { MockRegistry } from './mock' to satellite/index.ts to get the import to work without the addition export is the package
I couldn't refactor the tab components as you suggested, I had tried to do this before, its something to do with when react expands out the JSX, it generates react hook errors.
I have moved the whole thing for now - but we can do this later
I haven't added back the toolbar in an example as I want to do this an other step once there is a published toolbar package to refer to properly. But I have checked that this will works ok in the web example. I have also added a non compiled util.ts script to tidy away the code to add the toolbar.

I think this might be ready now - I've made to readme changes you suggested

I see there is still activity here. Let's not leave it hanging.

@kevin-dp I have done all the changes you suggested except the additional tests, and I have tested it manually against 0.0.9 with the web example.
Can we merge this now as I'm not really coding now and don't have time to think about how to set up the extra tests and Valter wanted this in for next week?

Hi @paulharter,
It feels like the unit tests suggestion are important. I understand that this will probably not help a lot with browser implementation, but for the sake of completeness let's make those tests.
This PR is dragging for too long. It's very near.
Best,

@kevin-dp @balegas I've added the extra api tests now

I've modified the debug toolbar slightly, added a couple more APIs for the connection status (toggle on/off) and an experimental shapes tab for viewing active shapes.
The last one uses a private field from the satellite implementation so before developing further I'll open a PR to expose some shape subscription information from the Satellite client for the debug toolbar to use safely.
I've also done an ugly inlining of CSS into the debug toolbar to make the DX better (now only need to dynamically import and call addToolbar(electric)) - will improve that over time but I had already spent too much time on CSS bunding, postprocessing, and TS errors...
@kevin-dp can you have another look so we can potentially merge this?


Shared folders are not listed in view.delta

According to working-with-links.md view.delta should return the shared folders contained in the drive.
However in my tests shared folders (that have been added to the drive) are never returned.

Content of my drive:

Output of /drive/items/root/view.delta?select=name:
{
<EMAIL_ADDRESS>"https://api.onedrive.com/v1.0/$metadata#drives('D4FE0EE4D78A90DC')/items(name)",
<EMAIL_ADDRESS>"...",
	"value": [
		{
			"name": "root"
		},
		{
			"name": "empty"
		}
	],
<EMAIL_ADDRESS>"..."
}

I noticed your response contains a nextLink - if you fetch that URL do you see SHARE-TEST come back?

No, this is the response:
{
<EMAIL_ADDRESS>"https://api.onedrive.com/v1.0/$metadata#drives('D4FE0EE4D78A90DC')/items(name)",
<EMAIL_ADDRESS>"https://api.onedrive.com/v1.0/drives('D4FE0EE4D78A90DC')/items('root%252F')/view.delta?$select=name&token=...",
<EMAIL_ADDRESS>"...",
	"value": [
		{
			"name": "root"
		}
	]
}

Hi @skilion, this should now be resolved! It will only work for new tokens though - a delta session that was started prior to the fix will continue to not return the mount points (you can get a new token by either not providing the token query string parameter).

Just tested and it's working, thank you.


googlechrome.github.io needs landing page

That lists all the samples.

+1 this.
Maybe we could list samples and their relative chrome status page as well.

@gauntface, if you've got the interest to take this on either here or over at https://github.com/GoogleChrome/chromium-dashboard, I think that strictly speaking, anything would be better than the nothing that we currently have. I don't believe anyone else is currently looking into this.

I played with it a while ago but we needed to integrate Jekyll more. If
someone is up for you will get much love frome me
P
On Tue, 7 Jul 2015 18:08 Jeffrey Posnick<EMAIL_ADDRESS>wrote:

@gauntface https://github.com/gauntface, if you've got the interest to
take this on either here or over at
https://github.com/GoogleChrome/chromium-dashboard, I think that strictly
speaking, anything would be better than the nothing that we currently have.
I don't believe anyone else is currently looking into this.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleChrome/samples/issues/37#issuecomment-119270291
.

After thinking about this some more, I'm of the opinion that this should be addressed via a new https://chromestatus.com/samples landing page, and we should create a top-level index.html in this repo that just contains a <meta> redirect there.
This repo is great for hosting new samples and offers a lot of flexibility around directory structure, including supporting JS/CSS files, and using HTTPS. This repo is not ideal for storing metadata about what each sample relates to (Networking, Multimedia, ES6, etc.) and metadata about which version of Chrome it's associated with. While we could try to add that metadata to each sample in a way that would be parseable by an index-generator, we'd effectively be replicating what already exists in the chromestatus.com backend. Without that metadata, putting together a meaningful browsing/search experience seems like an uphill battle.
Using a hypothetical https://chromestatus.com/samples as the canonical samples list also means that samples that predate this repo or that, for whatever reason, aren't hosted in this repo will show up in the list.


Numbers - Add math-php

https://github.com/markrogoyski/math-php

Please open a pull request to suggest something.


Making it work with yarn package manager

So i was trying to build this project with yarn, and I ran into this strange issue during compile:
TypeScript error: Argument of type '{ todoList: (state: any, action: Action<any>) => any; }' is not assignable to parameter of type 'ReducersMapObject'.
  Property 'todoList' is incompatible with index signature.
    Type '(state: any, action: Action<any>) => any' is not assignable to type 'Reducer<any>'.
      Types of parameters 'action' and 'action' are incompatible.
        Property 'payload' is missing in type 'AnyAction' but required in type 'Action<any>'.  TS2345

     9 | }
    10 |
  > 11 | export default (history: History) => combineReducers({
       |                                                      ^
    12 |   ...todoReducder
    13 | });

I've tried yarn import, inferring yarn.lock and so on. but nothing worked.
After more diggings, i found that by using yarn I was getting an older version of redux (3.7.2) by running yarn why
yarn why v1.15.2
[1/4] Why do we have the module "redux"...?
[2/4] Initialising dependency graph...
[3/4] Finding dependency...
[4/4] Calculating file sizes...
=> Found<EMAIL_ADDRESS>info Reasons this module exists
   - "@types#redux-logger" depends on it
   - Hoisted from "@types#redux-logger#redux"
info Disk size without dependencies: "248KB"
info Disk size with unique dependencies: "7.76MB"
info Disk size with transitive dependencies: "7.79MB"
info Number of shared dependencies: 5
=> Found "<EMAIL_ADDRESS>info This module exists because "@types#react-redux" depends on it.
info Disk size without dependencies: "228KB"
info Disk size with unique dependencies: "312KB"
info Disk size with transitive dependencies: "340KB"
info Number of shared dependencies: 3
Done in 0.65s.

and I was eventually able to resolve by adding  "redux": "^4.0.1" as a dependency.
Not sure if I should submit a pull request for this, after all I did go against the recommended npm installation approach. But i hope this can help others who wanted to use an alternative package manager. I have verified that by adding this constraint does not break the current npm build process.

should be solved in latest commit


test: fix tests for #251

addressing #254
i left comments in the code diff

Thanks for the quick fix!


Fixed broken link on the deploying to hubot page

I fixed a broken link on the deploying to hubot page.  The link to "Getting Started" results in a 404.  I've updated it to point to the /docs/index.md page.

Thanks for the PR. #1309 will fix this so links work properly on GitHub.com and on hubot.github.com


Generate classes by root namespace

A rebase of #40 on top of #41. @lmolkova it was impractical to do a traditional merge because there were just too many conflicts, so I listed you as a co-author on the commit. Hope that's ok with you.
This PR breaks out classes for each top level namespace in semantic conventions:

For stable conventions, classes are named {{Namespace}}Attributes.java
For incubating / experimental conventions, classes are named {{Namespace}}IncubatingAttributes.java

This omits the automatic metric generation of #40. Figure we can do that in a separate PR to keep changes organized.
Leaving this as a draft because we're waiting https://github.com/open-telemetry/build-tools/pull/243, which is currently only merged to the feature/codegen-by-namespace branch, which doesn't have an image published yet. I think https://github.com/open-telemetry/build-tools/pull/256 might be trying to solve that.

Update: A image has been published containing the logic form the feature/codegen-by-namespace branch. Marking this as ready for review since the new capabilities are expected to eventually merge into main.

We can't say for sure, but this issue contains feedback from a number of language maintainers which is telling:

js maintainer seems to indicate that grouping attributes by namespace isn't very important
dotnet maintainers say grouping attributes by namespace makes sense
go maintainer says "is desirable to have the semantic conventions laid out in files that matched their groups so we can find them easier". I think this is referring to grouping by namespace?
c++ maintainer says group by namespace sounds good
erlang maintainer agrees with c++ maintainer
swift and php don't seem to have a preference on grouping

Didn't here from all maintainers, but there seems to agreement about grouping by namespace amongst maintainers who expressed an opinion.
Not related to this comment, but related to this PR: This issue to have separate properties from stability and deprecation, and not remove deprecated attributes will be something we will want to take advantage of.

Do we know that this is the direction other languages are going to take? I like it, my only reservation is to minimize the number of times we change the structure before marking it stable.

Thanks @jack-berg for summarizing the feedback!
I was also checking with python SIG - they are fine with per-root-namespace approach https://github.com/open-telemetry/opentelemetry-python/pull/3586
@trask I only know about two possible directions:

current (file per signal or similar)
group by root namespace - this is a new feature in the build-tools

I think there could be something else, but nobody requests for it explicitly and nobody works on supporting it (build-tools changes would be necessary).
So at least for today, there is no third way.

FYI - This PR is broken against latest feature head.  You need to merge this: https://github.com/jack-berg/semantic-conventions-java/pull/1

@open-telemetry/java-approvers - WDYT? Are there any other questions / concerns about moving forward with this?

I didn't see any deprecated attributes (yet?), is this expected?

I didn't see any deprecated attributes (yet?), is this expected?

@trask good observation. I wouldn't expect to see any for 1.23.1, but I also don't see any in #46, which upgrades to 1.24.0 which splits out the deprecated and stability properties. I think I found the issue.

@jack-berg @trask I'm in doubt how  to approach stabilization and wonder if you have some thoughts on it
E.g.

attr foo.bar goes stable
it disappears from incubating
it appears in stable

Assuming application uses semconv.incubating.FooAttributes.FOO_BAR it will be a breaking change (or if instrumentation lib version is not aligned).
I wonder if we can do a better job if incubating artifact included all attributes - stable and experimental - then experimental-> stable transition would not be breaking.
When attribute becomes stable we could also mark corresponding constant as @Deprecated - use  semconv.FooAttributes.FOO_BAR instead

Ok, I've pushed a commit to use the latest release of the feature/codegen-by-namespace, which adds a --strict-validation. This allows it to work with 1.23.1 of semconv which didn't separate out the stability and deprecated properties.
The result is that we now see deprecated attributes, which addresses @trask comment. However, the deprecated annotations are missing the @Deprecated annotation, because 1.23.1 of semconv conflates stability and deprecated in a single property. This is resolved once we bump to 1.24.0 of semconv (#46). See the diff for properly working @Deprecated annotations.

I wonder if we can do a better job if incubating artifact included all attributes - stable and experimental - then experimental-> stable transition would not be breaking.
When attribute becomes stable we could also mark corresponding constant as @Deprecated - use  semconv.FooAttributes.FOO_BAR instead

I think that could work. Incubating is then a superset of stable.

I wonder if we can do a better job if incubating artifact included all attributes - stable and experimental - then experimental-> stable transition would not be breaking.
When attribute becomes stable we could also mark corresponding constant as @deprecated - use  semconv.FooAttributes.FOO_BAR instead

I think that could work. Incubating is then a superset of stable.

Tried it here - https://github.com/jack-berg/semantic-conventions-java/pull/2
Generation of @deprecated annotation is a bit ugly, but possible.

@lmolkova I've merged your PR and an additional commit that allows this to work as is. I think we should be good to merge this PR now.
@trask let me know if you have any additional thoughts.

@lmolkova I've merged your PR and an additional commit that allows this to work as is. I think we should be good to merge this PR now.
@trask let me know if you have any additional thoughts.

can you please revert it? It relies on unmerged changes in the build-tools that are unlikely to make it into the next build-tools release.  I don't think it's blocking since I assume it should be fine to add a bunch or stable attributes into incubating artifact later on.
In the meantime, I really want  to get your thoughts on  how this stable+unstable solution might look like.
Repeating the question I left on the python PR https://github.com/open-telemetry/opentelemetry-python/pull/3586#issuecomment-1973978521
Option 1: ship 2 artifacts  (semconv and semconv-incubating) - current solution
It's aligned with semantic versioning, easy to maintain.
Still having stable attributes as deprecated in the incubating artifact is not great.
Option 2: ship 1 artifact, but two different versions: stable and preview. Stable contains only stable semconv. Preview contains all of them.
Ideal in terms of semantic versioning. When attribute stabilizes, it remains in the same namespace and all libs/apps that depend on it don't need to change
The only downside is that it maybe harder to maintain and release.
Option 3: Probably not applicable to Java - ship 1 artifact, 1 version containing stable and experimental semconv. Emphasize that some parts are experimental via namespace (import semconv.experimental.foo_attributes, perhaps we should call it _experimental )?
It's easy to do, but violates semantic versioning. When attribute stabilizes, it moves to a different namespace, i.e. libs/apps that depend on it need to change.

I think the cleanest solution from user perspective would be option 2:
we'd have semantic-conventions-java:1.0.0 with stable only, semantic-conventions-java:1.0.0-beta.1 with all semconvs. Experimental -> stable transition requires no action on the user side.
It's also one of the common patters to ship experimental features.
We can also discuss at the SIG meeting,  but I always have a conflict discussing the same with Python SIG at the same time:)

@lmolkova I adjusted it so it only uses build-tools code merged to the feature branch which is published to docker hub

I'd suggest that we wait on merging this (or at least block on releasing it) until we have final decision on #50, so that we can limit the breaking changes to a single release

I think we should hold the release until we resolve #50, but we've already merged substantial breaking changes to main, so I don't see the value of holding off on this specific PR.


(supertuxkart) Outdated

Supertuxkart 1.2.0 just released: https://blog.supertuxkart.net/2020/08/supertuxkart-12-release.html
Changelog is here: https://github.com/supertuxkart/stk-code/blob/1.2/CHANGELOG.md, binaries moved to github here: https://github.com/supertuxkart/stk-code/releases/tag/1.2
;)

@pataluc @AdmiringWorm
I will submit a PR for this later today. It needs a better regex to get those new links.


docs: CSS scoping ADR & apply decision in real life

Bug: https://phabricator.wikimedia.org/T258774

The title in 8672f28 will need an update as seen in fb91cf8 once #105 is merged (incl. an update to storybook 6).

Oh. Now that we merged #105, the vue-components' storybook need a sass loader in dev, too.
I pointed #110 into this branch as 445ebd2 applies the ADR (in 1e2193f) - so the PR (primarily the first and last commit) only makes sense if we decide to go forward with what is discussed in https://github.com/wmde/wikit/pull/108#discussion_r459982127

Fixed the SASS introduction to add the SASS loader to storybook.


[5.3] Move the fire of the NotificationSent event after the actual sending

I think the event was fired first so that if sending throws an exception, the event still made it out into the wild?

But if it throws an exception, the NotificationSent event doesn't makes sense, isn't it?

I guess. :)

I'm just trying to explore the reasons for and against. This PR probably is fine. 👍


difference betwen dev & prod

Hey guys,
I have a problem with the datatable. In dev environement, the filters icon appears to be a css class. In prod environment, the filters icon's seems to be a img ( located in bundles/avanzuadmintheme/static/prod/styles/images/sort_asc.png ) but this doesnt exists (event after cache:clear, assets install, assetic:dump or fetch-vendors...)
Thank you for helping :)

Hmm, the prod directory seems to be outdated here.

had the same issue
2 solutions :)
using capifon to deploy my app into production - copying dev assets to prod
invoke_command "sh -c 'cd #{latest_release}/web/bundles/avanzuadmintheme/static && cp -pva dev/* prod/'"
or after deploy generate the prod assets :) - seems cleaner to me, but requires uglifyjs und uglifycss installed.
invoke_command "sh -c 'cd #{latest_release} && app/console avanzu:admin:build-assets --env=prod'"

Thank you @hjanuschka ,
an other solution I've found is to fetch vendors and install assets using the --env prod option.
php app/console avanzu:admin:fetch-vendor

app/console avanzu:admin:fetch-vendor --env=proddidn't work for me :D

@hjanuschka thank for pointing this out :)
if you like this adminthemebundle, have a look on this boilerplate I've made to kickstart project in a minute !  https://github.com/numerogeek/ultimate-symfony2-boilerplate

After fetching the vendor scripts (which should be optional since 1.3.0) you will need to regenerate the static assets as described here. The upgrade notice is obviously not very clear about that.


[AIRFLOW-481] Add Markovian

Dear Airflow Maintainers,
Please accept this PR that addresses the following issues:

https://issues.apache.org/jira/browse/AIRFLOW-481

Thx @waltherg  and welcome to the community!


Feat: Backend Integration for Scheduling of Post uploading

Describe the feature
I want to integrate the backend for scheduling the post where user can schedule the uploading of blogs
Add ScreenShots

Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[ ] I'm a VSoC'24 contributor
[X] I want to work on this issue

@Ayushmaanagarwal1211 you are assigned


added hola-mundo

What the program do ?
Added a simple html page with a js alert box.
In what programming language it is written?
html, js

THank you!


fix: remove max-workers flag for ansible

Description of the change:
Removes max-workers flag for ansible operators. They have been
replaced by --max-concurrent-reconciles.
Motivation for the change:
Follow up of #3435 on changing an implementation.
Checklist
If the pull request includes user-facing changes, extra documentation is required:

[ ] Add a new changelog fragment in changelog/fragments (see changelog/fragments/00-template.yaml)
[ ] Add or update relevant sections of the docs website in website/content/en/docs

Intentionally didn't add a changelog fragment here, because we already have a fragment for this in #3435 . But the changelog there says, its an addition for the flag instead of renaming. Can we create a separate PR for change that, or can that be changed manually later? @joelanford

@varshaprasad96 we can submit another PR to update the changelog fragment.


ModuleNotFoundError: No module named 'basicsr.archs.vgg_arch'

could you help me ?

The project is based on BasicSR. Before pushing the GitHub version, I deleted some unnecessary files while forgetting to delete the import code in the remained file.
You can delete the "from basicsr.archs.vgg.arch import VGGFeatureExtractor" in line 5 and delete the relevant code.


write_to_textfile: fails to overwrite file on windows

On linux, write_to_textfile can overwrite an existing file.
However, this fails if we try this on windows:
"C:\...\dashboard\metrics.py", line 169, in write_metrics
write_to_textfile(outfile, registry)
File "C:\...\env\lib\site-packages\prometheus_client\exposition.py", line 275, in write_to_textfile
os.rename(tmppath, path)
FileExistsError: [WinError 183] Cannot create a file when that file already exists: '..\metrics\cable_alert.prom.6092.3608' -> '..\metrics\cable_alert.prom'

should be good to close now with #650 merged. Thanks Chris!


Allow to override RELEASE operator references

This commits allows to override the Release operator image by setting the new image and tag in the RELEASE_IMAGE_REPO and RELEASE_IMAGE_TAG variables. It also allows to modify the reference to the SHA containing the operator resources.

I added another commit on this PR to be able to override the sha referencing the operator resources.

/retest

/retest


Fix Scratchpad assertion errors for sub-word acces

This would require RTL changes to the SCRATCHPAD wrappers to incorporate the "grounding" of d_channel data based on the a_channel mask (thus assuring X's never get written to the response fifo)
Or.... a simulation only initialization of scratchpad memory would suffice.
only applicable in simulating the ASIC version of the CEP, which does not initialize the scratchpad memory.
The ARM simulation model for the sramspo8192x64mt_rvt component does have tasks that support "backdoor" loading of main memory. These could be used to initialization the memory to all zeroes (or a backdoor load of code too)

The assertion is actually from the tlul_fifo component... which throws an assertion when an X gets read from the fifo.  Given that scratchpad accesses can be 8, 16, 32, or 64 bits.... this can happen a lot during the boot pocess.

As noted, this is only applicable when performing an ASIC simulation.  Performing the initialization in a way that does not substantially extend simulation time will require using some of the "hooks" (i.e., tasks) in the ASIC specific memory models.  As this is not applicable to the open source distribution, this issue will be closed... for now.


Autoindention after enter right paren/brace on empty row

This PR cover #93 except indention to different levels for paren and brace

This is fine, with one objection, use Microsoft.VisualStudio.Text.Span instead of Microsoft.VisualStudio.Text.ITrackingSpan (and textSnapshot.GetText(span) instead of trackingSpan.GetText(textSnapshot)). You want tracking spans if you plan to use a span that supports text edits between text snapshots. Since the spans in this code are discarded after use, plain Span is good enough and more idiomatic.

I make more ideomatic code.
@vosen, Is it enough for issue #93 or what else means in this one?


Package fontspec Error: The font "FontAwesome" cannot be found.

Existing state of 4be6c21 will not typeset examples on macOS and MacTex 2020 with font awesome bundled in repository  (v4.7.0). The bundled font is behind tex distribution current release (v5.13.0) but ahead of legacy (v<IP_ADDRESS>).

Appears related to https://github.com/posquit0/Awesome-CV/issues/181

Removing: https://github.com/spmcv/Awesome-CV/blob/4be6c213c8631a1fd68d1ad452e88b722b911f78/fontawesome.sty#L12
as mentioned in https://github.com/posquit0/Awesome-CV/issues/181#issuecomment-363289393 did indeed fix the issue.

This works because the font family is defined properly in awesome-cv.cls already.

i.e.: https://github.com/spmcv/Awesome-CV/blob/4be6c213c8631a1fd68d1ad452e88b722b911f78/awesome-cv.cls#L164


Feature/add dialog class

⚠  Requires #90 to be merged to master. Then rebase this branch
Add dialogClass option
Add the ability to specify a custom class for the modal-dialog
element, allowing users to change the width of the modal
Ref: https://getbootstrap.com/docs/5.0/components/modal/#optional-sizes
Supersedes #75, #88

Tested on BS3/Rails4.2/jQuery 1 and BS4/Rails6.1/jQuery 3


Updated .travis.yml to reflect changes in upcoming 1.0.0 release

Removed 5.3 and 5.4, added 5.5 and 5.6. Removed build.sh run to drastically cut down on test time.

Could we get the build.sh script to run only on the master branch? I agree it's slow and unnecessary for most pull requests, but it is useful for ensuring that the build script hasn't broken.

Travis will now check for the branch and if it is master, will run the build script.

:+1: Thanks!


Pre Launch Final Bug Squashing Pt 1

Types of changes

[x] Bugfix (change which fixes an issue)
[ ] New feature (change which adds functionality)
[ ] Refactor (change which changes the codebase without affecting its external behavior)
[ ] Non-breaking change (fix or feature –that would causes existing functionality to work as expected)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Purpose
Address black screen of death on History tab within React Router apps
Fix styling error in AtomsRelationship.tsx
Clean up code in StateRoutes
Clean up code in PerfView

Approach
Change line 111 in History.tsx. Now avoids a crash caused by moving to a route with no children.
        if (d.data.stateSnapshot.children[0].componentData.actualDuration){
          loadTime = d.data.stateSnapshot.children[0].componentData.actualDuration;
        } else{
          loadTime = 1;
        }

Set height of Sankey chart to 98% to allow space for buttons and fix layout bug.

Learning
Testing a broader range of apps exposed new bugs in code.

This PR was closed because it was made to the wrong repository!


Tests fail out the gate

Keep getting this error when testing the base model.
Is initialized!:
Error: failed to get recent blockhash: FetchError: request to http://localhost:8899/ failed, reason: connect ECONNREFUSED ::1:8899
I've tried while running validator locally and not. It doesn't work.
Is there a fix coming?

Tried to send on discord but as the many who have tried to verify will tell you. it doesn't work :/

FetchError: request to http://localhost:8899/ failed, reason: connect ECONNREFUSED

If you've started a validator locally I can't see how it is possible that nothing is running on this port.
How are you running the tests? Does solana-test-validator give you any error output?

Actually maybe use node 16 per https://github.com/solana-labs/example-helloworld/issues/392#issuecomment-1014965594

This could be related to Node version as the previous comment suggested but it also could be a Solana setup issue.
Closing because it's an old issue and there isn't enough information.


Minor changes to enable Linux testing

Handle line endings in query tests.
Fix #3658

cc @saurabh500

Clever. :smile:

:shipit:


Always use Cython implementation

What do you think of this? I had to introduce a hashable dummy type to work around https://github.com/google/jax/issues/4280, but I kinda like it because it uses the memory address of the underlying MPI object for caching (which should be more invariant than object id).
This removes quite a bit of boilerplate from the primitive implementations.
Fixes #9

I like the hashable wrapper idea.
I dislike the try-catch statement in to_mpi_ptr. Are you sure there is no other way?
I think we should also address #20 before merging this?

I can handle #20 but I don't see how it's related to this. Fixing #20 just means some additional checks in the xla_encode functions (which I don't touch here).  I can make another PR for the validation.
Why don't you like the try-except? The alternative is checking the type of the MPI object, and using addressof if it's MPI.Status and otherwise handleof. I like the duck approach a bit better but if you prefer I can change it to type checking (no strong opinion on that).

I'd expect a try-catch block to be much more expensive than checking if an object has the method handleof or not.
Then the code will be compiled in XLA so... who cares.
I'm not a python expert at all (Julia being my main language lately), so I trust your judgement in what is more efficient.
Your call!

No need to worry about performance here, this is in the microsecond range :)
(and is only executed during tracing)
try-except is more robust to upstream changes in mpi4py, so I'd rather keep that.
I'll resolve the conflicts and merge if there's nothing else?

yes


(fix) Correct font name

Correct broken url in docs/css/custom.css for Tajaval-Bold.

I intentionally used the extrabold ;)

Nothing I was wrong...


having a space in a .blend prevent mesh from being imported

Godot version
4.1.dev (4d5f10fc3a)
System information
macOS 13.3.0 - Vulkan (Forward+) - integrated Apple M1 Pro - Apple M1 Pro (8 Threads)
Issue description
When there is a space in the .blend file name the .bin file will have a corrupted name with %20instead of spaces. This will result in godot not finding the file and meshes not to be imported. Might be system/architecture dependant.
Steps to reproduce

open godot
add a .blend file with a space in it
See the errors

Minimal reproduction project
N/A

Duplicate of #76813 indeed, it's a Blender bug fixed in 3.6.


resolved meanings issues

The output of definitions have been cleared and simplified.

this PR creates no additional bugs or issues.
PLS do check.

This PR resolves issue #1


Push Rulesets

Repository Rules will now allow you to block commits on select file and path attributes.

Restrict file paths: Prevent commits that include changes in specified file paths from being pushed to the commit graph.
Restrict file path length: Prevent commits that include file paths that exceed a specified character limit from being pushed to the commit graph
Restrict file extensions: Prevent commits that include files with specified file extensions from being pushed to the commit graph.
Restrict file size: Prevent commits that exceed a specified file size limit from being pushed to the commit.

Repository and Organizations admins can control who can contribute to certain locations in a repository ensuring certain paths stay locked down. It also ensure the correct types of file are committed to a repo making it sure you don't accidentally add a binary file or SSH key to a repository.

🚢 This has shipped: https://github.blog/changelog/2024-04-18-push-rules-public-beta!