update OSAL for Class A & Associated unit tests

The currently released OSAL unit tests may not fully run with OSAL 4.1.1.
JSC has made updates so that they run with Linux and ARINC653 OSAL 4.1.1.
JSC is currently updating to work with VxWorks 6.7.
These updates are being tracked in the JSC subversion repo and need to be pushed into a proper git branch and further work continued from there.

Imported from trac issue 22.  Created by sduran on 2015-02-26T14:12:23, last modified: 2016-03-07T15:56:36

Trac comment by jphickey on 2015-02-26 14:26:37:
FYI - This is exactly the type of thing targeted by my trac #17.
The idea here is that all tests should run using ANY supported OSAL BSP, not just those that have been explicitly added to the unit test code.  This will make sure that the unit tests remain future proof as new BSPs are added.

Trac comment by jphickey on 2015-03-19 09:50:10:
The filesystem failures are (mostly) due to the BSP not creating the necessary mount points, and the Error Name failure was due to checking for a too-specific string that is just different in this version of OSAL.
These were all minor fixes - I rolled these into my current branch for trac #17 since this updated the pc-linux BSP for other reasons.  With this change the build tests all pass and it shows green.

Trac comment by glimes on 2015-02-26 15:33:40:
Current status for 32-bit X86 Linux unit tests as being run
by my tame Bamboo server today
on the ic-trac-18-2015-0212 branch
(currently at [changeset:f22ae86]) are:
{{{
FAILED [ ] OS_GetErrorName - #3 Undefined Error
FAILED [ ] OS_mv - #6 Nominal
FAILED [ ] OS_fsBytesFree - #4 Nominal
FAILED [ ] OS_mkfs - #5 Nominal
FAILED [ ] OS_initfs - #5 Nominal
FAILED [ ] OS_fsBlocksFree - #4 Nominal
}}}

The OS_GetErrorName failure is from ut_oscore_log.txt
The OS_mv failure is from ut_osfile_log.txt
The other four are from ut_osfilesys_log.txt

Trac comment by tngo on 2015-03-26 14:26:39:
The git osal branch created for this ticket is "trac-22-update-osal-unit-tests".

Trac comment by glimes on 2015-04-10 12:49:32:
Starting to merge your changes in now.
We may need to talk at some point about line-termination characters.

Trac comment by jphickey on 2015-06-04 16:19:44:
While these sound like great changes, I only request that these be pushed as separate change sets if possible, rather than one big lump-sum.  Can we possibly create a different trac ticket to merge in the MISRA/cppcheck/compiler warning fixes vs. the UT modifications?
Having smaller change sets just makes it easier to review and (usually) easier to merge them forward up to the latest development branch.

Trac comment by lprokop on 2015-06-04 15:51:20:
Expanding Scope of this ticket beyond unit tests to include vxworks OSAL source code updates to all OSAL files necessary to pursue a Class A safety-critical classification of this software for use by Orion and potentially others.  The CFS AES project performed static code analysis utilizing the Understand tool for all MISRA rules, ran static cppcheck tool for potential safety concerns, ran the existing unit tests on a vxworks/LEON3 target with !WindRiver Coverage analysis enabled to determine unit test code coverage, and held a general code inspection process.  Specific changes as a result will be as follows:

Updating all OSAL files to comply with "most" MISRA rules and cppcheck static analysis errors
Updating all OSAL files to remove compiler warnings
Expanding existing unit tests to provide full API-level test coverage based on exposed uncovered code
Adding a full ADDITIONAL unit test suite using UTAssert framework to provide full code and BRANCH level coverage of the OSAL source code.

Subsequent commits should address these issues.

Trac comment by lprokop on 2015-06-10 17:49:40:
commit:  [changeset:5d51482] osfile, osfilesys, ostimer changes for selected MISRA/code inspection findings
commit:  [changeset:c05b231] iosapi.c changes for selected MISRA/code inspection findings
commit:  [changeset:6bf0b29] osnetwork.c/osloader.c changes for MISRA/code inspection findings

Trac comment by sduran on 2015-06-10 10:57:34:
We are trying to check in the changes in small pieces.  Since the osal code and the matching unit tests have to be built with each other, we are keeping them in the same branch.

Trac comment by glimes on 2015-06-11 15:29:34:
I am a bit concerned about the "osprintf.c" sources,
which appear to be LGPL licensed (making our use of
this source a license violation).
If we can resolve the license issues, we would also want
to fix up the code to use va_arg properly as using pointer
manipulations in place of va_arg makes some presumptions
about the varargs implementation that are unsupportable.

Trac comment by jphickey on 2015-06-11 17:31:06:
My comments:

I agree fully with Greg's comment above regarding osprintf.c sources.  This appears to be an LGPL library and we cannot simply grab it and use it if that is the case.

I have started to review the changes in e.g. [changeset:5d51482] and I'm concerned that this appears to just be blindly changing {{{int}}} to {{{int32}}} any time the MISRA inspector complained about it.  This is not correct!

Just because MISRA points something out doesn't mean it's wrong.  {{{int}}} (not int32) is actually the correct C type to use in most of these cases //because the code is interacting with the C library//.
If a C library call defines the return type as an {{{int}}} we should store it in an {{{int}}}.  This is especially true if the library also defines constants to compare it against (e.g. the {{{OK}}} and {{{ERROR}}} constants).  It is entirely possible that a comparison such as:
{{{
if (result == ERROR)
}}}
Can be "true" if both {{{result}}} and {{{ERROR}}} are the same type (e.g. int), but "false" if they are different fundamental types (e.g. int32 and int).
Please see my CFE ticket for a detailed explanation of how this can happen:
https://babelfish.arc.nasa.gov/trac/cfs_cfe/ticket/26
(Note I didn't fix this for theoretical reasons - it actually doesn't equate on a 64-bit processor if you compare a "long" constant to a negative "int32" value, and this actually caused code to fail on a 64-bit build)

Trac comment by lprokop on 2015-06-17 17:31:48:
[changeset:7376f89] Added

Trac comment by lprokop on 2015-06-18 11:34:14:
Commit: [changeset:05a6a3f] More MISRA

Trac comment by glimes on 2015-06-18 11:43:08:
Is there a document handy that could be included in the project
to indicate exactly what rules are being enforced?

Trac comment by lprokop on 2015-06-18 15:38:10:
Replying to [comment:17 glimes]:

Is there a document handy that could be included in the project
to indicate exactly what rules are being enforced?
Not yet, but there will be.  I'm capturing it in a test plan, procedures & report document.

Trac comment by lprokop on 2015-06-25 17:05:14:
Replying to [comment:14 jphickey]:
Agreed on part 2 of this comment, Joe.   We were going back and forth on this one, change all so that you can clearly see everywhere that an "int" has to change if the common_types.h is changed based on migrating processors, or selectively in your case, not doing it for library or vxworks OS calls that expect or return the base type "int". Since they were ultimately the same base type, thought no harm, but good example.  So I went through all changes, and reverted those utilized in library/os calls or compared to return values for those functions.   Will commit this after testing... thanks!

Trac comment by lprokop on 2015-06-26 17:20:04:
commit: [changeset:ec67295]  Updated osapi.c for some race conditions, other files changed to revert blanket "int" to "int32" change retaining "int" with vxworks/lib functions.

Trac comment by abrown4 on 2015-07-06 19:09:37:
commit: [changeset:5b3e2b3] "Updated ostimer.c (to 100%) coverage with static fcn."
ostimer.c is now 100% covered, however tests fail given the issues that were discovered (see above).

Trac comment by abrown4 on 2015-07-01 17:42:03:
commit: [changeset:2c8edbe] Added ut-assert to unit-test-coverage
For coverage testing.  This can be moved or replaced later.
commit: [changeset:5cc4968] Added coverage tests for osnetwork.c
Identified issue #61 when testing with "make NONET=TRUE" (see Makefile).  100% coverage.
commit: [changeset:b412254] Add coverage tests for ostimer.c (with errors).
Identified issues #62, #65, #66, #69.  Not all tests pass in this changeset.  Maximum coverage achieved. (One statement, ostimer.c line 262, can't be covered due to the logic immediately preceeding it.)

Trac comment by abrown4 on 2015-07-07 18:00:53:
commit: [changeset:124dd01] Fixed incorrect logic in VxWorks OS_TimespecToUsec(), trac #62.
Built upon ostimer.c coverage tests on Track #22.

Trac comment by abrown4 on 2015-07-07 18:06:26:
commit: [changeset:baf36c4] Fixed incorrect logic in POSIX OS_TimespecToUsec(), Trac #62.
Built upon ostimer.c coverage tests on Track #22.

Trac comment by abrown4 on 2015-07-07 18:38:03:
commit: [changeset:cb5032b], Fixed "Unfreed" Table Entries after failure, VxWorks OS_TimerCreate().  Trac #66.
Built upon ostimer.c coverage tests on Track #22.

Trac comment by abrown4 on 2015-07-08 13:13:17:
commit: [changeset:1a6e450], Fixed VxWorks ostimer.c to properly use semaphore, Trac #69.
Built upon ostimer.c coverage tests on Track #22.
Made unit test changes to differentiate between failures due to Trac #69 and Trac #65.

Trac comment by abrown4 on 2015-07-09 14:10:21:
commit: [changeset:9fccc17] Switching default osnetwork.c coverage build to 32-bit.

Trac comment by abrown4 on 2015-07-08 16:50:40:
commit: [changeset:37dc473] Fixed ostimer.c to prevent unterminated strings, Track #65.
Built upon ostimer.c coverage tests on Track #22.  Bumped unit tests to 100% coverage with logic change.

Trac comment by dasp on 2015-07-14 11:19:42:
commit: [changeset:9519e1c7] Added UtAssert unit tests for osapi.c and osfileapi.c for VxWorks

Trac comment by abrown4 on 2015-07-13 19:23:17:
commit: [changeset:1e86ff7] Trac #22, Coverage tests for VxWorks osloader.c (in linux). Fixed header typos in osnetwork_testcase.c.

Trac comment by dasp on 2015-07-17 13:38:13:
commit:[changeset:730b2d2] Update to UtAssert unit test adaptor header file for osapi to fix compilation issue

Trac comment by abrown4 on 2015-07-15 11:57:17:
Identified issues #75, #76, #77, #78 for vxworks osloader.c.

Trac comment by abrown4 on 2015-07-17 18:03:09:
commit: [changeset:82f215e] Fixed osnetwork return codes with no network.  Fixes #61.
Tested with coverage tests on Trac #22 branch and in CFS_TST 0d3a682.

Trac comment by abrown4 on 2015-07-17 14:33:01:
Identified issues #80, #81, & #82 for osfileapi.c.
Identified issues #83, & #84 for osfileapi.c.

Trac comment by abrown4 on 2015-07-20 16:14:49:
commit [changeset:ecfe66e]  Trac #76, osloader mutex unit test update.
commit [changeset:be8d33e]  Trac #76, osloader mutex fix.

Trac comment by abrown4 on 2015-07-20 17:20:09:
commit: [changeset:8e3608d] Trac #78, Fixed OS_SymTableIterator() unchecked write().
Added a unit test to confirm fix.  Updated hardcoded values in other unit tests (no change in behavior or coverage).

Trac comment by abrown4 on 2015-07-20 16:48:17:
commit: [changeset:17a4c30] Trac #77, Fixed vxworks OS_SymTableIterator() name arg too long.

Trac comment by abrown4 on 2015-07-20 16:30:59:
commit: [changeset:0ca6b09] Fixed osloader OS_ModuleLoad() bug when module_name too long.  (Trac #75)

Trac comment by abrown4 on 2015-07-20 19:00:15:
commit: [changeset:803fe97] Trac #82, Fixed OS_GetErrorName() missing codes.  Also fixed the incorrect return for OS_ERROR_ADDRESS_MISALIGNED.

Trac comment by dasp on 2015-07-23 11:46:06:
commit: [changeset:0c91d99] Initial commit of UtAssert tests for osfilesys.c
commit: [changeset:0ac78a6] Updated UtAssert tests for osfilesys.c and files for developing tests in Linux

Trac comment by abrown4 on 2015-07-23 12:46:59:
Replying to [comment:41 jphickey]:

Please note again that there is UTassert framework and filesystem tests on the mainline development branch.  The last commit appears to be linux-targeted and any changes here really should be on the mainline branch.  (I was under the impression this branch was Vxworks-specific class A stuff only)

FYI, the commits here in the OSAL src/unit-tests-coverage/osfilesys-test handle two separate issues:

the unit test .c's & .h's used for the VxWorks OSAL coverage (_wb) tests
a Makefile and fakes/stubs for use on linux, where we can stub out and develop the tests much more quickly than for the VxWorks target.

We're committing the extra linux Makefiles & linux fakes so that we can share them and run each others' tests on linux first for development/debugging.  This has been an immense time-saver.  Sure, the organization may need work, but we can work on directory organization after we get the tests all up and going.

Trac comment by jphickey on 2015-07-23 11:51:38:
Please note again that there is UTassert framework and filesystem tests on the mainline development branch.  The last commit appears to be linux-targeted and any changes here really should be on the mainline branch.  (I was under the impression this branch was Vxworks-specific class A stuff only)

Trac comment by glimes on 2015-07-23 14:02:09:
OS_ShellOutputToFile case 4 in osfile is fragile: if "osapi.o" does not
exist in the current working directory when the test program is executed,
the test fails (and fails to tell us why).
This will break when we stage the test to a test target machine.
Because I suspect this isn't going to be high on your priority list,
I will augment my test setup scripts to work around the problem.
I mention this because ... well, that's the only real test failure
that I find when I run the results of merging trac-22 with development
on the 32-bit X86 PC-Linux test target.
Oh, did I mention that I'm successfully building the merge of trac-22
with development, and it is going fairly well? :) ;)
[edit below: removed changeset links as they are changing too rapidly
for a changeset link to be useful.]
So yes, I have a Bamboo plan branch (and an OSAL branch) that now
successfully run your tests, and a second set that runs the merge of your
branch with development. While I still have to update those branches manually
having these available should be helpful.
Your commit pace is fast enough that it seemed like a good idea to
get the bulk of the merge work done in advance, and to try to track
your updates periodically, so that when you are happy with the set
of tests you need for OSAL 4.1.1 we can bring them over to development
without months of merge pain ;)
Current reference information for the test branches and Bamboo plans,
as of last night's scheduled build:
babelfish/ic-gll-jsc-update
This branch contains ephemeral changes I need to apply so that
my existing scripts can build and run your tests. This branch
will be rebased and amended, so ''don't branch from it''.
This commit will descend from a recent commit on the trac-22
branch, usually the one at the tip when I do my update.
Sorry, no changeset link -- this is getting rebased and amended
so frequently that a changeset link would not be useful.
https://babelfish.arc.nasa.gov/bamboo/browse/CFS-CFSOSAL60/latest
This is where you find the latest build results for my slightly
edited version of your branch.
(wow, we've burned through 60 plan branches already!)
This builds and tests with the classic build mechanism.
It is possible to examine the Bamboo logs to find out exactly
which commit was used. If we have to do this often, I will
figure out a way to make it easier.
babelfish/ic-gll-jsc-merge
This is the merge of your branch with development (it does not
include the workaround branch mentioned above). For the record,
I used a tool called "git imerge" in combination with a meld hack
that helps resolve conflicts involving end-of-line changes.
Sorry, no changeset link -- this is getting rebased and amended
so frequently that a changeset link would not be useful.
This will usually merge from the same commit of trac-22 used
as the basis for the update branch.
This also merges in changes I need to make to development in order
to bring "classic build and test" back to a working condition.
This branch may have inconsistant end-of-line markers, inconsistant
whitespace, and an inconsistant idea of the width of a tab, as it is
intended to be as close to a "mechanical merge" as possible (to make
future merges easy).
As above, this branch will be rebased and amended, so ''don't branch from it''.
https://babelfish.arc.nasa.gov/bamboo/browse/CFS-CFSOSAL58/latest
This is where you find the latest build results for this branch.
This builds and tests with the classic build '''and''' the CMake system.
It is possible to examine the Bamboo logs to find out exactly
which commit was used. If we have to do this often, I will
figure out a way to make it easier.

Trac comment by jphickey on 2015-07-23 12:58:01:
Understood that it is easier to run fakes/stubs on Linux - my main concern is that I see unit test code that is going to conflict/duplicate/overlap stuff that is already on mainline development.
Has there been any consideration to rebase/merge this stuff with development?  There are quite a few commits on this branch and I would really like to see some of them get merged in so we can start using these improvements.  I believe this would also benefit your development efforts in that you would not need to re-do this unit test stuff and you could leverage what is already there, too.
FYI - WRT VxWorks, very very little has changed between {{{master}}} and {{{development}}} at least within the real implementation part i.e. the {{{src/os/vxworks}}} directory.

Trac comment by dasp on 2015-07-23 15:16:21:
commit: [changeset:0f36fb6] Minor updates to compile osfilesys.c tests on VxWorks

Trac comment by abrown4 on 2015-07-24 15:35:19:
commit: [changeset:67e90ac] Trac #81, fixed OS_Milli2Ticks() out of range bugs.
commit: [changeset:495f229] Trac #22, osfile_wb test bugfixes for VxWorks execution.
commit: [changeset:301a46a] Trac #22, fixed the osfile_wb test hanging on vxworks.

Trac comment by abrown4 on 2015-07-24 18:39:17:
Merge for ticket #80 onto branch #22.  These changes fix everything except for the *TimedWait() and *SemTake() -type functions.
commit: [changeset:1ab6906] Trac #80, fixed osapi.c task semaphore usage.
commit: [changeset:6b37df9] Trac #80, fixed osapi.c queue semaphore usage (except for OS_QueuePut() ).
commit: [changeset:d19e229] Trac #80, fixed osapi.c binary semaphore usage, except for OS_BinSemTimedWait().
commit: [changeset:a18e1d0] Trac #80, fixed osapi.c queue semaphore usage in OS_QueuePut().
commit: [changeset:be741ce] Trac #80, Continue osapi.c semaphore fixes.  Fixes to OS_QueuePut(), OS_BinSemGive(), OS_BinSemFlush().  Rolling back changes to: OS_BinSemTake(), OS_BinSemTimedWait() pending design decision.
commit: [changeset:2150e71] Trac #80, updated unit tests for binary semaphores.
commit: [changeset:c527718] Trac #80, osapi.c counting semaphore fixes (with exceptions).  Note: OS_CountSemTake() and OS_CountSemTimedWait() are not protected.
commit: [changeset:596f739] Trac #80, osapi.c mutex semaphore table fixes (with an exception) Note: OS_MutSemTake() was not changed.

Trac comment by abrown4 on 2015-07-24 18:53:38:
commit: [changeset:0383f17] Trac #81, Fixed osapi.c OS_Milli2Ticks() to return OS_ERROR.  Also removed hardcoded number in favor of including limits.h.

Trac comment by abrown4 on 2015-07-24 20:13:43:
commit: [changeset:41d0208] Trac #22, added osfilesys-test to linux Makefile.
commit: [changeset:d114775] Trac #22, linux osfilesys-test/Makefile writes to log.
commit: [changeset:d642ff9] Trac #22, Made oscore_wb testcase have similar stats on all platforms. The number of tests being run/pass/fail were slightly different.  (Now they're identical, useful for debugging.)

Trac comment by abrown4 on 2015-07-24 20:52:15:
Replying to [comment:45 dasp]:

commit: [changeset:0f36fb6] Minor updates to compile osfilesys.c tests on VxWorks

Identified issues #85, #86, #87, #88, #89, #90, #91, & #92 for osfilesys.c thanks to Alan's hard work.  Whew!

Trac comment by abrown4 on 2015-07-28 14:58:37:
commit: [changeset:270c2dc] Trac #69, Removed table locks in ostimer.c OS_TimerSignalHandler().
Locking a global table while calling a non-osal callback isn't a good idea.

Trac comment by abrown4 on 2015-07-27 20:09:51:
commit: [changeset:a0f957a] Trac #80, osfileapi.c semaphore locking fixes & rollbacks.
Rolled back many of the simple table accesses for risk reduction.
Added selected lock/unlock pairs for functions that may block or yield
during table modifications.

Trac comment by abrown4 on 2015-07-28 16:48:49:
commit: [changeset:76b7d35]  Trac #83, Fixed some osfileapi.c global table accesses.
OS_CloseFileByName() and OS_CloseAllFiles() peform the syscall close()
inside the mutex lock.  Changed to be like OS_close(), where close() is
called outside the lock.

Trac comment by abrown4 on 2015-08-19 19:29:19:
OSAL code inspection held Wed, 8/19/2015.  Comments were approved for updates - to be done in this ticket.

Trac comment by abrown4 on 2015-08-19 20:46:01:
commit: Trac #22, 8/19 Inspection item 7 (in excel sheet), [changeset:0a3a4f6] to [changeset:a4817be]. (variable initialization in all osal .c files)

Trac comment by abrown4 on 2015-08-19 19:30:16:
While performing code inspection updates, identified new issues #94, #95, and #96.

Trac comment by abrown4 on 2015-08-20 14:27:28:
commit: Trac #22, 8/19 Inspection items 9-33 (in excel sheet), [changeset:497c856] to [changeset:10c024d]. (osapi.c, osapi.h, and notes in all osal .c files)

Trac comment by abrown4 on 2015-08-24 17:00:46:
commit: [changeset:a1a870e] Trac #96, Removed unused variables from vxworks OS_TranslatePath(), osfilesys.c.

Trac comment by abrown4 on 2015-08-20 15:06:52:
commit: Trac #22, 8/19 Inspection items 44-45 (in excel sheet), [changeset:aa433bf] to [changeset:42565b2]. (osfilesys.c)

Trac comment by abrown4 on 2015-08-20 16:38:46:
commit: Trac #22, 8/19 Inspection items 50-60 (in excel sheet), [changeset:abdf238] to [changeset:6a9db70]. (osloader.c, osnetwork,c. and ostimer.c)
This should complete the changes from the 8/19 inspection comments.

Trac comment by abrown4 on 2015-08-24 17:38:41:
commit: [changeset:33afe2b] Trac #94, Fixed vxworks OS_unmount() to catch OS_TranslatePath() failures.
commit: [changeset:40531d9] Trac #94, Fixed vxworks OS_chkfs() to catch OS_TranslatePath() failures. Improved similar logic for OS_unmount() check as well.

Trac comment by abrown4 on 2015-09-09 11:12:45:
commit: [changeset:0df2dad] Trac #22, Fixed internal type mismatch on return type in OS_initfs, osfilesys.c.

Trac comment by glimes on 2016-03-07 15:56:36:
We believe we have picked all the meat off of these bones, if there is
anything else that needs to go in, we will want a followup ticket.


(SERVER-2259) Add clean action

This provides a clean action. The inputs (and their validation) is the same as the revoke action. The actual action is a superset of revocation (send a PUT to revoke and then a DELETE to clean up after ourselves). Consequently a lot of this is directly boilerplate from revoke.
I assume we might want to create a CA object that we can delegate the shared behaviors and validation to? Thought I'd run doing that by folks (in this or another PR).
At a minimum this needs a rebase, but I'ld love to get folks feedback on the approach/see if I'm missing anything.

Going to try this out.

Updated the comment and rebased.

This looks good.  Do we need to do any manual testing?

I'll try this out minus the use of the generate command on the VM I set up earlier.

rebased, think this is good to go now!


feat: added dependency info in csharp

Description

Updated docs/language/Csharp.md

Related issue(s)

Fixes #446

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 93.542%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2459

Relevant Lines:
2505

💛  - Coveralls

@jonaslagoni  please review 🙂

/rtm


Correctly handle handshake messages in ssl_read if renegotiation is disabled

Context: Unexpected handshake messages lead to connection failure in mbedtls_ssl_read when MBEDTLS_SSL_RENEGOTIATION is disabled. In the presence of package duplication, this can constitute a severe problem, as e.g. duplicated and delayed Finished messages would bring down the connection after the initial handshake has completed and application data is expected next. This was observed here.
Changes: This PR fixes the issue by restricting the MBEDTLS_SSL_RENEGOTIATION-guard in mbedtls_ssl_read to the code-path accepting renegotiations and the code-path aborting pending renegotiations. Further, it adds missing dependencies on MBEDTLS_SSL_RENEGOTIATION to some tests in ssl-opt.sh and includes a build and test run for !MBEDTLS_SSL_RENEGOTIATION to all.sh. Finally, some documentation improvements are made along the way.
Internal Reference: IOTSSL-1707

@hanno-arm Ok by me for you to rebase.

@mpg @gilles-peskine-arm Rebased following @mpg's suggestion. The previous branch is retained at https://github.com/hanno-arm/mbedtls/commits/iotssl-1707_backup and you can check that the diff between its head ce39904 and the new head 479e8e2 is empty.

I checked that the rebased version has an empty diff with the version I reviewed, so my approval is still valid.

@mpg @gilles-peskine-arm Opened the backport to Mbed TLS 2.1 as #1153. A backport to Mbed TLS 1.3 doesn't seem necessary as the issue only appears for DTLS. Do you agree?

@hanno-arm The motivating issue and main problematic use case is DTLS, but the code is also suboptimal in the plain TLS case. It results in the code attempting to decode the handshake message and rejecting the data later. This could potentially be a step in an attack path against some other validation step. So I think it should be backported to 1.3 as well for robustness.

@gilles-peskine-arm Oh, and also for TLS, even if MBEDTLS_SSL_RENEGOTIATION is disabled we should gracefully reject a renegotiation attempt from the peer even instead of failing.

CI failed only due to timing test (known issue), ok to merge


⚠ Bofrost.es has degraded performance

In ac63038, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1412 ms

Resolved: Bofrost.es performance has improved in 68715a1 after 20 minutes.


feat: [NPM] cleanup command & label node

Reason for Change:
Help AKS disable NPM on existing clusters.
Issue Fixed:
Requirements:

[x] uses conventional commit messages
[ ] includes documentation
[x] adds unit tests
[x] relevant PR labels added

Notes:
Cleanup Command
You can use the following on the NPM container to clean the kernel up:
command: ['/usr/bin/azure-npm', 'cleanup']

Node Labels
NPM will now add this Label to its Node: kubernetes.azure.com/azure-npm=installed.
When running the cleanup command, NPM will add this Label instead: kubernetes.azure.com/azure-npm=removed.

just curious, when would there case be that a node wouldn't have npm scheduled?

just curious, when would there be the case that a node wouldn't have npm scheduled?

In the future, AKS may allow users to disable NPM in an existing cluster, i.e., to change --network-policy=azure to --network-policy=none. This would e.g. help a customer upgrade from a cluster with NPM to a cluster with Cilium
During that upgrade, new nodes won't have NPM installed, and azure-npm=installed serves as a marker for nodes that used to have NPM

/azp run

/azp run

No worries, let me close this


Error when Importing DTDLs generated in IoT Central using ADT Explorer, CLI works fine

The dtdls are simple, created in Azure IoT Central and export directly, showed below:
[
  {
    "@id": "dtmi:dtdlv2:TempSensor:original;1",
    "@type": "Interface",
    "contents": [
      {
        "@id": "dtmi:dtdlv2:TempSensor:Temperature;1",
        "@type": [
          "Property",
          "Temperature"
        ],
        "displayName": {
          "en": "Temperature"
        },
        "name": "Temperature",
        "schema": "double",
        "unit": "degreeCelsius",
        "writable": true
      },
      {
        "@id": "dtmi:dtdlv2:TempSensor:Humidity;1",
        "@type": [
          "Property",
          "RelativeHumidity"
        ],
        "displayName": {
          "en": "Humidity"
        },
        "name": "Humidity",
        "schema": "double",
        "unit": "percent",
        "writable": true
      }
    ],
    "displayName": {
      "en": "TempSensor"
    },
    "@context": [
      "dtmi:iotcentral:context;2",
      "dtmi:dtdl:context;2"
    ]
  }
]

Following errors will pop up when import using ADT Explorer:

DocumentParseError: Failed to parse input. Error: jsonld.InvalidUrl: Dereferencing a URL did not result in a valid JSON-LD object. Possible causes are an inaccessible URL perhaps due to a same-origin policy (ensure the server uses CORS if you are using client-side JavaScript), too many redirects, a non-JSON response, or more than one HTTP Link Header was provided for a remote context.
at n. (https://explorer.digitaltwins.azure.net/static/js/2.2bafa1d5.chunk.js:2:7913129)
at c (https://explorer.digitaltwins.azure.net/static/js/2.2bafa1d5.chunk.js:2:7185557)
at Generator._invoke (https://explorer.digitaltwins.azure.net/static/js/2.2bafa1d5.chunk.js:2:7185345)
at Generator.E.forEach.e. [as throw] (https://explorer.digitaltwins.azure.net/static/js/2.2bafa1d5.chunk.js:2:7185980)
at s (https://explorer.digitaltwins.azure.net/static/js/2.2bafa1d5.chunk.js:2:7898628)

Need to remove "dtmi:iotcentral:context;2" from "@context" in order to perform a successfully import using ADT Explorer.
OR using CLI, the import will success without editing the original file.

Hi @pczhao1210 - I believe this should have been addressed in this commit https://github.com/Azure-Samples/digital-twins-explorer/commit/416aa851c24fd6b75b2c5b2dfd71bf12597318d3.
Let me know if you still see the issue!


.hex Files

On the FizzBuzz examples, the extension is .hex, which is a file encoded in hexadecimal. I reckon that the .hxg file extension is better. Also, the other files have the extension .hxg, so why not that one as well?

Oh, you're right. That was from an old version of the Esoteric IDE. I had
already fixed it but forgot to commit it. Will push those files later.
On Sun, Nov 15, 2015 at 9:24 AM, George Gibson<EMAIL_ADDRESS>wrote:

On the FizzBuzz examples, the extension is .hex, which is a file encoded
in hexadecimal http://file.org/extension/hex. I reckon that the .hxg
file extension is better.
—
Reply to this email directly or view it on GitHub
https://github.com/mbuettner/hexagony/issues/3.


Adonis azure

Help adonis publish azure.
Im try long time but no work adonis in azure.

Hey @fabiossantoss! 👋
This is more a question than an issue with the framework itself.
Please use the forum or the Discord Server for this kind of request.


Graphite-web Upgrade error

I'm trying to upgrade graphite-web to 1.1.4 from 0.9.15. I'm getting the below error when trying to execute command: django-admin.py runserver --pythonpath /opt/graphite/webapp --settings graphite.settings <IP_ADDRESS>:8080
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Performing system checks...

System check identified no issues (0 silenced).

You have 20 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener.
Run 'python manage.py migrate' to apply them.

December 07, 2018 - 20:51:41
Django version 1.11, using settings 'graphite.settings'
Starting development server at http://<IP_ADDRESS>:8080/
Quit the server with CONTROL-C.
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Unhandled exception in thread started by <function wrapper at 0x7f1b85a6c758>
Traceback (most recent call last):
  File "/usr/local/lib64/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib64/python2.7/site-packages/django/core/management/commands/runserver.py", line 117, in inner_run
    autoreload.raise_last_exception()
  File "/usr/local/lib64/python2.7/site-packages/django/utils/autoreload.py", line 250, in raise_last_exception
    six.reraise(*_exception)
  File "/usr/local/lib64/python2.7/site-packages/django/utils/autoreload.py", line 227, in wrapper
    fn(*args, **kwargs)
  File "/usr/local/lib64/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/local/lib64/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models()
  File "/usr/local/lib64/python2.7/site-packages/django/apps/config.py", line 202, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/opt/graphite/webapp/graphite/events/models.py", line 4, in <module>
    from tagging.models import Tag
  File "/usr/local/lib/python2.7/site-packages/tagging/models.py", line 10, in <module>
    from django.contrib.contenttypes.generic import GenericForeignKey
ImportError: No module named generic

Command pip show django whipser carbon graphite-web results in
Name: Django
Version: 1.11
Summary: A high-level Python Web framework that encourages rapid development and clean, pragmatic design.
Home-page: https://www.djangoproject.com/
Author: Django Software Foundation
Author-email<EMAIL_ADDRESS>License: BSD
Location: /usr/local/lib64/python2.7/site-packages
Requires: pytz
Required-by:

Any help appreciated, thanks.

Could you check if your django-tagging module is out of date? I suspect it needs to be updated.

I updated django-tagging to 0.4.5. I'm still getting the same error.

I bumped it to 0.4.6.
Requirement already satisfied: django-tagging in /usr/local/lib/python2.7/site-packages (0.4.6)

Is there another way to check what version of django-tagging I have?

check /usr/local/lib/python2.7/site-packages/tagging/models.py and see what is being imported, if it is from django.contrib.contenttypes.generic import GenericForeignKey then there is something off. It mus not contain the .generic part.

# For the ImportError: No module named fields. http://www.reemachugani.com/blog/2015/feb/14/no-module-named-fields-importing-GenericForeignKey/
# In the official django documentation for ContentTypes, the import statement used for GenericForeignKey is given as - `from django.contrib.contenttypes.fields import GenericForeignKey`
# This will raise an ImportError. Replace it with the following - `from django.contrib.contenttypes.generic import GenericForeignKey`
cd /usr/local/lib/python2.7/site-packages
find . -type f -exec sed -i 's/django.contrib.contenttypes.fields/django.contrib.contenttypes.generic/' {} +

Found the culprit, this was replacing the fields with generic.
The error is gone, but graphite doesn't start up
I'm following this doc: https://www.digitalocean.com/community/tutorials/how-to-keep-effective-historical-logs-with-graphite-carbon-and-collectd-on-centos-7 to see what am I doing wrong.

Documentation looks sane. What do you mean when saying "graphite doesn't start up"? graphite-web doesn't start? apache doesn't start? carbon doesn't start? what's in the logs?

As in the dashboard UI doesn't load.
django-admin.py runserver --pythonpath /opt/graphite/webapp --settings graphite.settings <IP_ADDRESS>:8080
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Performing system checks...

System check identified no issues (0 silenced).

You have 21 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener.
Run 'python manage.py migrate' to apply them.

December 09, 2018 - 18:35:56
Django version 1.11, using settings 'graphite.settings'
Starting development server at http://<IP_ADDRESS>:8080/
Quit the server with CONTROL-C.

This is what I see in UI.
This page isn’t working dashboard-stress-internal.devaws.net didn’t send any data.
ERR_EMPTY_RESPONSE

Did you try setting the SECRET_KEY and running the migrations as instructed in the log output above?

@DanCech I'll try that.

@DanCech @piotr1212
I tried a few combination of versions for a couple of days. I was not able to solve it, here are the details.
Versions:
django: 1.11.17
twisted: 18.7.0
graphite-web: 1.1.4
whisper: 1.1.4
carbon: 1.1.4
go-carbon: 0.13.0
https://github.com/lomik/go-carbon/releases/download/v0.13.0/go-carbon-0.13.0-1.x86_64.rpm
PYTHONPATH="/opt/graphite/lib/:/opt/graphite/webapp/"
This is the error I'm getting now.
[ec2-user@ip ~]$ ./dashboard.sh
starting carbon
2018/12/11 18:30:40 [WARNING] `common.log-level` and `common.logfile` is DEPRICATED. Use `logging` config section
starting graphite-web
starting dashboard
nohup: redirecting stderr to stdout
Starting dxedge-dashboard at port 9000...                             ERROR [Returned: 7]
ERROR [no response from server, timeout]

dashboard.sh
#!/bin/bash

source /home/ec2-user/.bashrc
export PATH=$PATH:/usr/local/bin

echo "starting carbon"
go-carbon -config /usr/local/etc/go-carbon.conf -daemon

echo "starting graphite-web"
nohup /opt/graphite/bin/run-graphite-devel-server.py /opt/graphite > graphite.out &

cd {{ dashboard_home }}
echo "starting dashboard"
mkdir logs
bin/service.sh start {{ dashboard_port }}

The same set up works fine for below versions.
Versions:
django: 1.6.7
twisted: 13.1.0
graphite-web: 0.9.15
whisper: 0.9.15
carbon: 0.9.15
go-carbon: 0.9.1
https://github.com/lomik/go-carbon/releases/download/v0.9.1/go-carbon-0.9.1-1.el6.x86_64.rpm
I'm using ansible for automation and I have removed this bit of code and there are no script changes apart from the version changes.
# Fix problems with Python imports.
# Because `defaults` is deprecated. See http://stackoverflow.com/questions/21058177/no-module-named-defaults-django-simple-friends
cd /opt/graphite
find . -type f -exec sed -i 's/django.conf.urls.defaults/django.conf.urls/' {} +
# For the ImportError: No module named fields. http://www.reemachugani.com/blog/2015/feb/14/no-module-named-fields-importing-GenericForeignKey/
# In the official django documentation for ContentTypes, the import statement used for GenericForeignKey is given as - `from django.contrib.contenttypes.fields import GenericForeignKey`
# This will raise an ImportError. Replace it with the following - `from django.contrib.contenttypes.generic import GenericForeignKey`
cd /usr/local/lib/python2.7/site-packages
find . -type f -exec sed -i 's/django.contrib.contenttypes.fields/django.contrib.contenttypes.generic/' {} +

Would appreciate any help on this.

@njchandu : unfortunately, scripts above (e.g. bin/service.sh start ) are not part of Graphite and we can't say anything about them or Ansible automation.
But from information above I can say that you didn't apply migrations, please do that using something like
cd /opt/graphite
sudo PYTHONPATH=/opt/graphite/webapp/ django-admin.py syncdb --settings=graphite.settings

or
cd /opt/graphite
sudo PYTHONPATH=/opt/graphite/webapp/ django-admin.py migrate --settings=graphite.settings --run-syncdb

I tried django-admin.py migrate --settings=graphite.settings --run-syncdb
That seemed to go through successfully. Here is the output.
/opt/graphite/webapp/graphite/settings.py:332: UserWarning: SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security
  warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security')
Operations to perform:
  Synchronize unmigrated apps: browser, composer, functions, metrics, render, staticfiles, whitelist
  Apply all migrations: account, admin, auth, contenttypes, dashboard, events, sessions, tagging, tags, url_shortener
Synchronizing apps without migrations:
  Creating tables...
    Running deferred SQL...
Running migrations:
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying account.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying dashboard.0001_initial... OK
  Applying events.0001_initial... OK
  Applying sessions.0001_initial... OK
  Applying tagging.0001_initial... OK
  Applying tagging.0002_on_delete... OK
  Applying tags.0001_initial... OK
  Applying url_shortener.0001_initial... OK

bin/service.sh start is this script https://gist.github.com/RadoBuransky/9128967
Upon starting the service by ./dashboard.sh I'm getting the below output
[ec2-user@ip-172-17-71-229 ~]$ ./dashboard.sh
starting carbon
2018/12/13 01:30:37 [WARNING] `common.log-level` and `common.logfile` is DEPRICATED. Use `logging` config section
starting graphite-web
starting dashboard
mkdir: cannot create directory ‘logs’: File exists
nohup: redirecting stderr to stdout
Starting dxedge-dashboard at port 9000...                             ERROR [Returned: 7]
ERROR [no response from server, timeout]

@njchandu : this script is not part of Graphite installation and not required for its work, and we not able to provide help about it.
Graphite-web is working in port 8080, try to open that port in web browser. Is it working? If not - what in the logs?

@deniszh I will re write the scripts and update it here.

@njchandu : no need to touch script for now, just use browser...

@deniszh where can I find the log file of graphite-web?
Thanks

Usually it's in /opt/graphite/webapp/log but it's configurable.

@deniszh
I believe it is breaking here
+ echo 'Moving graphite storage folders'
+ mkdir /mnt/data/graphite
+ mv /opt/graphite/storage /mnt/data/graphite/storage
+ ln -s /mnt/data/graphite/storage /opt/graphite/storage
+ chown -R ec2-user:ec2-user /mnt/data/
+ cd /opt/graphite/webapp/graphite/
+ python ./manage.py syncdb --noinput
python: can't open file './manage.py': [Errno 2] No such file or directory

and the script is
function configure_graphite(){
  echo "Graphite/Carbon configuration"
  # Update links
  GRAPHITE_DATA=${DATA_MOUNT}/graphite
  if [ ! -d "${GRAPHITE_DATA}" ]; then
    echo "Moving graphite storage folders"
    mkdir ${GRAPHITE_DATA}
    mv /opt/graphite/storage ${GRAPHITE_DATA}/storage
  else
    echo "Graphite storage folders exist - removing"
    rm -r /opt/graphite/storage
  fi
  ln -s ${DATA_MOUNT}/graphite/storage /opt/graphite/storage
  chown -R ec2-user:ec2-user ${DATA_MOUNT}/

  cd /opt/graphite/webapp/graphite/
  python ./manage.py syncdb --noinput
  chown -R ec2-user:ec2-user ${DATA_MOUNT}/graphite/storage
  chown -R ec2-user:ec2-user /opt/graphite/storage
}

Sorry, @njchandu, I'm really confused. 3 days ago you said that command was there and you ran it successfully - https://github.com/graphite-project/graphite-web/issues/2391#issuecomment-446809009
Even when you started this thread django-admin.py was there...
django-admin.py is part of Django installation if it's not there - then Django not installed properly.
Probably, you need to describe your setup and what you trying to achieve more thoroughly.

@deniszh
Apologies that happened because the set was happening from a root user. It works fine now. But, it doesn't load the data on the dashboard. The left navigation bar doesn't load. This is what I notice in the logs.
[16/Dec/2018 13:56:40] "GET / HTTP/1.1" 200 598
[16/Dec/2018 13:56:41] "GET /browser/header HTTP/1.1" 200 1167
[16/Dec/2018 13:56:41] "GET /composer? HTTP/1.1" 200 1643
[16/Dec/2018 13:56:41] "GET /static/js/browser.js HTTP/1.1" 404 94
[16/Dec/2018 13:56:41] "GET /static/js/ext/resources/css/ext-all.css HTTP/1.1" 404 113
[16/Dec/2018 13:56:41] "GET /static/js/ext/adapter/ext/ext-base.js HTTP/1.1" 404 111
[16/Dec/2018 13:56:41] "GET /static/js/ext/ext-all.js HTTP/1.1" 404 98
[16/Dec/2018 13:56:41] "GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[16/Dec/2018 13:56:41] "GET /static/js/composer.js HTTP/1.1" 404 95
[16/Dec/2018 13:56:41] "GET /static/js/completer.js HTTP/1.1" 404 96
[16/Dec/2018 13:56:41] "GET /static/img/carbon-fiber.png HTTP/1.1" 404 101
[16/Dec/2018 13:56:41] "GET /static/img/graphite-logo.png HTTP/1.1" 404 102
[16/Dec/2018 13:56:41] "GET /static/js/browser.js HTTP/1.1" 404 94
[16/Dec/2018 13:56:42] "GET /static/js/composer_widgets.js HTTP/1.1" 404 103
[16/Dec/2018 13:56:42] "GET /static/js/composer.js HTTP/1.1" 404 95
[16/Dec/2018 13:56:42] "GET /static/js/completer.js HTTP/1.1" 404 96

@njchandu:
you need to run collectstatic command python django-admin.py collectstatic --settings=graphite.settings

Hi @piotr1212 , I'm getting the same error when starting the django server with django-admin.py runserver --pythonpath /opt/graphite/webapp --settings graphite.settings <IP_ADDRESS>:8080

There are some issue with the way that the Django internal server handles static content, I can't remember the details, I think there was some security setting blocking this.
But why do you want to run the Django server? The recommended way is to use one of: apache+mod_wsgi, gunicorn, or uwsgi+nginx. as described in the documents https://graphite.readthedocs.io/en/latest/config-webapp.html

The original issue as reported seems to be solved, this is getting off-topic, closing. If you still have troubles please open a new issue.


Make trident ready for GitOps / ArgoCD

Hi,
we are using ArgoCD - a GitOps tool  - to configure our OpenShift clusters.
Every application in the cluster is described with YAML files which are in a Git repository. If someone wants to change a configuration of the cluster or manifests of an application running in the cluster he must create a pull request for that. If that PR is approved, the changes get merged in the Git repository. This triggers the GitOps tool (ArgoCD in our case) which applies the new manifests to the Kubernetes cluster.
Currently my problem is that Trident doesn't seem to be ready for it. For sure it creates some YAML files with --generate-custom-yaml but it does not create all manifests it will apply to the cluster on installation.
For instance the secret 'trident-csi' with lots of certificates is not generated. Because of that I can't use the YAML files directly in my Git repo to install trident.
My current workaround is to put a Kubernetes Job manifest in the Git repo which calls tridentctl in a trident container and afterwards it creates a backend by calling tridentctl create backend ... again.
After deletion of this Job from the Git repo some resources of Trident get deleted, some stay in the cluster (like podsecuritypolicy, ...). On the next installation of Trident tridentctl complains about resources already in the cluster and fails ...
I talked with fchuck in the slack channel about that and he told me that a Kubernetes operator for Trident will be available in a few weeks.
Every instance of Trident and each storage backend should be created / deleted by using custom resources, completely without an external command line tool. In this case we could put this CRs in the Git repo and manage Trident configuration / instances with them.
Also the installation/deinstallation of the Trident operator itself should be possible without a command line tool.
Is this possible ?
Thanks and greetings,
Josef

This issue is similar to https://github.com/NetApp/trident/issues/247.

Testing with latest v20.0.4,  the --generate-custom-yaml is still not generating the CSI secret.  Was it supposed to be fixed in this release ?

If using operator, most of the things can be done via ArgoCD, only thing that I am missing is creation of tridentBackend, as I have no idea how UUIDs are generated.

@jomeier Could you maybe share your current goals :) ?

Sure. An operator and its custom resources should 100% take care about that a software does what it is supposed to do.
I don't want to call any CLI tool to perform tasks. In the case of Trident we have to create the backend with the CLI tool of trident.
This should be not necessary.
I don't know other tools/operators that require CLI tools to perform its tasks.
Normally - as far as I know operators and I know lots of them - CLI tools only are optional but are not necessary to be called.

Hi @jomeier,
The feature request to add a Trident backend via the Trident Operator is scheduled to be included in the Trident 21.01 release. The tridentctl tool existed prior to Operators being available which is why the functionality to add, update, and remove backends exists in tridentctl. We do plan to make improvements to the Trident Operator going forward but still need to support customers that have created tooling around the tridentctl binary.

@gnarl:
Thanks for your answer. We are happily looking forward to version 21.01 then. It will save us lots of work and will make the installation of Trident much more simpler.
Seeing getting it working together with GitOps tools like ArgoCD is very cool !
Thank you.
If it is available and will work, i will close this ticket.

i stumbled upon this, while i was evaluating how i can fully gitops our cluster infrastructure components (one of those is netapp via trident). Hope 21.01 will come out soon

i stumbled upon this, while i was evaluating how i can fully gitops our cluster infrastructure components (one of those is netapp via trident). Hope 21.01 will come out soon

@jomeier,
We were not able to complete this feature for Trident 21.01. We are continuing work on enabling this feature in a future Trident release.

Hey @gnarl , do you know which future release will include this feature?

This issue is fixed with commit 3173b5f and will be included in the Trident 21.04 release.

Could you point to the doc where samples are provided etc? I can't seem to find them :(

Hi  @travisghansen,
This is for managing backends: https://netapp-trident.readthedocs.io/en/stable-v21.04/kubernetes/operations/tasks/managing-backends/index.html
For new samples (look for YAML-based tbc samples): https://github.com/NetApp/trident/tree/master/trident-installer/sample-input/backends-samples


Guess the VP leadboards

It would be fun to have some sort of leaderboard of each year's winner of the "Guess the VP" game we play on the server. It would require counting (or dumpping) the number of times FramedBot mentioned a player in #bot-games.

lmao good idea, i knew i should have coded this in the bot directly


Support TLS certificate validation for receivers

Enhancement Proposal
I am using the charm on edge rev 98 and am trying to use a slack receiver, but it fails with
:~$ kubectl logs alertmanager-0 -c alertmanager | less

2024-02-20T14:25:35.026Z [alertmanager] ts=2024-02-20T14:25:35.026Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml
2024-02-20T14:25:35.027Z [alertmanager] ts=2024-02-20T14:25:35.027Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml
2024-02-20T14:25:35.181Z [alertmanager] ts=2024-02-20T14:25:35.181Z caller=notify.go:745 level=warn component=dispatcher receiver=mattermost-notifications integration=slack[0] aggrGroup="{}/{juju_model=\"prod-github-runner\"}:{juju_application=\"grafana-agent-without-dashboard\", juju_model=\"prod-github-runner\", juju_model_uuid=\"4572b1cc-0a39-40b7-818d-c68ed553f11a\"}" msg="Notify attempt failed, will retry later" attempts=1 err="Post \"<redacted>\": x509: certificate signed by unknown authority"

The configuration I provided looks like this
global:
  http_config:
    tls_config:
      insecure_skip_verify: false
receivers:
- name: 'default'
- name: 'mattermost-notifications'
  slack_configs:
    - send_resolved: true
      api_url: 'https://fake-webhook-url'
route:
  group_by:
    - juju_application
    - juju_model_uuid
    - juju_model
  group_interval: 5m
  group_wait: 30s
  receiver: 'default'
  repeat_interval: 4h
  routes:
    - receiver: 'mattermost-notifications'
      matchers:
       - juju_model = prod-github-runner

The server in the api_url doesn't use a self-signed certificate, but one from Let's Encrypt. The reason certificate validation fails is probably because there are no ca certs in the container.
~$ kubectl exec -it alertmanager-0 -c alertmanager -- ls -l /etc/ssl/certs
total 0

I can fix this by setting insecure_skip_verify to true, but I think the charm should also support validation for common ca's or at least provide the ability to provide a ca file for https://prometheus.io/docs/alerting/latest/configuration/#tls_config.

@Abuelodelanada /etc/ssl/certs/ is empty, this may be related to https://github.com/canonical/rockcraft/issues/343#issuecomment-1946506475

Hi @cbartz !
Thanks for the report! We'll address this.

This seems to have been fixed in the latest alertmanager (rev 107):
kubectl exec -it alertmanager-0  -c alertmanager -- ls -l /etc/ssl/certs                                                                                1 ↵
Defaulted container "charm" out of: charm, alertmanager, charm-init (init)
total 584

I'll try it out and will close this issue once confirmed.

This has been fixed (tested with rev 107)


Please port to 3.68

Your plugin is a godsend. Please port to 3.68. =D

894bec38fbcd539814c07af384df757015ec08f2


UnrealEnginePython_20170203_4_14_python36_win64.zip fails to load.

Windows 10
UE-4.14.3
UnrealEnginePython_20170203_4_14_python36_win64
Starting from a new, empty project in UE 4.14.3, I launched and enabled the plugin.
An error pop-up is triggered during application start up:
Plugin `UnrealEnginePython` failed to load because module `UnrealEnginePython` could not be loaded. There may be an operating system error or the module may not be properly set up.

Hi, are you sure you have python3.6 in system PATH ? The non-embedded distribution expects to find python36.dll. Does it work with the embedded one ?

The short answer is: Yes, that was the issue.

@rdeioris I have the same problem. Win10, UE4.15, Python36
I have no problem launching python from the power shell but the plugin does not work.
I set the PATH variable to C:\Program Files\Python36
Steps:

download python 3.6 from https://www.python.org/downloads/
install for all users & set the path to C:\Program Files\Python36
install UE4 plugin
start project
fail: "plugin UnrealEnginePython failed to load because module UnrealEnginePython could not be loaded. There may be an operating system error or the module may not be properly set up."

I have the same problem on two different machines

@MatzeOGH  have you tried the embedded distribution ?
Can you try copying python36.dll into the plugin binary directory ?

It was my own fault.
I mistakenly installed the Win32 version


[Bug]: Can't import xcdat after updating from 0.3.0 to 0.5.0 (module 'xarray.core' has no attribute 'resample')

What happened?
I have updated xcdat using mamba the following way
(cdatm_py3) jypeter@obelix4 - ...jypeter - 45 >mamba update xcdat
[...]

Pinned packages:
  - python 3.8.*

Transaction

  Prefix: /home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3

  Updating specs:

   - xcdat
   - ca-certificates
   - certifi
   - openssl

  Package    Version  Build           Channel                   Size
──────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────

  + libxslt   1.1.33  hf705e74_1      conda-forge/linux-64     553kB
  + lxml       4.8.0  py38h0a891b7_3  conda-forge/linux-64       2MB

  Upgrade:
──────────────────────────────────────────────────────────────────────

  - openssl   1.1.1v  hd590300_0      installed
  + openssl   1.1.1w  hd590300_0      conda-forge/linux-64       2MB
  - xcdat      0.3.0  pyhd8ed1ab_0    installed
  + xcdat      0.5.0  pyhd8ed1ab_0    conda-forge/noarch        61kB

  Summary:

  Install: 2 packages
  Upgrade: 2 packages

  Total download: 4MB

──────────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] Y
lxml                                                 1.5MB @   4.9MB/s  0.3s
libxslt                                            553.1kB @   1.7MB/s  0.3s
openssl                                              2.0MB @   5.7MB/s  0.4s
xcdat                                               61.0kB @ 128.9kB/s  0.5s
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Afterwards, I can't import xcdat anymore
(cdatm_py3) jypeter@obelix4 - ...jypeter - 46 >python
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import xcdat
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/lib/python3.8/site-packages/xcdat/__init__.py", line 2, in <module>
    from xcdat.axis import (  # noqa: F401
  File "/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/lib/python3.8/site-packages/xcdat/axis.py", line 9, in <module>
    from cf_xarray.criteria import coordinate_criteria
  File "/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/lib/python3.8/site-packages/cf_xarray/__init__.py", line 1, in <module>
    from .accessor import CFAccessor  # noqa
  File "/home/share/unix_files/cdat/miniconda3_21-02/envs/cdatm_py3/lib/python3.8/site-packages/cf_xarray/accessor.py", line 42, in <module>
    xr.core.resample.Resample,
AttributeError: module 'xarray.core' has no attribute 'resample'

What did you expect to happen? Are there are possible answers you came across?
I expected xcdat to keep on working after the update :-)
Minimal Complete Verifiable Example (MVCE)
No response
Relevant log output
No response
Anything else we need to know?
What is the suggested way to update xcdat?
I have not found anything about that in the documentation
Environment

import xarray as xr
xr.show_versions()

INSTALLED VERSIONS
commit: None
python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)
[GCC 9.3.0]
python-bits: 64
OS: Linux
OS-release: 3.10.0-1160.83.1.el7.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.10.6
libnetcdf: 4.7.4
xarray: 2022.9.0
pandas: 1.2.3
numpy: 1.21.4
scipy: 1.7.0
netCDF4: 1.5.6
pydap: None
h5netcdf: 0.13.1
h5py: 2.10.0
Nio: None
zarr: 2.11.0
cftime: 1.6.2
nc_time_axis: 1.4.1
PseudoNetCDF: None
rasterio: None
cfgrib: <IP_ADDRESS>
iris: 3.1.0
bottleneck: 1.3.7
dask: 2021.02.0
distributed: 2021.02.0
matplotlib: 3.3.4
cartopy: 0.18.0
seaborn: 0.11.1
numbagg: None
fsspec: 0.8.7
cupy: None
pint: 0.20.1
sparse: 0.13.0
flox: 0.7.2
numpy_groupies: 0.9.22
setuptools: 49.6.0.post20210108
pip: 22.0.4
conda: None
pytest: 6.2.4
IPython: 7.21.0
sphinx: 3.5.2

I have just found #341 !
Maybe #341 should not have been closed before exploring A few things we might want to consider on our end @tomvothecoder mentioned at the end? Because we are just 1 year after the issue was reported and the issue is still occurring
xr.show_versions() does not report cf_xarray version. It appears I have an even older version (0.6.1) than the one mentioned in the original issue (0.7.0)
I have updated cf_xarray and I can import xcdat again! You may want to do something about this, even if it is likely that not many people around have an old version of xcdat and cf_xarray
(cdatm_py3) jypeter@obelix4 - ...jypeter - 48 >mamba update cf_xarray
[...]
  Package      Version  Build         Channel                Size
───────────────────────────────────────────────────────────────────
  Upgrade:
───────────────────────────────────────────────────────────────────

  - cf_xarray    0.6.1  pyh6c4a22f_0  installed
  + cf_xarray    0.8.4  pyhd8ed1ab_0  conda-forge/noarch     53kB

  Summary:

  Upgrade: 1 packages

  Total download: 53kB

───────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] Y
cf_xarray                                           53.3kB @ 325.9kB/s  0.2s
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(cdatm_py3) jypeter@obelix4 - ...jypeter - 49 >python
Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import xcdat
>>>

Thanks @jypeter! I accidentally closed the issue by linking it to a PR merge and forgot to go back and open an issue to constrain cf_xarray.
I'll update the v0.5.0 recipe to ensure this doesn't happen.

I accidentally closed the issue by linking it to a PR merge and forgot to go back and open an issue to constrain cf_xarray.

Actually the user who opened it closed the PR and I forgot to go back to it :(

I opened up an issue in the cf-xarray repo so they can possibly address it on their end.
In an hour or so, there will be a new version of v0.5.0 on conda-forge that includes this patch from this PR.

python: 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)

I also suggest you update your environments to at least python >=3.10. Python 3.8 is approaching end-of-life (https://devguide.python.org/versions/).


Incorrect workflow when finally clauses

Given the follwing pipeline:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: workflow
spec:
  finally:
  - name: cleanup
    taskRef:
      kind: Task
      name: echo
  - name: cleanup1
    taskRef:
      kind: Task
      name: echo
    runAfter:
      - cleanup
  tasks:
  - conditions:
    - conditionRef: always-true
    name: always-echo
    taskRef:
      kind: Task
      name: echo
  - conditions:
    - conditionRef: always-false
    name: never-echo
    taskRef:
      kind: Task
      name: echo
  - name: echo
    taskRef:
      kind: Task
      name: echo

The representation is the following:

There are several issues:

the finally tasks should be linked to always-echo and never-echo tasks as well
task cleanup1 should be executed after cleanup
there is a warning on the runAfter field to cleanup: Value is not accepted. Valid values: "always-echo", "never-echo", "echo".

new screenshot for finally diagram.

@jeffmaury we cannot configure the final task execution order as per doc: https://github.com/tektoncd/pipeline/blob/master/docs/pipelines.md#cannot-configure-the-final-task-execution-order

closing this issue. feel free to open if you find any issue.


New resource: Org Setting

upstream: https://github.com/Juniper/terraform-provider-mist/issues/43

Already implemented today https://www.pulumi.com/registry/packages/junipermist/api-docs/org/base/


Swapper flow improvements

This has been this way for a while, but permit2 makes it a bit worse of a UX in my opinion.
We ask the user to do 2 clicks on the bottom of the screen (preview trade)

and then confirm and trade

and then all of a sudden bury the user action mid page when we want them to sign.

With permit2, we prompt the user and if they are using an external wallet, they are taken away from the context of the page and the changes to UI are very subtle

Now the user is wondering what action to take since the UI change is minor and they never had to wait for a TX to confirm.
I would suggest:

Move all user actions to the bottom, so a user gets used to just clicking through the flow and not having to figure out wat do that is buried in our UI

re: > was asked to execute a legacy approval transaction  /  If a legacy approval is in place, we should not need a permit 2 signature also
Note, this isn't granting allowance to ZRX legacy router but rather this grants allowance to 0x000000000022d473030f116ddee9f6b43ac78ba3 i.e the Permit2 contract, which is required before signing the Permit2 message
re: finally was able to execute the transaction after a refresh
If I understand correctly, does it mean it's impossible to execute the trade without a refresh of the app?

@twblack88 @reallybeard - think we might want to fix this ASAP. It feels a bit broken currently with permit2

Onboard with moving all actions to the bottom button. @reallybeard and I started exploring something very similar this week so we can churn some mocks/flows pretty quickly.

@twblack88 @reallybeard do we have finalized mocks for this yet?

https://www.figma.com/design/VSXdqFlFkBhdcPs2WqJqMo/SWAP-UX?node-id=2644-8097&node-type=section&t=SPp8hvzHw1871zKG-11
looks like we didn't link from today.

This issue should prob be closed in favor of #8118

Yeah isn't https://github.com/shapeshift/web/issues/8118 what we groomed today and called a 5?

Closing in favour of https://github.com/shapeshift/web/issues/8100


Switch to using kubernetes_asyncio client

Currently we use the official kubernetes python API Client. This is a purely synchronous client, and we must use threadpools / threads to use it from our tornado app.
We should experiment with using https://github.com/tomplus/kubernetes_asyncio instead, which is the asynchronous version of the library. This should (hopefully) simplify our code and let us stop thinking about threads.

We should consider how well supported the async k8s library will be (in a year or two from now). Right now it seems like a hobby project with one main contributor.
In the past we have had some problems with leaks/things breaking because the official library had bugs/made changes which causes trouble "in real time", compared to getting thread pools right for which you have infinite amounts of time. I prefer that we struggle during coding to in real time. Especially as the thread pool pattern is well established and setup in our code base already.
Is there a particular bug that would be fixed by switching or feature it would enable?
Right now my assessment is:

would be cool from a coder's perspective
introduces a risky new dependency (low maintainer count)
reduces compatibility with other cloud k8s deployments(??)

Should we put this on our "to watch" list and assess how the project has progressed in attracting more maintainers in a few weeks or months?

I stepped back from this a little bit and realized what really needs to happen is a port of the 'SharedInformer' concept from the k8s go client to Python. It is an encapsulation of a list + watch that handles coalescing, retries and resyncs for you. The existence of this library is why most k8s related stuff is in Go, and why they function pretty well. We've fixed a number of issues in kubespawner where we (mostly @minrk) learnt why we need an encapsulation like this.
More information on the SharedInformer is at:

https://engineering.bitnami.com/articles/a-deep-dive-into-kubernetes-controllers.html
http://borismattijssen.github.io/articles/kubernetes-informers-controllers-reflectors-stores
This 11 part series of blog posts: https://lairdnelson.wordpress.com/2018/01/07/understanding-kubernetes-tools-cache-package-part-0/

I don't believe it is possible for a human being to write a correct implementation of this while mixing threads & async.
re: compatibility, I don't think that's a problem. When running inside a k8s cluster, clients always use in-cluster. Minikube uses certs here, which are also supported. The only thing not natively supported is directly talking from your local computer to an AKS kubernetes cluster, which isn't something BinderHub can do anyway now. In the future, everything is moving to using the 'exec' provider (where an external command is executed for auth tokens). This library seems to support that too.
Maintainership is the only long term issue, but I'm personally committed to contributing to this too if necessary. The following things make me not too concerned about it:

It is generated from the same official repository as the official client: https://github.com/kubernetes-client/gen/pull/60
It's using asyncio support in swagger-gen to do most of its work, and this is maintained upstream in swagger-gen
The async client for kubernetes will always be a separate package, since the official one wants to always be synchronous (and be compatible with 2.7 for the foreseeable future) https://github.com/kubernetes-client/python/issues/323

Kubespawner is a bit more complex than binderhub, so I wanted to try this out on binderhub to gain experience running it to see how it goes. BinderHub's use of the k8s API is fairly minimal, and I am 100% sure there are bugs here we aren't noticing.
Since kubernetes_asyncio wants to be backwards compatible with the synchronous library, the changes are somewhat minimal. I've started work on this with #745. I need to figure out all the threads in use and make sure I've converted them right, and understand enough of the testing infrastructure to fix that too.
In parallel, I'm also working on a separate library that should implement sharedinformers - https://github.com/yuvipanda/kubernetes-informers. This should have thorough tests for various issues that might arise. I'm happy to wait for that library to mature before using that in BinderHub if that sounds better.
/cc @minrk @tomplus

@yuvipanda Thanks for nice introduction :+1:
Yes, my library is very similar to the official library which you're using. It's also generated from a specification but it uses the asyncio variant of python module from swagger-codegen. I always contribute to both repository to make these libraries almost identical. Now it exists in a separate repository (as owners of the official library suggested) but I hope it'll be incorporated to kubernetes-client in the future.  I think your use case can help with it by showing that such library is also necessary for community.
Please let me know if you need any help with integration.


Add Ménesguen 2022 line shapes for Mg, Al, and Si

Original report by Joseph Fowler (Bitbucket: joe_fowler, ).

See doi:10.1016/j.radphyschem.2022.110048 or email from Ullom March 1, 2022.

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Add Mg, Al, and Si K-alpha from Ménesguen 2022
Fixes #215

Original comment by Joseph Fowler (Bitbucket: joe_fowler, ).

Removing milestone: v0.7 (automated comment)


Can't build in IOS xcode

Is working in android.
but is not working in IOS xcode
this is my error

please let me know if you guys have any lucks.
thank you.

Do you still have the error ?
I juste remove 'character' to solve this

oh thank you for the feedback! I will try remove 'character'

Same here. Getting errors in xCode

I am also getting the same errors in xcode

bump, still having this erros ,any solution? using swift 5

Line 669 of PhotoLibraryService.swift:
Change from url.characters.count to url.count
Find and replace all instances of UIImageOrientation with UIImage.Orientation
That should do it


Provide a way to unsubscribe from LogHandler/ErrorHandler

Description
There is currently no way to remove the referenced Actions from Producer/Consumer which might cause a memory leak.
var producer = new ProducerBuilder<string, string>(producerConfig)
		.SetLogHandler(LogHandler)
		.Build();

How to reproduce
Confluent.Kafka 2.3.0
librdkafka 2.3.0
Checklist
Please provide the following information:

[ ] A complete (i.e. we can run it), minimal program demonstrating the problem. No need to supply a project file.
[x] Confluent.Kafka nuget version.
[x] Apache Kafka version.
[ ] Client configuration.
[ ] Operating system.
[ ] Provide logs (with "debug" : "..." as necessary in configuration).
[ ] Provide broker log excerpts.
[ ] Critical issue.

What is the scenario for memory leaking? When you're done with the producer, the handlers would go with it.

Can you provide more details?

I am not sure what more details you want me to provide. I need a way to unsubscribe/unset reference to error/log handlers actions as described in the issue's title and description.
Namely
ProducerBuilder<,>.ErrorHandler
ProducerBuilder<,>.LogHandler
Producer<,>.errorHandler
Producer<,>.logHandler


test: stablize test_applied_lock_collector

Signed-off-by: youjiali1995<EMAIL_ADDRESS>
What problem does this PR solve?
Issue Number: close https://github.com/tikv/tikv/issues/9451
Problem Summary:
Fix an unstable test:
[2021-01-14T07:58:07.094Z] test server::gc_worker::test_applied_lock_collector ... thread 'main' panicked at 'header { error { message: "stale command" } current_term: 6 }', tests/integrations/server/gc_worker.rs:103:13

What is changed and how it works?
What's Changed:
Add retry mechanism.
Related changes

Need to cherry-pick to the release branch

Check List
Tests

Unit test

Release note

No release note

/merge

/merge


mavlink: 2019.12.12-1 in 'kinetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository mavlink to 2019.12.12-1:

upstream repository: https://github.com/mavlink/mavlink.git
release repository: https://github.com/mavlink/mavlink-gbp-release.git
distro file: kinetic/distribution.yaml
bloom version: 0.9.0
previous version for package: 2019.11.11-1

Holding for Kinetic sync.


Readiness probe is not considering the monitoring resources status

Currently the readiness probe only checks for the storagecluster status. For overall readiness of the deployer, its necessary to consider monitoring resource(prometheus and alertmanager) status as well.

Fixed


Upgrade to Kube 5.0 and Rails 4

Update .less files to Kube 5.0
Update dependencies for Rails 4

@alishan-l wow, great :bow:
Just a couple of things, before I can merge:

remove changes to .gitignore.
squash Update readme commits into one.

I'm also not sure about version change. Could you explain it? :smile:

Fixed as per your suggestions.
Thanks

@alishan-l amazing, thanks :+1:


rename name and description

Request
Consider renaming name to use and description to short.
Initially this felt more intuitive but since name now supports the full usage it might be a good idea to align the terms to cobra command.
Both could be supported for a while for transition.
Proposed solution
No response
Anything else?
No response

Going back on forth on this, so leaving it as it is for now.


[SPARK-22585][Core] Path in addJar is not url encoded

What changes were proposed in this pull request?
This updates a behavior of addJar method of sparkContext class. If path without any scheme is passed as input it is used literally without url encoding/decoding it.
How was this patch tested?
A unit test is added for this.

Test build #84262 has started for PR 19834 at commit 1fc5db3.

@srowen Let's continue our discussion here.

So I have removed those three commented lines. Is there anything else to do before merge?

Test build #84266 has started for PR 19834 at commit bd667d9.

Test build #84262 has finished for PR 19834 at commit 1fc5db3.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84262/
Test PASSed.

Test build #84266 has finished for PR 19834 at commit bd667d9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84266/
Test FAILed.

LGTM.

Another LGTM

retest this please

Test build #84295 has started for PR 19834 at commit bd667d9.

Test build #84295 has finished for PR 19834 at commit bd667d9.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84295/
Test FAILed.

retest this please

Test build #84302 has started for PR 19834 at commit bd667d9.

Test build #84302 has finished for PR 19834 at commit bd667d9.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84302/
Test PASSed.

Merged to master.

Thanks!


key/value pairs in JSON

i'm wondering if it's possible to return a key/value pair in the JSON, so that when you're typing something in typeahead, it shows you the value, but when you select the item you want it actually puts the key as the input value?
so i.e.
"59936":"Silver Mt. Zion"
if you type "Silver" and select Silver Mt. Zion, it puts 59936 as the value of the input?
I'm not seeing of a way off the bat to do this, but i'm assuming it is possible.

Valid question in 2021, too bad there is no reply


fix: replace bundledDeps with externalDeps

Re-fixes #42 after my confusion around option naming 🤦
New description:
/** Additional dependencies to externalize if not detected by `vite-plugin-externalize-deps` */
externalDeps?: Array<string | RegExp>

Note bundledDeps did not work (included node_modules in output), so it has been removed

@lachlancollins thanks - works well


Panic on startup

The alb-ingress-controller pod is hitting a panic on startup:
2017-07-14T17:47:38.914734272Z I0714 17:47:38.914658       1 util.go:98] Request: elasticloadbalancing/&{DescribeLoadBalancers POST / %!s(*request.Paginator=&{[Marker] [NextMarker]  }) %!s(func(*request.Request) error=<nil>)}, Payload: {
2017-07-14T17:47:38.914753288Z   PageSize: 100
2017-07-14T17:47:38.914757789Z }
2017-07-14T17:47:39.011688485Z I0714 17:47:39.011607       1 util.go:98] Request: elasticloadbalancing/&{DescribeTags POST / %!s(*request.Paginator=<nil>) %!s(func(*request.Request) error=<nil>)}, Payload: {
2017-07-14T17:47:39.011710728Z   ResourceArns: ["arn:aws:elasticloadbalancing:us-west-1:xxxxxxxxxxx:loadbalancer/app/uswest1dev-xxxxxxxxxx/xxxxxxxxxx"]
2017-07-14T17:47:39.011716872Z }
2017-07-14T17:47:39.019571477Z panic: runtime error: index out of range
2017-07-14T17:47:39.019629318Z
2017-07-14T17:47:39.019636238Z goroutine 1 [running]:
2017-07-14T17:47:39.019640073Z github.com/coreos/alb-ingress-controller/awsutil.(*ELBV2).DescribeTags(0xc42033f790, 0xc4201de268, 0x19491ed, 0xa, 0xc42061db30, 0x1, 0x1)
2017-07-14T17:47:39.019643963Z 	/home/josh/dev/go/src/github.com/coreos/alb-ingress-controller/awsutil/elbv2.go:293 +0x32e
2017-07-14T17:47:39.019648011Z github.com/coreos/alb-ingress-controller/controller.assembleIngresses(0xc420151b00, 0xc42033f870, 0xc0, 0x190dde0)
2017-07-14T17:47:39.019651406Z 	/home/josh/dev/go/src/github.com/coreos/alb-ingress-controller/controller/ingress.go:205 +0x256
2017-07-14T17:47:39.019655331Z github.com/coreos/alb-ingress-controller/controller.NewALBController(0xc42013c780, 0xc42032d080, 0x1941f2a)
2017-07-14T17:47:39.019658644Z 	/home/josh/dev/go/src/github.com/coreos/alb-ingress-controller/controller/controller.go:41 +0x1a8
2017-07-14T17:47:39.019661726Z main.main()
2017-07-14T17:47:39.019664489Z 	/home/josh/dev/go/src/github.com/coreos/alb-ingress-controller/main.go:43 +0x24e

I have a hunch it's related to IAM permissions as I had been slowly adding them as it hit denials, then after permitting "createloadbalancer", the next startup hit this.

This was related to permissions, once I added the list from this page, the problem went away: https://github.com/coreos/alb-ingress-controller/blob/master/examples/iam-policy.json

I see that this is closed, which is surprising considering that there are unhandled exceptions identified here.  Wouldn't we want to see those cleaned up before closing?  The error message gave 0 helpful output and required scouring the documentation to understand what was wrong.

A lot of work was done lately to do a better job for catching errors. I've been running the controller with read only permissions and been unable to cause crashes when failing to delete or create resources. thanks for the heads up!


Local Repository Try To Build TS Aode At Client Workspace

Hi, I am new at bazel. When I try to depend on a project from my other projects, it wants a typescript and other npm deps. My dependency is working without problem but if I wanna use it on any other bazel repository with local_repository. it is wanna install all npm deps to the main project. This is has no meaning. In the future, if I wanna use bazel code from github I hes to open it and find all deps.

Thanks, I did think my configs were wrong. I did not publish to npm because I learning bazel repository stuff.


When switching tabs, code inspector needs to know in order to connect to changed notebooks/consoles.

We can implement this with the new focus tracker, but currently it's an outstanding issue in the code inspector.
Feel free to re-assign this issue if somebody else feels like taking it. If not, I'll handle it when we have the mono-repo.
cc: @sccolbert @blink1073

I think this isn't blocked anymore, now that we are using the monorepo.


feat: use sha256 for Relay msgIdFn

Problem
There is an assumption that StrictNoSign messages always have msgId generated by sha256.
Solution
To reduce potential problems explicitly define msgIdFn and pass to GossipSub.
Notes

Resolves https://github.com/waku-org/js-waku/issues/1179

Test with go-waku fail, is it related?

Test with go-waku fail, is it related?

transient issue


add adios2 host dependency

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

First try adding this failed in staged-recipes, so isolating its addition here

First try worked, rebased on #5


Write documentation describing Toolbar expanded and compressed layout

Missing implementation document about the expanded and compressed layout that impact the Toolbar layout.

Duplicate of #1095.


A way to add error message to radio button validation or at least show default HTML5 message?

Currently there is no way to add error message to radio button for validation.  (similar to that for text input field) Even if there is no plan to add the error message in the future update, at least display the original HTML5 validation message? Now it is strange to just jump to that position with no message at all.

dropping left: -9999px and keeping opacity: 0 for radio styles would seem to fix this.  is there a good reason to throw the unstyled radio button off-screen like that?

@vml-rmott Those styles are already gone in the recent versions

In that case, "at least display the original HTML5 validation message" can be considered fixed.

I doubt if it can be counted as fixed in a CSS framework if it requires CSS override to just display an basic feature that plain HTML already provide.

in the latest version 0.100.1, the original HTML5 validation message is now shown, no override needed.


[Enhancement] Support olap table prune complex type column on FE

What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes:

Fixes #
Problem Summary(Required):

push down complex column to scan node, and compute access path
rewrite all complex column in whole plan, unpack the column to a scalar column on scan node (later, next PR)

use tree struct ColumnAccessPath to descibe the access path, we can support complex column auto derive on this struct
CREATE TABLE `sc0` (
  `v1` bigint NULL,
  `st1` struct<s1 INT, s2 INT> NULL,
  `st2` struct<s1 INT, s2 INT, sm3 MAP<INT, INT>> NULL,
  `st3` struct<s1 INT, s2 INT, sa3 ARRAY<INT>> NULL,
  `st4` struct<s1 INT, s2 INT, ss3 struct<s31 INT, s32 INT>> NULL,
  `st5` struct<s1 INT, s2 INT, ss3 struct<s31 INT, s32 INT>,  ss4 struct<s41 INT, s52 struct<s421 INT, s423 INT>>> NULL
) ENGINE=OLAP
DUPLICATE KEY(`v1`)
DISTRIBUTED BY HASH(`v1`) BUCKETS 3
PROPERTIES (
"replication_num" = "1",
"in_memory" = "false",
"storage_format" = "DEFAULT"
);

eg. select st2.s1, map_keys(st2.sm3), st3.sa3 from sc0
PLAN FRAGMENT 0(F00)
  Output Exprs:7: st2.s1 | 8: map_keys | 9: st3.sa3
  Input Partition: RANDOM
  RESULT SINK

  1:Project
  |  output columns:
  |  7 <-> 3: st2.s1
  |  8 <-> map_keys[(3: st2.sm3); args: INVALID_TYPE; result: ARRAY<INT>; args nullable: true; result nullable: true]
  |  9 <-> 4: st3.sa3
  |  cardinality: 1
  |
  0:OlapScanNode
     table: sc0, rollup: sc0
     preAggregation: on
     partitionsRatio=0/1, tabletsRatio=0/0
     tabletList=
     actualRows=0, avgRowSize=5.0
     ColumnAccessPath: [/st2/sm3/KEY, /st2/s1, /st3/sa3]
     cardinality: 1

eg. select st5.ss4.s52.s421, st5.ss3.s32 from sc0
PLAN FRAGMENT 0(F00)
  Output Exprs:7: st5.ss4.s52.s421 | 8: st5.ss3.s32
  Input Partition: RANDOM
  RESULT SINK

  1:Project
  |  output columns:
  |  7 <-> 6: st5.ss4.s52.s421
  |  8 <-> 6: st5.ss3.s32
  |  cardinality: 1
  |
  0:OlapScanNode
     table: sc0, rollup: sc0
     preAggregation: on
     partitionsRatio=0/1, tabletsRatio=0/0
     tabletList=
     actualRows=0, avgRowSize=3.0
     ColumnAccessPath: [/st5/ss4/s52/s421, /st5/ss3/s32]
     cardinality: 1

Checklist:

[x] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[ ] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

[FE PR Coverage Check]
:heart_eyes: pass : 239 / 255 (93.73%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/OlapScanNode.java
2
5
40.00%
[794, 795, 827]

:large_blue_circle:
com/starrocks/sql/plan/PlanFragmentBuilder.java
3
6
50.00%
[347, 348, 1214]

:large_blue_circle:
com/starrocks/planner/ScanNode.java
6
9
66.67%
[113, 114, 117]

:large_blue_circle:
com/starrocks/catalog/ColumnAccessPath.java
22
28
78.57%
[101, 105, 106, 107, 108, 109]

:large_blue_circle:
com/starrocks/sql/optimizer/Optimizer.java
8
9
88.89%
[413]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MaterializedViewRewriter.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/RelationTransformer.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/MvRewritePreprocessor.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/base/ColumnRefSet.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/Operator.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MVProjectAggProjectScanRewrite.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/BestIndexRewriter.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalOlapScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/implementation/OlapScanImplementationRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/tree/PruneSubColumnsRule.java
136
136
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/transformation/PruneScanColumnRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/physical/PhysicalScanOperator.java
4
4
100.00%
[]

select before_a, before_map, mmap[a] as after_a, mmap as after_map from (select map_apply((k,v)->(k+1,abs(v)) as mmap, mapp[a] as before_a, mapp as before_map from t)A;

what can be pushed down?
what if when a map, map[key],map_keys(map) occur in project, scan return 3 colums?
what if sub columns after some update functions? like map_apply, map_concat, map_filter?

I only support map_keys now, but we can support other experssion, I don't support because I don't know what you need
will query hole map
no matter what you do(update/concat/filter), the base operate is read data, so specify the data you only need to read.

like map['abc'],  only query data which the key is 'abc' , we can driver the path /map/abc, the path type is '/ROOT/SPECIAL_KEY'?  then we can do something in storage with this message? query materialized column? or driver a predicate?

What type of PR is this:

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Optimizer.java#512 result = new PruneSubfieldsForComplexType().rewrite(result, rootTaskContext);

will replace PruneSubfieldsForComplexType later

Optimizer.java#512 result = new PruneSubfieldsForComplexType().rewrite(result, rootTaskContext); Is this pr will replcae this line's work?

it's work for external table, I think will replace the rule later, but need modify be

[FE PR Coverage Check]
:heart_eyes: pass : 238 / 255 (93.33%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/OlapScanNode.java
2
5
40.00%
[794, 795, 827]

:large_blue_circle:
com/starrocks/sql/plan/PlanFragmentBuilder.java
3
6
50.00%
[347, 348, 1214]

:large_blue_circle:
com/starrocks/planner/ScanNode.java
6
9
66.67%
[113, 114, 117]

:large_blue_circle:
com/starrocks/catalog/ColumnAccessPath.java
22
28
78.57%
[101, 105, 106, 107, 108, 109]

:large_blue_circle:
com/starrocks/sql/optimizer/Optimizer.java
8
9
88.89%
[413]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/tree/PruneSubColumnsRule.java
135
136
99.26%
[283]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MaterializedViewRewriter.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/RelationTransformer.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/MvRewritePreprocessor.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/base/ColumnRefSet.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/Operator.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MVProjectAggProjectScanRewrite.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/BestIndexRewriter.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalOlapScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/implementation/OlapScanImplementationRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/transformation/PruneScanColumnRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/physical/PhysicalScanOperator.java
4
4
100.00%
[]

@Mergify rebase

SQL like select x from b join c on b.b1=c.c1
Will subfields b1 and c1's access path extract? I didn't see relevant code.

In the phase, join on-clause was pushed down expression to child

[FE PR Coverage Check]
:heart_eyes: pass : 277 / 297 (93.27%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/planner/OlapScanNode.java
2
6
33.33%
[732, 793, 794, 826]

:large_blue_circle:
com/starrocks/sql/plan/PlanFragmentBuilder.java
3
6
50.00%
[347, 348, 1214]

:large_blue_circle:
com/starrocks/planner/ScanNode.java
6
9
66.67%
[113, 114, 117]

:large_blue_circle:
com/starrocks/catalog/ColumnAccessPath.java
22
28
78.57%
[101, 105, 106, 107, 108, 109]

:large_blue_circle:
com/starrocks/sql/optimizer/Optimizer.java
8
9
88.89%
[412]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/tree/PruneSubfieldRule.java
172
175
98.29%
[351, 352, 353]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MaterializedViewRewriter.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/transformer/RelationTransformer.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/MvRewritePreprocessor.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/base/ColumnRefSet.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/qe/SessionVariable.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/Operator.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/MVProjectAggProjectScanRewrite.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/mv/BestIndexRewriter.java
4
4
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/tree/PruneSubfieldsForComplexType.java
2
2
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/logical/LogicalOlapScanOperator.java
11
11
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/implementation/OlapScanImplementationRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/rule/transformation/PruneScanColumnRule.java
1
1
100.00%
[]

:large_blue_circle:
com/starrocks/sql/optimizer/operator/physical/PhysicalScanOperator.java
4
4
100.00%
[]


[BUG]  The cpu/memory/storage chart does not show data

Describe the bug

Login to harvester with member role, the cpu/mem/storage chart on the dashboard home page is empty, and there is no node information on the host list page.

To Reproduce
Steps to reproduce the behavior:

Create a member user
login with the created member user

Expected behavior

When the user does not have access to the node, the host page and related chart should be hidden
Support bundle

Environment:

Harvester ISO version:
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630):

Additional context
Add any other context about the problem here.

Tested in Harvester master-1a861fc1-head and Rancher v2.6-948355cc596b8466a77aec5276640425a5935852-head.
Verification Steps

Create new user in Rancher
Import Harvester
Add new user as member user in RBAC in Harvester through virtualization management console
Log into Rancher with new user
Check dashboard for Harvester through Rancher RBAC login

Notes
I used external preferred ui-index and changed it to the Harvester one for validating the change in the Harvester dashboard.


Parsing uber zap log file

Fixes #55
-- Added a program to save logs from uber zap's generated log file.
-- Documentation for zap parser.

Let's skip it!


ADDITION: Shaarli

What this PR does / why we need it:
Adds Shaarli
Which issue (if any) this PR fixes:
Fixes # n/a
Any other useful info:
Shaarli is a what if Wallabag and MiniFlux had a baby... it's useful for creating a feed of links to share with friends and family.

If this is pulled into an Ansible role I can review and merge. There doesn't seem to have been any demand for it though, so if you're not up for updating it please close the PR.

Thanks for the initial response to my PR from 842 days ago.
Looks like a prime example for a plugin.

You're welcome


[T4A5][W10-B3]Vincent Seng

Ready for review

Some comments added. Please close the PR after reading comments.


[vcpkg docs] new README/Quick Start guide

cc @ras0219-msft
This is a first draft.

Should I update the chinese readme also?

@JackBoosY that would be great! just push to the branch :) (I think you can do that...)

/azp run

Seems some linkage are broken, I will fix that later.

The link issue is not resolved, please wait.

Done.


Fix Topics, Relationships, and See Also sections of multi-language symbols that don't have them

Bug/issue #, if applicable: rdar://92127060
Summary
When a symbol that's available in Swift and Objective-C whose Objective-C variant doesn't have topics, relationships, or See Also sections, the Objective-C documentation page inherits the topics, relationships, and See Also information of the Swift symbol, which is incorrect. This PR resolves the issue by not emitting Topics, Relationships, and See Also sections for those cases.
The work is split into three commits:

Do not emit relationship sections for variants with no relationships: when an Objective-C symbol has no relationships, do not emit relationships, instead of defaulting to the Swift variant's relationships.

Do not emit topics section for variants that don't have topics: if the Objective-C version of a symbol doesn't have topics, do not emit a Topics section for it, instead of defaulting to the Swift symbol's topics. This logic was implemented for Topics sections in articles, but not in symbols.

Do not emit See Also sections for variants that have no See Also topics: when an Objective-C symbol has no See Also topics, or when it has empty See Also topic groups, dont' emit a See Also section for it, rather than defaulting to the Swift's variant's See Also topics.

Dependencies
None.
Testing
Build documentation for symbols that have Topics, Relationships, and See Also information for only the Swift variant. Verify that the Objective-C documentation page doesn't contain information that's specific to the Swift symbol.
Checklist

[x] Added tests
[x] Ran the ./bin/test script and it succeeded
[x] Updated documentation if necessary

@swift-ci please test


Volunteer Page

I'm submitting a:

feature request

Expected behavior:
As a user, I should be able to volunteer for the warehouse.
Current behavior:
None
Possible Solution
Add a volunteer page to the adoption site. Volunteer page will include:

[ ] Volunteer requirements section

[ ] Volunteer FAQ section

[ ] A CTA to Volunteer, that will open an email to<EMAIL_ADDRESS>

@Jillzey please provide copy for Volunteer FAQ & Requirements sections here when available