Implement N4409 on top of HPX

Implement N3989 (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3989.html) on top of HPX. This finally would be the first step to expose parallel algorithms to application developers.
There is an updated version of the proposal document here (N4409): http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4409.pdf.
As always, this implementation will add HPX specific functionality. Several extensions are obvious right away:

Add a new execution policy task_execution_policy on top of what the proposal mandates. The difference to the already described parallel_execution_policy would be that all algorithms would return a future<> representing the original result.
Add possibility to specify HPX executors to be used with the par and task execution policies. This could be done by adding par(exec) and task(exec) as valid execution policy arguments.
Add functionality enabling the use of all algorithms for distributed use cases (see also #1338).

Another possible extension needs more investigation:

Allow for all algorithms to be invoked with sequences of futures themselves, where the predicates/operators are invoked only after the corresponding future became ready.

Here is the list of algorithms mandated by the proposal:

[x] adjacent_difference inner_product (see #1659)
[x] adjacent_find (see #1223)
[x] all_of any_of none_of (see #1181)
[x] copy copy_if copy_n (see #1147)
[x] move
[x] count count_if
[x] equal mismatch (see #1193, #1206)
[x] exclusive_scan inclusive_scan (see #1341, #1342)
[x] reduce transform (see #1183)
[x] fill fill_n
[x] find find_end find_first_of find_if find_if_not (see #1206, #1223)
[x] for_each for_each_n
[x] generate generate_n (see #1206)
[x] is_heap is_heap_until (see #2654)
[x] is_partitioned is_sorted is_sorted_until (see #1382)
[x] lexicographical_compare (see #1354)
[x] max_element min_element minmax_element (see #1231)
[x] make_heap (#4964)
[x] partial_sort (#5041)
[x] partial_sort_copy nth_element (#5592, #5630)
[x] sort (see #1850)
[x] stable_sort (see #4803)
[x] partition (see #2778)
[x] partition_copy (see #2716)
[x] stable_partition  (see #2345)
[x] remove remove_if (see #3086)
[x] remove_copy remove_copy_if (see #1385)
[x] replace replace_copy replace_copy_if replace_if (see #1225)
[x] reverse reverse_copy (see #1208)
[x] rotate rotate_copy (see #1212)
[x] search search_n (see #1317)
[x] set_difference set_intersection set_symmetric_difference set_union includes (see #1410)
[x] inplace_merge (see #2978)
[x] merge (see #2833)
[x] swap_ranges
[x] uninitialized_copy uninitialized_copy_n (see #1284)
[x] uninitialized_fill uninitialized_fill_n (see #1284)
[x] uninitialized_default_construct uninitialized_default_construct_n (see #2669)
[x] uninitialized_value_construct uninitialized_value_construct_n (see #2671)
[x] uninitialized_move uninitialized_move_n (see #2664)
[x] destroy destroy_n (see #2676)
[x] unique (see #2867)
[x] unique_copy (see #2754)

These were added by N4310:

[x] transform_reduce
[x] transform_exclusive_scan transform_inclusive_scan (see #1341, #1342)

These were added to C++20:

[x] shift_left shift_right (#5466)
[x] Add starts_with and ends_with algorithms (#5381)

@hkaiser  Could you please add a project description here https://github.com/STEllAR-GROUP/hpx/wiki/GSoC-2017-Project-Ideas

I'm preparing GSoC. I have a question.
When implementing parallel algorithms, can I allocate additional memory for optimization or parallelization of algorithms?
For some algorithms, there may be differences in implementation depending on whether additional memory allocation is allowed or not.

When implementing parallel algorithms, can I allocate additional memory for optimization or parallelization of algorithms?

That's a very good and controversial question. The maximum memory requirements for the parallel algorithms are not specified, only the computational complexity. While allocating memory itself does not change the computational complexity of an algorithm, often the fact that you allocate some intermediate buffer requires more data copying which in turn may change the complexity.
So the first rule of the game is not to exceed the complexity requirements as specified (e.g. don't make an algorithm (O(N) if it's supposed to be O(logN), etc.). If an additional allocation does not (indirectly) change the complexity, then please make sure (that even for large data arrays) this does not blow the memory requirements out of proportion.
Generally, I'd suggest to try to avoid memory allocations as much as possible (in the first step) and use the implementation allowing to do things without. I understand that additional allocations may improve the algorithm performance, or even it's complexity, but I'd like to make an implementation correct first before diving into possible optimizations.
I know I have not given a concrete answer to your question. I guess it's a case by case decision we'll have to make as we go.

@taeguk Most of the missing algorithms are usually implemented based on a variation of a parallel scan. We already have a handful algorithms based on a scan_partitioner (e.g. copy_if, remove_copy, inclusive_scan, etc.). I'd expect for the rest of those to be easily implementable using this very same (and already existing) scan_partitioner. I'd suggest for you to familiarize yourself with how we have implemented those existing algorithms as reusing the partitioner would significantly simplify implementing the missing algorithms.

@hkaiser Thanks for your answer and advice.
After reading your answer, I analyzed the code for copy_if, inclusive_scan, and scan_partitioner.
scan_partitioner seems to be helpful for the rest of algorithms.
For example, partition_copy can be easily implemented using scan_partitioner with referring to the implementation of copy_if. And unique_copy also seems to be able to be implemented using scan_partitioner through a little application in using it.
But still, further thought and discussion seem necessary for implementations of some algorithms.
For example, partition is not easy to implement because it is an in-place algorithm unlike copy_if and scan_partitioner.
And, merge also seems to be not easy even though it is an out-of-place algorithm.

@taeguk Sure. Please ask more questions if needed. You may want to focus on the algorithms which are easier to implement, first.

@hkaiser And I have a one more question.
Currently,  scan_partitioner uses a sequential approach as a way to propagate partition results in step 2.
(https://github.com/STEllAR-GROUP/hpx/blob/66ef6bd5d626fbaf16414b5687169c3197e42e9c/hpx/parallel/util/scan_partitioner.hpp#L269)
I think it will be more better by using an approach like:
https://upload.wikimedia.org/wikipedia/commons/thumb/8/81/Prefix_sum_16.svg/300px-Prefix_sum_16.svg.png
Of course, this is only meaningful if there are a large number of partitions and f2 is heavy function.
How do you think about it?

@taeguk I agree, this might be viable. Having an independent implementation for a scan-partitioner for performance comparisons is a good thing. Also, @biddisco was looking into improving the scan-based algorithms at some point, he might be able to shed some additional light onto all of this.
As a unrelated note: would you mind continuing this discussion on our mailing list<EMAIL_ADDRESS>or on the IRC channel (#ste||ar at freenode)?

@hkaiser @taeguk @biddisco
I missed the IRC discussion on that topic but I want to mention that I am already working on a very similar problem. My Master thesis is built on top of parallel prefix sum, I started working on implementing existing solution in HPX and for that part, I have to change the existing implementation of scan_partitioner.
In general, taeguk made a very good observation about scan_partitioner. However, the existing approach is absolutely fine for most applications because they tend to perform operations where computation cost of the operand << communication time. For example, all papers on MPI_Scan are focused on optimizing the communication in operations such as integer addition. Unfortunately, it is not easy to find examples in literature which consider more complex operations in parallel prefix sum. After all, it might be a rare case. A parallel scan with a serialized middle phase is a very poor choice there due to its lack of scalability. The theoretical maximum of scalability is reached for a number of workers equal to square root of the number of data items.
My case is completely different and I have a very "heavy" operation with an unpredictable runtime (multigrid for nonlinear optimization by gradient descent) which operates on a small amount of data.
The most important problem for me is to decrease the span of the whole algorithm. It does not have to be the work-efficient but the total number of operations by the slowest worker has to be as small as possible. Unfortunately, for that case the Blleloch scan, which is presented on the graph linked by taeguk, is not really the best solution. The idea behind this scan is to perform two tree sweeps, one upward and one downward, which in total gives you a theoretical span of a twice the logarithm of the number of workers. We can do better.
I have achieved the best results with parallel prefix sums which are not work-efficient i.e. the total number of operations performed by all workers is much higher than in Blelloch scan. It payoffs to do more work in parallel. Examples here are Kogge-Stone and Sklansky parallel prefix adders and both of them achieve the total span of just a logarithm of the number of workers; if prefer the former solution. The downside here is that those scans are inclusive and the third phase requires results of an exclusive scan. It has to be fixed by another round of communication where each partition shifts its result to a neighbor on its right. It is not a problem as long as the operand is "heavy" enough because it is much better to perform more communication instead of doing more work.
Then there are smaller optimizations and modifications to further decrease the span. Kogge-Stone scan leaves some room for improvement because the distribution of work is not equal among workers.
If I succeed, then definitely something similar would be available before GSoC starts. It may be a starting point or a competitive idea, we may end up with different solutions. However, it is very hard to invent anything new or different for prefix sums, the field has been widely studied since 60's.

@taeguk Just my curiosity: have you noticed the performance problem in scan_partitioner because of prior experience or knowledge about problems where parallel prefix sum is performed with computationally expensive operands? If yes, then would you mind sharing some details? I'm interested in approaches developed for this problem and, as I said, it's not a very common thing.

@mcopik I didn't claim to use Blleloch scan. Just, I wanted to mention that there may be a problem in implementation of scan_partitioner. Kogge-Stone and Sklansky parallel prefix adders seem to be good.
Let me tell you a little bit about the last discussion in IRC. @biddisco said that he implemented scan_partitioner using sequential approach for propagation because of extra cost outweighed the gains from parallel approach.
But, I thought the sequential approach can occur a problem if there are a large number of partitions and f2 is heavy function. So, we determined that an user selects what approach to use through algorithmic policy, or somesuch.
In parallel approach, determining what method we use is a issue which needs additional discussions.

My case is completely different and I have a very "heavy" operation with an unpredictable runtime (multigrid for nonlinear optimization by gradient descent) which operates on a small amount of data.
The most important problem for me is to decrease the span of the whole algorithm. It does not have to be the work-efficient but the total number of operations by the slowest worker has to be as small as possible.

@mcopik In this case, there is basical problem in scan_partitioner aside from the problem I presented. Currently, scan_partitioner first performs a sequential scan on each partition and then propagate partition results to other partitions from left to right. (the problem I presented is related in the latter)
So, the sequential scan on each partition in first step occurs the problem in situation which you suggested.

Marked inplace_merge as done because #2978 was merged.

Hello @hkaiser !! As Many algorithms are already implemented . But I have made one list of the unimplemented algorithms .
copy_backward
equal_range
is_permutation
lower_bound
upper_bound
move_backward
prev_permutation
next_permutation
nth_element
partition_point
pop_heap
push_heap
sort_heap
stable_sort
partial_sort
and as we have one numeric adjacent_difference (numeric algorithm) ,
These three numeric algorithms can also be implemented
accumulate
inner_product
partial_sum
As I have started learning about HPX , I have checked that these algorithms haven't been implemented yet. Is the implementation of these algorithms are not important for parallelism and concurrency ?

@victor-ludorum the algorithms listed here above are the ones specified for C++17.
accumulate, inner_product, and partial_sum are listed under a different name (reduce, transform_reduce, and inclusive_scan).
Somebody already tried to implement the heap algorithms, but that was abandoned (see #1914), feel free to revive that effort.
The algorithms related to sort are listed as not-implemented in our list above (nth_element, partial_sort, etc.)
I'm not sure if it's possible to parallelize the permutation algorithms.
copy_backwards and move_backwards can easily be implemented on top of the existing copy and move algorithms (it requires at least or bi-directional iterators, so you could wrap the given iterators into reverse_iterator), alternatively we'd need a separate (but similar to copy/move) implementation.
I don't know what is the difference between partition and partition_point.

Thanks @hkaiser sir !! I will definitely work on these algorithms. So numeric algorithms are already implemented . Remaining algorithms which is important can be implemented , I hope.

@fjtapia Just out of curiosity, we still have a couple of algorithms missing that are related to sorting (partial_sort, partial_sort_copy, and nth_element). Do you happen to have implementation available for those? Even some initial code would be very helpful. Our plan is to finally have all of the algorithms as specified by C++20 in place and these are the last missing pieces. Any help you could give would be most appreciated!

Hi Hartmut
Glad to contact you again.
I will prepare the implementation of the functions and send it to you to
examine. But it will be at the end of August because in two days I am going
on vacation.
Please send me a list of the functions to implement. If they are
single-thread or parallel, and if they have any conditions that must be
taken into consideration.
Regards
Francisco
El jue., 30 jul. 2020 a las 17:45, Hartmut Kaiser<EMAIL_ADDRESS>escribió:

@fjtapia https://github.com/fjtapia Just out of curiosity, we still
have a couple of algorithms missing that are related to sorting (
partial_sort, partial_sort_copy, and nth_element). Do you happen to have
implementation available for those? Even some initial code would be very
helpful. Our plan is to finally have all of the algorithms as specified by
C++20 in place and these are the last missing pieces. Any help you could
give would be most appreciated!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/STEllAR-GROUP/hpx/issues/1141#issuecomment-666472356,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA5O5GCKZPO5VOA4QRB67UTR6GITHANCNFSM4AP742IQ
.

Hi Hartmut Glad to contact you again. I will prepare the implementation of the functions and send it to you to examine. But it will be at the end of August because in two days I am going on vacation. Please send me a list of the functions to implement. If they are single-thread or parallel, and if they have any conditions that must be taken into consideration. Regards Francisco

Francisco, thanks for getting back so quickly, and thanks for your interest in helping! As said in my initial message, we are missing the implementations for the parallel versions of the following algorithms: partial_sort, partial_sort_copy, and nth_element (I believe I would be able to derive the partial_sort_copy from a partial_sort implementation myself, if needed - if that simplifies things). Also, we can always fall back to the std library versions for the sequential algorithms, so there is no need for you to look into those.

This has finally been done! Thanks to everybody who contributed to this task!


Add typescript type declaration

@w4ll3
While working on the gitcoin example we added the typescript type declarations to avoid lint issues when importing.
Signed-off-by: Tiago Nascimento<EMAIL_ADDRESS>

The code looks good to me. But I don't know how to test it. Is there a way to use or test this, or only using the gitcoin example?

I made a simple example that can be used to test it out here. You just need to compile DIDKit for node and link it using npm.

@theosirian great, that works, thanks. I think this is just blocked on the CI failure now, which is because the ssi ref must be updated. I'll open a separate PR for that.

The CI error should be fixed since #157. @theosirian can you rebase this branch onto latest main and squash the commits?

Done. Waiting on CI now.

Passed; merging.


CI/CD enhancements

A few useful change to improve our release/distribution cycle:

[x] compile manylinux wheels (see #188)
[x] set up CI to directly upload wheels to PyPI
[x] include Makefile to automise release workflow (see this intro, also see e.g. https://github.com/alan-turing-institute/CleverCSV/blob/master/make_release.py)
[x] enforce code formatting in released code (see #208, #256)
[x] add coverage for unit tests (see #394)
[ ] add code quality/climate checks (see https://lgtm.com, https://www.codacy.com, https://github.com/marketplace/code-climate, etc)
[ ] consider cibuildwheel package to simplify wheel building across platforms
[ ] improve packaging for macOS wheels with delocate, also included in cibuildwheel, also see https://iscinumpy.gitlab.io/post/azure-devops-python-wheels/
[x] distribute sktime via conda-forge: https://conda-forge.org/#contribute (see #326)
[x] set up pre-commit config (see #394)
[x] add Black for code formatting (see #394)
[x] add check-manifest (see #394)
[x] add bugbear to flake8 (see #394)
[x] update setup.cfg, see e.g. https://github.com/scikit-hep/boost-histogram/blob/develop/setup.cfg (see #394)
[ ] check out other flake8 extensions
[ ] add checks for broken website links: https://fastpages.fast.ai/fastlinkcheck/
[ ] use GitHub Action to check if Binder works and pre-build Docker images for quicker launch: https://github.com/jupyterhub/repo2docker-action#use-github-actions-to-cache-the-build-for-binderhub
[ ] explore pre-commit-ci as suggested in #794

Extra resources

For building/distribution on GitHub Actions checkout: https://scikit-hep.org/developer/gha_wheels
Useful links from reproducible research lunch
scikit-learn's release process
Check out https://python-poetry.org thanks to @hiqbal2
Consider setuptools_scm for versioning (software configuration management)

Hi @mloning, I would like to work on using cibuildwheel to simplify the workflow on package building. You can check our package that uses cibuildwheel, it is quite convenient, and we can even build wheels for architectures like aarch64.
I have checked all supported python versions and platforms of sktime on PyPI, all of them can be built with cibuildwheel.

@xuyxu sounds good

hi @mloning Im doing a tidy up, 200+ issues triggers me. Can this be closed?


Objects created in a different window are not reactively wrapped by store

Describe the bug
If you share a store setter from an iframe to its parent and call it via the parent, objects assigned to the store via this method, will not be reactive.
Your Example Website or App
https://stackblitz.com/edit/github-gerxph-42ncis?file=src%2Froutes%2Findex.tsx
Steps to Reproduce the Bug or Issue

Share a store setter from an iframe with its parent window
Listen to all values inside of the store, e.g. with createEffect(() => console.log(JSON.stringify(state))
Via the parent window, assign a new object to the store
Via the parent window, update a value inside of this new object
Notice how this change will not trigger a console.log

Expected behavior
Objects assigned to a store via a different window should become reactive like any other object.
Screenshots or Videos
No response
Platform

OS: [Linux]
Browser: [Chrome, Firefox]

Additional context
The reason why objects assigned this way will not be reactive, is the prototype check in https://github.com/solidjs/solid/blob/4d824b08d8534d2a079f9ca4c1ea980684c05582/packages/solid/store/src/store.ts#L75. Different windows have different Object.prototype and therefore an object assigned via another window, will not be considered as wrappable by solid store.
Since the prototype check works as intended for normal non-iframe use cases, a sensible way would be to just allow a hook into the check, e.g. something like this:
const wrappablePrototypes = [Object.prototype];
export const registerWrappablePrototype = (p) => wrappablePrototypes.push(p);

export function isWrappable(obj: any) {
  let proto;
  return (
    obj != null &&
    typeof obj === "object" &&
    (obj[$PROXY] ||
      !(proto = Object.getPrototypeOf(obj)) ||
      wrappablePrototypes.indexOf(proto) >= 0 ||
      Array.isArray(obj))
  );
}

registerWrappablePrototype would allow advanced developers to add parent/iframe Object.prototype, vice-versa.

Edit: Alternative: how about we use duck typing to figure out if proto is wrappable, if it quacks like a Object.prototype it must be one. We would just have to make sure that the duck typing works across realms.

Yeah this is one of those annoying things. Forcing this into user space seems really awkward. We need to consider the general solution. This check is to prevent classes. I'm deep into rethinking stores right now. Maybe that constraint is not necessary. It is worth investigation.

I also looked a bit deeper regarding the runtime typing approach. Most isPlainobject solutions out there sadly just do it with Object.prototype - (wheres the iframe love 🤣), but Lodash has a different solution, that works across realms from what I found (https://github.com/lodash/lodash/blob/4.17.15/lodash.js#L12045): Their Ctor instanceof Ctor check filters out classes and funcToString.call(Ctor) == objectCtorString takes care of the rest. I wonder what this would mean for performance though 🤔..

@ryansolid Is there anything I can do to speed up the solution-finding for this issue 🙂? Not gonna lie, being able to synchronize stores between realms without reconcile would be an absolute game changer for Nitropage, it would improve performance a lot and get rid of many sync timing issues.


Add JetBrains logos for donating OSS license

JetBrains has kindly donated OSS licenses for WebStorm and PhpStorm. Let's add their logos and some press text or links to our web page.
https://www.jetbrains.com/company/press/

Fixed!
http://nestor-qa.org/documentation/for-developers/oss-licenses


How to enable Aiven password

This is needed if the user wants to authenticate and log in directly without using any 3rd parties by using Aiven username and password.
fixes:https://github.com/aiven/devportal/issues/677

Thanks, I'll clarify the use of this feature first.

Yes, I was thinking that was something important to talk about. And I'd already written "Discuss pros and cons of using password vs token." in the outline to remind me.

I've added a note on the cli tools article about it and focused on the case that the user wants to authenticate and log in directly without using any 3rd parties by using Aiven username and password. Thanks again!


Fix the failing KubeRay tests running with downstream RHODS operator

Investigate and fix the following test failure when creating a RayJob with the downstream operator - might be related to [1]:
=== NAME  TestRayJobSubmissionRest
    ray_test.go:185:
        Unexpected error:
            <*errors.errorString | 0xc0004bbe70>:
            incorrect response code: 503 for creating Ray Job, response body:
...

Application is not available

The application is currently not serving requests at this endpoint. It may not have been started or is still starting.

                      Possible reasons you are seeing this page:

                        The host doesn't exist.
                        Make sure the hostname was typed correctly and that a route matching this hostname exists.

                        The host exists, but doesn't have a matching path.
                        Check if the URL path was typed correctly and that the route was created using the desired path.

                        Route and path matches, but all pods are down.
                        Make sure that the resources exposed by this route (pods, services, deployment configs, etc) have at least one pod running.

...
            }
        occurred

[1] https://issues.redhat.com/browse/RHODS-11106

Moved to https://github.com/red-hat-data-services/distributed-workloads/issues/5


Dockerfile templates

Alright, I think this and these changes to the clojure build definition should work to create leiningen and boot images in a way that is compatible with the official image build system.
All of the non-onbuild Dockerfiles are now generated from the Dockerfile-boot.template and Dockerfile-lein.template files by the update.sh script. If you make a change to either template (such as updating to a new lein or boot version), just run ./update.sh in the root of the project. The update.sh script is modeled after other examples the docker-library maintainers pointed me to, and it seems to require bash 4.x (notably I had to brew install bash even on macOS 10.12.2 b/c it ships with bash 3.x).
When / if this is merged, I'll PR the clojure build definition changes to the docker-library/official-images repo. Note that it also converts that file to their new format, which is documented here.

@cap10morgan could you note in README that Bash 4 is needed to run update.sh. Otherwise, LTGM. Please merge when you're ok with it. Thanks!

I like the image tag semantic clojure:boot vs clojure:lein, etc. Good stuff!

official-images PR is up

And now it's merged. I'll make an announcement on the clojure mailing list.


Write script to auto-populate database on initial set up

Overview
We have training data from various sources currently sitting in various CSV files. We also have a (probably empty) db table called training_claims.
Requirements

[ ] Write a script (in /tools/db) that auto-populates the database with some sample data.
[ ] Call this from db/main.py so that when a new db is created, it is populated

The code reads the training data from CSV files quite happily.
While this might have been worth fixing months ago, it isn't any more as the alternatives are working fine.


Adding uofn.txt file (uofn.edu)

Official website: uofn.edu
URL of all Technology courses: https://uofn.edu/colleges/science-technology#courses
Complementary Info: Heroku feature a case study of a project implemented by UofN Staff and some students: https://www.heroku.com/customers/uofn

@dudunato Thank you for your patience while waiting for a review and processing your request. Pull request merged. Please start applying for JetBrains educational subscription at https://www.jetbrains.com/shop/eform/students in an hour, to allow the changes to propagate through JetBrains systems.


Message cache missing messages sent by self.

Fixed. Sorry I keep pushing to master!
The problem was caused by messages sent not having a timestamp "ts" until returned in an reply_to from slack. Without this ts, it is impossible to reprint them in order, and they are skipped. This was fixed by properly handling reply_tos


PEP8 files and allow the cursor for the next pledge page to be returned

Note: You may need to view the diffs with whitespace changes disabled (?w=0).
This pull request contains three changes.  Firstly it runs the project files through a PEP8 checker to ensure they're matching the standard code style for Python projects.  Secondly, it allows the fetch_page_of_pledges method to return the cursor for the next page of pledges, this makes it much easier to iterate over the pledges for a campaign. Thirdly, it fixes a missing import (urllib) which affects the currently released version.
Example usage:
campaign_id = 123456
per_page = 20

pledges, cursor = api_client.fetch_page_of_pledges(campaign_id, per_page, return_cursor=True)

while 'next' in pledges['links']:
    # Get the next page
    pledges, cursor = api_client.fetch_page_of_pledges(campaign_id, per_page, cursor, return_cursor=True)

I've added return_cursor as a new kwarg to fetch_page_of_pledges with the default value set to False, this ensures the code does not break existing installations.
I've only tested this PR against Python 3.5, it may be that the urllib import paths are different in Python 2.x. If that's the case, let me know and I'll update the PR with additional imports for Python 2.

Hey there! Love the ideas, just a few comments:

In the future, separate the PEP8 styling changes and import changes (both of which are more like bug fixes than new features) from the other change would have been nice. Hypothetically, I'd hate to have to hold up those to obvious improvements due to a weak interface / implementation of the cursor feature.
For the PEP8 styling -- can we make sure all lines are shorter than 80 characters? I'm seeing

./setup.py:5:80: E501 line too long (96 > 79 characters)
./patreon/__init__.py:2:25: W292 no newline at end of file
./patreon/api.py:18:80: E501 line too long (94 > 79 characters)
./patreon/api.py:21:80: E501 line too long (86 > 79 characters)
./patreon/api.py:23:80: E501 line too long (94 > 79 characters)
./patreon/api.py:24:80: E501 line too long (93 > 79 characters)
./patreon/api.py:46:80: E501 line too long (97 > 79 characters)
./patreon/oauth.py:29:80: E501 line too long (87 > 79 characters)

I've left a small comment on import consistency for the cursor return implementation.

Thanks so much for your contributions! Excited to get this cleaned up and merged in :D

closed in favor of https://github.com/Patreon/patreon-python/pull/6
example usage of the new interface:
api_client = patreon.API(patron_access_token)
patrons_page = api_client.fetch_page_of_pledges(campaign_id, 10)
next_cursor = api_client.extract_cursor(patrons_page)
second_patrons_page = api_client.fetch_page_of_pledges(campaign_id, 10, cursor=next_cursor)


Devices which are connected on Mesh Repeater get not counted

When you have the following Configuration on your WiFi:

Only the directly Connected WiFi Devices will be "exported".
In my Case only
9 for 5.0Ghz
and
2 for 2.4Ghz
But my WiFi Network is much larger because of the Mesh Repeater which are handling most of the Network.
Metrics which get Exportet:
# HELP fritzbox_exporter_collectErrors Number of collection errors.
# TYPE fritzbox_exporter_collectErrors counter
fritzbox_exporter_collectErrors 6341
# HELP fritzbox_exporter_luaCollectErrors Number of lua collection errors.
# TYPE fritzbox_exporter_luaCollectErrors counter
fritzbox_exporter_luaCollectErrors 9423
# HELP fritzbox_exporter_results_cached Number of results taken from cache.
# TYPE fritzbox_exporter_results_cached counter
fritzbox_exporter_results_cached{Cache="LUA"} 259818
fritzbox_exporter_results_cached{Cache="UPNP"} 525168
# HELP fritzbox_exporter_results_loaded Number of results loaded from fritzbox.
# TYPE fritzbox_exporter_results_loaded counter
fritzbox_exporter_results_loaded{Cache="LUA"} 13889
fritzbox_exporter_results_loaded{Cache="UPNP"} 2.172327e+06
# HELP gateway_connection_max_bitrate connection max bitrate
# TYPE gateway_connection_max_bitrate gauge
gateway_connection_max_bitrate{connection="PPP",direction="Down",gateway="fritz.box",interface="WAN"} 4.9643334e+07
gateway_connection_max_bitrate{connection="PPP",direction="Up",gateway="fritz.box",interface="WAN"} 4.0569962e+07
# HELP gateway_connection_status Connection status (Connected = 1)
# TYPE gateway_connection_status gauge
gateway_connection_status{connection="IP",gateway="fritz.box",interface="WAN"} 1
gateway_connection_status{connection="Physical Link",gateway="fritz.box",interface="DSL"} 1
gateway_connection_status{connection="Physical Link",gateway="fritz.box",interface="WAN"} 1
# HELP gateway_connection_uptime_seconds Connection uptime
# TYPE gateway_connection_uptime_seconds gauge
gateway_connection_uptime_seconds{connection="IP",gateway="fritz.box",interface="WAN"} 12603
gateway_connection_uptime_seconds{connection="PPP",gateway="fritz.box",interface="WAN"} 12603
# HELP gateway_data_ecostat_cputemp cpu temperature from data.lua?page=ecoStat
# TYPE gateway_data_ecostat_cputemp gauge
gateway_data_ecostat_cputemp{gateway="fritz.box"} 55
# HELP gateway_data_ecostat_cpuutil percentage of cpu utilization from data.lua?page=ecoStat
# TYPE gateway_data_ecostat_cpuutil gauge
gateway_data_ecostat_cpuutil{gateway="fritz.box"} 64
# HELP gateway_data_ecostat_ramusage percentage of RAM utilization from data.lua?page=energy
# TYPE gateway_data_ecostat_ramusage gauge
gateway_data_ecostat_ramusage{gateway="fritz.box",ram_type="Dynamic"} 49.1
gateway_data_ecostat_ramusage{gateway="fritz.box",ram_type="Fixed"} 32.4
gateway_data_ecostat_ramusage{gateway="fritz.box",ram_type="Free"} 18.5
# HELP gateway_data_energy_consumption percentage of energy consumed from data.lua?page=energy
# TYPE gateway_data_energy_consumption gauge
gateway_data_energy_consumption{gateway="fritz.box",name="CPU"} 81
gateway_data_energy_consumption{gateway="fritz.box",name="DSL"} 100
gateway_data_energy_consumption{gateway="fritz.box",name="Phone"} 0
gateway_data_energy_consumption{gateway="fritz.box",name="System"} 54
gateway_data_energy_consumption{gateway="fritz.box",name="USB"} 0
gateway_data_energy_consumption{gateway="fritz.box",name="WLAN"} 100
# HELP gateway_data_energy_lan_status status of LAN connection from data.lua?page=energy (1 = up)
# TYPE gateway_data_energy_lan_status gauge
gateway_data_energy_lan_status{gateway="fritz.box",name="LAN 1"} 1
gateway_data_energy_lan_status{gateway="fritz.box",name="LAN 2"} 1
gateway_data_energy_lan_status{gateway="fritz.box",name="LAN 3"} 0
gateway_data_energy_lan_status{gateway="fritz.box",name="LAN 4"} 0
# HELP gateway_data_usb_storage_total total storage in bytes from data.lua?page=usbOv
# TYPE gateway_data_usb_storage_total gauge
gateway_data_usb_storage_total{devicename="Internal eStorage",devicetype="INTERNAL_STORAGE",gateway="fritz.box"} 4.25721856e+08
# HELP gateway_data_usb_storage_used used storage in bytes from data.lua?page=usbOv
# TYPE gateway_data_usb_storage_used gauge
gateway_data_usb_storage_used{devicename="Internal eStorage",devicetype="INTERNAL_STORAGE",gateway="fritz.box"} 3.018752e+06
# HELP gateway_dect_active is dect device currently active
# TYPE gateway_dect_active gauge
gateway_dect_active{gateway="fritz.box",id="1",model="FRITZ!Fon C5",name=" "} 0
# HELP gateway_device_modelname gateway device model name
# TYPE gateway_device_modelname gauge
gateway_device_modelname{description="FRITZ!Box 7490 113.07.29",gateway="fritz.box",hardwareversion="FRITZ!Box 7490",modelname="FRITZ!Box 7490",productclass="FRITZ!Box",softwareversion="113.07.29"} 0
# HELP gateway_host_active is host currently active
# TYPE gateway_host_active gauge
gateway_host_active{gateway="fritz.box",hostname="amazon-dfb7475e1",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="90:23:5b:3d:c2:8b"} 1
gateway_host_active{gateway="fritz.box",hostname="android-8f3324f3210d9174",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="e4:04:39:d9:35:a8"} 0
gateway_host_active{gateway="fritz.box",hostname="annabells-air",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="9c:f3:87:ac:5e:30"} 0
gateway_host_active{gateway="fritz.box",hostname="arlo-babycam",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="08:02:8e:2c:25:e4"} 1
gateway_host_active{gateway="fritz.box",hostname="bellas-iphone",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="be:e3:02:e0:03:e4"} 0
gateway_host_active{gateway="fritz.box",hostname="bellas-iphone",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="dc:9b:9c:c1:3e:13"} 0
gateway_host_active{gateway="fritz.box",hostname="buro-eg",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="58:d3:49:00:cc:d7"} 1
gateway_host_active{gateway="fritz.box",hostname="camera-hub-g3-cfb0",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="54:ef:44:38:cf:b0"} 1
gateway_host_active{gateway="fritz.box",hostname="docker-pi-2",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="e4:5f:01:85:96:e5"} 1
gateway_host_active{gateway="fritz.box",hostname="epson53423a",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="e0:bb:9e:53:42:3a"} 0
gateway_host_active{gateway="fritz.box",hostname="esp-70c003",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="84:0d:8e:70:c0:03"} 1
gateway_host_active{gateway="fritz.box",hostname="esplesekopf-251aa4-6820",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="8c:ce:4e:25:1a:a4"} 1
gateway_host_active{gateway="fritz.box",hostname="espresense-buero-eg",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="08:b6:1f:38:1a:b8"} 0
gateway_host_active{gateway="fritz.box",hostname="fb-buero",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="c2:39:6f:9d:75:fe"} 1
gateway_host_active{gateway="fritz.box",hostname="fb-flur",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="dc:39:6f:82:ce:0d"} 1
gateway_host_active{gateway="fritz.box",hostname="fb-kinderzimmer",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="42:49:79:e9:71:f0"} 1
gateway_host_active{gateway="fritz.box",hostname="fb-kueche",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="12:72:74:a0:8d:9a"} 1
gateway_host_active{gateway="fritz.box",hostname="filebrowser",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="02:42:c0:a8:b2:04"} 0
gateway_host_active{gateway="fritz.box",hostname="fritz.box",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="38:10:d5:a4:1b:08"} 1
gateway_host_active{gateway="fritz.box",hostname="garagentor",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="c4:5b:be:5f:d8:b0"} 1
gateway_host_active{gateway="fritz.box",hostname="homeassistant",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="52:54:00:57:8d:47"} 1
gateway_host_active{gateway="fritz.box",hostname="hs100",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="d8:47:32:a3:e1:2e"} 1
gateway_host_active{gateway="fritz.box",hostname="ipad-von-klaus-2",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="ee:84:f1:2f:eb:98"} 0
gateway_host_active{gateway="fritz.box",hostname="ipad-von-stefan",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="ae:3e:63:d7:75:26"} 1
gateway_host_active{gateway="fritz.box",hostname="iphone-x",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="02:63:38:b9:7e:7b"} 1
gateway_host_active{gateway="fritz.box",hostname="iphone11promax",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="d0:3f:aa:b2:50:06"} 1
gateway_host_active{gateway="fritz.box",hostname="jarvis",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="00:11:32:61:c8:1f"} 1
gateway_host_active{gateway="fritz.box",hostname="kp115",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="b0:a7:b9:c5:28:b5"} 1
gateway_host_active{gateway="fritz.box",hostname="linux",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="f0:81:73:7a:59:2c"} 1
gateway_host_active{gateway="fritz.box",hostname="macbookonstefan",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="00:23:6c:8a:2a:96"} 0
gateway_host_active{gateway="fritz.box",hostname="nb-ts-r0-939",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="a4:34:d9:cd:39:b7"} 0
gateway_host_active{gateway="fritz.box",hostname="nb-ts-r0-939",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="84:7b:eb:0c:50:50"} 1
gateway_host_active{gateway="fritz.box",hostname="nginxproxymanager",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="02:42:c0:a8:b2:05"} 1
gateway_host_active{gateway="fritz.box",hostname="none-5",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="7c:d5:66:8c:56:2e"} 1
gateway_host_active{gateway="fritz.box",hostname="p110",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="14:eb:b6:c1:45:ba"} 1
gateway_host_active{gateway="fritz.box",hostname="p110",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="14:eb:b6:c1:45:51"} 1
gateway_host_active{gateway="fritz.box",hostname="p110",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="34:60:f9:56:97:11"} 1
gateway_host_active{gateway="fritz.box",hostname="pc-192-168-178-20",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="32:e1:39:07:90:25"} 1
gateway_host_active{gateway="fritz.box",hostname="pc-192-168-178-41",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="00:25:29:24:f5:1b"} 1
gateway_host_active{gateway="fritz.box",hostname="pc-192-168-178-45",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="86:e1:bd:29:04:19"} 0
gateway_host_active{gateway="fritz.box",hostname="pc-192-168-178-64",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="d4:ad:fc:25:18:03"} 1
gateway_host_active{gateway="fritz.box",hostname="pc-192-168-178-84",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="84:2e:14:ee:34:b6"} 1
gateway_host_active{gateway="fritz.box",hostname="pi-hole",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="02:42:c0:a8:b2:03"} 1
gateway_host_active{gateway="fritz.box",hostname="rk3308-robot32",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="f0:c8:14:37:62:0a"} 1
gateway_host_active{gateway="fritz.box",hostname="samsung-the-frame",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="7c:0a:3f:72:be:70"} 1
gateway_host_active{gateway="fritz.box",hostname="shelly1-e8db84acb65d",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="e8:db:84:ac:b6:5d"} 1
gateway_host_active{gateway="fritz.box",hostname="shellyht-762587",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="c4:5b:be:76:25:87"} 0
gateway_host_active{gateway="fritz.box",hostname="smuehlendick",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress=""} 0
gateway_host_active{gateway="fritz.box",hostname="toniebox",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="68:47:49:28:0a:5b"} 0
gateway_host_active{gateway="fritz.box",hostname="tower",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="00:d8:61:a6:dd:3f"} 1
gateway_host_active{gateway="fritz.box",hostname="tower",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="22:b8:ea:ac:32:e6"} 1
gateway_host_active{gateway="fritz.box",hostname="tower",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="6a:4d:40:e4:4c:f6"} 0
gateway_host_active{gateway="fritz.box",hostname="unraid",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="1c:83:41:29:25:69"} 0
gateway_host_active{gateway="fritz.box",hostname="valhalla",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="a8:a1:59:1a:1d:3b"} 0
gateway_host_active{gateway="fritz.box",hostname="vmb3010",interfacetype="Ethernet",ipaddress="<IP_ADDRESS>",macaddress="9c:3d:cf:3d:bf:27"} 1
gateway_host_active{gateway="fritz.box",hostname="watermeter",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="30:c6:f7:22:1d:b4"} 0
gateway_host_active{gateway="fritz.box",hostname="watermeter-reed",interfacetype="",ipaddress="<IP_ADDRESS>",macaddress="08:b6:1f:35:05:f0"} 0
gateway_host_active{gateway="fritz.box",hostname="wonderwall-smart-bueroeg",interfacetype="802.11",ipaddress="<IP_ADDRESS>",macaddress="d8:1f:12:fb:dc:4d"} 1
# HELP gateway_max_bitrate max bitrate on gateway interface
# TYPE gateway_max_bitrate gauge
gateway_max_bitrate{direction="Down",gateway="fritz.box",interface="DSL"} 1.43949e+08
gateway_max_bitrate{direction="Down",gateway="fritz.box",interface="WAN"} 1.1411e+08
gateway_max_bitrate{direction="Up",gateway="fritz.box",interface="DSL"} 4.4796e+07
gateway_max_bitrate{direction="Up",gateway="fritz.box",interface="WAN"} 4.1486e+07
# HELP gateway_traffic traffic on gateway interface
# TYPE gateway_traffic counter
gateway_traffic{direction="Received",gateway="fritz.box",interface="DSL",unit="Packets"} 117803
gateway_traffic{direction="Received",gateway="fritz.box",interface="LAN",unit="Bytes"} 2.542578392e+09
gateway_traffic{direction="Received",gateway="fritz.box",interface="WAN",unit="Bytes"} 2.540815994e+09
gateway_traffic{direction="Received",gateway="fritz.box",interface="WAN",unit="Packets"} 117803
gateway_traffic{direction="Sent",gateway="fritz.box",interface="DSL",unit="Packets"} 328163
gateway_traffic{direction="Sent",gateway="fritz.box",interface="LAN",unit="Bytes"} 2.932797116e+09
gateway_traffic{direction="Sent",gateway="fritz.box",interface="WAN",unit="Bytes"} 7.13684903e+08
gateway_traffic{direction="Sent",gateway="fritz.box",interface="WAN",unit="Packets"} 328074
# HELP gateway_traffic_rate traffic rate on gateway interface
# TYPE gateway_traffic_rate gauge
gateway_traffic_rate{direction="Received",gateway="fritz.box",interface="WAN",unit="Bytes"} 3440
gateway_traffic_rate{direction="Sent",gateway="fritz.box",interface="WAN",unit="Bytes"} 2585
# HELP gateway_uptime_seconds gateway uptime
# TYPE gateway_uptime_seconds gauge
gateway_uptime_seconds{description="FRITZ!Box 7490 113.07.29",gateway="fritz.box"} 445992
# HELP gateway_wan_connection_status WAN Connection status (Connected = 1)
# TYPE gateway_wan_connection_status gauge
gateway_wan_connection_status{connection="PPP",connectiontype="IP_Routed",externalipaddress="<IP_ADDRESS>",gateway="fritz.box",interface="WAN",macaddress="38:10:d5:a4:1b:0c"} 1
# HELP gateway_wlan_current_connections current WLAN connections
# TYPE gateway_wlan_current_connections gauge
gateway_wlan_current_connections{gateway="fritz.box",wlan="2.4 GHz"} 9
gateway_wlan_current_connections{gateway="fritz.box",wlan="5 Ghz"} 2
# HELP gateway_wlan_status WLAN status (Enabled = 1)
# TYPE gateway_wlan_status gauge
gateway_wlan_status{gateway="fritz.box",wlan="2.4 Ghz"} 1
gateway_wlan_status{gateway="fritz.box",wlan="5 Ghz"} 1
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 3.125e-05
go_gc_duration_seconds{quantile="0.25"} 4.238e-05
go_gc_duration_seconds{quantile="0.5"} 4.784e-05
go_gc_duration_seconds{quantile="0.75"} 7.6931e-05
go_gc_duration_seconds{quantile="1"} 0.00017346
go_gc_duration_seconds_sum 0.453172652
go_gc_duration_seconds_count 7607
# HELP go_goroutines Number of goroutines that currently exist.
# TYPE go_goroutines gauge
go_goroutines 12
# HELP go_info Information about the Go environment.
# TYPE go_info gauge
go_info{version="go1.16.3"} 1
# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
# TYPE go_memstats_alloc_bytes gauge
go_memstats_alloc_bytes 8.083848e+06
# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
# TYPE go_memstats_alloc_bytes_total counter
go_memstats_alloc_bytes_total 2.901196468e+10
# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
# TYPE go_memstats_buck_hash_sys_bytes gauge
go_memstats_buck_hash_sys_bytes 1.677464e+06
# HELP go_memstats_frees_total Total number of frees.
# TYPE go_memstats_frees_total counter
go_memstats_frees_total 2.32893148e+08
# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
# TYPE go_memstats_gc_cpu_fraction gauge
go_memstats_gc_cpu_fraction 4.176786513288098e-06
# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
# TYPE go_memstats_gc_sys_bytes gauge
go_memstats_gc_sys_bytes 5.749184e+06
# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
# TYPE go_memstats_heap_alloc_bytes gauge
go_memstats_heap_alloc_bytes 8.083848e+06
# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
# TYPE go_memstats_heap_idle_bytes gauge
go_memstats_heap_idle_bytes 5.431296e+07
# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
# TYPE go_memstats_heap_inuse_bytes gauge
go_memstats_heap_inuse_bytes 1.1583488e+07
# HELP go_memstats_heap_objects Number of allocated objects.
# TYPE go_memstats_heap_objects gauge
go_memstats_heap_objects 44014
# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
# TYPE go_memstats_heap_released_bytes gauge
go_memstats_heap_released_bytes 5.3026816e+07
# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
# TYPE go_memstats_heap_sys_bytes gauge
go_memstats_heap_sys_bytes 6.5896448e+07
# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
# TYPE go_memstats_last_gc_time_seconds gauge
go_memstats_last_gc_time_seconds 1.6806195308395948e+09
# HELP go_memstats_lookups_total Total number of pointer lookups.
# TYPE go_memstats_lookups_total counter
go_memstats_lookups_total 0
# HELP go_memstats_mallocs_total Total number of mallocs.
# TYPE go_memstats_mallocs_total counter
go_memstats_mallocs_total 2.32937162e+08
# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
# TYPE go_memstats_mcache_inuse_bytes gauge
go_memstats_mcache_inuse_bytes 19200
# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
# TYPE go_memstats_mcache_sys_bytes gauge
go_memstats_mcache_sys_bytes 32768
# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
# TYPE go_memstats_mspan_inuse_bytes gauge
go_memstats_mspan_inuse_bytes 309808
# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
# TYPE go_memstats_mspan_sys_bytes gauge
go_memstats_mspan_sys_bytes 704512
# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
# TYPE go_memstats_next_gc_bytes gauge
go_memstats_next_gc_bytes 9.043472e+06
# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
# TYPE go_memstats_other_sys_bytes gauge
go_memstats_other_sys_bytes 3.649968e+06
# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
# TYPE go_memstats_stack_inuse_bytes gauge
go_memstats_stack_inuse_bytes 1.212416e+06
# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
# TYPE go_memstats_stack_sys_bytes gauge
go_memstats_stack_sys_bytes 1.212416e+06
# HELP go_memstats_sys_bytes Number of bytes obtained from system.
# TYPE go_memstats_sys_bytes gauge
go_memstats_sys_bytes 7.892276e+07
# HELP go_threads Number of OS threads created.
# TYPE go_threads gauge
go_threads 21
# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
# TYPE process_cpu_seconds_total counter
process_cpu_seconds_total 377.1
# HELP process_max_fds Maximum number of open file descriptors.
# TYPE process_max_fds gauge
process_max_fds 40960
# HELP process_open_fds Number of open file descriptors.
# TYPE process_open_fds gauge
process_open_fds 12
# HELP process_resident_memory_bytes Resident memory size in bytes.
# TYPE process_resident_memory_bytes gauge
process_resident_memory_bytes 4.0562688e+07
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.68016248053e+09
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 2.052722688e+09
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 30429
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

the exporter is just collecting, what is configured in your metrics.json and reported by the fritzbox. so I guess you need to figure out if (and how) the fritzbox is making the mash details available, I at least could not find any mesh info in the documentation:
https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AVM_TR-064_first_steps.pdf
if you figure out how to get it, you can simply add it to your metrics.json


Cannot import pyscipopt into python ( MacOS Catalina)

I am trying to use pyscipopt on a Mac. I downloaded the SCIP Optimization Suite in my application folder and installed it by instruction. And I installed pyscipopt by typing: pip3 install pyscipopt in the terminal.
When I tried to import pyscipopt in Python, following messages came up:
from pyscipopt import Model
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/init.py", line 10, in
from pyscipopt.scip      import Model
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so, 2): Library not loaded: libscip.7.0.dylib
Referenced from: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so
Reason: unsafe use of relative rpath libscip.7.0.dylib in /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so with restricted binary
I have no idea how to fix this problem. Could you pls give me a little help

Did you specify the environment variable SCIPOPTDIR to point to the SCIP installation directory?

yes, I tried export SCIPOPTDIR=/usr/local, where has a subdirectory lib and a subdirectory include. It didn't work.
And I tried export SCIPOPTDIR=/Applications/scipoptsuite-7.0.0, where contains the source codes of SCIP Optimizer. The same problem occurred.

@anqi9 Not sure this is directly related but I'm also investigating installation of SCIP in MacOS. Could you give us the ouptut of the following commands:
otool -L /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so

otool -l /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so | grep LC_RPATH -A2

otool -D /usr/local/lib/libscip.dylib

The followings are the outputs:
Leung@LeungdeMBP ~ % otool -L /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so:
libscip.7.0.dylib (compatibility version 7.0.0, current version 7.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
Leung@LeungdeMBP ~ % otool -l /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so | grep LC_RPATH -A2
cmd LC_RPATH
cmdsize 104
path /private/var/folders/2x/zklnbj6n7ss0ff0x6xqm3qlr0000gn/T/pip-install-9ysn85qh/pyscipopt/lib (offset 12)
Leung@LeungdeMBP ~ % otool -D /usr/local/lib/libscip.dylib
/usr/local/lib/libscip.dylib:
libscip.7.0.dylib

@anqi9 Not sure this is directly related but I'm also investigating installation of SCIP in MacOS. Could you give us the ouptut of the following commands:
otool -L /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so

otool -l /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/pyscipopt/scip.cpython-38-darwin.so | grep LC_RPATH -A2

otool -D /usr/local/lib/libscip.dylib

Can you try specifying the environment variable DYLD_LIBRARY_PATH to point to the SCIP bin directory which contains libscip.7.0.dylib.
The error here seems to be that this library cannot be found because it is not in any of the default search locations.

Can you try specifying the environment variable DYLD_LIBRARY_PATH to point to the SCIP bin directory which contains libscip.7.0.dylib.
The error here seems to be that this library cannot be found because it is not in any of the default search locations.

I have tried to set the environment variable as export DYLD_LIBRARY_PATH=\usr\local\lib, which contains libscip.7.0.dylib or export DYLD_LIBRARY_PATH=/Applications/scipoptsuite-7.0.0/build/lib which contains libscip.7.0.dylib as well. Neither of them works.
I have checked on the internet, from one the pages said:" The behaviour of dylib may be controlled by environment variables, however If System Integrity Protection is enabled, environment variables are ignored when executing binaries protected by System Integrity Protection". I'm not so sure whether this issue is related to the relative path.

I have encountered with this problem too. Methods to modify DYLD_LIBRARY_PATH are generally invalid due to Integrity Protection. I fixed this problem by changing the pathlibscip.7.0.dylib in ../pyscipopt/scip.cpython-37m-darwin.so to a more specific  /Users/apple/SCIPOptSuite-7.0.0-Darwin/lib/libscip.7.0.dylib, which is the actual path of my dynamic library for scip. Then, all goes right.
first, this is error message:
ImportError: dlopen(/Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so, 2): Library not loaded: libscip.7.0.dylib
  Referenced from: /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so
  Reason: image not found

you need use otool -L /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so to get a lock on our trouble:
libscip.7.0.dylib (compatibility version 7.0.0, current version 7.0.0)
ahh, a haunting ghost without specific path.
Then, use sudo install_name_tool -change libscip.7.0.dylib /Users/apple/SCIPOptSuite-7.0.0-Darwin/lib/libscip.7.0.dylib /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so   to fix it.

I have encountered with this problem too. Methods to modify DYLD_LIBRARY_PATH are generally invalid due to Integrity Protection. I fixed this problem by changing the pathlibscip.7.0.dylib in ../pyscipopt/scip.cpython-37m-darwin.so to a more specific  /Users/apple/SCIPOptSuite-7.0.0-Darwin/lib/libscip.7.0.dylib, which is the actual path of my dynamic library for scip. Then, all goes right.
first, this is error message:
ImportError: dlopen(/Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so, 2): Library not loaded: libscip.7.0.dylib
  Referenced from: /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so
  Reason: image not found

you need use otool -L /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so to get a lock on our trouble:
libscip.7.0.dylib (compatibility version 7.0.0, current version 7.0.0)
ahh, a haunting ghost without specific path.
Then, use sudo install_name_tool -change libscip.7.0.dylib /Users/apple/SCIPOptSuite-7.0.0-Darwin/lib/libscip.7.0.dylib /Users/apple/anaconda3/lib/python3.7/site-packages/pyscipopt/scip.cpython-37m-darwin.so  to fix it.

Thank you for the solution, it finally works.


Fix bottom app bar padding is not exist

Issue

close #62

Overview (Required)

This bug occurs when the navigation bar is below the app bar.
If navigation bar is bottom, add navigation bar height to app bar padding.

Links

N/A

Screenshot

Before
After

Nexus5X Android5.0

Nexus9 Android7.1.1

👀

@cheddar-dev
Hi,
Sorry for late reply to your comment.
I've investigate this issue and found one of a solution.
So, I take over the resolving of this issue :D
Sorry for the too late support, however thanks to you we have noticed the difficulty of this issue.
Thank you for your cooperation.

The following PR has been taken over.
#741


Adjust server-side to read web server-side

There are other kinds of servers than web. Just making that specific.

@Gabriel439 I seem to have broken the link. :( Could you fix it? I don't really understand how it works, sadly.

Actually, we should probably revert this anyway because I just realized that this section does include service programming, too


Is it possible to fcc hack the DJI FPV drone on latest firmware?

I love the DJI FPV drone due to the gimbal, superior image, and position hold for flying in mountains. Since I was forced to upgrade to latest 1.04 firmware to support DJI o3 Air unit I traded in the b3yond fcc hack that is now nolonger working. Is it technically possible to do a fcc hack by wtf for the DJI FPV drone? Is it possible to roll back the DJI FPV firmware only and not the others with the root?

Greate! I have this question too.

As i understood there is no way to root / setup wtf once you have fw .15 or newer.

I have root than upgraded

I have root than upgraded

Yes but the issue is that many have already the upgrade without root. There is no solution for that. My goggles were replaced by support with a version that had the newer firmware :(


CB-15469 New clusters: close the gate - execute the patcher scripts f…

…or all new clusters
See detailed description in the commit message.

source of the script: https://github.infra.cloudera.com/QE/deploy/blob/log4j-scripts/cdep/data/jndi_scripts/cm_cdp_cdh_log4j_jndi_removal.sh

Unfortunately during provision this can increase the duration of provision significantly:
(salt_3000.8) [root@adamdh1-master0 cloudbreak]# salt-run jobs.lookup_jid<PHONE_NUMBER>4124377625 | grep run_log4j_patch -A 10
/opt/salt_3000.8/lib/python3.6/site-packages/salt/utils/jinja.py:656: DeprecationWarning: 'contextfunction' is renamed to 'pass_context', the old name will be removed in Jinja 3.1.
  @jinja2.contextfunction
          ID: run_log4j_patch
    Function: cmd.script
        Name: patch
      Result: True
     Comment: Command 'patch' run
     Started: 11:41:33.782797
    Duration: 1486040.015 ms
     Changes:
              ----------
              pid:
                  12353
--
          ID: run_log4j_patch
    Function: cmd.script
        Name: patch
      Result: True
     Comment: Command 'patch' run
     Started: 11:41:33.218886
    Duration: 1486573.763 ms
     Changes:
              ----------
              pid:
                  8704
--
          ID: run_log4j_patch
    Function: cmd.script
        Name: patch
      Result: True
     Comment: Command 'patch' run
     Started: 11:41:33.230438
    Duration: 1486562.176 ms
     Changes:
              ----------
              pid:
                  8702
--
          ID: run_log4j_patch
    Function: cmd.script
        Name: patch
      Result: True
     Comment: Command 'patch' run
     Started: 11:41:33.610835
    Duration: 1486184.123 ms
     Changes:
              ----------
              pid:
                  8701
--
          ID: run_log4j_patch
    Function: cmd.script
        Name: patch
      Result: True
     Comment: Command 'patch' run
     Started: 11:41:34.315473
    Duration: 1485482.217 ms
     Changes:
              ----------
              pid:
                  8690

data mart cluster:
(salt_3000.8) [root@adamdm1-master0 cloudbreak]# salt-run jobs.lookup_jid<PHONE_NUMBER>1303930469 | grep run_log4j_patch -A 10
/opt/salt_3000.8/lib/python3.6/site-packages/salt/utils/jinja.py:656: DeprecationWarning: 'contextfunction' is renamed to 'pass_context', the old name will be removed in Jinja 3.1.
  @jinja2.contextfunction
          ID: run_log4j_patch
    Function: cmd.run
        Name: /opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}
      Result: True
     Comment: Command "/opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}" run
     Started: 13:13:23.082179
    Duration: 1889049.403 ms
     Changes:
              ----------
              pid:
                  12453
--
          ID: run_log4j_patch
    Function: cmd.run
        Name: /opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}
      Result: True
     Comment: Command "/opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}" run
     Started: 13:13:22.571081
    Duration: 1889430.015 ms
     Changes:
              ----------
              pid:
                  8783
--
          ID: run_log4j_patch
    Function: cmd.run
        Name: /opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}
      Result: True
     Comment: Command "/opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}" run
     Started: 13:13:22.543267
    Duration: 1889459.695 ms
     Changes:
              ----------
              pid:
                  8773
--
          ID: run_log4j_patch
    Function: cmd.run
        Name: /opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}
      Result: True
     Comment: Command "/opt/log4jpatch/scripts/cm_cdp_cdh_log4j_jndi_removal.sh 2>&1 | tee -a /var/log/log4jpatch.log && exit ${PIPESTATUS[0]}" run
     Started: 13:13:22.510583
    Duration: 1889639.736 ms
     Changes:
              ----------
              pid:
                  8786


MacOS Specific Productivity Tool Evaluation

Evaluate and add to new MacOS setup:

https://rectangleapp.com/ (replaces Spectacle)
https://www.raycast.com/
https://docs.folivora.ai/ -  BetterTouchTool
https://www.macbartender.com/
https://bjango.com/mac/istatmenus/

Rectangle is already in beleyenv (with a follow-up to check in config). Other than that, the only other thing I have not tried that is interesting is bartender.
Closing, but may re-visit when I have a macbook again (if ever)


feat buy & sell kin screen

Buy & Sell Screen
Account sheet

Remaining changes:

[ ] Translations
[ ] Analytics on video shared

On my P8P, the video isn't utilizing all of the space.

I'm also noticing the ripple on the share button come in and out very quickly when pressed. It appears very "grainy".

I think that's the native Android thing, can you check if you are experiencing the same when trying to share bills?

@jeffyanta updated the PR!

@cdhiraj40 just tried the latest changes. Looks like the sizing is correct. I am noticing the white highlight upon pressing the video extends beyond the video preview. Note the extra padding at the bottom, and curved edges.

@cdhiraj40 just tried the latest changes. Looks like the sizing is correct. I am noticing the white highlight upon pressing the video extends beyond the video preview. Note the extra padding at the bottom, and curved edges.

ah my bad, the latest changes should fix it!

On the share button issue, I've uploaded a video to show what's going on my device. You might need to zoom in on the top share button, since it's quite subtle.
https://github.com/code-payments/code-android-app/assets/5760385/d66c9909-35c7-4844-b506-aad261803264


[FIX] fix a typo in question 13: stopPropogation -> stopPropagation

I fixed a typo in question 13.
stopPropogation -> stopPropagation
reference: https://developer.mozilla.org/en-US/docs/Web/API/Event/stopPropagation

Thanks @mhsiungw for PR


Misuse of Jackson ObjectMapper

I noticed java-jwt doesn't use Jackson's ObjectMappers in the most optimal way. They are meant to be fully thread-safe (after configuration) and reusable, as their construction is a relative heavy process. Instead, java-jwt seems to create a new one for almost any operation, including JWTParser and JWTCreator construction. Even some getter-type parsing methods create one just to parse a single value.
In practice, they could be replaced with a single static instance.

I don't know what Jackson says about this but when these ObjectMappers are created, they are configured differently across this codebase. Re-using (and thus re-configuring) a ObjectMapper doesn't seem thread-safe to me at all. Could you provide some more insight and possibly a PR?

As far as I can tell, every call to JWTVerifier.verify() creates a new JWTDecoder, which creates a new JWTParser, which creates a new ObjectMapper. So even if application only verifies tokens, each verification creates a separate ObjectMapper.
Furthermore, at least JsonNodeClaim creates a single-use ObjectMapper for asArray, asList and as methods, as does PayloadDeserializer.getStringOrArray. I think most, if not all, of these could be merged, as I don't think their configurations clash with each other.
For reference, http://stackoverflow.com/a/3909846/434429 (answer by Jackson's developer)

Hello!
For our microservice applications we have implemented a ResourceServerTokenServices to do local JWT OAuth2 token validation using the JwtVerifier-class.
Recently we figured out that roughly 10% of our request fail due to JsonParseException. I was able to verify the underlying usage of JWTVerifier was causing the problem. Rough description: We have one service-class (singleton) which is responsible for the token-validation, this service-class utilizes JWTVerifier.class method verify(token).
Since the constructor-call looks like this: "new JWTVerifier(this.getPublicKeyFromCertificate(), this.getAudience(), this.getIssuer());", I assumed we could use this object thread-safe for multiple verify()-calls.
Turns out this does not work very well, but when creating a new instance of JWTVerifier each time we want to verify a token, everything's ok. I'd like to point out that this is not much more expensive in terms of calculation power (roughly 5%), but please add a bold hint / javadoc, so everyone can see how this class-method should be invoked!
com.auth0.jwt.internal.com.fasterxml.jackson.core.JsonParseException: Illegal character ((CTRL-CHAR, code 0)): only regular white space (\r, \n, \t) is allowed between tokens  at [Source: java.io.StringReader@2c28b43; line: 1, column: 2]         at com.auth0.jwt.internal.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1284)         at com.auth0.jwt.internal.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:588)         at com.auth0.jwt.internal.com.fasterxml.jackson.core.base.ParserMinimalBase._throwInvalidSpace(ParserMinimalBase.java:534)         at com.auth0.jwt.internal.com.fasterxml.jackson.core.json.ReaderBasedJsonParser._skipWSOrEnd(ReaderBasedJsonParser.java:1493)         at com.auth0.jwt.internal.com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:380)         at com.auth0.jwt.internal.com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:2601)         at com.auth0.jwt.internal.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2548)         at com.auth0.jwt.internal.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1805)         at com.auth0.jwt.JWTVerifier.decodeAndParse(JWTVerifier.java:236)         at com.auth0.jwt.JWTVerifier.verify(JWTVerifier.java:109)

@derandreasberger the verifier is meant to be reusable, can you provide more info since the exception can be caused by something unrelated to the lib?

@hzalaz I think i can not provide the whole bunch of code, but I'd like to display the problematic piece of Constructor/method invocation.
The original class had a private variable holding the JWTVerifier (method this.setJwtVerifier(...), respectively this.getJwtVerifier()). This needed to be replaced with a new class initiation.
`    private boolean verifyTokenAndSignature(String token) throws InvalidKeyException {
    boolean tokenVerificationOk = false;

    try {

        //DDS-1751: Concurrency issue fix (JWTVerifier is not thread-safe, checkout LocalTokenVerificationSingletonLoadTest for testing)

//            if (this.getJwtVerifier() == null) {
//                this.initTokenValidator();
//            }
//            Map<String, Object> map = this.getJwtVerifier().verify(token);
JWTVerifier jwtVerifier = new JWTVerifier(this.getPublicKeyFromCertificate(), this.getAudience(), this.getIssuer());
        Map<String, Object> map = jwtVerifier.verify(token);

        LocalTokenValidation.logger
                .debug("JwtVerifier return keys: " + Arrays.deepToString(map.keySet().toArray()));

        if (map != null && !map.isEmpty()) {
            LocalTokenValidation.logger.info("No Exception thrown, verification ok!");
            tokenVerificationOk = true;
        }

    } catch (InvalidKeyException | SignatureException | JWTVerifyException e) {

        LocalTokenValidation.logger.error("InvalidKeyException check if parameters are correct (aud: " + this.getAudience() + ", iss: " + this.getIssuer() + ", certificate: " + this.getCertifcateFileName() + ")", e);
        throw new InvalidKeyException(e);

    } catch (NoSuchAlgorithmException | IllegalStateException | IOException e) {

        LocalTokenValidation.logger.error("InvalidKeyException caused by ", e);
        throw new InvalidKeyException(e);

    }

    return tokenVerificationOk;
}

public void initTokenValidator() {

    LocalTokenValidationWithSingletonVerifier.logger.debug("Initializing TokenValidator");

    this.setJwtVerifier(
            new JWTVerifier(this.getPublicKeyFromCertificate(), this.getAudience(), this.getIssuer()));

}`

@derandreasberger I see no sign of a threading issue in the decoding code and the exception you got can be caused by a corrupted input that ends up with a malformed token passed to java-jwt. I said this since we only receive String not Streams so it's unlikely we are adding or modifying the string in some way to add these unexpected chars. We'll review this once we tackle the original issue reported here.
@arlampin its not the recommended way but I don't want to have a static singleton with the object mapper or keep it in a static variable in some already existing class. However, there are room for improvements like passing a single mapper to all who need to use them or use the recommended reader and writers (I am more inclined to do this) created from a single mapper that is stored in the verifier/creator since they are meant to be shared.
In order to make this clear, I'll close this one and create a new one stating this improvement


Improve AnyMock typing

Better type hint for AnyMock. Because PropertyMock is typed as Any, AnyMock also becomes Any and the type hint is not really useful in catching bugs.

Seems that the issue was fixed in typeshed:
https://github.com/python/typeshed/commit/43a9ab08d77af669652720e7579bc97956eb31aa


updating master

Codecov Report

Merging #195 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #195      +/-   ##
==========================================
- Coverage   99.16%   99.15%   -0.01%
==========================================
  Files          43       43
  Lines        1317     1308       -9
  Branches      152      152
==========================================
- Hits         1306     1297       -9
  Misses         11       11

Impacted Files
Coverage Δ

src/proptypes.js
100% <ø> (ø)
:arrow_up:

src/redux/actions/datatableActions.js
100% <ø> (ø)
:arrow_up:

src/redux/reducers/datatableReducer.js
98.72% <100%> (-0.01%)
:arrow_down:

src/components/DatatableCore/Body/Body.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c7c7fd...55302f4. Read the comment docs.


[Python][Docker] Cannot build Pyarro on python38 docker image

Hi all,

Trying to build pyarrow on a python38 docker image in order to build a aws-lambda layer.
I had success on doing this couple months back with the dockerfile attached. (Python37) but now getting the error either I try either python version below.

FROM lambci/lambda:build-python3.7

FROM lambci/lambda:build-python3.8

which leads me to think that something else has changed.

#8 6.648 Scanning dependencies of target boost_ep
#8 6.657 [  0%] Creating directories for 'boost_ep'
#8 6.689 [  1%] Performing download step (download, verify and extract) for 'boost_ep'
#8 8.038 -- boost_ep download command succeeded.  See also /arrow/cpp/build/boost_ep-prefix/src/boost_ep-stamp/boost_ep-download-*.log
#8 8.049 [  1%] No patch step for 'boost_ep'
#8 8.060 [  1%] No update step for 'boost_ep'
#8 8.071 [  2%] Performing configure step for 'boost_ep'
#8 8.088 CMake Error at /arrow/cpp/build/boost_ep-prefix/src/boost_ep-stamp/boost_ep-configure-RELEASE.cmake:16 (message):
#8 8.088   Command failed: 1
#8 8.088
#8 8.088    './bootstrap.sh' '--prefix=/arrow/cpp/build/boost_ep-prefix/src/boost_ep' '--with-libraries=filesystem,regex,system'
#8 8.088
#8 8.088   See also
#8 8.088
#8 8.088     /arrow/cpp/build/boost_ep-prefix/src/boost_ep-stamp/boost_ep-configure-*.log
#8 8.088
#8 8.088
#8 8.089 make[2]: *** [boost_ep-prefix/src/boost_ep-stamp/boost_ep-configure] Error 1
#8 8.089 make[1]: *** [CMakeFiles/boost_ep.dir/all] Error 2
#8 8.090 make: *** [all] Error 2

executor failed running [/bin/sh -c mkdir /arrow     && curl -o /tmp/apache-arrow.tar.gz -SL https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz     && tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1     && mkdir /arrow/dist     && export LD_LIBRARY_PATH=/dist/lib:$LD_LIBRARY_PATH     && mkdir -p /arrow/cpp/build     && cd /arrow/cpp/build     && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE     -DCMAKE_INSTALL_LIBDIR=lib     -DCMAKE_INSTALL_PREFIX=$ARROW_HOME     -DARROW_PARQUET=on     -DARROW_PYTHON=on     -DARROW_PLASMA=on
   -DARROW_WITH_SNAPPY=on     -DARROW_BUILD_TESTS=OFF     ..     && make     && make install]: exit code: 2
The terminal process "C:\WINDOWS\System32\cmd.exe /K C:\tools\cmder\vendor\bin\vscode_init.cmd /d /c docker build --pull --rm -f "Docker\pyarrow\Dockerfile" -t pyarrow37:lambci-lambda "Docker\pyarrow"" terminated with exit code: 1.

Note: Dockerfile has plenty of commented out code. Output above corresponds to the non commented out code built attempt.

Thank you
Alex

Reporter: Alexandre Campino
Original Issue Attachments:

Dockerfile

Note: This issue was originally created as ARROW-13123. Please see the migration documentation for further details.

Given this is an old issue, and there was no further response, going to close this.


Error when opening .ts file

My vim seems to hang when opening .ts files. I ran a git bisect and traced the problem commit to here: a4ffd2f37c5d535d62170f7e3021cd2981689988
 Current Filetype: typescript
Available Linters: ['eslint', 'tslint', 'tsserver', 'typecheck']
  Enabled Linters: ['eslint', 'tslint', 'tsserver', 'typecheck']
 Linter Variables:

let g:ale_typescript_tslint_config_path = ''
let g:ale_typescript_tslint_executable = 'tslint'
let g:ale_typescript_tslint_use_global = 0
let g:ale_typescript_tsserver_config_path = ''
let g:ale_typescript_tsserver_executable = 'tsserver'
let g:ale_typescript_tsserver_use_global = 0
 Global Variables:

let g:ale_echo_cursor = 1
let g:ale_echo_msg_error_str = 'Error'
let g:ale_echo_msg_format = '%s'
let g:ale_echo_msg_warning_str = 'Warning'
let g:ale_enabled = 1
let g:ale_fix_on_save = 0
let g:ale_fixers = {}
let g:ale_keep_list_window_open = 0
let g:ale_lint_delay = 200
let g:ale_lint_on_enter = 1
let g:ale_lint_on_save = 1
let g:ale_lint_on_text_changed = 'always'
let g:ale_linter_aliases = {}
let g:ale_linters = {}
let g:ale_open_list = 1
let g:ale_set_highlights = 1
let g:ale_set_loclist = 0
let g:ale_set_quickfix = 1
let g:ale_set_signs = 1
let g:ale_sign_column_always = 0
let g:ale_sign_error = '>>'
let g:ale_sign_offset = 1000000
let g:ale_sign_warning = '--'
let g:ale_statusline_format = ['%d error(s)', '%d warning(s)', 'OK']
let g:ale_warn_about_trailing_whitespace = 1
  Command History:

(started) ['/bin/zsh', '-c', 'node_modules/.bin/tsserver']

Could you follow the advice here and share the profiler results? https://stackoverflow.com/questions/12213597/how-to-see-which-plugins-are-making-vim-slow/12216578#12216578
Then I might be able to figure out what's making Vim "hang."

profile.log

Appears that restarting my computer has fixed the problem. I'll keep this issue open in case it's useful to dig into why I was in such a state.

All the time was being sent on simply sending a buffer to tsserver.
    2              7.115065         call ch_sendraw(job_getchannel(s:job_map[a:job_id].job), a:string)

I don't think there's anything I can do about that.


_tx_semaphore_created_ptr is always NULL

Describe the bug
I want to print the semaphore information, but the value of the pointer variable _tx_semaphore_created_ptr is always NULL.
Device: Cortex-M4
ThreadX Version: 6.4.1
Toolchain: arm-gnu-toolchain-13.2.Rel1-mingw-w64-i686-arm-none-eabi
To Reproduce
#include <tx_api.h>
#include <tx_semaphore.h>

// ...

static void show_sem(void)
{
    uart_printf("sem_ptr = %p\n", _tx_semaphore_created_ptr);
    uart_printf("sem_cnt = %d\n", _tx_semaphore_created_count);
}

// ...

I have created several semaphores in other files and they are running normally.
Expected behavior
expected:
sem_ptr = 0x2000****
sem_cnt = [greater than 0]

but the actual output
sem_ptr = 0
sem_cnt = 0

Hi @xray-bit ,
can you please give us 'minimal reproducible example'?
From what you wrote I would guess you have some bug and show_sem is running before you created the semaphores or after you deleted them.

Sorry, I was busy earlier. Thanks for the reply.
When I created a new project to reproduce the problem, it became normal again and I will check why the previous one was faulty.


[bazel] Handle site-specific environments through env

Due to the multi-tenant nature of our development system almost nothing we use is installed in the default directory.   Our PATH, LD_LIBRARY_PATH and PKG_CONFIG_PATH environments consist of long strings of site-specific paths holding everything.  We have environment module tools in place to quickly switch from one project to another, and we generally try to do everything through environment variables.  When file configurations are necessary, we try to store them in common space shared by the team.
Could we use environment variables to pass site-specific PATHs into bazel via bazelisk.sh.  For example one could create an OPENTITAN_BAZEL_BUILD_ENV variable, that could for instance contain any number of --action_env options for bazelisk.sh to pass into .bin/bazelisk.
Rationale for this request:
I realize that the same functionality can be accommodated using $HOME/.bazelrc, but this is suboptimal for two reasons:

Fixed configuration files would make it hard for us to reproduce issues or switch projects.   We generally discourage users from storing configuration information in their home directory for reasons related to reproducibility.
Our paths are configured via many different files that are updated independently.

For a slightly different approach, would an OPENTITAN_USER_BAZELRC_PATH work? Then, when it's defined, we can pass in a user-defined bazelrc arg in bazelisk.sh, and all the bazelrc flexibility exists for site-specific configuration.

Hi Alex,
As I was reading more about the documentation was just thinking something similar.  Another approach could be to have something like try-import .bazelrc-site in .bazelrc.  I'm giving that a try right now actually.   If that works then this would close both #14767 and #14768 for us.

Having a .bazelrc-site located in the repo, would work pretty well in our environment as we are already tracking the repo and so we could just make a common file that our environment script drops into the repo.   Would this be okay for the rest of the team?

Though I can see how having it completely outside the repo and pointed to by an environment variable would also be cleaner, as there may be less possibility for confusion & cross-site contamination

I think both approaches work fine. They are described in https://github.com/bazelbuild/bazel/issues/10904#issuecomment-818717854 . You can see there is quite a discussion there around a similar request, heh.
Compared to the wrapper script with the environment variable, the advantage of try-import is that you can potentially use the bazel executable directly on the command line, which is nice for bash completion on the interactive side. On the other hand, the environment variable is a bit less cumbersome for setup, since new worktrees can "just work" instead of requiring generation of the user bazelrc file again.
As a user of such a feature, your input will hold more weight about which is the nicer workflow. :)

Hey neat!  I knew this was probably discussed somewhere, but I just didn't go back far enough.  This is good stuff.  Thanks for the pointer.


Failed to do advanced query if interval is specified and endTime is not specified

Because Mr. G created a fake end time which is around year 3939, it gives a really huge number as interval section number.

@AlvISsReimu we should set endTime to time.Now()

I'll do the fix


Format flag for add and autoadd

Right now if I do an autoadd sometimes is in YAML and other times in JSON (but always has the yaml extension). I think, if the user can control the format, that would be the best.

Hmm, I've seen this before, but haven't been able to consistently reproduce. If you know a way to reproduce this let me know. Does specifying goss -g goss.yaml add/autoadd force it into YAML mode for you?

-g file.(yaml|json) will format the output accordingly if it is a new file (which is awesome, is this documented?).
So I think I found what was happening to me. If add or aa has nothing to return, a file with {} get created. Any subsequent adds or autoadds will be in JSON, regardless of -g. I noticed then when I attempted goss -g apache.yaml aa apache (which is incorrect for Ubuntu 14.04). Then I did goss -g apache.yaml aa apache2 and it was JSON.

The suggested edit above (latest commit) is good for #173 and avoids an extra few CPU cycles of hitting setStoreFormatFromData when StoreFormat is already set.
However a quick test of a non-existent resource in a new file still outputs an evil nil JSON object aka: {}
I'm kinda thinking here that a quick test for an empty resource before writing the file and printing a message and not an writing an empty file?
What do you think?

Good spot @CMeza99 I could reproduce it every time. If the file doesn't exist and you add / autoadd a resource that doesn't exist you get a file with {} in. I've created a PR that tackles this and another issue but would appreciate your feedback to add some context around how you're using goss.
The PR checks if the resulting config is empty, notifies the user that the configuration is empty and doesn't create a file however, I'm not sure if this behaviour would cause a problem (missing file) for your use-case, it might be better to create an empty file appropriately:
JSON:
{}

YAML:
---

What do you think?

@pysysops I could be wrong, but I believe the yaml driver in go writes an empty file as:
{}

Which is valid YAML, but confusing. I don't think it's that big of an issue if subsequent adds are in the correct format. That said, I think the cleanest solution is not creating a file at all (your solution).


Add Cloudera integration

What does this PR do?

This PR adds support for collecting metrics and service checks from Cloudera via Cloudera Manager.
Motivation

AI-2511
Additional Notes

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] PR title must be written as a CHANGELOG entry (see why)
[ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
[ ] PR must have changelog/ and integration/ labels attached
[ ] If the PR doesn't need to be tested during QA, please add a qa/skip-qa label.

Created https://datadoghq.atlassian.net/browse/DOCS-4546 for tracking the editorial review.


fix: truncate tag text instead of wrapping

#324  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Move key retrieval to Firebase

When I decompiled the app it still has the old values in it. So I am quite sure they deploy the new values using Firebase remote-config: https://firebase.google.com/docs/remote-config
A test to check this would be to install the official app on an android device without Google Play Services (e.g. a LineageOS custom ROM and not installing any GApps) and to see if the app gets the updated key or not (I suspect not: because Firebase depends on Google Play Services).
Afaik there is no method (yet) to eavesdrop on the Firebase/Google connection. Though you could try to check the contents of the local configuration SQLite database of the app.

@mfonville it is possible to intercept the firebase request using Fiddler (install Fiddler on a PC, and set the proxy on the phone wifi connection).
You then have to clear the application settings, and restart the application to force the application to retrieve the new keys.
We need to find a way to make that generic...

Adding this link for reference: https://danq.me/2018/08/07/android-emulator-https-intercept/


<IP_ADDRESS> update caused the mod to stop loading.

The mod wont load after the <IP_ADDRESS> update. Had to download the old patch from the site since rollback wasn't available due to not loading.

Manually download and install latest release.


Call for co-maintainers

leptos-hotkeys is pretty stable at this point, but there are still rough edges that need some love like #139 and #141. Unfortunately, I don't have the time, squeeze, and the Leptos 0.7 experience to steer this project.
Calling for co-maintainers to everyone who contributed before and/or have significant leptos experience.
The codebase is very small and pretty simple. Happy to chat with anyone interested.

@phillipbaird Would you be interested to become the maintainer of this crate? :)


Sub-fields support?

Fuse.js allows us to do stuff like this:
{
  keys: [
    {
      name: 'title',
      weight: 1.0,
    },
    {
      name: 'body',
      weight: 0.75,
    },
    {
      name: 'user.login',
      weight: 0.3,
    },
    {
      name: 'labels.name',
      weight: 0.3,
    },
  ],
}

...which is really nice.
AFAICT from the docs/guide, here I would need to flatten the data, right?
But how would that work with e.g. my labels?
Thank you!

A function can be used to 'extract' a field from a document, this is specified when configuring the index, e.g.

var fooFieldExtractor = function (doc) {
  return doc.foo.bar.baz
}

lunr(function () {
  this.field("foo", extractor:
    return doc.field.subField
  }


NotImplementedError: Unsupported ONNX ops of type: RandomNormal

I'm hitting this Unsupported ONNX ops error when trying to convert a VAE from PyTorch to ONNX to CoreML.
The only place I use randn() in the model is in the reparametrize function:
def reparametrize(self, mu, logvar):
    std = logvar.mul(0.5).exp_()
    if self.have_cuda:
        eps = torch.randn(self.bs, self.nz, device='cuda')
    else:
        eps = torch.randn(self.bs, self.nz)
    return eps.mul(std).add_(mu)

Is there some way to add this by hand? I'm surprised that this is an issue, since generating a random normal tensor is pretty standard.
NOTE: I do recognize that, in many cases, only the decoder would be converted for a VAE, using a noise vector as input (i.e., from the host app). However, I'm using the VAE for "correcting" inputs, so to speak, so I need to input an image. I suppose it's conceivable that I could convert the encoder and decoder separately, and in my host app use the encoder to get my mu, logvar, then send that into the decoder... but that seems like a lot of messing around...

Custom layers.
Since random data generation is n to common for inference tasks, this particular operation is not present in the CoreML spec.

Ugh... okay, I think I've seen a blog post on that. Will give it a go. Thanks.

Do you have any suggestions as to how this might be done? The ONNX RandomNormal custom layer is failing in Xcode because the layer has no inputs. Can I somehow add a bogus input of some kind?

Yes you may need to add an input. You can add sth like mean and variance and then produce the output which are samples drawn from that distribution.

Issue resolved with suggestion of adding custom layer.


[Bug] There is an problem with the execution of the lag function on nested array type fields

Search before asking

[X] I had searched in the issues and found no similar issues.

Version
2.1
What's Wrong?
表dwd_content_detail中存在字段：feature_embedding ARRAY<ARRAY< FLOAT >>，且部分记录中该字段值为空，然后当执行sql（ SELECT id,feature_embedding,lag(feature_embedding,1,null) over (partition by id) as feature_embedding_lag FROM dwd_content_detail）时会报错：
org.jkiss.dbeaver.model.sql.DBSQLException: SQL 错误 [1105] [HY000]: errCode = 2, detailMessage = No matching function with signature: lag(ARRAY, BIGINT, NULL_TYPE).
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:614)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$2(SQLQueryJob.java:505)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:524)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:976)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:4133)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:190)
What You Expected?
execute normally
How to Reproduce?
execute sql：
SELECT id,feature_embedding,lag(feature_embedding,1,null) over (partition by id) as feature_embedding_lag FROM dwd_content_detail
Anything Else?
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Table structure and query statements?

这是一封自动回复邮件。已经收到您的来信，我会尽快回复。谢谢！
                                                              ---万大友

Table structure and query statements?

CREATE TABLE IF NOT EXISTS dwd_content_detail
(
id varchar(80),
feature_embedding array<array>
)
UNIQUE  KEY(id)
DISTRIBUTED BY HASH(id) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"enable_unique_key_merge_on_write" = "true",
"light_schema_change" = "true",
"store_row_column" = "true",
"compression"="zstd"
);
SELECT id,feature_embedding,lag(feature_embedding,1,null) over (partition by id) as feature_embedding_lag FROM dwd_content_detail


False positive from devicelogin

Description
False authentication positive reported via devicelogin from az cli 2.10.
I run az login, copy the code into aka.ms/devicelogin, enter my account details and receive a successful authentication report, however I cannot execute any az commands, as i'm told to run 'az login' again.
I installed Ubuntu 17.04 to see if a fresh OS and different distribution makes a difference, azure-cli 2, and also receive the same issue.
Environment summary
Install Method: curl -L https://aka.ms/InstallAzureCli | bash
CLI Version: 2.10
OS Version: RHEL 7.3 & Ubuntu 17.04
Shell Type: BASH 4.2.46

Does your account have subscriptions? if yes, maybe you can share the verbose log (--debug) and let me take a look?

Emailed you the debug log. Let me know if I can provide further info. Not being able to use Azure CLI is a major issue for me. Thx

Thanks. I have recommended you open an internal ticket to let AAD ESTS take a look.

I am closing this since you have an internal ticket opened


Fix for issue #478. Mtime of output of phony edges should be the latest mtime of its inputs

Fix for issue #478
Observed : Real edges depending on a phony edge will not be marked as dirty or rebuilt if the phony's (real) inputs are updated
Expected : An edge should always be rebuilt if its inputs or transitive inputs are newer than the output's mtime
Change : Node::mtime_ was overloaded, 0 represented "does not exist". This change disambiguates it by adding Node::exists_. Then to fix the observed behaviour, Node::UpdatePhonyMtime was added to update the mtime if the node does not exist.
I've added tests BuildTest.PhonyUseCase# GraphTest.PhonyDepsMtimes
Unit tests will also test for always-dirty behaviour if a phony rule has no inputs

Gonna close this in favor of #1021, the rebase.  (The comment history here will be readable via a link from there.)


Fork/Clone issues

Forked the repository and tried cloning locally:
billy:sdp billy$ git clone https://github.com/wbuchanan/college-going-stata.git
Cloning into 'college-going-stata'...
remote: Counting objects: 127, done.
remote: Compressing objects: 100% (98/98), done.
remote: Total 127 (delta 25), reused 127 (delta 25), pack-reused 0
Receiving objects: 100% (127/127), 3.17 MiB | 584.00 KiB/s, done.
Resolving deltas: 100% (25/25), done.
Error: unknown command "filter-process" for "git-lfs"
Run 'git-lfs --help' for usage.
fatal: The remote end hung up unexpectedly
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

It might make more sense to separate code/analysis from the data.  It will keep the repositories lighter and avoid community members who are new to Git/GitHub having to learn about the large file system storage capabilities that GitHub has to work with the code.  Maybe mirroring the data on a few different servers might be helpful (and a few functions could be used to grab those data).

Anyone who runs into the same issue will need to make sure they have git-lfs installed on their system.  I was able to use brew install git-lfs to install it on my machine followed by sudo git lfs install --system to install it for any user on my personal machine and that took care of the error above.