IndexError on printing token.orth_ / incorrect token printing token.lower_ ...but only in some cases

System specs: Linux Mint 16, 64-bit, Python 3.4, using Anaconda
stack trace:
Traceback (most recent call last):
  File "/blah/blah/test.py", line 324, in <module>
    induceFail()
  File "/blah/blah/test.py", line 319, in induceFail
    print(l.orth_)
  File "spacy/tokens.pyx", line 427, in spacy.tokens.Token.orth_.__get__ (spacy/tokens.cpp:8080)
  File "spacy/strings.pyx", line 71, in spacy.strings.StringStore.__getitem__ (spacy/strings.cpp:1631)
IndexError:<PHONE_NUMBER>

Alright, this is a weird one. It involves passing tokens around in different lists, and only certain configurations will reliably cause the error. This error cropped up in a dataset I am actually using, where I wanted to pass the token objects around but only in certain cases. I got around this error by begrudgingly just passing the string representation of the token object. Anyway, I've created a small set of code that reliably induces the error on my machine so you can hopefully debug it:
from spacy.en import English

def blah(toks):
    ''' Take the tokens from nlp(), append them to a list, return the list '''
    lst = []
    for tok in toks:
        lst.append(tok)
        # printing the tok.orth_ at this point works just fine
        print(tok.orth_)
    return lst

def induceFail():
    nlp = English()
    samples = ["a", "test blah wat okay"]
    lst = []
    for sample in samples:
        # Go through all the samples, call nlp() on each to get tokens,
        # pass those tokens to the blah() function, get a list back and put all results in another list
        lst.extend(blah(nlp(sample)))
    # go through the list of all tokens and try to print orth_
    for l in lst:
        # here is where the error is thrown
        print(l.orth_)

induceFail()

Now, if you replace samples with the following sample, it works just fine!
samples = ["will this break", "test blah wat okay"]

And note that printing other attributes like pos_, dep_, and lower_ work without causing an IndexError, BUT don't print the correct thing, which leads me to believe some funny pointer dereferencing bug is causing this (the address is incorrect for certain symbols or something). It seems to only throw an error on the orth_ attribute. For example, I changed to printing lower_ and got the following output:
a
test
blah
wat
okay
a
test
blah
neighbour       <-------------- Wait...what? Neighbour isn't in any of the samples...
okay

Notice neighbour? Where the heck did it get that?. This is in an entirely new Python process. I'm not doing any multiprocessing / multithreading. I'm not using multiple instances of the nlp() object. So somehow an address got messed up, it's getting into memory it's not supposed to, but it is a valid address and it prints what is there? I have no idea.
So then I ran it again. And this time, instead of "neighbour" it was "Milk".
I hope the error is reproducible on your end. I apologize if it isn't. I swear I'm not crazy, you might just need to change the values in the samples list.

This should be fixed in version 0.63. Are you on that version yet?
Before the latest version, Token was supposed to be a weak reference. The underlying data is a C array of structs. This data is owned by the Tokens object. Once the Tokens object passes out of scope, the underlying C data is due for collection. If you're still holding references to Token objects, and you access a property that's proxied to that C data, all bets are off.
The recent update tries to check whether any Token objects will outlive the Tokens object. If so, it gives them a copy of the C data. I didn't document any of this properly, and I'm sorry that this seems to have wasted some of your time. I hope to have less broken release processes soon.
But, aside from this: what you're doing is supposed to be a rare edge-case. My intention is for users to maintain a reference to the Tokens object, and access the Token objects through it. Is there a reason you want to create your own list? It might indicate a weakness in my API.

I am running this on version 0.63, sorry I didn't mention that. Strange that it works on your machine, but not mine. But now that I understand you have to keep a reference to the Tokens object, I can do what I wanted to earlier without the error. It might be good to look into this a bit more to make sure you really are preventing those tokens from losing the necessary data; I'm positive I'm on 0.63 (I even checked tokens.pxd and it has this change: "cdef int take_ownership_of_c_data(self) except -1"
The reason I was doing that was I have a large set of tweets, each of which I was turning into a Tokens object by doing nlp(tweet_text). I didn't know you needed to keep a reference to it so I was just passing it into a function that goes and does the analysis I want. In this case I was using the dependency parse to extract Subject, Verb, Object structures. In that function, I was finding those subject, verb, and object tokens and was trying to append them to another list that I return at the end. So down the line I wanted to examine the tokens that were returned. I figured it might be useful to have all of the information, like tok.pos_, tok.dep_, etc, which is why I was adding the token to a list. To work around it, I simply appended the tok.lower_ string representation. Which so far is fine, but maybe I'd want more info later.
Don't worry about the bugs, I understand this is the early stage of development. I do want to commend you on your work so far, though. The dependency parsing works pretty darn well, even on informal text from the internet. And it is by far the fastest and easiest to use in Python. I've tried TurboParser and TweeboParser and both of those are very difficult to work with in Python. It's so simple to traverse the tree using rights and lefts as well. Really well done!

It looks like this is a packaging problem: the version on pip is failing my tests, but master is working. I think I didn't rebuild the file properly when I released 0.63. Working on it.

Working on this. I think I have it replicating on my server, but not on my laptop. Memory errors like this are difficult, because tests can pass accidentally, depending on whether the memory was over-written.

Okay, try v0.65. This is working on my laptop, server, and on Travis.
Please watch out for, and report, memory leaks in the new implementation. There's a reference cycle between Tokens and Token. I've run the parser over lots of documents, but the problems might only arise when the Token objects are used in some non-trivial way.

It seems to be working on my end now with v0.65!

Great!


"Cairo must be installed to use the PNG backend" error after trying to run Vega.jl + Cairo.jl + Gadfly.jl

I just installed the new stable version of Julia (Version 0.4.0-rc1+7 (2015-09-10 20:01 UTC))
And I am trying to run Gadfly together with Vega to generate a figure of a word-cloud. I am using simple examples extracted from http://johnmyleswhite.github.io/Vega.jl/wordcloud.html and http://gadflyjl.org/
I am using Vega.jl to create the word-cloud:
    julia> Pkg.checkout("Vega")
    INFO: Checking out Vega master...
    INFO: Pulling Vega latest master...
    INFO: No packages to install, update or remove

    julia> using Vega

    julia> corpus = [
           "Julia is a high-level, bla bla bla....",
           "Julia programs are organized ble ble ble..."
           ];

    julia> wc = wordcloud(x = corpus);

And I am trying to use Cairo.jl + Gadfly.jl to plot the word-cloud into a PNG figure. I am doing the following:
    julia> Pkg.checkout("Cairo")
    INFO: Checking out Cairo master...
    INFO: Pulling Cairo latest master...
    INFO: No packages to install, update or remove

    julia> Pkg.checkout("Gadfly")
    INFO: Checking out Gadfly master...
    INFO: Pulling Gadfly latest master...
    INFO: No packages to install, update or remove

    julia> using Gadfly

    julia> draw(PNG("myplot.png", 4inch, 3inch), wc)
    ERROR: Cairo must be installed to use the PNG backend.
     in PNG at /home/cdesantana/.julia/v0.4/Compose/src/Compose.jl:137

Any idea about it? Is it a problem with my installation of Cairo or Gadfly? Is it a compatibility problem between Gadfly and Vega?
Thanks for any tip!
P.S.: I submitted the same issue in Gadfly github: https://github.com/dcjones/Gadfly.jl/issues/686

This is most certainly an issue with something other than Vega. The only thing required for Vega are the JavaScript libraries, there is no PNG exporting (though, it might be possible to do in the future natively in this package).
Given that wc is of type VegaVisualization when it goes into draw, there might be a method needed to be defined with Gadfly. While I've got nothing against Gadfly, I haven't given any thought on how to integrate with it either :)

If you come up with a way to save PNGs, please let me know.  I'm planning
to add Vega.jl as a backend in Plots.jl, and it will be important for me to
be able to save PNGs for the examples markdown page.  Hacks are accepted,
but I would prefer that it didn't require Gadfly or lots of other
dependencies.
On Fri, Sep 11, 2015 at 9:16 AM, Randy Zwitch<EMAIL_ADDRESS>wrote:

This is most certainly an issue with something other than Vega. The only
thing required for Vega are the JavaScript libraries, there is no PNG
exporting (though, it might be possible to do in the future natively in
this package).
Given that wc is of type VegaVisualization when it goes into draw, there
might be a method needed to be defined with Gadfly. While I've got nothing
against Gadfly, I haven't given any thought on how to integrate with it
either :)
—
Reply to this email directly or view it on GitHub
https://github.com/johnmyleswhite/Vega.jl/issues/39#issuecomment-139543277
.

Thanks for your responses.
I agree with @tbreloff, it would be great to be able to save things we do with Vega.jl into a figure, doesn't matter the format, actually (PNG, PDF, EPS, TIFF, ...). In my case, I would like to include a wordcloud in a latex manuscript, so EPS, PDF or PNG would be my preferences.
I will let you know If I find a way by myself.

Can one of you throw that up as an issue?

Done: https://github.com/johnmyleswhite/Vega.jl/issues/40

Closing, if you figure it out your approach is almost certainly a hack using Gadfly (and I mean that in the best way possible)


Place crop area on image on init

Hi, I'm switching from jCrop to Darkroom.js. I need to set the cropping area on init so that the user finds the cropping grid already placed on the image on load. In jCrop i used setSelect : [0, 0, x, y]. What is the Darkroom.js equivalent?
Thanks!

Hello. You can do this: https://github.com/MattKetmo/darkroomjs/blob/e7d119fb370fd30be9d64c57348f3aabdf6c01b4/demo/index.html#L161


Add colors to XLSX INTO-function

I found that if save Excel file with XML extension and then rename it to XLS, Excel and LibreOffice can read it with colors, etc. I am going to change XLS() into-function from HTML to XML format.

Done. Now it works for Excel and LibreOffice as well. This is a code sample:
var opts = {
  headers:true,
  column: {style:{Font:{Bold:"1"}}},
  rows: {1:{style:{Font:{Color:"#FF0077"}}}},
  cells: {1:{1:{
    style: {Font:{Color:"#00FFFF"}}
  }}}
};
alasql('SELECT * INTO XLSXML("restest280b.xls",?) FROM ?',[opts,data]);

See the working example in jsFiddle
I will describe in wiki how to use it.


Doesn't appear to work under macOS

macOS 10.13.3 High Sierra
Golang go1.10rc1
When I run the example app, it just gives me the zero-values for the struct, not the values from the export commands. I seem to remember I had similar problems when I used os.Getenv but I had hoped this package solved those problems.

Can you share some code that reproduces the issue? If os.Getenv doesn't work then it's pretty unlikely that envconfig will.

Thanks for the reply. Just the example script and code on the page https://github.com/kelseyhightower/envconfig will show the problem. I suspect this is a deep underlying problme with macOS, rather than any code here.

Can you eliminate variables here? I just got it to work on OS X 10.12.6 and go 1.9.2, you may not be able to try a different OS version but can you see if it works on your machine under go 1.9?
One other thing comes to mind: what shell do you use? It could be that the export syntax in the README doesn't get picked up by your shell, so the go program never sees them and prints out zero values.

I think your second suggestion is significant here. I'm using ZSH as my shell, and others have suggested this may be the problem. I guess I'll have to pester the ZSH people...!

I received this post in another forum. Seems to answer the question.
https://forum.golangbridge.org/u/NobbZ
tl;dr
source env.sh
go run main.go


No Title Bar Not Working

I should of been more specific. I have CSS to hide title bars for every window in GNOME. It for some reason does not work for this specific collection of spicetiy themes. I was just simply asking if this is some known issue, or am I going to be given a non-answer and have my thread closed immediately again.

Sorry, I didn't want to be rude, but from your previous informations I wasn't able to understand much (you could have reopened that issue again instead of this new one btw): all I read was like "I want to hide the title bar in Spotify, how can I do it?"
However I'm pretty confident that this problem is not related to the themes but to your Desktop Environment (I cannot directly test it) since the theme apply just to the window content while the title bar are handled by the OS.

If you look at bittersweet it should be pretty clear where the theme end and when the Desktop environment starts


feat: Added ability to enforce numeric map keys are encoded as numbers rather than strings

Closing in favor of #5


Invalid SERVER_NAME + url_for leads to AttributeError

In our environment we have some security scanners running which generate artificial HTTP requests. Since they are closed source and I can't generate these calls with other tools, I created the below example which starts at the flask level and assumes that an invalid server passed thru the levels.
from flask import Flask, url_for

app = Flask(__name__)

@app.errorhandler(400)
def do_400(e):
    url_for("foo")
    return "error", 400

res = app(
    {
        "ACTUAL_SERVER_PROTOCOL": "HTTP/1.1",
        "PATH_INFO": "/",
        "QUERY_STRING": "",
        "REQUEST_METHOD": "GET",
        "REQUEST_URI": "/",
        "SCRIPT_NAME": "",
        "SERVER_NAME": "foobar/..",
        "SERVER_PROTOCOL": "HTTP/1.1",
        "wsgi.url_scheme": "https",
    },
    lambda x, y: None,
)

Error
Traceback (most recent call last):
  File "work/venv/lib/python3.12/site-packages/werkzeug/routing/map.py", line 258, in bind
    server_name = server_name.encode("idna").decode("ascii")
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".pyenv/versions/3.12.0/lib/python3.12/encodings/idna.py", line 173, in encode
    raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long
encoding with 'idna' codec failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1458, in dispatch_request
    self.raise_routing_exception(req)
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1440, in raise_routing_exception
    raise request.routing_exception  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/ctx.py", line 316, in __init__
    self.url_adapter = app.create_url_adapter(self.request)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1883, in create_url_adapter
    return self.url_map.bind_to_environ(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/werkzeug/routing/map.py", line 371, in bind_to_environ
    return Map.bind(
           ^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/werkzeug/routing/map.py", line 260, in bind
    raise BadHost() from e
werkzeug.exceptions.BadHost: 400 Bad Request: The browser (or proxy) sent a request that this server could not understand.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 2190, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1486, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1341, in handle_user_exception
    return self.handle_http_exception(e)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1281, in handle_http_exception
    return self.ensure_sync(handler)(e)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "work/foo.py", line 9, in do_400
    url_for("foo")
  File "work/venv/lib/python3.12/site-packages/flask/helpers.py", line 225, in url_for
    return current_app.url_for(
           ^^^^^^^^^^^^^^^^^^^^
  File "work/venv/lib/python3.12/site-packages/flask/app.py", line 1686, in url_for
    rv = url_adapter.build(  # type: ignore[union-attr]
         ^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'build'

As you can see, the server_name passed to flask is invalid. This should normally not happen, but let's just assume that it might happen. In my case I use cheroot and it let's it thru.
Then the server name should be encoded with IDNA which fails and that is fine. However, then the custom error handler for 400 is called and in my case it contains a url_for call. This call then fails with an AttributeError because the request was never fully consumed by flask.
Even if this seems now a little bit artificial, I would like to ask, if flask could improve its handling here, I have the following ideas

Don't call custom error handlers when such an error occurs
Have a nicer exception than AttributeError so that the caller of url_for can handle it gracefully
Handle a None url_adapter in url_for

Environment:

Python version: 3.12.0
Flask version: 2.3.3
Werkzeug: 2.3.8
This error also happens with flask/werkzeug 3.0

I'm having a hard time evaluating this. None of the suggested fixes are really useful.
1 is not possible. 3 is not possible, except to "handle" it by raising an error, which is what happens when the app context (instead of request context) can't create an adapter. 2 is possible, at least to raise a clearer error message closer to where the problem occurs, as explained for 3, but it shouldn't be something the dev should consider handling.
There is a little known feature called app.url_build_error_handlers, which is a list of handlers to call when adapter.build() raises a BuildError. Right now, it's only called when adapter.build() fails with a BuildError, not for other errors or errors raised by Flask when wrapping that call. This could be extended so that the RuntimeError and ValueError that url_for raises in certain situations would be a BuildError instead and pass through the same handlers. By default, no handlers means that the error is re-raised.

Thanks for checking this. I know it's a weird one, with not a clear path forward.
I like the idea to register a url build error handler 👍

Hi, I now found a reproducible example by just providing an invalid Host header value. I also reported it to cheroot: https://github.com/cherrypy/cheroot/issues/646

After thinking about this more, I'm going to close it for now. It seems to be mainly a server configuration issue rather than an issue with Flask. I don't think spending time resurrecting/changing an unused feature in Flask is really worth it for this.

Anyways, thanks for the time analysis this. I hugkly appreciate it!


Sometimes clicking on the map doesn't work, or takes a long time to give visual feedback.

Hey @hellais, I definitely remember this being an issue occasionally, but I'm having a bit of an issue reproducing it. I added some log + timestamps to the printout, and have been trying to keep an eye on when this happens. Let me know if you're noticing where abouts in the control flow it does thanks to the timers. Maybe it's another DOM load delay, so maybe we need to be a bit more liberal with in-view?

I am also having problems reproducing it in an accurate way. What I noticed is that when it happens, as soon as I open the developer console the map registers the click event and performs the redirect, so I am not really sure how is best to debug it.

It seems like the issue is related to the $location.path() statement not being executed properly. This could perhaps be a fix to the problem: http://stackoverflow.com/a/11932283.

Oh so maybe it's because it's outside of the regular angular processing and it just waits till the next render? I'll investigate.

I've added $scope.$apply to the click function in c7814ea.

I believe this has now been resolved since I haven't seen it anymore in recent versions of it.


Can't find module with extension .es6

VSCode Version: 1.2.1
OS Version: Windows 8.1

Some projects I work on use .es6 extension for JavaScript files that import/export using ES6 module syntax.
While webstorm and webpack seem to have no issues with this setup, VSCode gives a red squiggly saying [js] Cannot find module './filename.es6'.
Is there some way to get VS Code to find modules imported that do not have a .js extension? We would like to use .jsx in a similar fashion.
I have this for a .jsconfig:
    {
        "compilerOptions": {
            "target": "ES6",
            "module": "commonjs"
        },
        "files": [
            "**/*.jsx",
            "**/*.es6",
            "**/*.js"
        ]
    }

...and tried adding .es6 under user settings for VSCode, but suspect I did it wrong or that doesn't solve the issue.
Also opened StackOverflow here:
https://stackoverflow.com/questions/38046039/vs-code-es6-extensions-cant-find-module

@aikeru jsx is support but es6 to my knowledge not. Moving to TS team for further clarification.

TS team: for 2.0 I saw that you support arbitrary file extensions by use setting the kind on open. Would there be more we need to do to make this work.

This issue was moved to Microsoft/TypeScript#9551

@dbaeumer   I appreciate the quick response and clarification. If I just rename the files to ".js" extension, everything works in VS Code. It's only the difference of extension that seems to matter. I'd like to switch, but you know how it is... :)
Does this further confirm that it is, indeed, a TypeScript issue?

It is a TS issue. Especially when it comes to resolving file. Consider the case weher eyou have a JS file A which import a JS file B. The import statement usually doesn't include the file extension. So the TS/JS server (yes the JS support is power by the TS language server :-)) needs to know the list of valid file extensions that could contain JS code.

@dbaeumer well, the issue has been closed on the TS side as "works as intended".
This seems to be saying that for all the Microsoft tooling that depends on TS for module intellisense, you must either use one of the 4 supported extensions or it wont work. If true, it also means VS Code isn't viable for codebase(s) that I work on day-to-day (even if the syntax highlighter has no problem with it). :(
I really hope this isn't the case, since plenty of the community supports configurable extensions (especially ".es6", even if it's not the best extension to use).
Other competing tools (like Webstorm) work just fine with this setup.
I really hope this is reconsidered!

Sadly, this still seems to be an issue in VS Code (January 2017 v1.9) and the issue on the TypeScript repo seems to have been closed without taking this use case into account https://github.com/Microsoft/TypeScript/issues/10939
This is one of the things driving teams I work with away from VS Code. Regardless of whether it's a good idea, the reality is asking teams to move away from .ES6 extension (and virtually every editor/IDE has the ability to configure a custom extension for a given file type) is a non-starter just for the sake of supporting a specific editor/tool.

@mjbvz @waderyan can you follow up.

+1


some s3d code cause gpu stall

This code :
#if mobile
	fragmentShaderSource = "precision highp float;\n" + fragmentShaderSource;
#end

is inducing stall (and cannot work) on low end mobile gpu (SGT and other powervr).
There should be
#ifndef GL_FRAGMENT_PRECISION_HIGH
precision mediump float;
#else
precision highp float;
#end
``` in its place.

in S3DView.hx:122

precision high is not always available :)

Thanks !

Apéro !


forge script fails to verify on mumbai

Component
Forge
Have you ensured that all of these are up to date?

[X] Foundry
[ ] Foundryup

What version of Foundry are you on?
No response
What command(s) is the bug in?
forge script
Operating System
Linux
Describe the bug
as seen here in CI https://github.com/rainprotocol/rain.orderbook/actions/runs/5291795497/jobs/9577824407
forge script deploys a contract but fails to verify it
We haven't found any matching bytecode for the following contracts: [0xe80e098a8379eee4c05100e36088fd13e9e7b9b3].

This may occur when resuming a verification, but the underlying source code or compiler version has changed.

This seems strange to me as the code should be built and deployed as-is by CI without any modifications or "resuming" a verification.

Ref https://github.com/foundry-rs/foundry/issues/5168

it verifies now strangely even with the same error https://github.com/rainprotocol/rain.orderbook/actions/runs/5291881682/jobs/9578030758?pr=1

Closing, we merged #5167 and will be available in the next nightly


add session and credential management

Implement SessionSevice() and CredentialService
Implement PasswordAbuseDetection
setCredential() now will both set and verify the credential from token
Use HMAC256 for JWT encrpytion, can be modified to unsymmetric version later
Secret for HMAC256 now is hard-coded in KeyService
Reconstruct previous code to ease the session management
Modify .scripts/ to make it compatible with the current implementation

Since the code is written before the time that the last PR had been approved, we don't have the chance to rebase upon the release branch. Can we still rebase and delete history commits now?


Error: Using Batch processing with a Pydantic models doesn't folllow under the main idea of batch processing

Expected Behaviour
Lambda shouldn't failure if there is at least one successful record and return {batchItemFailures: [...] }
in the example the result of the lambda should be like this:
{    "batchItemFailures": [          {             "itemIdentifier": "messageId-2"         }     ] }
Current Behaviour
Code throw exception without processing all batches
Traceback (most recent call last):   File "C:\Users\Crumpet\Desktop\AWS\powertools\model_test.py", line 90, in <module>     lambda_handler(event,{})   File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\tracing\tracer.py", line 305, in decorate     response = lambda_handler(event, context, **kwargs)   File "C:\Users\Crumpet\Desktop\AWS\powertools\model_test.py", line 42, in lambda_handler     processed_messages: List[Union[SuccessResponse, FailureResponse]] = processor.process()   File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 97, in process     return [self._process_record(record) for record in self.records]   File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 97, in <listcomp>     return [self._process_record(record) for record in self.records]   File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 474, in _process_record     data = self._to_batch_type(record=record, event_type=self.event_type, model=self.model)   File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 347, in _to_batch_type     return model.parse_obj(record)   File "pydantic\main.py", line 526, in pydantic.main.BaseModel.parse_obj   File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__ pydantic.error_wrappers.ValidationError: 1 validation error for OrderSqsRecord body -> item   value is not a valid dict (type=type_error.dict)
Code snippet
import json

from typing import Any, List, Literal, Union
from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.utilities.batch import BatchProcessor, EventType, batch_processor
from aws_lambda_powertools.utilities.parser.models import SqsRecordModel
from aws_lambda_powertools.utilities.data_classes.sqs_event import SQSRecord
from aws_lambda_powertools.utilities.batch import (BatchProcessor,
                                                   EventType,
                                                   FailureResponse,
                                                   SuccessResponse,
                                                   batch_processor)

from aws_lambda_powertools.utilities.parser import BaseModel, validator

class Order(BaseModel):
    item: dict

class OrderSqsRecord(SqsRecordModel):
    body: Order

    # auto transform json string
    # so Pydantic can auto-initialize nested Order model
    @validator("body", pre=True)
    def transform_body_to_dict(cls, value: str):
        return json.loads(value)

processor = BatchProcessor(event_type=EventType.SQS, model=OrderSqsRecord)
tracer = Tracer()
logger = Logger()

@tracer.capture_method
def record_handler(record: OrderSqsRecord):
    return record.body.item

@tracer.capture_lambda_handler
def lambda_handler(event, context):
    batch = event["Records"]
    with processor(records=batch, handler=record_handler):
        processed_messages: List[Union[SuccessResponse, FailureResponse]] = processor.process()

    for message in processed_messages:
        status: Union[Literal["success"], Literal["fail"]] = message[0]
        result: Any = message[1]
        record: SQSRecord = message[2]

    logger.info(processor.response())
    return processor.response()

if __name__ == "__main__":
    event = {
        "Records": [
            {
            "messageId": "messageId-1",
            "receiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
            "body": '{"item": 1}',
            "attributes": {
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1545082649183",
                "SenderId": "1",
                "ApproximateFirstReceiveTimestamp": "1545082649185"
            },
            "messageAttributes": {},
            "md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
            "awsRegion": "us-east-2"
            },
{
            "messageId": "messageId-2",
            "receiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
            "body": 'Hi',
            "attributes": {
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1545082649183",
                "SenderId": "AIDAIENQZJOLO23YVJ4VO",
                "ApproximateFirstReceiveTimestamp": "1545082649185"
            },
            "messageAttributes": {},
            "md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
            "awsRegion": "us-east-2"
            }
        ]
    }
    lambda_handler(event,{})

Possible Solution
I think it can be easily fixed if we add code for bring to the model under try section (https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/aws_lambda_powertools/utilities/batch/base.py#L474)
But perhaps there are some reasons because of which they decided not to do so. Please share your thoughts
Steps to Reproduce
You can try code snippet. also you can try any test case where you have more than two records in one batch, and one record doesn't follow under your model
AWS Lambda Powertools for Python version
latest
AWS Lambda function runtime
3.9
Packaging format used
PyPi
Debugging logs
Traceback (most recent call last):
  File "C:\Users\Crumpet\Desktop\AWS\powertools\model_test.py", line 90, in <module>
    lambda_handler(event,{})
  File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\tracing\tracer.py", line 305, in decorate
    response = lambda_handler(event, context, **kwargs)
  File "C:\Users\Crumpet\Desktop\AWS\powertools\model_test.py", line 42, in lambda_handler
    processed_messages: List[Union[SuccessResponse, FailureResponse]] = processor.process()
  File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 97, in process
    return [self._process_record(record) for record in self.records]
  File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 97, in <listcomp>
    return [self._process_record(record) for record in self.records]
  File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 474, in _process_record
    data = self._to_batch_type(record=record, event_type=self.event_type, model=self.model)
  File "C:\Users\Crumpet\AppData\Local\Programs\Python39\lib\site-packages\aws_lambda_powertools\utilities\batch\base.py", line 347, in _to_batch_type
    return model.parse_obj(record)
  File "pydantic\main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for OrderSqsRecord
body -> item
  value is not a valid dict (type=type_error.dict)

@LuckIlNe thank you so much for this super detailed bug report!
There was a reason before we added Pydantic support, so we should definitely push that line within the try block as a fix.
Feel free to make a PR and we can review, amend any necessary details, and merge it tomorrow morning (CET tz).
I'm working on a new Batch improved experience tomorrow and can prioritise this bugfix if you're tight on bandwidth.
Thank you!

Notes for maintainers:
Paid additional tech debts as part of this PR

[x] SQS, DynamoDB Streams, and Kinesis Data Streams now support the use of Json[Model] without mypy erroring out: https://github.com/awslabs/aws-lambda-powertools-python/pull/2099/files#diff-24525fde52e7050cebd8041038fb1ca36688c0f14e16c2f3960fa178c0e1d63fR20
[x] Moved all duplicate models (SQS/DDB/Kinesis) into a sample_models.py  for functional tests in batch: https://github.com/awslabs/aws-lambda-powertools-python/pull/2099/files#diff-24525fde52e7050cebd8041038fb1ca36688c0f14e16c2f3960fa178c0e1d63fR1
[x] Moved all duplicate record handlers for parser as fixtures (ideally these will be functions themselves later not fixtures to easily trace it)

Congrats on your first contribution @LuckIlNe we appreciate a lot! I'm gonna finish the newly improved batch experience to cut boilerplate, along with doc fixes, and will merge this today.
This is the new concise experience with no need for a decorator + additional return.


Add Python3.8 to TravisCI and no Python27

TL;DR
Add Python3.8 to TravisCI and stop supporting Python27.
Type

[x] Bug Fix
[ ] Feature
[ ] Plugin

Are all requirements met?

[x] Code completed
[ ] Smoke tested
[ ] Unit tests added
[ ] Code documentation added
[ ] Any pending items have an associated Issue

Complete description
Add Python 3.8 as a job in TravisCI.
Tracking Issue
https://github.com/lyft/flyte/issues/534
Follow-up issue
N/A

Hmm, it turns out the last 2.x release of pyspark doesn't support Python 3.8. :( Any plan to upgrade pyspark to 3.x?

@akhurana001 ?

3.8

Yes, at some point, we would remove support for 2.4.x . I have https://github.com/lyft/flyte/issues/483 to track changing the default to 3.0 , targeting end of Q4 for that.


Fix header being overriden in auth

I noticed that auth calls like updateUser would fail due to a wrong JWT token. The cause for this was, that the execute method would override the Authentication header to the anon key. I changed the call, such that the execute method does not override existing headers.

Hi @leoMehlig I don't see this issue happening on my tests, do you have a reproducible example?

Does it not override the session.accessToken with the API key for you?

Does it not override the session.accessToken with the API key for you?

It doesn't, I'm verifying with the following test: https://github.com/supabase/supabase-swift/blob/main/Tests/IntegrationTests/AuthClientIntegrationTests.swift#L139

Does it not override the session.accessToken with the API key for you?

It doesn't, I'm verifying with the following test: https://github.com/supabase/supabase-swift/blob/main/Tests/IntegrationTests/AuthClientIntegrationTests.swift#L139

I think the difference in the test is, that it does not have a Authorization header set in the text, but in production this is set to the apiKey.


Some formatting in the readme file

I am sorry to bother you but I'll be honest I am doing this for the hacktoberfest and I''ll be very greatful if you''ll accept my pull
request , thank you so much .

Thanks for your honesty, but we do not appreciate PRs that do not make useful changes. Even less if they introduce mistakes (space before the ,). So, this is [invalid] - please make meaningful contributions that don't contradict the Hacktoberfest spirit.

Thanks for your honesty, but we do not appreciate PRs that do not make useful changes. Even less if they introduce mistakes (space before the ,). So, this is [invalid] - please make meaningful contributions that don't contradict the Hacktoberfest spirit.

Ok , I understand and I am sorry for bothering you .

But really. There are so many project who are looking for someone to help writing/improving documentation. Pick something, and make a decent contribution. It doesn't take that long and you don't have to feel bad for cheating.


Ugly formatting of lambdas in function arguments

This diff was created with rustfmt 0.9.0.
-                    parent.keys().iter().filter_map(|(k, v)| if delegation.key_ids().contains(k) { Some(v) } else { None }),
+                    parent.keys().iter().filter_map(
+                        |(k, v)| if delegation
+                            .key_ids()
+                            .contains(k)
+                        {
+                            Some(v)
+                        } else {
+                            None
+                        },
+                    ),

I would have expected it to look like this
-                    parent.keys().iter().filter_map(|(k, v)| if delegation.key_ids().contains(k) { Some(v) } else { None }),
+                    parent.keys().iter().filter_map(|(k, v)| {
+                        if delegation
+                            .key_ids()
+                            .contains(k)
+                        {
+                            Some(v)
+                        } else {
+                            None
+                        },
+                    }),

Dropping the |(k, v)| argument down a line does not seem like the appropriate place and seems inconsistent with what Rust or even other languages use as convention.
(sorry for the excessive whitespace)

I believe this is fixed on master/rustfmt-nightly

Wunderbar :)

Closing since this is fixed on master/rustfmt-nightly.


Add Hailfire by Rukkus

Song Name
Hailfire
Artist Name
Rukkus
Source
Youtube
Youtube Link
w67yHgaqn4g
Direct File Link
No response
Song ID
62885
Start Offset [ms]
0
Extra
Created from Auto Nong.
Created for level: 70371020
Downloaded: Yes

.


Add channelcount information on audio tracks

For get the channels count, only it's necessary read the value parametrer on AudioChannelConfiguration node.
Two examples:

2 channels

<AdaptationSet id="11" contentType="audio" lang="en" subsegmentAlignment="true">
      <Representation id="14" bandwidth="132318" codecs="mp4a.40.2" mimeType="audio/mp4" audioSamplingRate="48000">
        <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
        <BaseURL>a-eng-0128k-aac.mp4</BaseURL>
        <SegmentBase indexRange="745-1844" timescale="48000">
          <Initialization range="0-744"/>
        </SegmentBase>
      </Representation>
    </AdaptationSet>

6 channels

<AdaptationSet
 mimeType="audio/mp4"
 codecs="dtse"
 lang="en"
 audioSamplingRate="48000"
 startWithSAP="1"
 subsegmentStartsWithSAP="1" >
 <AudioChannelConfiguration
 schemeIdUri="urn:dts:dash:audio_channel_configuration:2012"
 value="6"/>
 <ContentComponent id="100" contentType="audio"/>
 <Representation id="a1" bandwidth="192000">
 <BaseURL>dtse_192_dash.mp4</BaseURL>
 <SegmentBase indexRangeExact="true" indexRange="852-920"/>
 </Representation>
 <Representation id="a2" bandwidth="256000">
 <BaseURL>dtse_256_dash.mp4</BaseURL>
 <SegmentBase indexRangeExact="true" indexRange="853-921"/>
 </Representation>
 <Representation id="a3" bandwidth="384000">
 <BaseURL>dtse_384_dash.mp4</BaseURL>
 <SegmentBase indexRangeExact="true" indexRange="852-920"/>
 </Representation>
 <Representation id="a4" bandwidth="510000">
 <BaseURL>dtse_510_dash.mp4</BaseURL>
 <SegmentBase indexRangeExact="true" indexRange="853-921"/>
 </Representation>
 </AdaptationSet>

We would expect something like this for audio track:
{active: true, bandwidth: 132318, channelcount: 2, hasOutputRestrictions: false, height: null, id: 20, kind: null, language: "en", type: "audio", width: null}

Sounds fairly easy.  We'll see if we can fit it into v2.0.0.  One change, though:  I'd prefer the member to be named "channels" instead of "channelcount".  Any objections?

Go!! channels name is perfect

Actually, now that we've looked into it, this is not so clear.  The DASH spec has some very confusing text about the meaning of the value attribute, and we can't convince ourselves that this is supposed to be the number of channels.  The spec text around this also refers to multiple other spec documents, one of which we don't currently have access to.
Because of all this, I am going to defer this enhancement.  We can look again after v2.0.0 is complete.  Sorry!

@joeyparrish , do you have any update about this?

Sorry, I do not have an update.  I need to track down the relevant spec text again, as I have lost my notes on the topic.  This time, I will leave my notes on github.

A while ago, I filed Dash-Industry-Forum/DASH-IF-IOP#124 for clarification.  The DASH-IF have updated their site to be more clear.  However, we still don't have access to the ISO/IEC 23003-3 spec required.

ExoPlayer is using the value attribute for audioChannel: https://github.com/google/ExoPlayer/blob/release-v2/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java#L708-L717
And a pull request of DolbyLaboratories (https://github.com/google/ExoPlayer/pull/3085) is using the same (https://github.com/DolbyLaboratories/ExoPlayer/blob/a9062ff28378734c0bab7086a6c7664554c37a79/library/dash/src/main/java/com/google/android/exoplayer2/source/dash/manifest/DashManifestParser.java#L689-L700)

@avelad, the interpretation of value depends on the scheme ID.  I will be publishing a more detailed plan soon.


Redress - Buyer Guidelines

Redress
Most buyer or company concerns can be resolved by use of our Tiki support site at https://support.mytiki.com/. If Tiki is unable to resolve your concerns and a dispute remains between you and a user or you and Tiki this Section explains how the parties have agreed to resolve it.
A.	Must Arbitrate All Claims Except Intellectual Property, Unauthorized Use, Piracy, or Theft
YOU AND THE USER AGREE TO RESOLVE ALL DISPUTES AND CLAIMS BETWEEN EACH OTHER IN INDIVIDUAL BINDING ARBITRATION.
However, this does not apply to the following types of claims or disputes, which you or Tiki may bring in any court with jurisdiction: (i) claims of infringement or other misuses of intellectual property rights, including such claims seeking injunctive relief; and (ii) claims related to or arising from any alleged unauthorized use, piracy, or theft.
This does not prevent you from bringing your dispute to the attention of any federal, state, or local government agencies that can, if the law allows, seek relief from us for you.
An arbitration is a proceeding before a neutral arbitrator, instead of before a judge or jury. Arbitration is less formal than a lawsuit in court, and provides more limited discovery. It follows different rules than court proceedings, and is subject to very limited review by courts. The arbitrator will issue a written decision and provide a statement of reasons if requested by either party. YOU UNDERSTAND THAT YOU AND THE USER(s) ARE GIVING UP THE RIGHT TO SUE IN COURT AND TO HAVE A TRIAL BEFORE A JUDGE OR JURY.
B. Try to Resolve Dispute Informally First
You and the User(s) agree to make reasonable, good-faith efforts to informally resolve any dispute before initiating arbitration. A party who intends to seek arbitration must first send the other a written notice that describes the nature and basis of the claim or dispute and sets forth the relief sought. If you and the User(s) do not reach an agreement to resolve that claim or dispute within thirty (30) calendar days after the notice is received, you or the User(s) may commence an arbitration. Written notice to Tiki must be sent via postal mail to: ATTN: Arbitration Notice, Tiki Corporation, 1814 Hayes St, Nashville, TN 37203.
C. Arbitration Rules and Fees
The U.S. Federal Arbitration Act applies to this section as far as your country’s laws permit. The arbitration will be governed by the Consumer Arbitration Rules (or the Commercial Arbitration Rules, if the Consumer Arbitration rules are inapplicable) of the American Arbitration Association ("AAA") as modified by this Agreement. Rules are available at http://www.adr.org. The arbitrator is bound by the terms of this Agreement.
The AAA will administer the arbitration. Outside the U.S., Tiki will recommend a neutral arbitration provider that uses these or similar rules. It may be conducted through the submission of documents, by phone, or in person in the country where you live or at another mutually agreed location.
If you seek $10,000 or less, Tiki agrees to promptly reimburse your filing fee and your share if any of AAA’s arbitration costs, including arbitrator compensation, unless the arbitrator determines your claims are frivolous or were filed for harassment. Tiki agrees not to seek its attorneys’ fees or costs unless the arbitrator determines your claims are frivolous or were filed for harassment. If you seek more than $10,000 and the AAA Consumer Arbitration Rules do not apply, the AAA’s arbitration costs, including arbitrator compensation, will be split between you and Valve according to the AAA Commercial Arbitration Rules.
D.  THE ARBITRATOR MAY AWARD RELIEF ONLY IN FAVOR OF THE INDIVIDUAL PARTY SEEKING RELIEF AND ONLY TO THE EXTENT OF THAT PARTY’S INDIVIDUAL CLAIM. You and Valve also agree not to seek to combine any action or arbitration with any other action or arbitration without the consent of all parties to this Agreement and all other actions or arbitrations.
This Agreement does not permit class, collective, or representative arbitration. A court has exclusive authority to rule on any assertion that it makes.
E. What Happens if Part of Section 11 Is Found Illegal or Unenforceable
If any part of the above (Dispute Resolution/Binding Arbitration/Class Action Waiver) is found to be illegal or unenforceable, the rest will remain in effect (with an arbitration award issued before any court proceeding begins), except that if a finding of partial illegality or unenforceability would allow class, collective, or representative arbitration, all of the above will be unenforceable and the claim or dispute will be resolved in court.
F. Miscellaneous
Except as otherwise expressly set forth in this Agreement, in the event that any provision of this Agreement shall be held by an arbitrator, court, or another tribunal of competent jurisdiction to be illegal or unenforceable, such provision will be enforced to the maximum extent permissible and the remaining portions of this Agreement shall remain in full force and effect. This Agreement, including any Payment Terms, Rules of Use, and the TIKI Privacy Policy constitutes and contains the entire agreement between the parties with respect to the subject matter hereof and supersedes any prior oral or written agreements. You agree that this Agreement is not intended to confer and does not confer any rights or remedies upon any person other than the parties to this Agreement.
Tiki’s obligations are subject to existing laws and legal processes and Tiki may comply with law enforcement or regulatory requests or requirements notwithstanding any contrary term.
You agree to comply with all applicable import/export laws and regulations. You agree not to export the content or data or allow the use of your Account by individuals of any terrorist-supporting countries to which encryption exports are at the time of exportation restricted by the U.S. Bureau of Export Administration. You represent and warrant that you are not located in, under the control of, or a national or resident of any such prohibited country.
The U.S. Federal Arbitration Act applies to this Section 11 as far as your country’s laws permit. The arbitration will be governed by the Consumer Arbitration Rules (or the Commercial Arbitration Rules, if the Consumer Arbitration Rules are inapplicable) of the American Arbitration Association ("AAA") as modified by this Agreement. Rules are available at http://www.adr.org. The arbitrator is bound by the terms of this Agreement.
The AAA will administer the arbitration. Outside the U.S., Tiki will select a neutral arbitration provider that uses these or similar rules. It may be conducted through the submission of documents, by phone, or in person in the country where you live or at another mutually agreed location.
If you seek $10,000 or less, Tiki agrees to promptly reimburse your filing fee and your share if any of AAA’s arbitration costs, including arbitrator compensation, unless the arbitrator determines your claims are frivolous or were filed for harassment. Tiki agrees not to seek its attorneys’ fees or costs unless the arbitrator determines your claims are frivolous or were filed for harassment. If you seek more than $10,000 and the AAA Consumer Arbitration Rules do not apply, the AAA’s arbitration costs, including arbitrator compensation, will be split between you and Tiki according to the AAA Commercial Arbitration Rules.
This Agreement does not permit class, collective, or representative arbitration. A court has exclusive authority to rule on any assertion that it does.

moved into #590


Add retry/backoff to get diagnostics data

The hm-config container currently crashes with the following error if it is launched before the hm-diag container has been fully initiated:
 gateway-config  2021-10-18 17:44:50,728 - [DEBUG] - gatewayconfig.gatewayconfig_app - (gatewayconfig_app.py).__init__ -- /opt/gatewayconfig/gatewayconfig_app.py:(43) - Read eth0 mac address 00:BD:27:EF:6D:E7 and wlan0 FF:FF:FF:FF:FF:FF
 gateway-config  Traceback (most recent call last):
 gateway-config    File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
 gateway-config      "__main__", mod_spec)
 gateway-config    File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
 gateway-config      exec(code, run_globals)
 gateway-config    File "gatewayconfig/__main__.py", line 74, in <module>
 gateway-config      main()
 gateway-config    File "gatewayconfig/__main__.py", line 29, in main
 gateway-config      start()
 gateway-config    File "gatewayconfig/__main__.py", line 63, in start
 gateway-config      FIRMWARE_VERSION
 gateway-config    File "/opt/gatewayconfig/gatewayconfig_app.py", line 47, in __init__
 gateway-config      pub_key = diagnostics_json['PK']
 gateway-config  KeyError: 'PK'
 gateway-config  Sentry is attempting to send 1 pending error messages
 gateway-config  Waiting up to 2 seconds
 gateway-config  Press Ctrl-C to quit
 gateway-config  Agent registered
[CHG] Controller 00:1A:7D:DA:71:13 Pairable: yes
 gateway-config  [bluetooth]# pairable off
 gateway-config  [bluetooth]# quit
method return time=1634579096.751134 sender=org.freedesktop.DBus -> destination=:1.52 serial=3 reply_serial=2
 gateway-config     array [
 gateway-config        string "org.freedesktop.DBus"
 gateway-config        string ":1.52"
 gateway-config        string "com.helium.Miner"
 gateway-config        string ":1.46"
 gateway-config     ]
 gateway-config  DBus is now accepting connections
 gateway-config  2021-10-18 17:44:59,378 - [DEBUG] - __main__ - (__main__.py).validate_env -- gatewayconfig/__main__.py:(50) - Starting with the following ENV:
 gateway-config<EMAIL_ADDRESS> gateway-config          BALENA_APP_NAME=HELIUM-TESTNET
 gateway-config          BALENA_DEVICE_UUID=204f46e27d5a44bc65fac817ebc77e5b
 gateway-config          VARIANT=NEBHNT-OUT1
 gateway-config          ETH0_MAC_ADDRESS_FILEPATH=/sys/class/net/eth0/address
 gateway-config          WLAN0_MAC_ADDRESS_FILEPATH=/sys/class/net/wlan0/address
 gateway-config          DIAGNOSTICS_JSON_URL=http://localhost:80?json=true
 gateway-config          ETHERNET_IS_ONLINE_FILEPATH=/sys/class/net/eth0/carrier
 gateway-config          FIRMWARE_VERSION=2<IP_ADDRESS>
 gateway-config
 gateway-config  2021-10-18 17:44:59,648 - [DEBUG] - gatewayconfig.gatewayconfig_app - (gatewayconfig_app.py).__init__ -- /opt/gatewayconfig/gatewayconfig_app.py:(43) - Read eth0 mac address 00:BD:27:EF:6D:E7 and wlan0 FF:FF:FF:FF:FF:FF
 gateway-config  Traceback (most recent call last):
 gateway-config    File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
 gateway-config      "__main__", mod_spec)
 gateway-config    File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
 gateway-config      exec(code, run_globals)
 gateway-config    File "gatewayconfig/__main__.py", line 74, in <module>
 gateway-config      main()
 gateway-config    File "gatewayconfig/__main__.py", line 29, in main
 gateway-config      start()
 gateway-config    File "gatewayconfig/__main__.py", line 63, in start
 gateway-config      FIRMWARE_VERSION
 gateway-config    File "/opt/gatewayconfig/gatewayconfig_app.py", line 47, in __init__
 gateway-config      pub_key = diagnostics_json['PK']
 gateway-config  KeyError: 'PK'

This is not a big deal, as it is recovering automatically. However, we should clean this up with some better retry logic in the near future.

The only thing I'm not sure of is where it says "Only array form is supported" for depends on. Does that mean we can't pass a healthcheck to it?
Also I'm pretty sure I read somewhere that the supervisor automatically kills and restarts a container if a healthcheck fails. But can't seem to find where I read that. But worth keeping in mind anyway

FYI @vpetersson https://github.com/NebraLtd/helium-miner-software/commit/73a759e4d32484f33ff8ef86c5ed1de14708501a#r58324969

Superseeded by https://github.com/NebraLtd/helium-miner-software/issues/177


[#3099] Fix chat messages collapsing when clicking on details element

Would love another set of eyes on this one. It fixes all the issues with using <details> elements in chat card descriptions, except when you collapse the details section the chat card doesn't reclaim the height properly:

I think to reclaim the space we need to clear the fixed height so the container can recalculate its scrollHeight appropriately, then set the container to the scrollHeight size if we want to show a CSS transition.
Looking at this code though, there are some things I would like to clean up:

We shouldn't consume click events on the details at all really. It should really only be collapsible from the summary (that's what has the cursor style implying interactivity). Then we don't need to check for links and things at the top, we can just let those have their normal click behaviour.
In order to handle things that might resize the details container, we should clear the fixed height once the CSS transition finishes. We only use fixed heights because CSS transitions (rather annoyingly) do not work on auto-sized containers. So we should use transitionend listeners with once: true to clear the fixed height when expanding is completed, and then re-add the fixed height just before collapsing so that the transition to 0 animates correctly. Such code is a bit messy, and there are other places we have to do this dance (like the accordion) so it might be nice to build some generic helpers that let us do this easily.

I'll raise an issue for the above, and if you don't mind, we could mark this issue as blocked and bump it to 3.1? I don't think this issue is so urgent that it needs to be fixed immediately, just a little annoying.

There is a somewhat inconvenient/non-intuitive, but solid work around using grid that would allow us to avoid having to manually set the height: keithjgrant.com/posts/2023/04/transitioning-to-height-auto/#with-grid
I've used it at work successfully and it should be supported on all of the browser versions we already require.

That's a great tip; if we can do it purely with CSS that would really be ideal.


methylation call-threshold on non-singleton events

Hi @jts
I'm running nanopolish on fast5 files to detect promoter methylation on a pretty dense CpG island. After nanopolish, I summarize the results with calculate_methylation_frequency.py
As far as I understand, the default call-threshold in calculate_methylation_frequency set to log_lik_ratio > 2.0 for a singleton event. This threshold is multiplied by the number of CpGs in a non-singleton event. A non-singleton event with 10 CpGs will have a threshold for being called or not called of 20.
Does it make sense to relax the threshold for non-singleton events?
Cheers
S

This is an open question still but I think it is fair to try to relax the threshold. You may also try to reduce the minimum distance between CpGs (—min-separation) to break up the groups as the default value of 10 is fairly conservative.
Jared

On Nov 26, 2021, at 9:02 AM, SkabbiVML @.***> wrote:

Hi @jts
I'm running nanopolish on fast5 files to detect promoter methylation on a pretty dense CpG island. After nanopolish, I summarize the results with calculate_methylation_frequency.py
As far as I understand, the default call-threshold in calculate_methylation_frequency set to log_lik_ratio > 2.0 for a singleton event. This threshold is multiplied by the number of CpGs in a non-singleton event. A non-singleton event with 10 CpGs will have a threshold for being called or not called of 20.
Does it make sense to relax the threshold for non-singleton events?
Cheers
S
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

Thanks so much for your reply @jts, I appreciate it. I actually asked you about setting the distance parameter between CpGs previously, it was a big help as well.

Ah my apologies, I didn't link the username between those two issues.


Type error for Extended TransactionClient

Hi,
first of all thanks for the library: I really needed something to upgrade my Redis cache from Prisma middleware to extensions.
Unfortunately I found a type error when trying to define the new type for Prisma.TransactionClient after the extension.
I am following the suggestion posted here https://github.com/prisma/prisma/issues/20738#issuecomment-1807917019 and it works with a basic extension that I created to prevent full table delete and update (query $allModels deleteMany/updateMany).
PrismaRedisExtension, though, makes it fail.
Here's the example code:
import { PrismaClient } from '@prisma/client';
import { CacheConfig, PrismaRedisExtension } from '@yxx4c/prisma-redis-extension';
import Redis from 'ioredis';

const extendPrismaClient = (client: PrismaClient) => {
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  let xclient: any = client;
  if (process.env.REDIS_URL) {
    const redis = new Redis(process.env.REDIS_URL);
    const auto = {
      ttl: 15 * 60, // Time-to-live for caching
    };

    const cache: CacheConfig = {
      ttl: 15 * 60, // Time-to-live for caching
      stale: 1 * 60, // Stale time for caching
      storage: {
        type: 'redis',
        options: {
          client: redis,
          invalidation: {referencesTTL: 60}, // Invalidation settings
          // log: Logger, // Logger function
        },
      }, // Storage configuration for async-cache-dedupe
    };

    xclient = client.$extends(PrismaRedisExtension({ auto, cache, redis }));
  }

  return xclient;
}

export type ExtendedPrismaClient = ReturnType<typeof extendPrismaClient>;
// https://github.com/prisma/prisma/issues/20738#issuecomment-1807917019
export type ExtendedTransactionClient = Parameters<Parameters<ExtendedPrismaClient["$transaction"]>[0]>[0];
class MyPrisma {
  private client: ExtendedPrismaClient;

  getClient() {
    if (!this.client) {
      const _client: PrismaClient = new PrismaClient();
      this.client = extendPrismaClient(_client);
    }
    return this.client;
  }
}

const prisma = new MyPrisma();

export { prisma };

The error:
TS2344: Type 'unknown' does not satisfy the constraint '(...args: any) => any'.
  Type '{}' provides no match for the signature '(...args: any): any'.
  | export type ExtendedPrismaClient = ReturnType<typeof extendPrismaClient>;
  | // https://github.com/prisma/prisma/issues/20738#issuecomment-1807917019
  >| export type ExtendedTransactionClient = Parameters<Parameters<ExtendedPrismaClient["$transaction"]>[0]>[0];
   |                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

My TypeScript is not good enough to understand the intricacies that I saw in Prisma definitions (and yours!), so even if I'd really like to help I don't know where to start.
Thanks
paolo

Hi @pmosconi,
Given the uncertainty about your specific use case and your message expressing uncertainty about TypeScript requirements, please find my suggestion below:
import {PrismaClient} from '@prisma/client';
import {CacheConfig, PrismaRedisExtension} from '@yxx4c/prisma-redis-extension';
import Redis from 'ioredis';

/**
 * Extends the Prisma Client with Redis caching functionality.
 * Assumes that REDIS_URL is always present in the environment variables.
 * @returns Extended Prisma Client with caching.
 */
export const extendPrismaClient = () => {
  // Initialize Prisma Client
  const client = new PrismaClient();

  // Initialize Redis with the provided REDIS_URL, asserting it is non-null
  const redis = new Redis(process.env.REDIS_URL!);

  // Auto configuration for caching time-to-live
  const auto = {
    ttl: 15 * 60, // Time-to-live for caching
  };

  // Cache configuration for async-cache-dedupe
  const cache: CacheConfig = {
    ttl: 15 * 60, // Time-to-live for caching
    stale: 1 * 60, // Stale time for caching
    storage: {
      type: 'redis',
      options: {
        client: redis,
        invalidation: {referencesTTL: 60}, // Invalidation settings
      },
    },
  };

  // Extend Prisma Client with Redis caching
  return client.$extends(PrismaRedisExtension({auto, cache, redis}));
};

If this doesn't meet your requirements, kindly provide detailed information on how you intend to utilize the extended Prisma and specify situations when you anticipate REDIS_URL might not be available.

Hi @pmosconi,
It seems like you're encountering an issue with your TypeScript types. The error you're seeing, Parameters<ExtendedPrismaClient['$transaction']>[0] being unknown, is because TypeScript cannot infer the type properly.
To resolve this error, you need to adjust the type of xclient in the extendPrismaClient function. Currently, it's being set as any, which defeats TypeScript's type checking. Instead, you should try to maintain type safety by explicitly defining the type of xclient based on the return type of client.$extends().
Here's what you can do:
const extendPrismaClient = (client: PrismaClient) => {
  let xclient: ExtendedPrismaClient = client; // Adjust the type here
  // Rest of your code remains the same...
};

By doing this, TypeScript will have a better understanding of the type of xclient, which should resolve the error you're encountering.
P.S. It's worth mentioning that combining extensions could lead to issues. You may want to review the Prisma documentation for more information on this.

Hi yxx4c,
I had intially overlooked the fact that in the end xclient was set to any, I've been able to fix things by updating Prisma to the latest available release, currently 5.9.1, and by a simple change:
const extendPrismaClient = (client: PrismaClient) => {
  let xclient = client; // let's assume the type is PrismaClient
  // Rest of the code remains the same...

  // @ts-expect-error - the type changes to ExtendedPrismaClient as we really want
  xclient = client.$extends(PrismaRedisExtension({ auto, cache, redis }));

// Rest of the code remains the same...
};

About combining extensions: I am well aware, and in this case there are no issues. It's worth mentioning that the two extensions are orthogonal: one is related to delete and update operations while the other is primarily concerned with caching read operations.


Bad data slows down profiler

De-noising data in general will help on overall performance by:

making the profiler work more efficient

improving accuracy

This requires that errors occurred for a given column are counted and the processing of that column abandoned when these trespass a given threshold:
For example, in data.gov. I observe multiple messages like:
WARN  preanalysis.PreAnalyzer - Error while parsing: For input string: "523986004252398600465239860072"
In any case, this requires a more in depth study of what other errors are causing trouble.

Did it, fixed it


Installation on Raspbian OS

ZestDB has a docker build for HypriotOS. Since Raspbian(the official OS) is currently supporting docker, is there any chance to have a docker build for it?

My understanding is we are going to use a 64bit OS on the Pi3. It should run fine though on Raspbian now that that Docker is supported.


Implement arrayMult function

There is currently a method called arrayMult that lives in /lib/arrayMult.js.
It is incomplete and needs to be fixed!
This method should not modify the original array.
It will ignore anything that is not a number.
It will take an array of numbers an return the result of all the items multiplied.
Ex:
arrayMult([1, 2, 3, 4]) // ---> 24
arrayMult([3, "test", 0, 4] // ---> 0
arrayMult([]) // --> 0
arrayMult(["pen", "pineapple", "apple", "pen"]) // --> 0

Acceptance Criteria:

Running yarn test arrayMult results in tests passing.
You have written a skeleton method for someone else to work on.
You have written tests surrounding your skeleton method.
Running yarn lint does not print any errors to the console!
Optional: write one or two more tests surrounding the method you are solving to account for potential edge cases.

Please include the skeleton of a new method + an accompanying test for someone else to work on at the time of creating a pull request! A pull request will most likely be denied if it does not contain a skeleton method for someone else to work on! For more information, please read the Contributing Guide.
Thank you so much for your contribution!

Requires  #550  to be merged first

#551 @BlakeGuilloud I would like to work on this


set max stream duration to the value specified in VS

No one has responded yet on this https://github.com/envoyproxy/envoy/issues/15530. So instead of keeping master broken for long time - I am setting MaxStreamDuration to the value specified in VS.
I will do a follow-up PR once a decision comes on that.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ X] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[X] Does not have any changes that may affect Istio users.

Thanks.


"redeem" event not working

client.on("redeem",(channel,user,type,tags)=>{
console.log("AA")
})
Actual behaviour:
no log
Expected behaviour:
log "AA"
Error log:
Insert your error log here

Server configuration

Operating system:W10
Node version (if applicable):
NPM version (if applicable):
tmi.js version:1.8.5

Redemptions only work with tmi.js if the reward requires user input. Twitch may also remove channel point redemptions from IRC at any point but I feel rewards with a message will stay. Try logging the incoming IRC and figure out if you're actually receiving any data when a reward is redeemed.
client.connect()
.then(() => {
    client.ws.addEventListener('message', ({ data }) => {
        console.log(data);
    });
});


slate will not be compatible with html-webpack-plugin 4.x

Problem
The html-webpack-include-chunks.js is using the htmlWebpackPluginAlterChunks hook which will be removed in the upcoming version of the plugin.
https://github.com/Shopify/slate/blob/d67f445ea4dda555cce836f58d78f23ec3b55b84/packages/slate-tools/tools/webpack/html-webpack-include-chunks.js#L17-L26
Maybe you could switch to htmlWebpackPluginBeforeHtmlProcessing.
Please take a look at https://github.com/jantimon/html-webpack-plugin/blob/webpack-4/lib/hooks.js#L29-L78 for the upcoming hook interface and let me know if this would still work for you.

@jantimon Thanks for letting us know!

I don't expect any further breaking changing to the current hook interface:

Could you please give the new events a try and let me know if they work for your case?
https://github.com/jantimon/html-webpack-plugin/tree/webpack-4#events
beforeAssetTagGeneration hook
    AsyncSeriesWaterfallHook<{
      assets: {
        publicPath: string,
        js: Array<{string}>,
        css: Array<{string}>,
        favicon?: string | undefined,
        manifest?: string | undefined
      },
      outputName: string,
      plugin: HtmlWebpackPlugin
    }>

alterAssetTags hook
    AsyncSeriesWaterfallHook<{
      assetTags: {
        scripts: Array<HtmlTagObject>,
        styles: Array<HtmlTagObject>,
        meta: Array<HtmlTagObject>,
      },
      outputName: string,
      plugin: HtmlWebpackPlugin
    }>

alterAssetTagGroups hook
    AsyncSeriesWaterfallHook<{
      headTags: Array<HtmlTagObject | HtmlTagObject>,
      bodyTags: Array<HtmlTagObject | HtmlTagObject>,
      outputName: string,
      plugin: HtmlWebpackPlugin
    }>

afterTemplateExecution hook
    AsyncSeriesWaterfallHook<{
      html: string,
      headTags: Array<HtmlTagObject | HtmlTagObject>,
      bodyTags: Array<HtmlTagObject | HtmlTagObject>,
      outputName: string,
      plugin: HtmlWebpackPlugin,
    }>

beforeEmit hook
    AsyncSeriesWaterfallHook<{
      html: string,
      outputName: string,
      plugin: HtmlWebpackPlugin,
    }>

afterEmit hook
    AsyncSeriesWaterfallHook<{
      outputName: string,
      plugin: HtmlWebpackPlugin
    }>


Configuring CATALINA_BASE

Describe the bug
I have multiple tomcat instances on the same server to simulate different environments (DEV, TEST, QA, ...).
To select the correct tomcat-server, I usually configure CATALINA_BASE to point to the correct directory.
This is done by having multiple run configurations (one for each environment), and configuring the following VM option:
-Dcatalina.base=D:/Path/to/my/alternative/catalineBase
This works with the default Tomcat functionality in IntelliJ Ultimate, but not with SmartTomcat - the option is ignored.
The tomcat server starts, but no application is deployed.
Intellij & SmartTomcat Version
IntelliJ IDEA 2020.3.3 (Ultimate Edition)
Build #IU-203.7717.56, built on March 15, 2021
Runtime version: 11.0.10+8-b1145.96 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: ParNew, ConcurrentMarkSweep
Memory: 1981M
Cores: 8
Non-Bundled Plugins: CheckStyle-IDEA, com.dubreuia, com.poratu.idea.plugins.tomcat
SmartTomcat version: 3.8.3

@jakobmaier I suggest you create multiple tomcat server and run/debug instance to support your requirement.

Latest Smart Tomcat now supports overriding catalina.base via the VM option:
-Dcatalina.base=D:/Path/to/my/alternative/catalineBase

Closing it.


[Gally]: master <- dev

Automatically created by Git-Ally

:tada: This PR is included in version 4.0.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Implement numpy's import_array and friends in Cython

The numpy macros import_array (and similarly import_umath) have a return statement in them, making them not really useful for Cython. The underscored functions _import_array and _import_umath are more useful. This PR implements import_array and import_umath as cdef functions in Cython instead of using the numpy macros.

Who would be using these?

Who would be using these?

They are used in a few places in SageMath. For more context, see https://trac.sagemath.org/ticket/21695

For some reason I thought we were automatically calling these when numpy was cimported. But, no, and as they're called manually, this should be good (better in fact).


Warn about MRI 1.9 super infinite recursion bug.

Closes #814.

Seems there are some specs that generate the warning missing an allow_warning

Seems there are some specs that generate the warning missing an allow_warning

There failures are in rspec-core's specs. This is surprising.  I'm going to dig into it now.

Seems there are some specs that generate the warning missing an allow_warning

Nope, it was a bug related to re-inclusion that I have now addressed.

And now it's green :).

LGTM


_stare_column_name is not carried over if we do a subset

print(df._stare_column_name)
stare_cover

print(df.iloc[0:2]._stare_column_name)
stare

seems to have been related to this effort with geopandas.
https://github.com/geopandas/geopandas/pull/60/files#
Fixed by adding a class "_metadata" variable holding the names of the instance variables that will be carried over during the pandas finalize() step of subsetting

magically resolved. Might have just come downstream


Notebook cells will not run ==> Error: cannot delete password (source: jupyter (extension))

Environment data

VS Code version: 1.56.2
Jupyter Extension version: v2021.6.832593372
Python Extension version: v2021.5.842923320
OS: Linux (Opensuse Tumbleweed)
Python and/or Anaconda version: 3.8.10
Type of virtual environment used: venv
Jupyter server running: Local

Expected behaviour
Expect basic cell to run through with no vscode errors. I have been attempting mainly with a cell that only has imports.
Actual behaviour
When I click to run any cell the only response is an error popup on vscode that reads: cannot delete password (source: jupyter (extension))

The output of the cell is shown here:

Steps to reproduce:

Open any .ipynb file in vscode (jupyter environment loads successfully and correctly as far as I can see)
Click the run button on the first cell (jupyter seems to recognize the venv kernel even though vscode says it's not started)
Error pops up immediately in the bottom right corner of vscode window

Logs

Output for Jupyter in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Jupyter)
Info 2021-05-19 21:31:52: Executing cell NotebookImport#3
Info 2021-05-19 21:31:52: Waiting for jupyter server and web panel ...
Error 2021-05-19 21:31:52: Exception attempting to start notebook:  Error: Cannot delete password
at E.$deletePassword (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1763:64048)
Info 2021-05-19 21:31:52: Exception executing cell NotebookImport#3:  Error: Cannot delete password
at E.$deletePassword (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1763:64048)
Info 2021-05-19 21:31:52: Finished executing cell NotebookImport#3
Error 2021-05-19 21:31:52: DataScience Error Error: Cannot delete password
at E.$deletePassword (file:///usr/share/code/resources/app/out/vs/workbench/workbench.desktop.main.js:1763:64048)
Info 2021-05-19 21:31:54: Trusting notebook...

xxx

Thanks for the bug.
It looks like your encrypted storage is corrupted and our attempts to cleanup after it are failing. You might be able workaround this by deleting your user settings (although this will reset all of your settings if you do)
On windows those settings are stored here:
C:\Users\<user name>\AppData\Roaming\Code\User\globalStorage
If you stop VS code and delete the files and directories under that location, the problem should go away.
This was likely caused by this fix here:
https://github.com/microsoft/vscode-jupyter/commit/54bc65d32fc2d767879d525a8c3fe452541a8205

Thanks for the feedback. This workaround does not appear to have worked.
The settings you referred to on my OS (openSUSE Tumbleweed) are apparently at this location:
/home//.config/Code/User/globalStorage
I delete all of the contents of that directory after vscode is closed. New contents appear when I open vscode again. The jupyter notebook cell reacts the same (with an error).
Could this be related to admin rights somehow? Any other ideas?

This should only go down this path if you had been using a remote jupyter server before. You could try deleting your workspace and user settings.json (well really just the settings prefixed with 'jupyter')

Perfect! Thanks for the tip.
Switched the jupyter.jupyterServerType value to "local" in the the workspace and the user settings and now the cells run as expected.
This is strange because I have not used a remote server yet just FYI.

Do you have another machine? There is a sync settings option that might have moved it over. That might even be the cause of the problem as the expected URI isn't in the storage.
Or possibly if your workspace settings came down from another user submitting them to source control.

I used to use a Windows machine but I haven't used sync settings since I switched over. I only enabled the sync settings as an attempted fix to this problem actually. Thought it would be useful in the future as well though.
I'm not worried about it at this point but if you need more info on this bug I'll provide it. Just happy to know it's being checked out and I can still work.

I don't have the specific validation on this (requires windows + corrupted storate), but resolved based on no incoming customer issues since then.


Improve wording for multi-process docs

Currently, there is no direct implication that device-to-device links are a requirement.
Though it is a very small change, it would have saved me a couple of hours in trying to leverage NCCL's ability to create rings over ethernet to run JAX.
I'm sure others will also appreciate this.

@googlebot I signed it!

This change LGTM, thanks for the contribution and sorry for the confusion!
However, it may be possible to use NCCL over ethernet with JAX -- my understanding is that JAX doesn't care if devices communicate over ethernet or dedicated links, as long as there's some kind of protocol that manages the communication and can be targeted by JAX's backend. I think the trick with NCCL is initializing it correctly. @hawkinsp can you say more?

@skye If that is the case, I would be very interested in having documentation covering this. I could help with that.
I also wonder if there is a way to retrigger the build of the documentation (which seems to have failed due to timeout?).

JAX does support cross-host (i.e., ethernet or infiniband) GPU communication via NCCL already. However, we don't advertise the feature yet or provide a public API for it.

The docs build was unrelatedly broken when you created your PR (not your fault!). It's fixed now though, and your change is live: https://jax.readthedocs.io/en/latest/multi_process.html


[Suggestion] Alternate methods for obtaining trophies

While trophies being rare drops from mobs works well for most hostile mobs, other mobs are quite unlikely to ever be seen as trophies, usually passive mobs that players aren't really incentivized to kill.
To alleviate this, there could possibly be other methods that trophies could be obtained, manageable by config. If possible, these could even be supported by datapacks, i.e. listing an event that could cause a trophy to spawn.
Config options could include whether alternate methods are enabled, and whether they should have a different chance for spawning trophies.
Some alternate methods to obtain trophies could include the following:

Breeding mobs
Taming mobs
Building mobs (Iron Golem)
Fishing (Cod, Salmon, Pufferfish, Tropical Fish)
Trading with mobs (Villagers, Piglins)
Other unique mob behaviors (i.e. saving beached whales in Alex's Mobs, feeding Tazmanian Devils from Alex's Mobs Rotten Flesh, harvesting ores from Quark's Toretoises, etc.)

While I agree that acquiring some trophies can be a bit of a problem, I'm honestly not sure whether I want to do something about it. I'd need a lot of custom behavior for entities that work in all kinds of different ways. There's trophies for over 150 entities already, and I'll probably be adding even more in the future. Things would quickly become an unmaintainable mess, as I'd have to spend a significant of time fixing things every time mods update their entities or when new Minecraft versions come out.
Even if I manage to keep things relatively simple, balancing many different methods for obtaining trophies such that they drop with more or less the same rate would still be very difficult and time-consuming.
In general I just think it's a bit beyond the scope of the mod.


Docs

Incremental changes, nothing life changing of course.

NOTE:

It's a good idea to open an issue first to discuss potential changes.
Please make sure that you are NOT opening a PR to fix a potential security vulnerability. Instead, please follow the Security guidelines to disclose the issue to us confidentially.

☕ Reasoning

🧢 Checklist

[ ] Documentation
[ ] Tests
[ ] Ready to be merged

🎫 Affected issues
Please scout and link issues that might be solved by this PR.
Fixes: INSERT_ISSUE_LINK_HERE
📌 Resources

Security guidelines
Contributing guidelines
Code of conduct
Contributing to Open Source

Made those "up and running" changes to index that I lost earlier in the day. I didn't see that there was a conflict on 3 or 4 lines before I tried to submit the PR, but I went ahead and removed the ts-error from my version.


fix for doyou89/RPBot#3

Added check for self.items being empty.

@deepio-oc, Thank you for your contribution.
Could you add some unit test to be able to check if your code operates well in your case?

@deepio-oc,
Could you add some unit tests to be able to check if your code operates well in your case?


WanMode not found

I've updated the livebox component, and now I've got this error :
2020-11-20 10:01:17 ERROR (MainThread) [homeassistant.components.sensor] Error while setting up livebox platform for sensor
Traceback (most recent call last):
File "/srv/homeassistant/lib/python3.9/site-packages/homeassistant/helpers/entity_platform.py", line 199, in _async_setup_platform
await asyncio.shield(task)
File "/var/www/house_bone/homeassistant/custom_components/livebox/sensor.py", line 17, in async_setup_entry
if "ETHERNET" not in nmc["WanMode"].upper():
KeyError: 'WanMode'

Fix 1.6.2


Increment package versions for servicebus releases

API change check
API changes are not detected in this pull request.


custom configuration for the rake tasks

I see in the README that you can provide custom configuration in spec_helper.rb, test_helper.rb, etc.
This affects the configuration when running tests and generating a report.
However, how do you provide configuration when running the knapsack rake tasks?
For example, I'm using rspec, and have a custom configuration to change the report_path.
This correctly saves the report where I want it to when I run rspec to generate the report, but how do I tell rake knapsack:rspec to use the alternate report location?

You can use KNAPSACK_REPORT_PATH
KNAPSACK_REPORT_PATH="knapsack_custom_report.json" CI_NODE_TOTAL=2 CI_NODE_INDEX=0 bundle exec rake knapsack:rspec

You may want to check https://knapsackpro.com with more advanced features like Queue Mode to get optimal test suite split.

Thanks. I see now that the environment variable is mentioned in the README; I just didn't see it.

Cool. I'm closing the issue then.
BTW I recommend seeing video how Knapsack Pro Queue Mode works to get optimal test suite split.
https://www.youtube.com/watch?v=pkdLzKlnlQg


detect mouse and keyboard

im running raylib in native mode..
When i start my program with mouse and keyboard plugged in at boot.. they work fine.
if i disconnect and reinsert them.. they do not come back..
also if i dont have a keyboard plugged in when i start my program they will not work.
how does raylib detect input devices?
thank you

is it on Raspberry Pi?

is it on Raspberry Pi?

yes..
at command prompt it detects the mouse and keyboard if plugged in after boot... or it removed and replugged in..

it seem what ever you have when you start the program works.. as long as you dont disconnect it

@gezzuzz inputs processing in RPI native is different than other platforms, actually, it has been reviewed several times... Current implementation uses dev events in separate threads to detect inputs...

There is a function in core.c.. InitEvdevInput
if i call it. the keyboard starts to work fine.
Since its not in raylib.h by default i can not access it.
i was able to make it work by removing Static
then i added it to raylib.h
RLAPI void InitEvdevInput(void);
Im not sure it this is the proper way.. But i would like to make InitEvdevInput public.
Is there is a reason not to make this function public?
i figured i could call this function on a timer so that i can redetect input devices..

Better yet i can check for changes to "/dev/input/" if any devices get added to removed it shows here.. then i call InitEvdevInput to reset everything.
            GetDirectoryFiles("/dev/input/",&fileCount);
            if(fileCount != lastDeviceCount){
                InitEvdevInput();
                lastDeviceCount = fileCount;
            }

@gezzuzz that's nice... but for the moment no plans to expose that function...

Closing this issue.


Storage of SignedBeaconBlock

The P2P specification lists validations before forwarding signed_beacon_block, which implies you will forward it, but doesn't have a requirement to.
Similarly, the request/response section implies nodes should record signed_beacon_block in order to respond with it, "if they have it".
However, in phase 0 fork choice on_block only signed_block.message, the BeaconBlock is stored.
Note: In the current dev branch, the definition of Store has been changed to store BeaconBlockHeader, instead of BeaconBlock, although the change has not yet been propagated to all functions.
The specification does talk about messages without parents or in the future being delayed, again implying storage.
Should there be an explicit mention, even if just something like "clients SHOULD keep a record of messages received, including signature", if not something formal like Store?

The requirement of the fork choice is only validated the blocks/headers so that's why the naive spec stores them there, whereas the network spec needs signed blocks to be passed around for others to validate.
There are statements like
Clients MUST support requesting blocks since the latest finalized epoch.

I suppose we can change this to "signed blocks" to be more explicit

That's probably fair enough; I was concentrating on the "if they have it and it exists in the range" exception, and interpreting the "MUST support requesting" as simply sure, I'll accept your request, but because I never stored any blocks I don't have them and so support your request by responding with nothing. (Before the signing root change, the Store contained full blocks, including signatures, so the clients would have the blocks.)

Changing it to explicit statement like "Clients MUST keep a record of signed blocks seen since the weak subjectivity period and MUST support requests for them up to the given head_block_root." would remove any ambiguity about the "if they have it" as an escape clause.
But not keeping blocks would probably be a silly thing to do anyway, so probably doesn't matter if you just want to close the issue.

The latter part of "Clients MUST keep a record of signed blocks seen since the weak subjectivity period and MUST support requests for them up to the given head_block_root." is not desirable (assuming you mean "serve" for "support") as it opens the way for bad implementations (or just greedy peers) to be able to continually request blocks from you and you be obliged to return them.

to be able to continually request blocks from you and you be obliged to return them

I should probably update the issue with the full text, so there is no confusion.
The current spec already says "MUST support requests", so that part hasn't changed.
The next line is "Clients MUST respond with at least one block, if they have it and it exists in the range. Clients MAY limit the number of blocks in the response."
Prior to the hash_signing_root change, the full messages were recorded in Store, so if a node had seen blocks, they would "have it" and must respond, with at least one block.
Since the change, only beacon_block is stored (not signed_beacon_block), and the upcoming dev changes cut this down to beacon_block_header.
Therefore, a client could throw away all blocks, and rely in the "if they have it" as an escape clause to respond with nothing, instead of a minimum of one block.
Adding in the "MUST keep a record of signed blocks" is simply re-instating the requirement to record them (as they are no longer in Store), which from the other discussions sounds like what the intention is.

I put up a pull request for consideration, especially seeing as @mcdee seems to be worried about the requirement that clients MUST store and respond with blocks, when I think that was what the intention is (and is what the spec required in 0.9.2 and earlier),

as it opens the way for bad implementations (or just greedy peers) to be able to continually request blocks from you and you be obliged to return them.

"Support" here means that you have the ability to serve. A client is never required to "serve" due to local bandwidth constraints, peer abuse, etc. A client that connects to a peer that won't serve them should drop and find better peers.

Prior to the hash_signing_root change, the full messages were recorded in Store, so if a node had seen blocks, they would "have it" and must respond, with at least one block.

The "Store" is just an example of how the functionality of the fork choice can be incredibly naively implemented. The "Store" is not prescriptive. The validation conditions on on_block, on_attestation, and the results of get_head are prescriptive.
I wouldn't get too hung up on the mechanics here in fork choice

Clients MUST support requesting blocks since the start of the weak subjectivity period and up to the given head_block_root.

This statement dictates that a client must have access to SignedBeaconBlocks since weak subjectivity. I'm not certain we need to state explicitly to store them.

Clarification merged, so closing issue.


Tests-alexrb-csl1160-structure

@gromakovsky By the way, a heads-up: the only remaining Arbitrary module in src is in src/Pos/Block/Arbitrary.hs, but this one cannot be moved into core because it depends on block modules that themselves cannot be moved outside of src.

By the way, a heads-up: the only remaining Arbitrary module in src is in src/Pos/Block/Arbitrary.hs, but this one cannot be moved into core because it depends on block modules that themselves cannot be moved outside of src.

Yes, it's fine to Pos.Block.Arbitrary where it is now, except that TxPayload and related should be moved to txp as I wrote above.

Pos.DHT.Arbitrary is probably a leftover.

@gromakovsky Done. As soon as CI passes you can merge this.


Support scanning multiple ranges

Feature Request
Scanning multiple ranges (in Coprocessor) is not efficient currently because for each range we need to rebuild the rocksdb iterator. This results in a strange case that scanning from a to z can be faster than scanning from a to c, from f to g, from x to z.
We can support scanning multiple (ordered) ranges in the mvcc scanner, by using the same rocksdb iterator. Each time for a new range, we can next() for several times and then seek() for it, just like what we did in current mvcc process.

If there are a lot of tombstone keys between ranges, reusing iterator may perform worse.

How about creating a iterator pool in memory and always find the nearest iterator?

@zhangjinpeng1987 PTAL

@breeswish Can you provide more details for what kind of SQL can send queries that scan multiple ranges in one Coprocessor query.

@zhangjinpeng1987 There are many, for example, select * from table where id not in (...).

I think it worth to reuse iterator in a coprocessor request.

@zhangjinpeng1987 Yes I agree. It's this issue's purpose. Also @ilovesoup has some demands of scanning multiple ranges.


Performance tuning by template literal

Well, tried using Array.join() function to beautify compiled code, but it made json string conversion speed slowly. Also, comparing string concatenation and template literal, template literal is about 1.5x times faster.
Therefore, utilize the template literal enhance performance

Succeeded


Progress Dialog Create when the data fetch from the firebase

On Main Activity you need to create a Progress Dialog when the data fetch from the firebase ..

Hey @maityamit, I am interested in contributing to this issue.
Thanks
Aritra

@aritra-tech It's cool.

I am interested to contribute in this issue JWOC

@shubhankar-bt
It is first cum first serve basis.
I am creating some new issue
I will update on discord .
Then you will get priority on new issues .
This issue is already assign to @aritra-tech
Thanks Guys

@shubhankar-bt
Please Comment on the #5
It is one issue left yet , you can do this

@aritra-tech
You code merged ..


Request: Integration of Homebrew

Would be great to have this app in the Homebrew repositories. Would be help to keep it automaticaly updated

The app needs to be signed to live in the official repo, and I have no plans on doing that 💸, but you can install the cask.
See this thread about it.
I believe you have to run brew livecheck —tap mracos/tap though to get the latest version.