adding PDF latex support

This is an attempt at adding PDF latex support. It currently does this:

adds a --build pdf_latex option, which:
runs sphinx with the latex builder
triggers some different TOC logic because there are some features (like "expand_sections") that don't make sense in latex
Then tries to run the sphinx Makefile command latexmk -pdf -dvi- -ps- '{path_tex_file.name}'

however, this isn't working for me.
First, it seems that Sphinx is getting different toctrees with the latex builder compared with the HTML builder, and I don't understand why.
Second, the latexmk command keeps failing for me saying that it cannot find the .tex file, even though I think it does exist. But maybe this is related to the problem above.
cc @mmcky in case you want to take a look at spot-checking this. I don't think we want to keep this particular implementation in the long term, but I wanted something so that we can start previewing outputs with PDF in the CLI

hey @choldgraf looking at this PR -- is there a reason you want to build latex outside of sphinx. For now we can use the latexpdf builder and pass configuration variables in the same was as html. Sphinx will then handle to pdf generation call with latex_engine

This small example might be good for testing PDF output: https://github.com/ExecutableBookProject/quantecon-mini-example.
If it works we can include that in the blog post as an alpha feature.

@choldgraf looking at name conventions. Sphinx took the approach of using the order of how things are built when making names for builders. I see you have used pdf_html -- for pdf support built from html. The builder in sphinx would most likely be htmlpdf (and latexpdf is pdf built from latex. Are these builder names negotiable? :-)

@choldgraf @AakashGFude this will work to build a pdf file using --build latexpdf but currently triggers the makefile in _build/latex. This enables the pass through of sphinx latex configuration but isn't a good final solution as I don't think it would work on windows. Not sure why latexpdf can't be send through to sphinx as a builder itself.

Hey @mmcky , the makefile in _build/latex get created automatically right? So, it might create a make.bat in windows. And we can trigger that depending on the os we are in?

indeed we could -- but my preference would be to leverage sphinx directly with latexpdf.

@mmcky I think sphinx also does not call latexpdf directly through the build (directly through the application class). Instead, it seems like it uses the same approach like us:-

Here, run_generic_build calls the sphinx.application class
All the more convincing, since there is no latexpdf builder as well :-

.
So, Instead of calling sphinx.application directly like we are calling, we can use a different function like https://github.com/sphinx-doc/sphinx/blob/3.x/sphinx/cmd/make_mode.py#L158 , which calls sphinx.application indirectly with edge case handling here?
Or we can simply handle windows like the above latexpdf builder function is doing in our repo?

Oh I see thanks @AakashGfude maybe that is the easiest path to take given it is only 8 lines of code and pretty clear.

@choldgraf this should be pretty complete for triggering sphinx builds of the generic pdflatex.
We now need to figure out the best way to:

[ ] add latex configuration variable overrides.

currently I run all builds using latex_engine = xelatex as that is robust to the use of unicode etc. But we really need to parameterise this. Should I add a latex section to the config.yml?

re: the name, if you guys think that it is more intuitive to use htmlpdf rather than pdf_html, then I'm OK changing it. I think the first question should be "which convention is better, regardless of Sphinx's convention?" If the answer is "they're equal", then I think we can go with whatever Sphinx does as a next step.
re: configuration, yep I think adding a latex: section to the configuration could be used for configuring latex specifically, similar to what I do with an html: section. The trick will be figuring out config that is truly latex-specific, but we can always discuss!

If you wanna go the latexpdf route, wanna change the HTML config to htmlpdf as well?
If we think this convention is a good one then I will change it in a separate PR I think so as not to confuse the purpose of this PR.

The confusing part of using latexpdf is the standard builder for latex is called pdflatex so the convention is up for debate really. I just liked the logic of latex then pdf so latexpdf :-). I also try and avoid using _ as I find them harder for autofill in terminals.
I will work on adding latex in the config now. There are quite a few latex specific elements such as latex_engine etc.
@choldgraf re: make. The current implementation is under the hood with an auto built Makefile and make.bat file for windows generated by sphinx -- so the user should never see it. Do you find triggering latexmk via a subprocess better than calling make all that has all the latex configuration controlled by sphinx.

re: the naming, I tend to go in the opposite direction when it comes to the name order. In my mind, a user doesn't think to themself "I want some latex files, and then a PDF". They think "I want a PDF (from some latex files)" so it's why I tend to put the final product first rather than later. It has the added benefit that multiple paths to the same output (pdf_latex and pdf_html) have the same root pdf_. I'd also be fine with pdflatex and pdfhtml?
Another option is that we could follow pandoc's approach and have --pdf-engine. Then the command would be something like jupyter-book build mybook/ --build pdf --pdf-engine {latex or html} (with one of them the default).
If that's the case, I almost wonder if we could also expose another level of CLI so people could do

jupyter-book build pdf mybook/ --pdf-engine {latex or html}
or
jupyter-book build html mybook/

and ideally just jupyter-book build mybook/ would default to html. That might not be possible though.
re: the latex config - as a general note, I would try to err on the side of as few configuration options as possible. I think we want the minimal set of configuration, and only add new configuration as it's really needed, to avoid making the UI too complex for people.
re: Make - I don't know what the best approach is. I am worried about relying on an external makefile in general...IMO if we can get something working with pure python that is always preferable to using another in-between technology like make, but if using make allows us to reduce a lot of complexity then it'd be OK

re: the naming, I tend to go in the opposite direction when it comes to the name order.

I am happy with that and it makes equal sense. So lets run with
htmlpdf, latexpdf
I would prefer no _ if that is ok?

Another option is that we could follow pandoc's approach and have --pdf-engine.

Perhaps we can just have a --builder option setting that can switch the backend builder where html is default. That would mean:
jupyter-book build pdf mybook/ --builder {latex, pdflatex, htmlpdf}

re: the latex config - as a general note, I would try to err on the side of as few configuration options as possible

We don't have to really support passthrough at the top level right? I mean if we add a latex: section it will pass through all those options as global config right? So we can essentially support all of sphinx configuration pretty simply as the options from yaml will get transferred over to the book_config object.

re: Make - I don't know what the best approach is. I am worried about relying on an external makefile in general

We could call the function that does this in sphinx I guess but given this is how sphinx does it under the hood then we would just be deferring a call to make to sphinx right?

I would prefer no _ if that is ok?

sounds good - so pdfhtml and pdflatex. Wanna change pdf_html to pdfhtml throughout as well?
You probably also need a rebase because some of the PDF HTML stuff changed to add more tests.

Perhaps we can just have a --builder option setting that can switch the backend builder where html is default

This basically just means to change --build to --builder, which I think is a good idea. +1 to that change in this PR.

I mean if we add a latex: section it will pass through all those options as global config right?

good point - that does seem reasonable. So you're saying "anything inside of latex: will be passed directly to the latex configuration as key-val pairs"?

but given this is how sphinx does it under the hood then we would just be deferring a call to make to sphinx right?

True - though I think it raises a lot of confusion for people in Sphinx. And I think they might be considering changing this in the future as well. How about as a starting point we use the Sphinx makefile approach that you propose, and we open up an issue to keep track of potentially moving that to Python. In this PR we'll need to make sure that we properly catch errors etc that happen when make is ran - we don't want users to see raw "make" errors (see the newly added _error( calls after you rebase)

@choldgraf when you get a chance would you mind having a quick look at these tests. Not sure why the merge of master is causing failures. I have reviewed my resolution to merge conflicts and I think they are ok.

@choldgraf do you know why the CI isn't installing the pyppetteer package
ImportError:
149
===============================================================================
150

151
Generating PDF from book HTML requires the pyppetteer package. Install it first.
152

153
===============================================================================

I believe you haven't modified the pdf_html -> pdfhtml in the github workflow, so it's not finding that extras list

doh -- thanks @choldgraf.

Just a quick note - could you remember to add a test? I think that'll fix codecov's diff report as well.

Working on the test now. Just trying to figure out how to pass through the document name for the generic latex builder.

As an aside - is there any tex distribution that is conda installable? I feel like that would simplify things for many users if it were possible this way...

note this might need a rebase for docs and for tests

As an aside - is there any tex distribution that is conda installable? I feel like that would simplify things for many users if it were possible this way...

I haven't come across one but it would be great if there was.

I wonder if this has enough of the texlive packages
https://anaconda.org/conda-forge/texlive-core

I think we will certainly need a prebuilt docker environment for any tests that include latex. I might try the conda-forge package to see if it contains enough of texlive to pass tests (as an alternative). But 15 minutes is too long.

@choldgraf these are all done in python/pip environments. Do you have a prefered recipe for using miniconda and setting up from environment.yml

hmmm nope not really, usually I just wget miniconda from the anaconda website, and then run their instructions to install, then just start going from there

hey @choldgraf -- looking at the templates for config.yml have we assumed to date that the masterdoc is called index? I don't see a setting to update this.
I am trying to pass through an update to targetname but need to build the following tuple:
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-latex_documents
but don't seem to have access to all the standard elements so the pdflatex build is producing output pdf called python.pdf.

Nope, it updates the master doc name to the first page of the TOC (here: https://github.com/ExecutableBookProject/cli/blob/890bc55d4743e74cc2951b7b0fdeaf99423a0e5f/jupyter_book/toc.py#L174)

does anyone understand circleci paths. It seems circleci adds a context based on the yaml structure. Is this true?
It looks like the latex outputs are here:
/tmp/pytest-of-circleci/pytest-0/test_pdflatex0/_build/latex

OK after some digging the issue with this build is due to
$$
\begin{align*}
\mbox{mean} la_{tex} \\ \\
math blocks
\end{align*}
$$

sphinx turns $$ into:
\begin{equation*}
\begin{split}
\begin{align*}
\mbox{mean} la_{tex} \\ \\
math blocks
\end{align*}
\end{split}

which causes this expression to be malformed latex. This is because align is an environment where we should used aligned
https://tex.stackexchange.com/questions/401201/difference-between-align-and-alignedt

The tricky part here is that the notebook renders the nested math environment case.

The following still needs to be done:

[ ] update latex_document tuple creation with custom options such as targetname for specified filenames
[ ] author config is not being passed to the latexpdf builder context

So no align* will work with the builder? We should definitely document that because I doubt users will be able to debug that one themselves

well it's because it get's wrapped an another math environment so latex complains. I'm not sure why notebook supports it when it really should use aligned to be true to LaTeX and you think of $$ as a math environment. This is a confusing issue for users (certainly needs to be documented). An alternative though is to try and intercept these cases in the LaTeX builder but that is difficult without writing our own.

So no align* will work with the builder? We should definitely document that because I doubt users will be able to debug that one themselves

In fact @mmcky has reminded me of this at least 20 times and yet I still forget and use align*.

I'm not sure why notebook supports it when it really should use

This is closely related to what I have already discussed here https://github.com/ExecutableBookProject/MyST-NB/issues/99#issuecomment-608545104 and here https://github.com/ExecutableBookProject/MyST-NB/issues/126#issuecomment-608600110, and why I'm not convinced that https://github.com/ExecutableBookProject/MyST-NB/pull/121 should be necessary, since it isan additional "post-processed"/HTML only treatment of math that should really be handled at the markdown parsing/MyST-Parser level, to ensure consistency.

I'm not sure why notebook supports it when it really should use

This is closely related to what I have already discussed here ExecutableBookProject/MyST-NB#99 (comment) and here ExecutableBookProject/MyST-NB#126 (comment), and why I'm not convinced that ExecutableBookProject/MyST-NB#121 should be necessary, since it is an additional "post-processed"/HTML only treatment of math that should really be handled at the markdown parsing/MyST-Parser level, to ensure consistency.

thanks for linking these two issues @chrisjsewell.

Note at the moment, the nowrap option of https://www.sphinx-doc.org/en/1.0/ext/math.html, is always set to False, but it may be the case that we want to do additional check here whether the math is enclosed in latex tags of not and set nowrap accordingly.

I agree with @chrisjsewell on this -- being general is slower to get things working but ultimately the better approach. I guess we would need to infer latex environments to enable a nowrap request.

I think this has already been explained (and lost in an earlier thread) but I'm still unsure of how/if mathjax (or katex) is invoked if not from an html script tag ala https://github.com/ExecutableBookProject/MyST-NB/pull/121
As far as the align environment goes, the one semi-esoteric feature that I use most frequently is the ams subequations environment, which is available as a mathjax extension and is semi-emulated in bookdown using \tag:  like this example.  It's pretty hard to work around its absense when you need to discuss the 4th equation in a system of 8 equations.

but I'm still unsure of how/if mathjax (or katex) is invoked if not from an html script tag

It is loaded via sphinx.ext.mathjax (see the code here), which is loaded by default in the HTML builder (see here).
So I think (a) we should be setting configuration via the sphinx options, not custom JS, and (b) mathjax should not be set to find just any dollar or latex tags in the HTML page, it should only be applied to math that has already been identified during the parsing phase, and enclosed in divs with the math class.

As far as the align environment goes, the one semi-esoteric feature that I use most frequently is the ams subequations environment

I would add this to the existing issues, or open a new feature request. But essentially, for an output agnostic solution, I think it would be desirable, that the latex is actually parsed, e.g. using https://github.com/alvinwan/TexSoup, to extract information like labels/tags and what environment the math is wrapped in.

Will do -- although it's not a high priority for me and the bookdown community seems to be living without it -- more along the lines of "easy things should be easy and hard things should be possible".  If myst has math feature-parity with current bookdown, then we can use their workaround for now, which is just to have a single global counter and increment it for every tagged subequation:  e.g. https://stackoverflow.com/a/55924252

thanks for this discussion guys. For now I will add a comment to the docs regarding latex environments within $$ signs, but I will open a LaTeX meta issue that captures this useful discussion / ideas about better parity between outputs.

@choldgraf I think your idea of splitting the tasks is a good one for latex checks. I will separate that out into a new .circleci task. So we will have tests, html, and latex checks?

yeah for now that is fine with me, so long as both tests pass then I think it's fine

hey @choldgraf I updated click to use --builder as a new option but seems to have upset the default build. With click is the function name automatically an option? Or do I need to add build back in?

hmmm, what is the error? It looks OK to me

ah I think I found a spurious build still hanging around.

Updating the targetname for latex documents isn't trivial as they are all tied up in the latex_documents tuple.
https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-latex_documents
@choldgraf is there a way to fetch the config from the sphinx application prior to running the sphinx_build?

@choldgraf I think I have an idea. I have found a suitable entry point in sphinx.py.

@choldgraf do you know why app.config.author is unknown for jupyter_book/book_template given author is specified in _config.yml?

@choldgraf I see. So config_yaml doesn't get added to the book_config that is passed to sphinx. https://github.com/ExecutableBookProject/cli/blob/master/jupyter_book/commands/init.py#L69
Do you see these as different configuration to sphinx items? We could have a list of matching top level sphinx options and then pass them through so sphinx can use them for all builders.

@choldgraf I have added some passthrough of top level config like author and title fields to book_config. You may want to review in case you think this should be done differently.

Last thing left is to migrate pdflatex testing to use a docker container with texlive preinstalled.

[ ] setup texlive docker for pdflatex testing

It is a bit convoluted, but the YAML config does make it into the sphinx build. The path to the YAML file is added as config here: https://github.com/ExecutableBookProject/cli/blob/master/jupyter_book/commands/init.py#L68
and then there's a Sphinx event that updates the config with those values here: https://github.com/ExecutableBookProject/cli/blob/master/jupyter_book/yaml.py#L19

It is a bit convoluted, but the YAML config does make it into the sphinx build. The path to the YAML file is added as config here: https://github.com/ExecutableBookProject/cli/blob/master/jupyter_book/commands/init.py#L68
and then there's a Sphinx event that updates the config with those values here: https://github.com/ExecutableBookProject/cli/blob/master/jupyter_book/yaml.py#L19

I am not sure that this is working as intended as I had to add some code to this PR to get a few elements working such as author. Let's link up to discuss.

this looks promising if we used github actions for CI when building with texlive

https://github.com/xu-cheng/latex-action

to make the swap we will need to use netlify previews on github actions.

why not do both? use netlify to preview the site HTML, and use github actions just for the latex builds and tests

@mmcky where are we on this one? I'd like to get all of these PRs cleaned up before we make an "official" beta release of jupyter-book. I am happy to treat latex building as an "alpha" feature that is experimental and make that clear in the documentation, if we only have partial functionality...

why not do both? use netlify to preview the site HTML, and use github actions just for the latex builds and tests

OK sure. Will try this approach for running latex tests.

@mmcky where are we on this one?

Been stuck learning github actions for deployments on QuantEcon projects. I'll work on this PR and tidy up the loose ends tonight / tomorrow.

ok great - once we get this merged then we can make an official "beta" release in jupyter-book

The difficulty with using the latex action is that it wants to build tex files using an entrypoint. Which is great but makes it very difficult to test the cli directly. We could build latex files and then build them to pdf using the action but that kind of doesn't fully test the cli. So I think we are back to using a custom docker image to support latex builds. If people are happy with a separate 15 minute test (to install texlive) then I can make this ready pretty quick. We can then improve using a Docker environment for the test.

Hmmm, why not use the docker image that the latex action uses: https://github.com/xu-cheng/latex-docker then we can install python and jupyter book into that, and use it to test the latex build

hey @choldgraf I have been reading some of the docs but it doesn't seem like you can setup a Github Action to run in a docker environment (like circleci). It seems to support loading a docker container action which we could setup with software preloaded and then run the jb command and get the results.
It looks like circleci can use docker containers as the runner. I don't see the equivalent of this https://circleci.com/docs/2.0/custom-images/ anywhere for github actions.

Let's just use circleci for the latex tests then?

Let's just use circleci for the latex tests then?

Yeah. I have just triggered a cache request for tmp/texlive folder and will see how much that speeds things up. Hopefully a 3Gb cache request is ok :-)

I'd that doesn't work, could you try a latex image and just install python onto it with miniconda or something?

I have done some reading and a 3Gb Docker container takes a while to come in from dockerhub. I am hoping the cache option on circleci is faster (within network) 🤞

Sounds good, I am +1 on whatever you guys think is the best approach

@choldgraf the cache approach worked nicely. It takes ~2m now with retrieval of the 1.6Gb cache of texlive.

So this is ready for review. I think the only contentious part may be how latex options are passed through to the sphinx.app -- but I thought using a latex section in the yaml would be the best approach.

@choldgraf do you know why this is getting caught in pre-commit
Fix End of Files.........................................................Failed
17
hookid: end-of-file-fixer
18

19
Files were modified by this hook. Additional output:
20

21
Fixing jupyter_book/book_template/notebooks.ipynb

that file hasn't been edited in this PR.

@mmcky it looks like the tests are failing on github actions because we're running the PDF tests when we should only be running those on circleci: https://github.com/ExecutableBookProject/cli/pull/73/checks?check_run_id=621445961#step:7:84
Can we fix that, and then are we ready to merge this as a "more beta than most other features" feature?

@choldgraf do you see pdfhtml as an html or pdf feature? I have current switched both off in pytest for github actions. If that is OK we can remove the headless chrome setup as well.

@choldgraf what is black complaining about?
https://github.com/ExecutableBookProject/cli/pull/73/checks?check_run_id=624524989
do I need to commit the diffs listed?

Is there a reason why you are not just using pre-commit install?
Oh please educate me! These pre-commits are driving me nuts :-)

Oh please educate me! These pre-commits are driving me nuts :-)

https://pre-commit.com/#3-install-the-git-hook-scripts
So in the terminal just cd into the root folder of the repository, then run pre-commit install. That's it lol (it adds some stuff in the .git folder), then when you git commit, it will automatically run, only on files that have been modified

@choldgraf all tests passing :-). When you decide to merge please Squash as this is pretty untidy. Cheers.

amazing! I'll plan to take a final check pass tomorrow, and then merge and begin The Great Migration.

ok, let's 🚢 it and now I'll begin the process of rebasing.


mfem::ParFiniteElementSpace::GetEssentialTrueDofs

I am not completely sure what is the issue, however, for NCMesh the checks do not pass and the code generates an error. An example file with mesh is attached.
tst.cpp.gz
manifold.msh.gz
A run on a single process works fine. The issue have been observed on 2, 4 and 8 processes.

A potential workaround/solution is to comment out the if line here: https://github.com/mfem/mfem/blob/9a73c2d94c5ecff6dd64c9f7f575e55da0b250cf/fem/pfespace.cpp#L1013 or in other words to call Synchronize for NC meshes as well.

This should be resolved by #2722. I tried the code and mesh posted above and they seem to work fine with that PR.


Bug / Patch: System instructions are lost in VertexAiGeminiChatModel

Bug description
The system instructions I was providing indicated a JSON output format, but were getting ignored in Spring AI. The same exact instructions and prompt in the test console work perfectly.
https://console.cloud.google.com/vertex-ai/generative/multimodal/create/text
I debugged into this and even thought I could put in a PR for it, but honestly, I don't see where this goes wrong. In  private GeminiRequest createGeminiRequest(Prompt prompt) I can clearly see my system instructions and the code looks like it adds them to the model:
              String systemContext = prompt.getInstructions()
			.stream()
			.filter(m -> m.getMessageType() == MessageType.SYSTEM)
			.map(m -> m.getContent())
			.collect(Collectors.joining(System.lineSeparator()));

		if (StringUtils.hasText(systemContext)) {
			generativeModel.withSystemInstruction(ContentMaker.fromString(systemContext));
		}

		return new GeminiRequest(toGeminiContent(prompt), generativeModel);

But in the debugger, at the return statement, the system instructions are empty:

Environment
Please provide as many details as possible: Spring AI version, Java version, which vector store you use if any, etc
    implementation platform("org.springframework.ai:spring-ai-bom:1.0.0-SNAPSHOT")
    implementation 'org.springframework.ai:spring-ai-vertex-ai-gemini'
    implementation 'org.springframework.ai:spring-ai-vertex-ai-embedding-spring-boot-starter'

I should note, until this morning the code did not work at all, but the PR from last night solved that :)
Steps to reproduce
See code below.
Expected behavior
The system instructions are passed along in the gemini request.
Minimal Complete Reproducible example
package my.rag;

import org.springframework.ai.chat.messages.Message;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.model.Generation;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.vertexai.gemini.VertexAiGeminiChatModel;
import org.springframework.ai.vertexai.gemini.VertexAiGeminiChatOptions;

import com.google.cloud.vertexai.VertexAI;

import java.util.ArrayList;

public class Test {
    public static void main(String[] args) {
                var chatModel = new VertexAiGeminiChatModel(
                new VertexAI(Config.GOOGLE_CLOUD_PROJECT_ID, Config.GOOGLE_GEMINI_API_LOCATION),
                VertexAiGeminiChatOptions.builder()
                        .withModel(VertexAiGeminiChatModel.ChatModel.GEMINI_1_5_FLASH)
                        .withTemperature(1f)
                        .build());
        var messages = new ArrayList<Message>();
        var systemMessage = new StringBuilder();
        systemMessage.append("You are a helpful assistant, providing guidance based on the information provided below. Always return as JSON format:\n")
        .append("Format example:\n")
        .append("{\"response\": \"This is the useful response from the helpful assistant\", \"link\": \"https://link-for-more-information\"}");
        System.out.println(systemMessage.toString());
        messages.add(new SystemMessage(systemMessage.toString()));
        var userMessage = "What is the URL for the google search engine?";
        messages.add(new UserMessage( userMessage));
        System.out.println(userMessage);

        ChatResponse response = chatModel.call(new Prompt(messages));
        for (Generation aiResult : response.getResults()) {
            System.out.println(aiResult.getOutput().getContent());
        }
    }
}

Running this code the RAG responds with something to the effect:
The URL for the Google search engine is simply:

**https://www.google.com**

In the console I get
{"response": "The URL for the Google search engine is www.google.com", "link": "https://www.google.com"}

I see it now, this PR should resolve it:
https://github.com/spring-projects/spring-ai/pull/1031

Thank you @turchinc  for catching and resolving this issue!
Highly appreciated!

@tzolov  thanks for applying the fix! I can confirm my app is working now with snapshot.


Support R2DBC

There's a project building a reactive alternative to JDBC: https://github.com/r2dbc
Supporting this could be valuable, as they currently have drivers for more than just postgres, and a standard API for reactive database stuff could be better cross-inspired.

Sounds interesting. Would you like to give it a try?

I'm afraid not. I just wanted to make sure you were aware of each other, as a (hopefully) future user.

That's ok. Thanks for the heads up, we will get to it as soon as we can.

I could possibly look into this one in a few weeks. I'll be doing some deeper investigation into R2DBC shortly and might have the cycles to connect the two projects together.

Sounds great. Please, assign this to yourself when you start to work on it so that people can see that somebody is taking care of it. Unless somebody else wants to send a PR before you start to work on it :)

@aguibert That would be great. It might actually be quite easy to support these drivers, since our dependency to Axle is really pretty localized.

@gavinking  @DavideD Would you prefer to have this worked in sooner or later? On one hand it would be nice to get it integrated sooner so we can be mindful of the requirements of abstracting over both APIs from the start, but on the other hand it will slow us down a bit to need to create extra abstractions over both APIs as we iterate on the HibernateRX design itself

I don't know, do we really need that much more abstraction to support it? Maybe you can have a quick look and let us know more about how hard it is going to be? (Unless you have already looked into it)

Would you prefer to have this worked in sooner or later?

I don't see it as something we need to have in the first release, but if it's straightforward, it would certainly be nice to have.

I don't know, do we really need that much more abstraction to support it?

I doubt it. Interaction with the database is already pretty well encapsulated within just a couple of classes.

it will slow us down a bit to need to create extra abstractions over both APIs as we iterate on the HibernateRX design itself

I don't imagine the impact will be enough to slow us down.

Note that with #155 it is now in principle possible to integrate other reactive database drivers.

I'm going to close this one now because in #154 I introduced an extension point that allows users to experiment with integrating alternative reactive database drivers. If someone actually gets motivated to do this, would could consider accepting a contribution, but we don't have time to work on this right now (that's really quite a lot of work in terms of testing, etc), and @OrangeDog says he doesn't have time to work on it either.
To be clear, I'm not ruling out OOTB support for other driver libs in future, just saying that we've already done what we have time for right now, and now there's nothing stopping other folks from playing with this for themselves.


Add resultcode enum

@KadoBOT github action tests are failing can you check ?

@KadoBOT github action tests are failing can you check ?

@rikterbeek good catch!


fix snippet data table reference

resolves #862
@mattwynne

Thanks @charlierudolph! I'm looking forward to more of your contributions to cucumber-ruby...


Allow selected data points to be shown as dots

Would be nice if it's possible to add dots to show some selected data points.
See attached file for an example.
That dot shows 2 data points. In this example, it shows where portfolio is versus a benchmark) and where they are in the quartile ranking. This aids in visualizing where a particular point you are observing in relation to the quartiles grouping.
Also, if would be nice to add some sample data set to aid in learning how to use the visualization. Cheers!

Thanks for the suggestion and this would probably ideal to be using in combination with the 'highlight' option of Power BI.


Setting a high page limit for Meraki get networks

Because some people have a lot of networks we need to raise the default number:
https://developer.cisco.com/meraki/api-v1/get-organization-networks/
If this doesn't work we can look at iterating over pages or supporting tag based search.

Convo on discord: https://discord.com/channels/887507000722882570/888243620367458374/1135995542468571257

I added a pagination approach which should work better.


Load content servers using new web api endpoint

ClientServerList message is no longer sent on login
https://steamapi.xpaw.me/#IContentServerDirectoryService
Not sure if you want to retain the contentServersReady and corresponding event for some kind of backwards compatibility.

Yes, the contentServersReady property and event need to be included still. You can just set/emit them in the logon callback.

Added logic for contentServersReady to logon and logoff callbacks

Thanks, I'll make some changes before pushing it out for backwards compatibility.


Download buttons won't show until entering the inspect element tool

Can't reproduce. Share more details about your browser, extension version, score being viewed, etc.

Fixed in df36815a2b26271d4f9a56d3a2636ac671112dd2. Update to v0.35.20


how to define the batch size

Thanks to your codes!
I saw you used train_on_batch to start training. But how to define the batch size if you used that? I just guess that the batch size is always 1.

This repo follows the original faster-rcnn paper, see section 3.1:

The   RPN   can   be   trained   end-to-end   by   back-
propagation  and  stochastic  gradient  descent  (SGD)
[35]. We follow the “image-centric” sampling strategy
from [2] to train this network. Each mini-batch arises
from a single image that contains many positive and
negative  example  anchors

It could certainly be modified to do larger batches. You'll have to make sure to crop the images to all have equal size though, then concatenate them into a batch.


Error: Duplicate key for property path:

Description
When attempting to dynamically create pages, I get:
Error: Duplicate key for property path: /ltc-terms

Steps to reproduce
In my gridsome.config.js, I have:
// This is where project configuration and plugin options are located.
// Learn more: https://gridsome.org/docs/config

// Changes here require a server restart.
// To restart press CTRL + C in terminal and run `gridsome develop`

module.exports = {
  siteName: '',
  plugins: [
    {
      use: '@gridsome/source-filesystem',
      options: {
        path: 'content/posts/*.md',
        typeName: 'BlogPage'
      }
    },
    {
      use: '@gridsome/source-filesystem',
      options: {
        baseDir: 'content/pages',
        path: '**/*.md',
        typeName: 'MainPage'
      }
    },
    {
      use: `gridsome-plugin-netlify-cms`,
      options: {
        publicPath: `/admin`
      }
    }
  ]
}

My gridsome.server.js:
// Server API makes it possible to hook into various parts of Gridsome
// on server-side and add custom data to the GraphQL data layer.
// Learn more: https://gridsome.org/docs/server-api

// Changes here require a server restart.
// To restart press CTRL + C in terminal and run `gridsome develop`

const nodeExternals = require('webpack-node-externals')

module.exports = function (api) {
  api.loadSource(({ addContentType }) => {
    // Use the Data Store API here: https://gridsome.org/docs/data-store-api
  })

  api.createPages(async ({ createPage, graphql }) => {
    // Use the Pages API here: https://gridsome.org/docs/pages-api
    const blog = await graphql(`{
      allBlogPage {
        edges {
          node {
            id
            date
            summary
            path
            tags
            excerpt
            title
            content
          }
        }
      }
    }`)

    blog.data.allBlogPage.edges.forEach(({ node }) => {
      createPage({
        path: `/${node.path}`,
        component: './src/templates/Blog.vue',
        context: {
          id: node.id,
          path: node.path,
          summary: node.summary,
          content: node.content,
          title: node.title,
          date: node.date
        }
      })
      console.log('createPage', node.path)
    })
  })

  api.createPages(async ({ createPage, graphql }) => {
    const main = await graphql(`{
        allMainPage {
          edges {
            node {
              id
              date
              path
              title
              content
            }
          }
        }
      }`)

    main.data.allMainPage.edges.forEach(({ node }) => {
      createPage({
        path: `${node.path}`,
        component: './src/templates/MainPage.vue',
        context: {
          id: node.id,
          path: node.path,
          body: node.content,
          title: node.title,
          date: node.date
        }
      })
      console.log('createPage', node.path, node.content)
    })
  })

  api.chainWebpack((config, { isServer }) => {
    config.mode('development')
    if (isServer) {
      config.externals([
        nodeExternals({
          whitelist: [/^Vuetify/]
        })
      ])
    }
  })
}

An example of content/pages/ltc-terms.md
---
title: LTC Terms
date: 2012-09-06
path: /ltc-terms
---

LTC Terms go here

Expected result
I expect the main pages to be created just like the blog post pages. Their format identical to blog posts. Yet I get weird issues, blog posts don't need a pre-pended /, but it complains if I don't add it for the main pages. When I do, I get:
Actual result
createPage /ltc-terms
LTC Terms go here
 (this is just a console.log)

Error: Duplicate key for property path: /ltc-terms
    at UniqueIndex.set (/Users/atomic/ltc/gridsome-ltctree/node_modules/lokijs/src/lokijs.js:7329:17)

    at Collection.add (/Users/atomic/ltc/gridsome-ltctree/node_modules/lokijs/src/lokijs.js:5974:31)
    at Collection.insertOne (/Users/atomic/ltc/gridsome-ltctree/node_modules/lokijs/src/lokijs.js:5738:17)
    at Collection.insert (/Users/atomic/ltc/gridsome-ltctree/node_modules/lokijs/src/lokijs.js:5677:21)
    at Route.addPage (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/pages/pages.js:447:19)
    at Pages.createPage (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/pages/pages.js:175:18)
    at Object.createPage (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/app/actions.js:258:24)
    at Template.createPage (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/plugins/TemplatesPlugin.js:375:23)
    at createTemplate (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/plugins/TemplatesPlugin.js:295:24)
    at TemplatesPlugin.api.createManagedPages (/Users/atomic/ltc/gridsome-ltctree/node_modules/gridsome/lib/plugins/TemplatesPlugin.js:332:9)
    at process._tickCallback (internal/process/next_tick.js:68:7)

I have only a single file in that directory, but it has the same issues with all files in that directory. I thought perhaps pages was a global restricted word, so renamed it to main with no luck.
Environment
Paste the information here as shown by gridsome info
It says gridsome info is not a command, so here is my package.json:
{
  "name": "ltctree",
  "private": true,
  "scripts": {
    "build": "gridsome build",
    "develop": "gridsome develop",
    "explore": "gridsome explore"
  },
  "dependencies": {
    "@gridsome/cli": "^0.3.1",
    "@gridsome/source-filesystem": "^0.6.2",
    "@gridsome/transformer-remark": "^0.4.0",
    "@mdi/font": "^4.5.95",
    "apollo-fetch": "^0.7.0",
    "axios": "^0.19.0",
    "global": "^4.4.0",
    "graphql-tag": "^2.10.1",
    "gridsome": "^0.7.1",
    "gridsome-plugin-netlify-cms": "^1.0.9",
    "moment": "^2.24.0",
    "netlify-cms": "^2.9.7",
    "stylus": "^0.54.7",
    "stylus-loader": "^3.0.2",
    "vue-the-mask": "^0.11.1",
    "vuetify": "^2.0.10"
  },
  "devDependencies": {
    "material-design-icons-iconfont": "^5.0.1",
    "webpack-node-externals": "^1.7.2"
  }
}

Thanks in advance for any help troubleshooting this!

Hey @phillipweston! 👋
I may be wrong, but I believe the file in content/pages are automatically created (since you used @gridsome/source-filesystem).
Therefore, since you are trying to add a page manually in your gridsome.server.js it complains about a duplicated path (because it has already been added).

huh @RomainLanz, thanks for the tip! my understanding was that @gridsome/source-filesystem would just pull the content of those pages into GraphQL. Is there something else I use if I want them ingested, but not auto created?

Thanks for your comment, it was useful in troubleshooting the problem.
If the typeName matches a valid template/<typeName>.vue, it will automatically create the pages. If you want @gridsome/source-filesystem to simply import items into GraphQL for your own use, then make sure the template name does not match the type.

@phillipweston Gridsome newbie here. I have been reading both this issue page and "Next and previous node in collections" Issue 177, and I guess I'm missing what you're saying here and what @hjvedvik says here, because I can't make it work — I keep getting the Duplicate key for property path error. Here is what I have taken your last comment to mean (but obviously I'm wrong):
If gridsome.config.js includes the following:
  plugins: [
    {
      use: '@gridsome/source-filesystem',
      options: {
        path: './content/posts/**/*.md',
        typeName: 'Post'
      }
    },
  ],
  templates: {
    Post: [
      {
        path: '/posts/:year/:month/:fileInfo__name',
        component: './src/templates/Post.vue',
      }
    ],
  },

... then gridsome.server.js should not include the following:
      createPage({
        path: node.path,
        component: './src/templates/Post.vue',
        context: {
          id: node.id,
          prevId: prev ? prev.node.id : null,
          nextId: next ? next.node.id : null
        }
      })

I have tried assigning different templates in gridsome.config.js and different components in gridsome.server.js but keep getting that Duplicate key for property path error, so I'm pretty sure I've simply misunderstood what you and @hjvedvik were saying. I would be grateful if you could set me straight on an instruction that sounds simple but, at least for me, clearly isn't.

@brycewray I was able to resolve this issue by simply assigning a different name for my Post.vue file (by renaming it to BlogPost.vue and changing all relevant imports).
To my understanding, because the typeName fed into @gridsome/source-filesystem is Post, and the template inside /src/template is also named Post.vue, @gridsome/source-filesystem will pick up all Post type pages and populate them automatically. And when you use createPage() afterwards, because Post type pages are already generated, gridsome will eventually complain about Duplicate keys.

Also, as a reference, the noxify/gridsome-starter-casper-v2 Gridsome theme did a very nice job for generating previous and next nodes.

@spencerwooo Thanks very much for this helpful information.


[Inventory Management] Add a central class for caching/parsing inventory & static data

Short Description: One of the biggest pains adding more complicated features is that the inventory parsing logic is spread out all over the place. This means that individual cell_workers must often parse the giant response dict themselves, which is very error prone and leads to a bunch of duplicated code. This PR adds a central class that parses every item we get back in the GET_INVENTORY request in to more usable classes so that it easy to retrieve inventory data from anywhere. This is cached at the module level and we should try to only initialize this once per tick.
Layered on top of this is the ability to load a static json file to get more data that is relevant to a certain type of object. For example, parsing pokemon.json to get next evolution, cost, etc.
The usage for this is very simple
from pokemongo_bot import inventory

# Access various inventories
inventory.candies()
inventory.pokemons()
inventory.pokedex()
inventory.items()

# Some example uses
if inventory.pokedex().captured(pokemon_id):
  print('Have captuered {}'.format(inventory.Pokemons.name_for(pokemon_id))

# Access static data
inventory.Pokemons.name_for(pokemon_id)
inventory.Pokemons.first_evolution_id_for(pokemon_id)

I also added two small examples of how this can be used.

Output # of candy after capturing a pokemon
Replaced all logic for fetching/tracking Candy in EvolvePokemon

I have been running with this for a while. I also implemented a few more complex features using this class (optimizing evolutions for xp/unseen pokemon), keeping the number of pokemon you need to perform the maximum # of evolutions, but I will send them out as separate PRs since they require a little clean-up.
In the long run we should probably move ALL inventory management through this mechanism, but I didn't want to track down every place we parse the response_dict directly so I added this in a way that should be easy to move towards incrementally.

Very nice, just this evening I also started to look at "software inventory".
I came here to look if I could find a place to dump a minimal example and this PR might just do the trick *g*
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import json
from pgoapi import PGoApi

def main():
    api = PGoApi()
    api.set_position(LAT, LON, 0.0)
    api.login('ptc', 'USER', 'PASS')

    req = api.create_request()
    req.get_player()
    req.get_inventory()

    import json
    with open('data.json', 'w') as fp:
        json.dump(response, fp)

if __name__ == '__main__':
    main()

The resulting data.json can be read with
jq '.responses.GET_INVENTORY.inventory_delta.inventory_items[] | .inventory_item_data | [ {pokemon: [ .pokemon_data.pokemon_id , .pokemon_data.cp]}, { pokedex: [.pokedex_entry.pokemon_id, .pokedex_entry.times_captured]}, {item: [.item.item_id, .item.count]} ]' -c data.json
This PR is much nicer, of course! (y)
But maybe us "inventory-minded" can find a place for an inventory discussion somewhere? I am excited to see this PR because although I never do computer games I discovered this one last week explicitly because it generates "data with a story" as a basis for data analysis. So my ~50 bots are meant to produce "software inventory over time". (I do have an automated PTC account 'sniper', btw. The CSRF-token is easy to get with curl and their Django checks for the referer which took me a day to realize. The Gmail API then gives access to the confirmation link. Starting all bots with generated systemd services "pumps data" *yay* )
With that in place inventory processing becomes the next important item.
potential usecases for better access to inventory data

visualization of the data, possibly over time
comparison between bots
identification of rare Pokemons in the "bot army"
periodically dumping a bot's inventory

data enrichment
Some data is not present in the GET_INVENTORY structure and could be nice to persist otherwise:

location where it was captured
any kind of geo-data analysis is then possible

@cprior  I definitely was mostly thinking about code structure when I wrote this, but I like a lot of your ideas. My measly 3 bot army does not have a need for super advanced analytics I guess lol :p.
I think the main advantage of this model is that we should be able to extend the classes as much as we want, since they are organized more by conceptual objects than how the response dict is set up. This then allows use to add logic specific to different types in a central, re-usable way. Layering static data on top of the classes already makes them not strictly dealing with the inventory so it could be reasonable to add in some more logic (such as rarity, comparison etc). This could also be a separate class that just uses the Inventory component for the analytices
As for the accessing, I think there are two things we can do:

I could add a getter for the entire inventory (had that initially)
could also require every InventoryComponent to implement some kind of "serialize()" function that would give a json representation of the component. Then we could dump to a file for history etc.

DUDE I LOV YOU

@aeckert come to slack and let's have a chat. I have plans too add superpowers to your PR.

@aeckert : Yes, an equivalent of .toString() like .toJson() or similar is what I had in mind 4 hours ago. In Python this could be just a json.dump() away.
Your "cached at the module level " deserves highlighing, as I managed to get a 'ServerSideRequestThrottlingException("Request throttled by server... slow down man")' right away :)
The full inventory is already placed in the web directory so my above example is probably not the best. But this is written by  cell_workers/transfer_pokemon.py

Ah, and talking about "inventory data": As far as I understand there might well be additional Pokemons in the (near?) future. Just something to keep in mind -- this is all rather "NoSQL" than RDMBS, so to say.

Can you solve merge conflicts, please?

Sure fixing now.

@aeckert sorry to bother, but merge conflicts again :/

Seems like the Pokemon class that has some display logic that was added here: https://github.com/PokemonGoF/PokemonGo-Bot/pull/2527/files#diff-d81926cdea81e55e6b994231eda7f95dR27
Could be consolidate with the pokemon class added in this PR.

@TheSavior Yea we probably should consolidate those. Will try to do that in a separate PR though as I have flaky internet the next few days and would like to get this in.
My plan was to slowly go through and use this class to clean up code in various cell workers in a separate PR.

@douglascamata Fixed conflicts :)

@TheSavior no internally it will remap Pidgeotto to Pidgey and maintain one count. This is also what is received in the response to a GET_INVENTORY request. The logic is to get the first evolution for the passed in pokemon number and then retrieve the number of candies for that. That logic is in the inventory.py file.

:+1:

@aeckert This is great. Doing a better job of caching the inventory will help make everything better and reduce the number of unnecessary requests we need to make to the api, speeding everything up and improving the amount of xp/hr everyone can get.
Now that this is in, it should hopefully be relatively easy to add PRs and get them merged that use this in more places.

@aeckert See #3256


Update FE to reflect there are more pQTL studies than just Sun et al.

Taking the next variant page as a reference https://genetics.opentargets.org/variant/2_102309902_G_A
The FE is not flexible enough to accept other pQTL studies that have been recently added to the Genetics Portal.

If we attend to the API response for the same page, it contains all the information in the same way we represent eQTL studies.

We can see how the response contains the information for the other studies (e.g.PIETZNER).
Unless there is something I'm not anticipating it looks we just want an implementation exactly the same as we have for the eQTL studies. We will also want to remove all references to Sun et al. when we refer to the aggregation of all pQTL studies.
Happy to explain in more detail

@d0choa this label comes from the BE, in order to hide the label in the FE, I'll need to add custom conditions depending on the variant schema and parse the text. I do not think that is the best approach to add that logic that will need to be added in multiple places and will need different places (e.g sourceDescriptionBreakdown sourceLabel)

By now for the current release, this will be addressed by the front end (PR) , for the future, I opened an issue for a new data/BE implementation


Passkey signatures failing for some hashes

I was running tests on a fork of the smart wallet and getting errors for some tests.
I was following the same signing procedure as in this repo, and eventually found that the signature malleability check in WebAuthn-Sol was returning the error due to s being too large.
This test shows that for some hashes the values returned from vm.signP256 are causing the problem. I added a test testValidateSignatureWithPasskeySignerWithSCheck which passes when we take the compliment of s.
Is this something that should be handled by signP256? Maybe I have misunderstood what webAuthn.messageHash should be and that is causing the wrong signatures?

Hey, thanks for this! Here is how you can normalize s values control for this. We'd expect clients to do this normalization.

Ah thanks, yea I see it is expected then. Will take it into account from here


Review request for Microsoft.DBforPostgreSQL to add stable release of version 2024-08-01

ARM (Control Plane) API Specification Update Pull Request
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

Purpose of this PR
What's the purpose of this PR? Check the specific option that applies. This is mandatory!

[X ] New API version for an existing resource provider. (If API spec is not defined in TypeSpec, the PR should have been created in adherence to OpenAPI specs PR creation guidance).

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[ X] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[X ] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to the diagram Step 2, "ARM API changes review", for this PR.

Additional information

 Viewing API changes
Added new GA API version 2024-08-01 This stable version introduces following:

Geo + CMK - Revive Dropped
Storage auto growth
IOPS scaling
New location capability api
Long Term Retention Backup
Server Logs
Migrations
Migration Pre-validation
Read replicas - Switchover (Site swap)
Read replicas - Virtual Endpoints
Private Endpoints
Azure Defender / Threat Protection APIs
PG 16 support
PremiumV2_LRS storage type support
Location capability changes for SSDv2
Migration Roles
Migration Instance Resource Id to support Private endpoint Migrations
On-demand backup

Suppressing failures
If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
suppressions guide to get approval.

Getting help

First, please carefully read through this PR description, from top to bottom. Please fill out the Purpose of this PR and Due diligence checklist.
If you don't have permissions to remove or add labels to the PR, request write access per aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories
To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
For help with ARM review (PR workflow diagram Step 2), see https://aka.ms/azsdk/pr-arm-review.
If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

Hi @mikekistler These changes were approved in pr #30433 but I abandoned and to redo here.
Breaking changes previously reviewed and approved in https://github.com/Azure/azure-rest-api-specs/pull/29295

/azp run

Hi @tadelesh could we please get a review of this PR to validate the breaking change for sdk-for-go. Check shows it succeeded.


recommendation for handling optional attributes (in keys or elsewhere)

Hi there,
thanks a lot for the release of v1.15.0 which resolves all blockers for me, so I can finally upgrade (currently using 1.10.2). 🎉
While upgrading I encountered a couple of issues related to the handling of optional attributes and I am not sure how to solve them:

optional attributes
I have an optional attribute on my Entity (not part of a key) declared like device?: string;. When I try to update an object and set the value of object to undefined (e.g. with an update body like {device: undefined} ) I get the following error: Invalid UpdateExpression: An expression attribute value used in expression is not defined; attribute value: :UE_device.
It seems typedorm filters out undefined values and the generated dynamo expression is missing this value. Is this a bug or is it intended behaviour? What is the recommended way to handle this case? Extending the type definition to device?: string | null; and updating with {device: null} works, but might not be desired in all cases.

nullable attributes as part of key
When using an attribute with type device?: string | null; in a key, I get the following error: Error: "device" is used in key XYZ, thus it's type must be or scalar type, if attribute type is Enum, please set "isEnum" to true in attribute decorator. I can remove the null but then I am not able to correctly set the attribute to be undefined. I read about the isSparse option, which is supposed to be released in v1.16.0 but seems to be already included in v1.15.0, but enabling this on the index does not seem to change the behaviour.

It might also be a good idea to have some examples with optional values (and sparse indices) in the readme as I guess this is a very common case.

Thanks for reporting this @silberistgold. Looks like a bug to me, I'll have a look.
Ideally I would expect the TypeDORM to auto remove the attribute from the attribute update expression when the value is set to undefined and when it is of type null it should let it go as is mainly because there could be reasons where someone would be wanting to null out a value for an attribute in DynamoDB.

I'm running into this issue as well I'm not sure what the "blessed" approach is.
I'd prefer to do
@Attribute({
  default: null // <- Not allowed currently, so I set it in my data access layer that calls TypeDORM
})
myOptionalProperty: string | null;

Along with a sparse index (isSparse) but that's not working, I get the following error:
Error: "myOptionalProperty" is used in key MYPREFIX#{{myOptionalProperty}}, thus it's type must be or scalar type, if attribute type is Enum, please set "isEnum" to true in attribute decorator.

However if I change that to
@Attribute()
myOptionalProperty?: string;

And I don't define it (either set it to undefined or don't touch it at all) I get the following error:
{
  "name": "SparseIndexParseError",
  "message": "Failed to parse \"MYPREFIX#{{myOptionalProperty}}\""
}

My index looks like this:
GSI2: {
  partitionKey: 'MYPREFIX#{{myOptionalProperty}}',
  sortKey: 'MYPREFIX2{{myOptionalProperty2}}', // I'm using another optional property here
  type: INDEX_TYPE.GSI,
  isSparse: true // <- This should be all I need right?
},

This also seems to be a problem if you specify a key but assign the value as undefined. For example:
const userId = 1
const email =<EMAIL_ADDRESS>const nickname = undefined
entityManger.update(
  User,
  {userId: userId},
  {
    email: email,
    nickname: nickname <--- This will result in an error because the key will exist, but be undefined
  }
)

Since JSON.stringify doesn't stringify undefined values, this can get missed when logging the update expression to console, but if you use the following logic (for example I added this to the transactWrite function document-client-v3.js):
const replacer = (key, value) => value === undefined ? null : value
console.log(JSON.stringify(input, replacer))

This will result in the following update expression:
{
    "Update": {
        "TableName": "Users",
        "Key": {
            "PK": "1",
            "SK": "info"
        },
        "ReturnConsumedCapacity": null,
        "UpdateExpression": "SET #UE_email= :UE_email, #UE_nickname = :UE_nickname",
        "ExpressionAttributeNames": {
            "#UE_email": "email",
            "#UE_nickname": "nickname",
        },
        "ExpressionAttributeValues": {
            ":UE_email"<EMAIL_ADDRESS>            ":UE_nickname": null, <--- THIS IS ACTUALLY UNDEFINED AND CAUSES AN ERROR
        }
    }
}

Note that this problem is entirely avoided if you never include a key with the undefined value in the update body:
const userId = 1
const email =<EMAIL_ADDRESS>const nickname = undefined
entityManger.update(
  User,
  {userId: userId},
  {
    ...(email && {email: email}),
    ...(nickname && {nickname: nickname}),  <--- does not cause an error since the undefined key will not be spread
  }
)


When FlatList wrapped inside ScollView  , onEndReached doesnt work expected.

[ ] Review the documentation: https://facebook.github.io/react-native
[ ] Search for existing issues: https://github.com/facebook/react-native/issues
[ ] Use the latest React Native release: https://github.com/facebook/react-native/releases

Environment
Environment:
OS: Linux 4.10
Node: 10.10.0
Yarn: 1.9.4
npm: 6.4.1
Watchman: Not Found
Xcode: N/A
Android Studio: Not Found
Packages: (wanted => installed)
react: 16.3.1 => 16.3.1
react-native: https://github.com/expo/react-native/archive/sdk-27.0.0.tar.gz => 0.55.2
Description
I have a multiple views that are wrapped inside ScrollView so that everthing gets scrollable.  I have read that it is not recommended to use ScrollView as a parent wrapper . But I want things to be entirely scrollable together. So I see issue with onEndReached , it is invoking on load. Can any one help me with this where all components are scrollable and also flatlist pagination is done in right way.
Reproducible Demo
<ScrollView>
<CoverPhoto/>
<Text>{title}</Text>
<FlatList
data={this.state.people}
renderItem={}
keyExtractor={(item, index) => item.personId}
onEndReached={() => {
console.log(" On End Reached");
this.getPeople();
}}
onEndReachedThreshold={0.01}
ListFooterComponent={() => {
if (this.state.hasNextPage)
return ;
else return null;
}}
/>
</ScrollView>

Why not put those top views in the header?
<FlatList
data={this.state.people}
renderItem={}
keyExtractor={(item, index) => item.personId}
onEndReached={() => {
console.log(" On End Reached");
this.getPeople();
}}
onEndReachedThreshold={0.01}
ListHeaderComponent={() => (
<View>
<CoverPhoto/>
<Text>{title}</Text>
</View>
))
ListFooterComponent={() => {
if (this.state.hasNextPage)
return ;
else return null;
}}
/>

Why not put those top views in the header?
<FlatList
data={this.state.people}
renderItem={}
keyExtractor={(item, index) => item.personId}
onEndReached={() => {
console.log(" On End Reached");
this.getPeople();
}}
onEndReachedThreshold={0.01}
ListHeaderComponent={() => (
<View>
<CoverPhoto/>
<Text>{title}</Text>
</View>
))
ListFooterComponent={() => {
if (this.state.hasNextPage)
return ;
else return null;
}}
/>

Oh yeah!! It works :+1: My header component takes more than half of the screen and yeah it scrolls along with the list.

@kelset close :)

Not solution
Header will always get re-rendered

i think this problem should be resolve, rather than avoid.

The purpose of the OP has been resolved entirely, where the request was to add static content above and below a Flatlist; there is a clear API for that use case, header and footer. Making a Flatlist with the same scroll direction as a ScrollView work, is not something that will be supported as it just doesn't make sense in relation to the FlastList's purpose: Virtualization of children, for which it needs a fixed height, otherwise how do you decide to stop rendering children?
If you can come up with a credible use case where you would require this, feel free to open a feature request and we can discuss the use case and implementation challenges in detail. I'm locking this issue for now, as it is completely resolved as far as I can see.


Only the note 7 moves

Describe your problem here.
function onBeatHit()
noteTweenX(tween, 4, getRandomInt(0, 1250), 0.1);
noteTweenX(tween3, 5, getRandomInt(0, 1250), 0.1);
noteTweenX(tween2, 6, getRandomInt(0, 1250), 0.1);
noteTweenX(tween1, 7, getRandomInt(0, 1250), 0.01)
end
Only the note 7 moves
Are you modding a build from source or with Lua?
Lua
What is your build target?
Windows x64
Did you edit anything in this build? If so, mention or summarize your changes.
No

I fixed it


Andrs-Betancourt_CU-3azapjk

Creates Endpoint for the user to Change Their Password

Task linked: CU-3azapjk Create Change Password Endpoint


Assign multiple  IPs with same netmask to one interface failed.

Under Linux or windows, it's allowed to assign multiple  IPs with same netmask to one interface.
Could you implement such a feature in dpdk-ans?
Thanks very much!

fix it in dpdk-17.05.2 ans


Documentation issue: Developer Guide cloudformation resource "name"

According to the 2020-07 Developers Guide pg 9 in referring to Cloudformation Resources:
name
The name that is associated with the AWS CloudFormation StackSets. The provided name is used to provide a more user-friendly name for a stack set.
Type: String Required: Yes
Valid Values: a-z, A-Z, 0-9, and an underscore (). Any other character is automatically replaced with an underscore ().
This should be accepting a hyphen (-) and not an underscore as hyphens are valid in stack names, and underscores are not.  Including an underscore results in the CloudformationResource build step to fail which can be tracked down in the step function:
"errorMessage": "An error occurred (ValidationError) when calling the CreateStackSet operation: 1 validation error detected: Value 'CustomControlTower-cloudformation_example_name' at 'stackSetName' failed to satisfy constraint: Member must satisfy regular expression pattern: [a-zA-Z][-a-zA-Z0-9]*"

@jdaves-ucar Thanks for bringing this to our attention. Can you please advise if you have defined the name of the stack set as 'cloudformation_example_name' in the manifest.yaml or with another special character?
Also, what is the current version of the solution are you using?
A quick review of the latest code shows that the solution does not update the stack set name with underscores.

Running 1.2.0.  The names in the manifest that failed were defined with underscores based on the incorrect documentation as cloudformation_example_name, no substitution attempted.

@jdaves-ucar Thanks for confirming the stack set name. Apologies for the confusion. We have created a ticket to update the documentation.
Please remove the underscore from the name of the stack set in the manifest file to resolve this issue.
Thanks.

@jdaves-ucar We updated the developer guide to reflect this change.


[New Component] - MudTextM3

Initial creation of MudTextM3 based on material 3 typography specifications - https://m3.material.io/styles/typography/overview

Thanks for the PR. Have two opinions.

What about using "Typo" and "Size" as parameter names?

We need to arrange API page also for the new component. And in drawer's nav menu, components should be in alphabetical order.

Thanks for the PR. Have two opinions.

What about using "Typo" and "Size" as parameter names?
We need to arrange API page also for the new component. And in drawer's nav menu, components should be in alphabetical order.

I’ll amend my pr with these items

checked in changes based on review, let me know if any additional changes are required

checked in changes based on review, let me know if any additional changes are required

Thanks, really good effort. I would only say, we can enhance the docs. We can seperate typo groups and users can easily show which ones are Display or Body one with a label.

Sorry i didn't see the second picture. Thanks again 🥇


Versions in the requirements

I was wondering if specific versions listed in the requirements file were intentionally fixed? I.e., is this version of pandas
https://github.com/ABCD-STUDY/nda-abcd-s3-downloader/blob/d57585d63f190b922953b1acfd7c6361d71a0fd0/requirements.txt#L11
specifically necessary for this tool to run? I have a much newer version and trying to uninstall it for this old version causes my computer to choke due to version incompatibilities with the newer versions of Python.
Similarly, if this requirement could be loosened, could the same be applied broadly to others?

No I haven't run into any issues with newer versions of pandas. It's more of a recommendations.txt than a requirements.txt. That said we do like to keep our software up to date, so if you run into issues with a newer version of a python package please open another issue and we will work on updating the repository accordingly.


Match BOM artifact to jenkins.version used

In #632, the jenkins.version was bumped from 2.387.3 to 2.440.1 but the BOM was not.
I discovered that during BOM release.
I cannot prove that this on its own would fix the BOM release.

Testing done
I ran mvn verify locally with no problem.

Proposed upgrade guidelines
N/A
Localizations

[ ] English
[ ] German
[ ] French
[ ] Slovak
[ ] Czech
[ ] ...

Submitter checklist

[ ] The Jira / Github issue, if it exists, is well-described.
[ ] The changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developers, depending on the change) and are in the imperative mood (see examples).

The changelog generator for plugins uses the pull request title as the changelog entry.
Fill in the Proposed upgrade guidelines section only if there are breaking changes or changes that may require extra steps from users during the upgrade.

[x] There is automated testing or an explanation that explains why this change has no tests.
[ ] New public functions for internal use only are annotated with @NoExternalUse. In case it is used by non java code the Used by {@code <panel>.jelly} Javadocs are annotated.

[ ] New or substantially changed JavaScript is not defined inline and does not call eval to ease the future introduction of Content Security Policy (CSP) directives (see documentation).
[x] For dependency updates, there are links to external changelogs and, if possible, full differentials.
[ ] For new APIs and extension points, there is a link to at least one consumer.
[ ] Any localizations are transferred to *.properties files.
[ ] Changes in the interface are documented also as examples.

Maintainer checklist
Before the changes are marked as ready-for-merge:

[ ] There is at least one (1) approval for the pull request and no outstanding requests for change.
[ ] Conversations in the pull request are over, or it is explicit that a reviewer is not blocking the change.
[ ] Changelog entries in the pull request title and/or Proposed changelog entries are accurate, human-readable, and in the imperative mood.
[ ] Proper changelog labels are set so that the changelog can be generated automatically. See also release-drafter-labels.
[ ] If the change needs additional upgrade steps from users, the upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the pull request title (see example).
[ ] java code changes are tested by automated test.

I have the same two tests failures locally. Not sure it's "flaky".

I also have several tests failing:
[ERROR] Tests run: 31, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 355.6 s <<< FAILURE! -- in org.jenkins.plugins.lockableresources.LockStepTest
[ERROR] org.jenkins.plugins.lockableresources.LockStepTest.unlockButtonWithWaitingRuns -- Time elapsed: 180.1 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 180 seconds
        at java.base/java.lang.Object.wait0(Native Method)
        at java.base/java.lang.Object.wait(Object.java:366)
        at org.jenkinsci.plugins.workflow.test.steps.SemaphoreStep.waitForStart(SemaphoreStep.java:169)
        at org.jenkins.plugins.lockableresources.LockStepTest.unlockButtonWithWaitingRuns(LockStepTest.java:767)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:656)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.lang.Thread.run(Thread.java:1583)

[INFO] Running org.jenkins.plugins.lockableresources.LockStepWithRestartTest
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 78.36 s -- in org.jenkins.plugins.lockableresources.LockStepWithRestartTest
[INFO] Running org.jenkins.plugins.lockableresources.LockableResourceApiTest
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 20.96 s <<< FAILURE! -- in org.jenkins.plugins.lockableresources.LockableResourceApiTest
[ERROR] org.jenkins.plugins.lockableresources.LockableResourceApiTest.reserveUnreserveApi -- Time elapsed: 16.03 s <<< FAILURE!
java.lang.AssertionError:

Expected: is <true>
     but: was <false>
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
        at org.jenkins.plugins.lockableresources.LockableResourceApiTest.reserveUnreserveApi(LockableResourceApiTest.java:41)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:656)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
        at java.base/java.lang.Thread.run(Thread.java:1583)

[INFO] Running org.jenkins.plugins.lockableresources.LockableResourceManagerTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.79 s -- in org.jenkins.plugins.lockableresources.LockableResourceManagerTest
[INFO] Running org.jenkins.plugins.lockableresources.LockableResourceRootActionSEC1361Test
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.96 s -- in org.jenkins.plugins.lockableresources.LockableResourceRootActionSEC1361Test
[INFO] Running org.jenkins.plugins.lockableresources.LockableResourceTest
[INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.064 s -- in org.jenkins.plugins.lockableresources.LockableResourceTest
[INFO] Running org.jenkins.plugins.lockableresources.NodesMirrorTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.34 s -- in org.jenkins.plugins.lockableresources.NodesMirrorTest
[INFO] Running org.jenkins.plugins.lockableresources.PressureTest
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.47 s -- in org.jenkins.plugins.lockableresources.PressureTest
[INFO] Running org.jenkins.plugins.lockableresources.actions.LockableResourcesRootActionTest
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 40.52 s -- in org.jenkins.plugins.lockableresources.actions.LockableResourcesRootActionTest
[INFO] Running org.jenkins.plugins.lockableresources.util.SerializableSecureGroovyScriptTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 s -- in org.jenkins.plugins.lockableresources.util.SerializableSecureGroovyScriptTest
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   LockableResourceApiTest.reserveUnreserveApi:41
Expected: is <true>
     but: was <false>
[ERROR] Errors:
[ERROR]   LockStepTest.unlockButtonWithWaitingRuns:767->Object.wait:366->Object.wait0:-2 » TestTimedOut test timed out after 180 seconds
[INFO]
[ERROR] Tests run: 165, Failures: 1, Errors: 1, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  13:56 min
[INFO] Finished at: 2024-04-19T14:58:41Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.2.5:test (default-test) on project lockable-resources: There are test failures.
[ERROR]
[ERROR] Please refer to /home/ubuntu/lockable-resources-plugin/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException


Running inference gives AttributeError: Missing attribute "MODEL

(centro) [user@remote centroids-reid]$ python inference/create_embeddings.py --config_file="configs/256_resnet50.yml" GPU_IDS [1] DATASETS.ROOT_DIR './test/probe/' TEST.IMS_PER_BATCH 128 OUTPUT_DIR './output-dir' TEST.ONLY_TEST True MODEL.PRETRAIN_PATH './logs/market1501/256_resnet50/train_ctl_model/version_0/auto_checkpoints/resnet50-19c8e357.pth'
INFO:main:Preparing data using <class 'inference_utils.ImageDataset'> dataset class
./test/probe/ 8
['./test/probe/ursula.jpg', './test/probe/angela.jpg', './test/probe/charles.jpg', './test/probe/biden.jpg', './test/probe/Yoshi.jpg', './test/probe/justin.jpg', './test/probe/macron.jpg', './test/probe/boris.jpg']
Traceback (most recent call last):
File "/home/user/anaconda3/envs/centro/lib/python3.7/site-packages/pytorch_lightning/utilities/parsing.py", line 184, in getattr
return self[key]
KeyError: 'MODEL'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "inference/create_embeddings.py", line 76, in
model = CTLModel.load_from_checkpoint(cfg.MODEL.PRETRAIN_PATH)
File "/home/user/anaconda3/envs/centro/lib/python3.7/site-packages/pytorch_lightning/core/saving.py", line 158, in load_from_checkpoint
model = cls._load_model_state(checkpoint, strict=strict, kwargs)
File "/home/user/anaconda3/envs/centro/lib/python3.7/site-packages/pytorch_lightning/core/saving.py", line 198, in _load_model_state
model = cls(_cls_kwargs)
File "./train_ctl_model.py", line 29, in init
super().init(cfg, **kwargs)
File "./modelling/bases.py", line 70, in init
self.backbone = Baseline(self.hparams)
File "./modelling/baseline.py", line 50, in init
last_stride = cfg.MODEL.LAST_STRIDE
File "/home/user/anaconda3/envs/centro/lib/python3.7/site-packages/pytorch_lightning/utilities/parsing.py", line 186, in getattr
raise AttributeError(f'Missing attribute "{key}"') from exp
AttributeError: Missing attribute "MODEL"
I can run the training but anytime I run any of the inference I receive this error.

I found a complex solve, if anyone get simplify thanks. (a condition could also allow to do this without creating a new file)

In **inference/create_embeddings.py **
CHANGE [13] from train_ctl_model import CTLModel to from test_ctl_model import CTLModel and
CHANGE [77] model = CTLModel.load_from_checkpoint(cfg.MODEL.PRETRAIN_PATH) to model = CTLModel.load_from_checkpoint(cfg.MODEL.PRETRAIN_PATH,cfg=cfg)

In /centroids-reid/
COPY train_ctl_model.py and name it test_ctl_model.py in the same directory.

In test_ctl_model.py
CHANGE [23] from modelling.bases import ModelBase to from modelling.bases_test import ModelBase

In /centroids/modelling/
COPY bases.py and name it bases_test.py.

In bases_test.py
Remove the following lines:
[75]self.contrastive_loss = TripletLoss(self.hparams.SOLVER.MARGIN, self.hparams.SOLVER.DISTANCE_FUNC)
[78]self.xent = CrossEntropyLabelSmooth(num_classes=self.hparams.num_classes)
[79-81]self.center_loss = CenterLoss(num_classes=self.hparams.num_classes, feat_dim=d_model)
[82]self.center_loss_weight = self.hparams.SOLVER.CENTER_LOSS_WEIGHT
[87]self.fc_query = torch.nn.Linear(d_model, self.hparams.num_classes, bias=False)
[88]self.fc_query.apply(weights_init_classifier)
[90]self.losses_names = ["query_xent", "query_triplet", "query_center"]
[91]self.losses_dict = {n: [] for n in self.losses_names}

This has allowed me to run create_embeddings.py

thanks @majrblais! This worked for me

Hi @majrblais @anchalgupta2412
I think you are using the wrong weights for the model.
Based on this script
python inference/create_embeddings.py --config_file="configs/256_resnet50.yml" GPU_IDS [1] DATASETS.ROOT_DIR './test/probe/' TEST.IMS_PER_BATCH 128 OUTPUT_DIR './output-dir' TEST.ONLY_TEST True MODEL.PRETRAIN_PATH './logs/market1501/256_resnet50/train_ctl_model/version_0/auto_checkpoints/resnet50-19c8e357.pth'

you are using the imagenet weight resnet50-19c8e357.pth rather than the weight which is provided by the author | link
You need to download and store the resnet50-19c8e357.pth/r50_ibn_a.pth in the models folder and pass the market1501_resnet50_256_128_epoch_120.ckpt to MODEL.PRETRAIN_PATH. Here is the example command:
python inference/create_embeddings.py \
--config_file="configs/256_resnet50.yml" \
GPU_IDS [0] \
DATASETS.ROOT_DIR '/data/my-image-dataset/' \
TEST.IMS_PER_BATCH 2 \
OUTPUT_DIR 'output-dir' \
TEST.ONLY_TEST True \
MODEL.PRETRAIN_PATH "path/to/your/market1501_resnet50_256_128_epoch_120.ckpt"

P/s: About step 6. In /home/user/anaconda3/envs/centro/lib/python3.7/site-packages/pytorch_lightning/core/saving.py
ADD [210] if "state_dict" in checkpoint: before model.load_state_dict(checkpoint['state_dict'], strict=strict)
I do not recommend you change the source code of the library, this will result in many think like wrong code behavior, hard to maintain, etc. In my opinion, I think this modification will ignore the weight and not load the weight in the model. This means you are using the model which is initialized randomly from scratch or using the Imagenet weights

@namphuongtran9196 hi i followed your instructions for create_embeddings but still am stuck with missing attribute 'MODEL'.
I also have the same issue for get_similar file. This was the error given:
`INFO:main:Preparing data using <class 'inference_utils.ImageDataset'> dataset class
D:\QingRong\centroids-reid\data\query-images 1
['D:\QingRong\centroids-reid\data\query-images\rdj-query.jpg']
Traceback (most recent call last):
File "D:\QingRong\Programs\anaconda3\envs\ctlreid\lib\site-packages\pytorch_lightning\utilities\parsing.py", line 184, in getattr
return self[key]
KeyError: 'MODEL'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\QingRong\centroids-reid\inference\create_embeddings.py", line 76, in
model = CTLModel.load_from_checkpoint(cfg.MODEL.PRETRAIN_PATH)
File "D:\QingRong\Programs\anaconda3\envs\ctlreid\lib\site-packages\pytorch_lightning\core\saving.py", line 158, in load_from_checkpoint
model = cls._load_model_state(checkpoint, strict=strict, kwargs)
File "D:\QingRong\Programs\anaconda3\envs\ctlreid\lib\site-packages\pytorch_lightning\core\saving.py", line 198, in _load_model_state
model = cls(_cls_kwargs)
File "D:\QingRong\centroids-reid.\train_ctl_model.py", line 29, in init
super().init(cfg, **kwargs)
File "D:\QingRong\centroids-reid.\modelling\bases.py", line 70, in init
self.backbone = Baseline(self.hparams)
File "D:\QingRong\centroids-reid.\modelling\baseline.py", line 50, in init
last_stride = cfg.MODEL.LAST_STRIDE
File "D:\QingRong\Programs\anaconda3\envs\ctlreid\lib\site-packages\pytorch_lightning\utilities\parsing.py", line 186, in getattr
raise AttributeError(f'Missing attribute "{key}"') from exp
AttributeError: Missing attribute "MODEL"`

Hi @CobaltConcrete
You can try to follow these commands
git clone git@github.com:mikwieczorek/centroids-reid.git
cd centroids-reid
pip install -r requirements.txt
mkdir models && wget https://download.pytorch.org/models/resnet50-19c8e357.pth -O models/resnet50-19c8e357.pth
mkdir samples && wget https://th.bing.com/th/id/OIP.muvVOaVid2C2LRH26luRggHaJX?pid=ImgDet&rs=1 -O samples/person.jpg
python inference/create_embeddings.py --config_file=configs/256_resnet50.yml GPU_IDS "[0]" DATASETS.ROOT_DIR './samples' TEST.IMS_PER_BATCH 1 OUTPUT_DIR 'outputs' TEST.ONLY_TEST True MODEL.PRETRAIN_PATH "market1501_resnet50_256_128_epoch_120.ckpt"

Maybe you will meet some problems with the CUDA device. You can try to use docker to build the library. Here is the Dockerfile
#
# Created on Fri Mar 24 2023
#
# Copyright (c) 2023 Tran Phuong Nam
# Contact<EMAIL_ADDRESS>#

FROM namphuongtran9196/pycuda:python3.8.16-cuda11.0.3-cudnn8-ubuntu20.04 AS build

ENV DEBIAN_FRONTEND noninteractive

ARG USER=vscode

ENV DEBIAN_FRONTEND noninteractive
# Update apt
RUN apt update -y
RUN apt install -y git vim nano unzip

COPY ./centroids-reid /centroids-reid

# Install python dependencies
RUN pip3 install --upgrade pip
RUN cd /centroids-reid && pip3 install -r requirements.txt


How to unit test the toast?

Is it possible to unit-test this library? Like adding a full integration test to verify the toast has been launched or maybe a way to mock the toast to be used inside a Jest unit test? If so, is it possible to add more info about it in the docs

Hey @MarlonAEC .  The fastest way to test toasts in your project is by using *ByText query.


LaTeX submission as zip includes unnecessary files

When the export is created, it looks like it is pulling things in from the entire project, not just images referenced in the tex document. We do need to process the notebooks, but we don't need to copy/write all of the files.
Any thoughts @fwkoch on how to achieve this?

This was partially completed in #674, however, that is still rendering the outputs that are in notebooks which may not make it all the way to the article. We should take a pass at defining our transforms as plugins (e.g. in #636) and then defer the writing of images until the last stage. That will also help us use and improve the plugin infrastructure.

@fwkoch is this now completely fixed with the image updates in #721?


Error while ng build with AOT

Hi,
I am using ngx-leaflet and it builds up somehow when I dont using AOT flag. But as soon as I use AOT flag it is showing me this error.
My module file:
import { LeafletModule } from '@asymmetrik/ngx-leaflet';
...

@Pipe({ name: 'safeHtml' })
export class SafeHtmlPipe implements PipeTransform {
    constructor(private sanitized: DomSanitizer) { }
    transform(value) {
        return this.sanitized.bypassSecurityTrustHtml(value);
    }
}

@NgModule({
    ...
    imports: [
       ...,
        LeafletModule.forRoot()
    ],
...
})

export class TaskModule { }

ERROR in Error encountered resolving symbol values statically. Could not resolve @asymmetrik/ngx-leaflet relative to /builds/vernuso/trust/src/app/task/task.module.ts., resolving symbol TaskModule in /builds/vernuso/trust/src/app/task/task.module.ts, resolving symbol TaskModule in /builds/vernuso/trust/src/app/task/task.module.ts, resolving symbol TaskModule in /builds/vernuso/trust/src/app/task/task.module.ts
ERROR in multi ./src/styles.css ./src/css/mapping.css ./~/quill/dist/quill.core.css ./~/quill/dist/quill.snow.css ./~/leaflet/dist/leaflet.css
Module not found: Error: Can't resolve '/builds/vernuso/trust/node_modules/leaflet/dist/leaflet.css' in '/builds/vernuso/trust'
@ multi ./src/styles.css ./src/css/mapping.css ./~/quill/dist/quill.core.css ./~/quill/dist/quill.snow.css ./~/leaflet/dist/leaflet.css

Am I missing anything here? Please help.

Are you using ng cli? Can you share some more code from the project? If I had to guess, maybe there's an issue with the tsconfig for your aot build?

Yes, I am using cli. Below is the packages.json

"dependencies": {
    "@agm/core": "^1.0.0-beta.1",
    "@angular/common": "^2.3.1",
    "@angular/compiler": "^2.3.1",
    "@angular/core": "^2.3.1",
    "@angular/flex-layout": "^2.0.0-rc.1",
    "@angular/forms": "^2.3.1",
    "@an
```gular/http": "^2.3.1",
    "@angular/material": "^2.0.0-beta.2",
    "@angular/platform-browser": "^2.3.1",
    "@angular/platform-browser-dynamic": "^2.3.1",
    "@angular/router": "^3.3.1",
    "@angular2-material/core": "^2.0.0-alpha.7-4",
    "@asymmetrik/ngx-leaflet": "^2.5.1",
    "@types/leaflet": "^1.2.0",
    "leaflet": "^1.2.0",
  },

and tsconfig.json

{
"compilerOptions": {
"baseUrl": "",
"declaration": false,
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
"lib": ["es6", "dom"],
"mapRoot": "./",
"module": "es6",
"moduleResolution": "node",
"outDir": "../dist/out-tsc",
"sourceMap": true,
"target": "es5",
"typeRoots": [
"../node_modules/@types"
]
}
}

I can't reproduce this. But, I have an example ng cli project that I'm building for a tutorial I'm writing on how to use this plugin. You can find the repo here: https://github.com/Asymmetrik/ngx-leaflet-tutorial-ngcli
You could compare your setup to mine and see where they diverge. That might help you track down your issue.
The error output you're seeing might be related

I can see you are using angular 4. Is this the requirement? I am using angular 2.3
Please suggest.

I don't think I'm doing anything ng4 specific. More specifically, I don't think I changed anything when I went from 2.x to 4. You could try changing the github project I referenced with ng 2.x and see if it works.

Did you get anywhere with this? I just tested the tutorial with ng2 and it seems fine

Gonna close this cause I haven't heard anything in a while. Let me know if there are any specific issues that you encounter with AOT.


Undefined method `Schema' for Dry::Validation:Module

I'm new to dry-rb so please excuse me if this is not the right place for this bug report.
Describe the bug
According to the 1.0.0 documentation, one of the ways to declare a schema is in the following:
require 'dry-validation'

schema = Dry::Validation.Schema do
  required(:email).filled(:str?)
  required(:age).filled(:int?, gt?: 18)
end

This breaks for me with the following error:

undefined method `Schema' for Dry::Validation:Module (NoMethodError)

To Reproduce
The following script fails with the mentioned error
require 'bundler/inline'

gemfile do
  source 'https://rubygems.org'
  gem 'dry-validation', '1.0.0'
end

require 'dry-validation'

schema = Dry::Validation.Schema do
  required(:email).filled(:str?)
  required(:age).filled(:int?, gt?: 18)
end

Expected behavior
It should work as documented.
Your environment

Affects my production application: NO
Ruby version: 2.6.3
OS: Manjaro Linux

Sorry about the confusion. This is old syntax. We had a bug on the website where it showed incorrect version number next to the docs. I've switched to 1.0.0 for all the default docs so this should no longer be an issue. See dry-rb/dry-rb.org/issues/312 for more context.


Investigate missing records

When loading a lot of data via Webhook => Clickhouse sink
There seems to be data missing, not sure if it's because the laptop goes to sleep and the Clickhouse sink isn't pushing data or Clickhouse stops processing
🤷♂  but seems like we would need to re-add wait_for_async_insert=1

>1M is fine, but I had to restart a bunch of times and I was missing blocks of data
The issue is there's data loss in between the process
The Webhook continues to send data, but for some reason it's not being stored in Clickhouse (not sure where it's getting lost)

The common factor that seems to be creating issues is restarts. Stopping the process midway does not store everything fully. Restarting it then skips these records that are not fully saved.
A solution would be to store the last completed cursor in the database and continue from there.


Adding 'None' text to show when user has no groups

Fixes #
Improving UX.
Describe the changes you have made in this PR -
Adding a text widget showing 'None' when the user is not a member of any group. Improves UX.
Screenshots of the changes (If any) -
Before :

After:

Note: Please check Allow edits from maintainers. if you would like us to assist in the PR.

Instead of "None", it would be better if we give a informative message. For example, For Groups You're in, "You are currently not in any groups, create or join a group to get started".

Instead of "None", it would be better if we give a informative message. For example, For Groups You're in, "You are currently not in any groups, create or join a group to get started".

Won't it look a bit redundant and repetitive having the same message(you are currently not in any groups....) shown under both the Groups You Mentor and Groups You're In headings when we have given a Make New Group button just above it?
I guess we can remove the headings altogether and just display a single "You're not in any groups..." message in this case. Something like this :

Let me know your thoughts.

Instead of "None", it would be better if we give a informative message. For example, For Groups You're in, "You are currently not in any groups, create or join a group to get started".

Won't it look a bit redundant and repetitive having the same message(you are currently not in any groups....) shown under both the Groups You Mentor and Groups You're In headings when we have given a Make New Group button just above it?
I guess we can remove the headings altogether and just display a single "You're not in any groups..." message in this case. Something like this :

Let me know your thoughts.

I was just pointing out example for "Groups your In". I wanted your to decide the informative messages rather than just keeping just "None". Also, it's better to have both Mentored Groups and Joined Groups separate. It helps teachers/others to moderate.

Does this look fine??

Does this look fine??

Better! Also, get some views in the #ux channel.

It would be better if we could add a small illustration to this as well. also remove the grey bg, it looks odd

we can show the two sections in a way we display projects created by the user and the ones starred by the user?

Since the PR is now stale and due to new changes, I am closing this PR. Feel free to open another one anytime :)
Thanks for Contributing 👍


fleep.get returns nothing regarding type, extension, or mime

I have the following code
with open("back number - sister.mp3", "rb") as file:
    info = fleep.get(file.read(128))
    print(info.type)
    print(info.extension)
    print(info.mime)

But the output shows nothing
[]
[]
[]

I tried exiftool & it shows the correct info
ExifTool Version Number         : 11.11
File Name                       : back number - sister.mp3
Directory                       : .
File Size                       : 3.8 MB
File Modification Date/Time     : 2018:12:15 18:57:00+09:00
File Access Date/Time           : 2019:02:02 18:00:44+09:00
File Inode Change Date/Time     : 2018:12:15 18:57:28+09:00
File Permissions                : rw-r--r--
File Type                       : MP3
**File Type Extension             : mp3**
MIME Type                       : audio/mpeg
MPEG Audio Version              : 1
Audio Layer                     : 3
Audio Bitrate                   : 128 kbps
Sample Rate                     : 44100
Channel Mode                    : Joint Stereo
MS Stereo                       : On
Intensity Stereo                : Off
Copyright Flag                  : False
Original Media                  : True
Emphasis                        : None
Encoder                         : LAME3.99r
Lame VBR Quality                : 4
Lame Quality                    : 3
Lame Method                     : CBR
Lame Low Pass Filter            : 17 kHz
Lame Bitrate                    : 128 kbps
Lame Stereo Mode                : Joint Stereo
ID3 Size                        : 128
Title                           :
Artist                          :
Album                           :
Year                            :
Comment                         :
Genre                           : None
Duration                        : 0:04:07 (approx)

Link to the trouble file back number - sister.mp3
Could you please help?

I added a new PR #10 for this issue. Could you please check?

the same issue for me: Many mp3 files just deliver an empty result.

I added a new PR #10 for this issue. Could you please check?

Thank you. fix the issue for me