BraceLabel text can not use chinese

Description of bug / unexpected behavior
I found that text is converted to tex mobject by default in BraceLabel. The default latex does not support Chinese
Therefore I use config["tex_template"]=TexTemplateLibrary.ctex
Switch to ctex that supports Chinese
The following error will appear
IndexError                                Traceback (most recent call last)
Cell In[10], line 1
----> 1 get_ipython().run_cell_magic('manim', '-qm  Test', '\nconfig["tex_template"]=TexTemplateLibrary.ctex\nprint(config["tex_template"].tex_compiler)\nclass AcuteAngle(Scene):\n    def construct(self):\n        line_1=Line([-1,0,0],[1,0,0])\n        text=BraceLabel(line_1,text="中文")\n        self.add(line_1,text)\n')

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\IPython\core\interactiveshell.py:2475, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
   2473 with self.builtin_trap:
   2474     args = (magic_arg_s, cell)
-> 2475     result = fn(*args, **kwargs)
   2477 # The code below prevents the output from being displayed
   2478 # when using magics with decodator @output_can_be_silenced
   2479 # when the last Python token in the expression is a ';'.
   2480 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\manim\utils\ipython_magic.py:141, in ManimMagic.manim(self, line, cell, local_ns)
    139     SceneClass = local_ns[config["scene_names"][0]]
    140     scene = SceneClass(renderer=renderer)
--> 141     scene.render()
    142 finally:
    143     # Shader cache becomes invalid as the context is destroyed
    144     shader_program_cache.clear()

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\manim\scene\scene.py:223, in Scene.render(self, preview)
    221 self.setup()
    222 try:
--> 223     self.construct()
    224 except EndSceneEarlyException:
    225     pass

File <string>:4, in construct(self)

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\manim\mobject\svg\brace.py:204, in BraceLabel.__init__(self, obj, text, brace_direction, label_constructor, font_size, buff, brace_config, **kwargs)
    202     self.label = self.label_constructor(font_size=font_size, *text, **kwargs)
    203 else:
--> 204     self.label = self.label_constructor(str(text), font_size=font_size)
    206 self.brace.put_at_tip(self.label)
    207 self.add(self.brace, self.label)

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\manim\mobject\text\tex_mobject.py:280, in MathTex.__init__(self, arg_separator, substrings_to_isolate, tex_to_color_map, tex_environment, *tex_strings, **kwargs)
    273 try:
    274     super().__init__(
    275         self.arg_separator.join(self.tex_strings),
    276         tex_environment=self.tex_environment,
    277         tex_template=self.tex_template,
    278         **kwargs,
    279     )
--> 280     self._break_up_by_substrings()
    281 except ValueError as compilation_error:
    282     if self.brace_notation_split_occurred:

File D:\ProgramData\anaconda\envs\manimtest\lib\site-packages\manim\mobject\text\tex_mobject.py:354, in MathTex._break_up_by_substrings(self)
    352     sub_tex_mob.submobjects = [VectorizedPoint()]
    353     last_submob_index = min(curr_index, len(self.submobjects) - 1)
--> 354     sub_tex_mob.move_to(self.submobjects[last_submob_index], RIGHT)
    355 else:
    356     sub_tex_mob.submobjects = self.submobjects[curr_index:new_index]

IndexError: list index out of range

Also I need to add that when I use the example in the documentation, the result is normal, proving that my ctex is installed correctly
%%manim -qm  LaTeXTemplateLibrary

class LaTeXTemplateLibrary(Scene):
    def construct(self):
        tex = Tex('Hello 你好 \\LaTeX', tex_template=TexTemplateLibrary.ctex, font_size=144)
        self.add(tex)

Expected behavior
After switching to ctex, Chinese and English are displayed normally
How to reproduce the issue
from manim import *
#Manim Community v0.17.3

%%manim -qm  Test

config["tex_template"]=TexTemplateLibrary.ctex
print(config["tex_template"].tex_compiler)
class AcuteAngle(Scene):
    def construct(self):
        line_1=Line([-1,0,0],[1,0,0])
        text=BraceLabel(line_1,text="中文")
        self.add(line_1,text)

Additional media files

Images/GIFs

Logs
Terminal output

PASTE HERE OR PROVIDE LINK TO https://pastebin.com/ OR SIMILAR

System specifications
System Details

OS (with version, e.g., Windows 10 v2004 or macOS 10.15 (Catalina)):
RAM:
Python version (python/py/python3 --version):
Installed modules (provide output from pip list):

PASTE HERE

LaTeX details

LaTeX distribution (e.g. TeX Live 2020):
Installed LaTeX packages:

FFMPEG
Output of ffmpeg -version:
PASTE HERE

Additional comments

I think I may have found the point of the problem.
when using
text=BraceLabel(line_1,text="Chinese")
tex file is like this
\documentclass[preview]{standalone}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{dsfont}
\usepackage{setspace}
\usepackage{tipa}
\usepackage{relsize}
\usepackage{textcomp}
\usepackage{mathrsfs}
\usepackage{calligra}
\usepackage{wasysym}
\usepackage{ragged2e}
\usepackage{physics}
\usepackage{xcolor}
\usepackage{microtype}
\usepackage[UTF8]{ctex}
\linespread{1}

\begin{document}

\begin{align*}
chinese
\end{align*}

\end{document}

and the result is correct. So on this basis, I modified the "Chinese" to "chinese中文版"
the MiKTex console will output
This is XeTeX, Version 3.141592653-2.6-0.999995 (MiKTeX 23.4) (preloaded format=xelatex.fmt)
 restricted \write18 enabled.
entering extended mode
(8581c9c711b1f36b.tex
LaTeX2e <2022-11-01> patch level 1
L3 programming layer <2023-03-30>
(D:\MiKTeX\tex/latex/standalone\standalone.cls
Document Class: standalone 2022/10/10 v1.3b Class to compile TeX sub-files stan
dalone
(D:\MiKTeX\tex/latex/tools\shellesc.sty)
(D:\MiKTeX\tex/generic/iftex\ifluatex.sty
(D:\MiKTeX\tex/generic/iftex\iftex.sty))
(D:\MiKTeX\tex/latex/xkeyval\xkeyval.sty
(D:\MiKTeX\tex/generic/xkeyval\xkeyval.tex
(D:\MiKTeX\tex/generic/xkeyval\xkvutils.tex
(D:\MiKTeX\tex/generic/xkeyval\keyval.tex))))
(D:\MiKTeX\tex/latex/standalone\standalone.cfg)
(D:\MiKTeX\tex/latex/base\article.cls
Document Class: article 2022/07/02 v1.4n Standard LaTeX document class
(D:\MiKTeX\tex/latex/base\size10.clo)) (D:\MiKTeX\tex/latex/preview\preview.sty
(D:\MiKTeX\tex/latex/luatex85\luatex85.sty)
(D:\MiKTeX\tex/latex/preview\prtightpage.def)))
(D:\MiKTeX\tex/generic/babel\babel.sty (D:\MiKTeX\tex/generic/babel\xebabel.def
)
*************************************
* Local config file bblopts.cfg used
*
(D:\MiKTeX\tex/latex/arabi\bblopts.cfg)
(D:\MiKTeX\tex/latex/babel-english\english.ldf))
(D:\MiKTeX\tex/generic/babel/locale/en\babel-english.tex)
(D:\MiKTeX\tex/latex/base\inputenc.sty

Package inputenc Warning: inputenc package ignored with utf8 based engines.

) (D:\MiKTeX\tex/latex/base\fontenc.sty (D:\MiKTeX\tex/latex/lm\t1lmr.fd))
(D:\MiKTeX\tex/latex/lm\lmodern.sty) (D:\MiKTeX\tex/latex/amsmath\amsmath.sty
For additional information on amsmath, use the `?' option.
(D:\MiKTeX\tex/latex/amsmath\amstext.sty
(D:\MiKTeX\tex/latex/amsmath\amsgen.sty))
(D:\MiKTeX\tex/latex/amsmath\amsbsy.sty)
(D:\MiKTeX\tex/latex/amsmath\amsopn.sty))
(D:\MiKTeX\tex/latex/amsfonts\amssymb.sty
(D:\MiKTeX\tex/latex/amsfonts\amsfonts.sty))
(D:\MiKTeX\tex/latex/doublestroke\dsfont.sty)
(D:\MiKTeX\tex/latex/setspace\setspace.sty) (D:\MiKTeX\tex/latex/tipa\tipa.sty
(D:\MiKTeX\tex/latex/base\fontenc.sty (D:\MiKTeX\tex/latex/tipa\t3enc.def
(D:\MiKTeX\tex/latex/lm\t1lmss.fd)))) (D:\MiKTeX\tex/latex/relsize\relsize.sty)
 (D:\MiKTeX\tex/latex/base\textcomp.sty)
(D:\MiKTeX\tex/latex/jknappen\mathrsfs.sty)
(D:\MiKTeX\tex/latex/fundus-calligra\calligra.sty)
(D:\MiKTeX\tex/latex/wasysym\wasysym.sty)
(D:\MiKTeX\tex/latex/ragged2e\ragged2e.sty)
(D:\MiKTeX\tex/latex/physics\physics.sty
(D:\MiKTeX\tex/latex/l3packages/xparse\xparse.sty
(D:\MiKTeX\tex/latex/l3kernel\expl3.sty
(D:\MiKTeX\tex/latex/l3backend\l3backend-xetex.def))))
(D:\MiKTeX\tex/latex/xcolor\xcolor.sty
(D:\MiKTeX\tex/latex/graphics-cfg\color.cfg)
(D:\MiKTeX\tex/latex/graphics-def\xetex.def)
(D:\MiKTeX\tex/latex/graphics\mathcolor.ltx))
(D:\MiKTeX\tex/latex/microtype\microtype.sty
(D:\MiKTeX\tex/latex/etoolbox\etoolbox.sty)
(D:\MiKTeX\tex/latex/microtype\microtype-xetex.def)
(D:\MiKTeX\tex/latex/microtype\microtype.cfg))
(D:\MiKTeX\tex/latex/ctex\ctex.sty (D:\MiKTeX\tex/latex/ctex\ctexhook.sty)
(D:\MiKTeX\tex/latex/ctex\ctexpatch.sty) (D:\MiKTeX\tex/latex/base\fix-cm.sty
(D:\MiKTeX\tex/latex/base\ts1enc.def))
(D:\MiKTeX\tex/latex/ctex/config\ctexopts.cfg)
(D:\MiKTeX\tex/latex/ctex/engine\ctex-engine-xetex.def
(D:\MiKTeX\tex/xelatex/xecjk\xeCJK.sty
(D:\MiKTeX\tex/latex/l3packages/xtemplate\xtemplate.sty)
(D:\MiKTeX\tex/latex/fontspec\fontspec.sty
(D:\MiKTeX\tex/latex/fontspec\fontspec-xetex.sty
(D:\MiKTeX\tex/latex/base\fontenc.sty)
(D:\MiKTeX\tex/latex/fontspec\fontspec.cfg)))
(D:\MiKTeX\tex/xelatex/xecjk\xeCJK.cfg)))
(D:\MiKTeX\tex/latex/zhnumber\zhnumber.sty
(D:\MiKTeX\tex/latex/zhnumber\zhnumber-utf8.cfg))
(D:\MiKTeX\tex/latex/ctex/scheme\ctex-scheme-chinese.def
(D:\MiKTeX\tex/latex/ctex/config\ctex-name-utf8.cfg))
(D:\MiKTeX\tex/latex/tools\indentfirst.sty)
(D:\MiKTeX\tex/latex/ctex\ctex-c5size.clo)
(D:\MiKTeX\tex/latex/ctex/fontset\ctex-fontset-windows.def))
(D:\MiKTeX\tex/latex/ctex/config\ctex.cfg) (8581c9c711b1f36b.aux)
(D:\MiKTeX\tex/latex/tipa\t3cmr.fd)
Preview: Fontsize 10.53937pt
Preview: PDFoutput 1
(D:\MiKTeX\tex/latex/microtype\mt-cmr.cfg) (D:\MiKTeX\tex/latex/lm\ot1lmr.fd)
(D:\MiKTeX\tex/latex/lm\omllmm.fd) (D:\MiKTeX\tex/latex/lm\omslmsy.fd)
(D:\MiKTeX\tex/latex/lm\omxlmex.fd) (D:\MiKTeX\tex/latex/amsfonts\umsa.fd)
(D:\MiKTeX\tex/latex/microtype\mt-msa.cfg)
(D:\MiKTeX\tex/latex/amsfonts\umsb.fd)
(D:\MiKTeX\tex/latex/microtype\mt-msb.cfg)
(D:\MiKTeX\tex/latex/jknappen\ursfs.fd)

LaTeX Font Warning: Font shape `U/rsfs/m/n' in size <10.53937> not available
(Font)              size <10.95> substituted on input line 26.

(D:\MiKTeX\tex/latex/wasysym\uwasy.fd)

LaTeX Font Warning: Font shape `U/wasy/m/n' in size <10.53937> not available
(Font)              size <10.95> substituted on input line 26.

Missing character: There is no 中 in font rm-lmr10!
Missing character: There is no 文 in font rm-lmr10!
Missing character: There is no 中 in font rm-lmr10!
Missing character: There is no 文 in font rm-lmr10!
Preview: Tightpage 0 0 0 0
[1] (8581c9c711b1f36b.aux)

LaTeX Font Warning: Size substitutions with differences
(Font)              up to 0.41063pt have occurred.

 )
Output written on 8581c9c711b1f36b.pdf (1 page).
SyncTeX written on 8581c9c711b1f36b.synctex.gz.
Transcript written on 8581c9c711b1f36b.log

Looks like a font error.
and if  I use the tex
%%manim -qm  LaTeXTemplateLibrary

class LaTeXTemplateLibrary(Scene):
    def construct(self):
        tex = Tex('你好', tex_template=TexTemplateLibrary.ctex,font_size=144)
        self.add(tex)

and the tex file like this:
\documentclass[preview]{standalone}

\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{lmodern}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{dsfont}
\usepackage{setspace}
\usepackage{tipa}
\usepackage{relsize}
\usepackage{textcomp}
\usepackage{mathrsfs}
\usepackage{calligra}
\usepackage{wasysym}
\usepackage{ragged2e}
\usepackage{physics}
\usepackage{xcolor}
\usepackage{microtype}
\usepackage[UTF8]{ctex}
\linespread{1}

\begin{document}

**\begin{center}
你好
\end{center}**

\end{document}

the result is correct.
so {align*} +chinese character will result in the "missing character" font error.
I don't know latex syntax very well, so can anyone help me with this problem?

so {align*} +chinese character will result in the "missing character" font error. I don't know latex syntax very well, so can anyone help me with this problem?

even in ctex the Chinese characters are only supported in text mode. The labels in the Brace() object are typeset in math-mode. Even standard European text would end up wrongly formatted if you place it into a MathTex() rather than a Tex().
Two solutions:
a) in BraceLabel() add the argument label_constructor=Tex in order to switch from math-mode to text-mode
b) alternatively you can place your label as "\text{中文}"

so {align*} +chinese character will result in the "missing character" font error. I don't know latex syntax very well, so can anyone help me with this problem?

even in ctex the Chinese characters are only supported in text mode. The labels in the Brace() object are typeset in math-mode. Even standard European text would end up wrongly formatted if you place it into a MathTex() rather than a Tex().
Two solutions: a) in BraceLabel() add the argument label_constructor=Tex in order to switch from math-mode to text-mode
b) alternatively you can place your label as "\text{中文}"
https://docs.manim.community/en/stable/reference/manim.mobject.svg.brace.BraceLabel.html?highlight=bracelabel
You are welcome to discuss questions like this on the Discord server https://discord.com/channels/581738731934056449/1019649969596153968

I have used your solution 1. It worsk well. Thank you very much.
Then I have two questions to ask:

I saw in the documentation that Tex is a subclass of MathTex, so I don't think there is much difference between the two? I don't quite understand the main difference between the two modes math-mode and text-mode. Are there more reference articles?
I still need to use Chinese tex in the future, such as

Tex(r"$\dfrac{你好}{中文}$", font_size=28,tex_environment="center")

In TeXworks I use lualatex as tex_compiler to compile correctly

So I specify config["tex_template"].tex_compiler="lualatex" in config
But it still makes an error in the function self._break_up_by_substrings() in manim\mobject\text\tex_mobject.py.
File D:\ProgramData\anaconda\envs\ManimCommunity\Lib\site-packages\manim\mobject\text\tex_mobject.py:449, in Tex.__init__(self, arg_separator, tex_environment, *tex_strings, **kwargs)
    446 def __init__(
    447     self, *tex_strings, arg_separator="", tex_environment="center", **kwargs
    448 ):
--> 449     super().__init__(
    450         *tex_strings,
    451         arg_separator=arg_separator,
    452         tex_environment=tex_environment,
    453         **kwargs,
    454     )

File D:\ProgramData\anaconda\envs\ManimCommunity\Lib\site-packages\manim\mobject\text\tex_mobject.py:280, in MathTex.__init__(self, arg_separator, substrings_to_isolate, tex_to_color_map, tex_environment, *tex_strings, **kwargs)
    273 try:
    274     super().__init__(
    275         self.arg_separator.join(self.tex_strings),
    276         tex_environment=self.tex_environment,
    277         tex_template=self.tex_template,
    278         **kwargs,
    279     )
--> 280     self._break_up_by_substrings()
    281 except ValueError as compilation_error:
    282     if self.brace_notation_split_occurred:

File D:\ProgramData\anaconda\envs\ManimCommunity\Lib\site-packages\manim\mobject\text\tex_mobject.py:354, in MathTex._break_up_by_substrings(self)
    352     sub_tex_mob.submobjects = [VectorizedPoint()]
    353     last_submob_index = min(curr_index, len(self.submobjects) - 1)
--> 354     sub_tex_mob.move_to(self.submobjects[last_submob_index], RIGHT)
    355 else:
    356     sub_tex_mob.submobjects = self.submobjects[curr_index:new_index]

IndexError: list index out of range

Please come over to the Discord, there is a Chinese community as well!
And it's much more convenient to answer questions there in the forum.
both Tex() and MathTex() send their arguments to the external LaTeX program to render the text. LaTeX has to principal modes of typesetting, "math mode" and "text mode". The difference between Tex() and MathTex() is whether they start in text mode or math mode (to keep it very short here). But you can switch between the two using $...$ or \(...\) to invoke math mode and \text{...} to get to text mode.
So instead of
Tex(r"$\dfrac{你好}{中文}$", font_size=28,tex_environment="center")

you need to do
MathTex(r"\dfrac{\text{你好}}{\text{中文}}", font_size=28)

or
Tex(r"$\dfrac{\text{你好}}{\text{中文}}$", font_size=28)

Even with Latin-character text one would have to use the \text inside a math environment (which you absolutely need for the \frac ) in order to get the correct typesetting of text in the nominator and denominator.
MathTex(r"\dfrac{Hello}{Chinese language}", font_size=28)

MathTex(r"\dfrac{\text{Hello}}{\text{Chinese language}}", font_size=28)

I don't know what luatex makes differently, but if you copy the complete settings from your luatex template over to your Tex-templates in Manim, and also select luatex as the tex compiler to be used you should and will get the same result as in TexWorks, because Manim does nothing else then sending the arguments of your commands to the selected LaTeX-compiler.
https://docs.manim.community/en/stable/faq/general.html?highlight=discord#where-can-i-find-more-resources-for-learning-manim

Interestingly the ctex template in Manim appears to be made for latex and not for xelatex, yet it invokes xelatex, which on my system results in metafont errors. The following code just runs fine on my system:
class AcuteAngle(Scene):
    def construct(self):
        config.tex_template=TexTemplateLibrary.ctex
        config.tex_template.tex_compiler="latex"
        config.tex_template.output_format=".dvi"
        line_1=Line([-1,0,0],[1,0,0])
        text=BraceLabel(line_1,text=r"\text{中文}")
        self.add(line_1,text)
        frac =  Tex(r"$\dfrac{\text{你好}}{\text{中文}}$", font_size=50).to_edge(UP)
        self.add(frac)

And now with also fonts SimHei and KeiTi installed, the script also runs with xelatex as LaTeX compiler
class AcuteAngle(Scene):
    def construct(self):
        config.tex_template=TexTemplateLibrary.ctex
        line_1=Line([-1,0,0],[1,0,0])
        text=BraceLabel(line_1,text=r"\text{中文}")
        self.add(line_1,text)
        frac =  Tex(r"$\dfrac{\text{你好}}{\text{中文}}$", font_size=50).to_edge(UP)
        self.add(frac)

Interestingly the ctex template in Manim appears to be made for latex and not for xelatex, yet it invokes xelatex, which on my system results in metafont errors. The following code just runs fine on my system:
class AcuteAngle(Scene):
    def construct(self):
        config.tex_template=TexTemplateLibrary.ctex
        config.tex_template.tex_compiler="latex"
        config.tex_template.output_format=".dvi"
        line_1=Line([-1,0,0],[1,0,0])
        text=BraceLabel(line_1,text=r"\text{中文}")
        self.add(line_1,text)
        frac =  Tex(r"$\dfrac{\text{你好}}{\text{中文}}$", font_size=50).to_edge(UP)
        self.add(frac)

Thank you very much for your solution, it seems that I should learn more about latex. BTW I can't join you through the discord link you provided.  I haven't used discord before, but I think this shows that I don't have access @uwezi

Interestingly the ctex template in Manim appears to be made for latex and not for xelatex, yet it invokes xelatex, which on my system results in metafont errors. The following code just runs fine on my system:
class AcuteAngle(Scene):
    def construct(self):
        config.tex_template=TexTemplateLibrary.ctex
        config.tex_template.tex_compiler="latex"
        config.tex_template.output_format=".dvi"
        line_1=Line([-1,0,0],[1,0,0])
        text=BraceLabel(line_1,text=r"\text{中文}")
        self.add(line_1,text)
        frac =  Tex(r"$\dfrac{\text{你好}}{\text{中文}}$", font_size=50).to_edge(UP)
        self.add(frac)

Thank you very much for your solution, it seems that I should learn more about latex. BTW I can't join you through the discord link you provided. I haven't used discord before, but I think this shows that I don't have access @uwezi

I have actually found the link in the Manim documentation, so there's no need for you to worry about it anymore.


EXAMPLE NOT WORKING

Hi Marvin ,
I'm trying to use your library and start with the example provided , but it doesn't work .
I set the new level from the serial monitor and nothing happens , level always stays @ 255   .
I'm using Arduino IDE 1.8.7
Can you please help me in this matter.
Thank you .

So I have done some more research and it looks like the example is working on a ESP 8266 , any body knows why is not working on Uno / Nano ? anything to  do with the eeprom ?

Hi,
Maybe your problem is related to  issue #23 ?
Change  these lines in the example
const unsigned long upCourseTime = 30 * 1000;
const unsigned long downCourseTime = 45 * 1000;

for example like that
const unsigned long upCourseTime = 30u * 1000;
const unsigned long downCourseTime = 45u * 1000;

After these changes it's working on Mega.
You could also uncomment
#define DEBUG
in Shutters.h to see what's going on.

I have discovered why is not working , It's something to do with the Eeprom .
It works as intended on ESP 8266 , but on Nano , Uno , Mega I had to comment everything related about the ESP 8266 Eeprom (flash)  specific settings like :
#ifdef ESP8266
EEPROM.begin(512);
#endif
Maybe somebody has an Idea what going on , but this was the fix , to have it working on non ESP devices .


What about continuous webjobs

As i understand it the recurring webjobs is using scheduler under the hood, isnt that correct? Is my recurring webjobs going to stop working?
This only talks about the scheduled webjob, but the continuous webjob is still fully functional, correct?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a8d495c3-9afc-f758-5d04-c47a89b9d895
Version Independent ID: 5b95e62c-7a05-353d-5766-62ccd4cd4833
Content: Migrate from Azure Scheduler to Azure Logic Apps - Azure Scheduler
Content Source: articles/scheduler/migrate-from-scheduler-to-logic-apps.md
Service: scheduler
GitHub Login: @derek1ee
Microsoft Alias: deli

@mslot Thank you for your query, we will look into it and get back to you at the earliest.

Or is the CRON expressions that can be used when adding a webjob through the portal actually not a part of the scheduler? I am afraid of using the recurring bits if it stops working.

@mslot  Apologies for delayed response on this, i have reached out to our internal product team on this and will keep you updated.

@BharathNimmala-MSFT thanks. It might be that recurring Web jobs is a new feature, and the "old" way is to use a scheduler. But it is kind of hard to read from docs if using a recurring Web job is a bad practice or a logic app should be used. I think it is a naming thing where recurring Web jobs clash with scheduling a Web job because it is two different features but it is hard to tell what feature is referred when reading this doc, or questions on fx stackoverflow etc.

@mslot Thanks for sharing additional context on this, will request our document author @derek1ee to weigh in your feedback and further clarify and also see if document needs to be updated accordingly.

The Webjob schedule trigger has a few implementations, depending on when you created those you may have a schedule that's backed by Scheduler or you may not.
If you have a Webjob schedule that's backed by a Scheduler resource, you will need to update it. The easiest way to verify is to navigate to Scheduler resources in the subscription in the attached e-mail, and see if they are calling into your Webjob.

@derek1ee It is a pretty new Web job and I haven't any email. I don't have a scheduler resource in my subscription. My webjob is scheduling, does this mean that there is a new way for it to schedule?


Erlang 18 compatibility

Changed erlang:now() (deprecated in Erlang 18) to os:timestamp.
This way it should be compatible with Erlang 18 as well as older versions.
(Old PR: https://github.com/gleber/erlcloud/pull/212)

Thanks for the fix


Add support for private repos

It would be useful if people could build Binders for private repos. This is especially common in educational settings, where repos need to be kept private to prevent students from cheating on each other (or to prevent cheating by students in the following year!)
This should just requires letting the web app / build system clone private repos. Let's try to spec out here what that should look like.
cc @rahuldave

Did this get implemented by any chance?


Remove istio.io/rev from injection template

Fixes #33237 by not overwriting istio.io/rev on injection. Adds revision to the sidecar.istio.io/status annotation.
[ ] Configuration Infrastructure
[ ] Docs
[x] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[ ] Does not have any user-facing changes. This may include CLI changes, API changes, behavior changes, performance improvements, etc.

@Monkeyanator: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
0ccfc5034dadb6cf79256b5f03cbae5658be281f
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
0ccfc5034dadb6cf79256b5f03cbae5658be281f
link
/test release-notes_istio

integ-pilot-k8s-tests_istio
0ccfc5034dadb6cf79256b5f03cbae5658be281f
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
0ccfc5034dadb6cf79256b5f03cbae5658be281f
link
/test integ-pilot-k8s-tests_istio

release-notes_istio
adde5765f331f8c010688ee55e60509e2d229ec5
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
adde5765f331f8c010688ee55e60509e2d229ec5
link
/test release-notes_istio

integ-pilot-multicluster-tests_istio
adde5765f331f8c010688ee55e60509e2d229ec5
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
adde5765f331f8c010688ee55e60509e2d229ec5
link
/test integ-pilot-multicluster-tests_istio

gencheck_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test gencheck_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
adde5765f331f8c010688ee55e60509e2d229ec5
link
/test integ-pilot-multicluster-tests_istio

gencheck_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test gencheck_istio

unit-tests_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

gencheck_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test gencheck_istio

unit-tests_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test unit-tests_istio

integ-pilot-multicluster-tests_istio
4c2ee480d2dabeb5ee45ad658c5030d50a245b67
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
5d52d82fba891248b3788ab43b3db59d952566d6
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
5d52d82fba891248b3788ab43b3db59d952566d6
link
/test integ-pilot-multicluster-tests_istio

unit-tests_istio
fcefc80c85f55bf99d4990345a9ed20f7b85e23b
link
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

unit-tests_istio
fcefc80c85f55bf99d4990345a9ed20f7b85e23b
link
/test unit-tests_istio

integ-pilot-multicluster-tests_istio
fcefc80c85f55bf99d4990345a9ed20f7b85e23b
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@Monkeyanator: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-multicluster-tests_istio
43d1fd64998fd7c1379c8b75c08a163f8dd59198
link
/test integ-pilot-multicluster-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

Random note: this seems to have broken way less than expected. istioctl uninstall actually uses /syncz/ to warn about removing in-use control plane and only 1 integ test relied on the label

Hi, as the istio.io/rev has been removed, what's the best way to get pods in old version or canary version when doing a canary upgrade?


Updating dataframe

Codecov Report

Merging #122 into master will decrease coverage by 0.18%.
The diff coverage is 94.77%.

@@            Coverage Diff             @@
##           master     #122      +/-   ##
==========================================
- Coverage   93.75%   93.56%   -0.19%
==========================================
  Files          12       12
  Lines        1392     1430      +38
==========================================
+ Hits         1305     1338      +33
- Misses         87       92       +5

Impacted Files
Coverage Δ

streamz/sources.py
80.51% <100%> (-0.25%)
:arrow_down:

streamz/batch.py
100% <100%> (ø)
:arrow_up:

streamz/core.py
95.6% <100%> (ø)
:arrow_up:

streamz/collection.py
87.7% <89.47%> (+0.06%)
:arrow_up:

streamz/dataframe.py
97.68% <95.2%> (-1.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update abc102d...4433fa0. Read the comment docs.


Constraint failure while replaying some mainnet TX with access list

Some tx on mainnet like
[0xceeac2b8e36e6de05bb0a625781f044e4be833b917c984584c3839755e78ce83]
(https://etherscan.io/tx/0xceeac2b8e36e6de05bb0a625781f044e4be833b917c984584c3839755e78ce83)
or
0x94604c865c331ae66a80eb199264b3f317e4aab125a3463aea527c6f77e12898
would encounter constraint error and lookup failure while being replayed by our tools. The error looks lile:

[2024-03-12T12:53:18Z ERROR mainnet] ERR: Constraint 4 ('section_rlc accumulation: r = rand^20, section_rlc' = section_rlc * r + field_rlc'') in gate 1212 ('tx access list') is not satisfied in Region 310 ('tx table aux') at offset 669
- Column('Advice', 3 - )@0 = 1
- Column('Advice', 349 - )@0 = 0
- Column('Advice', 370 - )@0 = 1
- Column('Advice', 371 - )@0 = 1
- Column('Advice', 373 - )@1 = 0x21561e53104b2a06ab619c4b89402a3f6bad6f6d993a155c982422de19ec08b0
- Column('Advice', 375 - )@0 = 1
- Column('Advice { phase: Phase(1) }', 351 - )@0 = 0xb6dd20903504285b86a714980fe1b17c0e4eaa4f1a842619fbea211f5079d0a
- Column('Advice { phase: Phase(1) }', 351 - )@1 = 0x17ee1648645d019d4d554685dd551353fcde3b1b7da642b4bed33dd93d5b8218
- Column('Fixed', 0 - )@0 = 1
[2024-03-12T12:53:18Z ERROR mainnet] ERR: Lookup Sig table lookup(index: 92) is not satisfied in Region 310 ('tx table aux') at offset 12

It can be reproduced by following command (suppose we have a geth node with debug API availiable in localhost:8545
RUST_LOG=debug TX_ID=0x94604c865c331ae66a80eb199264b3f317e4aab125a3463aea527c6f77e12898 GETH0_URL=http://localhost:8545 cargo test --features=scroll --release test_mock_prove_tx

Guess the constraint issue if caused by empty storageKey slot in access list.
Tx with access list without empty slot of storageKey would not raise this issue

it is signature lookup fail in tx circuit,  waitting Pr ray's tx circuit audit fix merge then re-test.

after run testing against latest develop branch,  guess not signature data error.
      "v": 1,
        "r": "0xb747fec9bd97c46da0af4750d0c97423392fd045512b566d72519e102173d6d6",
        "s": "0x1390614df6318f45bcc1d189d846135d843982e5de35913a289376b20066b07e",

seems access list encoding issue, will dig more.

it is caused by access list data, if tx has no access list , can pass after signature fix in tx circuit .


Failed: 'consul_z1/0 (xxxxxxxx)' is not running after update. Review logs for failed jobs: consul_agent, metron_agent (00:10:32)

Hi evryone
I have a issue in bosh deploy.
I can deploy all the instances but i have this problem.
Started preparing deployment > Preparing deployment. Done (00:00:01)
Started preparing package compilation > Finding packages to compile. Done (00:00:01)
Started updating instance consul_z1 > consul_z1/6b67631f-0beb-4143-bfed-16b97910041b (0) (canary). Failed: 'consul_z1/0 (6b67631f-0beb-4143-bfed-16b97910041b)' is not running after update. Review logs for failed jobs: consul_agent, metron_agent (00:10:32)
Error 400007: 'consul_z1/0 (6b67631f-0beb-4143-bfed-16b97910041b)' is not running after update. Review logs for failed jobs: consul_agent, metron_agent
Task 41 error
consul_z1/0 (6b67631f-0beb-4143-bfed-16b97910041b)* failing
consul_agent failing
metron_agent failing
consul_z1/6b67631f-0beb-4143-bfed-16b97910041b:/var/vcap/data/sys/log# tail -f metron_agent/metron.log
2017/08/04 23:47:35 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/04 23:54:23 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/04 23:56:03 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/04 23:57:51 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/04 23:59:21 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/05 00:01:01 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/05 00:02:41 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
2017/08/05 00:04:21 Could not use GRPC creds for client: failed to load keypair: tls: failed to find any PEM data in certificate input
consul_z1/65d2eb14-b072-49e9-9b31-46daac3cb863:/var/vcap/sys/log# tail -f consul_agent/consul_agent.stderr.log
error during start: timeout exceeded: "Get http://<IP_ADDRESS>:8500/v1/agent/self: dial tcp <IP_ADDRESS>:8500: getsockopt: connection refused"
error during start: timeout exceeded: "Get http://<IP_ADDRESS>:8500/v1/agent/self: dial tcp <IP_ADDRESS>:8500: getsockopt: connection refused"
error during start: timeout exceeded: "Get http://<IP_ADDRESS>:8500/v1/agent/self: dial tcp <IP_ADDRESS>:8500: getsockopt: connection refused"
error during start: timeout exceeded: "Get http://<IP_ADDRESS>:8500/v1/agent/self: dial tcp <IP_ADDRESS>:8500: getsockopt: connection refused"
error during start: timeout exceeded: "Get http://<IP_ADDRESS>:8500/v1/agent/self: dial tcp <IP_ADDRESS>:8500: getsockopt: connection refused"
Can anyone can help me?

Try generating your certs this way: https://github.com/cloudfoundry-incubator/consul-release/blob/master/scripts/generate-certs

hi rowanjacobs,
I try with https://github.com/cloudfoundry-incubator/consul-release/blob/master/scripts/generate-certs but i have the same issue.


Only work around dup lib issue on appleclang

This was causing linker errors using home-brew clang on Mac.

Cherry-picked to v10.0.x


HttpPostRequestDecoder may cause memory leak

set HttpDataFactory to use disk = false like below:

HttpDataFactory factory = new DefaultHttpDataFactory(false, CharsetUtil.UTF_8);

and then create an HttpPostRequestDecoder instance:

HttpPostRequestDecoder httpDecoder = new HttpPostRequestDecoder(factory, httpRequest);

after decoder is destroyed the reference count of httpRequest is incorrect.
in the method createAttribute of DefaultHttpDataFactory, MemoryAttribute is not added to HttpData list:
@Override
    public Attribute createAttribute(HttpRequest request, String name, long definedSize) {
        if (useDisk) {
            Attribute attribute = new DiskAttribute(name, definedSize, charset, baseDir, deleteOnExit);
            attribute.setMaxSize(maxSize);
            List<HttpData> list = getList(request);
            list.add(attribute);
            return attribute;
        }
        if (checkSize) {
            Attribute attribute = new MixedAttribute(name, definedSize, minSize, charset, baseDir, deleteOnExit);
            attribute.setMaxSize(maxSize);
            List<HttpData> list = getList(request);
            list.add(attribute);
            return attribute;
        }
        MemoryAttribute attribute = new MemoryAttribute(name, definedSize);
        attribute.setMaxSize(maxSize);
        return attribute;
    }

so on destroying
@Override
    public void cleanAllHttpData() {
        Iterator<Entry<HttpRequest, List<HttpData>>> i = requestFileDeleteMap.entrySet().iterator();
        while (i.hasNext()) {
            Entry<HttpRequest, List<HttpData>> e = i.next();

            // Calling i.remove() here will cause "java.lang.IllegalStateException: Entry was removed"
            // at e.getValue() below

            List<HttpData> list = e.getValue();
            for (HttpData data : list) {
                data.release();
            }

            i.remove();
        }
    }

the MemoryAttributes created above are not released.
I'm not sure if it is a bug.
Expected behavior
the reference count of httpRequest before decoding should equals to the one after decoder is destroyed.
Actual behavior
if the original reference count of http request is 1, and the http request contains 2 fields , then after destroying the reference count is 3 !
Steps to reproduce
Minimal yet complete reproducer code (or URL to code)
send a form :
public void testSubmitForm() throws IOException {
        String boundary = "---------------------------ABCDEFGHIJKLMNOPQRSTUVWXYZ";
        URL url = new URL("http://<IP_ADDRESS>:1084/test");
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setConnectTimeout(5000);
        connection.setReadTimeout(30000);
        connection.setDoOutput(true);
        connection.setDoInput(true);
        connection.setUseCaches(false);
        connection.setRequestMethod("POST");
        connection.setRequestProperty("Connection", "Keep-Alive");
        connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.1; zh-CN; rv:<IP_ADDRESS>)");
        connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);

        try (OutputStream out = new DataOutputStream(connection.getOutputStream())) {
            out.write(("\r\n--" + boundary + "\r\n").getBytes(StandardCharsets.UTF_8));
            out.write("Content-Disposition: form-data; name=\"username\"\r\n\r\n".getBytes(StandardCharsets.UTF_8));
            out.write("chinnsii".getBytes(StandardCharsets.UTF_8));

            out.write(("\r\n--" + boundary + "\r\n").getBytes(StandardCharsets.UTF_8));
            out.write("Content-Disposition: form-data; name=\"password\"\r\n\r\n".getBytes(StandardCharsets.UTF_8));
            out.write("chinnsii".getBytes(StandardCharsets.UTF_8));

            byte[] endData = ("\r\n--" + boundary + "--\r\n").getBytes();
            out.write(endData);
            out.flush();
        }

        try (BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()))) {
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
        }
    }

decode to a hash map
Map<String, Object> convert(FullHttpRequest httpRequest) throws IOException {
        HttpDataFactory factory = new DefaultHttpDataFactory(false, CharsetUtil.UTF_8);
        HttpPostRequestDecoder httpDecoder = new HttpPostRequestDecoder(factory, httpRequest);
        try {
            httpDecoder.setDiscardThreshold(0);
            Map<String, Object> params = new HashMap<>();
            List<InterfaceHttpData> InterfaceHttpDataList = httpDecoder.getBodyHttpDatas();
            for (InterfaceHttpData data : InterfaceHttpDataList) {
                if (data != null) {
                    if (InterfaceHttpData.HttpDataType.FileUpload == data.getHttpDataType()) {
                        FileUpload fileUpload = (FileUpload) data;
                        byte[] bytes = fileUpload.get();
                        if (!fileUpload.isInMemory()) {
                            fileUpload.delete();
                        }
                        params.put(data.getName(), file);
                    }
                    if (data.getHttpDataType() == InterfaceHttpData.HttpDataType.Attribute) {
                        Attribute attribute = (Attribute) data;
                        params.put(attribute.getName(), attribute.getValue());
                    }
                }
            }
            return params;
        } finally {
            System.out.println("74------------->" + httpRequest.refCnt());
            httpDecoder.destroy();
            System.out.println("76------------>" + httpRequest.refCnt());
        }
}

where 74--------------> is 4, and 76----------------->is 3
Netty version
4.1.51.Final
JVM version (e.g. java -version)
1.8
OS version (e.g. uname -a)
MAC

Hi, I'm trying to reproduce your code in a unit test but I'm not finding ContentConverterException nor UploadedFile. My version is 4.1.61.Final-SNAPSHOT (currently the last in branch 4.1) Are those some "theoretical classes" or is this in a previous version.
Thanks in advance for your time

Hi, I'm trying to reproduce your code in a unit test but I'm not finding ContentConverterException nor UploadedFile. My version is 4.1.61.Final-SNAPSHOT (currently the last in branch 4.1) Are those some "theoretical classes" or is this in a previous version?
Thanks in advance for your time
Disclaimer: This may be my first contribution to the project. Please forgive my newbie questions

thank you for replying. sorry for my posting, UploadedFile and ContentConverterException is my local definition, I will modify the code.

I'm terribly sorry for not being able to read your code correctly, but file variable is not being resolved in convert(), did you meant bytes declared before and not being used after?

byte[] bytes = fileUpload.get();
if (!fileUpload.isInMemory()) {
    fileUpload.delete();
}
params.put(data.getName(), file);

The API says:
 /**
   * Remove all InterfaceHttpData from virtual File storage from clean list for all requests
   */
  void cleanAllHttpData();

So MemoryAttribute is not concerned by this method.
Just for note, for both Memory or Disk or Mixed mode factories, the release has to be done as:
  for (InterfaceHttpData httpData: decoder.getBodyHttpDatas()) {
      httpData.release();
      factory.removeHttpDataFromClean(request, httpData);
  }
  factory.cleanAllHttpData();
  decoder.destroy();

Relates to https://github.com/netty/netty/issues/8640

@jameskleeh  Yes and no... It is the current behavior.
I give the right way to clean up, whatever Memory, Disk or Mixed mode in my last comment.

@fredericBregier So, should this issue be discarded?

@aepedraza I think so.
Note however on your example code:
 byte[] bytes = fileUpload.get(); // This get all content from File if any to memory (could lead to Out Of Memory)
 if (!fileUpload.isInMemory()) {
     fileUpload.delete();
 }
 params.put(data.getName(), file) // I don't know to what file point to?

I would recommend to free Memory, Mixed, Disk based factory as the following, once all stuffs are done.
 for (InterfaceHttpData httpData: decoder.getBodyHttpDatas()) {
       httpData.release();
       factory.removeHttpDataFromClean(request, httpData);
 }
 factory.cleanAllHttpData();
 decoder.destroy();

@devilyard This could be related to #10973 and might be fixed by #11001. Can you try out version 4.1.60.Final or newer, and report back if that fixes the issue?

@devilyard (ignore my previous comment – the web page was out of date when I posted) Does Frederics help solve the issue you are seeing? If so, then I think we can close this issue.

yes, i tried the latest 4. 1. 62 version,  the issue is fixed


Custom Object Detection - ssd-mobilenet-v2 with TRT optmization

Hi,
I successfully converted my frozen_inference_graph.pb to an uff file and created and Engine following the https://github.com/AastaNV/TRT_object_detection instructions.
It's running very well with an inference time of 25ms, but I would like to try it with detectnet-camera.py (it's pretty organized and easy to use),  what are the commands that I should use to run on this file?
I have the following files:
ssd_mobilenet_v2_coco_2018_03_29 - network
tmp.uff - temporary uff file
TRT_ssd_mobilenet_v2_coco_2018_03_29.bin - optimized file used for inference
coco.py - list with Classes
Thanks in Advance.

Hi, how do i start to collect and train my custom dataset? Can you help me for what i need to do this? Thanks.

Hi Deniz,
I followed this page https://tensorflow-object-detection-api-tutorial.readthedocs.io/en/latest/training.html, I used the ssd_mobilenet_v2_coco_2018_03_29 model that has a very good accuracy and it's very fast.
Then, I optmized the model following these instructions https://github.com/AastaNV/TRT_object_detection

Hi Dusty,
I tried to edit the detectNet.h/cpp files adding my custom model close to the SSD_MOBILENET_V2 info as you mentioned on this issue, on ./jetson_inference/c/detecNet.cpp I added the information on line 260-261 (just below the SSD_MOBILENET_V2 info) as per below:
else if( networkType == SSD_MOBILENET_V2 )
return Create("networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff", "networks/SSD-Mobilenet-v2/ssd_coco_labels.txt", threshold, "Input", Dims3(3,300,300), "NMS", "NMS_1", maxBatchSize, precision, device, allowGPUFallback);
else if( networkType == SSD_MOBILENET_V2_WEEDS )
return Create("networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff", "networks/SSD-Mobilenet-v2-Weeds/voc-weed-labels.txt", threshold, "Input", Dims3(3,300,300), "NMS", "NMS_1", maxBatchSize, precision, device, allowGPUFallback);
on the same file I added o  line 315-216 the following info:
else if( strcasecmp(modelName, "ssd-mobilenet-v2") == 0 || strcasecmp(modelName, "coco-ssd-mobilenet-v2") == 0 || strcasecmp(modelName, "ssd-mobilenet") == 0 )
type = detectNet::SSD_MOBILENET_V2;
else if( strcasecmp(modelName, "ssd-mobilenet-v2-weeds") == 0 || strcasecmp(modelName, "coco-ssd-mobilenet-v2-weeds") == 0 || strcasecmp(modelName, "weeds") == 0 )
type = detectNet::SSD_MOBILENET_V2_WEEDS;
Inside the file ./jetson-inference/c/detecNet.h I inserted the following info on line 197:
SSD_MOBILENET_V2_WEEDS	/**< SSD Inception-v2-Weeds UFF model, trained on MS-COCO */
Then, I noticed that I had to change my engine name to tmp_weeds.uff.1.1.GPU.FP16.engine, just a continuation of my .uff file tmp_weeds.uff.
Finally I used the following commands:
python3 detectnet-camera.py --network=/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff --input_blob=Input --output_blob=NMS --output_count=NMS --class_labels=/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/voc-weed-labels.txt --camera=/dev/video1
and I got the following error:
device GPU, /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff initialized.
W = 7  H = 100  C = 1
detectNet -- maximum bounding boxes:  100
detectNet -- loaded 4 class info entries
detectNet -- number of object classes:  4
jetson.utils -- PyCamera_New()
jetson.utils -- PyCamera_Init()
[gstreamer] initialized gstreamer, version <IP_ADDRESS>
[gstreamer] gstCamera attempting to initialize with GST_SOURCE_NVARGUS, camera /dev/video1
[gstreamer] gstCamera pipeline string:
v4l2src device=/dev/video1 ! video/x-raw, width=(int)1280, height=(int)720, format=YUY2 ! videoconvert ! video/x-raw, format=RGB ! videoconvert !appsink name=mysink
[gstreamer] gstCamera successfully initialized with GST_SOURCE_V4L2, camera /dev/video1
jetson.utils -- PyDisplay_New()
jetson.utils -- PyDisplay_Init()
[OpenGL] glDisplay -- X screen 0 resolution:  1280x800
[OpenGL] glDisplay -- display device initialized
[gstreamer] opening gstCamera for streaming, transitioning pipeline to GST_STATE_PLAYING
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> videoconvert1
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> videoconvert0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> v4l2src0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert1
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer changed state from READY to PAUSED ==> videoconvert0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> v4l2src0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> videoconvert0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> v4l2src0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg stream-start ==> pipeline0
[gstreamer] gstCamera onPreroll
[gstreamer] gstCamera -- allocated 16 ringbuffers, 2764800 bytes each
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer msg async-done ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline0
[gstreamer] gstCamera -- allocated 16 RGBA ringbuffers
python3: nmsPlugin.cpp:106: virtual int nvinfer1::plugin::DetectionOutput::enqueue(int, const void* const*, void**, void*, cudaStream_t): Assertion `status == STATUS_SUCCESS' failed.
Aborted (core dumped)
When I followed this instruction (link) I managed to make my CUSTOM detection work on detectNet model, but I don't think it's the proper way to make it work.
Am I missing some command or other thing to make my custom model work by itself without substituting the defautl ssd-mobilenet-v2?
Thanks.

Then, I noticed that I had to change my engine name to tmp_weeds.uff.1.1.GPU.FP16.engine, just a continuation of my .uff file tmp_weeds.uff.

Hi @leandrovrabelo , you should delete tmp_weeds.uff.1.1.GPU.FP16.engine and allow the application to generate it the first time it runs.   When you renamed it, it is probably using the old TensorRT engine from previous model.
If you delete it, and run the application again, is it able to generate the TensorRT engine file?
Also, I recommend that you test on a static image (detectnet-console) first before trying camera on a new model.

Hi Dusty,
I tested it on detecnet-console, I deleted the engine that I created and here is what happened:
[TRT]   TensorRT version 6.0.1
[TRT]   loading NVIDIA plugins...
[TRT]   Plugin Creator registration succeeded - GridAnchor_TRT
[TRT]   Plugin Creator registration succeeded - GridAnchorRect_TRT
[TRT]   Plugin Creator registration succeeded - NMS_TRT
[TRT]   Plugin Creator registration succeeded - Reorg_TRT
[TRT]   Plugin Creator registration succeeded - Region_TRT
[TRT]   Plugin Creator registration succeeded - Clip_TRT
[TRT]   Plugin Creator registration succeeded - LReLU_TRT
[TRT]   Plugin Creator registration succeeded - PriorBox_TRT
[TRT]   Plugin Creator registration succeeded - Normalize_TRT
[TRT]   Plugin Creator registration succeeded - RPROI_TRT
[TRT]   Plugin Creator registration succeeded - BatchedNMS_TRT
[TRT]   Could not register plugin creator:  FlattenConcat_TRT in namespace:
[TRT]   completed loading NVIDIA plugins.
[TRT]   detected model format - UFF  (extension '.uff')
[TRT]   desired precision specified for GPU: FASTEST
[TRT]   requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]   native precisions detected for GPU:  FP32, FP16
[TRT]   selecting fastest native precision for GPU:  FP16
[TRT]   attempting to open engine cache file /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff.1.1.GPU.FP16.engine
[TRT]   cache file not found, profiling network model on device GPU
[TRT]   device GPU, loading /usr/bin/ /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff
[TRT]   UFFParser: Parsing Input[Op: Input].
[TRT]   UFFParser: Input -> [1,1,1]
[TRT]   UFFParser: Applying order forwarding to: Input
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/weights[Op: Const].
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/weights -> [3,3,3,32]
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/weights
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/Conv2D[Op: Conv]. Inputs: Input, FeatureExtractor/MobilenetV2/Conv/weights
[TRT]   FeatureExtractor/MobilenetV2/Conv/Conv2D: kernel weights has count 864 but 288 was expected
[TRT]   FeatureExtractor/MobilenetV2/Conv/Conv2D: count of 864 weights in kernel, but kernel dimensions (3,3) with 1 input channels, 32 output channels and 1 groups were specified. Expected Weights count is 1 * 3*3 * 32 / 1 = 288
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/Conv2D -> []
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/Conv2D
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma[Op: Const].
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma -> [32]
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta[Op: Const].
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta -> [32]
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean[Op: Const].
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean -> [32]
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance[Op: Const].
[TRT]   UFFParser: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance -> [32]
[TRT]   UFFParser: Applying order forwarding to: FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance
[TRT]   UFFParser: Parsing FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNorm[Op: BatchNorm]. Inputs: FeatureExtractor/MobilenetV2/Conv/Conv2D, FeatureExtractor/MobilenetV2/Conv/BatchNorm/gamma, FeatureExtractor/MobilenetV2/Conv/BatchNorm/beta, FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_mean, FeatureExtractor/MobilenetV2/Conv/BatchNorm/moving_variance
[TRT]   UffParser: Parser error: FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNorm: The input to the Scale Layer is required to have a minimum of 3 dimensions.
[TRT]   failed to parse UFF model '/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff'
[TRT]   device GPU, failed to load /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff
detectNet -- failed to initialize.
jetson.inference -- detectNet failed to load built-in network '/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff'
PyTensorNet_Dealloc()
Traceback (most recent call last):
File "detectnet-console.py", line 51, in
net = jetson.inference.detectNet(opt.network, sys.argv, opt.threshold)
Exception: jetson.inference -- detectNet failed to load network
jetson.utils -- freeing CUDA mapped memory
When I use the engine with name tmp_weeds.uff.1.1.GPU.FP16.engine see what happens:
[TRT]   TensorRT version 6.0.1
[TRT]   loading NVIDIA plugins...
[TRT]   Plugin Creator registration succeeded - GridAnchor_TRT
[TRT]   Plugin Creator registration succeeded - GridAnchorRect_TRT
[TRT]   Plugin Creator registration succeeded - NMS_TRT
[TRT]   Plugin Creator registration succeeded - Reorg_TRT
[TRT]   Plugin Creator registration succeeded - Region_TRT
[TRT]   Plugin Creator registration succeeded - Clip_TRT
[TRT]   Plugin Creator registration succeeded - LReLU_TRT
[TRT]   Plugin Creator registration succeeded - PriorBox_TRT
[TRT]   Plugin Creator registration succeeded - Normalize_TRT
[TRT]   Plugin Creator registration succeeded - RPROI_TRT
[TRT]   Plugin Creator registration succeeded - BatchedNMS_TRT
[TRT]   Could not register plugin creator:  FlattenConcat_TRT in namespace:
[TRT]   completed loading NVIDIA plugins.
[TRT]   detected model format - UFF  (extension '.uff')
[TRT]   desired precision specified for GPU: FASTEST
[TRT]   requested fasted precision for device GPU without providing valid calibrator, disabling INT8
[TRT]   native precisions detected for GPU:  FP32, FP16
[TRT]   selecting fastest native precision for GPU:  FP16
[TRT]   attempting to open engine cache file /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff.1.1.GPU.FP16.engine
[TRT]   loading network profile from engine cache... /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff.1.1.GPU.FP16.engine
[TRT]   device GPU, /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff loaded
[TRT]   Deserialize required 3516711 microseconds.
[TRT]   device GPU, CUDA engine context initialized with 3 bindings
[TRT]   binding -- index   0
-- name    'Input'
-- type    FP32
-- in/out  INPUT
-- # dims  3
-- dim #0  3 (SPATIAL)
-- dim #1  300 (SPATIAL)
-- dim #2  300 (SPATIAL)
[TRT]   binding -- index   1
-- name    'NMS'
-- type    FP32
-- in/out  OUTPUT
-- # dims  3
-- dim #0  1 (SPATIAL)
-- dim #1  100 (SPATIAL)
-- dim #2  7 (SPATIAL)
[TRT]   binding -- index   2
-- name    'NMS_1'
-- type    FP32
-- in/out  OUTPUT
-- # dims  3
-- dim #0  1 (SPATIAL)
-- dim #1  1 (SPATIAL)
-- dim #2  1 (SPATIAL)
[TRT]   binding to input 0 Input  binding index:  0
[TRT]   binding to input 0 Input  dims (b=1 c=3 h=300 w=300) size=1080000
[TRT]   binding to output 0 NMS  binding index:  1
[TRT]   binding to output 0 NMS  dims (b=1 c=1 h=100 w=7) size=2800
device GPU, /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff initialized.
W = 7  H = 100  C = 1
detectNet -- maximum bounding boxes:  100
detectNet -- loaded 4 class info entries
detectNet -- number of object classes:  4
Segmentation fault (core dumped)

Thank you so much for your answer.

Hi Dusty, i'm still trying to figure out how to make my custom model work properly without substituting the default ssd-mobilinet-v2 model.
Do you think that this error is related to some dimentions that are set wrong?
'[TRT] UffParser: Parser error: FeatureExtractor/MobilenetV2/Conv/BatchNorm/FusedBatchNorm: The input to the Scale Layer is required to have a minimum of 3 dimensions.
[TRT] failed to parse UFF model '/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff'
[TRT] device GPU, failed to load /usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff
detectNet -- failed to initialize.
jetson.inference -- detectNet failed to load built-in network '/usr/local/bin/networks/SSD-Mobilenet-v2-Weeds/tmp_weeds.uff'
PyTensorNet_Dealloc()
Traceback (most recent call last):
File "detectnet-console.py", line 51, in
net = jetson.inference.detectNet(opt.network, sys.argv, opt.threshold)
Exception: jetson.inference -- detectNet failed to load network
jetson.utils -- freeing CUDA mapped memory'
if it really don't work, should I retrain my model with https://github.com/dusty-nv/pytorch-ssd or should I try the depth branch of jetson inference first? https://github.com/dusty-nv/jetson-inference/tree/depth

Hello, great work by  leandrovrabelo.
Does this conversion from Tensorflow 1.x to .uff and engine file work for every Tensorflow1 models from the zoo or is it specific for Mobilenet-SSD and Inception-SSD?

hi @ThiloFink , I don't know, I just tested on these models.


Conveyors render incorrectly when viewed at an angle

Description
When viewed at roughly a 45 degree angle, conveyors render with the wrong graphic.
Reproduction
Rotate the cargo shuttle 45 degrees and view it from the station.
Alternatively, spin your view with Num7/9 and watch them dance.
Screenshots

containment field beams do the same thing


output results of the same trained model are inconsistent

I have found the reason. In the methond  DRLAgent.DRL_prediction,   the parameter deterministic must be set as following:
action, _states = model.predict(trade_obs, deterministic=True)


Cannot access logstash environmental variables

I installed logstash via Docker and I keep getting the error when I am trying to access an environmental variable locally on the machine. I've tried adding it in /etc/environment and /etc/default/logstash both getting the same error. Any idea where to store this variable on the docker installation?
I'm using the variable in an input in logstash called ${Timestamp}

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Cannot evaluate ${Timestamp}. Replacement variable Timestamp is not defined in a Logstash secret store or as an Environment entry and there is no default value given.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:50:in `block in replace_placeholders'", "org/jruby/RubyString.java:2541:in `gsub'", "/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:35:in `replace_placeholders'", "/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:23:in `deep_replace'", "/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:82:in `block in config_init'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:81:in `config_init'", "/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:60:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:233:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:166:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:71:in `plugin'", "(eval):8:in `'", "org/jruby/RubyKernel.java:994:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}

What I'm trying to do in the end is I have a node script that finds the most recent Timestamp in my SQL db so then I can import only new ES documents to that SQL db. Once the node script runs, it needs to save that timestamp locally for LS to access to use as a filter.

When I ran logstash as sudo -u root /usr/share/logstash/bin/logstash then it could access /etc/environment folder with the timestamp in there. Problem solved.


Craft 3 module generator zip file is empty

Generated zip file has only .craftplugin file. This one is named site and has Service, Twig Extension, Variables, with comments turned on.
sitemodule.zip

Is this still happening for anyone @marionnewlevant @boscho87 ?

Seems fixed will check tomorrow

😅👍🏾


fix: util/mkinstallvars.pl mistreated LDLIBS on Unix (and Windows)

Don't do comma separation on those platforms.
Fixes #24986

@floppym, could you try this patch and confirm that it fixes your issue (#24986)?

Seems to work for me. After patching openssl and recompiling, I got this:
$ pkg-config --static --libs libcrypto
-lcrypto -ldl -pthread -Wl,--push-state,--as-needed,-latomic,--pop-state

I also confirm this resolves the issue.

This pull request is ready to merge

merged to master and openssl-3.3.  Thank you!


Add files via upload

@03lizchavez Looks like you tried uploading a file directly to the textEncoding-Hub, and it created a branch from the main repo.
This isn't quite what we want you to do. We want you to be making the changes from your local computer, using git commands. When you use these, you keep a local folder that you keep connected to the repository as it changes with everyone's contributions. When you add files by upload, they can come from anywhere on your computer. You update only the remote (online web) repo, and your local computer's clone (version of the repo) falls out of sync--you aren't pulling in changes from others and merging them with your own.
So, yes--use the shell, and at your clone of this textEncoding-Hub use the git commands to git pull (update your clone of the repo). Then save your XML file in the directory where we asked you to store it, and use the git commands to add, commit, and push it up. That will let you effectively label your change with the commit message, and make it part of the logged history of the git repo on both your computer and the remote web version.

You can always "add files via upload", but the problem with this is your local clone of the repo falls out of sync with the remote. People use that "add files via upload" feature when they really aren't working with git tools and have no other way to send files. That's not you--though let me know if you need help with your git commands working. Sometimes there's a little confusion at first or something we need to do to set up your git config on a new computer.


Create SFC.
Bind the SG to SFC.
Now delete the SG.
Delete DA
Get an error message - Distributed Appliance is currently protecting a workload
This suggests that probably there is a SG being protected, whereas, in reality we need to delete the SFC to delete the DA.

Additional Information
WORKAROUND - delete the SFC and then DA delete will be successful
Environment

OSC Version/Build: Version: 1.0.0-SNAPSHOT (Build:112-g42e1905, 2017-10-11T18:46:02Z)
Cloud Provider Info:
Plugin Info:
Other:

Status

[ ] Reproduced
[ ] In Progress
[ ] Validated     =>    Verified in Version/Build:

please Handle #544 as part of this change


Change field name in result of some command to fix English mistakes

Fields to rename in result of some command:

list host, list service, show service, show host...

rename activate to active

show host

rename childs to children

list poller

rename is_activate to is_active

hello,
I renamed childs to children but for "is_activate" and "activate" I don't changed because it is write like that in centreon's documentation.
Thank you for your contribution to improving the tool.


Added a clone method to BasisSet and subclasses.

If I remember correctly, it was because I needed a way to make a copy of the full GaussianSet or SlaterSet while only working with abstract BasisSets. This seemed like the cleanest way to do it.
For instance, I used it in vtkOpenQubeElectronicData::DeepCopy to copy the basis set without needing to know what type it was.

OK, maybe we revisit this but it seems reasonable to merge for now. I would like to find a cleaner way to handle this in VTK.


Iframe accessibility

Pull Request Test Coverage Report for Build<PHONE_NUMBER>3
Details

8 of 9 (88.89%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.005%) to 99.66%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/modules/iframe/components/activateMapButton/ActivateMapButton.js
6
7
85.71%

Totals

Change from base Build<PHONE_NUMBER>2:
-0.005%

Covered Lines:
13516

Relevant Lines:
13518

💛  - Coveralls


Separate --break-on-reject logic between --date and --match-filters

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm requesting a feature unrelated to a specific site
[X] I've looked through the README
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
Currently there is a shortcoming of two conceptually different features in yt-dlp which are munged together,

date functionality
match filter functionality

These in practice represent different concerns. When either of these features is needed but not both there is no problem and all is good. However, when both of these features are required yt-dlp's current design is inadequate. Take for example the use,

I want to download all news newer then 6 months, pertaining to Ukraine.

For "I want to download all news newer then 6 months" you would use the arguments --date today-6month --break-on-reject. And, for "pertaining to Ukraine", you would use --match-title ukraine. The problem is currently when you issue the --date $foo --break-on-reject you stop downloading metadata when the videos come back out of date: this is a really nice feature and a must for many channels. When you add the --match-title it changes the behavior to stop downloading on EITHER the first video that is out of date, OR the first video that fails to match the title. If the title filter is ever more selective, you won't get the date range requested. This muxing of filters is not intuitive.
It would be my desire to allow better communication between the index and these conditions so they could be pushed down and handled better. If nothing else, the index should typically be able to handle the date predicate in the initial query (maybe index doesn't even need to be paginated) and if it can't do that we can generalize the behavior to handle it, but that this generalized behavior to handle it should be distinct from match-title.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[ ] If using API, add 'verbose': True to YoutubeDL params instead
[ ] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>[b532a3481] (zip)
[debug] Python 3.11.2 (CPython x86_64 64bit) - Linux-6.1.0-11-amd64-x86_64-with-glibc2.36 (OpenSSL 3.0.9 30 May 2023, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.3-1 (setts), ffprobe 5.1.3-1
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2022.09.24, sqlite3-2.6.0
[debug] Proxy map: {}
[debug] Loaded 1855 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version<EMAIL_ADDRESS>Current version<EMAIL_ADDRESS>Current Build Hash: cedf44a9cf3c241520a692c1006d276e6f434d4d945727c47cb1a70d32953f4a
yt-dlp is up to date<EMAIL_ADDRESS>

--match-filters already lets you filter by date AND/OR title
e.g. this will download videos that were uploaded in the past 6 months AND have "ukraine" in the title:
--match-filters "upload_date >= 20230314 & title ~= '(?i)ukraine'"

vs. this will download videos that were uploaded in the past 6 months OR have "ukraine" in the title:
--match-filters "upload_date >= 20230314" --match-filter "title ~= '(?i)ukraine'"

For "I want to download all news newer then 6 months" you would use the arguments --date today-6month --break-on-reject

You meant --dateafter, right?

FWIW we're using https://github.com/yt-dlp/yt-dlp/blob/master/yt_dlp/YoutubeDL.py#L291 behind-the-scenes for date range. AND/OR isn't the issue, it's that --break-on-reject will break eagerly for both date AND/OR match-filter.
Basically the feature request is introduce a new --break* flag, that can dictate whether to break on date reject or match filter reject.
The use case is to filter ukraine vids (but not break) in the last X daterange, and break once we're out of the date range.

A more appropriate title for this feature request is more-so Separate --break-on-reject logic between daterange and match-filters

Both --break-on-reject and the --date* options are mostly redundant now. You can achieve what you want with just --match-filter and --break-match-filter

Well, that's f*in cool. I don't know how I overlooked that in the docs, but that's on me. Thanks @pukkandan.


QA/QC | Check logistic and linear models from skilearn.

AdaptiveBridge should be aligned with sklearn data requirements and errors.


Do not send exception details in the HTTP reply

Full exception details are still written to the log file.

Coverage decreased (-0.008%) to 96.842% when pulling bf59f245e1a379c2dd7602216f2dbccdc868d133 on no-backtrace-via-http into 2ba25dea1f39f4ab4b03c53af95aa48077b392ec on master.

I'd merge it, but not enough privs.

A good compromise we use is to generate an ID with uuidv4, return that ID in the HTTP response and write the exception to the log file with the ID.
On 26 Oct 2016, at 18:21, Stefan Nordhausen<EMAIL_ADDRESS>wrote:

Full exception details are still written to the log file.
You can view, comment on, or merge this pull request online at:
https://github.com/ImmobilienScout24/afp-core/pull/5
Commit Summary
Do not send exception details in the HTTP reply
File Changes
M src/main/python/aws_federation_proxy/wsgi_api/wsgi_api.py (17)
M src/unittest/python/api_endpoint_tests.py (14)
Patch Links:
https://github.com/ImmobilienScout24/afp-core/pull/5.patch
https://github.com/ImmobilienScout24/afp-core/pull/5.diff
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.


TPPG::GetTimeInCycle etc.

Not really a bug in the code as it is an annoyance with one of our compilers at guelph. A call of ppg->GetTimeInCycle(griffin->GetTime()) was returning weirdness. The compiler was throwing no warnings about the fact that the function is defined as TPPG::GetTimeInCycle(ULong64_t) and was not treating it properly. I am going to add a TPPG::GetTimeInCycle(Double_t) which should do the appropriate fix to the time argument that is being used so one does not have to be super careful about using GetTimeStamp() rather than GetTime().

I think this is actually just caused by GetTime() now returning ns not 10*ns. Solution is to be a smarter user, or use the PPG functions built into TGRSIDetectorHit.


Setting storage class does not support deconstructors

When setting the storage, sorl does not allow for a class instance being deconstructible, which is necessary to be able to use backends with arguments.
sorl.thumbnails.images.py
cls = self.storage._wrapped.__class__

For a deconstructible class this can use self.storage.deconstruct (checking whether the class is deconstructible first to ensure backwards compatibility).
deserialize could check whether the storage is a list (deconstructible) or a single value (old method)

sorl-thumbnails now privileges a storage alias from the Django-configured STORAGES ([6fc7575]), so I guess this is no longer an issue.


v.0.6 to Daily Dev

See commit details for more information.

Hello @mikkokotila! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file talos/commands/deploy.py:

Line 75:1: W293 blank line contains whitespace

In the file talos/commands/reporting.py:

Line 71:9: E722 do not use bare 'except'

In the file talos/logging/logging_run.py:

Line 21:80: E501 line too long (83 > 79 characters)

In the file talos/metrics/entropy.py:

Line 30:80: E501 line too long (85 > 79 characters)

In the file talos/scan/scan_round.py:

Line 44:9: E722 do not use bare 'except'

In the file talos/templates/models.py:

Line 7:80: E501 line too long (80 > 79 characters)
Line 48:80: E501 line too long (80 > 79 characters)

In the file test/core_tests/test_scan.py:

Line 183:80: E501 line too long (81 > 79 characters)


Updated Jan 2022

Trello card
https://trello.com/c/T4TyDmCh/1739-get-an-adviser-multiple-changes

@Tomcowan1 Just checking if this PR should be closed - think it was superseded by https://github.com/DFE-Digital/get-teacher-training-adviser-service/pull/911 ?

Closing this PR as it was superseded by https://github.com/DFE-Digital/get-teacher-training-adviser-service/pull/911


Remove DotNet-Helix-Service direct references to nuget.org

Run-CodeInspections.ps1 and update-database.ps1 have direct references to packages on nuget.org rather than using our ADO feeds (use the internal feeds)

Release Note Category

[ ] Feature changes/additions
[ ] Bug fixes
[x] Internal Infrastructure Improvements

Release Note Description
No release notes needed.

PR: https://dev.azure.com/dnceng/internal/_git/dotnet-helix-service/pullrequest/38772


Provide example for basic authentication.

Ok, I was blind. The blocking version documents it well. I've just copied it over to the async call.


[Logging]: SD Card Logging

Add SD card logging

OTA
WIFI events
MQTT events
SNTP events

https://github.com/aqandu/airu-v2-fw/pull/18


[WSL2] "rustup update" causes "getrandom not ready" when run on initial VM boot

Your Windows build number:  (Type ver at a Windows Command Prompt)

10.0.18963.1000

What you're doing and what's happening:  (Copy&paste the full set of specific command-line steps necessary to reproduce the behavior, and their output. Include screen shots if that helps demonstrate the problem.)

Running Rust programming language environment's "rustup update" command immediately on first boot of a WSL2 environment, either by typing into the terminal during bootup or by running directly as a command:
C:\Users\brion>wsl --shutdown

C:\Users\brion>ubuntu run /home/brion/.cargo/bin/rustup update
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'
thread 'main' panicked at 'could not initialize thread_rng: All entropy sources failed (permanently unavailable); cause: getrandom not ready (not ready yet); cause: Resource temporarily unavailable (os error 11)', /cargo/registry/src/github.com-1ecc6299db9ec823/rand-0.6.5/src/rngs/thread.rs:80:17
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

What's wrong / what should be happening instead:

Command should run to completion, checking for toolchain updates and installing them if necessary. It runs fine if run a few seconds after the VM boots, just not immediately.

Strace of the failing command, if applicable:  (If some_command is failing, then run strace -o some_command.strace -f some_command some_args, and link the contents of some_command.strace in a gist here).

https://gist.github.com/brion/d8f0ec7f2fd00d069e85a4e9ce4bfbfd

Added more boot entropy in 18990.

Can confirm that in build 18990 I no longer get the low-entropy panic from rustup called in this way. (The command eventually does fail in a different way, but I'll track that down separately if it continues...) Thanks for the update!


Add sphinx documentation

Thanks @cav71 for starting to work on this :+1:
To help document the different CMake function, I suggest you look at https://github.com/python-cmake-buildsystem/python-cmake-buildsystem/issues/211

Hi,
I've back-ported the scikit style.

Hi, this is the documentation backport from the cpython branch #226

More documentation


a doubt about repoository url

hi,
is there somehow to show on the page the address of the repo so anyone can git clone path?
(gitweb shows url, and i am missing that from glist)

Just enable it in the config.ini.

didnt worked for me:
https://github.com/klaussilveira/gitlist/pull/688/files#diff-0f101cdb11b6f7d962eaf5a2ed73e3fd
https://github.com/klaussilveira/gitlist/pull/688
is it allready in the gitlist 0.4.0 ?
dunno what i did wrong :/

This feature has been introduced in ecc8e97117d5f2d7b1541f5bc7689a158579f1f and is part of v0.6.0. Please upgrade - v0.4.0 is more than 4 years old.

hi @alehaa ,
i followed the instructions:

Do not download a branch or tag from GitHub, unless you want to use the development version. The version available for download at the website already has all dependencies bundled, so you don't have to use composer or any other tool.

i downloaded from http://gitlist.org/ but it is 0.4.0 (the file says 0.5.0 but the page shows 0.4.0)
i will upgrade, should i clone the master or is any other "stable" and "bundled" target should i look for?

The "full" tarball is attached to the release (not to be confused with tag!). https://github.com/klaussilveira/gitlist/releases/tag/0.6.0

hi @alehaa ,
now i got the release tarball, but:
[fcgid:warn] [pid 2214] [client <IP_ADDRESS>:56904] mod_fcgid: stderr: PHP Catchable fatal error:  Argument 1 passed to Twig_Filter::__construct() must be an instance of string, string given, called in /home/git/public_html/src/Application.php on line 70 and defined in /home/git/public_html/vendor/twig/twig/lib/Twig/Filter.php on line 35
:/

Seems you're affected by #747. Use PHP 7 if available, otherwise you'll have to wait for a fixed release.

well, i did a dist-upgrade yesturday (debian 8->9) just to get the php7 :)

Sure you're using PHP 7? Because this issue says it's not aware of the string type hint - so it's not running in PHP 7.

ps: @alehaa i dont have composer file at all in this new install

yes @alehaa you are right! php7 on the system, php on apache! :/
thanks!


Large erosions failing

commit 2fa42995c1285a975b165eb119755f35886bb429 fixes an issue reported on the mailing list earlier in the year. Large erosions were producing blank images. The problem was caused by incorrect numeric type when setting up boundary conditions. Two tests of large structuring elements have been included.
@hjmjohnson @fbudin69500 @jhlegarreta @ut666 : Not sure of how the InsightSoftwareConsortium fork is being updated wrt to threading architecture changes etc, so filing this issue so you can handle the merge in your preferred fashion.

@richardbeare Sorry for the long delay. I had completely missed this. The upstream changes you mention were brought to the downstream ISC fork successfully:
https://github.com/InsightSoftwareConsortium/LabelErodeDilate/commit/2fa42995c1285a975b165eb119755f35886bb429
According to this thread in discourse, CI images use the latest ITK released, including alpha versions:
https://hub.docker.com/r/insighttoolkit/module-ci/tags/
The CI is currently failing for unrelated reasons, so I'd say that the issue can be closed. If legacy release versions of ITK are used in the CI and the threading changes are not being checked against for this module, issues may arise in the future, so I'd say that we can re-open the issue then if the mentioned commit is the cause for those failures.

That said, it would be nice if the test set (dataset, parameters, etc.) that caused the issue was incorporated to the current set of test cases, if not already done, for the sake of completeness.

There are new tests as part of this fix - simply larger erosions/dilation than were tested previously. The problem arose at about 16 - as 16^2 is 256 and the bug was causing an 8 bit value to be used somewhere...

👍 Thanks @richardbeare . Then, since it is being tested and has been solved, I'd be for closing the issue.


Add Ning to Contributors list

Hi  @WenNingz, I'm quite elated about your pull request. I wanna evolve this project to addresses various problems faced by first-time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a pull request and learned that I merged it?
We’ve recently added social share  to our web app. Could you please go to https://roshanjossey.github.io/first-contributions/#social-share and share your first contribution to open source?
Also, check out projects with easy issues while you’re there.

Well, at first, I tried to find open source project to contribute but none of it is easy to start at first. Because mostly open source project didn't teach how to properly setup and contribute in that project. Or it was hard to understand to contribute it. So it was kind of hard for first timer like me.
This one is the first one open source community that I contributed. At first, its kind of nervous but the tutorial was easy to follow. Now, it kinda makes me have a little bit confident to contribute to other open source project again.

I’d urge you to continue contributing to other open source projects.
We also have a slack team to help/mentor people trying to contribute to open source projects.
You can join the team by going to https://goo.gl/PM1Jdh


Allow _ as placeholder inside array literal for pipes

Hello,
It would be awesome if you could use the _ placeholder for putting a value into an array on a pipeline.
Here is an example from a real world babel plugin:
function transformFunction t, functionType, id, {node: {params, body, generator, async}}: path
    path.replaceWith
    <| expandParams t, (tail params), body, async
    |> t.returnStatement
    |> -> t.blockStatement [it]
    |> functionType id, (head params), _, generator, async

It is very close to be as beautiful as it could be. This pipe works well, and I'm just being picky of course, but, why the following will not work ?
function transformFunction t, functionType, id, {node: {params, body, generator, async}}: path
    path.replaceWith
    <| expandParams t, (tail params), body, async
    |> t.returnStatement
    |> -> t.blockStatement [_]
    |> functionType id, (head params), _, generator, async

I tried all kind of combinations [ _ ], |> [ _ ], but none of them work.
Of course I can use Array.of _ , but then I think it will be not much difference from my current solution.
Regards

A solution that does not introduce any runtime overhead (or at least, any intermediate function) would be this:
function transformFunction t, functionType, id, {node: {params, body, generator, async}}: path
    path.replaceWith
    <| expandParams t, (tail params), body, async
    |> t.returnStatement
    |> Array.of
    |> t.blockStatement
    |> functionType id, (head params), _, generator, async

In general, f g _ expands to f -> g it, not -> f g it, which is what you seem to expect.
So t.blockStatement [_] would expand to t.blockStatement -> [it].
What you want would be composition |> t.blockStatement << Array.of (or just |> Array.of |> t.blockStatement if you ere fine with that).

@vendethiel seems that we just posted at the same time, with a very similar solution.
As I said, it is not a very important thing and I'm quite happy with the current solution, it's just I usually prefer literal arrays over functions that builds arrays.

In my opinion, for this specific case, -> t.blockStatement [it] is the clearest way to express this. _ has the problem, hinted at by ven's comment, that it could be confusing where the implicit function begins—this problem is mitigated by making the placeholder _ only work as the argument of a function application and letting that application be the entirety of the implicit function thus created. It's a limited but easy to understand rule. For that reason, I'm reluctant to let the _ roam further than that. (There's a long history of people proposing various other uses in #19, although beware: some of that conversation refers to outdated LS features.) it doesn't have that problem because the function arrow is explicit.
If the intermediate function in the compiled code, and not the LS syntax, is what bothers you, then perhaps LiveScript should automatically inline functions that are immediately piped to if they meet some simplicity requirements (no variables declared, no use of this (unless it's a bound ~> function) or arguments, maybe some others?), the same way it inlines such functions when written with placeholder _. In other words, perhaps both a |> f _ and a |> -> f it should compile to f(a).

Hello @rhendric
As you said, I think the current syntax is good enough, I was just proposing if it were simple, but obviously it is not.
One of my main concern however was the usage of unnecessary inline functions as you said. I think this function is simple enough to be inlined. In fact I switched to using Array.of inside the pipe because the compiled code is much simpler and concise.
So, in summary I subscribe your latest proposal


Readme: contributors section

Styling won't work.
Workaround — SVG: https://pragmaticpineapple.com/adding-custom-html-and-css-to-github-readme/

Ok. Styling won't work due to security reasons.


VSCode should give better errors when failing to install extension dependencies.

I don't know if VSCode just handles all extension responsibilities to the extensions themselves, but I think extensions should be like any other package manager you name it APT, PIP, BREW, etc, if something fails, is the package manager responsibility to report a failure or error.
I've already ran into several issues installing vscode extensions offline that needed other stuff installed to work, but extensions vsix installed with zero warnings. This gives the impression everything is good.
Just to name a few:

python, (this used to work before but then they added jupyter extension as a dep)
remote ssh, (this one does give you some sensible warnings I'll admit)
c++, (language tools)

So if an extension has a config where it lists all deps whether it is another extension or something else installed in X path. VSCode should check whether everything was installed, if not give errors/warnings.

Are you installing a vsix or from Marketplace?

Are you installing a vsix or from Marketplace?

Ah forgot that in the description, yes vsix, but in my opinion either install it should warn the user about failing deps installs, not just silently sort of install half way. Is Vscode fully dependent on extensions to do these warnings?

Side loading using vsixs is a special case and it can be used for various purposes - like installing a vsix to test or installing while offline. It is expected by the user to make sure user installs all dependencies if installing by vsix.
We try to install the deps if user is online and if offline we do not restrict because of above reasons. If you try to install from gallery then we make sure we install all dependencies like other package manager.


Fix attachment upload

Currently the attachment upload does not work because jQuery sends the
data to "/attachments" (even after changing the url property to include
the trailing slash in "/attachments/"). Thus I also renamed the route
to /attachments/upload.
This fixes in-house file uploads.

Great! Thanks, @xdarklight


Rename AuthenticationClient module to CDAP

This means you instantiate CDAP::AuthenticationClient instead of AuthenticationClient::AuthenticationClient (the docs were wrong, anyway)...

LGTM


Option 2 Installation

Option 2: JupyterLab UI and Server-Side Extension
Looking at the ./install_tarball.sh
It is calling 0.1.1 when it should be 0.1.5.
This is calling issues when running the shutdown extension, can we update the build file to the latest version.
https://github.com/aws-samples/sagemaker-studio-auto-shutdown-extension/blob/main/install_tarball.sh

Hi @jkapurchicago , the UI version of the extension has not been updated since 0.1.1, and so we've left it as-is. We also encourage you to use the server-side extension as it is regularly tested and maintained.
That said, can you share what issues you're facing when running the UI version?

Closing due to inactivity. Please reopen if the issue still exists.


Add Rubocop to Rake infrastructure

Coverage remained the same at 100.0% when pulling 07c9abab52b5929bdfe6f8557836cc47e616929f on stephengroat:rubocop-rake into 50e09f973ec01ccb0751741eed4d78acb1c3cdb3 on dtan4:master.


NSM entry not removed from registry after nsmgr deletion

Expected Behavior
after running
kubectl delete -f k8s/conf/nsmgr.yaml

the network service manager should not be seen with:
kubectl describe networkservicemanagers.networkservicemesh.io

Current Behavior
currently, deleting the nsmgr pod doesn't remove the entry from the registry

That was discussed in #767. I think there are the ideas.

@ivanayov Some nuance here.  If a NSMgr restarts, we don't want to remove it's record.  It's not clear how to detect the difference between  deletion vs a restart or upgrade.
Additionally, there may be NSE records associated with that NSMgr that need to be cleaned up correctly.
One way we considered was possibly using the lastseen field in the NetworkServiceManager CRD...  the original idea was to have some period (5 minutes) and have the NSMgr update that field every that many minutes.  If its lastseen is more than three periods old, consider it gone.
The problem with the lastseen is it has some scaling issues.  If I have 5k Nodes in a cluster, each updating every 5 minutes... that means on average 5k/300s ~=17 updates/s... which then also means 17 updates mirrored out to each node per second (as nsmd-k8s subscribes to notifications for each  node).
All of these things are solvable... just wanted to point out its a more interesting problem than it might originally seem :)

We can use K8s Garbage collection ([1], [2], [3]) , i.e. add ownerReferences
There are 3 kinds of policies: foreground - owner cannot be deleted before any other object it owns, background - once owner is deleted, all owned objects are deleted, orphan - owned objects are left orphaned, removing the reference to the owner.
I’m checking how it can be configured together with liveness and readiness probes.
[1] https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/
[2] https://thenewstack.io/deletion-garbage-collection-kubernetes-objects/
[3] https://blog.openshift.com/garbage-collection-custom-resources-available-kubernetes-1-8/

this issue is related to #928

I have paused work on this issue and will be able to continue at the end of June.

@haiodo @matrohon Issue can be closed