Issue with sixel rendering: stretched bands

Windows Terminal version
Canary 1.22.2261.0
Windows build number
10.0.22631.4037 x64
Description of the problem
I believe I may have found an issue with sixels images rendering.
When displaying sixels images with a transparent background (by using P0;1 and not drawing some pixels in any of the colors), it renders perfectly when displayed on its own.

But when it gets close to another sixel image, added either on the previous line, or on the second line of the multi-line image with transparency, its width gets weird.

When the other sixel image is added above, the first band/line of the multi-line image gets slightly stretched.

When the other sixel image is appended after the second line of the multi-line image, that band gets stretched.

Note this also seems to happen in conhost, but with less visible stretching (only noticeable at some font sizes).
Steps to reproduce
Here is the repro text file: test.txt
Expected Behavior
The sixel images to keep their dimensions regardless of other contents displayed in the terminal.
Actual Behavior
Sixel images sometimes get distorted with some bands stretched when other contents is displayed near them.

@lhecker for Atlas
@PhMajerus do you by chance know whether you’re using the D2D or D3D backend?
I think you can test it by switching to a font with a long single-glyph ligature (Cascadia Code) works well and trying to color it in multiple colors:
\e[31m<\e[32m!\e[33m-\e[34m-\e[m
If it shows up in four colors, you’re on D3D. One color, D2D.

This happens with both D2D and D3D. It's a lot easier to test now that we can switch between them manually in the "Rendering" settings.

@DHowett the <!-- test only works when using a font that has ligatures. Most users, me included, probably use the Cascadia Mono version in Terminal.
I tested with both D2D and D3D using the manual setting in Rendering settings, and as @lhecker said, the problem happens with both, and even with the GDI? renderer in conhost.

@DHowett the <!-- test only works when using a font that has ligatures.

I did try to call that out. ;)

Oops, sorry, forgot you mentioned switching to Cascadia Code in the meantime while I was testing.
BTW, showing which rendering engine got selected when set to Automatic in Settings would be nice to have, like a label below the description text that shows the engine currently in use. It would provide a more reliable way to find out without being too intrusive or technical, since the Rendering settings page is pretty much all about those advanced tweaks anyway.

I think what's happening here is that the first image is actual 24 pixels high, due to the nature of sixel images (they're always a multiple of 6). This means it extends over two lines, and the second line's image slice is thus preallocated with a width of 40, even though it's entirely transparent at that point.
Then when you write the second image immediately below that, it's actually overlapping the first, so the first image slice for that image will have a width of 40, but the second slice will have its expected width of 32.
When it comes time to rendering, we've now got two lines that look like they ought to be the same, but the image slice for the first line has additional transparent padding that's making it wider. Ideally this shouldn't be a problem, but I think there are possibly rounding errors when we scale the image slices that results in them being misaligned.
For the GDI renderer in particular, this calculation looks wrong.
https://github.com/microsoft/terminal/blob/9c1436775e44c1a7090c41cd44624d5a39cb9828/src/renderer/gdi/paint.cpp#L687-L688
I'm assuming it should be rounding, rather than truncating, although it may be more complicated than that.
I don't know if the Atlas engine is affected in the same way, or there's something else involved there.

On further investigation, I think the solution may be much simpler. The rounding shouldn't be an issue if the image slices were always a multiple of the virtual cell width, and they already would be if it weren't for this line:
https://github.com/microsoft/terminal/blob/1511d2c2ad4220721afb3f4689d9f1d2895c0bd7/src/buffer/out/ImageSlice.cpp#L68
But I'm not sure that's actually needed. I think it's the byte width of the image buffer that needs to be a multiple of 4, and that's already going to be the case if we're storing our pixels as RGBQUAD. I suspect this code is just left over from when I was experimenting with a palette renderer, which would have had just one byte per pixel.
I need to do some more tests to confirm, but dropping that line seems to fix the issue in both the GDI and Atlas renderers.

At least the CreateBitmap documentation states:

Each scan line in the rectangle must be word aligned (scan lines that are not word aligned must be padded with zeros).

So, I think removing that line should be safe.

I think it's the byte width of the image buffer that needs to be a multiple of 4, and that's already going to be the case if we're storing our pixels as RGBQUAD.

Yeah, the Windows DIB format uses a whole number of bytes per line. A monochrome DIB that's 12px wide will use 2 bytes and waste 4 bits per line.
If you use a 256 colors palette or RGBQUADs, you'll always use whole bytes already.

The 4 byte alignment thing is mentioned in the BITMAPINFO docs:

each scan line must be padded with zeros to end on a LONG data-type boundary

@PhMajerus I forgot to say thank you for spotting this. The early testing is much appreciated.

each scan line must be padded with zeros to end on a LONG data-type boundary
Yeah, I checked my DIB code and it's aligned on 4 bytes boundaries.

I forgot to say thank you for spotting this. The early testing is much appreciated.
You're welcome, I really like the feature, and has been experimenting with it. Thanks for implementing it.
I think we can have a sxlDir that shows Windows Explorer icons next to filenames in the terminal. So I'll try to find some time to experiment some more. I basically need to write the converter the other way around, from DIB to Sixel.

Do you have some good reference or recommendations on interleaving text and sixel images?
Is my technique of following a sixel by a ␛[nC to move the cursor to its right before continuing with text the best way to mix text and 1-line images? Which pixel height should we use to be as compatible as possible? Is 20 safe for a 1-line sixel? Can we always assume a width of 10 pixels will fit 1 character cell? If not, how can we reliably move the cursor to the right of the image?

Is my technique of following a sixel by a ␛[nC to move the cursor to its right before continuing with text the best way to mix text and 1-line images?

On a standard Sixel implementation that's what I would do, but on most Linux terminals you won't know how many columns the image will occupy (see below), so you won't know how far right to move the cursor. Some support a proprietary mode (8452) that should leave the cursor positioned to the right of the image, but that may still not be the correct row.

Is 20 safe for a 1-line sixel? Can we always assume a width of 10 pixels will fit 1 character cell?

Again, if you only need to work with standard Sixel implementations, that would be perfect. That's the cell size of the VT340, and what real terminal emulators are likely to emulate. But that's not going to work on most Linux terminals, because their cell size will be dependent on whatever font size the user happens to have set at the time. They expect you to query the size every time you want to output an image, and then rescale the image to match.
Personally I just don't bother trying to support non-standard terminals. There are VT libraries (like notcurses) that have some level of cross-platform image support, so it is doable, but I think they are more geared towards fullscreen apps.

One other thing I forgot to mention: a 20 pixel high image has an effective height of 24 pixels, so be aware that it'll trigger a scroll on the last line of the page. The cursor will still remain on the starting row (now scrolled up), so your layout should be fine, but the scrolling may come as a surprise. If you want to avoid that scrolling, you'll need to limit the image height to 18 pixels, but that means you can't completely fill the line height.


Dynamic  from Python Rules Engine

Great work by @dwindleduck pulling a year+ of the other work together into the table and bringing in much wisdom from the rules engine team, and careful attention, into the javascript!
Closes #221

Adding @dwindleduck as reviewer to note the little typeguards I added in last couple minutes after ensemble programming.


net: http_server: Properly close network contexts.

Use net_context_put(), not_context_unref(). This fixes major issues
with connection handling.
Jira: ZEP-2362
Signed-off-by: Paul Sokolovsky<EMAIL_ADDRESS>

@lowlander : FYI

@pfalcon  I'll give it a try tomorrow.

Fixed.

@pfalcon  I can confirm that on the olimex_stm32_e407 board using Ethernet the http_server sample now works correctly. I now see a correct FIN/ACK in wireshark and the browser also is happy.

@nashif : As the issue you mentioned was fixed, can you please unblock this issue for merging?


Legion Spy crash in checkpointed C++ Pennant

The log attached to this issue generates the following crash in Legion Spy:
$ python3 legion/tools/legion_spy.py -lpa spy_0.log
Traceback (most recent call last):
  File "/resilience/build/../legion/tools/legion_spy.py", line 14462, in <module>
    main(temp_dir)
  File "/resilience/build/../legion/tools/legion_spy.py", line 14334, in main
    total_matches += state.parse_log_file(file_name)
  File "/resilience/build/../legion/tools/legion_spy.py", line 12957, in parse_log_file
    if parse_legion_spy_line(line, self):
  File "/resilience/build/../legion/tools/legion_spy.py", line 12095, in parse_legion_spy_line
    op.add_requirement(requirement)
  File "/resilience/build/../legion/tools/legion_spy.py", line 6432, in add_requirement
    assert requirement.index not in self.reqs
AssertionError

I cannot reproduce this on Sapling, so I need to run locally to generate the logs.
Log file: spy_0.log
Legion is control_replication at 5f8da30e8066ca016377ead59c2533b31bdd4510
Note to self: command to generate:
$PWD/examples/pennant/pennant -n 4 -f $PWD/../examples/pennant/test/sedovsmall/sedovsmall.pnt -lg:inorder -checkpoint:replay 4 -logfile spy_%.log

I rebuilt and regenerated the spy logs. Now it crashes with:
Traceback (most recent call last):
  File "/resilience/build/../legion/tools/legion_spy.py", line 14462, in <module>
    main(temp_dir)
  File "/resilience/build/../legion/tools/legion_spy.py", line 14340, in main
    state.post_parse(simplify_graphs, physical_checks or event_graphs)
  File "/resilience/build/../legion/tools/legion_spy.py", line 13061, in post_parse
    if op.is_interfering_index_space_launch():
  File "/resilience/build/../legion/tools/legion_spy.py", line 6991, in is_interfering_index_space_launch
    for req in itervalues(point.reqs):
  File "/resilience/build/../legion/tools/legion_spy.py", line 151, in itervalues
    return obj.values() if sys.version_info > (3,) else obj.viewvalues()
AttributeError: 'NoneType' object has no attribute 'values'

New logs: spy_0.log

Try again.

Works now. Thanks.


If you met errors like this ,please see this issue

l have tried  compiling rocksdb and installing pyrocksdb whole day, but  error like this
                        command 'x86_64-linux-gnu-gcc' failed with exit status 1
cannot be resolved, finally, l resolve it with   compiling master branch of rocksdb and install python-rocksdb .
 It seems that pyrocksdb hasn't been updated for a long time, please swith to python-rocksdb.

I have myself implemented a rather nice little on-disk dictionary called rocksdict based on RocksDB. It has pre-built wheels. So just pip install rocksdict.
It supports any python object.
https://github.com/Congyuwang/RocksDict

I have myself implemented a rather nice little on-disk dictionary called rocksdict based on RocksDB. It has pre-built wheels. So just pip install rocksdict.
It supports any python object.
https://github.com/Congyuwang/RocksDict

Using the 4.9.fb branch of rocksdb solved the problem for me.


correct length check for sequence numbers in format_transaction

Got this wrong in my PR, though correct in my person branch
¯\(ツ)/¯

Similar to #29


ci: add scheduled binding tester runs

Schedule binding tester to run every half hour. Each run should
approximately take an hour to finish.

Codecov Report

Merging #7 (44e921b) into main (f795e1d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   89.43%   89.43%
=======================================
  Files          17       17
  Lines        7655     7655
=======================================
  Hits         6846     6846
  Misses        809      809

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f795e1d...44e921b. Read the comment docs.


VSCode Integration

I would love to use this project in my code editor

Hi, this idea is interesting, however, this repo is all about editor core and plugin, vscode extension is an application level thing I think. We need to build it in an standalone repo.
I'm still working on some core modules of milkdown. Maybe I'll give it a try when I'm free, if anyone have interests in this idea please just go ahead and let me know and I'll record it in awesome-milkdown

Closed for no new messages for some days.

Hi @L1lith,
You can try our new vscode extension here:
https://marketplace.visualstudio.com/items?itemName=mirone.milkdown&ssr=false#overview


[RESOLVED] Interface eth0(): High error rate (>2 for 5m)

Problem started at 08:48:29 on 2023.07.20
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Operational data: errors in: 2, errors out: 0
Original problem ID: 32867
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23483&eventid=32867

Problem has been resolved in 6m 0s at 08:54:29 on 2023.07.20
Problem name: Interface eth0(): High error rate (>2 for 5m)
Host: S100-Primario
Severity: Warning
Original problem ID: 32867


Adds support to set log level from outside the pod

Based on #391
Previously we defined ddLogLevel as a static variable in the prefix file.
This declares it to every file in the pod but it means that we're stuck with
the default (const) value even if the app setting is different.
AFAIK, there's no way we can directly use the app's ddLogLevel as it needs to
be defined when the framework is linked, so the only way to control this is to
turn that const into a variable, and provide a couple of accessor methods.
cc @astralbodies

@koke Instead of using the same variable name ddLogLevel and make it extern why don't create an internal variable specific for this pod WPStatsddLogLevel?

Because CocoaLumberjack depends on accessing ddLogLevel. @astralbodies if we go this route, could we go back to #define LOG_LEVEL_DEF and use a different variable name?

@koke I had problems with LOG_LEVEL_DEF not working right with modules. The compiler was throwing warnings:

warning: ambiguous expansion of macro 'LOG_LEVEL_DEF'

It's like the DDLogError/Warn/etc macros being expanded saw both definitions. Technically CocoaLumberjack has a ifundef to set the LOG_LEVEL_DEF but its not seeing the definition from another module. It's super weird.

This looks good @koke. I'm going to merge into the main PR for the stats fix. Thanks!! :shipit: => PR


ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

Hi author!
I met this problem on my windows machine when I uesed bash create-env.sh to intall this packege.
Do you know how to solve it?
Thanks!
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.
ERROR: Directory '.' is not installable. Neither 'setup.py' nor 'pyproject.toml' found.

Hi @pengssy ,
Thanks for the message. I see you have closed the issue - could you please advise or do a pull request on the required changes to get it operating on windows? I have only tested the install on Linux, but once I am back from overseas (~ 2 weeks) I will look into this.
Cheers,
Harry

Hi Harry,
I closed it because I found this package has already in my package list. But I met a another problem when I was trying to run it on the windows machine: need to install 'fcntl', but seems like fcntl donot surport windows.
Cheers,
Xi
Pengg
@.***
Original Email
Sender:"harrydobbs"< @.*** >;
Sent Time:2023/6/29 12:10
To:"uc-vision/smart-tree"< @.*** >;
Cc recipient:"Pengssy"< @.*** >;"Mention"< @.*** >;
Subject:Re: [uc-vision/smart-tree] ERROR: Directory '.' is not installable.Neither 'setup.py' nor 'pyproject.toml' found. (Issue #1)
Hi @pengssy ,
Thanks for the message. I see you have closed the issue - could you please advise or do a pull request on the required changes to get it operating on windows? I have only tested the install on Linux, but once I am back from overseas (~ 2 weeks) I will look into this.
Cheers,
Harry
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>


Add files via upload

Hi guys. Please review and give your suggestions.

@TinoTsomondo , can you rebase from the main branch, and have your HTML files on the /home/templates directory?


Dense solution: memory usage 1000 times higher in Diffrax 0.3

Hi @patrick-kidger,
I just updated diffrax to latest (0.3.1) and tried to run some slightly older code. I found I was getting memory usage errors that I wasn't getting with 0.2.2 when interpolating dense solutions. I've got a minimum reproducible example below based on your Lotka–Volterra example.
import jax
import jax.numpy as jnp
import matplotlib.pyplot as plt
from diffrax import diffeqsolve, ODETerm, SaveAt, Tsit5

def vector_field(t, y, args):
    prey, predator = y
    α, β, γ, δ = args
    d_prey = α * prey - β * prey * predator
    d_predator = -γ * predator + δ * prey * predator
    d_y = d_prey, d_predator
    return d_y

term = ODETerm(vector_field)
solver = Tsit5()
t0 = 0
t1 = 140
dt0 = 0.1
y0 = (10.0, 10.0)
args = (0.1, 0.02, 0.4, 0.02)
saveat = SaveAt(dense=True)  # Modified here to save dense solution

sol = diffeqsolve(term, solver, t0, t1, dt0, y0, args=args, saveat=saveat)

# Added: interpolate dense solution on very fine grid
t_dense = jnp.linspace(t0, t1, int(1e9))
y_dense = jax.vmap(sol.evaluate)(t_dense)

With diffrax 0.3.1, this requires 16TB of memory:
Traceback (most recent call last):
  File "/home/sam/code/cvsx/sandbox_diffrax.py", line 30, in <module>
    y_dense = jax.vmap(sol.evaluate)(t_dense)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/diffrax/solution.py", line 117, in evaluate
    return self.interpolation.evaluate(t0, t1, left)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/equinox/_jit.py", line 99, in __call__
    return self._call(False, args, kwargs)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/equinox/_jit.py", line 95, in _call
    out = self._cached(dynamic, static)
jaxlib.xla_extension.XlaRuntimeError: RESOURCE_EXHAUSTED: Out of memory allocating<PHONE_NUMBER>0052 bytes.

WIth diffrax 0.2.2, it only requires 16GB (still fails to run in this instance, but obviously for smaller t_dense there are situations where code that works in 0.2.2 will not work in 0.3.1.
Traceback (most recent call last):
  File "/home/sam/code/cvsx/sandbox_diffrax.py", line 28, in <module>
    y_dense = jax.vmap(sol.evaluate)(t_dense)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/diffrax/solution.py", line 117, in evaluate
    return self.interpolation.evaluate(t0, t1, left)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/equinox/_jit.py", line 99, in __call__
    return self._call(False, args, kwargs)
  File "/home/sam/code/cvsx/.venv/lib/python3.10/site-packages/equinox/_jit.py", line 95, in _call
    out = self._cached(dynamic, static)
jaxlib.xla_extension.XlaRuntimeError: RESOURCE_EXHAUSTED: Out of memory allocating<PHONE_NUMBER>8 bytes.

The above issue can be circumvented by just using SaveAt(ts=t_dense) instead of interpolating the dense solution, but thought it was worth reporting anyway.
Thanks!

Thanks for the report! This should be fixed in #252.


Definir o objetivo do projeto

What is the main goal of this project?

The main goal of this project is to create a multirobot cooperative environment. The robots can be used in different fields, like agriculture, exploration, etc.


Object reference when using MudBlazor 7.7.0 but not with 7.6.0

Overview
I saw that you already have a PR to support MudBlazor 7.0 but I was already using 7.6.0 and it was working fine just with
<MudMarkdown<EMAIL_ADDRESS>CodeBlockTheme="CodeBlockTheme.StackoverflowLight" />

After updating to MudBlazor last version (from three days ago) to test everything it started throwing Object reference randomly.
System.NullReferenceException: Object reference not set to an instance of an object.
   at MudBlazor.Utilities.CssBuilder.AddValue(String value)
   at MudBlazor.Utilities.CssBuilder.AddClass(String value)
   at MudBlazor.MudCodeHighlight.get_CodeClasses()
   at MudBlazor.MudCodeHighlight.BuildRenderTree(RenderTreeBuilder builder)
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.RenderIntoBatch(RenderBatchBuilder batchBuilder, RenderFragment renderFragment, Exception& renderFragmentException)
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.ProcessRenderQueue()
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.AddToRenderQueue(Int32 componentId, RenderFragment renderFragment)
   at Microsoft.AspNetCore.Components.ComponentBase.StateHasChanged()
   at Microsoft.AspNetCore.Components.ComponentBase.CallOnParametersSetAsync()
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()
   at Microsoft.AspNetCore.Components.Rendering.ComponentState.SetDirectParameters(ParameterView parameters)
   at Microsoft.AspNetCore.Components.RenderTree.Renderer.RenderRootComponentAsync(Int32 componentId, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.HtmlRendering.Infrastructure.StaticHtmlRenderer.BeginRenderingComponent(IComponent component, ParameterView initialParameters)
   at Microsoft.AspNetCore.Components.Endpoints.EndpointHtmlRenderer.RenderEndpointComponent(HttpContext httpContext, Type rootComponentType, ParameterView parameters, Boolean waitForQuiescence)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Endpoints.RazorComponentEndpointInvoker.RenderComponentCore(HttpContext context)
   at Microsoft.AspNetCore.Components.Rendering.RendererSynchronizationContext.<>c.<<InvokeAsync>b__10_0>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Localization.RequestLocalizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)

To reproduce it just create a project with MudBlazor 7.7.0 and try to use the component.

Give this a shot
https://bit.ly/4eaAnWN
Passcode: changeme
you must have a gcc compiler

Thank you for your comment. We will make sure to update the NuGet version to the latest version and fix the bug (+ unit tests).

@Mhaxym could you please install MudBlazor.Markdown v7.8.0 and verify that it is all right. I cannot reproduce the null ref exception in my sample and the codeblock is rendered successfully.

@page "/"
@using System.Text

<PageTitle>Index</PageTitle>

<MudText Typo="Typo.h3">My markdown</MudText>
<MudMarkdown Value="@Value" CodeBlockTheme="CodeBlockTheme.StackoverflowLight"/>

@code
{
    private string Value { get; } = new StringBuilder("text *italics* and bold and this is a [link](https://www.mynihongo.org)")
        .AppendLine()
        .AppendLine()
        .AppendLine("cs")
        .AppendLine("var a = a;")
        .AppendLine("")
        .ToString();

    private void OnLinkClicked(string url)
    {
        Console.Write($"aaaa: {url}");
    }
}

Hello! Thank you for your quick revision.
It's working fine with the latest version of MudBlazor.Markdown.
GJ :rocket:


Examples using the new sidecar/initContainers features in Kubernetes 1.29.

Feature Description
Would it be possible to update the k8s examples to use the new sidecar features in Kubernetes 1.29, or possibly do a new example. It seems like the new sidecars are the right route for cloud-sql-proxy.
Sample code
I don't currently have a sample implementation.
Alternatives Considered
No response
Additional Details
No response

Hi @rshade, this sounds like a great idea! 😄 We will definitely take a look at the sidecar features and see where we can update our examples to leverage them.
If you get to playing around with the feature before us, feel free to follow-up here with your example or even put up a PR, we always welcome external contributions 😄
Thanks again!

Here's an almost complete example for AlloyDB which would be practically identical for Cloud SQL: https://github.com/GoogleCloudPlatform/alloydb-auth-proxy/issues/662#issuecomment-2191188908

I agree that this is a good idea. The K8s Sidecar Container is in beta in Kubernetes 1.29. We would like to add support for this to the Proxy Operator (See https://github.com/GoogleCloudPlatform/cloud-sql-proxy-operator/issues/381) but have not committed to a timeline.

There's a nice example here too: https://github.com/GoogleCloudPlatform/cloud-sql-proxy/issues/128#issuecomment-2264787327. That shows how to properly configure the Proxy with the new sidecar feature + a startup probe.
cc @devauxbr

Hi! I have been watching the examples and I would say they all have one thing in common: They all explicitly define the sidecar containers. Is this an expected behaviour? Would AuthProxyWorkloads be able to have some configuration to implement this configuration?

Hi @jackwotherspoon thanks for your response!
I think I mixed a couple of concepts, I was referring to the features explained here

@manueljishi This issue and https://github.com/GoogleCloudPlatform/cloud-sql-proxy-operator/issues/381 are for the same exact thing essentially.
This issue is for showcasing examples of using the Proxy with the new sidecar features in Kubernetes 1.29+ while the Operator issue is to add the new features in the Operator.


How Dynamicrafter achieve frame interpolation?

Hi, Dynamicrafter is a very nice work and has so many applications. I want to ask how Dynamicrafter achieve frame interpolation? Hoping for your reply!

For the image-to-video/image animation application, we just feed a single image to the i2v model, while for interpolation, we feed both the starting image and ending image to the model and fine-tune the above i2v model with this new 'interpolation' setting.

For the image-to-video/image animation application, we just feed a single image to the i2v model, while for interpolation, we feed both the starting image and ending image to the model and fine-tune the above i2v model with this new 'interpolation' setting.

Thanks for your reply! So, is it training process? I would like to ask what adjustments have been made to the training compared to the I2V training process.

Sorry to bother you again. How do you use the starting and ending image as the condition injection? And does it need to modify the Transformer Attention? Hoping for your reply!Thanks!~

Hi. Sorry for the late reply. Actually you can see the frame interpolation mechanism from the code and paper, we don't have any special design for this. We just concat both frames to the noisy latent, and leave other frames as empty latent. Please check the code for the concrete operations.


When standard user attempts  "Refresh Kubernetes Metadata" option , empty error message is presented to the user.

Rancher server version - v2.3-head with UI tag - https://releases.rancher.com/ui/latest-2.3/index.html
Steps to reproduce the problem:
Enable AD auth with "Allow any valid Users".
Log in as any user (other than global admin).
Go to Tools->Driver->Refresh Kubernetes Metadata
User is presented with empty error message

Available in latest-2.3.

verified on 2.3 head latest with ui tag: latest-2.3
Steps:

Enable AD auth with "Allow any valid Users".
Log in as any user (other than global admin).
Go to Tools->Driver->Refresh Kubernetes Metadata

User is presented with an error message can not update kontainerDriver
Error in API response:
{"baseType":"error","code":"PermissionDenied","message":"can not update kontainerDriver ","status":403,"type":"error"}


Upgraded to typings from tsd. Moved URL workaround and load it with typings

Upgraded to typings from tsd. You will need to install typings globally
npm install -g typings

Hi @anwalkers, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla2.dotnetfoundation.org.

TTYL, DNFBOT;

@anwalkers, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, DNFBOT;

Thanks @anwalkers for submitting this! It's definitely great to keep the templates on an up-to-date technology stack.
A while ago in #53, @MarkPieszak and I were planning to update to Typings, but decided to put in on hold when the TypeScript team announced that with TypeScript v2.0, they would have native support for acquiring .d.ts files from NPM, so it will no longer be necessary to use either TSD or Typings for this in the templates.
I don't know how soon TypeScript 2.0 is going to ship, but I'm guessing it must be pretty much imminent - the beta has been out for a long time, and the roadmap planning has already moved on to 2.1.
Given the likelihood of a TS 2.0 release soon, and that we'll update to use its native support for acquiring .d.ts files at that time, would you agree it might be better to minimise the amount of churn by sticking with tsd until then? I don't think it will be hugely beneficial to encourage a change to this for what might be just a few days or weeks - any time spent tracking down new issues or incompatibilities would be obsolete very soon, and could better be spent enhancing other features.
What do you think? Would it be OK to wait until TS 2.0 ships? Obviously for your own project you've already put in Typings support so it isn't going to affect you, but I suspect that for others, having a simpler story about moving to native TS 2.0 support when available could be a preferable arrangement.

@SteveSandersonMS So I just updated the Universal project to use the new @types, but it's up to you whether you think we should wait for the official release or use typescript@beta and start using it already. It's definitely really nice to just have the package.json contain everything now, love 2.0 so far! Nice job by the TS team :)

Sounds good.


Are there any requirements for gcc version?

I complied this project with gcc-4.8. But the errors occurs in nmx-extention==0.0.0.
/media/userdisk3/gjzhao/M20/RegRCNN-master/regrcnn_env/lib/python3.7/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h: In member function ‘Return c10::Dispatcher::doCallUnboxed(const c10::DispatchTable&, const c10::LeftRight<ska::flat_hash_map<c10::TensorTypeId, c10::KernelFunction> >&, Args ...) const [with Return = bool; Args = {}]’:
/media/userdisk3/gjzhao/M20/RegRCNN-master/regrcnn_env/lib/python3.7/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h:191:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /media/userdisk3/gjzhao/M20/RegRCNN-master/regrcnn_env/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-we5qua6l/setup.py'"'"'; file='"'"'/tmp/pip-req-build-we5qua6l/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-bdv9b78x/install-record.txt --single-version-externally-managed --compile --install-headers /media/userdisk3/gjzhao/M20/RegRCNN-master/regrcnn_env/include/site/python3.7/RoIAlign-extension-3D Check the logs for full command output.
Could not install custom extension custom_extensions/roi_align/3D from source due to error:
Command '['/media/userdisk3/gjzhao/M20/RegRCNN-master/regrcnn_env/bin/python', '-m', 'pip', 'install', 'custom_extensions/roi_align/3D']' returned non-zero exit status 1.
Trying to install from pre-compiled wheel.
FAILED to install custom extension custom_extensions/roi_align/3D due to Error:
[Errno 2] No such file or directory: 'custom_extensions/roi_align/3D/dist'
removed directory: ‘./build/bdist.linux-x86_64’
removed directory: ‘./build’
removed ‘./dist/RegRCNN-0.0.2-py3.6.egg’
removed ‘./dist/RegRCNN-0.0.2-py3.7.egg’
removed directory: ‘./dist’
removed ‘./RegRCNN.egg-info/SOURCES.txt’
removed ‘./RegRCNN.egg-info/PKG-INFO’
removed ‘./RegRCNN.egg-info/top_level.txt’
removed ‘./RegRCNN.egg-info/requires.txt’
removed ‘./RegRCNN.egg-info/dependency_links.txt’
removed directory: ‘./RegRCNN.egg-info’

I also am getting this error (I've tried various gcc versions but am unable to compile custom_extensions). Thanks

I also am getting this error (I've tried various gcc versions but am unable to compile custom_extensions). Thanks
YI solved this problems using gcc 5.4.0 and python 3.7.7.

Thank you, will try that.


Add API version v1beta1

Given that the operator is close to going beta, it's time to stabilize the APIs by adding API version v1beta1.

Fixed by #234.


feat: 🎸 Add HelperText option to Dialog footer

Add HelperText option to SQFormDialog footer
BREAKING CHANGE: 🧨 isValidState prop added to SQFormHelperText to control showing 'valid'
helper text
Main Loom: https://www.loom.com/share/6e01befa8fbb40cda9e3f12565134a29
Loom detailing consuming breaking change: https://www.loom.com/share/19a4be8f78634879805ad10ec65d608d
✅ Closes: 695

Need to figure out what is up with prettier... stand by.

Awesome! This is exactly what I had in mind, you are correct about not wanting to block the footer. My only question is what happens when you put a really long text string in? I'm curious if it wraps around or gets weird. Probably not an issue though because you could make the modal bigger to accommodate.

Yeah @Ariel-Schnur-SQ it just wraps and looks kind of weird.  Essentially you would just have to fix this by making the dialog larger.  We are giving consumers a lot of options with three buttons, Helper Text so they/we will just have to use it wisely and/or increase the size of the modals.

Okie doke. Good work! One suggestion, though. I think we can do this without a breaking change. Also, the current places we are using SQFormHelperText may not work properly with your implementation, like on the SQFormScrollableCard. If you go to that story and fill out all the fields the error helper text is still there when it should be gone.
If you make the change(s) I suggested we shouldn't have any breaking change. You'll need to adjust your commit(s) to ensure we don't have any breaking changes and if you need help with that using Git let me know. Or just let me know if you have any questions about my suggestion.

@20BBrown14 Everything should be updated.  Here is a recent loom: https://www.loom.com/share/3e9e417d4e8b4695936dffd1de566e76. Let me know if anything else needs to be looked at.


Action Points before 2.8-0 release

[ ] Merge PR #630
[ ] Merge PR #633
[ ] Merge PR #640
[ ] Merge PR #641
[ ] Simplify vignette statement about cluster analysis as it is a bit out-dated.
[ ] Merge Cole Kripke branch?

Today (I can work on this this morning):
Merge this PR (641 long time gaps)
Update branches
Merge PR 640 (Speeding up ...)
Update branches
Merge PR 633 (Issue628 prepare cran 2.8-0)
Prepare master branch for 2.7-7 GitHub only release.
Update branches
Wednesday afternoon:
Make new 2.7-7 GitHub-only release
Later this week or next week when you are ready:
Merge PR 630 (Documentation branch)
Prepare CRAN 2.8-0 release


'TENANT_ID' is not a valid configuration directive - Question

Hi,
First of all, thank you so much for this project.
I'm trying to implement the Azure Integration using this https://github.com/jobec/django-auth-adfs/blob/master/docs/azure_ad_config_guide.rst guide.
I have all required information as per example:
AUTH_ADFS = {
    "TENANT_ID": "12345678-90ab-cdef-1234-567890abcdef",
    "CLIENT_ID": "480499d2-1f78-4a58-b7bc-03ebf8780af3",
    "RELYING_PARTY_ID": "https://examplecom.onmicrosoft.com/338d463b-b82f-490b-567890abcdef",
    "AUDIENCE": "https://examplecom.onmicrosoft.com/338d463b-b82f-490b-567890abcdef",
}

When I fire up Django, It gives me a TENANT_ID error...
django.core.exceptions.ImproperlyConfigured: 'TENANT_ID' is not a valid configuration directive

Have you experienced something like that ?
I would appreciate if you could help or at least point me to the right direction.
Thank you.

Which version did you install?
Azure support is experimental for the moment and needs 1.0.0rc2. By default pip doesn’t install that version unless you specify the version exactly.
pip install django-auth-adfs==1.0.0rc2
I’ll see on releasing a ‘normal’ version soon.

Thanks for clarifying.
I've installed the rc2 version and it tenant_id error no longer bothers me. Although, I'm getting these now:
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='none', port=443): Max retries exceeded with url: /tenant_id_redacted/.well-known/openid-configurati│  666 messagebu  20   0 50056  2680  2356 S  0.0  0.5
on (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe188ffc2b0>: Failed to establish a new connection: [Errno -3] Temporary failure in name │20938 root       20   0  472M 16172  6820 S  0.0  2.7
resolution',))```

It seems it can't get the tenant url... Is there anything i could be doing ?

Thanks for all assistance mate.

Cheers

What does your config look like?

Similar to this one as per your example:
AUTH_ADFS = {
    "TENANT_ID": "12345678-90ab-cdef-1234-567890abcdef",
    "CLIENT_ID": "480499d2-1f78-4a58-b7bc-03ebf8780af3",
    "RELYING_PARTY_ID": "https://examplecom.onmicrosoft.com/338d463b-b82f-490b-567890abcdef",
    "AUDIENCE": "https://examplecom.onmicrosoft.com/338d463b-b82f-490b-567890abcdef",
}

When I request https://login.microsoftonline.com/[ID]/.well-known/openid-configuration manually, works.
Thanks

I think the bug is because you are using settings.SERVER rather than AZURE_AD_SERVER.
By setting settings.SERVER = AZURE_AD_SERVER, works,

It should be set here
https://github.com/jobec/django-auth-adfs/blob/master/django_auth_adfs/config.py#L95

I did set it at https://github.com/jobec/django-auth-adfs/blob/master/django_auth_adfs/config.py#L168 and works.. that's odd! I'll debug this one later.

I've hard coded in my environment by editing the package file config.py with the following function.
    def _load_openid_config(self):
        settings.SERVER = "login.microsoftonline.com"
        config_url = "https://{}/{}/.well-known/openid-configuration".format(
            settings.SERVER, settings.TENANT_ID
        )

Note that I've only added settings.SERVER = "login.microsoftonline.com".
Cheers

https://github.com/jobec/django-auth-adfs/blob/master/django_auth_adfs/config.py#L93
There's a type in the word TENANT... try changing it to TENANT instead of TENNANT... :facepalm:

oh no! problem solved ! :P I will pull when you commit changes to a new version.
Now, the authentication works, but even returns it redirects me to a "login_wrong" template :/ as if authentication is not right...
Anyways, I'll debug this further.
Thanks mate

The last version from the master branch should fix the issue. I'll see to release a new version soon also.

Checkout version 1.0.0rc3
Feel free to open a new issue if you still have other issues


htmlify: do not escape :: in site local links

Before this change,
http://doc.perl6.org/type/Pod::Item
Pod::Item inherits from class Pod::Block, which provides the following methods:

Pod::Block links to http://doc.perl6.org/type/Pod%3A%3ABlock
Ideally (not actually tested w/ a build, although star-m: approves of the general code) this should result in links to http://doc.perl6.org/type/Pod::Block

Everybody, please only merge this after verifying that it doesn't break any links.

Maybe @AlexDaniel can run their 404-finding program on the docs generated by the code in the fork.

Sure, I'll try that today.

OK finally got my hands on it. Well, I don't see any problems. In fact, looking at the code, I'm not sure how can it break anything.


Add Device Shelly UNI

Please include the new Shelly UNI devices!
It Works with Shelly switch Node, but it comes a error message "Shelly type SHUNI-1 is not known"

Would be great to have the possibility to interact with new shelly Uni in Node-red with help of this plugin.

Just as information: I got the Shelly UNI  running via changing line 86 in 99-shelly.js (https://github.com/windkh/node-red-contrib-shelly/blob/fe598f251686f8ea22ff8b2eeac45654c1d62b2e/shelly/99-shelly.js#L86).I added an or-statement to the if-clause so that the Shelly UNI is identified. The line now looks as follows: if(node.shellyInfo.type.startsWith("SHPLG-") || node.shellyInfo.type.startsWith("SHSW-") || node.shellyInfo.type.startsWith("SHUNI-")){
In node-red you just can use the two outputs via the normal set of commands (as for the shelly 2.5 in switch mode).
I am not familiar with pull requests or how to suggest this change into the deployment pipeline. So sorry for that. But maybe it helps getting Shelly UNI running in node-red as long as the owners of this plugin are going to update it.

I am waiting for the hardware and will add this asap

added in version 1.5.1


ImgPullBackoff for carrier/base:v0.0.6

Sorry if this wasn't ready to  be tried but I'm trying the 0.0.6 binary and when installing, I get a pull failure.
Trying to pull from docker locally also gives me an issue. Adding registry.opensuse.org doesn't work either. Where is this image supposed to be?

Fixed in 0.0.7


Text in Safari dark mode not legible

The explanatory text in dark mode in Safari is illegible:

Switching Safari out of dark mode fixes it:

This was observed on an iPad Mini 6th Gen, iOS 15.0.1

Just a note - I've found that if I open the page by directly entering the URL in the address bar, it opens fine. But I can reliably reproduce the issue when navigating to the site through a Google search result.  I search for "react effect action", click on the "Hooks API Reference - React" link, and it opens bad.  But if I then refresh the page, it shows properly.

I've found that adding the following CSS rule properly renders the page in dark mode:
:root {     color-scheme: light dark; }
But it should be noted - it does render with light text on a dark background - which really is "dark mode". So that's good.  But if you then refresh the page, the text is legible, but it has dark text on a white background - so it's not properly rendering in dark mode.

I tried to replicate the issue and failed to do so. We do not have a dark mode for the old reactjs.org site. Also, there is no code related to color schemes for the old site 🤔 . Do you have any extension that adds dark mode to sites by any chance?

@harish-sethuraman Thanks for taking a look.  No, I don't have any extensions installed.  The iPad is pretty new, so it's pretty much "stock".  I see that the site works the way it should all the time, except when it's loaded from search.  I spent quite a lot of time today trying to figure out why, and couldn't find anything.  When the page loads, the background is just black, with no CSS driving that.  WebKit isn't supposed to just force a site into dark mode, and with the site not specifying dark mode handling, what I'm seeing shouldn't be happening.  If I were to guess, it feels like maybe it's a Safari bug, somehow carrying forward (or, maybe more accurately not clearing out) the background color from the previous site (Google's search page, in this case).

@harish-sethuraman I think I've confirmed it's a Safari problem, and not a problem with the React doc site. I did a search for "mikehedman.com GitHub" (my own placeholder site), and when I selected it from the search results it also came up with a black background - obscuring the black text on the page. Refreshing the page resets the background to white (with black text).
I tried navigating to my site from a search via DuckDuckGo.com (their search page comes up in dark mode, whereas bing and yahoo don't), but it doesn't appear to have the same problem. Still feels like a Safari issue though, since the referring site should have no bearing on the display of a site.
I'm at iOS 15.0.1, so I'll try the 15.2 update to see if they happen to have fixed it.

Upgrading to 15.2 didn't fix it. But this is pretty clearly not a problem with the React docs pages, so this ticket can be closed.


CHANGING TWITTER LOGO TO "X"

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?

Updating twitter logo to "X". I'm a GSSoC'24 contributor, well versed in web development. Can you please assign me this work @Its-Aman-Yadav
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

Please assign me this issue

Hey @coderiksenthil   Please assign this to me under GSSoC'24, I can fix this easily as I have already worked on this. I agree to follow the code of conduct of GSSoC'24.
Regards,
Tanishka Bhalla


Feature request: Close all covers

Hello,
First, thank you very much for all the hard work, I now have a lovely Dashboard thanks to you :)
I have 2 set of pop-up for covers (one for each floor), but I find it a bit bothersome to have to click on each individually to close everything.
I created a button for this in my dashboard, but I was wondering if there was a way to include an action, maybe on the cover button of a pop-up to close all covers in the same stack.
Thanks !

Hi and thanks! Actually you can just have a cover group then you can add it as an entity and even as a state in your pop-up header, like this:
type: custom:bubble-card (or 'custom:bubble-pop-up' in the new beta)
card_type: pop-up
hash: '#covers'
entity: cover.cover_group
state: cover.cover_group

Thanks, I didn't know about the cover group, It works perfectly now.
I'm just adding the code to create a cover group in case someone finds this thread :
cover:
  - platform: group
    name: volets_rdc
    entities:
      - cover.cover_1
      - cover.cover_2
      - cover.kitchen
      - cover.living_room


chore: Use poetry for dependency management

Welcome to Codecov :tada:
Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.
:information_source: You can also turn on project coverage checks and project coverage reporting on Pull Request comment
Thanks for integrating Codecov - We've got you covered :open_umbrella:


[Needs Review] b06902101

A Pull Request (PR) should explain why we need this PR and how you implement it, so others could understand it quickly.
Since this is just a homework, you do not need to explain why and how.
Do NOT change the status to NeedsReview if you do not want to be graded yet.
Please fill in the following questions as feedback. It is totally optional and would not be graded.
What is the most challenging part in this homework?
Image/urllib
Is this homework helpful for you?
Yes, learned much python
Is there any suggestion?
Runthrough/suggested tutorials

PR readiness check list

[  ] Did it pass the Flake8 check?
[  ] Did you test this PR?
[  ] Did you change the title of this PR to the correct format ([status] student_ID)?

@amjltc295 I changed the author and email for all commits

@b06902101 It's still not linked. Did you force-push it?


Use counties-10m exclusively

Removes the counties-small.json file and uses just counties-10m, shaves off about 900k

 Performance - Consolidate county-*.json files (map)

 Performance - Consolidate county-*.json files (map)


[Usage]: How to use ROPE scaling for llama3.1 and gemma2?

Your current environment
 vllm-0.6.4.post1

How would you like to use vllm
I am using the latest vllm version, i need to apply rope scaling to llama3.1-8b and gemma2-9b to extend the the max context length from 8k up to 128k.
I using this command:
python -m vllm.entrypoints.openai.api_server --model NousResearch/Meta-Llama-3.1-8B-Instruct  --max-model-len 12000 --gpu-memory-utilization 0.95 --rope-scaling '{"factor": 8.0, "type": "dynamic"}'

I don't know how to use/tune these args --rope-scaling  and --rope-theta to be assure that i can serve 128k tokens per requests (or context length is extended to 128k tokens).
what do they mean?
Hardware A10 GPU with 24G RAM
btw I got this error:
$ python -m vllm.entrypoints.openai.api_server --model NousResearch/Meta-Llama-3.1-8B-Instruct  --max-model-len 120000 --gpu-memory-utilization 0.95 --rope-scaling '{"factor": 8.0, "type": "dynamic"}'
INFO 11-21 13:28:24 api_server.py:585] vLLM API server version 0.6.4.post1
INFO 11-21 13:28:24 api_server.py:586] args: Namespace(host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='NousResearch/Meta-Llama-3.1-8B-Instruct', task='auto', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', chat_template_text_format='string', trust_remote_code=False, allowed_local_media_path=None, download_dir=None, load_format='auto', config_format=<ConfigFormat.AUTO: 'auto'>, dtype='auto', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=120000, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=1, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=False, disable_sliding_window=False, use_v2_block_manager=False, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.95, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling={'factor': 8.0, 'type': 'dynamic'}, rope_theta=None, hf_overrides=None, enforce_eager=False, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=None, qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', override_neuron_config=None, override_pooler_config=None, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False)
INFO 11-21 13:28:24 api_server.py:175] Multiprocessing frontend to use ipc:/tmp/2a3bc2d6-292b-410b-bb5d-f83383760498 for IPC Path.
INFO 11-21 13:28:24 api_server.py:194] Started engine process with PID 29655
INFO 11-21 13:28:24 config.py:112] Replacing legacy 'type' key with 'rope_type'
INFO 11-21 13:28:29 config.py:350] This model supports multiple tasks: {'embedding', 'generate'}. Defaulting to 'generate'.
WARNING 11-21 13:28:29 arg_utils.py:1013] Chunked prefill is enabled by default for models with max_model_len > 32K. Currently, chunked prefill might not work with some features or models. If you encounter any issues, please disable chunked prefill by setting --enable-chunked-prefill=False.
WARNING 11-21 13:28:29 arg_utils.py:1075] [DEPRECATED] Block manager v1 has been removed, and setting --use-v2-block-manager to True or False has no effect on vLLM behavior. Please remove --use-v2-block-manager in your engine argument. If your use case is not supported by SelfAttnBlockSpaceManager (i.e. block manager v2), please file an issue with detailed information.
INFO 11-21 13:28:29 config.py:1136] Chunked prefill is enabled with max_num_batched_tokens=512.
INFO 11-21 13:28:29 config.py:112] Replacing legacy 'type' key with 'rope_type'
INFO 11-21 13:28:37 config.py:350] This model supports multiple tasks: {'generate', 'embedding'}. Defaulting to 'generate'.
WARNING 11-21 13:28:37 arg_utils.py:1013] Chunked prefill is enabled by default for models with max_model_len > 32K. Currently, chunked prefill might not work with some features or models. If you encounter any issues, please disable chunked prefill by setting --enable-chunked-prefill=False.
WARNING 11-21 13:28:37 arg_utils.py:1075] [DEPRECATED] Block manager v1 has been removed, and setting --use-v2-block-manager to True or False has no effect on vLLM behavior. Please remove --use-v2-block-manager in your engine argument. If your use case is not supported by SelfAttnBlockSpaceManager (i.e. block manager v2), please file an issue with detailed information.
INFO 11-21 13:28:37 config.py:1136] Chunked prefill is enabled with max_num_batched_tokens=512.
INFO 11-21 13:28:37 llm_engine.py:249] Initializing an LLM engine (v0.6.4.post1) with config: model='NousResearch/Meta-Llama-3.1-8B-Instruct', speculative_config=None, tokenizer='NousResearch/Meta-Llama-3.1-8B-Instruct', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=120000, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=NousResearch/Meta-Llama-3.1-8B-Instruct, num_scheduler_steps=1, chunked_prefill_enabled=True multi_step_stream_outputs=True, enable_prefix_caching=False, use_async_output_proc=True, use_cached_outputs=True, chat_template_text_format=string, mm_processor_kwargs=None, pooler_config=None)
INFO 11-21 13:28:40 selector.py:135] Using Flash Attention backend.
INFO 11-21 13:28:43 model_runner.py:1072] Starting to load model NousResearch/Meta-Llama-3.1-8B-Instruct...
INFO 11-21 13:28:44 weight_utils.py:243] Using model weights format ['*.safetensors']
Loading safetensors checkpoint shards:   0% Completed | 0/4 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:  25% Completed | 1/4 [00:00<00:02,  1.43it/s]
Loading safetensors checkpoint shards:  50% Completed | 2/4 [00:01<00:01,  1.16it/s]
Loading safetensors checkpoint shards:  75% Completed | 3/4 [00:02<00:00,  1.10it/s]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:03<00:00,  1.39it/s]
Loading safetensors checkpoint shards: 100% Completed | 4/4 [00:03<00:00,  1.30it/s]

INFO 11-21 13:28:48 model_runner.py:1077] Loading model weights took 15.2075 GB
INFO 11-21 13:28:49 worker.py:232] Memory profiling results: total_gpu_memory=21.99GiB initial_memory_usage=15.53GiB peak_torch_memory=16.38GiB memory_usage_post_profile=15.55GiB non_torch_memory=0.33GiB kv_cache_size=4.17GiB gpu_memory_utilization=0.95
INFO 11-21 13:28:49 gpu_executor.py:113] # GPU blocks: 2136, # CPU blocks: 2048
INFO 11-21 13:28:49 gpu_executor.py:117] Maximum concurrency for 120000 tokens per request: 0.28x
ERROR 11-21 13:28:49 engine.py:366] The model's max seq len (120000) is larger than the maximum number of tokens that can be stored in KV cache (34176). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.
ERROR 11-21 13:28:49 engine.py:366] Traceback (most recent call last):
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 357, in run_mp_engine
ERROR 11-21 13:28:49 engine.py:366]     engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 119, in from_engine_args
ERROR 11-21 13:28:49 engine.py:366]     return cls(ipc_path=ipc_path,
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 71, in __init__
ERROR 11-21 13:28:49 engine.py:366]     self.engine = LLMEngine(*args, **kwargs)
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/llm_engine.py", line 350, in __init__
ERROR 11-21 13:28:49 engine.py:366]     self._initialize_kv_caches()
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/llm_engine.py", line 500, in _initialize_kv_caches
ERROR 11-21 13:28:49 engine.py:366]     self.model_executor.initialize_cache(num_gpu_blocks, num_cpu_blocks)
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/executor/gpu_executor.py", line 120, in initialize_cache
ERROR 11-21 13:28:49 engine.py:366]     self.driver_worker.initialize_cache(num_gpu_blocks, num_cpu_blocks)
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/worker/worker.py", line 268, in initialize_cache
ERROR 11-21 13:28:49 engine.py:366]     raise_if_cache_size_invalid(num_gpu_blocks,
ERROR 11-21 13:28:49 engine.py:366]   File "/home/vllm-env/lib/python3.9/site-packages/vllm/worker/worker.py", line 498, in raise_if_cache_size_invalid
ERROR 11-21 13:28:49 engine.py:366]     raise ValueError(
ERROR 11-21 13:28:49 engine.py:366] ValueError: The model's max seq len (120000) is larger than the maximum number of tokens that can be stored in KV cache (34176). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/home/vllm-env/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/vllm-env/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 368, in run_mp_engine
    raise e
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 357, in run_mp_engine
    engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 119, in from_engine_args
    return cls(ipc_path=ipc_path,
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 71, in __init__
    self.engine = LLMEngine(*args, **kwargs)
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/llm_engine.py", line 350, in __init__
    self._initialize_kv_caches()
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/llm_engine.py", line 500, in _initialize_kv_caches
    self.model_executor.initialize_cache(num_gpu_blocks, num_cpu_blocks)
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/executor/gpu_executor.py", line 120, in initialize_cache
    self.driver_worker.initialize_cache(num_gpu_blocks, num_cpu_blocks)
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/worker/worker.py", line 268, in initialize_cache
    raise_if_cache_size_invalid(num_gpu_blocks,
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/worker/worker.py", line 498, in raise_if_cache_size_invalid
    raise ValueError(
ValueError: The model's max seq len (120000) is larger than the maximum number of tokens that can be stored in KV cache (34176). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.
[rank0]:[W1121 13:28:49.681361114 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present,  but this warning has only been added since PyTorch 2.4 (function operator())
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<MQLLMEngineClient.run_output_handler_loop() done, defined at /home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/client.py:178> exception=ZMQError('Operation not supported')>
Traceback (most recent call last):
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/client.py", line 184, in run_output_handler_loop
    while await self.output_socket.poll(timeout=VLLM_RPC_TIMEOUT
  File "/home/vllm-env/lib/python3.9/site-packages/zmq/_future.py", line 382, in poll
    raise _zmq.ZMQError(_zmq.ENOTSUP)
zmq.error.ZMQError: Operation not supported
Traceback (most recent call last):
  File "/home/vllm-env/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/vllm-env/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 643, in <module>
    uvloop.run(run_server(args))
  File "/home/vllm-env/lib/python3.9/site-packages/uvloop/__init__.py", line 82, in run
    return loop.run_until_complete(wrapper())
  File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
    return future.result()
  File "/home/vllm-env/lib/python3.9/site-packages/uvloop/__init__.py", line 61, in wrapper
    return await main
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 609, in run_server
    async with build_async_engine_client(args) as engine_client:
  File "/home/vllm-env/lib/python3.9/contextlib.py", line 181, in __aenter__
    return await self.gen.__anext__()
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 113, in build_async_engine_client
    async with build_async_engine_client_from_engine_args(
  File "/home/vllm-env/lib/python3.9/contextlib.py", line 181, in __aenter__
    return await self.gen.__anext__()
  File "/home/vllm-env/lib/python3.9/site-packages/vllm/entrypoints/openai/api_server.py", line 210, in build_async_engine_client_from_engine_args
    raise RuntimeError(
RuntimeError: Engine process failed to start. See stack trace for the root cause.

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

@imkero @DarkLight1337 could you please help?

You may have to set max_num_seqs to a small value (e.g. 2) to avoid OOM.

@DarkLight1337 So this represent the concurrent num sequences (requests) can be running inferencing at the same time?

Yes

How do these args influence the max-context length? how to use the best values or to tune them?

Since more memory is needed for longer context length, you should reduce max_num_seqs so that the overall memory usage remains within your GPU VRAM.

Sorry I mean the ROPE args [--rope-scaling  and --rope-theta] how they influence the extended max context length?

@DarkLight1337 what does it mean this?

WARNING 11-24 11:04:06 arg_utils.py:1013] Chunked prefill is enabled by default for models with max_model_len > 32K. Currently, chunked prefill might not work with some features or models. If you encounter any issues, please disable chunked prefill by setting --enable-chunked-prefill=False.

It means if your model produces incorrect results, you should disable chunked prefill.

@DarkLight1337
I can run llama3.1-70b-awq with context length 120k on GPU with 80G RAM. and I can send a prompt of length 110k and generate 5k completion tokens.
However I couldn't for gemma2-9b model. I got this error:

command:
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 python -m vllm.entrypoints.openai.api_server --model google/gemma-2-9b-it   --max-model-len 120000 --max-num-seqs 2  --gpu-memory-utilization 0.95 --rope-scaling '{"factor": 8.0, "type": "dynamic"}'

error:

INFO 11-24 11:38:03 metrics.py:449] Avg prompt throughput: 89.4 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 62.8%, CPU KV cache usage: 0.0%.
INFO 11-24 11:38:08 metrics.py:449] Avg prompt throughput: 5001.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 62.8%, CPU KV cache usage: 0.0%.
INFO 11-24 11:38:14 metrics.py:449] Avg prompt throughput: 2788.9 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 62.8%, CPU KV cache usage: 0.0%.
ERROR 11-24 11:38:14 engine.py:135] RuntimeError('CUDA error: an illegal memory access was encountered\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n')

As before, I'm not involved with this part so I can't help much. Probably the model doesn't support long context well.

Anyone can help in this? please could you tag him/her?

full log:
INFO 11-24 11:45:06 engine.py:267] Added request chatcmpl-7220ee5627654025b2d55e07797abe82.
INFO 11-24 11:45:07 metrics.py:449] Avg prompt throughput: 62.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 60.3%, CPU KV cache usage: 0.0%.
INFO 11-24 11:45:12 metrics.py:449] Avg prompt throughput: 3362.8 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 60.3%, CPU KV cache usage: 0.0%.
INFO 11-24 11:45:17 metrics.py:449] Avg prompt throughput: 2245.5 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 60.3%, CPU KV cache usage: 0.0%.
INFO 11-24 11:45:22 metrics.py:449] Avg prompt throughput: 1956.3 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 1 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 60.3%, CPU KV cache usage: 0.0%.
INFO 11-24 11:45:23 model_runner_base.py:120] Writing input of failed execution to /tmp/err_execute_model_input_20241124-114523.pkl...
WARNING 11-24 11:45:23 model_runner_base.py:143] Failed to pickle inputs of failed execution: CUDA error: an illegal memory access was encountered
WARNING 11-24 11:45:23 model_runner_base.py:143] Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
WARNING 11-24 11:45:23 model_runner_base.py:143]
CRITICAL 11-24 11:45:23 launcher.py:99] MQLLMEngine is already dead, terminating server process
INFO:     <IP_ADDRESS>:58389 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
ERROR 11-24 11:45:23 engine.py:135] RuntimeError('Error in model execution: CUDA error: an illegal memory access was encountered\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n')
ERROR 11-24 11:45:23 engine.py:135] Traceback (most recent call last):
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/worker/model_runner_base.py", line 116, in _wrapper
ERROR 11-24 11:45:23 engine.py:135]     return func(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/worker/model_runner.py", line 1654, in execute_model
ERROR 11-24 11:45:23 engine.py:135]     hidden_or_intermediate_states = model_executable(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 11-24 11:45:23 engine.py:135]     return self._call_impl(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 11-24 11:45:23 engine.py:135]     return forward_call(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/model_executor/models/gemma2.py", line 425, in forward
ERROR 11-24 11:45:23 engine.py:135]     hidden_states = self.model(input_ids, positions, kv_caches,
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/compilation/decorators.py", line 143, in __call__
ERROR 11-24 11:45:23 engine.py:135]     return self.forward(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/model_executor/models/gemma2.py", line 297, in forward
ERROR 11-24 11:45:23 engine.py:135]     hidden_states, residual = layer(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 11-24 11:45:23 engine.py:135]     return self._call_impl(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 11-24 11:45:23 engine.py:135]     return forward_call(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/model_executor/models/gemma2.py", line 229, in forward
ERROR 11-24 11:45:23 engine.py:135]     hidden_states = self.self_attn(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 11-24 11:45:23 engine.py:135]     return self._call_impl(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 11-24 11:45:23 engine.py:135]     return forward_call(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/model_executor/models/gemma2.py", line 169, in forward
ERROR 11-24 11:45:23 engine.py:135]     attn_output = self.attn(q, k, v, kv_cache, attn_metadata)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
ERROR 11-24 11:45:23 engine.py:135]     return self._call_impl(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
ERROR 11-24 11:45:23 engine.py:135]     return forward_call(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/attention/layer.py", line 99, in forward
ERROR 11-24 11:45:23 engine.py:135]     return self.impl.forward(query,
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/attention/backends/flash_attn.py", line 671, in forward
ERROR 11-24 11:45:23 engine.py:135]     output = torch.ops.vllm.unified_flash_attention(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/_ops.py", line 1116, in __call__
ERROR 11-24 11:45:23 engine.py:135]     return self._op(*args, **(kwargs or {}))
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/attention/backends/flash_attn.py", line 890, in unified_flash_attention
ERROR 11-24 11:45:23 engine.py:135]     prefill_output = flash_attn_varlen_func(  # noqa
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/vllm_flash_attn/flash_attn_interface.py", line 1155, in flash_attn_varlen_func
ERROR 11-24 11:45:23 engine.py:135]     return FlashAttnVarlenFunc.apply(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/autograd/function.py", line 575, in apply
ERROR 11-24 11:45:23 engine.py:135]     return super().apply(*args, **kwargs)  # type: ignore[misc]
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/vllm_flash_attn/flash_attn_interface.py", line 633, in forward
ERROR 11-24 11:45:23 engine.py:135]     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _flash_attn_varlen_forward(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/vllm_flash_attn/flash_attn_interface.py", line 91, in _flash_attn_varlen_forward
ERROR 11-24 11:45:23 engine.py:135]     out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = torch.ops.vllm_flash_attn_c.varlen_fwd(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/_ops.py", line 1116, in __call__
ERROR 11-24 11:45:23 engine.py:135]     return self._op(*args, **(kwargs or {}))
ERROR 11-24 11:45:23 engine.py:135] RuntimeError: CUDA error: an illegal memory access was encountered
ERROR 11-24 11:45:23 engine.py:135] Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
ERROR 11-24 11:45:23 engine.py:135]
ERROR 11-24 11:45:23 engine.py:135]
ERROR 11-24 11:45:23 engine.py:135] The above exception was the direct cause of the following exception:
ERROR 11-24 11:45:23 engine.py:135]
ERROR 11-24 11:45:23 engine.py:135] Traceback (most recent call last):
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 133, in start
ERROR 11-24 11:45:23 engine.py:135]     self.run_engine_loop()
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 196, in run_engine_loop
ERROR 11-24 11:45:23 engine.py:135]     request_outputs = self.engine_step()
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 214, in engine_step
ERROR 11-24 11:45:23 engine.py:135]     raise e
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/engine/multiprocessing/engine.py", line 205, in engine_step
ERROR 11-24 11:45:23 engine.py:135]     return self.engine.step()
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/engine/llm_engine.py", line 1454, in step
ERROR 11-24 11:45:23 engine.py:135]     outputs = self.model_executor.execute_model(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/executor/gpu_executor.py", line 125, in execute_model
ERROR 11-24 11:45:23 engine.py:135]     output = self.driver_worker.execute_model(execute_model_req)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/worker/worker_base.py", line 343, in execute_model
ERROR 11-24 11:45:23 engine.py:135]     output = self.model_runner.execute_model(
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
ERROR 11-24 11:45:23 engine.py:135]     return func(*args, **kwargs)
ERROR 11-24 11:45:23 engine.py:135]   File "/root/miniconda3/envs/vllm-env/lib/python3.9/site-packages/vllm/worker/model_runner_base.py", line 146, in _wrapper
ERROR 11-24 11:45:23 engine.py:135]     raise type(err)(f"Error in model execution: "
ERROR 11-24 11:45:23 engine.py:135] RuntimeError: Error in model execution: CUDA error: an illegal memory access was encountered
ERROR 11-24 11:45:23 engine.py:135] Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
ERROR 11-24 11:45:23 engine.py:135]
INFO:     Shutting down

@WoosukKwon @youkaichao

i think gemma2 only supports 8k context.

@youkaichao
ROPE args [--rope-scaling and --rope-theta] how they influence the extended max context length?

@youkaichao do they have effect on the model while inferencing when we set these values with vllm?

@youkaichao How it gonna extend the context length? can you elaborate? if i need to extend length of llama3.1-8b from 8k to 128k, can I do that? how? by only using these args? --rope-scaling and --rope-theta
how to configure them?

any updates on this? i am also looking to extend context len of gemma2 beyond 8192

@youkaichao I need to extend llama3.1-8b (which is trained on 8k length) to 120k length.
Can I do it using these args --rope-scaling '{"factor": 15.0, "type": "dynamic"}'  --rope-theta 1200000  in inference time?

after reading the full thread:
to extend llama3.1-8b (which is trained on 8k length) to 120k length , you should add:
--max-model-len 12000 --rope-scaling '{"factor": 8.0, "type": "dynamic"}' --rope-theta 1200000
--max-model-len 12000 tells vllm the model supports 120k context, and --rope-scaling '{"factor": 8.0, "type": "dynamic"}' --rope-theta 1200000 tells vllm how to support 120k context.
the error from the original post:

ValueError: The model's max seq len (120000) is larger than the maximum number of tokens that can be stored in KV cache (34176). Try increasing gpu_memory_utilization or decreasing max_model_len when initializing the engine.

means your gpu does not have enough gpu memory to serve 120k context. you should try a GPU with larger memory, or try --cpu-offload-gb to offload some model weights to CPU so that kv cache can be larger.
any remaining questions?

Thanks @youkaichao for your response.
However, why we dont use factor=15 which is equal to new_context_length / old_contect_length  = 120k/8k?
what is the equation you used to set the rope theta? I check this paper and they provide with equation that you result in OverflowError: math range error to compute it.
also the equation is mentioned here at this blog

I don't know how to set these parameters :(


redirect to spawn page

as I stop everware instance gracefully, then start it again, and then open in the browser URL:
http://localhost:8000/hub/user/anaderi/tree
it says:

Your server is starting up.
This can take seconds or minutes, please be patient.
You will be redirected automatically when it's ready for you.
This page is updated automatically.

while in the logs it complains that no such container anaderi-jupyter :
but apparently nothing is being created. in logs:
[D 2016-04-28 00:33:13.714 Everware log:100] 304 GET /hub/user/anaderi/tree?get_logs=1<EMAIL_ADDRESS>8.00ms
[D 2016-04-28 00:33:16.217 Everware spawner:226] Getting container: jupyter-anaderi
[I 2016-04-28 00:33:16.221 Everware spawner:234] Container 'jupyter-anaderi' is gone

IMO, if there is no such container , it should redirect to 'spawn' page.
PS hasn't checked other user pages, but probably those also need such redirection.

Need to redirect if there is no build in progress and it's not failed


Added EMPD2 samples contributed by Joannin

Hi! I'm your friendly automated EMPD-admin bot!
Thank you very much for your data submission! I will now run some tests on your data.
In the meantime: please review your test data on the EMPD viewer using
this link: https://EMPD2.github.io/?repo=EMPD2/EMPD-data&branch=Joannin&meta=Joannin.tsv
This bot is helping you merging your data. I am checking for common issues and I can fix your climate, country or elevation data.
Type @EMPD-admin help in a new comment in this PR for usage information.
Make sure, that all the commands for the @EMPD-admin are on one single line and nothing else. Only
@EMPD-admin help
will work, not something like
<other words> @EMPD-admin help
or
@EMPD-admin help <other words>
Note that you can also run these tests on your local machine with
pytest.

Hi! I'm your friendly automated EMPD-admin bot!
This is just to inform you that I tested your data submission in your PR (Joannin.tsv) and found it in an excellent condition!

@EMPD-admin finish --commit

Hi! I'm your friendly automated EMPD-admin bot!
I processed your command and hope that I can help you!

finish --commit

Finished the PR!
You may want to have a final look into the viewer (https://EMPD2.github.io/?repo=EMPD2/EMPD-data&branch=Joannin) and then merge it.


conditionally hidden fields are never shown for (android, iOS) compiled apps (permanently not showing whether or not they are hide on event or show on event)

not sure how to go about debugging this. if there is something i can do, please let me know!

fields either are either permanently shown, or never show, even though running on web  / livereload  works fine

I will need to hook up an app with GapDebug to get to the bottom of this. Are you using Ionic?

yes, using ionic

I'm trying to create a demo ionic project for you...
when i try to compile there is an error. following suggestion from Toub on https://github.com/webpack/webpack/issues/4072
we can see it's trying to find ng2-formio/dist/formio.config which doesn't exist ... do you have an old reference somewhere?
File not found:  HarmonyImportDependency {
module: null,
request: 'ng2-formio/dist/formio.config',
userRequest: 'ng2-formio/dist/formio.config',
range: [ 1065, 1129 ],
importedVar: 'WEBPACK_IMPORTED_MODULE_9_ng2_formio_dist_formio_config',
loc: SourceLocation { index: -1 } }

ignore previous comment. it was my bad in the app using a previous rc

@travist  loading using raw JSON fails with hidden fields
I have compiled an apk here that you can try: https://filebin.ca/3NwPF1pQLhSr/test.apk
The following code is the key piece, rending the form twice, one using server, the other using JSON.
<formio src='https://auicqnmuwlxvnoh.form.io/form4'></formio>
<formio [form]='{"components":[{"key":"panel2","input":false,"title":"","theme":"default","components":[{"conditional":{"eq":"","when":null,"show":""},"tags":[],"type":"checkbox","validate":{"required":false},"clearOnHide":true,"value":"","name":"","hidden":false,"persistent":true,"protected":false,"defaultValue":false,"key":"showdatagrid","datagridLabel":true,"label":"show datagrid","hideLabel":true,"tableView":true,"inputType":"checkbox","input":true},{"conditional":{"eq":"true","when":"showdatagrid","show":"true"},"tags":[],"type":"datagrid","clearOnHide":true,"hidden":false,"persistent":true,"protected":false,"key":"datagrid","label":"datagrid","tableView":true,"components":[{"conditional":{"eq":"","when":null,"show":""},"tags":[],"hideLabel":true,"type":"columns","columns":[{"components":[{"tags":[],"hideLabel":true,"type":"textfield","conditional":{"eq":"","when":null,"show":""},"validate":{"customPrivate":false,"custom":"","pattern":"","maxLength":"","minLength":"","required":false},"clearOnHide":true,"hidden":false,"persistent":true,"unique":false,"protected":false,"defaultValue":"","multiple":false,"suffix":"","prefix":"","placeholder":"","key":"name","label":"name","inputMask":"","inputType":"text","tableView":true,"input":true}]},{"components":[{"tags":[],"type":"textfield","conditional":{"eq":"","when":null,"show":""},"validate":{"customPrivate":false,"custom":"","pattern":"","maxLength":"","minLength":"","required":false},"clearOnHide":true,"hidden":false,"persistent":true,"unique":false,"protected":false,"defaultValue":"","multiple":false,"suffix":"","prefix":""  ,"placeholder":"","key":"yodel","label":"yodel","inputMask":"","inputType":"text","tableView":true,"input":true}]}],"key":"columns","input":false}],"tree":true,"input":true}],"type":"panel","tags":[],"conditional":{"show":"","when":null,"eq":""}}],"type":"panel","tags":[],"conditional":{"show":"","when":null,"eq":""}}'> </formio>

I can share the code with you if that would help.
The first instance of the form works fine. the JSON version of the form immediately after does not.

code is available here
https://github.com/mtsang/formiobuildissue.git
running build.sh should build it..

In addition, it appears as though loading data into the form also doesn't work if loaded from JSON (when it is built to ios or android...)

@travist, does the ng2-formio component know how to read from a src=file:// URI ?
(thinking of using this workaround until the root issue gets resolved...)

this is fixed with rc.33


BGP is not advertising

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[ x ] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[ x ] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
I upgraded from 23.1.11_1 to 23.7.1_3 and it broke the BGP. BGP is not sending updates. I have three sites and so far I have upgraded the main and a remote sites. The 2nd remote site is still at 23.1.11. The 23.1.11 is still sending routes to 23.7.1, but not receiving any BGP routes from 23.7.1 firewalls.
Both 23.7.1 firewalls are receiving routes from the 23.1.11.
Tip: to validate your setup was working with the previous version, use opnsense-revert (https://docs.opnsense.org/manual/opnsense_tools.html#opnsense-revert)
To Reproduce
Steps to reproduce the behavior:

Upgrade to 23.7.1
Enable eBGP
Check the route table
The route table will not show any routes from 23.7.1 firewalls

Expected behavior
The expected behavior should be the peer BGP firewalls should send BGP routes to its neighbors.
Describe alternatives you considered
N/A
Screenshots
N/A
Relevant log files
2023-08-21T20:15:15	Warning	bgpd	[YTHK0-FSPPJ][EC 33554500] sendmsg_nexthop: zclient_send_message() failed
2023-08-21T20:15:14	Warning	zebra	[VXKFG-8SJRV][EC<PHONE_NUMBER>] Client 'bgp' encountered an error and is shutting down.
2023-08-21T20:15:14	Warning	zebra	[VXKFG-8SJRV][EC<PHONE_NUMBER>] Client 'vnc' encountered an error and is shutting down.
2023-08-21T20:15:05	Error	bgpd	[MVZKX-EG443][EC 33554452] bgp_process_packet: BGP OPEN receipt failed for peer: <IP_ADDRESS>

Additional context
N/A
Environment
Software version used and hardware type if relevant, e.g.:
OPNsense 23.7.1_3-amd64
FreeBSD 13.2-RELEASE-p2
OpenSSL 1.1.1v 1 Aug 2023

Same as this https://github.com/opnsense/plugins/issues/3520


[otp/testing] device_id being set randomly makes testing difficult

I'm working on writing a test that ensures the MaskROM is checking the manifest usage constraints correctly against the hardware. This involves toggling the 4 selector bits in the manifest and testing each of the constraints. One of these usage constraints is the DEVICE_ID which is currently being set randomly in the OTP (e.g. https://github.com/lowRISC/opentitan/blob/master/hw/ip/otp_ctrl/data/otp_ctrl_img_rma.hjson#L268). This makes testing it somewhat difficult. Can this value be set to a constant?

@cfrantz @msfschaffner

yeah i dont see why not. For DV purposes, we have other ways of overriding this to be a random value for our purposes.
@cindychip @sriyerg

Fine with me - though lets keep this open until we re-randomize it in DV.

@milesdai is this addressed? can we close the issue?

Not yet -- I'm returning to this test soon and will deal with this issue then.

Ok sounds good, thanks!

This problem is solved by creating an overlay that overwrites the the device ID in Bazel.


Full Layout Restore Seems to Incorrectly Restore Splits

I saved the current layout to a member variable FullAppLayout and it didn't restore the split sizes correctly.

I reproduced this in the demo app. I think it's just a matter of the timing getting the current layout. I don't think the technique I'm using will be good enough (dock everything in their default locations and then immediately grab the layout as the "default"). I think I will have to use the layout builder to build the default layout, then either restore from the layout file if there is one or restore the default.


please add linux support...and deb files

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

https://github.com/Trackhe/kitematic/releases

@thakkarsagar12 / @Trackhe assuming this one can be closed? confirm that I can install and run kitematic on Ubuntu 18.04.4 LTS, preconditions to be installed:

xterm
docker
docker-compose (optional)

note, I should restart Ubuntu after kitematic installation to make it work...


[honeyswap] Added Honeyswap strategy

Fixes # .
Changes proposed in this pull request:

Added Honeyswap strategy

Thanks for the advice i went ahead and made some edits let me know if its good now! I run npm test and it passes but it did it last time as well.

Strategy is live: https://snapshot.org/#/strategy/honeyswap


[DOCFIX] Fix Journal management docs

@ggezer PTAL, thanks

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4421/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/4422/
Test PASSed.

alluxio-bot, merge this please.

alluxio-bot, cherry-pick this to branch-2.0 please.


embedding videos from BR Mediathek

Can we add BR Mediathek as a source option for our video learning object?
e.g. https://www.br.de/mediathek/video/respekt-10062018-pluralismus-warum-die-demokratie-von-mehr-vielfalt-profitiert-av:5abbbdeb512d0600188593cc
embedding code can be found if you click on "Teilen"

Yes, should be possible. Are the videos always online (instead of the timed deletion because legal stuff in ARD + ZDF mediathek)?
TODO:

[ ] Map https://www.br.de/mediathek/video/respekt-10062018-pluralismus-warum-die-demokratie-von-mehr-vielfalt-profitiert-av:5abbbdeb512d0600188593cc to https://www.br.de/mediathek/embed/respekt-10062018-pluralismus-warum-die-demokratie-von-mehr-vielfalt-profitiert-av:5abbbdeb512d0600188593cc
[ ] Use provided embed code with that url (or some similar embed code)

If you use the provided embed code, you don't have to map, it is ready to use:

Unfortunately, the videos are not always online - only for 5 years...

@br-oliver:  That's true, but we don't want the user to have to enter that code. So what we are probably going to do is that the user provides the usual video url and we use that entered url and render the embed code.
@SimonKoehl Is the 5-year limit a problem? Resp @br-oliver: do you know if the embedding handles "expired" videos correctly and shows a good error message (Like: "Leider ist dieser Inhalt nicht mehr verfügbar?")? Or can we check via API when the video will expire? Then it wouldn't be such a big problem IMHO if we only use BR videos to enrich the content but don't rely on them to be there.

The reason I'm asking: It would be a shame if a good learning entity relies on the video to be there since then the content has to be rewritten after at most 5 years (depending on how long the video is already online). Furthermore: @SimonKoehl: What about the license? If it's cc anyway, we might be better off downloading the video and reuploading it to e.g. Wikimedia Commons (the license would allow that). If not, we have a slight license problem and need to indicate the different license in the content (which we don't do currently I suppose since we only show the license of the "outer" entity and not the embedded one).

About license: embeded videos also show the license

@inyono As for the error message: yes. But the licence is not CC, so you would have to use the embedding method to display the content-  then you are legally fine.
The API (in construction) will deliver the date the video expires - and, fyi,  the laws that force public stations to take down their content are in discussion and about to change - in favour of longer availability of videos - we will know details next year.

@Knorrke Nice, didn't know.
Okay, everything's fine by me, then 👍

ok, so let's get started with the current circumstances and hope for better laws regarding the expire date and licences in public broadcasting in the future

@SimonKoehl can you create the license for those videos?

@Knorrke done https://de.serlo.org/license/manage