Raw H264 stream: problem with framerate

Hi, I can stream out of a RPi the raw data of an H264 video.
If I want to obtain a well-formed MP4 file, I would save the video as H264 and then wrap into an MP4 container:

ffmpeg -r 25 -i raw_video.h264 -vcodec copy wrapped_video.mp4
or
MP4Box -add raw_video.h264 wrapped_video.mp4
It works, but this is of course not possible when streaming.

If I save the raw H264 and play it with CreateFFmpegInteropMSSFromStream, the video is super-fast. I suppose the problem is due to the missing framerate.
From the ffmpeg rtsp help I don't see any useful options to make it work.
Suggestions?
TIA

Can you try with the code from pull request #163?
We have an issue with small samples that might address the fast playback.

I cloned the repo, pulled the #163, recompiled the Win10 solution and run the C# example.
When I load the H264 raw video file, the result is:
MF_MEDIA_ENGINE_ERR_DECODE : HRESULT - 0xC00D3E84
I never had this error in my previous tests ... do you have any idea?
Using another H264 file I can see the first frame but the video doesn't play.
Is there some ffmpeg option to pass in the propertyset to support the H264 raw data?

Hi @khouzam thanks again for your support.
I did other tests with my own UWP app using the modified control but the problem is exactly the same.
Do you have any other suggestions?

Hi @khouzam sorry for pinging, but I was wondering if you have any solution for this problem.
Thanks

Hi @raffaeler
Is there a repro I can run on my side to see what's going on?

Would you like me to send a raw video captured with the default Raspberry PI utility?
It is not "processed" in any way, just raw H264 stream saved on disk.
I am trying to show it as a stream without encapsulate in the MP4 container.

Yes, please, anything that can replicate the issues that you are seeing.

Hi @khouzam here is a video captured as a raw stream.
This specific video can be opened with VNC (this isn't always true). But if I try to watch it with FFmpegInterop, it plays in about 1 second instead its duration is about 12 seconds.
raw-video.zip
Please let me know if you need another one (different length, etc.)
Thank you

Ok, I've found the issue. Because this is a raw stream, the timestamps for the samples aren't and we're not handling this case properly. It should be a simple fix, because the samples do have duration, so we'd just have to keep increasing the timestamps for the current position based on the sample durations

Awesome. Do you believe to bea ble to make the fix before Wed? If I have time I would like to show the streaming in my demo @ Ignite next week.

Here you go, please try the fix on https://github.com/Microsoft/FFmpegInterop/tree/gillesk/fixNoPts.

I just fetched the branch, recompiled everytyhing and it works!
I will make other tests but it looks amazing.
Thank you

I've created Pull Request #171 to address the issue. As mentioned above, we'll keep track of the current presentation time based on the timestamp and the duration of samples as we're playing a stream and if a sample doesn't have a timestamp, it will infer the timestamp from the running timestamp.

@khouzam how many samples do you need to make the CreateFFmpegInteropMSSFromStream return a valid decoder?
I modified my sample in order to read the bytes from a websocket but CreateFFmpegInteropMSSFromStream returns null.
TIA

I'm not sure about this. It will all depend on the how much data FFmpeg needs to identify the stream, this is not something that FFmpegInterop is doing. On the other hand, we're creating a wrapper on the stream, perhaps this is what is failing. Would it be possible to debug into this?

Yes I identified the problem two minutes ago.
The H264 stream begins with 00 00 00 01. If the stream already started, FFmpegInterop does not understand the 'random' sequence of bytes and failed.
I expected instead that it continue to read data (and discard) until finding the magic sequence.

BTW I am trying right now to identify the magic sequence and see if there is anything else to do.

@khouzam  Apparently this is not enough. Even after recognizing the magic bytes sequence and feeding the decode with that, the factory method returns null.
I am not an expert of H264 therefore I don't know if the beginning of the sequence (the beginning of the file in your case) is special.

I tried removing random bytes from the raw file and it keeps working. I'm guessing that FFmpegInterop.cpp@205 is failing this call:
		// Convert asynchronous IRandomAccessStream to synchronous IStream. This API requires shcore.h and shcore.lib
		hr = CreateStreamOverRandomAccessStream(reinterpret_cast<IUnknown*>(stream), IID_PPV_ARGS(&fileStreamData));

The web socket is most likely not castable to a RandomAccessStream.

@khouzam
After recognizing the correct byte[] sequence, now I get a valid decoder,  but still doesn't work.
My steps are:

I create an InMemoryRandomAccessStream as an intermediary buffer since the FFmpegInterpo need an IRandomAccessStream.
I pull the data correctly from the websocket
I write the data to the memory stream using the output stream obtained (before looping) with GetOutputStreamAt(0)
But now the FFmpegInterop does not pull from my memory stream (is position is always 0).

@khouzam how many bytes do you need in the stream before rendering the image?
I see that if I buffer the data coming from the websocket, I can see an image (just the first frame) but then it fails with a weird c++ exception as if it is looking for "future" data that is not still available.
If the stream is a file stream, you have all the data at once, but processing from the network is of course very different...

@khouzam do you believe it is possible to have a short chat together to explain what is not working now.
The goal is very close but the live stream still doesn't work.
TIA

Sure,
Can you send me an email and we can set something up.

@khouzam
I got a same problem as raffaeler, we have a media server wrap the raw h264 to flv and stream out through the websocket.
no we want play it with uwp app, and use the ffmpeginterop with the same problem.
the mediaelement control only can play the stream for few seconds when i change the maxdelay to 1000.
then the mediaelement stoped play, but websocket is still working on receive data.
         _ws = new StreamWebSocket();
        await _ws.ConnectAsync(new Uri(wsuri.Text, UriKind.Absolute));
        var stream = new Windows.Storage.Streams.InMemoryRandomAccessStream();
        var output = stream.GetOutputStreamAt(0);
        var writer = new DataWriter(output);
        int maxdelay = 1000;
        int delay = maxdelay;
        int index = 0;
        bool isplayed = false;
        DataReader reader = new DataReader(_ws.InputStream);
        do
        {
            var loaded = await reader.LoadAsync(10240);
            var buf = reader.ReadBuffer(loaded);
            writer.WriteBuffer(buf);
            await writer.FlushAsync();
            await writer.StoreAsync();

if (delay > 0)
{
delay--;
Debug.Write("B");
                continue;
            }
            // //delay = maxdelay;

            //Debug.Write(".");
            Debug.WriteLine(media.CurrentState);
            try
            {
                //await output.WriteAsync(buf);
               delay = maxdelay;
                isCreated = true;
                    //stream.Seek((ulong)0);
                    var decoder = FFmpegInterop.FFmpegInteropMSS.CreateFFmpegInteropMSSFromStream(
                        stream, false, false, options);
                    if (decoder == null)
                    {
                        Debug.WriteLine("decoder was null, will retry");
                        continue;
                    }

                    var source = decoder.GetMediaStreamSource();
                    media.RealTimePlayback = true;
                    media.SetMediaStreamSource(source);
                    media.Play();

            }

catch (Exception err)
{
Debug.WriteLine(err.ToString());
stream.Dispose();
reader.Dispose();
return;
}
        }
        while (true);

Creating an InMemoryRandomAccessStream will not work as it will only copy the current portion of the data. There needs to be a wrapper around the WebSocket that provides the stream interface to the MediaStreamSource

After thinking about it some more, the easiest thing that might be done would be to mimic the way we wrap the RandomAccessStream into an IStream interface and wrap the WebSocket inputstream. We could then have a CreateMediaStreamSource taking a WebSocket as an input parameter.
I haven't tried this but it should be the easiest step forward.

I have try the way not just as FFmegainterop does, return a sample from websocket. and set a mediasource to mediaelement, it works.
but if i want use the FFmegainterop, is it possible.
I don't understand what your meaning about to mimic the way we wrap the RandomAccessStream into an IStream interface and wrap the WebSocket inputstream.
I try to change the parameter to IIputstream of FFmpegInteropMSS::CreateMediaStreamSource()
but i see you cast the RandomAccessStream to Istream.
hr = CreateStreamOverRandomAccessStream(reinterpret_cast<IUnknown*>(stream), IID_PPV_ARGS(&fileStreamData));
how can i Cast the IInputstream to a istream?

@liujingsheng I did not understand if @khouzam are asking us to make a test, or if instead he is anticipating a change for the ffmpeginterop codebase that he will do.
@khouzam could you please clarify?
TIA

@khouzam do you have any update? Thanks :)

@raffaeler Can't you just use CreateFFmpegInteropMSSFromUri? This is how streaming scenarios are usually handled. If you can't use that, you need to add overloads to FFmpegInterop which take IInputStream. You must provide a custom implementation for FileStreamRead which directly works on IInputStream (it can't be wrapped to IStream). Pass NULL as FileStreamSeek in avio_alloc_context, since the stream cannot be seeked.

@lukasf I can't because I am streaming using websockets from the RPi.
But as you can read from the previous posts, I am already trying to read from the stream, but this is not enough:

the ffmpeginterop does not identify the h364 key frame. So I decoded manually the frames and start feeding the component with the h264 as I see the key frames.
even after that, the ffmpeginterop tries to read more than is currently available from the stream, and then hangs. I can therefore see few images and then stops.

I published the example and I am not alone with this problem, therefore I believe there is a problem in the ffmpeginterop component.

You absolutely cannot use InMemoryRandomAccessStream for your scenario. As I wrote, you need to get the IInputStream from your WebSocket. Then have ffmpeg read directly from that IInputStream, by providing a custom FileStreamRead implementation. Only this will make sure that ffmpeg will not read beyond what is available.

You have 2 choices, depending on how brave you are.

use the easy way lukasf describes.

create your own implementation of IRandomAccessStream wrapping an InputStream. This is much more complicated, but it guves you more flexibility in the long term.

Either way, when you run out of data you need to provide mock frames if the stream isnt over. I dont think ffmpeg interop supports this scebario.

I do not see a need to provide mock frames. Every read call on a socket is delayed until at least one byte is available (or socket is closed). So if the socket does not provide data fast enough, playback will be delayed automatically. No need to patch mock frames into the stream (which could get horribly complicated).
Having FileStreamRead directly read the IInputStream is not only easier, but also more performant, compared to wrapping the IInputStream in a IRandomAccessStream, then wrapping that IRandomAccessStream in a COM IStream, then using a FileStreamRead to read form that IStream.

Depending on the scenario and device, the system might enter a power saving mode or kill the app if frames are not provided in time. I had to deal with this scenario a few years back but there were tablets doing the playback on battery power.
You either need mock frames or call a method to notify progress on getting the next frame (i dont remember its name).

You are talking about ReportSampleProgress.
https://docs.microsoft.com/en-us/uwp/api/windows.media.core.mediastreamsource#Windows_Media_Core_MediaStreamSource_SampleRequested
This could be needed if you expect longer interruptions (depends on the scenario).
If you are playing video (as seems to be the case here), I think you can also use DisplayRequest to keep display (and system) active as long as your are playing. The system will not enter standby as long as a DisplayRequest is active. This should be easier than the above method. But it does not work for pure audio playback (where you'd want display to go off but still keep the system on).

Streams on UWP are a kind of pain.
I will probably implement an InputStream, but it is embarassing seeing how far it would be easier with the classical streaming model in a desktop app.

I generally like the async approach. But I agree that stream/data handling in UWP is overly complicated (and in some cases also pretty inefficient, forcing unneccessary copy operations). They could have made the API much easier and still async.
Mixing async and non-async code is always a bit tricky. But then it's also not that hard. In c++/cx you can use create_task to get a task from the ReadAsync call result, then call task->get() to wait for and get the result. This will be blocking, but it's not a problem in our context.

I love the async approach too. But the API design is overcomplicated.
The shape on .net new networking libraries is far better and not too distant from the classical approach.
I studied, used and presented in conferences the async streaming and storage APIs in winrt, but they present multiple pitfalls, like the difficulties in implementing custom versions of stream and buffer objects.

Yes, reading and writing data is very complicated. Especially the locked up IBuffer is hard to handle. It should be a one-liner to read and write binary data, from both managed and unmanaged code, as well as creating an IBuffer or InMemoryRandomAccessStream from existing data (without copy!).

Buffer imutability helps prevent some typical buffer overflow attacks.

Yeah, but it's not like they are really immutable. People who really need raw access will get it anyways. Hackers who want raw access will also get it. So I am not sure how much that really helps, given the cost of all that (in developer frustration, especially for beginners).

Buffers are never really immutable. The process memory is accessible by any code running in it or out of it, given the required privileges by the OS.
In one of my demos I show how the sandbox protects other processes from the sandboxed one. But also how easy is injecting code inside an UWP app and hijacking xaml or javascript UIs.


Fixup an issue in the input IR

... where the amount of data moved between src and dst of dma_memcpy_nd mismatches.
DDR side memref size is vector_size which is 4k. L1 side memref size is lineWidthInBytes which is 1k.

Cc: @hunhoffe

Thanks, merged as part of #701.


[HOPS-1155]  Remove Client cache invalidation

[HOPS-1155]  Remove Client cache invalidation
[HOPS-1155]  Reduce the minimum CRL fetcher service interval
Make sure there is no duplicate PR for this issue

Please check if the PR fulfills these requirements

[ ] Tests for the changes have been added and passed (for bug fixes / features)
[x] HOPS JIRA issue has been opened for this PR
[x] All commits have been squashed down to a single commit
[x] The commit message has the following format: [HOPS-XXX] message

Post a link to the associated JIRA issue
https://hopshadoop.atlassian.net/browse/HOPS-1155

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug fix

What is the new behavior (if this is a feature change)?

Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)

Other information:

Jenkins test this please


Capture Feign data using Java Aspect

Hi,
I'm tring to use Aspect to capture data from feign, i found a closed issue (https://github.com/spring-cloud/spring-cloud-openfeign/issues/59), but that solution return the below exception
@Pointcut("execution(* org.springframework.cloud.netflix.feign.ribbon.LoadBalancerFeignClient.*(..))")
public void feignClientPointcut() {}

Exception in thread "main" org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration': Initialization of bean failed; nested exception is java.lang.IllegalArgumentException: warning no match for this type name: org.springframework.cloud.netflix [Xlint:invalidAbsoluteTypeName]

After starting the spring i had the message
2019-12-02 15:23:38.484  INFO 9919 --- [           main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration' of type [org.springframework.cloud.autoconfigure.ConfigurationPropertiesRebinderAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying)
I'm using this repository to test it https://github.com/FeeFelipe/azpecLogger

Are you asking a question or reporting a bug?

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


NPM error when searching for gajus.Scream!

https://github.com/gajus/brim/blob/809d68475504aa2482fadca1d90cb7620fb11bbd/src/brim.js#L19

NPM error? Not clear what you are asking for.
Closing until further details provided.

@gajus I'm trying to use Scream + Brim with webpack into a Vue.JS system, but when I try to create an instance of Brim, using the following code:
const scream = Scream({
    width: {
        portrait: 320,
        landscape: 568
    }
});

const brim = Brim({
    viewport: scream
});

The brim instance has not been created because of this:
https://github.com/gajus/brim/blob/809d68475504aa2482fadca1d90cb7620fb11bbd/src/brim.js#L19
For some reason the Scream from NPM package is not same of gajus.Scream instance.
Any idea of how to solve it?

Have you inspected the contents of the entry point of those scripts?, i.e. look at the package.json main value and what script that executes.
I have not touched Scream/ Brim in a long time.

Oh, I see. This line of code was from Brim.
https://github.com/gajus/brim/blob/809d68475504aa2482fadca1d90cb7620fb11bbd/src/brim.js#L19-L21
Yeah, this obviously does not make sense.
Looks like the only fix would be to add scream as a dependency of Brim and check instanceof of the imported dependency.

Thanks a lot @gajus
It works ;D

Hey @gajus thanks for this update, but apparently isn't working:

I run it before: npm update:
"brim": "^1.0.12",

Can you try to look into the code?
I am out of capacity to take on this at the moment.

I am fine with the instanceof check being completely removed.

Actually, it will be my next suggestion, haha, just check for not null value and remove instance of check.


Extracting Username from Token

I have an ASP.NET Core web api and a mvc client application secured by IdentityServer4.
When an web API request is made by the client, i need to do additional filtering on the data returned based on the username of the logged in user. Hence would like to extract the username of the logged in user in the web api controlled action. Is there a way to do this?

Add that claim to the UserClaims collection on the configured ApiResource: https://identityserver4.readthedocs.io/en/release/reference/api_resource.html

Thanks Brock. That worked!!!
From: Brock Allen<EMAIL_ADDRESS>Sent: Monday, April 3, 2017 8:30 PM
To: IdentityServer/IdentityServer4<EMAIL_ADDRESS>Cc: Vicky Malhar<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [IdentityServer/IdentityServer4] Extracting Username from Token (#1017)
Add that claim to the UserClaims collection on the configured ApiResource: https://identityserver4.readthedocs.io/en/release/reference/api_resource.htmlhttps://urldefense.proofpoint.com/v2/url?u=https-3A__identityserver4.readthedocs.io_en_release_reference_api-5Fresource.html&d=DwMCaQ&c=ItdvPXPJc-zW5sqOZ-7AtXIt0TnvciGmwfBte4XFNFM&r=YjFV-Uuj5frMz78y99GWlpjNlKvwiQeqvBrs5JrsZak&m=1iGbSXCVTAFkJ7NQ8ydT2hPZF7HQ1THHkcMpP6QPRt4&s=Uty_zxKLhXGzWiLti_9a7Fr3y88iI7U4MPq0sEmVPb8&e=
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IdentityServer_IdentityServer4_issues_1017-23issuecomment-2D291342329&d=DwMCaQ&c=ItdvPXPJc-zW5sqOZ-7AtXIt0TnvciGmwfBte4XFNFM&r=YjFV-Uuj5frMz78y99GWlpjNlKvwiQeqvBrs5JrsZak&m=1iGbSXCVTAFkJ7NQ8ydT2hPZF7HQ1THHkcMpP6QPRt4&s=WgZUYMvGQIu88MZVdK501x_VzMWVK3C7NIbAxrRCKZ4&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AWGoZJJEc3nETvo2xhqc8coXSODrbIUGks5rsY8YgaJpZM4Mx6hp&d=DwMCaQ&c=ItdvPXPJc-zW5sqOZ-7AtXIt0TnvciGmwfBte4XFNFM&r=YjFV-Uuj5frMz78y99GWlpjNlKvwiQeqvBrs5JrsZak&m=1iGbSXCVTAFkJ7NQ8ydT2hPZF7HQ1THHkcMpP6QPRt4&s=3bW11_2AM2Y-pe6qXPXnstvcQxt-btvNqb4LmYnX98I&e=.

This message is intended only for the designated recipient(s). It may contain confidential, privileged or proprietary information. If you are not a designated recipient,you may not review, copy or distribute this message. If you receive this message in error, please notify the sender by reply email and delete this message.
WARNING: All email sent to or from the First Reserve corporate email system is subject to archiving, monitoring and/or review by First Reserve personnel.

Add that claim to the UserClaims collection on the configured ApiResource: https://identityserver4.readthedocs.io/en/release/reference/api_resource.html

can you please share the answer again, the page is not accessible :(

Just look for the same topic in the latest docs:

For clarity, this refers to the GetApis method within the example Config.cs. Below is an example which adds claim types to the API Resource:
    public static IEnumerable<ApiResource> GetApis()
    {
        return new ApiResource[]
        {
            new ApiResource("api1", "My API", claimTypes: new string[] { JwtClaimTypes.GivenName, JwtClaimTypes.FamilyName, JwtClaimTypes.Email, JwtClaimTypes.Role, "office" })
        };
    }


Added: Eloquent JavaScript

I would like to refer to this book for the classes

@divyanshu-rawat you may review the changes I made to this PR


Platform Build Missing Files

Hello,
I was able setup hello world on web, tizen and webos. But was getting this issue #653 .
However I found that when build in platforms, files are missing in project folder and not copying to server folder. Because when rnv run -p web shows missing files in web console like favicon, manifest. Please see folder structure.
PS
Yeshans-MacBook-Pro:ITSthe1TV yeshansachithak$ cd renative-tv-app/
Yeshans-MacBook-Pro:renative-tv-app yeshansachithak$ rnv build -p web

┌──────────────────────────────────────────────────────────────────────────────┐
│                                                                              │
│        ██████╗ ███████╗███╗   ██╗ █████╗ ████████╗██╗██╗   ██╗███████╗       │
│        ██╔══██╗██╔════╝████╗  ██║██╔══██╗╚══██╔══╝██║██║   ██║██╔════╝       │
│        ██████╔╝█████╗  ██╔██╗ ██║███████║   ██║   ██║██║   ██║█████╗         │
│        ██╔══██╗██╔══╝  ██║╚██╗██║██╔══██║   ██║   ██║╚██╗ ██╔╝██╔══╝         │
│        ██║  ██║███████╗██║ ╚████║██║  ██║   ██║   ██║ ╚████╔╝ ███████╗       │
│        ╚═╝  ╚═╝╚══════╝╚═╝  ╚═══╝╚═╝  ╚═╝   ╚═╝   ╚═╝  ╚═══╝  ╚══════╝       │
│                                                                              │
│        Version: 0.31.3                                                  │
│        https://renative.org                                             │
│        🚀 Firing up!...                                                 │
│        $ rnv build -p web                                               │
│        Start Time: 5/17/2021, 2:02:34 PM                                     │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

(node:28936) Warning: Accessing non-existent property 'cat' of module exports inside circular dependency
(Use `node --trace-warnings ...` to show where the warning was created)
(node:28936) Warning: Accessing non-existent property 'cd' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'chmod' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'cp' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'dirs' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'pushd' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'popd' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'echo' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'tempdir' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'pwd' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'exec' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'ls' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'find' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'grep' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'head' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'ln' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'mkdir' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'rm' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'mv' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'sed' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'set' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'sort' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'tail' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'test' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'to' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'toEnd' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'touch' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'uniq' of module exports inside circular dependency
(node:28936) Warning: Accessing non-existent property 'which' of module exports inside circular dependency
[ task ] registerEngines[1]
[ task ] configureRuntimeDefaults[1] appId:null
[ task ] checkAndMigrateProject[1]
[ task ] parseRenativeConfigs[1]
[ task ] getWorkspaceDirPath[1]
[ task ] loadPluginTemplates[1]
[ task ] _parsePluginTemplateDependencies[1] scope:root
[ task ] loadEngines[1]
[ task ] _generatePlatformTemplatePaths[1]
[ info ] Current Engine: engine-rn-web
[ task ] initializeTask[1] build
[ task ] => [build]
[ task ] [build] checkIfProjectAndNodeModulesExists[1]
[ hook ] [build] executePipe ('build:before')
[ task ] [build] taskRnvBuild[1] parent:null
[ task ] => [build] => [package]
[ hook ] [package] executePipe ('package:before')
[ task ] [package] taskRnvPackage[1] parent:build
[ task ] => [package] => [configure]
[ hook ] [configure] executePipe ('configure:before')
[ task ] [configure] taskRnvConfigure[1]
[ task ] => [configure] => [platform configure]
[ hook ] [platform configure] executePipe ('platform:configure:before')
[ task ] [platform configure] taskRnvPlatformConfigure[1]
[ task ] => [platform configure] => [project configure]
[ hook ] [project configure] executePipe ('project:configure:before')
[ task ] [project configure] taskRnvProjectConfigure[1]
[ task ] [project configure] checkAndMigrateProject[2]
[ task ] [project configure] parseRenativeConfigs[2]
[ task ] [project configure] getWorkspaceDirPath[2]
[ task ] [project configure] loadPluginTemplates[2]
[ task ] [project configure] _parsePluginTemplateDependencies[2] scope:root
[ task ] [project configure] loadEngines[2]
[ task ] [project configure] _generatePlatformTemplatePaths[2]
[ task ] [project configure] checkAndCreateProjectPackage[1]
[ task ] => [project configure] => [workspace configure]
[ task ] [workspace configure] taskRnvWorkspaceConfigure[1]
[ task ] <= [project configure] <= workspace configure
[ task ] [project configure] checkIfTemplateConfigured[1]
[ task ] => [project configure] => [install]
[ hook ] [install] executePipe ('install:before')
[ task ] [install] taskRnvInstall[1] requiresInstall:false:true
[ hook ] [install] executePipe ('install:after')
[ task ] <= [project configure] <= install
[ task ] [project configure] checkCrypto[1]
[ task ] [project configure] configureRuntimeDefaults[2] appId:helloworld
[ task ] [project configure] applyTemplate[1] renative-template-hello-world=>undefined
[ task ] [project configure] _applyTemplate[1] current:renative-template-hello-world selected:undefined
[ task ] [project configure] configureRuntimeDefaults[3] appId:helloworld
[ task ] => [project configure] => [install]
[ hook ] [install] executePipe ('install:before')
[ task ] [install] taskRnvInstall[2] requiresInstall:false:true
[ hook ] [install] executePipe ('install:after')
[ task ] <= [project configure] <= install
[ task ] => [project configure] => [app configure]
[ hook ] [app configure] executePipe ('app:configure:before')
[ task ] [app configure] taskRnvAppConfigure[1]
[ task ] [app configure] listAppConfigsFoldersSync[1] ignoreHiddenConfigs:true
[ task ] [app configure] parseRenativeConfigs[3]
[ task ] [app configure] generateBuildConfig[1] size:123.72 KB
[ task ] [app configure] getWorkspaceDirPath[3]
[ task ] [app configure] generateBuildConfig[2] size:123.72 KB
[ task ] [app configure] generateBuildConfig[3] size:123.72 KB
[ task ] [app configure] loadPluginTemplates[3]
[ task ] [app configure] _parsePluginTemplateDependencies[3] scope:root
[ task ] [app configure] loadEngines[3]
[ task ] [app configure] generateBuildConfig[4] size:123.72 KB
[ task ] [app configure] _generatePlatformTemplatePaths[3]
[ task ] [app configure] generateBuildConfig[5] size:134.19 KB
[ task ] [app configure] generateBuildConfig[6] size:134.19 KB
[ task ] [app configure] getWorkspaceDirPath[4]
[ task ] [app configure] generateBuildConfig[7] size:134.19 KB
[ task ] [app configure] generateLocalConfig[1] resetAppId:false
[ task ] [app configure] generateBuildConfig[8] size:134.19 KB
[ info ] [app configure] Current App Config: helloworld
[ hook ] [app configure] executePipe ('app:configure:after')
[ task ] <= [project configure] <= app configure
[ task ] [project configure] resolvePluginDependants[1]
[ task ] [project configure] configurePlugins[1]
[ task ] [project configure] versionCheck[1]
[ task ] [project configure] versionCheck:rnvRunner:0.31.3,rnvProject:0.31.3[1]
[ task ] [project configure] configureRuntimeDefaults[4] appId:helloworld
[ task ] [project configure] copyRuntimeAssets[1]
[ task ] [project configure] parseFonts[1]
[ task ] [project configure] configureTemplateFiles[1]
[ task ] [project configure] fixRenativeConfigsSync[1]
[ task ] [project configure] checkAndCreateGitignore[1]
[ task ] [project configure] configureEntryPoints[1]
[ warn ] [project configure] Extra platform ios will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform android will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform androidwear will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform androidtv will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform tvos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform macos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform windows will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform tizenmobile will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform tizenwatch will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform chromecast will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform firefoxos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform firefoxtv will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ warn ] [project configure] Extra platform kaios will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
[ task ] [project configure] generateRuntimeConfig[1]
[ task ] [project configure] overrideTemplatePlugins[1]
[ task ] [project configure] parsePlugins[1]
[ hook ] [project configure] executePipe ('project:configure:after')
[ task ] <= [platform configure] <= project configure
[ task ] [platform configure] isPlatformSupported[1]
[ task ] [platform configure] checkAndConfigureSdks[1]
[ task ] [platform configure] isBuildSchemeSupported[1]
[ info ] [platform configure] Current Build Scheme: debug
[ task ] [platform configure] checkSdk[1]
[ task ] [platform configure] _isSdkInstalled[1]
[ task ] [platform configure] configureRuntimeDefaults[5] appId:helloworld
[ task ] => [platform configure] => [install]
[ hook ] [install] executePipe ('install:before')
[ task ] [install] taskRnvInstall[3] requiresInstall:false:true
[ hook ] [install] executePipe ('install:after')
[ task ] <= [platform configure] <= install
[ task ] [platform configure] [1] taskRnvPlatformConfigure hasBuildFolderPresent:true
[ task ] [platform configure] createPlatformBuild[1]
[ task ] [platform configure] injectPlatformDependencies[1]
[ hook ] [platform configure] executePipe ('platform:configure:after')
[ task ] <= [configure] <= platform configure
[ task ] [configure] copySharedPlatforms[1]
[ task ] [configure] configureWebProject[1]
[ task ] [configure] copyAssetsFolder[1]
[ task ] [configure] configureCoreWebProject[1]
[ task ] [configure] _generateWebpackConfigs[1]
[ task ] [configure] parsePlugins[2]
[ task ] [configure] copyBuildsFolder[1]
[ task ] [configure] copyTemplatePluginsSync[1] (./platformBuilds/helloworld_web)
[ task ] [configure] parsePlugins[3]
[ hook ] [configure] executePipe ('configure:after')
[ task ] <= [package] <= configure
[ hook ] [package] executePipe ('package:after')
[ task ] <= [build] <= package
[ task ] [build] buildWeb[1]
✔ Executing: npx cross-env PLATFORM=web NODE_ENV=production  node /Users/yeshansachithak/.npm-packages/lib/node_modules/rnv/node_modules/webpack/bin/webpack.js -p --config ./platformBuilds/helloworld_web/webpack.config.prod.js
[ success ] [build] Your Build is located in ./platformBuilds/helloworld_web/project .
[ hook ] [build] executePipe ('build:after')
[ task ] <= build

 $ rnv build -p web - Done! 🚀
┌──────────────────────────────────────────────────────────────────────────────┐
│  🚀  SUMMARY | 5/17/2021, 2:03:43 PM                                         │
│  $ rnv build -p web                                                          │
├──────────────────────────────────────────────────────────────────────────────┤
│  ReNative Version: 0.31.3                                               │
│  Project Name ($package.name): renative-tv-app                          │
│  Project Version ($package.version): 0.1.0                              │
│  Workspace ($.workspaceID): rnv                                         │
│  Platform (-p): web                                                     │
│  Engine ($.platforms.web.engine): engine-rn-web                         │
│  App Config (-c): helloworld                                            │
│  Build Scheme (-s): debug                                               │
│  Supported Platforms: tizen, web, webos                            │
│  Env Info: darwin | x64 | node v14.16.1                                 │
│  Executed Time: 0h:1m:8s:376ms                                          │
│                                                                              │
│ [ warn ] [project configure] Extra platform ios will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform android will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform androidwear will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform androidtv will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform tvos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform macos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform windows will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform tizenmobile will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform tizenwatch will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform chromecast will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform firefoxos will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform firefoxtv will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ warn ] [project configure] Extra platform kaios will be ignored because it's not configured in your ./renative.json: { defaults.supportedPlatforms } object.
│ [ success ] [build] Your Build is located in ./platformBuilds/helloworld_web/project .
│                                                                              │
│  Project location:                                                           │
│  ./platformBuilds/helloworld_web/project                                │
└──────────────────────────────────────────────────────────────────────────────┘
Yeshans-MacBook-Pro:renative-tv-app yeshansachithak$

Folder structure:-

Please guide, Thanks

I was able to build the App with release schema. Which give all the files in Project Folder.
rnv build -p web -s release


Check if Mystic Light is running on launch

If Mystic Light is running and lighting is on, you'll get weird effects since both MSIRGB and Mystic Light will be accessing the hardware at the same time. This problem has been more common than I expected and therefore MSIRGB should warn the user beforehand that another application is already changing LED settings and potentially stop that application before continuing.
The simplest option, and also less likely to break, is to check if the "LEDKeeper.exe" process and ML service are running. This should cover both MSI Mystic Light and MSI Gaming App. Must check if LEDKeeper might stay alive even if no LED effects are enabled, although if it isn't, there is no harm in stopping the service anyway.

Fixed. Not auto-stopping any MSI application because there seems to be no way other than messing with the registry. Stopping the MSI service makes the MSI application no longer open, but the MSI service must be stopped before stopping LEDKeeper, and stopping MSI from changing the LED effect.


smtp errro 554

please help me fix it

duplicate of
https://github.com/3s3s/opentrade/issues/112

Set true and try
https://github.com/3s3s/opentrade/blob/master/server/constants.js#L11

Thanks i think it worked fine, now do what? i'm using digitalocean ubuntu so open in which browser localhost:40403?

Your vpsip:40443

???

show the above error while using VPN and without VPN i can't get connected

You set first email id
Example : support email<EMAIL_ADDRESS>No reply email<EMAIL_ADDRESS>Then run node main.js it's work with or without vpn

it works fine but i meant when i want to open it in browser get that error

i'm using putty to connect to digitalocean droplet, but i really don't how access opentrade in my browser,
i also receive email notifications too

Be sure, that this port is opened and you configure your nginx server.
https://medium.com/@abhaytalreja/configure-digital-ocean-for-ssl-https-meteor-app-as-a-bonus-part-4-6b5a00b581ef

How to setup SSL for opentrade domain

I've setup it using this tutorial, you need to paste External IP instead of domain, if you haven't
https://medium.com/@abhaytalreja/configure-digital-ocean-for-ssl-https-meteor-app-as-a-bonus-part-4-6b5a00b581ef

@CryptorClub  can you give me your email?
i can't fix the problem because of lack of knowledge, i give you my ubunta ip address and login details please fix my problem i really appreciate that
Thanks

<EMAIL_ADDRESS>

@aminaqi just run this command
curl ipinfo.io/ip
You will get your servers IP address
Then type in browser this ip with port, like you do before. You need, to understand wow to setup it, if you want to manage it

@aminaqi
I sent you email, I fixed it
Guys for SSL you just install nginx and install letsencrypt for nginx, you will have certificates and everything.there are many tutorials on digitalocean for ubuntu

@coinnetworkpw dont forget to set a cron job to automatically redo the certificate every 90 days ;)

I get error after setup unexpected json

@jonn4y I know :)
This is not our first time doing it.
We are setting up exchanges for our clients from peatio and this script since last year and always using nginx.
@parwezmansuri do you have your coin deamons running ?

Last time I check exchange it's work other coin deamons but at this time my vps stuck so still I am waiting for resolve my vps issue. Then after I setup exchange.


v3.3.5

https://argoproj.github.io/argo-workflows/releasing/
https://github.com/argoproj/argo-workflows/security/advisories/GHSA-cmv8-6362-r5w9

87470e1c2 fix: Added artifact Content-Security-Policy (#8585)
dc8fef3e5 fix: Support memoization on plugin node. Fixes #8553 (#8554)
c3ae56565 fix: Do not log container not found (#8509)
c61770622 fix: Pod OOMKilled should fail workflow. Fixes #8456 (#8478)
a648ccdcf fix: prevent backoff when retryStrategy.limit has been reached. Fixes #7588 (#8090)
aa9ff17d5 fix: Remove path traversal CWE-23 (#8331)
0cdd2b40a fix: update docker version to address CVE-2022-24921 (#8312)
9c901456a fix: Default value is ignored when loading params from configmap. Fixes #8262 (#8271)
954a3ee7e fix: install.yaml missing crb subject ns (#8280)
e232340cc fix: grep pattern (#8238)
12cd8bcaa fix: requeue not delete the considererd Task flag (#8194)

CI running.

CI passed.


Bump framework version

Please refer the $subject

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3581645090

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3581645090
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/3581645090

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/3581645090
Status: success


Support WebSocket

https://en.wikipedia.org/wiki/WebSocket

great!

Gently ask any progress here? We‘re planing to start a websocket based project using the wonderful armeria server...

Hi! There's no progress yet. I think it will take place after 1.0. We always look for volunteers, so please let us know if you have an idea. :bowing_man:

I also need to add documentation and support WebSocket extensions as follow-ups.


Make it easier to clear cabal caches on more buildkite agents

[x] Add "hack" for clearing buildkite cabal caches on more than just 1 buildkite agent

Comments

I went with 5 steps because the buildkite agents seem to be split over 5 machines

not sure whether there are e.g. several containers with separate caches on each machine
If needed we can increase the number of steps in the future.

Motivation
Need was discovered when attempting to merge #3522, whose cabal caches where somehow incompatible with those of master. Buildkite steps would not succeed on the PR without clearing the caches, and if a step completed after doing so (but not merge to master), it would break CI in other PRs.
Having the opportunity to clear the caches on "hopefully all" agents  should make dealing with cache-breaking PRs easier in the future.

bors r+


Seus renewed causing textures to be slightly see through

I know modders don't usually bother with shaders as optifine is a finicky thing and so many shaders exist
but this is something I haven't seen before in a mod

I guess the reason for that is, that all frame and illusion blocks are registered as translucent blocks, because you might want to mimic glass or slime blocks (which are translucent blocks). At least that's what I assume. I will test it with different render types (cutout, cutout_mipped), but right now I don't have much time (studies); and I did not encounter this problem before as I usually play Minecraft without shaders

Registering them as translucent blocks leads to draw order weirdness even without shaders, Optifine or any of that sort of funny business. For example, opaque framed blocks are drawn over effects such as Astral Sorcery's light beams when behind them.

Seus based shaders seem to have trouble whit this (and other mods) anyways.
the ptgi shader for example turns those blocks to water


Make SQL strings generated from Exprs "prettier"

Is your feature request related to a problem or challenge?
Part of https://github.com/apache/datafusion/issues/9494
As @backkem  says https://github.com/apache/datafusion/pull/10528#issuecomment-2116068547 on https://github.com/apache/datafusion/pull/10528
Currently, expressions from the DataFusion SQL unparser (aka expr --> String) are somewhat ugly
For example the expression col("a").eq(lit(5)) would be rendered as a = 5 by most poeple if they did it manaully, but DataFusion's unparser currently renders it like "a" = 5 (with extra quotes).
DataFusion also puts in quotes to make the order of operations explicit -- so instead of a < 5 AND b < 10 it would render ("a" < 5) AND ("b" < 10)
The current unparser is conservative and likely works well for when generating SQL for consumptions by other database systems. However, the SQL is not as nice for human consumption
Here is another instance from the example
https://github.com/apache/datafusion/blob/98647e842a85b768ea0cb0f8ccf1016636001abb/datafusion-examples/examples/plan_to_sql.rs#L50-L53
Describe the solution you'd like
If we want to make the generated SQL easier to read by humans / more succint, these steps will have to be made "smarter".
Describe alternatives you've considered
Potential ideas:

We'll have to add in the math rules to avoid unneeded parentheses
(likely dialect specific) rules for determining of quoting is needed.

Note that the latter likely involves listing out the reserved keywords for each dialect.
Additional context
No response

Provide something I surveyed.
I think we can follow how Calcite handles the quoted issue. The SqlDialect of Calcite has a check rule identifierNeedsQuote.
https://github.com/apache/calcite/blob/aba64f0b217093b500629fe07a0befdc68293fbc/core/src/main/java/org/apache/calcite/sql/SqlDialect.java#L413-L415
It can be overridden according to the specific data source, such as BigQuery:
https://github.com/apache/calcite/blob/50a20824c4536450dcae963b5e757cf4bbc7e406/core/src/main/java/org/apache/calcite/sql/dialect/BigQuerySqlDialect.java#L106-L109
They implement some rules, such as regex patterns or reserved word lists. I think a dialect-specific rule is a nice choice.

Indeed, there is already a function on the sqlparser::dialect trait that takes this into account:
https://github.com/sqlparser-rs/sqlparser-rs/blob/54184460b5d873a67c2801e8b7c6e4f145bc65df/src/dialect/mod.rs#L113-L116
The dialect specific implementations just need to be expanded on. For now they just always return a conservative quote character.

https://github.com/sqlparser-rs/sqlparser-rs/blob/54184460b5d873a67c2801e8b7c6e4f145bc65df/src/dialect/mod.rs#L113-L116
The dialect specific implementations just need to be expanded on. For now they just always return a conservative quote character.

I think it's same as the dialect in datafusion::unparser::dialect
https://github.com/apache/datafusion/blob/e7858ff0ab1c282ab46bd93cabc3dc83db583165/datafusion/sql/src/unparser/dialect.rs#L28
However, what we need is a checker to check if the identifier needs to be quoted.
I think I can make a PR for DefaultDialect first.

As the mentioned in dialect.rs
https://github.com/apache/datafusion/blob/e7858ff0ab1c282ab46bd93cabc3dc83db583165/datafusion/sql/src/unparser/dialect.rs#L19
I think we need to use the Dialect in sqlparser-rs instead and extract identifier_needs_quote in #10573 to sqlparser-rs. Just like https://github.com/sqlparser-rs/sqlparser-rs/pull/1170

Yes, these are basically the same object. The one in DataFusion was put there temporarily until the trait extension in the sqlparser repo is landed and pushed to crates.io.

https://github.com/apache/datafusion/pull/10392 is the upgrade to sqlparser -- I think it is pretty close but @tisonkun  hit an issue during upgrade.

#10392 is the upgrade to sqlparser -- I think it is pretty close but @tisonkun hit an issue during upgrade.

We may need a 0.46.1 for resolving the regressions:

https://github.com/sqlparser-rs/sqlparser-rs/pull/1278
And what's still unclear how to fix, https://github.com/apache/datafusion/pull/10392#discussion_r1605947314

I'm not sure but I think we can merge #10573 first because it also fix many unpasring tests. Then, I'll create PR for sqlparser to add the check rule in dialect.

FWI https://github.com/apache/datafusion/pull/10573 is merged!

Do we split off a ticket reduce the nr of brackets emitted?

Do we split off a ticket reduce the nr of brackets emitted?

Excellent call -- I filed https://github.com/apache/datafusion/issues/10633


How to skip individual processing tests (by default)

the individual process tests are quite heavy and large in numbe
It's probably best to skip them by default and make it opt-in in some way

playing with the idea to make a ProcessTestRunner subclass that skips everything


Added a Floyd's Algorithm

Please review my code and merge it under hactoberfest.

Please mark it under hactoberfest accepted .


⚠ Mobitopup has degraded performance

In e66cfb7, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 3094 ms

Resolved: Mobitopup performance has improved in 0dd27bf.


ue 5.2

Plugin did not work with ue 5.2
Just testet to build android and get many errors.

so ts sdk for unreal is dead right?


[BUG] AttributeError: 'NoneType' object has no attribute 'startswith'

Description
Error
File [~/.pyenv/versions/3.11.4/envs/crewai/lib/python3.11/site-packages/tiktoken/model.py:86](http://<IP_ADDRESS>:8889/~/.pyenv/versions/3.11.4/envs/crewai/lib/python3.11/site-packages/tiktoken/model.py#line=85), in encoding_name_for_model(model_name)
     81 else:
     82     # Check if the model matches a known prefix
     83     # Prefix matching avoids needing library updates for every model version release
     84     # Note that this can match on non-existent models (e.g., gpt-3.5-turbo-FAKE)
     85     for model_prefix, model_encoding_name in MODEL_PREFIX_TO_ENCODING.items():
---> 86         if model_name.startswith(model_prefix):
     87             return model_encoding_name
     89 if encoding_name is None:

AttributeError: 'NoneType' object has no attribute 'startswith'

Code
from dotenv import load_dotenv
from crewai import Agent
from langchain_openai import AzureChatOpenAI

load_dotenv()

azure_llm = AzureChatOpenAI(
    azure_endpoint=os.environ.get("AZURE_OPENAI_ENDPOINT"),
    api_key=os.environ.get("AZURE_OPENAI_KEY"),
    openai_api_version=os.environ.get("AZURE_OPENAI_VERSION"),
    azure_deployment=os.environ.get("AZURE_OPENAI_DEPLOYMENT")
)

azure_agent = Agent(
  role='Example Agent',
  goal='Demonstrate custom LLM configuration',
  backstory='A diligent explorer of GitHub docs.',
  llm=azure_llm,
  verbose=True,
)

Steps to Reproduce
None
Expected behavior
None
Screenshots/Code snippets
None
Operating System
macOS Sonoma
Python Version
3.11
crewAI Version
0.51.1
crewAI Tools Version
0.8.3
Virtual Environment
Venv
Evidence

Possible Solution
None
Additional context
None

thanks to @farzad528 on this post: https://github.com/crewAIInc/crewAI/issues/1235
Documents and code of examples need to be updated.
mode should be used to define AzureChatOpenAI, not azure_deployment.
azure_llm = AzureChatOpenAI(
    azure_endpoint=os.environ.get("AZURE_OPENAI_ENDPOINT"),
    api_key=os.environ.get("AZURE_OPENAI_KEY"),
    openai_api_version=os.environ.get("AZURE_OPENAI_VERSION"),
    model=os.environ.get("AZURE_OPENAI_DEPLOYMENT")
)

@martin-wan01 I was also seeing that issue but am really struggling to connect to the model on azure ai. I've been able to connect using their example using from azure.ai.inference import ChatCompletionsClient but keep getting the error Task output: Agent stopped due to iteration limit or time limit. when using it as an LLM through CrewAi.
I was wondering if you've maybe got a fix for this?
For the model I have tried the variants below with the same result:

llama3 (for my llama3.1 model)
llama3.1 (for my llama3.1 model)
<NAME_OF_LLAMA_3.1_MODEL> (for my phi3.5 custom model)

See below for my code example:

from langchain_openai import AzureChatOpenAI
from crewai import Agent, Task, Crew
import os

azure_llm = AzureChatOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    azure_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version=os.environ["OPENAI_API_VERSION"],
    model="llama3.1"
)

general_agent = Agent(role = "Math Professor",
                      goal = """Provide the solution to the students that are asking mathematical questions and give them the answer.""",
                      backstory = """You are an excellent math professor that likes to solve math questions in a way that everyone can understand your solution""",
                      allow_delegation = False,
                      verbose = True,
                      llm = azure_llm)

task = Task(description="""what is 3 + 5""",
             agent = general_agent,
             expected_output="A numerical answer.")

crew = Crew(
            agents=[general_agent],
            tasks=[task],
            verbose=True
        )

result = crew.kickoff()

print(result)

Looks like the issue I was having was that I am using the AzureML through the new Azure AI Studio and trying to connect to a model there. I got it working using the AzureMLChatOnlineEndpoint from langchain. See example in link below:
https://python.langchain.com/v0.1/docs/integrations/chat/azureml_chat_endpoint/

I modified the code and it worked for me:
I added the model_name = "gpt-4", parameter for at the end see code below:
llm_azure = AzureChatOpenAI(
    api_version="",
    azure_deployment="",
    azure_endpoint=os.getenv(""),
    api_key=os.getenv(""),
    temperature=0,
    max_tokens=None,
    timeout=None,
    max_retries=2,
    streaming=True,
    model_name = "gpt-4"
)

Please open again - https://python.langchain.com/docs/integrations/chat/azure_chat_openai/ is still wrong and causes this issue.


Updated to typeahead v0.11.1

Added a couple extra files from the twitter-typeahead distribution:

bloodhound.min.js
typeahead.jquery.min.js

Can you coordinate w/ PR #17

@yourabi I just pushed a commit to update README.md to refer to the new version.  I'd say this #16 and #17 are identical.  Go ahead and choose one and merge it so I can stop pointing to a fork in my app :-)
Do let me know if you'd like me to change anything in the PR.
Thanks!

@yourabi is there anything you need changed in this PR to merge it?  Alternatively you can merge in #17 and close this one.  Either one will allow me to point my codebase to your repo for the typeahead gem instead of my fork of it :-)  Do let me know!
Thanks!


Fixed Ctrl+Alt shortcuts conflicting with AltGr

Summary of the Pull Request
This moves the detection of AltGr keypresses in front of the shortcut handling.
This allows one to have Ctrl+Alt shortcuts, while simultaneously being able to use the AltGr key for special characters.
References
This fixes a regression in v0.3.2142.0, which automatically creates Ctrl+Alt+[0-9] shortcuts for switching between tabs. These conflicted with the detection of the AltGr key. in SendKeyEvent().
Refixes #521.

PR Checklist

[ ] Closes #521
[x] CLA signed. If not, go over here and sign the CLA
[ ] Tests added/passed
[ ] Requires documentation to be updated
[ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

Detailed Description of the Pull Request / Additional comments
This PR has the flaw that it doesn't allow one to define AltGr shortcuts anymore.
Considering that it's probably more important to get a fix out of the door quickly I personally see this as a minor flaw though. VS Code has the same "flaw" for instance and doesn't allow AltGr combinations either.
We could fix it more correctly later on, by modifying the IKeyBindings interface and KeyChord class.
Validation Steps Performed

Enter AltGr-characters on my german keyboard and ensuring they appear in my shell.
Ensure that Ctrl+Alt+[0-9] shortcuts simultaneously work.

I've based this PR on a08666b btw, which is the common ancestor between master and release-0.3.
It should be possible to merge this PR into both branches without rebasing or cherry picking, etc.

@DHowett-MSFT No problem. 😅 I guess I'm just
I've amended this PR with comments for the two new functions. I've also tried to give them slightly more obvious names (although I failed at it I guess lol).
I'm not entirely sure how to beautify the code any further.
There's probably a reason for it, but IMHO the architecture between TerminalControl::TermControl, Core::Terminal and Console::VirtualTerminal feels kinda... redundant, as all 3 of them fiddle around with vkeys and modifiers. I believe merging these classes and resplitting the responsibilities might yield subjectively "better" code in the long run.

E.g. my thoughts regarding character handling...
You could for instance abstract AltGr, shortcut, etc. handling into one class each and concate them together to a stream of key/character event processors which implement the full, final keyboard handling.
TermControl would also implement the common interface and be the final (self-referential) receiver in the chain.
That way you do away with the current, loopy design, of TermControl handling a bit of the keys, pushing it down to Terminal, which implements a bit of the handling, which then pushes it down to VirtualTerminal which implements a bit of the handling and finally calls TermControl again through a std::function callback, which then stringifies the character/escape-sequence array and then calls TermControl again through a std::function callback which then sends it to the shell.
It's very "loopy". 😄

Alright... If you need any more of my AltGr expertisms: I've applied to the open position as well now. 😄😄

@DHowett-MSFT do the thing


show first two letters of days

i want to show the first two letters for days that start with the same letter ( T for tuesday and TH for Thursday , Su for sunday and S for saturday) how can I do this in the xml?

You can extend WeekdaysPicker and then overwrite getDayLetter:
https://github.com/DavidProdinger/Weekdays-Selector/blob/a01735ef1c8b6cb957545bdf59753c7a1fddccd9/weekdaysselector/src/main/java/com/dpro/widgets/WeekdaysPicker.java#L276-L279

Hi, I did what you suggested but the OVERRIDE annotation doesnt work with this function. I can override basically anything else but this.

You can extend WeekdaysPicker and then overwrite getDayLetter:
https://github.com/DavidProdinger/Weekdays-Selector/blob/a01735ef1c8b6cb957545bdf59753c7a1fddccd9/weekdaysselector/src/main/java/com/dpro/widgets/WeekdaysPicker.java#L276-L279

This is a private method, you won't be able to override this.
@taouichaimaa There is currently no functionality to get a day as two letters.

Ups sorry looked to fast ...


License?

Hi @CraigRonald555,
Thank you so much for developing this plugin! To ensure that everyone can use and contribute to it legally and safely, could you please add the licensing information? This would help clarify usage rights for all users.

Thank you!

My bad - this plugin is free to use for personal and commercial projects. I will confirm this with an appropriate license update before tomorrow.
Thank you for bringing this to my attention.

That's great to hear, and thank you for the quick update! If you're considering which license to use, the MIT license is a popular choice that could suit your needs. It's simple, permissive, and widely used in many open-source projects.
Here's a handy link to a GitHub tutorial that can guide you through adding a license to your repository: Adding a License to a Repository.
Thanks again for your responsiveness and for sharing your work with the community!

@jdachtera  The license should be added now, thank you for providing the link. Take care :)


Bars aren't to scale

The bars aren't to scale. This makes it hard to build an intuition about how much bigger one budget is over another.

This image implies that 50% is approximately 15%.

Potential further reading: https://infovis-wiki.net/wiki/Lie_Factor

Yeah, this is something we have to solve- thank you for raising the issue. The bars are using the same scale, but there is an extra padding on the bottom of each bar so that there is something to grab when the bar is at 0%.
We could adjust the script so that a drag event on the light grey background bar would increase the bar height. We could also add, as @austindavis suggested, a tap event handler, where the user could tap the bar and type in their desired percentage.


Sink does not work with Microsoft.Azure.DocumentDB 1.15.0

There has apparently been a change within Microsoft.Azure.DocumentDB 1.15.0 that doesn't work with the sink.  The following error is generated when using it with the latest version of the Azure DocumentDB Sink.
System.MissingMethodException: Method not found: 'Void Microsoft.Azure.Documents.Client.DocumentClient..ctor(System.Uri, System.String, Microsoft.Azure.Documents.Client.ConnectionPolicy, System.Nullable1)'.    at Serilog.Sinks.AzureDocumentDb.AzureDocumentDBSink..ctor(Uri endpointUri, String authorizationKey, String databaseName, String collectionName, IFormatProvider formatProvider, Boolean storeTimestampInUtc, Protocol connectionProtocol, Nullable1 timeToLive)
at Serilog.LoggerConfigurationAzureDocumentDBExtensions.AzureDocumentDB(LoggerSinkConfiguration loggerConfiguration, Uri endpointUri, String authorizationKey, String databaseName, String collectionName, LogEventLevel restrictedToMinimumLevel, IFormatProvider formatProvider, Boolean storeTimestampInUtc, Protocol connectionProtocol, Nullable`1 timeToLive)
Looking at the code for Microsoft.Azure.DocumentDB.Client.DocumentClient, a new parameter was added to the constructor and set to default as NULL but just updating the NuGet Package causes it to break.

@acgritt, thanks for reporting this issue. I'll investigate it over weekend.

@acgritt, Please try latest pre-release version (3.6.8-dev-00148) and let me know if it works or not.
Install-Package Serilog.Sinks.AzureDocumentDb -Pre

@saleem-mirza - any idea when you can push this fix as a released version ??
Thanks,

@saleem-mirza - We are starting a new aspnet core project very soon and would love to use this sink for logging.

@aagarwal10 I'm traveling and hopping to be back in my desk by 20th. I'll release it soon.

@saleem-mirza - Thanks, Appreciate it.


chore: Use version 0.0.7 of signed provisioning scripts

chore: Use version 0.0.7 of signed provisioning scripts
Remove outdated 0.0.2

source of this change:  https://github.com/Azure/aks-engine/pull/4134

feat: Updating kubelet/kube-proxy to run with as high priority processes (#4073)
feat: Add WinDSR support (#4104)

@mainred v0.0.7 does not work with WinDSR so I closed this PR. Will open a new PR with new signed provisioning scripts.


missing filed(s) in frontend function / Placeholders Form data fields selector

I have two (or more) Relations with an attribute type "N:1" related to another Relation. These attributes have the same name. I have also a Form related to this another Relation. The form contains all "1:N" fields. The form has assigned a Frontend function. I can see only one "N:1" filed in the function placeholders form data fileds selector . If I rename the "N:1" attributes in Relations to different names, then all fileds will be in the fileds selector.
Tested in v3.1.3, v3.2.5 and v3.3.

Thank you very much for the report. Fix will follow soon.


next-auth login() redirection doesn't work

Use case:
I used the repo below and added next-auth using Cognito provider:
https://github.com/plevavas/monorepo
https://next-auth.js.org/getting-started/example
https://next-auth.js.org/providers/cognito
The redirection to Cognito login does work locally when debugging in vs code
Issue:
The redirection to Cognito login does NOT work when deployed with<EMAIL_ADDRESS>

Hey @adamsh25 !
Any chance you might have a logs on your hand? Also, if you say that redirection does not work - do you get some Internal Error? Does not not function at all? Do you get stuck on API route?

Hey @sladg I will need to redeploy this test, I will be able to do so over the weekend.

I did replicate the behaviour, tested on Cognito and latest Next-auth. It indeed does not redirect properly.
From what I was able to gather around Github, it seems like cookies/headers are not passed properly through Cloudfront, I tried out tweaking Origin Request and Cache policies, but no luck. I wasn't able to find what headers/cookies are used/necessary, it will probably require digging into source code of the next-auth.
Let me know if you find some more info!

Might be related to: https://github.com/nextauthjs/next-auth/pull/5561

@adamsh25 was there any luck resolving this with newest version of Next-auth?


Did the mesh init() signature from the scene example change?

I tried running from https://xxyxyz.org/rendering-tutorial/  and got this:
Traceback (most recent call last):
  File "…scene_try.py", line 13, in <module>
    s.add(mesh(
TypeError: __init__() takes 2 positional arguments but 5 were given

So I added some parentheses to make the mesh faces a single tuple and now I'm waiting for the render :)
from flat import diffuse, mesh, scene

gray = diffuse((0.7, 0.7, 0.7))
red = diffuse((0.7, 0.2, 0.2))
green = diffuse((0.2, 0.7, 0.2))
lamp = diffuse((0.7, 0.7, 0.7), (1000.0, 1000.0, 1000.0))

s = scene()
# Scene by Harrison Ainsworth / HXA7241
# http://www.hxa.name/minilight/
s.camera((0.278, 0.275, -0.789), (0.278, 0.275, 0), 50)
s.environment((0.0906, 0.0943, 0.1151), (0.1, 0.09, 0.07))
s.add(mesh((
    ((0.556, 0.000, 0.000), (0.006, 0.000, 0.559), (0.556, 0.000, 0.559)),
    ((0.006, 0.000, 0.559), (0.556, 0.000, 0.000), (0.003, 0.000, 0.000)),
    ((0.556, 0.000, 0.559), (0.000, 0.549, 0.559), (0.556, 0.549, 0.559)),
    ((0.000, 0.549, 0.559), (0.556, 0.000, 0.559), (0.006, 0.000, 0.559)))), gray)
s.add(mesh((
    ((0.006, 0.000, 0.559), (0.000, 0.549, 0.000), (0.000, 0.549, 0.559)),
    ((0.000, 0.549, 0.000), (0.006, 0.000, 0.559), (0.003, 0.000, 0.000)))), red)
s.add(mesh((
    ((0.556, 0.000, 0.000), (0.556, 0.549, 0.559), (0.556, 0.549, 0.000)),
    ((0.556, 0.549, 0.559), (0.556, 0.000, 0.000), (0.556, 0.000, 0.559)))), green)
s.add(mesh((
    ((0.556, 0.549, 0.559), (0.000, 0.549, 0.000), (0.556, 0.549, 0.000)),
    ((0.000, 0.549, 0.000), (0.556, 0.549, 0.559), (0.000, 0.549, 0.559)))), gray)
s.add(mesh((
    ((0.343, 0.545, 0.332), (0.213, 0.545, 0.227), (0.343, 0.545, 0.227)),
    ((0.213, 0.545, 0.227), (0.343, 0.545, 0.332), (0.213, 0.545, 0.332)))), lamp)
s.add(mesh((
    ((0.474, 0.165, 0.225), (0.426, 0.165, 0.065), (0.316, 0.165, 0.272)),
    ((0.266, 0.165, 0.114), (0.316, 0.165, 0.272), (0.426, 0.165, 0.065)),
    ((0.266, 0.000, 0.114), (0.266, 0.165, 0.114), (0.316, 0.165, 0.272)),
    ((0.316, 0.000, 0.272), (0.266, 0.000, 0.114), (0.316, 0.165, 0.272)),
    ((0.316, 0.000, 0.272), (0.316, 0.165, 0.272), (0.474, 0.165, 0.225)),
    ((0.474, 0.165, 0.225), (0.316, 0.000, 0.272), (0.474, 0.000, 0.225)),
    ((0.474, 0.000, 0.225), (0.474, 0.165, 0.225), (0.426, 0.165, 0.065)),
    ((0.426, 0.165, 0.065), (0.426, 0.000, 0.065), (0.474, 0.000, 0.225)),
    ((0.426, 0.000, 0.065), (0.426, 0.165, 0.065), (0.266, 0.165, 0.114)),
    ((0.266, 0.165, 0.114), (0.266, 0.000, 0.114), (0.426, 0.000, 0.065)))), gray)
s.add(mesh((
    ((0.133, 0.330, 0.247), (0.291, 0.330, 0.296), (0.242, 0.330, 0.456)),
    ((0.242, 0.330, 0.456), (0.084, 0.330, 0.406), (0.133, 0.330, 0.247)),
    ((0.133, 0.000, 0.247), (0.133, 0.330, 0.247), (0.084, 0.330, 0.406)),
    ((0.084, 0.330, 0.406), (0.084, 0.000, 0.406), (0.133, 0.000, 0.247)),
    ((0.084, 0.000, 0.406), (0.084, 0.330, 0.406), (0.242, 0.330, 0.456)),
    ((0.242, 0.330, 0.456), (0.242, 0.000, 0.456), (0.084, 0.000, 0.406)),
    ((0.242, 0.000, 0.456), (0.242, 0.330, 0.456), (0.291, 0.330, 0.296)),
    ((0.291, 0.330, 0.296), (0.291, 0.000, 0.296), (0.242, 0.000, 0.456)),
    ((0.291, 0.000, 0.296), (0.291, 0.330, 0.296), (0.133, 0.330, 0.247)),
    ((0.133, 0.330, 0.247), (0.133, 0.000, 0.247), (0.291, 0.000, 0.296)))), gray)

s.render(200, 200, 10).tonemapped().png('cornellbox.png')

Yes, it did, it's mentioned in 0.2 release notes (see "depreciation of *sequence"). The rendering example was written for 0.1 so I kept it as it was.

I understand you might want to keep the example as is for historical tracking. But please consider your users trying to learn the library have very few examples to consult. Maybe you could add a note next to the example? Maybe a link to a repo with up to date examples as mentioned on issue https://github.com/xxyxyz/flat/issues/12 ...


CMake Generate step failed.  Build files cannot be regenerated correctly.

(tensorml) user@10324cdf-227b-465d-98af-cabfa0a5996c:~/TensorRT-LLM$ python3 ./scripts/build_wheel.py --trt_root /home/user/TensorRT-LLM/TensorRT-<IP_ADDRESS>
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: build in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 1)) (1.0.3)
Requirement already satisfied: torch in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 2)) (2.1.0)
Requirement already satisfied: transformers==4.31.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 3)) (4.31.0)
Requirement already satisfied: diffusers==0.15.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 4)) (0.15.0)
Requirement already satisfied: accelerate==0.20.3 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 5)) (0.20.3)
Requirement already satisfied: colored in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 6)) (2.2.3)
Requirement already satisfied: polygraphy in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 7)) (0.49.0)
Requirement already satisfied: onnx>=1.12.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 8)) (1.15.0)
Requirement already satisfied: mpi4py in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 9)) (3.1.5)
Requirement already satisfied: tensorrt>=8.6.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 10)) (9.1.0.post12.dev4)
Requirement already satisfied: numpy in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 11)) (1.26.0)
Requirement already satisfied: cuda-python==12.2.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 12)) (12.2.0)
Requirement already satisfied: sentencepiece>=0.1.99 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 13)) (0.1.99)
Requirement already satisfied: wheel in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 14)) (0.41.3)
Requirement already satisfied: lark in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from -r requirements.txt (line 15)) (1.1.8)
Requirement already satisfied: filelock in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (3.13.1)
Requirement already satisfied: huggingface-hub<1.0,>=0.14.1 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (0.19.4)
Requirement already satisfied: packaging>=20.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (23.2)
Requirement already satisfied: pyyaml>=5.1 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (2023.10.3)
Requirement already satisfied: requests in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (2.31.0)
Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (0.13.3)
Requirement already satisfied: safetensors>=0.3.1 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (0.4.0)
Requirement already satisfied: tqdm>=4.27 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from transformers==4.31.0->-r requirements.txt (line 3)) (4.66.1)
Requirement already satisfied: Pillow in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from diffusers==0.15.0->-r requirements.txt (line 4)) (9.4.0)
Requirement already satisfied: importlib-metadata in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from diffusers==0.15.0->-r requirements.txt (line 4)) (6.8.0)
Requirement already satisfied: psutil in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from accelerate==0.20.3->-r requirements.txt (line 5)) (5.9.5)
Requirement already satisfied: cython in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from cuda-python==12.2.0->-r requirements.txt (line 12)) (3.0.5)
Requirement already satisfied: pyproject_hooks in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from build->-r requirements.txt (line 1)) (1.0.0)
Requirement already satisfied: typing-extensions in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from torch->-r requirements.txt (line 2)) (4.8.0)
Requirement already satisfied: sympy in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from torch->-r requirements.txt (line 2)) (1.12)
Requirement already satisfied: networkx in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from torch->-r requirements.txt (line 2)) (3.2.1)
Requirement already satisfied: jinja2 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from torch->-r requirements.txt (line 2)) (3.1.2)
Requirement already satisfied: fsspec in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from torch->-r requirements.txt (line 2)) (2023.10.0)
Requirement already satisfied: protobuf>=3.20.2 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from onnx>=1.12.0->-r requirements.txt (line 8)) (4.25.1)
Requirement already satisfied: tensorrt-libs==9.1.0.post12.dev4 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from tensorrt>=8.6.0->-r requirements.txt (line 10)) (9.1.0.post12.dev4)
Requirement already satisfied: tensorrt-bindings==9.1.0.post12.dev4 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from tensorrt>=8.6.0->-r requirements.txt (line 10)) (9.1.0.post12.dev4)
Requirement already satisfied: nvidia-cuda-runtime-cu12 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from tensorrt-libs==9.1.0.post12.dev4->tensorrt>=8.6.0->-r requirements.txt (line 10)) (12.3.101)
Requirement already satisfied: nvidia-cudnn-cu12 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from tensorrt-libs==9.1.0.post12.dev4->tensorrt>=8.6.0->-r requirements.txt (line 10)) (<IP_ADDRESS>)
Requirement already satisfied: nvidia-cublas-cu12 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from tensorrt-libs==9.1.0.post12.dev4->tensorrt>=8.6.0->-r requirements.txt (line 10)) (<IP_ADDRESS>)
Requirement already satisfied: zipp>=0.5 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from importlib-metadata->diffusers==0.15.0->-r requirements.txt (line 4)) (3.17.0)
Requirement already satisfied: MarkupSafe>=2.0 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from jinja2->torch->-r requirements.txt (line 2)) (2.1.3)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 3)) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 3)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 3)) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from requests->transformers==4.31.0->-r requirements.txt (line 3)) (2023.7.22)
Requirement already satisfied: mpmath>=0.19 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from sympy->torch->-r requirements.txt (line 2)) (1.3.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12 in /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages (from nvidia-cudnn-cu12->tensorrt-libs==9.1.0.post12.dev4->tensorrt>=8.6.0->-r requirements.txt (line 10)) (12.3.103)
-- The CXX compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- NVTX is disabled
-- Importing batch manager
-- Building PyTorch
-- Building Google tests
-- Building benchmarks
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda-12.2/bin/nvcc
-- CUDA compiler: /usr/local/cuda-12.2/bin/nvcc
-- GPU architectures: 70-real;80-real;86-real;89-real;90-real
-- The CUDA compiler identification is NVIDIA 12.2.140
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-12.2/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /usr/local/cuda-12.2/include (found version "12.2.140")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- ========================= Importing and creating target nvinfer ==========================
-- Looking for library nvinfer
-- Library that was found /home/user/TensorRT-LLM/TensorRT-<IP_ADDRESS>/targets/x86_64-linux-gnu/lib/libnvinfer.so
-- ==========================================================================================
-- ========================= Importing and creating target nvuffparser ==========================
-- Looking for library nvparsers
-- Library that was found nvparsers_LIB_PATH-NOTFOUND
-- ==========================================================================================
-- CUDAToolkit_VERSION 12.2 is greater or equal than 11.0, enable -DENABLE_BF16 flag
-- CUDAToolkit_VERSION 12.2 is greater or equal than 11.8, enable -DENABLE_FP8 flag
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- COMMON_HEADER_DIRS: /home/user/TensorRT-LLM/cpp;/usr/local/cuda-12.2/include
-- TORCH_CUDA_ARCH_LIST: 7.0;8.0;8.6;8.9;9.0
-- Found Python3: /home/user/mambaforge/envs/tensorml/bin/python3.11 (found version "3.11.6") found components: Interpreter Development Development.Module Development.Embed
-- Found Python executable at /home/user/mambaforge/envs/tensorml/bin/python3.11
-- Found Python libraries at /home/user/mambaforge/envs/tensorml/lib
-- Found CUDA: /usr/local/cuda-12.2 (found version "12.2")
-- Found CUDAToolkit: /usr/local/cuda-12.2/include (found version "12.2.140")
-- Caffe2: CUDA detected: 12.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda-12.2/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-12.2
-- Caffe2: Header version is: 12.2
-- /usr/local/cuda-12.2/targets/x86_64-linux/lib/libnvrtc.so shorthash is 000ca627
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- Added CUDA NVCC flags for: -gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_90,code=sm_90
-- MKL_ARCH: None, set to ` intel64` by default
-- MKL_ROOT /home/user/mambaforge/envs/tensorml
-- MKL_LINK: None, set to ` dynamic` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
-- MKL_THREADING: None, set to ` intel_thread` by default
-- MKL_MPI: None, set to ` intelmpi` by default
CMake Warning at /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
  CMakeLists.txt:281 (find_package)

-- Found Torch: /home/user/mambaforge/envs/tensorml/lib/python3.11/site-packages/torch/lib/libtorch.so
-- TORCH_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0
-- Building for TensorRT version: 9.1.0, library version: 9
-- Using MPI_CXX_INCLUDE_DIRS: /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi;/usr/lib/x86_64-linux-gnu/openmpi/include
-- Using MPI_CXX_LIBRARIES: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so;/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- USE_CXX11_ABI: False
-- The C compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found Python: /home/user/mambaforge/envs/tensorml/bin/python3.11 (found version "3.11.6") found components: Interpreter
-- ========================= Importing and creating target nvonnxparser ==========================
-- Looking for library nvonnxparser
-- Library that was found /home/user/TensorRT-LLM/TensorRT-<IP_ADDRESS>/targets/x86_64-linux-gnu/lib/libnvonnxparser.so
-- ==========================================================================================
-- Configuring done (31.5s)
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDNN_LIB
    linked by target "tensorrt_llm_static" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm
    linked by target "tensorrt_llm" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm
    linked by target "nvinfer_plugin_tensorrt_llm" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm/plugins
NCCL_LIB
    linked by target "tensorrt_llm_static" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm
    linked by target "tensorrt_llm" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm
    linked by target "runtime_src" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm/runtime
    linked by target "nvinfer_plugin_tensorrt_llm" in directory /home/user/TensorRT-LLM/cpp/tensorrt_llm/plugins

-- Generating done (0.2s)
CMake Generate step failed.  Build files cannot be regenerated correctly.
Traceback (most recent call last):
  File "/home/user/TensorRT-LLM/./scripts/build_wheel.py", line 248, in <module>
    main(**vars(args))
  File "/home/user/TensorRT-LLM/./scripts/build_wheel.py", line 149, in main
    build_run(
  File "/home/user/mambaforge/envs/tensorml/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake -DCMAKE_BUILD_TYPE="Release" -DBUILD_PYT="ON"  -DTRT_LIB_DIR=/home/user/TensorRT-LLM/TensorRT-<IP_ADDRESS>/targets/x86_64-linux-gnu/lib -DTRT_INCLUDE_DIR=/home/user/TensorRT-LLM/TensorRT-<IP_ADDRESS>/include -S "/home/user/TensorRT-LLM/cpp"' returned non-zero exit status 1.

Managed to build the wheel by following instructions on docker.multi, without using the docker image!


fix bug: generate uuid; added param policy into client

AT

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


[TRGs]: Add distroless as aligned allowed docker base image

Description
distroless is a lightweight image, that can be used, if you want to just run a simple binary for example.
We need to collect the necessary information for the "Notice on docker images" and add it to the list of aligned images

Hi @SebastianBezold, do we still need that? Is distroless used anywhere by us?

Hi @tomaszbarwicki,
since this is not urgent, and I am not sure if there are publicly available container scan results, we can drop the issue for now and revisit, if it becomes relevant again.
I thought we had examples basing on distroless, but cannot find it ad-hoc


Farthest point cloud in the opposite direction of movement disappears when moving go2

Hi, thank you for sharing the code!
When i start ros2 launch go2_robot_sdk robot.launch.py, I can see the point cloud data for point_cloud2, but the point cloud seems to have a range. When I move go2, the farthest point cloud in the opposite direction of movement disappears. It seems that the complete map cannot be saved. And I saw the same situation in the gif of Go2 Air/PRO/EDU lidar point cloud in your README. So, how can I save the complete point cloud map? Is there any setting that needs to be modified?

Hello, yes. You can save it via the params
export MAP_SAVE=True
export MAP_NAME="3d_map"
After this you can use Meshlab to view it


typedef redefinition with different types ('int' vs '_mode_t' (aka 'unsigned short')) on vanilla install

I'm gettting typedef redefinition with different types ('int' vs '_mode_t' (aka 'unsigned short')) on a fairly vanilla install, closely following the instructions in README.md.
This machine (x64) already had VS2019 and MSYS2 installed. (I won't be using VS2019 for GNUstep projects, but it's worth noting that it's there)
I installed the latest clang using winget -- as per README.md ...
$ clang --version
clang version 18.1.5
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: C:/msys64/mingw64/bin

I downloaded and installed GNUstep 2023-12-28 Debug and Release builds and installed them both at C:\GNUstep -- such that I end up with C:\GNUstep\x64\Debug\... and C:\GNUstep\x64\Release\... -- as per README.md ...
Here's the source file -- copy/pasted from README.md ...
#include <Foundation/Foundation.h>
int main(int argc, char *argv[])
{
    NSLog(@"Hello Objective-C");
    return 0;
}

These same steps  -- copy/pasted from README.md -- produce the same error in all of the provided MSYS2 shell environments ...
me@MYPC MSYS ~
$ whereis clang
clang: /mingw64/bin/clang.exe

me@MYPC MSYS ~
$ export PATH="$PATH:/c/GNUstep/x64/Debug/bin/"

me@MYPC MSYS ~
$ cd /c/Users/me/Desktop/GSTEST/

me@MYPC MSYS /c/Users/me/Desktop/GSTEST
$ clang `gnustep-config --objc-flags` -c test.m
In file included from test.m:1:
In file included from C:/GNUstep/x64/Debug/include/Foundation/Foundation.h:200:
In file included from C:/GNUstep/x64/Debug/include/dispatch/dispatch.h:30:
C:/GNUstep/x64/Debug/include/os/generic_win_base.h:31:13: error: typedef redefinition with different
      types ('int' vs '_mode_t' (aka 'unsigned short'))
   31 | typedef int mode_t;
      |             ^
C:/msys64/mingw64/include/sys/types.h:77:17: note: previous definition is here
   77 | typedef _mode_t mode_t;
      |                 ^
1 error generated.

Windows 10 Pro 22H2
VS2019 (Community) 16.11.27

It looks like your shell is set up with MSYS2’s "MINGW64" environment and it’s picking up MinGW Clang and headers from that environment. For using GNUstep please use the plain "MSYS" environment, i.e. the first one listed here and then load the Visual Studio environment inside it (e.g. using vcvarsall.bat).


Program freezes after run_to_position command

Hi there,
i've implemented a simple python script with a board camera for Pan/Tilt Tracking Control.
I am using two motors for tracking an object. For move to the desired position i am using x_motor.run_to_position(x) and y_motor.run_to_position(y). After a while the program freezes.
These are the terminal messages after a keyboard interruption
_File "/usr/local/lib/python3.9/dist-packages/buildhat/motors.py", line 210, in _run_positional_ramp
self.hat.rampcond[self.port].wait()
File "/usr/lib/python3.9/threading.py", line 312, in wait
waiter.acquire()
KeyboardInterrupt
Thx in advance for your support!
code excerpt (inside a while loop):
```
if Track:
# calculating the center of gravity of contour
if cog_contour_x:
cog_contour_x =  0.8cog_contour_x + 0.2(x+w/2) # low-pass filter
else:
cog_contour_x = x+w/2
if cog_contour_y:
cog_contour_y = 0.8cog_contour_y + 0.2(y+h/2) # low-pass filter
else:
cog_contour_y = y+h/2
            # calculating distance between cog_contour and the center of the window
            error_x = cog_contour_x - dispW/2
            panAngle += error_x/PIXEL2DEGREE_RATE

            if panAngle > MAX_PAN_ANGLE:
                panAngle = MAX_PAN_ANGLE
            if panAngle < -MAX_PAN_ANGLE:
                panAngle = -MAX_PAN_ANGLE

            if abs(error_x) > MIN_PIXEL_ERROR:
               x_motor.run_to_position(panAngle)

            error_y = cog_contour_y - dispH/2
            tiltAngle -= error_y/PIXEL2DEGREE_RATE

            if tiltAngle > MAX_TILT_ANGLE:
                tiltAngle = MAX_TILT_ANGLE
            if tiltAngle < -MAX_TILT_ANGLE:
                tiltAngle=-MAX_TILT_ANGLE

            if abs(error_y) > MIN_PIXEL_ERROR:
               y_motor.run_to_position(tiltAngle)

Hi there again,
i've observed a further behavior of my motor. Sometimes i've stoped the program without freezes. Then i've got following terminal messages:
serial.serialutil.PortNotOpenError: Attempting to use a port that is not open
Am i stressing the motor to much?

The serial error is from the serial port connection to the BuildHAT.  My assumption is that you are sending commands too fast to the BuildHAT, it's getting overloaded and acting strange.

Thank you very much for your reply!
I've tried to slow down the frequency of updating the motor position. It became better, but object tracking with build hat seems to be too slow. I am now trying with 2 servos. It seems to be a bit more robust.

see comments above


Revert "Revert "allows zone.js to cleanup error stacktraces""

Reverts microsoft/azuredatastudio#7499
Keeping this for later when I can do more tests.

@aaomidi let's merge to unblock the CI queue.


land-sea mask

Hi NeuralGCM team,
I wonder how the land-sea mask is specified in NeuralGCM. For example, in the 1.4-deg deterministic model, is it based on the NaN values after regridding the 0.25-deg resolution ERA5 SST using xarray_utils.regrid?
I noticed that the very first inference demo yielded quite different locations of NaN values after regridding from the current demo. The current demo has more ocean (i.e., no-NaNs) area. Maybe you changed xarray_utils.regrid a little?
Thanks!
Yu

Our trained models used regridding as shown in the current online docs/demo notebook.
The original demo notebook was using a mistaken setting (horizontal_interpolation.ConservativeRegridder with skipna=False) which resulted in NaN values on coastlines, that needed to be filled with nearest neighbors values.
I wrote a detailed guide to regridding here. Please take a look and let me know if you still have questions.

This information is very helpful, Shoyer! I have a question about the guide where you mentioned that "NeuralGCM’s surface model also includes a mask that ignores values over land." After regridding the ERA5 data to the targeted Gaussian grid, do the land values or locations that will be ignored correspond to NaNs?

I have a question about the guide where you mentioned that "NeuralGCM’s surface model also includes a mask that ignores values over land." After regridding the ERA5 data to the targeted Gaussian grid, do the land values or locations that will be ignored correspond to NaNs?

In principle, the only locations that take sea surface temperature input account are locations where the land/sea mask is less than 1, so the details of the NaN filling should not matter.
In practice, the model does seem to make slightly different predictions when NaN values are filled in an inconsistent fashion. I don't think we ever quite tracked down why this is the case -- possibly there are some inconsistencies in because ERA5's land/sea mask and SST or sea ice fields.

I am asking because I am trying to couple NeuralGCM with a statistical SST model, and I need to know the exact ocean grids from which NeuralGCM reads SST information.
So, after regridding the 0.25-degree ERA5 SST data to the targeted Gaussian grid, are the grids with NaN values actually a subset of the land grids? In other words, do the grids with non-NaN values include all ocean grids and some coastal land grids?
If this is the case, I can provide NeuralGCM with SST on all grids with non-NaN values, since NeuralGCM won't read the coastal land grids anyway.

So, after regridding the 0.25-degree ERA5 SST data to the targeted Gaussian grid, are the grids with NaN values actually a subset of the land grids? In other words, do the grids with non-NaN values include all ocean grids and some coastal land grids?

Yes, this is correct.

Thanks!


Fix initial value of all store

With this change this useStore(initialValue) (or with  getStore / withStore) is valid. However, is doing an Object.assign with the parameter + the current store, this way only add the missing store variables and is not removing the existint ones.
const { getStore } = createStore({ cart: { price: 0 }})
getStore({ username: 'Aral' })
// Now the store is { username: 'Aral', cart: { price: 0 } }
// Now the initial store is { username: 'Aral', cart: { price: 0 } }

The difference with setStore is that setStore is replacing all the store
without touching the initialStore:
const { getStore } = createStore({ cart: { price: 0 }})
const [,setStore] = getStore()
setStore({ username: 'Aral' })
// Now the store is { username: 'Aral' }
// The initial store is the same that before: { cart: { price: 0 } }

Proposal for future releases: RenamegetStore to another name because do more things than get... For example myStore...

I need to reduce 2 bytes 😅

go for that 2 bytes! 💪

I close this PR because with this other PR there is no more initial value, so the same can be done directly with setStore(store => ({ ...store, newProperty: 'blabla' })). And it's already normal that useStore({ someValue: 2 }) does not work, because as it already exists the store does not change it. So it only works by properties (useStore.someValue(2)). What I will do is explain it better in the documentation.