🐛 Android: incorrect resolution/size photo output.

What were you trying to do?
On android, I get a max photo output of 1080x1920 resolution. While the camera is capable of taking pictures of 3000x4000. It looks like the camera is zoomed in somehow, but i have zoom on 0.
I tried the expo-camera/react-native-camera library and their output size is correct.
What happened instead?
Photo output is max 1080x1920 while the camera is capable of 3000x4000
Relevant log output
No response
Devices
Samsung A41
Samsung A40
Samsung Galaxy S9
Samsung Galaxy S10
Sony xperia XZ1
VisionCamera Version
2.5.0
Additional information

[ ] I am using Expo
[X] I have read the Troubleshooting Guide
[X] I agree to follow this project's Code of Conduct
[X] I searched for similar issues in this repository and found none.

Please share the code of the camera component. Are you using format={...}? What device={...} are you using? expo-camera and react-native-camera both do not support custom/multiple devices selection nor format selection

Hi Marc,
Thank you for your reply.
The device I'm using is a Samsung A41.
1. Did you use format={...}
Yes, I did use format={...} no luck either. i used the sort format function from the documentation to get the highest resolution
https://mrousavy.github.io/react-native-vision-camera/docs/guides/formats#sort
output:
{"autoFocusSystem": "none", "colorSpaces": ["jpeg"], "fieldOfView": 69.64898258937245, "frameRateRanges": [{"maxFrameRate": 30, "minFrameRate": 1}, {"maxFrameRate": 10, "minFrameRate": 10}, {"maxFrameRate": 15, "minFrameRate": 15}, {"maxFrameRate": 20, "minFrameRate": 15}, {"maxFrameRate": 20, "minFrameRate": 20}, {"maxFrameRate": 30, "minFrameRate": 5}, {"maxFrameRate": 30, "minFrameRate": 30}], "isHighestPhotoQualitySupported": true, "maxISO": 1600, "maxZoom": 4, "minISO": 100, "photoHeight": 1440, "photoWidth": 2560, "supportsPhotoHDR": false, "supportsVideoHDR": false, "videoHeight": 1440, "videoStabilizationModes": ["off", "auto", "standard"], "videoWidth": 2560}
as you can see the max resolution is 1440x2560
Output from the devices is only 1: ["wide-angle-camera"] while the phone has 3 cameras.
Both expo-camera en react-native-camera gives me a resolution of 2250x4000
When i take a photo, the output is: {"height": 1080, "isRawPhoto": false, "metadata": null, "path": "/data/user/0/com.cameratest/cache/mrousavy99938243064036335.jpg", "width": 1440}
Code from camera component.
export default function App() {
  const [hasPermission, setHasPermission] = React.useState(null);

  const cameraRef = React.useRef();
  const devices = useCameraDevices();
  const device = devices.back;

  const formats = React.useMemo(
    () => device?.formats.sort(sortFormats),
    [device?.formats],
  );

  React.useEffect(() => {
    (async () => {
      const cameraPermission = await Camera.getCameraPermissionStatus();
      const microphonePermission = await Camera.getMicrophonePermissionStatus();

      if (microphonePermission === 'not-determined') {
        await Camera.requestMicrophonePermission();
      }
      if (cameraPermission === 'not-determined') {
        await Camera.requestCameraPermission();
      }

      setHasPermission(cameraPermission === 'authorized');
    })();
  }, []);

  const handleCaputre = async () => {
    console.log('asdasd');
    const data = await cameraRef.current.takePhoto({
      enableAutoStabilization: true,
      skipMetadata: true,
      photoCodec: 'jpeg',
      quality: 100,
    });

    console.log(data);
  };

  if (hasPermission === null) {
    return <View />;
  }

  if (hasPermission === false) {
    return <Text>No access to camera</Text>;
  }

  return (
    <SafeAreaView style={styles.container}>
      <View
        style={{
          ...StyleSheet.absoluteFillObject,
          display: 'flex',
          justifyContent: 'center',
          alignItems: 'center',
        }}>
        <Camera
          ref={cameraRef}
          style={{...StyleSheet.absoluteFillObject}}
          device={device}
          format={formats[0]}
          zoom={0}
          isActive={true}
          photo={true}
          video={true}
          audio={false}
        />
        <Pressable
          style={{...StyleSheet.absoluteFillObject, zIndex: 999999}}
          onPress={handleCaputre}
        />
      </View>
    </SafeAreaView>
  );
}

Any update on this? The issue still persist in version 2.5.0.
I'm not able to get a high resolution output (tested it on 3 different android devices so far). The max resolution i'm getting is 2560x1440. While react-native-camera gives me 4000x3000, using the same camera.
Please clarify this for me? In case I'm missing something or this is an actual bug.
Thanks

CameraX does not expose an API that provides all of the information we need for formats. I'm patching this information together and apparently this returns wrong sizes (using CameraCharacteristics), I can take a look soon

Hi @mrousavy,
Thanks for your reply. Glad you want to take a look at it. Other than that, this library is really great and has a lot of potential for a go-to camera library for React native.
I will try to help where i can (as my java/kotlin knowledge is almost zero). i can do some research.

Maybe this is something to look after?
"You can always use camera2 api StreamConfigurationMap.getOutputSizes() to get supported resolution even in CameraX. As CameraX is build on top of camera2 it shouldn't matter"
source: https://stackoverflow.com/questions/60901831/how-to-get-available-resolutions-using-camerax-on-android

Yes, I know, that's what I'm doing https://github.com/mrousavy/react-native-vision-camera/blob/ee762ce5f5f6d6738d3e62b9042c90322f0b09ef/android/src/main/java/com/mrousavy/camera/CameraViewModule.kt#L207
You could help debug this code to find out why it isn't working

Thanks for pointing me in the right direction. I will help debug this and come back to you.

I noticed that if you don't use a custom format and let cameraX determine the configuration, the output photo is not zoomed in.
So I just created a patch to use setTargetAspectRatio based on the format.photoHeight and format.photoWidth instead of the setTargetResolution with the format.photoSize, it does the job for me: https://gist.github.com/jeremybarbet/c89698df5ddd8f312de04aed4f2dc555
I tried to see why setTargetResolution is not successfully using the resolution passed from the format but I can't see from where the issue is coming from.

Nice job @jeremybarbet!
This always chooses the best resolution available though... If you find a solution that also chooses the correct resolution, please let me know/create a PR! Maybe the rotation is messed up?

Maybe related, but I noticed when hacking the FrameProcessor to capture a specific resolution, I had to reverse the width and height in order for CameraX to select the exact resolution described:
        // Hack! Ensure Preview and imageAnalysisBuilder are seeing them same thing
        var resolutionFlipped = Size(format.videoSize.height, format.videoSize.width)
        previewBuilder.setTargetResolution(resolutionFlipped)
        imageAnalysisBuilder.setTargetResolution(resolutionFlipped)

If I didn't reverse it, AndroidX is choosing a best fit on the wrong aspect ratio.

Good catch @ldstein! Camera sizes are specified at landscape orientation, maybe CameraX is different in that case? Maybe CameraX changes that representation depending on Phone Orientation?
If you fixed the issue, could you shoot me a PR?

Looking into it. https://github.com/ldstein/react-native-vision-camera/commit/a63c2924bb542e8c4f4ef0568ff3ffe96284f67b
Not confident yet this is consistent across all Android devices.

@ldstein  this might be interesting:

Source: CameraX Configuration Options (Specify Resolution)

@julianklumpers did you manage to solve this resolution problem ?
I also get a max resolution of 2560x1440 on my Samsung A41 while react-native-camera gives me 4000x3000.

For Android devices I had to swap the format image and video sizes before setting it for the Camera component.
if (format && Platform.OS === 'android') {
    ;[format.photoHeight, format.photoWidth] = [format.photoWidth, format.photoHeight]
    ;[format.videoHeight, format.videoWidth] = [format.videoWidth, format.videoHeight]

    setFormat(format)
}

this way I got correct video preview size and correct camera output.

lol that's gotta be one of the weirdest JS code snippets I've ever seen

Okay but seems like rotation is applied incorrectly in native

@GaspardC No did not solve the resolution problem yet unfortunately. I can try @viljark solution and hope it works for me to. But all of this seems like a temporary hack/fix. I did not have the time and knowledge to look into the native Java/Kotlin code to help solve it. So I hope @mrousavy could help with this because it is a real blocker for us right now for our product to use this library at the moment.

I don't have any free time right now, been busy with the app development agency I'm running.
I'll take a look when I have free time :)

For Android devices I had to swap the format's image and video sizes before setting it for the Camera component.
if (format && Platform.OS === 'android') {
    ;[format.photoHeight, format.photoWidth] = [format.photoWidth, format.photoHeight]
    ;[format.videoHeight, format.videoWidth] = [format.videoWidth, format.videoHeight]

    setFormat(format)
}

this way I got correct video preview size and correct camera output.

So this trick works for all of my test phones (Android 9-10), but for my test tablet (Lenovo TB-X505L, Android 10) I don't need to switch the values.

@viljark
Thank you for your workaround. I was finally able to get the resolution i was looking for.
There is only one problem, the width and height are now switched.
{"photo": {"height": 3024, "isRawPhoto": false, "metadata": {"Orientation": 6, "{Exif}": [Object], "{TIFF}": [Object]}, "path": "/data/user/0/com.cameratest/cache/mrousavy5629928989675797420.jpg", "width": 4032}}
I would expect the height to be 4032 and the width  to be 3024. I think this has something to do with rotation?
Did you also have this output?

I noticed the same issue as @viljark when I first investigated this in September. Seems like some camera sensors are mounted in portrait orientation and some in landscape orientation.
Unfortunately, I was unable to work out how to coax this info from CameraX to intelligently flip format dimensions and had to move on to other projects.

i did some research and find out that the orientation number that comes back from the captured photo metadata, in my case {"Orientation": 6, ... is related to the image below.
But when switching the format width and height as @viljark suggested, the camera finally give back the correct resolution, but the metadata is not updated.

source: https://www.impulseadventure.com/photo/exif-orientation.html

@julianklumpers from my testing I have seen that you will always get the width > height (on android phones at least) and the rotation is then specified by the rotation option on the exif info (so orientation flag 6 === Rotate 90 CW). Most of the places (RN Image, browsers, file viewers) show the image correctly and take the rotation flag into account.
Since some of our logic required the image widht and height be correct without the orientation flag, I created a workaround by using expo-image-manipulator to process the image and then rewriting the image width and height.
import { manipulateAsync, SaveFormat } from 'expo-image-manipulator'
const photo = await cameraRef.current!.takePhoto({
	qualityPrioritization: 'speed',
	enableAutoStabilization: true,
})

const result = await manipulateAsync(photo.path, [], {
	format: SaveFormat.JPEG,
	compress: MAX_IMAGE_QUALITY,
})
photo.path = result.uri
photo.width = result.width
photo.height = result.height

For Android devices I had to swap the format's image and video sizes before setting it for the Camera component.
if (format && Platform.OS === 'android') {
    ;[format.photoHeight, format.photoWidth] = [format.photoWidth, format.photoHeight]
    ;[format.videoHeight, format.videoWidth] = [format.videoWidth, format.videoHeight]

    setFormat(format)
}

this way I got correct video preview size and correct camera output.

So this trick works for all of my test phones (Android 9-10), but for my test tablet (Lenovo TB-X505L, Android 10) I don't need to switch the values.

Some additional testing revealed, that there is no need to swap the width and height if the camera device ise set while the phone is in landscape. 🤷

The default camera preview resolution I get is 864 x 480.
I want to set the resolution to 1920 x 1080 with the following code:
  const format = React.useMemo(() => {
    const desiredWidth = 1920;
    const desiredHeight = 1080;
    if (device){
      for (let index = 0; index < device.formats.length; index++) {
        const format = device.formats[index];
        console.log("h: "+format.videoHeight);
        console.log("w: "+format.videoWidth);
        if (format.videoWidth == desiredWidth && format.videoHeight == desiredHeight){
          console.log("select format: "+format);
          return format;
        }
      };

    }
    return undefined;
  }, [device?.formats])

But I only get 640x480.

The default camera preview resolution I get is 864 x 480.
I want to set the resolution to 1920 x 1080 with the following code:
  const format = React.useMemo(() => {
    const desiredWidth = 1920;
    const desiredHeight = 1080;
    if (device){
      for (let index = 0; index < device.formats.length; index++) {
        const format = device.formats[index];
        console.log("h: "+format.videoHeight);
        console.log("w: "+format.videoWidth);
        if (format.videoWidth == desiredWidth && format.videoHeight == desiredHeight){
          console.log("select format: "+format);
          return format;
        }
      };

    }
    return undefined;
  }, [device?.formats])

But I only get 640x480.

Did you got solution for this i am also facing the same issue ?

Just wanted to mention that switching metadata width and height is also happening on iPhone 11. Plus orientation is also 6 when taking portrait picture.

Photo & video sizes not configurable neither with presets nor with format props. (With format you can actually change size - but it will be random, not even close to width/height of specified format).
I had to hardcode desired resolution with patch: https://github.com/mrousavy/react-native-vision-camera/files/11178551/react-native-vision-camera%2B2.15.4.patch

Hey @garmoshka-mo! That patch on iOS is not safe and might crash on a few devices.
On Android; yes, I left a few comments in the codebase warning that this is not accurate on Android due to CameraX's design. I will rewrite the android part to Camera2 in the V3 PR, sponsor me on GitHub to support this effort. I need a lot of free time to do that.

Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3.
I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here?
Here's an instruction on how you can test that: https://github.com/mrousavy/react-native-vision-camera/pull/1674#issuecomment-1684104217
If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉
Otherwise please let me know and I'll keep this issue open to keep track of it.
Thank you!

Hey - I'm tracking Orientation in this feature request/issue now: https://github.com/mrousavy/react-native-vision-camera/issues/1891 ✨
Make sure to upvote or sponsor to support this feature, and leave a comment if you have any additional thoughts/ideas.

@mrousavy can one go straight from v2 to v3?

I think you might need to change the way you get a CameraDevice, other than that yes


look into XDD inconsistencies

look into XDD inconsistencies
*5f6d0e20a58f1dfd52184931 document has 2 entries in xdd?
*Only 8% (807 of 10,000) have abstracts???
*Essentially no documents with images found?

*xDD is grabbing this from Pubmed Extracts as well as Fulltext. This results in 2 distinct objects. They are aware of this issue and will be working on a merge in the future
*We are now able to provide a new flag for our query called "field" on this we can provide fields that are required such as "abstract". This will now filter out on their side those that do not have abstracts instead of filtering out on our side

Using the ASKEM object API we are able to search for tables and figures if we believe that to be more important than papers


Fix: Typography

Issue Number
fixes #141
Describe the changes you've made
Typography configuration has changed and font sizes are increased. Also fixed font sizes in some files.
Describe if there is any unusual behavior (Any Warning) of your code(Write NA if there isn't)
NA
Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.
[x] The title of my pull request is a short description of the requested changes.

@Abhishek-kumar09 please review this and let me know any changes.

Please also share screenshots of pages

do I need to reduce the font size more?

Make this a bit smaller and thicker

The less there will be changes in current view of app, the better it would be

Does this look good?

Hi @faisaljamil25
This is the figma design: https://www.figma.com/file/3QxrkclcjQFa2dplfWrQ8v/CFC-Application-(Copy)
Please ensure the changes align with the design.

Hi @faisaljamil25
This is the figma design: https://www.figma.com/file/3QxrkclcjQFa2dplfWrQ8v/CFC-Application-(Copy)
Please ensure the changes align with the design.

@Abhishek-kumar09 I have almost made the typography and font sizes identical or near to the one that you have provided and also changed some places where font sizes and font weights are hardcoded but still, a lot of places are left to be redefined and it would take a whole lot of time to correct each place to make as identical with the design (say if I increase font-weight of h3 to make align at one place it may be possible that it breaks font style at some other place and then I have to go to those individual places to change their variants and who knows correcting that place font style may lead to change in font-weight of some other variant and the loop continues). So, I think it would be better if we define a common typography configuration and then change the font style while working on those files themselves. What do you think?

@Abhishek-kumar09 have a look at this:

Looks good, Thanks @faisaljamil25


Update data-lake-analytics.md

Added overview, example code, and samples. Removed client section as there is only a management API. @CamSoper I will also have team review before sign-off.

#sign-off

:x: Validation status: errors

File
Status
Preview URL
Details

api/overview/azure/index.yml
:x:Error

Details

:x:Error

Details

api/overview/azure/index.yml

[Error] Error transforming model "W:\wvq0-o-unknown\raw-\zpc3wo0c.x5s\rawmodel\api\overview\azure\index.raw.json" generated from "api/overview/azure/index.yml" using "ManagedReference.html.primary.js". Error running Transform function inside template preprocessor:

[Error] Error transforming model "W:\wvq0-o-unknown\raw-\zpc3wo0c.x5s\rawmodel\api\overview\azure\index.raw.json" generated from "api/overview/azure/index.yml" using "ManagedReference.html.primary.js". Error running Transform function inside template preprocessor:

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@JeremyLikness - Two things:

Can you please add the strong types in the example?  Scott called out that he doesn't want vars in these examples.

To save me a step (and to make sure we preserve you as a contributor) can you please create a new PR with this same file going against the master branch instead?  I want to use the files from the aug-1-stage branch as the starting point (because I generated the templates for you guys there) but I want to check them directly into master.   I'll send an email to the team to that effect.

:x: Validation status: errors

File
Status
Preview URL
Details

api/overview/azure/index.yml
:x:Error

Details

:x:Error

Details

api/overview/azure/index.yml

[Error] Error transforming model "W:\b5ch-o-14d97248\raw-\52wfmejc.2xi\rawmodel\api\overview\azure\index.raw.json" generated from "api/overview/azure/index.yml" using "ManagedReference.html.primary.js". Error running Transform function inside template preprocessor:

[Error] Error transforming model "W:\b5ch-o-14d97248\raw-\52wfmejc.2xi\rawmodel\api\overview\azure\index.raw.json" generated from "api/overview/azure/index.yml" using "ManagedReference.html.primary.js". Error running Transform function inside template preprocessor:

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Band and card containers - fix bug with dynamic context setting

Band and card containers detect their background colors and if it is darkest, darker/dark, complement, or accent it will update the pfe-cta component to add the on="dark" attribute.  Right now if you load pfe-card before pfe-band, this inheritance works great.
<script>
require([
      '../../pfe-cta/pfe-cta.umd.js',
      '../../pfe-card/pfe-card.umd.js',
      '../pfe-band.umd.js'
    ]);
</script>

If you load pfe-band and then pfe-card, the pfe-band sets the on="dark" on ctas inside the card context (because the card has not updated yet to add the container attribute).
<script>
require([
      '../pfe-band.umd.js',
      '../../pfe-card/pfe-card.umd.js',
      '../../pfe-cta/pfe-cta.umd.js'
    ]);
</script>

Need to fix the this so that the order of loading does not matter; maybe by queueing off something other than [type="container"]?

This is reworked in https://github.com/patternfly/patternfly-elements/pull/392


Fix: addon version bug

Signed-off-by: 楚岳<EMAIL_ADDRESS>Description of your changes

Fixes #
I have:

[ ] Read and followed KubeVela's contribution process.
[ ] Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
[ ] Run make reviewable to ensure this PR is ready for review.
[ ] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested

Special notes for your reviewer

contained in #3533


Link to EntityMention in the v1beta2 Entity documentation results in a NotFound.

This is an issue with the online documentation.
There is a link to EntityMention in:
https://googlecloudplatform.github.io/google-cloud-node/#/docs/language/0.11.0/language/v1beta2/data_types?method=Entity
The link address is:
https://googlecloudplatform.github.io/google-cloud-node/#/docs/language/0.11.0/Data Types%2Fv%2Fdata_types?method=EntityMention
But it redirects to the Not Found page:
https://googlecloudplatform.github.io/google-cloud-node/#/docs/google-cloud/0.56.0/not-found

Actually, I think there are quite a few links that are broken. The link to TextSpan in the EntityMention documentation is broken as well.

Closing as a duplicate of #2546


Fix button overflow issue

Horizontal scroll is added to the chiplets

Thank you for your contribution 😄

@NewtonFernandis cam you assign me this issue?


Add a few extensions

@willingc I think this is good to go, shall I/you merge it and bump/publish?

Thanks :D


Buffering/Delegate client

To solve #23 (Resume games), an approach could be that the Client object that is passed to a game is actually a delegating client that delegates to the real client if it is online, otherwise it buffers up a bunch of events (only the important ones) and sends them when the client is online again.

Using the DSL approach of making games helped with this a whole lot more. Then the client can simply request the current game view when joining.
Also, the fact that the game needs to be loadable from database also has an impact here.


New Steam Game Pad UI (Steam Deck) Prevents Proper Setup

The UI falsely displays network scanning errors during setup. This causes the inability to properly set up Steam.
Possibly related, the UI shows the following exception:

One workaround is to use the normal desktop UI initially to log into Steam by setting the startup flag -nogamepadui. Example section in config.toml:
 [[apps]]
title = "Steam"
start_virtual_compositor = true

[apps.runner]
type = "docker"
name = "WolfSteam"
image = "ghcr.io/games-on-whales/steam:edge"
mounts = ["/run/udev:/run/udev:ro"]
env = [
"STEAM_STARTUP_FLAGS=-nogamepadui","PROTON_LOG=1","RUN_GAMESCOPE=true","GOW_REQUIRED_DEVICES=/dev/uinput /dev/input/event* /dev/dri/* /dev/nvidia*",
]
devices = []
ports = []
base_create_json = """
{
  "HostConfig": {
    "IpcMode": "host",
    "CapAdd": ["SYS_ADMIN", "SYS_NICE"],
    "Privileged": false
  }
}
\
"""

After logging in, Steam can continue to be used with the flag omitted if desired.

Thanks to @Drakulix we might have fixed the issues with the Steam container, you can try it out by updating the steam container
docker pull ghcr.io/games-on-whales/steam:edge

and restarting Wolf. There's no need for the Privileged: true flag and you should be able to see the new steam deck UI and even stream it to multiple clients.
Feel free to reopen this if it's not working for you.

You don't need privileged, you just need the pivot_root syscall, which is blocked by the default seccomp profile.
A custom profile should fix this, we should probably provide one for steam.
However privileged being broken is something we should fix address, because there are other reason for using it.

You can edit the game settings and set the resolution to "Native" to fix that.


BlockBlobClient throws System.IO.IOException when it is used for uploading a large file.

BlockBlobClient throws the exception when it is used for uploading a large file (1.2Gb) to Azure Gov Blob Storage. The issue is actual for large files only. It is interesting that BlobClient uploads the same file successfully. The details are below.
Library: 12.1.0 (https://www.nuget.org/packages/Azure.Storage.Blobs/12.1.0)
App: .NET Core 2.2, Console application
Exception:
"System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.Net.Http.HttpRequestException: Error while copying content to a stream. ---> System.IO.IOException: Unable to read data from the transport connection: The I/O operation has been aborted because of either a thread exit or an application request. ---> System.Net.Sockets.SocketException: The I/O operation has been aborted because of either a thread exit or an application request
--- End of inner exception stack trace ---
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
at System.Net.Security.SslStreamInternal.g__CompleteAsync|36_1[TWriteAdapter](ValueTask writeTask, Byte[] bufferToReturn)
at System.Net.Security.SslStreamInternal.WriteAsyncChunked[TWriteAdapter](TWriteAdapter writeAdapter, ReadOnlyMemory1 buffer)    at System.Net.Security.SslStreamInternal.<WriteAsyncInternal>g__ExitWriteAsync|35_0[TWriteAdapter](ValueTask task)    at System.Net.Http.HttpConnection.WriteAsync(ReadOnlyMemory1 source)
at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
at Azure.Core.RequestContent.StreamContent.WriteToAsync(Stream stream, CancellationToken cancellation)
at Azure.Core.Pipeline.HttpClientTransport.PipelineRequest.PipelineContentAdapter.SerializeToStreamAsync(Stream stream, TransportContext context)
at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
--- End of inner exception stack trace ---
at System.Net.Http.HttpContent.CopyToAsyncCore(ValueTask copyTask)
at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken)
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithNtConnectionAuthAsync(HttpConnection connection, HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message)    at Azure.Core.Pipeline.RequestActivityPolicy.ProcessNextAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean isAsync)
at Azure.Core.Pipeline.RequestActivityPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean isAsync)    at Azure.Core.Pipeline.BufferResponsePolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)    at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)    at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline, Boolean async)
at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)    at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory1 pipeline)
at Azure.Storage.Blobs.BlobRestClient.BlockBlob.UploadAsync(ClientDiagnostics clientDiagnostics, HttpPipeline pipeline, Uri resourceUri, Stream body, Int64 contentLength, Nullable1 timeout, Byte[] transactionalContentHash, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentHash, String blobCacheControl, IDictionary2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable1 encryptionAlgorithm, Nullable1 tier, Nullable1 ifModifiedSince, Nullable1 ifUnmodifiedSince, Nullable1 ifMatch, Nullable1 ifNoneMatch, String requestId, Boolean async, String operationName, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary2 metadata, BlobRequestConditions conditions, Nullable1 accessTier, IProgress1 progressHandler, Boolean async, CancellationToken cancellationToken)    at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadAsync(Stream content, BlobHttpHeaders httpHeaders, IDictionary2 metadata, BlobRequestConditions conditions, Nullable1 accessTier, IProgress1 progressHandler, CancellationToken cancellationToken)"
Code:
`
var sharedKeyCredential = new StorageSharedKeyCredential("account-name", "account-key");
var blobServiceClient = new BlobServiceClient(
new Uri("https://miniostoragepoc.blob.core.usgovcloudapi.net"),
sharedKeyCredential);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient("test-container-for-large-files");

await containerClient.CreateIfNotExistsAsync();

using (FileStream uploadFileStream = File.OpenRead(@"c:\large_file.MP4"))
{
    BlockBlobClient blobClient = containerClient.GetBlockBlobClient("large_file.MP4");
    await blobClient.UploadAsync(uploadFileStream); // <-- The exception is here
}

`

@tg-msft, could you take a look?

It looks like BlockBlobClient.UploadInteral() doesn't support parallel and multi-part uploads:
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/BlockBlobClient.cs#L492
But BlobClient.StagedUploadAsync() does:
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/BlobClient.cs#L966
This is also a problem in AppendBlobClient.AppendBlockInternal() and PageBlobClient.UploadPagesAsync():
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/AppendBlobClient.cs#L679
https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/storage/Azure.Storage.Blobs/src/PageBlobClient.cs#L796
I think we can re-used BlobClient.StagedUploadAsync()'s logic for BlockBlobClient.UploadInteral(), but it would need to be re-written or generalized for AppendBlobClient.AppendBlockInteral() and PageBlobClient.UploadPagesInternal()


Slow execution of delete operations

Greetings,
I'm using Dexie.js for working with a pretty large database (around 600,000 rows at present), and while most operations (including import) are reasonably fast, deleting rows is painfully slow (maybe around 80 rows per second). So basically a table I can populate in 10-15 seconds takes like 10 minutes to selectively delete from. Is there something silly about the way I'm doing things, or is deletion just always going to be slow?
Database initialization:
    init() {
        if (this.db !== null) {
            return Promise.reject('database already initialized');
        }

        this.db = new Dexie('dict');
        this.db.version(1).stores({
            terms: '++id, dictionary, expression, reading',
            kanji: '++, dictionary, character',
            entities: '++, dictionary',
            dictionaries: '++, title, version',
            meta: 'name, value',
        });
    }

Code to delete data:
    deleteDictionary(title) {
        if (this.db === null) {
            return Promise.reject('database not initialized');
        }

        return this.db.terms.where('dictionary').equals(title).delete().then(() => {
            return this.db.kanji.where('dictionary').equals(title).delete();
        }).then(() => {
            return this.db.entities.where('dictionary').equals(title).delete();
        }).then(() => {
            return this.db.dictionaries.where('title').equals(title).delete();
        });
    }

I've tried breaking up the delete operations, using bulkDelete, but nothing seems to help in getting performance to be acceptable in my extension. Any advice would be much appreciated.
edit: I'm running the Dexie beta, but I had same issue on 1.5

That was an issue before (#208) but a lot of effort has been done to solve #208. Delete operations have  been optimized as close to maximum as possible using different approaches for different browsers (avoiding certain bugs in IE and edge). However the speed may vary between browsers. But 100 rows a second seems like it used to behave before the fix.
What browser engine are you using and which version? What version of Dexie are you using?

Are you using a transaction?

Thank you for the quick response! My code is running on Chrome Version 54.0.2840.90 (64-bit) on Linux Mint. I am using the Dexie 2.0.0-beta release (the one linked on the GitHub page). I don't know if it makes any difference, but this is a browser extension.
I am not using a transaction, and am trying to just get the rows dropped by any means possible.
Please let me know if there is anything I can do to help you investigate this issue.

I think it's becaused you are deleting the items one by one which results in that IndexedDB needs to reoptimize its internal structure because every other operation is a a query while the previous was  delete operation. You're also awaiting one transaction commit for each deletion which is also slow.
You could do it all in one parallell query. See if it helps:

    deleteDictionalries (titles) {
        return db.transaction('rw', this.db.terms, this.db.kanji, ..., ()=>{
            return Promise.all([
                this.db.terms.where('dictionary').anyOf(titles).delete(),
                this.db.kanji.where('dictionary').anyOf(titles).delete(),
                ...
            ]);
        });
    }

If the query is deleting hundreds of thousands of entries, this may take time anyway. In that case, you could use a limit(1000) and show some progress while doing the massive delete, then rerun it until the delete operation completes with zero deletions.

I see. Of some reason, delete seems to be slower than inserts. Manipulations are much slower than reads. Queries should be fast if using indexes to query even with large data.
Seems that you've stumbled on a bug in chrome similar to the one described in #208 that made Edge corrupt its indexedDB. Dexie has a built-in protection from that bug for ie/edge but not for chrome. But you can avoid it by adding a limit() to the delete query.
IndexedDB has a delete method where you can pass an IDBKeyRange as argument instead of just a single key. Deletes normally becomes up to 5 times faster when doing so. But probably this should be avoided also with chrome if the IDBKeyRange resolves to a huge number of records to avoid this bug. If you pass a limit to the delete query, Dexie cannot use an IDBKeyRange but has to delete items one by one instead. But it does it by first querying items to delete, then bulkDeletes them, so it's still OK in performance.
Let me know if you can use the limit ()  approach and at what speed it deletes your records.
I would actually advice you NOT to use this loop in a transaction if it takes time. It would starve other requests. Test if a limit of 1000 is good enough and log or show some progress before rerunning the 'limited' delete query.

Great, I will do this in my extension then. I already tried this approach just to verify that there is something going on and that Dexie is not hung, and it does appear to work (albeit very slowly). It will also allow me to display a progress bar for these long delete operations, so users can see that something is happening, at least.
Closing this issue as it sounds like a problem with the underlying IndexedDB technology and not the Dexie library. Thanks again for your advice. Aside from this one annoying issue, Dexie is working great in my extension 😄


MINOR: Exclude jline, netty and findbugs annotations

These dependencies are unnecessary and they are acquired
transitively via zkclient (jline, netty) and reflections (annotations).
Ewen did the hard work in figuring out why we have unexpected
additional dependencies since 0.9.x.

@ijuma We also transitively depend on annotations via the reflections library in connect-runtime. That library is LGPL. I did a quick test and it looks like we can just remove the dependency -- I think its only used for static analysis bug finding and doesn't seem to trigger any errors when its filtered (I just built, removed connect/runtime/build/dependant-libs/annotations-2.0.1.jar and tested the /connector-plugins endpoint for Connect).
Could we update this patch to also remove that jar, validate the output of releaseTarGzAll to not include any of these, run system tests, then merge this?

@ijuma @ewencp Currently we are using a fairly lenient resolution policy in Gradle. If you like we can open a jira to track strictly defining our dependencies (including transitive ones) so that we don't accidentally include any unknown dependencies as versions/dependencies change. This increases the fragility of the build and requires more explicit control/management, but its sounds like thats what we are looking for.

@ewencp, will do. @granthenke, feel free to file the JIRA. For a project like Kafka, it makes sense to be strict about dependencies.

I updated the PR with @ewencp's and @granthenke's suggestions. I checked that jline, junit, netty and annotations are excluded via ./gradlew *:dependencies and looking at the files generated by releaseTarGzAll.
System tests started:
https://jenkins.confluent.io/job/system-test-kafka-branch-builder/451/

@ijuma LGTM as long as the system tests pass

LGTM

LGTM.

Branch-builder test passed with 1 failed, confirmed as unrelated by @ewencp and @hachikuji.
+1 and merging to trunk and 0.10.0 branch

I had an issue with the merge script, so cherry-picked manually to 0.10.0:
https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=shortlog;h=refs/heads/0.10.0


Edit Goal Modal

[ ] The form shows input fields for User's to edit their saved goals
[ ] The form also has a button to close it if the user so wishes to exit the form
[ ] The form has a submit button to save the new changes that have been edited

Testing link => https://goals.zuri.chat/
Description: This is a modal for editing goals which is displayed after clicking on the Edit Button which lies in the dropdown menu button. (This was used while waiting for the main goal button to be added)


Invalid directory metadata served by S3-based repositories

What happened?

Reposilite version
3.x
Relevant log output
No response

Reproduced with MinIO:


Ordering is broken when using map_partitions returning a MultiIndex DataFrame

I understand that Dask doesn't yet support MultiIndex DataFrame. I noticed that if a function return a MultiIndex df when you use map_partitions and use MultiIndex DF as meta then the order is broken. Eg: value of some columns are in different columns.
This was a behavior that I observer with 0.13 and I'm not of the support for MultiIndex DF for some of this operations and the behavior may have changed since 0.13.
If this operation is not supported then It may be better to fail if the user pass a MultiIndex DF as meta. It's possible to make it works if you sort the columns on the meta DF and the output.

Thanks for the bug report.  Are you able to provide a minimum failing
example?  http://stackoverflow.com/help/mcve
On Mon, Mar 6, 2017 at 3:13 AM, Rach Belaid<EMAIL_ADDRESS>wrote:

I understand that Dask doesn't yet support MultiIndex DataFrame. I noticed
that if a function return a MultiIndex df when you use map_partitions and
use MultiIndex DF as meta then the order is broken. Eg: value of some
columns are in different columns.
This was a behavior that I observer with 0.13 and I'm not of the support
for MultiIndex DF for some of this operations and the behavior may have
changed since 0.13.
If this operation is not supported then It may be better to fail if the
user pass a MultiIndex DF as meta. It's possible to make it works if you
sort the columns on the meta DF and the output.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/2046, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszNShLCwpfkY3gGWVuIlOjCHtvW0Xks5ri8A9gaJpZM4MT1rs
.

Of course. Let me put an example together.

My past code had a similar logic to this but I'm not able to reproduce the ordering problem with this small example. I will close this ticket for now but I will try to check if my previous commit work with dask 0.14 and come up with a better example
from dask import dataframe as ddf
import pandas as pd
from dask.dataframe.utils import make_meta

sub_df_1_type = {
    'b': 'float64',
    'a': 'float64',
    'date': 'datetime64[ns]',
}

sub_df_2_type = {
    'a': 'object',
    'b': 'float64',
    'x': 'float64',
    'token': 'object',

}

make_meta(sub_df_1_type), make_meta(sub_df_2_type)
meta_df = pd.concat({'group1': make_meta(sub_df_1_type), 'group2': make_meta(sub_df_2_type)}, axis=1)
meta_df
df = pd.DataFrame([{'a': 1}]*1000)
returning_df = pd.concat({'group1': pd.DataFrame([{'a': 1, 'b': 2, 'date': '2011-01-01'}]*1000), 'group2': pd.DataFrame([{'a': 'woow', 'x': 4, 'token': 'something'}]*1000)}, axis=1)
dask_df = ddf.from_pandas(df, chunksize=10)
ddf.map_partitions(lambda *args, **kwargs: returning_df, meta=meta_df).compute()


Control revision updates for PG STIG V1R3

A compilation of the changes required for Rev 3 of the PG STIG.  No major functional changes - primarily housekeeping.

Overall it is not recommended to allow sudo access for postgres user, let us discuss with @jconway

I understand regarding the comment regarding sudo access - but as that was committed to the V1R3 version of the STIG; I was updating the test to conform to the approved document.


Do not show tournament games in the queue to the public

(Or, make this a toggleable option for any given tournament. Tho that would be harder)
The queue is nice to keep suspense, and to giveusers a feel for when their matches might be played.
However, for tournaments, those factors aren't so useful. Also, the queue can expose winners of the tournament, especially the final tournament, as watchers can count the number of matches to infer the number of rounds, and infer whether the low seed has reset the bracket (forcing an extra match) or if the high seed won in the first match
Before you submit this issue...
If you are not a Battlecode dev, please keep the triage label.
If you are a Battlecode dev, the following would be great, but optional.

[ ] If you know the priority, assign a label for it. (the red labels)
[ ] If you know the module(s), assign a label (or labels) for it. (the green labels)
[ ] If you know the type(s), assign a label (or labels) for it. (the yellow labels)
[ ] If you know this is a simple issue or fix, assign good first issue. (the blue label)
[ ] If you did not fill any of the above, please keep the triage label.

Description
Please include a description of the issue below, including steps to reproduce.

...dupe of #737, i can't read


Compatibility issue with latest RazorEngine build

I'm getting following error Azure Mobile App template. Created the instance in Azure, and then downloaded the Visual Studio package. It works in RazorEngine v 3.4.2. But breaks in 3.5.0 and above.
An error has occurred.Method not found: 'System.String RazorEngine.Templating.ITemplate.Run(RazorEngine.Templating.ExecuteContext)'.System.MissingMethodException at Microsoft.Azure.Mobile.Server.Content.HtmlActionResult.ExecuteAsync(CancellationToken cancellationToken)
at System.Web.Http.Controllers.ApiControllerActionInvoker.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ActionFilterResult.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Filters.AuthorizationFilterAttribute.d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.AuthenticationFilterResult.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Web.Http.Controllers.ExceptionFilterResult.d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Web.Http.Dispatcher.HttpControllerDispatcher.d__1.MoveNext()
Check corresponding ticket at https://github.com/Antaris/RazorEngine/issues/284

If this is still happening, can you open it against the server: https://github.com/Azure/azure-mobile-apps-net-server


It seems that SKUs have been deprecated - can this support Prices instead?

Feature request
Is your feature request related to a problem? Please describe.
Stripe support told me that SKUs are from an older version of the API and are not really supported anymore -- they prefer to use various Prices instead.
Describe the solution you'd like
Ideally, this demo would come with Prices supported out of the box.

I literally just spent 1 week working on the implementation of Products, Sku, and Order, before realizing that Sku and Order were deprecated ...
I don't understand why it is.
If we had a product with 10 sku object, we now need to create 10 products each with a Price object. And so we can no longer even manage the stocks, product dimensions, etc.
It's a shame ... and they should indicate it more clearly on Stripe, I wasted a lot of time for nothing

I did the same thing, @bastien70. 🤦 These demos from the official Stripe Github need to be clearly marked *DEPRECATED*.

@spsaucier-bakkt It's really annoying ...
So how did you proceed on your side?
Did you keep the SKU objects?
Or have you replaced SKU objects with products?

How can we use price object to manager sku? Because this object don`t have attr props for example


multi-gpu lora finetune falcon

Whether to support multi-gpu lora fine-tuning falcon? thx!

Multi-gpu would be good.  Accelerate is used, which helps with multi-gpu.  Is there as reason why multi-gpu is not an option already?

https://github.com/rmihaylov/falcontune/issues/7

Falcontune uses RefinedWeb to instantiate a model of type RefinedWebModel, which I think is causing issues with multiple gpus. When I built the code after replacing RefinedWebModel with AutoModelForCausalLLM in 'load_model' function, it worked on multiple gpus without any issues.

@praveenkanithi does falcontune work the replacement or are you saying the training code works with other models supported by refinedwebmodel?  If it works, is it a simple search and replace?

It's the first one. I'm just talking about how I fixed the issue of running Falcon on multiple gpus.

@praveenkanithi that is great, I would very much like to get falcontune working with multi-gpu.  I replaced RefinedWebModel with AutoModelForCausalLLM and now I see: "You are using a model of type RefinedWeb to instantiate a model of type AutoModelForCausalLLM. This is not supported for all configurations of models and can yield errors.", seems like I am headed in the right direction.
However, after mapping I immediately get the error: "ValueError: Pointer argument (at 1) cannot be accessed from Triton (cpu tensor?)"
Here are my parameters:
Parameters:
-------config-------
dataset='./alpaca_data_cleaned.json'
data_type='alpaca'
lora_out_dir='.models/checkpoints/falcon-40b-instruct-4bit-alpaca/'
lora_apply_dir=None
weights='./models/falcon/gptq_model-4bit--1g.safetensors'
target_modules=['query_key_value']
------training------
mbatch_size=1
batch_size=2
gradient_accumulation_steps=2
epochs=3
lr=0.0003
cutoff_len=256
lora_r=8
lora_alpha=16
lora_dropout=0.05
val_set_size=0.2
gradient_checkpointing=False
gradient_checkpointing_ratio=1
warmup_steps=5
save_steps=50
save_total_limit=3
logging_steps=5
checkpoint=False
skip=False
world_size=1
ddp=False
device_map='auto'
I have tried messing with the world size to enable ddp, but that created other issues.  I am running with the following command:
falcontune finetune
--model=falcon-40b-instruct-4bit
--weights=./models/falcon/gptq_model-4bit--1g.safetensors
--dataset=./alpaca_data_cleaned.json
--data_type=alpaca
--lora_out_dir=.models/checkpoints/falcon-40b-instruct-4bit-alpaca/
--backend=triton
--mbatch_size=1
--batch_size=2
--epochs=3
--lr=3e-4
--cutoff_len=256
--lora_r=8
--lora_alpha=16
--lora_dropout=0.05
--warmup_steps=5
--save_steps=50
--save_total_limit=3
--logging_steps=5
--target_modules='["query_key_value"]'

Any update on this? Tnx

@codybum, can you give me the full error and where are you getting this error?
FYI, I'm not running falcontune as they specified in this repository. Instead of installing this repository using setup.py, I made a few changes for falcontune imports (you need to do it in multiple files), so I can run using python ./falcontune/run.py with appropriate arguments.

@praveenkanithi If its not too much trouble perhaps you can post a working fork?
Here is the full error:
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please run
python -m bitsandbytes
and submit this information together with your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
bin /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib64'), PosixPath('/usr/local/nvidia/lib')}
warn(msg)
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] as expected! Searching further paths...
warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
/usr/local/lib/python3.10/dist-packages/bitsandbytes/cuda_setup/main.py:149: UserWarning: Found duplicate ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] files: {PosixPath('/usr/local/cuda/lib64/libcudart.so'), PosixPath('/usr/local/cuda/lib64/libcudart.so.11.0')}.. We'll flip a coin and try one of these, in order to fail forward.
Either way, this might cause trouble in the future:
If you get CUDA error: invalid device function errors, the above might be the cause and the solution is to make sure only one ['libcudart.so', 'libcudart.so.11.0', 'libcudart.so.12.0'] in the paths that we search based on your env.
warn(msg)
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 8.0
CUDA SETUP: Detected CUDA version 118
CUDA SETUP: Loading binary /usr/local/lib/python3.10/dist-packages/bitsandbytes/libbitsandbytes_cuda118.so...
You are using a model of type RefinedWeb to instantiate a model of type AutoModelForCausalLLM. This is not supported for all configurations of models and can yield errors.
The model weights are not tied. Please use the tie_weights method before using the infer_auto_device function.
The model weights are not tied. Please use the tie_weights method before using the infer_auto_device function.
The safetensors archive passed at models/falcon-40b-instruct-4bit/gptq_model-4bit--1g.safetensors does not contain metadata. Make sure to save your model with the save_pretrained method. Defaulting to 'pt' metadata.
Parameters:
-------config-------
dataset='pathicd-large-v1-train-cancer_falcon.json'
data_type='alpaca'
lora_out_dir='custom-falcon-40b_pathicd-large-v1-train-cancer_epochs1-4bit'
lora_apply_dir=None
weights='models/falcon-40b-instruct-4bit/gptq_model-4bit--1g.safetensors'
target_modules=['query_key_value']
------training------
mbatch_size=1
batch_size=2
gradient_accumulation_steps=2
epochs=1
lr=0.0003
cutoff_len=256
lora_r=8
lora_alpha=16
lora_dropout=0.05
val_set_size=0.2
gradient_checkpointing=False
gradient_checkpointing_ratio=1
warmup_steps=5
save_steps=50
save_total_limit=3
logging_steps=5
checkpoint=False
skip=False
world_size=1
ddp=False
device_map='auto'
Converted as Half.
trainable params: 8355840 || all params:<PHONE_NUMBER> || trainable%: 0.7767877541695225
Found cached dataset json (/root/.cache/huggingface/datasets/json/default-0181237cb7d9e7a7/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96)
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 536.91it/s]
Loading cached split indices for dataset at /root/.cache/huggingface/datasets/json/default-0181237cb7d9e7a7/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-2956635c10ad56da.arrow and /root/.cache/huggingface/datasets/json/default-0181237cb7d9e7a7/0.0.0/8bb11242116d547c741b2e8a1f18598ffdd40a1d4f2a2872c7a28b697434bc96/cache-c7be19ca6dfe40e6.arrow
Run eval every 4148 steps
Found safetensors installation, but --save_safetensors=False. Safetensors should be a preferred weights saving format due to security and performance reasons. If your model cannot be saved by safetensors please feel free to open an issue at https://github.com/huggingface/safetensors!
PyTorch: setting up devices
The default value for the training argument --report_to will change in v5 (from all installed integrations to none). In v5, you will need to use --report_to all to get the same behavior as now. You should start updating your code and make this info disappear :-).
You have loaded a model on multiple GPUs. is_model_parallel attribute will be force-set to True to avoid any unexpected behavior such as device placement mismatching.
git root error: Cmd('git') failed due to: exit code(128)
cmdline: git rev-parse --show-toplevel
stderr: 'fatal: detected dubious ownership in repository at '/workspace'
To add an exception for this directory, call:
git config --global --add safe.directory /workspace'

The following columns in the training set don't have a corresponding argument in PeftModelForCausalLM.forward and have been ignored: token_type_ids, input, instruction, output. If token_type_ids, input, instruction, output are not expected by PeftModelForCausalLM.forward,  you can safely ignore this message.
***** Running training *****
Num examples = 33,190
Num Epochs = 1
Instantaneous batch size per device = 1
Total train batch size (w. parallel, distributed & accumulation) = 2
Gradient Accumulation steps = 2
Total optimization steps = 16,595
Number of trainable parameters = 8,355,840
Automatic Weights & Biases logging enabled, to disable set os.environ["WANDB_DISABLED"] = "true"
0%|                                                                                | 0/16595 [00:00<?, ?it/s]You're using a PreTrainedTokenizerFast tokenizer. Please note that with a fast tokenizer, using the __call__ method is faster than using a method to encode the text followed by a call to the pad method to get a padded encoding.
Traceback (most recent call last):
File "", line 21, in matmul_248_kernel
KeyError: ('2-.-0-.-0-83ca8b715a9dc5f32dc1110973485f64-d6252949da17ceb5f3a278a70250af13-3b85c7bef5f0a641282f3b73af50f599-3d2aedeb40d6d81c66a42791e268f98b-3498c340fd4b6ee7805fd54b882a04f5-e1f133f98d04093da2078dfc51c36b72-b26258bf01f839199e39d64851821f26-d7c06e3b46e708006c15224aac7a1378-f585402118c8a136948ce0a49cfe122c', (torch.float16, torch.int32, torch.float16, torch.float16, torch.int32, torch.int32, 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32'), (256, 64, 32, 8), (True, True, True, True, True, True, (False, False), (True, False), (True, False), (False, False), (False, False), (True, False), (False, True), (True, False), (False, True), (True, False), (False, True), (True, False), (True, False)))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/falcontune", line 33, in
sys.exit(load_entry_point('falcontune==0.1.0', 'console_scripts', 'falcontune')())
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/run.py", line 87, in main
args.func(args)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/finetune.py", line 162, in finetune
trainer.train()
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1645, in train
return inner_training_loop(
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 1938, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2759, in training_step
loss = self.compute_loss(model, inputs)
File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 2784, in compute_loss
outputs = model(**inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/operations.py", line 581, in forward
return model_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/utils/operations.py", line 569, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/usr/local/lib/python3.10/dist-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/peft/peft_model.py", line 678, in forward
return self.base_model(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 1078, in forward
transformer_outputs = self.transformer(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 973, in forward
outputs = block(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 641, in forward
attn_outputs = self.self_attention(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/accelerate/hooks.py", line 165, in new_forward
output = old_forward(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/falcon/model.py", line 493, in forward
fused_qkv = self.query_key_value(hidden_states)  # [batch_size, seq_length, 3 x hidden_size]
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(input, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/model/lora.py", line 59, in forward
result = self.quant_instance.forward(x)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/quantlinear.py", line 13, in forward
out = AutogradMatmul.apply(
File "/usr/local/lib/python3.10/dist-packages/torch/cuda/amp/autocast_mode.py", line 103, in decorate_fwd
return fwd(_cast(args, cast_inputs), **_cast(kwargs, cast_inputs))
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/autograd.py", line 11, in forward
output = tu.triton_matmul(x, qweight, scales, qzeros, g_idx, bits, maxq)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/triton_utils.py", line 246, in triton_matmul
matmul_248_kernel[grid](input, qweight, output,
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/custom_autotune.py", line 97, in run
timings = {config: self._bench(*args, config=config, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/custom_autotune.py", line 97, in
timings = {config: self._bench(*args, config=config, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/custom_autotune.py", line 79, in _bench
return triton.testing.do_bench(kernel_call, rep=40, **bench_kwargs)
File "/usr/local/lib/python3.10/dist-packages/triton/testing.py", line 143, in do_bench
fn()
File "/usr/local/lib/python3.10/dist-packages/falcontune-0.1.0-py3.10.egg/falcontune/backend/triton/custom_autotune.py", line 70, in kernel_call
self.fn.run(*args, num_warps=config.num_warps, num_stages=config.num_stages, **current)
File "", line 43, in matmul_248_kernel
ValueError: Pointer argument (at 1) cannot be accessed from Triton (cpu tensor?)

@praveenkanithi Here is the Dockerfile I am using to build the environment.
FROM nvidia/cuda:11.8.0-devel-ubuntu22.04
ENV PYTHONUNBUFFERED=1
ENV TORCH_CUDA_ARCH_LIST="3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX"
SYSTEM
RUN apt-get update --yes --quiet && DEBIAN_FRONTEND=noninteractive apt-get install --yes --quiet --no-install-recommends
software-properties-common
build-essential apt-utils
python3-dev
python3-pip
wget curl vim git ca-certificates kmod
&& rm -rf /var/lib/apt/lists/*
RUN git clone https://github.com/rmihaylov/falcontune.git
RUN mv falcontune /workspace
WORKDIR /workspace
RUN pip install --upgrade pip
COPY requirements_update.txt /workspace/requirements_update.txt
RUN pip install -r requirements_update.txt
RUN python3 setup.py install
RUN pip install ninja
RUN python3 setup_cuda.py install
RUN pip install scipy


Object grasping problem

I'm facing strange problem with grasping objects. On the video bellow You can see what is happening. I'm controlling arm with Moveit.
https://github.com/xArm-Developer/xarm_ros2/assets/52660101/b47db6e4-4f4a-449c-a684-8c004c389708
The problem is that parts of gripper are disconnecting when it is closing. Do you have any tested way to grasp object? What I'm doing wrong?

I see that in xarm_ros repo gazebo_grasp_fix plugin is used to grasp objects. Aren't there any alternatives for ROS 2?

It seems this plugin is not compatible with ROS2 yet, currently we have not found a good solution for this issue. Will keep an eye on it.

I got the same issue with this.
https://github.com/user-attachments/assets/35123a5b-da59-42f6-83e1-4558b692f3b2

@HansRen1024 here is a ROS2 ported version of gazebo_grasp_fix plugin. It will help attach the object to gripper when there are opposing forces detected. However, it seems not to be as stable as the behavior on ROS1 gazebo, the gripper still deforms when the fingers are in contact with the object while the attaching function works fine, not sure which part causes this issue. You may try giving a proper position command to the gripper according to the object size, thus making the deformation as small as possible.


Query does not works with array of search criterias

Array of multiple search criteria is not working, just the first one works.
// Does not works, just the subject criteria is respected -(
$folder->query()->where([['subject' => 'foo'], ['UNSEEN']]);

// Works, all criterias are respected -)
$folder->query()->where(['subject' => 'foo'])->where(['UNSEEN']);

Looking the docs at Custom search criteria, since it uses an array of arrays, I came to the conclusion that multiple search criterias shoud work but, it may be not true. Yet, it would be cool if it works.

Loos like the where method just return the first instead to push it to the array.
So, as a quick solution I did my own function to increment wheres:
public function addWhere(object $query, array $search): object
    {
        foreach ($search as $k => $v) {
            if (is_array($v)) {
                $query = $this->addWhere($query, $v);
            } else if (is_numeric($k)) {
                $query->where($v);
            } else {
                $query->where([$k => $v]);
            }

        }

        return $query;
}

// Usage
$query = $this->addWhere($folder->query(), [['subject' => 'foo'], ['UNSEEN']]);

Maybe this logic can be imported to the where method itself in the future. ;)

Hi @EthraZa ,
what do you think of replacing https://github.com/Webklex/php-imap/blob/23c80eb698e9f9d3c10297a8ff54e1965a58e0fb/src/Query/WhereQuery.php#L126-L146 with this:
public function where($criteria, $value = null) {
    if (is_array($criteria)) {
        foreach ($criteria as $key => $value) {
            if (is_numeric($key)) {
                $this->where($value);
            }
            $this->where($key, $value);
        }
    } else {
        $criteria = $this->validate_criteria($criteria);
        $value = $this->parse_value($value);

        if ($value === null || $value === '') {
            $this->query->push([$criteria]);
        } else {
            $this->query->push([$criteria, $value]);
        }
    }

    return $this;
}

I believe this would work as well and would allow multiple search criteria as long as the provider accepts them:
$query = $folder->query()->where([['subject' => 'foo'], ['UNSEEN']]);

Best regards,

Hi @EthraZa ,
this change is now available via the v3.0.0-alpha release.
Best regards,


Create data models for data retrieved from API.

Context
Organize data fetched from NOAA API. Should also consider how to store or cache data from a request to avoid unnecessary calls to the API.
Acceptance Criteria

[x] Will create model for annual station data: AnnualStationSummary
[x] Model will store data locally to avoid unnecessary calls to API.
[x] Model will load data locally when it is present.

Dev Notes

Add models to dir: models/noaa/
Stores data in dir: data/noaa

Tasks

[x] Create branch station-summary-models off main.
[x] Stub out class WeatherStation that loads from static config file.
[x] Add proof for caching station json data.
[x] Integrate caching in WeatherStation.
[x] Add download action to download annual datasets.
[x] Download Santa Ana station datasets.
[x] Stub out class AnnualStationSummary.
[x] Add method to compute average max temp by day of year.
[x] Add method to get max rain by year.
[x] Add AnnualStationSummary string with avg min temp, max temp, and total rain.
[x] Move data dir to root.
[x] Create PR into main branch.
[ ] Review PR.
[ ] Merge PR.
[ ] Close and delete branch.

Pull Request: https://github.com/klenwell/dr-george/pull/8


Add latin1 transcoding bindings

Fixes #10.
It's a little messy because latin1 has some special casing compared to other formats.

@Nugine, just a friendly ping so this doesn't get lost. No worries if you can't get to it immediately.


An option to toggle if the pin tab icon always shows up on the tab

I pin tabs a lot in Visual Studio 2022 and the same feature on Visual Studio Code could be a bit more appealing. Since the feature may interfere with the experience of a lot of other users (it may just not be for everyone after all) I am requesting maybe an option on the preferences (also because it may make the UI look too "bloated" as the icon would be there).
Another thing that could be added is another option to define if the icon shows up only when hovering a specific tab, any tab (this is to prevent the icon from blocking the file names) or always.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

@bpasero If I have related suggestions regarding this feature should I create separate issues or add the new info in here??

Maybe comment here and then we can see whether a new issue is warranted.

Yes, this would be very useful in a use case such as mine - I don't use tabs and have a limit of 1 opened file per editor group. When I need some more files to be available on ctrl+tab in the same editor group, I pin existing one and add the next one. While doable via a keyboard shortcut, a visible pin icon would surely be of help for us mere mouse users!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Entities not available

Intro
After installing this integration it worked perfectly.  Until one day it did not.  I cannot say what exactly i have done: I updated various parts of HA and i installed other add-ons and integrations.
To solve i re-installed the extension... without any visible improvement.
Bug
Instead of the name and the value of the sensors i get: Entity not available: binary_sensor.rav4_xxxxx
After writing this i tried to create a new card.  And could not find any RAV4 sensor anymore!!
To Reproduce
Create a card with multiple entities, select any from the pulldown list.

Post your logs

Hi there,
Few days ago, after last update to HA 2021.11 this integration stopped working - cannot login anymore with strange error (see below) - no changes have been made on HA apart from its regular update to the latest release:

two more screens - I think it shows where the problem is:

This was just fixed in the latest release, please update 😊

done - thank you ;-)

Good to hear, will keep this open to make sure he is not having another separate problem, though i suspect it is the same. 😊

@PGutti Do you still have this issue?

No!  All variables/sensores now contain a value. Super.
Regards
Peter

El 23 nov 2021, a las 18:44, Simon Hansen @.***> escribió:
@PGutti https://github.com/PGutti Do you still have this issue?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/DurgNomis-drol/ha_toyota/issues/83#issuecomment-976923029, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJRSHW6LXITTPSSPC3J5LHLUNPHHRANCNFSM5H7TAMSA.


Two instances for AWS document

It may be better to use two instances in the AWS document, so tests can be done in meaningful way.

IPERF example has been explained with two AWS VMs here


Readme

#resolves #1

Changes Unknown when pulling 83ab74ecb9bd7022c947ea26da6ba3e72e7d4d0f on readme into ** on master**.

Changes Unknown when pulling 83ab74ecb9bd7022c947ea26da6ba3e72e7d4d0f on readme into ** on master**.

:+1:


stopping lxd also stops all containers

When stopping lxd, all containers are also stopped.
This looks to me like unwanted behaviour, though I can understand if it is needed.
This makes upgrading lxd hard and needs to be planned during maintenance hours.

This has to do with distro packaging and some limitations of existing init systems like systemd.
LXD itself will not stop containers on exit, it will only do so when it receives SIGPWR which is what the systemd unit in Ubuntu does.
Ideally we'd have systemd pass SIGPWR during host shutdown and do a regular SIGTERM in other cases, but this doesn't appear to be possible, at least not in any document kind of way...
Closing this LXD issue since LXD itself supports both and does the right thing by default, it's only limitations of the init system which prevents from using this properly :(

You could file a bug against lxc/lxd-pkg-ubuntu where the systemd unit lives. As I said, I'm not aware of any way to distinguish from "systemctl stop lxd" or "systemctl restart lxd" and the same action being triggered at host shutdown, but having an issue for it would be worthwhile nonetheless.


Barcode detector polyfill can not detect code_128 format on Safari.

I used this library to recognize qr_code and barcode with format code_128.
It works for qr_code but failed to recognize barcode with format code_128 it on Safari.
this is my code:
// BarcodeReader.tsx
import { BarcodeDetectorPolyfill } from "@undecaf/barcode-detector-polyfill";
import { useEffect } from "react";

const BarcodeReader = () => {
      function readBarcode() {
            if (!window["BarcodeDetector"]) {
	            window["BarcodeDetector"] = BarcodeDetectorPolyfill;
            }
            navigator
            .mediaDevices
	    .getUserMedia({
		    video: true,
		    audio: false,
	    })
	    .then((stream) => {
            if ("BarcodeDetector" in window) {
	          const videoEl = document.querySelector("video") as HTMLVideoElement;
	          videoEl.srcObject = stream;
	          videoEl.play();

	          const barcodeDetector = new window.BarcodeDetector({
		          formats: ["code_128"],
	          });
                   const barcodes = await barcodeDetector.detect(videoEl);
	           console.log(barcodes); // print empty array for code_128 but work for qr_code on Safari!
	    })
	    .catch(console.log);
       }
       useEffect(() => {
	    readBarcode();
	}, []);
       return (
          <div>
              <video height={220} width={300} />
          </div>
	);
}
export default BarcodeReader;

Is there a way to detect code_128 barcode on Safari?

I suspect that the video height and width settings might influence the videoHeight and videoWidth attributes (i.e. the intrinsic resolution) of the video. If that is the case then the resolution might be too low to detect Code-128 but still sufficient to detect QR codes.
Could you therefore please also log videoEl.videoHeight and videoEl.videoWidth to see whether that is true?
Another test for my hypothesis would be to test whether other bar codes such as ean_13 also cannot be detected.

No activity here for the last four weeks, so closing. Please reopen if necessary.


[Bug]: Build Scucess,But Unable to call the library from anywhere else

Describe the issue
Unable to call the library from anywhere else
[cmake] CMake Error at /Users/baiyue/mino/cpp/abseil-cpp/build/abslConfig.cmake:32 (include):
[cmake]   include could not find requested file:
[cmake]
[cmake]     /Users/baiyue/mino/cpp/abseil-cpp/build/abslTargets.cmake

Steps to reproduce the problem
git clone https://github.com/abseil/abseil-cpp.git --recursive
cd abseil-cpp
git submodule update --init --recursive
mkdir build && cd build
cmake ..
make

What version of Abseil are you using?
latest
What operating system and version are you using?
lates mac
What compiler and version are you using?
clang latest
What build system are you using?
mac os
Additional context
No response

I don't know what "Build Scucess,But Unable to call the library from anywhere else" means, and this report lacks complete reproduction instructions.


Merge 'GO:0016180  scnRNA processing' into 'GO:0070918 primary sncRNA processing'

[x] The transcription Working Group proposes to merge 'GO:0016180  scnRNA processing' into 'GO:0070918 primary sncRNA processing'. The reason for merging is that only miRNAs are known to have two defined intermediate products (pre and pri-mRNA) that is useful to distinguish because pri-miRNA processing  takes place in the nucleus while the pre-miRNA processing takes place in the cytosol.

[x] Also, the term label will be changed to 'small regulatory ncRNA maturation'
[ ] A logical definition will be made with SO:0000370
[x] The term will be made 'do not annotate (it should be possible to specify a more precise substrate: miRNA, siRNA, piRNA, etc)

Move under 'GO:0070918  'small regulatory ncRNA maturation':

[ ] GO:0043628 ncRNA 3'-end processing + change label to 'small regulatory ncRNA 3'-end processing'
[ ] GO:0031054 'pre-miRNA processing'
[ ] GO:0031053 'primary miRNA processing'

Change label: 'GO:0140891 tRNA-derived small RNA processing' to 'tRNA-derived regulatory ncRNA processing', to match definition (A process leading to the generation of a functional small regulatory non-coding RNA derived from a tRNA)


ConfigProto

I use your exellent file but I cant run it.
I Run :
python test.py --input_images /path/to/image_0 /path/to/image_1 --resume /path/to/model.ckpt
After running above script, utilize GPU-id is asked, (-1:CPU). You can use other learning configs (like --n_epoch or --batch_size) see all arguments in train.py, regards.  but this error is gone:
AttributeError: module 'tensorflow' has no attribute 'ConfigProto'

You must be using 2.x version of tensorflow and 'ConfigProto' is deprecated in this version. In order to make this work, write this 'tf.compat.v1.ConfigProto' instead of 'tf.ConfigProto'.


CPP-103 Compression support

Thanks again for the PR, but htere hasn't been any follow up on this so I'm going to close it for now. We can reopen it in another PR in the future.


State 270

Three States.

Not Scanned:

Scanned and not yet approved:

Scanned and approved:

Possible Action Item
Should the "Refresh QR-Code" button still be there? I think it was only relevant when we had a state of denial.... Which we don't.

@popkinj  there is an issue with the redirect that seems to have been caused by another commit (it will send you to a malformed URL). I have a change ready in another branch, disregard that when cleaning-up the requested changes 🙂


Detect when a student got absorbed by the blackhole

Hello,
I have opened this pull request to add a new feature to your project. With these changes, the extension can now properly handle students who have been blackholed, ensuring that no date is displayed in the web browser for these cases.
Thank you for considering this enhancement. I look forward to your feedback.
Best regards,
Lirone.

LGTM!


Component: Per Page Feedback (informs DS)

The per page feedback widget used on covid19.ca.gov is already a web component that accepts the url of the site being used and submits data to a scalable Azure CosmosDB via serverless endpoint.
This component has not yet been used on any other sites although it should work if some style issues are ironed out.
In one of the wireframe review sessions with the cannabis team they mentioned that this component should appear lower in the footer. It is placed at the desired location in the latest version of the wireframes.
This component was integrated into the WordPress site theme by Danny Guzman using a script include hack that added the web component js and then inserted the custom element into the footer. This was done without modifying the theme but allowed it to appear on all pages.
We need to modify the web component as required to make it more easily reusable and then work with the WordPress theme team again in the production site to integrate it into the footer.

Seems like this needs to be scoped out further and broken down.

Ticket #5 contains the prerequisites to make the front and backend of the existing component usable on any site: https://github.com/cagov/design-system/issues/5
This ticket will track the integration of the latest version of the feedback component on the cannabis site.
When we include components that are fully encpasulated so can be added with a custom element and a script tag we should have a standard include method that allows versioning.
This ticket should cover the following tasks:

[ ] Create offsite component inclusion method - possibly ability to retrieve versioned js from CDN
[ ] Document inclusion method
[ ] Integrate this element into cannabis production site, if necessary move into footer with no dependency client side script
[ ] Scripts for multiple components should be combined and deferred when appropriate

Here is design feedback. You can see detailed design in the "Components and Styles" page on Figma.

Add 5px corner radius to container
Change background color to Cannabis secondary color (#2F4C2C)
Move Submit button below form so that it does resize when form grows [see attached screenshot showing the problem]
Make the top line of text for all states appear in the same position to avoid jarring movement. ("Did you find what you were looking for?" / "Great! What were you looking for today?" / "Thank you for your comments!")
Change hover state color to #d5dbd5 (80% tint of default color) and underline text on hover

style fixes applied, packages published, bundles recreated, remaining css variable issue will require one more package bump

Remaining style fixes:

Remove bold treatment from text
Can we remove the color animation behavior when you hover on the buttons?

Last issue where the buttons wrapped weird on larger phones was fixed


Remote code execution vulnerability due to arbitrary file creation.

We find the stored part of this file by searching the key functions.

View calls in this section

Network truncation of parameter transfer in this part.

Try to modify to JSP webshell.

The error reported here is an error occurred during XML parsing, but the file has been written into the server.

Find this directory.

Of course, this directory can't access JSP. Try to cross directory with relative path.

Successfully cross directory and get webshell.

the key functions are in ureport2-core/src/main/java/com/bstek/ureport/provider/report/ReportProvider.java or ureport2-core/src/main/java/com/bstek/ureport/provider/report/classpath/ClasspathReportProvider.java or ureport2-core/src/main/java/com/bstek/ureport/provider/report/file/FileReportProvider.java ?  we want to reproduce this vul as well


createServer api Can data configuration be supported

Clear and concise description of the problem
The createserver API currently supports parsing vite config. TS or vite config. js
Suggested solution
Add configuration item data directly
Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[ ] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

could you specify Add configuration item data directly more?

At present, it seems that configuration item data parsing is not supported. Only configuration files can be parsed

It is hoped that vite, like webpack, can import user configuration with user customization, and vite implements automatic file parsing internally, which makes it extremely inconvenient to merge external configuration files when developing the internal cli of the company

InlineConfig extends UserConfig, can you call createServer({...option1, ...option2, ...option3}) ?

yes

It's still not clear to me what this feature this issue is requesting. I'm assuming you want to call createServer without Vite loading the configFile by default? configFile exists in the inline config only so we can turn it off according to the docs. For config merging, Vite also exposes mergeConfig() so it should be easy to resolve a Vite config programmatically. There's also loadConfigFromFile() if you want to load non-js config files. I believe Vite has the tools to achieve this currently, so I'll close this for now, but feel free to point out if I'm mistaken somewhere. We can re-open this and further discuss about it.


ISSUE: socket failure on profile creation

After user creates a profile (create-profile not signup), socket does not open. This is fixed by simple refresh manually in the terminal, so maybe all I have to do is put in a scroll to refresh feature...
or find out why this happening

Fixed as of commit #92


Remove serde crate

Remove serde from core-graphics-rs. The serialization function moves to webrender.
https://github.com/servo/servo/issues/15607

This change is

@bors-servo: r+
Thanks!

:pushpin: Commit d503560 has been approved by jdm

:hourglass: Testing commit d503560bb1cea03753146f1ce14dfa41be2ce27d with merge aa1a2cc65dc0e26a265350c5cfb28159ca32a264...

:sunny: Test successful - status-travis
Approved by: jdm
Pushing aa1a2cc65dc0e26a265350c5cfb28159ca32a264 to master...


how to use pretrained bert model in fastbert?

I have pretrained bert model in pytorch with config and vocab file. I want to use it for multilabel classification task. My question is how to use it?

I have the same question...
Thanks firstly for giving a method.

Look at the predict_batch method.

There's a kind example in new-toxic-multilabel.ipynb
Here, you will have to change the model_name option to Bert. In addition, I changed several lines to adjust to my dataset and to print metrics to tensorboard.


Regression: white objects suddenly render yellow or blue

Description
Regression: white objects suddenly render yellow or blue.
Some parts of models that don't use textures have started to render yellow or blue.

The bike parts above and the cubes around the shark are untextured and should render white; they did when I uploaded those models.
Live Demo
https://sleet-kookaburra.glitch.me/
Browser Affected

[x] Chrome
[ ] Edge
[ ] Firefox
[ ] Helios
[ ] IE
[ ] Safari

OS

[ ] Android
[ ] iOS
[ ] Linux
[ ] MacOS
[x] Windows

Versions

model-viewer: latest

@soraryu I'm not able to reproduce this problem in your demo.
I have tried running the test case in Chrome 79 on Windows 10 via BrowserStack:

Can you offer any additional details about your setup? GPU? Windows / Chrome versions etc?

Sorry, ticked the wrong OS. This happens on Android 10, Xiaomi Mi 9, latest Chrome.

Reproduction attempts
Here are a few shots from phones on BrowserStack with the same chipset (Snapdragon 855) / browser (Chrome 79) combo:

Samsung Galaxy S10
Oppo OnePlus 7

So, Samsung seems fine, but there definitely seems to be funky with the OnePlus. To my eyes, the lighting on those squares looks darkened in that render.
In the JS console, there isn't a whole lot of actionable info, although it does look like there are some non-power-of-two textures in the mix:

For the OnePlus, I also saw this happening on the bike model, which looks pretty bad:

On both of those phones, these were the warnings I got in the console for the bike model, but only the OnePlus had major rendering issues:

Stepping back for a bit, I tried dropping the bicycle model into the Scene Viewer preview tool (https://vr.google.com/scene-viewer-preview) and noticed right away that it has hundreds of errors (as far as the preview tool is concerned) and also is reportedly very complex.
Analysis
It seems like there is definitely a discoloration problem that we can reproduce. We can probably use the Oppo OnePlus 7 emulator on BrowserStack to isolate the problem and look into it. cc @elalish
The yellow discoloration remains illusive. I guess we will need to get ahold of a Xiaomi 9 to verify the issue. It's possible that a fix for the Oppo will help the Xiaomi, but there is really no way to be sure.
As far as the bike is concerned, I'm worried that even if there is a <model-viewer> bug affecting it, it would be very difficult to tweeze it out of the nest of other problems in that model. @soraryu would you be willing to look into some of the errors reported by the Scene Viewer preview tool and let us know if you can reduce or eliminate them?

Yep, I know the bike has lots of warnings, but until the 0.8 update has rendered fine in both model-viewer and SceneViewer on any device I tried (and that's a lot, ~20 different mobiles). Note that I have another issue open here (that you forwarded to the scene viewer team iirc) about the web preview being incorrect and models working fine on actual devices.
For reproduction, the shark is a much lighter model. I have an assumption what the issue is (affects all objects that don't have a texture assigned). I'll try to make a very simple repro.

Thanks @soraryu , and acknowledged that there are outstanding threads of discussion related to the bike.
I ordered us a Xiaomi Mi 9 for local testing, but we probably won't be able to do anything until after the new year.

So this is definitely a regression between 0.7.2 and 0.8.0.
I uploaded a single white cube and it shows the same behaviour. I then made an array of cubes of varying smoothness / metallicness and the result is that all things that have
a) no texture
b) metallic 0
are yellow.
You can switch model-viewer versions here (0.7.2/latest dist/master):
https://glitch.com/edit/#!/sleet-kookaburra

@soraryu Yes, I'd guess this is a regression we picked up from bumping our version of three.js. I'll look into it. I still need to find a device that repros it.

Two more tests with external three.js look ok (of course I'm not sure that's the same version you're using):

Ran into this issue on my Pixel 3 when looking at a demo from another unrelated issue (#950): https://model-viewer-dynurl-bug.glitch.me/

I built a minimal repro of this on three: https://github.com/mrdoob/three.js/issues/18315


Hey there! So you want to contribute to FirebaseUI? Before you file this pull request, follow these steps:

Read the contribution guidelines.
If this has been discussed in an issue, make sure to mention the issue number here.  If not, go file an issue about this to make sure this is a desirable change.
If this is a new feature please co-ordinate with someone on FirebaseUI-Android to make sure that we can implement this on both platforms and maintain feature parity.

[_****_](

[ ] @ #url

)


Error launching Flutter App - Permission denied @ dir_s_mkdir

This issue is very serious and has been reported in CocoaPods repo.
https://github.com/CocoaPods/CocoaPods/issues/7927

When I downloaded flutter I saved it in the document location.
~/Documents/development/flutter

I believe this is tracked as https://github.com/flutter/flutter/issues/19338 ?

That CocoaPods issue is closed with a comment "it does seem to be related to something that Flutter is doing" but I don't know what that could be.  This is a duplicate of https://github.com/flutter/flutter/issues/19338, which is closed because I couldn't get more information and the crash location moved
Are you still this or a similar issue on CocoaPods >1.6.0 since that's the new minimum, and CocoaPods no longer will crash in the same place?
See https://github.com/flutter/flutter/issues/19338#issuecomment-503776123

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.


shim: add wait flag

Add a struct used to contain both BlockRef and the wait flag, to make it reusable for both block and blob queries (and all other possible new queries).
The struct should be used with clap(flatten) to make all the fields appear as options in the query command.
TODOs:

[ ] Better naming
[ ] Better logging

closes #171

Current dependencies on/for this PR:

main

PR #196   👈

This stack of pull requests is managed by Graphite.


Documentation

Minor. This entry appears twice

New doc solves the issue. SEE https://itisfoundation.github.io/osparc-simcore-python-client/#/


🔧 Support Dict & List Nested Field

Hi, this is PR for https://github.com/Frankkkkk/pykorm/issues/16
Plan to support native Open API structure integration, This will support the reference of Kubernetes native Model.

Hi Frank, thank you for the code review.
This helped me a lot,  I submitted a new code, in response to your comment.

Hi @AnyISalIn Looks quite good to me !
There's just 2 minor changes that I'd like (newline + kube fixture in test_model_setattr) but if you're okay with it I can edit your commit and then merge-it ?
Let me know,
Cheers

@Frankkkkk No problem!

I squashed your commits + added another small to support the failing test. Should be good :+1:


Answers?

Hey, thanks for this impressive work.
I am trying to run the benchmark locally and seems like the dataset doesn't have the correct answer?

Hi @falmanna,
Thanks for reaching out.
Thanks for reaching out! We left the 'answer' fields empty to prevent unauthorized training on our data. Yet, we can give you the accuracy of your generated answers by your model. To evaluate your model:

Use our evaluation script.
Replace the inference code with your model's code.
Run the script to generate outputs.
Submit the outputs to us, and we'll provide the evaluation results/accuracy.

This ensures the benchmark's integrity while allowing you to assess your model's performance. Let me know if you have any other questions!

Thanks @ahmedheakl
I actually switched the implementation to litellm lib and was trying with few VLMs locally.
I will reachout over LinkedIn for a more detailed inquiry.
Best.


docs(guides) update tree shaking guide, make syntax consistent

Update tree shaking guide
Update code to have consistent single quote in js examples

Thanks!