Model Adapter of a model within a model

Using Hive in Flutter, I have to generate a Model Adapater by annotating my Model:
import 'package:connectivity/connectivity.dart';
import 'package:flutter/material.dart';
import 'package:hive/hive.dart';

part 'internet_connection.g.dart';

@HiveType(typeId: 3)
class InternetConnection {
  InternetConnection({
    @required this.status,
  });

  @HiveField(0)
  ConnectivityResult status;
}

This fails and throws the Exception

[ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: HiveError: Cannot write, unknown type: ConnectivityResult. Did you forget to register an adapter?

From what I can gather, I will need to do additional annotation to have the Model ConnectivityResult be seen by Hive. What additional logic would I need to provide to get around this?
Version

Platform: iOS, Android
Flutter version: 1.23.0-18.1.pre • channel beta
Hive version: ^1.4.4+1

Did you register your adapters after Hive.initFlutter()?
if (!Hive.isAdapterRegistered(3)) {
  Hive.registerAdapter<InternetConnection>(
    InternetConnectionAdapter(),
  );
}

Run this for every Model you created before opening boxes.

By default HiveObjects only support dart types, like String, int, List, Map. If you want to save a different type, it will need to also be registered as an adapter, even enum needs an adapter.
And if I'm not mistaking, to save an object inside another object, the first object would need to be already in a box.

@mbaumgartenbr While it is possible for me to add Hive Adapter logic to a Model that I have control of (say I have a Model that has properties of another Model within my codebase that I've written) the core concept of the issue at hand is that the ConnectivityResult Model/Enum is a pub package from the Connectivity package. So I can't just edit their Model since it's outside of my control.
How do I go about accomplishing such a task?

Oh, sorry, i didn't see that it was a package, the way I would approach it is by converting the enum to String back and forth.
String enumToString(ConnectivityResult value) {
  return value.toString().split('.').last;
}

ConnectivityResult stringToEnum(String value) {
  return <String, ConnectivityResult>{
    'mobile': ConnectivityResult.mobile,
    'wifi': ConnectivityResult.wifi,
    'none': ConnectivityResult.none,
  }[value];
  // [ ... ]
}

You could also use the same approach but storing it as int using the index of each enum.

Thanks @mbaumgartenbr. To clarify, your provided code logic would go inside the Hive Adapter I assume?

Not really, it could live wherever you want, they could be private to the model file, the toString could be an extension on ConnectivityResult, they could be getters of your model. Whatever suits you :)

@mbaumgartenbr Thanks for the continued help. I'm trying to decipher your last message. I've updated my Model as follows:
import 'package:connectivity/connectivity.dart';
import 'package:flutter/material.dart';
import 'package:hive/hive.dart';

part 'internet_connection.g.dart';

@HiveType(typeId: 3)
class InternetConnection {
  InternetConnection({
    @required this.status,
  });

  @HiveField(0)
  ConnectivityResult status;
}

dynamic get status => status.EnumToString;

extension EnumToString on ConnectivityResult {
  String get status {
    return 'test'; //just to test if this works or not!
  }
}
String enumToString(ConnectivityResult value) {
  return value.toString().split('.').last;
}

ConnectivityResult stringToEnum(String value) {
  return <String, ConnectivityResult>{
    'mobile': ConnectivityResult.mobile,
    'wifi': ConnectivityResult.wifi,
    'none': ConnectivityResult.none,
  }[value];
}

I then run flutter pub run build_runner build --delete-conflicting-outputs. I still receive the same result when I try to save to the box:

HiveError: Cannot write, unknown type: ConnectivityResult. Did you forget to register an adapter?

While some of this is over my head, trying to record how to make this logic work would also help developers who stumble on this same issue. I also have other Models using package Models that I have to refactor this to work with Hive. If you are able to let me know what I'm missing in my updated Model that would be greatly appreciated.

Oh, sorry if I can't make myself clear, english is not my mother language.
You cannot store an enum inside a HiveObject if that enum doesn't have  a TypeAdapter.
To overcome that, make your model like so:
@HiveType(typeId: 3)
class InternetConnection extends HiveObject {
  InternetConnection({@required ConnectivityResult status}) {
    this.status = status.toString().split('.').last;
  }

  @HiveField(0)
  String status;

  ConnectivityResult get statusAsEnum {
    return <String, ConnectivityResult>{
      'mobile': ConnectivityResult.mobile,
      'wifi': ConnectivityResult.wifi,
      'none': ConnectivityResult.none,
    }[status];
  }
}

If you would want to move the logic for the enumToString convertion off of your model, use an extension:
extension EnumX on Enum {
  String asString() => return this.toString().split('.').last;
}

Then pass to your constructor:
final conn =InternetConnection(status: ConnectivityResult.mobile.asString());

Your updated Model example provides the following Adapter:
// GENERATED CODE - DO NOT MODIFY BY HAND

part of 'internet_connection.dart';

// **************************************************************************
// TypeAdapterGenerator
// **************************************************************************

class InternetConnectionAdapter extends TypeAdapter<InternetConnection> {
  @override
  final int typeId = 3;

  @override
  InternetConnection read(BinaryReader reader) {
    final numOfFields = reader.readByte();
    final fields = <int, dynamic>{
      for (int i = 0; i < numOfFields; i++) reader.readByte(): reader.read(),
    };
    return InternetConnection(
      status: fields[0] as ConnectivityResult,
    );
  }

  @override
  void write(BinaryWriter writer, InternetConnection obj) {
    writer
      ..writeByte(1)
      ..writeByte(0)
      ..write(obj.status);
  }

  @override
  int get hashCode => typeId.hashCode;

  @override
  bool operator ==(Object other) =>
      identical(this, other) ||
      other is InternetConnectionAdapter &&
          runtimeType == other.runtimeType &&
          typeId == other.typeId;
}

When I try to save to a box with the value of the InternetConnection model, it throws a new Exception:

[ERROR:flutter/lib/ui/ui_dart_state.cc(177)] Unhandled Exception: type 'String' is not a subtype of type 'ConnectivityResult' in type cast
E/flutter ( 7333): #0      InternetConnectionAdapter.read

pointing explicitly to status: fields[0] as ConnectivityResult, in the InternetConnectionAdapter read method.

Oops, my mistake! I thought that the constructor wouldn't matter. Just change to the second approach:
// [ ... ]
InternetConnection({@required this.status});
// [ ... ]

Then make sure to do the enum conversion when you create the class:
final conn = InternetConnection(
  status: ConnectivityResult.wifi.asString(),
)


[question] how to hide button link info in lower left

I am hoping someone may know the answer to this. not truly an issue just a question.
when I hover over the buttons they show link info in lower left.

javascript:void('source')
etc

I have attempted the following config flags but no luck.

Is there a config option to hide link info?

linkShowAdvancedTab: false,
linkShowTargetTab: false,
linkJavaScriptLinksAllowed: false,
showDisplayTextForElement: false,

Please ask ckeditor questions in ckeditor forum. I found similar question with answer here
                this.ckeConfig = {
                    height: 500,
                    language: "en",
                    allowedContent: true,
                    linkShowAdvancedTab: false,
                    linkShowTargetTab: false,
                    linkJavaScriptLinksAllowed: false,
                    showDisplayTextForElement: false,
                    extraPlugins: "divarea"
                };

This solution doesn't work in Chrome.

For anyone who comes across this looking for the same solution; here is mine for chrome.
add this to your CKEDITOR config.
on: {
        'contentDom': () => {
          var element = document.getElementsByClassName('cke_button')
          for (var i = 0; i < element.length; i++) {
            element[i].addEventListener('mouseover', (e)=>{
              e['toElement'].removeAttribute('href')
            })
          }
        }
      },


I'm still new to this, so please forgive any naivete

So I had this up and running, everything was fine.
Things happened, and I hadn't used it in awhile.
Now, when I try to use it, Alexa tells me that there's a problem with the requested skills response.
I've dug in and tried to figure out where the error is, but am stumped.
RealDNS is operating fine.
iTerm2 shows server.js is running fine.
if i run the localhost:5005/... commands in a browser, it yields results and does what I ask.
When I run tests in the Amazon Alexa skills section, it tells me "The remote endpoint could not be called, or the response it returned was invalid."
Did something change that I need to account for?

Can you hit yourip:5005?

Huh. No I can't; getting a "this site can't be reached error."  Odd; can't think of what changed, 'cause I know I could do this back when it was working...

@validoption Most likely you have a new external ip or are not using static ip's internally.

I've reviewed my RealDNS and my Airport TimeCapsule settings, and nothing has changed from how I had it set up.  All IPs are the same numbers (I wrote them down when I first set this all up). Yet I can't reach my DDNS IP, my LAN IP, or the Private IP specified in the Network>Port Settings> Personal Web Sharing section.  the only thing that works with :5005/zones is localhost in the browser.
Clearly, I'm missing something staring me in the face.  Is this a case of delete everything and start over?

Have your firewall settings changed?

Just double-checked on your question, and no they haven't.

So I went into Network>Port Settings>Personal Web Sharing entry I previously created and changed the Private IP to align with my Local IP.  Now, when I try myddnsIP:5005/zones, it yields the same data that localhost or Local IP did shortly ago.  However, there's still a disconnect somewhere in both the testing of the Alexa skill in developer.amazon and her response that there is a problem with the requested skill response.

closing due to inactivity; reopen if help still needed


ETag is empty

Hi, I'm using State store (Azure Cosmos) in my application. When I call GetStateEntry, ETag is empty.
var state = await daprClient.GetStateEntryAsync(StoreName, vote.articleid)
state.ETag is empty
I need to avoid optimistic concurency. I need to get exception in case of another process updated entity.
I'm using this commands to save data:
var options = new StateOptions() {Concurrency = ConcurrencyMode.FirstWrite};
await state.SaveAsync(options);
RELEASE NOTE: rc1

I'm using this commands to save data:

var options = new StateOptions() {Concurrency = ConcurrencyMode.FirstWrite};
await state.SaveAsync(options);

I realize that you mentioned the SDK didn't include an ETag - however when you do have the ETag you'll want use TrySaveAsync to ensure that it is passed to the state store.

Thanks, I tried to use for saving data:
var options = new StateOptions() {Concurrency = ConcurrencyMode.FirstWrite};
await state.TrySaveAsync(options);

But the problem remain. Data are overriden.

You might want to use the GetStateAndETagAsync API to get the state entry and the etag and as @rynowak suggested, TrySaveStateAsync API to save.

@vinayada1 thanks for that. Can you help me how to use it ? state.TrySaveAsync doesn't exists
var state = await daprClient.GetStateAndETagAsync<MyItem>(StoreName, key);
...
var options = new StateOptions() {Concurrency = ConcurrencyMode.FirstWrite};
await state.TrySaveAsync(options);

@jjindrich: Please take a look at the sample in the linked PR. I will check this in for future reference

Excellent, many thanks !


Ascensor pages can't be printed

Seems that using Ascensor makes the page unprintable. Safari shows a blank print preview and Chrome give the message "Print preview failed". I don't have a Windows machine to test IE.
I'm thinking this is something to do with the positioning?

did that resolve your issue?


Adding integration test retries

🔧 Changes
There have been occasional, intermittent testing failures in CI that are resolved by retrying the testing stage. This PR adds an automatic retry to the integration tests to automate the process of retrying failed tests.
📚 References
Commander CLI Docs: retries
📝 Checklist

[x] All new/changed/fixed functionality is covered by tests (or N/A)
[x] I have added documentation for all new/changed functionality (or N/A)

Codecov Report
Patch and project coverage have no change.

Comparison is base (0c7896b) 60.15% compared to head (40f7fbe) 60.15%.

:exclamation: Current head 40f7fbe differs from pull request most recent head c8367f6. Consider uploading reports for the commit c8367f6 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #698   +/-   ##
=======================================
  Coverage   60.15%   60.15%
=======================================
  Files          89       89
  Lines       11494    11494
=======================================
  Hits         6914     6914
  Misses       4087     4087
  Partials      493      493

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[BUG] Backing Image Data Inconsistency if it's Exported from a Backing Image Backed Volume

Describe the bug (🐛 if you encounter this issue)
The volume encounters a data integrity issue, if it's from a backing image, and such backing image is exported by a volume backed by another backing image.
To Reproduce

Preparing a Harvester v1.2.0-head cluster (for using volume as an OS root disk)
Uploading Ubuntu jammy cloud image (latest version), img1
Creating volume vol1 from img1
Creating a VM with vol1 as the root disk, and writing some data
Issuing sync on filesystem, e.q. $ sudo sync
Stopping VM, and waiting until vol1 is completely detached
Exporting vol1 as img2
Creating  vol2 from img2
Comparing data content between vol1 and vol2, the data is inconsistent
Running fsck on vol2, fsck will report filesystem errors and require data repair

Expected behavior
The data of vol1 and vol2 should be consistent.
Support bundle for troubleshooting
supportbundle_066aa8ed-a12b-4ccc-971c-59404ee44cd4_2023-10-16T07-46-44Z.zip
Environment

Longhorn version: v1.4.3
Harvester version: v1.2.0-head
Number of Longhorn volumes in the cluster: 2
Impacted Longhorn resources:

Volume names: pvc-b8bd9013-e8e9-449d-9c25-6c6c0824d0b3, pvc-de64dd95-b0c1-4ce9-864a-88d38e06d9d3

Additional context

If vol1 is not backed by backing image, there is no data inconsistency found between vol1 and vol2
A single node Harvester cluster should be enough to reproduce this issue

cc @shuo-wu

Some observations with the following steps:

Preparing a Harvester v1.2.0-head single node cluster (for using volume as an OS root disk)
Uploading Ubuntu jammy cloud image (latest version), img1 with replica number is 1
Checking data usage of img1 (mapped to bi default-image-s9rfj), which is 640MB
Creating volume os-vol (mapped to volume pvc-c3b1a434-86ee-453b-afc7-a52fe113c059) from img1 with claim size 10GB
Creating a VM with  os-vol as the root disk
Stopping VM, and waiting until os-vol is completely detached
Exporting os-vol as os-vol.img (mapped to bi default-image-j8ghr) with replica number is 1
Checking replica data usage of os-vol, which is 523MB
Checking data usage of os-vol.img, which is 2507MB

For the data usage, os-vol.img(2507MB) is much greater than the summation of img1(640MB) and os-vol(523MB)
The os-vol.img's data usage shouldn't be greater than the summation of img1's and os-vol's ?

Creating volume copy-vol from os-vol.img with claim size 10GB
Comparing data content between os-vol and copy-vol with $cmp -l os-vol copy-vol
From the compare result cmp.log, there are several continuous byte ranges overwritten with zero in the synthetic volume, copy-vol

Thanks

Note: From Longhorn side, we can't reproduce the issue
Longhorn side reproduce steps

Create a BackingImageA
Create a Volume with the BackingImageA
Write some data to the Volume
Get the checksum of the Volume
Export the BackingImageB from the Volume
Get the checksum of the BackingImageB
Checksum is the same

@WebberHuang1118 Please work with @ChanYiLin to reproduce the issue.

From the above reproduce steps https://github.com/longhorn/longhorn/issues/6899#issuecomment-1782369530,
After step7 in https://github.com/longhorn/longhorn/issues/6899#issuecomment-1782369530,
since the VM is stopped and the volume is detached, IMHO, this issue could be reduced to "if the exported backing image content is the same as the source volume ?"
As I described in step9 https://github.com/longhorn/longhorn/issues/6899#issuecomment-1782369530,
the original backing image size is 640MB and the volume is 523MB, however, the synthesized backing image's grows to 2507MB (> (640MB + 523MB)).
If we create a new volume and compare its content to the source volume, the content is inconsistent. However, if the source volume is not a backing image backed volume, there is no such data inconsistency.

I found a method to reproduce a similar case without Harvester-specific operations (no VM operation, only creating PVC from backing image and I/O access)
Detailed steps:

Uploading Ubuntu jammy cloud image (latest version), img1
Creating PVC os-vol from img1 with claim size 10GB and 1 replica number
Applying the following yaml to mount os-vol as block mode in the podapiVersion: v1
kind: Pod
metadata:
  name: block-volume-test
  namespace: default
spec:
  containers:
  - name: volume-test
    image: ubuntu
    imagePullPolicy: IfNotPresent
    securityContext:
      privileged: true
    command: ["/bin/sleep"]
    args: ["3600"]
    volumeDevices:
    - devicePath: /dev/os-vol
      name: os-vol
  volumes:
    - name: os-vol
      persistentVolumeClaim:
        claimName: os-vol

Issuing filesystem expand and I/O on os-vol

Enter pod block-volume-test
Install fdisk

$ apt update; apt install -y fdisk

Expanding filesystem on os-vol

Check device (/dev/sda for exmaple) mapped to os-vol by device number
$ fdisk /dev/sda

d 1 n 1 default default No w

$ resize2fs /dev/sda1

Mounting /dev/sda1 and writing I/O into it

$ mkdir /data; $ mount -t ext4 /dev/sda1 /data
$ dd if=/dev/zero of=/data/file bs=1M count=2048

Flushing data to storage for os-vol

$ umount /data

Delete pod block-volume-test, and waiting until os-vol's LH volume detach

Exporting os-vol as os-vol.img

I found a method to reproduce a similar case without Harvester-specific operations (no VM operation, only creating PVC from backing image and I/O access)
Detailed steps:

Uploading Ubuntu jammy cloud image (latest version), img1
Creating PVC os-vol from img1 with claim size 10GB and 1 replica number
Applying the following yaml to mount os-vol as block mode in the podapiVersion: v1
kind: Pod
metadata:
  name: block-volume-test
  namespace: default
spec:
  containers:
  - name: volume-test
    image: ubuntu
    imagePullPolicy: IfNotPresent
    securityContext:
      privileged: true
    command: ["/bin/sleep"]
    args: ["3600"]
    volumeDevices:
    - devicePath: /dev/os-vol
      name: os-vol
  volumes:
    - name: os-vol
      persistentVolumeClaim:
        claimName: os-vol

Issuing filesystem expand and I/O on os-vol

Enter pod block-volume-test
Install fdisk

$ apt update; apt install -y fdisk

Expanding filesystem on os-vol

Check device (/dev/sda for exmaple) mapped to os-vol by device number
$ fdisk /dev/sda

d 1 n 1 default default No w

$ resize2fs /dev/sda1

Mounting /dev/sda1 and writing I/O into it

$ mkdir /data; $ mount -t ext4 /dev/sda1 /data
$ dd if=/dev/zero of=/data/file bs=1M count=2048

Flushing data to storage for os-vol

$ umount /data

Delete pod block-volume-test, and waiting until os-vol's LH volume detach

Exporting os-vol as image os-vol.img
Creating PVC copy-vol from os-vol.img with claim size 10GB
Comparing data content between os-vol and copy-vol with $cmp -l os-vol copy-vol

The data content is different between the two PVC, os-vol and copy-vol

Finding filesystem error on copy-vol with $e2fsck -c /dev/copy-vol

There is no filesystem error on os-vol

I can reproduce the issue, will look into how we export BackingImage

BackingImage Export Mechanism
Volume is composed of a series of snapshots and a BackingImage
Volume = BackingImage -> snap1 -> head

When Exporting Volume to another BackingImage code

create a snapshot
create an in-memory replica by OpenSnapshot() from that snaphot
preload() the block index map (including BackingImage)

so we know where to find the latest data in each block

use SyncContent() from sparse-tools to sync the replica content to BackingImage Datasource

we use GetDataLayout() to see if the block is hole or data

Volume Cloning Mechanism
Is totally the same as exporting BackingImage except it won't load BackingImage when doing `preload()

Volume Cloning Test

Creating BackingImage img1 with Ubuntu jammy cloud image (latest version)
Creating StorageClass

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: longhorn-test
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
  numberOfReplicas: "1"
  staleReplicaTimeout: "2880"
  backingImage: "img1"

Creating PVC os-vol from img1 with claim size 5GB and 1 replicas
Mount os-vol as block mode in the pod
Issuing filesystem expand and I/O on os-vol
Cloning os-vol to another volume os-cloned by applying the YAML

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: os-cloned
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Block
  storageClassName: longhorn-test
  resources:
    requests:
      storage: 5Gi
  dataSource:
    name: os-vol
    kind: PersistentVolumeClaim

attach both os-vol and os-cloned to the host and check the checksum are the same

BackingImage Export Test - FAILED
Following the same steps mentioned here, but don't apply resizefs
Still inconsistent

Consider Volume Clone(with partition recreation) works but BackingImage export(with partition recreation) doesn't.
I am thinking somehow the partition information are not correctly export when exporting with BackingImage
The only difference between Volume Clone and BackingImage export is that when
doing preload() to build replica.volume.Location
replica.volume.location = array of sectorIndex -> diskIndex

Volume Clone won't consider BackingIamge index in this case, so the export won't export BackingImage data
BackingImage Export will consider BackingImage when the sector has no new data, so it will export BackingImage data

Volume Clone won't consider BackingIamge index in this case, so the export won't export BackingImage data
BackingImage Export will consider BackingImage when the sector has no new data, so it will export BackingImage data

Sorry...? Do you mean that VolumeExport won't retrieve the backing image fiemap/sector list hence the BackingImage data won't be exported? But why does Volume Cloning Test get passed?

Hi @shuo-wu
When doing VolumeExport here
It will create a replica and preload the replica to build the r.volume.location index map
In Volume Cloning when doing it, since it don't clone the BackingImage, so when doing preload(), it ignore the BackingImage by init the index map with "0".
In BackingImage Export, it will consider BackingImage, so the preload() will init the index map with "1" which is the index of BackingImage.
Volume Clone only compressed all the Snapshots into one Snapshot and sync to the target Volume
BackingImage Export compressed all the Snapshots + BackingImage and sync to the target BackingImage
In the test, we mount the BackingImage to the Pod.
We recreate the partition and resize the filesystem and write some date
If we just do Volume Clone, then the new target Volume + Original BackingImage will be consistent
If we do BackingImage Export, then the new empty Volume + New BackingImage will be inconsistent

Does the volume clone test and the failed backing image test mean that the new partitioning info can be cloned correctly via VolumeClone but cannot be exported via ExportingBackingImage?

Yes, that is what I observed here.

I am thinking if it is because
assuming snapshot chain as follow
0. nil
1. BackingImage [1, 1, 0, 0]
2. Snap1 [1, 0, 0, 1]

Case 1 - Volume Clone: Export without BackingImage
first init to all 0, location = [0, 0, 0, 0]
r.volumes.preload(), location:
i = 0, nil                          skip => location = [0, 0, 0, 0]
i = 1, BackingImage [1, 1, 0, 0],   skip => location = [0, 0, 0, 0]
i = 2, Snap1        [1, 0, 0, 1]         => location = [2, 0, 0, 2]

ExportVolume and GetDataLayout=> [2, 0, 0, 2] which is [Data, Hole, Hole, Data]

Case 2 - BackingImage Export: Export with BackingImage
first init to all 1, location = [1, 1, 1, 1]
r.volumes.preload(), location
i = 0, nil                          skip => location = [1, 1, 1, 1]
i = 1, BackingImage [1, 1, 0, 0],   skip => location = [1, 1, 1, 1]
i = 2, Snap1        [1, 0, 0, 1]         => location = [2, 1, 1, 2]

ExportVolume and GetDataLayout => [2, 1, 1, 2] which is [Data, Data, Data, Data]

When SyncContent(), we actually check if the sector is Data or Hole by checking if the DiskIndex is 0 or non-0
So in the Case 1, we still punch hole when for sector 2, 3
But in the Case 2, since we init the index map to all "1", so all the sector is Data, we don't do punch hole. Then there might have some sector should be Hole because it don't have data in BackingImage either, but we fill the sector with 0 instead of Hole?
GetDataLayout()
cc @shuo-wu @PhanLe1010

The question is, in case 2, location = [2, 1, 1, 2] means [reading data in snap1, reading data in BI, reading data in BI, reading data in snap1 ].  In other words, the final data set should be identical...

Yeah so the problem is still
Why the partition info is not exported or is corrupted during exporting BackingImage…

This example assumes that the logical (virtual) size of the backing image is equal to the logical size of snapshot:
assuming snapshot chain as follow
0. nil
1. BackingImage [1, 1, 0, 0]
2. Snap1 [1, 0, 0, 1]

Case 1 - Volume Clone: Export without BackingImage
first init to all 0, location = [0, 0, 0, 0]
r.volumes.preload(), location:
i = 0, nil                          skip => location = [0, 0, 0, 0]
i = 1, BackingImage [1, 1, 0, 0],   skip => location = [0, 0, 0, 0]
i = 2, Snap1        [1, 0, 0, 1]         => location = [2, 0, 0, 2]

ExportVolume and GetDataLayout=> [2, 0, 0, 2] which is [Data, Hole, Hole, Data]

Case 2 - BackingImage Export: Export with BackingImage
first init to all 1, location = [1, 1, 1, 1]
r.volumes.preload(), location
i = 0, nil                          skip => location = [1, 1, 1, 1]
i = 1, BackingImage [1, 1, 0, 0],   skip => location = [1, 1, 1, 1]
i = 2, Snap1        [1, 0, 0, 1]         => location = [2, 1, 1, 2]

ExportVolume and GetDataLayout => [2, 1, 1, 2] which is [Data, Data, Data, Data]

In reality, they are not:

The ubuntu image has 2.2 GiB logical size:➜  Downloads qemu-img info jammy-server-cloudimg-amd64.img
image: jammy-server-cloudimg-amd64.img
file format: qcow2
virtual size: 2.2 GiB<PHONE_NUMBER> bytes)
disk size: 642 MiB
cluster_size: 65536
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

While the Longhorn snapshot has 10GB logical size

Revisiting 2 cases:

mount the BackingImage to the Pod + write some data => Export BackingImage is consistent
-> In this case, all the write fall into the first 2.2GB region of the backing image file
mount the BackingImage to the Pod + recreate partition&resizefs + write some data => Export BackingImage is inconsistent and the filesystem is corrupted
-> In this case, some data might fall outside of the 2.2GB region of the backing image file. So something like:assuming snapshot chain as follow
0. nil
1. BackingImage [1, 1, 0, 0]
2. Snap1        [1, 0, 0, 1, 0, 1, 0]

I am wondering if reading data from the qcow2 backing image outside of its virtual size can cause any consistency? Could we test with a raw backing image instead to see if we can hit the same issue @ChanYiLin ?

An other testing idea I am thinking about is:

Create an empty qcow2/raw image file of 1MB
Create a volume with that backing image and the volume size is 16MB
Write some data at the 2MB offset to the volume
Exporting the backing image from the volume
Read and compare the data from newly exported backing image and the old volume. Which regions of the data are different?

Hi @PhanLe1010
Thanks for the advice! Here is the testing results
BackingImage Export (use raw image) - FAILED

Create a raw image (I convert the jammy image from qcow2 to raw) and a Volume with it
Mount to the Pod
fdisk recreate the partition and resize2fs to resize the filesystem
Mounting device and writing I/O into it

$ mkdir /data; $ mount -t ext4 /dev/sdb1 /data
$ dd if=/dev/zero of=/data/file bs=1M count=2048
$ sync

Export the Volume to another BackingImage
Create a Volume with the BackingImage

Two file sizes are slightly different
root@kworker2:/var/lib/longhorn# du -sm replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/*
1	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/revision.counter
0	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/volume-head-001.img
1	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/volume-head-001.img.meta
2141	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/volume-snap-test-export-806bbe7b.img
1	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/volume-snap-test-export-806bbe7b.img.meta
1	replicas/pvc-b334ec5f-ab8f-4285-bffe-743b542ec494-9a715c65/volume.meta

root@kworker2:/var/lib/longhorn# du -sm backing-images/test-raw-61fde503/*
1517	backing-images/test-raw-61fde503/backing
1	backing-images/test-raw-61fde503/backing.cfg

root@kworker2:/var/lib/longhorn# du -sm backing-images/test-export-c504be3c/*
3868	backing-images/test-export-c504be3c/backing
1	backing-images/test-export-c504be3c/backing.cfg

Attach two volume to the host and the cksum are not the same
root@kworker2:/dev/longhorn# ll
total 0
drwxr-xr-x  2 root root    80 Jan 11 03:24 ./
drwxr-xr-x 20 root root  4260 Jan 11 03:24 ../
brw-rw----  1 root root 8, 32 Jan 11 03:24 pvc-6a0211dd-09d8-4bb7-a95e-d054988dd958
brw-rw----  1 root root 8, 16 Jan 11 03:17 pvc-b334ec5f-ab8f-4285-bffe-743b542ec494
root@kworker2:/dev/longhorn# cksum pvc-b334ec5f-ab8f-4285-bffe-743b542ec494
969496369<PHONE_NUMBER> pvc-b334ec5f-ab8f-4285-bffe-743b542ec494
root@kworker2:/dev/longhorn# cksum pvc-6a0211dd-09d8-4bb7-a95e-d054988dd958
2208768380<PHONE_NUMBER> pvc-6a0211dd-09d8-4bb7-a95e-d054988dd958

Exported BackingImage with the Volume has no filesystem corrupted issue
root@block-volume-test:/# ls -al /dev/os-export
brw-rw---- 1 root root 8, 32 Jan 11 03:35 /dev/os-export
root@block-volume-test:/# ls -al /dev/sdc
brw-rw---- 1 root disk 8, 32 Jan 11 03:35 /dev/sdc
root@block-volume-test:/# e2fsck -c /dev/sdc1
e2fsck 1.46.5 (30-Dec-2021)
Checking for bad blocks (read-only test): done
cloudimg-rootfs: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

cloudimg-rootfs: ***** FILE SYSTEM WAS MODIFIED *****
cloudimg-rootfs: 73666/645120 files (0.0% non-contiguous), 949441/1282299 blocks

The data written in to the device are the same
root@block-volume-test:/# mkdir /data
root@block-volume-test:/# mkdir /data-export
root@block-volume-test:/# mount -t ext4 /dev/sdb1 /data
root@block-volume-test:/# mount -t ext4 /dev/sdc1 /data-export
root@block-volume-test:/# cksum /data/file
2532515601<PHONE_NUMBER> /data/file
root@block-volume-test:/# cksum /data-export/file
2532515601<PHONE_NUMBER> /data-export/file

Conclusion

With using raw image, two volumes are still not consistent, and the size are different but almost the same.
The filesystem in this case is not corrupted

An other testing idea I am thinking about is:

Create an empty qcow2/raw image file of 1MB

Create a volume with that backing image and the volume size is 16MB

Write some data at the 2MB offset to the volume

Exporting the backing image from the volume

Read and compare the data from the newly exported backing image and the old volume.

Which regions of the data are different?
Does the newly exported backing image contain all zero? -> If yes, it means the backing image layer overwrite the snapshot layer?

Hi @PhanLe1010
Thanks for the idea, I will try
But we have confirmed that using qcow2 image,
if we directly mount the filesystem, without recreating the partition, the exported BackingImage will be consistent.
Here is the test case

Thanks for the idea, I will try
But we have confirmed that using qcow2 image,
if we directly mount the filesystem, without recreating the partition, the exported BackingImage will be consistent.
Here is the test case

I think the new test idea is different than the consistent test case because the new test idea does write data to outside of the backing image boundary (similar to when you recreating the partition and resize the filesystem to be bigger than the virtual size of the backing image). WDYT?

Hi @PhanLe1010 I think you are right
Backing Image Export Test (using empty raw image with larger volume size) - FAILED

Create a empty raw image and upload as a BackingImage empty-raw

$ truncate -s 2M empty-raw

Create a 16MB Volume test-empty with the raw image
Attach to the host
dd 1MB data starts from 3MB offset

$ dd of=/dev/longhorn/test-empty if=/dev/urandom bs=1M count=1 oflag=seek_bytes seek=3145728

export the Volume to another BackingImage empty-raw-export
Create a 16MB Volume test-empty-export with the BackingImage empty-raw-export
Attach to the host

Test Result
test-empty-export is empty
and the comparison result shows that it dffers from 3MB offset
brw-rw----  1 root root 8, 48 Jan 11 05:35 test-empty
brw-rw----  1 root root 8, 64 Jan 11 05:36 test-empty-export
root@kworker1:/dev/longhorn# cat test-empty-export
root@kworker1:/dev/longhorn# cmp test-empty test-empty-export
test-empty test-empty-export differ: byte 3145729, line 1
root@kworker1:/dev/longhorn#

This is actually fit the testing results from @WebberHuang1118 testing
The cmp.log he provided also showed that the different bytes started after 2.2GB which is the size of the jammy ubuntu image.
And the differences were also because those values were now "0"

Awesome. It means somehow the region outside of the backing image boundary (3MB offset to 4MB offset region) in the backing image layer is overwriting the snapshot layer.
Maybe next step would be adding some logs to the preload() function to see if we can identify the fault logic (why choosing the backing image layer instead of snapshot layer for the region 3MB-4MB)?

I did intentionally print the r.volume.location after the preload and also print the segments sending during the exporting
Here is the results

As you can see the preload location I think is correct, there is a series of 2 which means the data from the snapshot
But when doing exporting, it only sent one segment [D 0 4096]
Note: I add the log message here, this function will be executed by multiple goroutine in parallel when sending segment

Sorry @ChanYiLin. I will have to continue tomorrow.

sure no problem, thanks for your help!
I think we are pretty close to the answer

it seems the preload result and the sending segment intervals are correct
now the problem might happen in reading the data with the given segments?

Interesting finding @ChanYiLin !

Event it was reading [2MB, 4MB] position from the replicas, the data is all 0

What about when cloning, ReadAt() also returns all 0 in the region [2MB, 4MB] ?

@PhanLe1010 you can search [DEBUG in syncDataInterval]: Sending batch: [     512:    1024](512), dataBuffer: in the file
that is the data sending from the sorce volume the target volume
Since the sending batch size is 2MB and the sector size is 4KB so the  [     512:    1024] means [2MB: 4MB]

Data only exists in [3MB: 4MB]
As you can see from the previous test, the whole [2MB:4MB] data sending is "0"
But Here is the result of using clone
The result of preload location is correct and the when sending the batch, it is sending Sending batch: [     768:    1024](256) exactly 1MB because [2MB: 3MB] is Hole now, it didn't even try to read from that offest
and the result is correct
lhim1_new3.txt

I think I found the bug
When reading [2MB-4MB] from the Volume
[2MB-3MB] is BackingImage
[3MB-4MB] is snap1

https://github.com/longhorn/longhorn-engine/blob/deb8b18a1558ddf4f359146b289d5469bef7cc6d/pkg/replica/diff_disk.go#L202-L241
Here when we try to read the data

If the target disk is out of bound, it could because of Volume Expansion or BackingImage.

In this case the preload location indicated that [2MB-3MB] is BackingImage and tried to read from it.

Then we will directly return 0 count because we failed to read any data to the buf

	// May read the expanded part
	if offset >= size {
		return 0, nil
	}

For [3MB-4MB], the target disk is correct, and we read the data from snap1 to the buf and the count would be 1MB
But we cut the return buf with count after reading from the file here. So the return data abandoned the data we read from the snap1 and only leave the first half of "0" data.

func ReadDataInterval(rw ReaderWriterAt, dataInterval Interval) ([]byte, error) {
	data := AllocateAligned(int(dataInterval.Len()))
	n, err := rw.ReadAt(data, dataInterval.Begin)
	if err != nil {
		if err == io.EOF {
			log.Debugf("Have read at the end of file, total read: %d", n)
		} else {
			log.WithError(err).Errorf("Failed to read interval %+v", dataInterval)
			return nil, errors.Wrapf(err, "failed to read interval %+v", dataInterval)
		}
	}
	return data[:n], nil
}

I wonder do we really need to cut it with count?
cc @shuo-wu @PhanLe1010 @WebberHuang1118

By not cutting the data data[:n]
The Volume are now consistent even testing with the original Test Case

Agreeing with removing the cut for ReadDataInterval.
On the other hand, I am thinking if we need to improve diffDisk.fullReadAt result asl well.
In the above case, we try to read data [2MB, 4MB], then diffDisk tells us the read count is 1MB. Intuitively users will consider that the read content is at the begin of the buffer then they will discard the latter 1MB. But actually NO...
What we can improve here is: When we read a data interval like [out of bound part of the backing image/smaller snapshot,data, out of bound part of the backing image/smaller snapshot], the returned read length should be len([out of bound part of the backing image/smaller snapshot, data]) rather than len([data])

From my perspective, I think we should still return len([data]) for diffDisk.fullReadAt
The reason is that, from the caller's perspective, it is reading from the "Volume"
func (r *Replica) ReadAt(buf []byte, offset int64) (int, error) {
	r.RLock()
	c, err := r.volume.ReadAt(buf, offset) // which will call diffDisk.fullReadAt()
	r.RUnlock()
	return c, err
}

So it has no knowledge about the snapshot chain underneath.
If it reads outside of the boundary of the "Volume", we do return io.ErrUnexpectedEOF error here
But if it only reads outside of the boundary of "specific snapshot image", and we allow it to do so, then we should consider it as reading a bunch of "0", and that is also true.
The caller should not take care of the situation. Instead it might get confused that it doesn't get EOF error but how come the data length is not correct.

After discussing with @ChanYiLin, we agree that Longhorn should return the length the caller wants to read if there is no error EOF

Hi @ChanYiLin ,
Following test steps, this issue verified pass on master-head, but in v1.6.x-head, the two backing image data were still not consistent.
The measter-head engine images build time is later then v1.6.x-head,  not sure if this means that v1.6.x engine does not contain your fix, could you help to check? thank you.
> docker images
REPOSITORY                   TAG           IMAGE ID       CREATED       SIZE
longhornio/longhorn-engine   master-head   16d097e2f0e6   4 hours ago   343MB
longhornio/longhorn-engine   v1.6.x-head   1c7202bbd25e   5 hours ago   343MB

@chriscchien 1.6 PR was just merged, so please wait for https://drone-publish.longhorn.io/longhorn/longhorn-engine/810 finished.

Verified pass on longhorn master(longhorn-engine ac4748), longhorn v1.6.x(longhorn-engine 0b553a) with test steps
I could reproduce this issue before the fix, after the fix, the two volumes data are identical.


Adding in a Makefile.boot filename for .boot detection

Looks like we added support for .boot in https://github.com/github/linguist/pull/2254 without considering the discussion we had in https://github.com/github/linguist/pull/1894
This pull request adds Makefile.boot which seems to be the dominant file type using the .boot extension.

Fix for failure in https://github.com/github/linguist/pull/2299


Broken Docker image on Docker Hub

I'm not sure if you're aware or not, but there is a Docker image on Docker Hub - https://hub.docker.com/r/beamaustralia/react-env - that runs a static website using react-env.
This was fantastic back when I first found it. However, it's now not been updated in 2 years and doesn't work with the current version of react-env. Which means that if you use the latest react-env in your project but try to use this docker image to deploy it then you'll likely waste hours trying to work out why things are broken.
I know it's not actually yourself that created this image, and I can't work out who did create it in order to message them, but it seemed worth mentioning if it can somehow stop other people having the same waste of time that I've just had!
Cheers

Hello! It was actually me who created this image. I should deprecate it, there are instructions on how to build your own docker image with the react-env binary. It is super easy. We simply dont want to maintain one!

Thank you for the heads up!


Infer Zipkin base path based on current URL.

Will add tests after getting an OK that the approach seems ok.
We previously passed a base path for zipkin through the server via a property. But I think it's relatively robust to just infer the base path from the browser URL.
There is a difference regards to find traces URL - now they render at e.g., /zipkin/?endTs..., previously it was /zipkin?endTs... (note no trailing slash in that one). I remember we had an issue where we removed the trailing slash for this page, but I think with the unlaunch of classic (hence, no switching between the two UIs), this isn't an issue anymore, but I can't remember it well enough to be sure. Note that the server will redirect /zipkin?endTs... to /zipkin/?endTs... so both URLs will still be accepted.
Unblocks #2519

@jorgheymans would you mind giving this a spin? I think you helped verify once upon a time..

I'll try and give this a spin today.
Op di 25 feb. 2020 07:15 schreef Adrian Cole<EMAIL_ADDRESS>
@adriancole approved this pull request.
I'd like to get manual verification with a proxy as used before, but
doesn't seem like that's in the cards.
So, verification will happen after release when someone tries to use it!
Worst case we have a small patch, which is better than holding back change.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/openzipkin/zipkin/pull/2982?email_source=notifications&email_token=AABPKAFYMT4VRM7ORKT2KXLRESZRPA5CNFSM4KZQVQ52YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCWYI2KQ#pullrequestreview-363892010,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABPKAGG7CL2RXJRSHCVGPLRESZRPANCNFSM4KZQVQ5Q
.

Thanks a lot @jorgheymans! I don't have a config on hand which is why setting that up is a bit low priority for me - as @adriancole says, I think we're still better with a chance of it working rather than no chance, even if we don't get to manually verify :)

Found the reason to not infer base path :) Static resources are still in the HTML so need to tweak base tag so they're served properly. Going to send another PR


[Consumer Refactor]Background thread

The skeleton of the background thread.  Impl to be added.
Detail design see: https://cwiki.apache.org/confluence/display/KAFKA/[Draft]+Consumer+Threading+Model+Refactoring
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

closing this as I want to move it to AK.


References under paths are compared only by ref name - oasdiff should compare the ref values.

Describe the bug
When a Path Item Object contains a $ref, oasdiff compares only textual changes to the $ref field.
Ideally, oasdiff should compare the referenced definitions of this path item(s) to each-other.
To Reproduce
Steps to reproduce the behavior:
Run oasdiff -base ref1.yaml -revision ref2.yaml
ref1.yaml
openapi: 3.0.1
info:
  title: Test API
  version: v1
paths:
  /partner-api/test/some-method:
    $ref: '#/components/paths/path1'
components:
  paths:
    path1:
      get:
        responses:
          "200":
            description: Success

ref2.yaml
openapi: 3.0.1
info:
  title: Test API
  version: v1
paths:
  /partner-api/test/some-method:
    $ref: '#/components/paths/path2'
components:
  paths:
    path2:
      get:
        responses:
          "200":
            description: Success

Result:
paths:
    modified:
        /partner-api/test/some-method:
            ref:
                from: '#/components/paths/path1'
                to: '#/components/paths/path2'

I'm happy to work on this issue.

@reuvenharrison I had some attempts to fix this by creating new diff category:
// RefDiff describes the changes between content refs
type RefDiff struct {
	RefModifled []string `json:"refModified,omitempty" yaml:"refModified,omitempty"`
}

type RequestBodyDiff struct {
        ...
	RefDiff			*RefDiff     	    `json:"ref,omitempty" yaml:"ref,omitempty"`
}

Do you think this is the good way?

Going to use RefDiff         *ValueDif

Hi @reuvenharrison @wtrocki . Thanks again for the great work on oasdiff. We use it daily.
This impacts us as well, especially for analyzing Post Body parameters. From what I've seen, request bodies are usually referenced via component schemas rather than via Request body objects under a path.
We have some automation that you're aware of that compares about 200 swagger docs versus others. I generated a parameter specific report and there were only about 5 instances where post body parameters were defined in RequestBody objects rather than via locally referenced schemas.
Do you have an idea of when this might be supported or on the roadmap? If it's not on the roadmap, perhaps we could get some pointers on where to contribute for this. Not sure of how widespread the changes would need to be.

Hi @kmcquade
I'll take a look at this asap.

Hi @kmcquade
Regarding the original issue, "References under paths are compared only by ref name":
I cannot replicate this.
For example, comparing the following shows the full change to the referenced paths:
openapi: 3.0.1
info:
  title: Test API
  version: v1
paths:
  /partner-api/test/some-method:
    $ref: '#/components/paths/path1'
components:
  paths:
    path1:
      get:
        responses:
          "200":
            description: Success

openapi: 3.0.1
info:
  title: Test API
  version: v1
paths:
  /partner-api/test/some-method:
    $ref: '#/components/paths/path2'
components:
  paths:
    path2:
      get:
        responses:
          "201":
            description: Success

❯ oasdiff diff data/1 data/2 --exclude-elements endpoints
paths:
    modified:
        /partner-api/test/some-method:
            ref:
                from: '#/components/paths/path1'
                to: '#/components/paths/path2'
            operations:
                modified:
                    GET:
                        responses:
                            added:
                                - "201"
                            deleted:
                                - "200"

Hi again @kmcquade
Regarding the issue you mentioned of referenced schemas under request body.
I cannot replicate this one either:
openapi: 3.0.0
info:
  version: "1.0.0"
  title: home-iot-api
  description: The API for the EatBacon IOT project
paths:
  /devices:
    post:
      responses:
        '200':
          description: successfully registered device
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/DeviceRegistrationInfo'
components:
  schemas:
    DeviceRegistrationInfo:
      type: string

openapi: 3.0.0
info:
  version: "1.0.0"
  title: home-iot-api
  description: The API for the EatBacon IOT project
paths:
  /devices:
    post:
      responses:
        '200':
          description: successfully registered device
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/DeviceRegistrationInfo'
components:
  schemas:
    DeviceRegistrationInfo:
      type: number

❯ oasdiff diff data/1 data/2 --exclude-elements endpoints
paths:
    modified:
        /devices:
            operations:
                modified:
                    POST:
                        requestBody:
                            content:
                                mediaTypeModified:
                                    application/json:
                                        schema:
                                            type:
                                                added:
                                                    - number
                                                deleted:
                                                    - string
components:
    schemas:
        modified:
            DeviceRegistrationInfo:
                type:
                    added:
                        - number
                    deleted:
                        - string


update TraceLayer save_inference_model api doc and dygraph_to_static example code

update TraceLayer save_inference_model api doc and the example code of dygraph_to_static guide doc

The new api has been merged with https://github.com/PaddlePaddle/Paddle/pull/31989


Introduce skewing pass

This PR introduces a skewing pass as part of implementing time-tiling.
Data accesses remain the same after skewing.
Known issues/to fix in parallel with reviewing are commented in code.

Codecov Report

Merging #856 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #856      +/-   ##
==========================================
- Coverage   88.89%   88.87%   -0.02%
==========================================
  Files         102      102
  Lines       11472    11490      +18
  Branches     2258     2261       +3
==========================================
+ Hits        10198    10212      +14
- Misses        989      993       +4
  Partials      285      285

Impacted Files
Coverage Δ

devito/dle/blocking_utils.py
89.11% <ø> (ø)
:arrow_up:

devito/symbolics/manipulation.py
90.14% <100%> (+0.14%)
:arrow_up:

devito/dse/rewriters.py
93.8% <100%> (+0.44%)
:arrow_up:

devito/ir/support/space.py
86.2% <100%> (+0.04%)
:arrow_up:

devito/ir/iet/nodes.py
93.47% <0%> (-1.05%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2764109...099228c. Read the comment docs.

As I was discussing with @navjotk , this PR should probably be closed as we realised that skewing is a clustering-level transformation, rather than a DSE-level transformation. To clarify:

A DSE pass works "one Cluster at a time", which gets transformed in complete isolation.
A clustering pass works (recursively) on groups of Clusters sharing an iteration space.

My making skewing a clustering pass we would catch all Clusters sharing (for example) the time dimension and we would be able to transform all of them coherently and in a single go.
Thoughts? Questions?

Thoughts? Questions?
Yes we can close it for the moment, I can continue working on it on the background.


unrecognized arguments: --password Pa$$w0rd01

I'm using Azure CLI 2.4.0 on MacOS.
When I follow the instruction towards the creation of the RBAC security:
az ad sp create-for-rbac --name TTapp --password Pa$$w0rd01
The CLI gives me error: "az: error: unrecognized arguments: --password Pa19428w0rd01"

I encountered the same error when following the instructions.
So instead of creating a Service Principal through Azure CLI, I created it through the Azure DevOps user interface (Azure DevOps > Project settings > Pipelines - Service connections > New service connection > Azure Resource Manager > Service principal (automatic))


More specific whitelisting

Is it possible to be more specific in the whitelisting as I'd prefer not to have to allow access to the entirety of Azure blob storage. Can you specify the Defender ATP specific blobs, rather than just everyone's Azure storage?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ef3b1143-53ee-892c-59c8-5cfa12e017c2
Version Independent ID: 29f6c231-15bd-296b-cca9-7ef6ba8192df
Content: Microsoft Defender ATP for Linux - Windows security
Content Source: windows/security/threat-protection/microsoft-defender-atp/microsoft-defender-atp-linux.md
Product: w10
Technology: windows
GitHub Login: @Dansimp
Microsoft Alias: dansimp

They’re still recommending the same for Mac clients too. Got fixed for Windows a while back and so far no issues with our Mac clients: https://github.com/MicrosoftDocs/windows-itpro-docs/issues/4942

Hello @kengoodwin,
as @phillym noticed, more detailed URLs could be found here.
Also, you can use the list of IP addresses published here, for example, to whitelist blob storage for Australia SE, you can use IP addresses name "Storage.AustraliaSoutheast"
Thank you

Hello @kengoodwin,
as @phillym noticed, more detailed URLs could be found here.
Also, you can use the list of IP addresses published here, for example, to whitelist blob storage for Australia SE, you can use IP addresses marked with name "Storage.AustraliaSoutheast".
Thank you

kengoodwins point, like mine before it is that it is unwise to exclude all of Azure blob storage from TLS interception and proxy auth, if those are controls you have in place already. Microsoft have specific buckets for these services and we need these to be explicitly documented so we only need to exclude the bucket DNS name and not every possible Azure Blob bucket.

Dear @Dansimp,
could you please assist?
Thank you

@kengoodwin @phillym - Thank you for submitting feedback.
From our understanding, the issue has been resolved based on this merged commit https://github.com/MicrosoftDocs/windows-itpro-docs/commit/26b93c9.
Thank you for your contribution to make the docs better! Much appreciated!


Fornax watch exits after file generation

Describe the bug
Running dotnet fornax watch generated files as expected, but exited immediately after generation without further input.
To Reproduce
Steps to reproduce the behaviour:

Run the commands specified here: https://github.com/ionide/ionide.github.io/blob/dev/README.md
Wait for file generation to complete
The command exits

Expected behaviour
The watch command should still be running after generation for inspection of the generated site.
Screenshots
λ dotnet fornax watch
[13:12:38] 'C:\Alt\Git\ionide.github.io\_public\index.json' generated in 381ms
[13:12:42] 'C:\Alt\Git\ionide.github.io\_public\callToAction.html' generated in 994ms
[13:12:45] 'C:\Alt\Git\ionide.github.io\_public\index.html' generated in 617ms
[13:12:47] 'C:\Alt\Git\ionide.github.io\_public\sponsors.html' generated in 734ms
[13:12:50] 'C:\Alt\Git\ionide.github.io\_public\Editors/editoresOverview.html' generated in 535ms
[13:12:52] 'C:\Alt\Git\ionide.github.io\_public\Editors/index.html' generated in 567ms
[13:12:55] 'C:\Alt\Git\ionide.github.io\_public\Libraries/eventhorizon.html' generated in 995ms
[13:12:58] 'C:\Alt\Git\ionide.github.io\_public\Libraries/fsanalyzers.html' generated in 58ms
[13:13:01] 'C:\Alt\Git\ionide.github.io\_public\Libraries/waypoint.html' generated in 4ms
[13:13:04] 'C:\Alt\Git\ionide.github.io\_public\Tools/dpi.html' generated in 788ms
[13:13:07] 'C:\Alt\Git\ionide.github.io\_public\Tools/forge.html' generated in 809ms
[13:13:10] 'C:\Alt\Git\ionide.github.io\_public\Tools/fornax.html' generated in 535ms
[13:13:12] 'C:\Alt\Git\ionide.github.io\_public\Tools/fsac.html' generated in 636ms
[13:13:15] 'C:\Alt\Git\ionide.github.io\_public\Editors/Code/editing.html' generated in 854ms
[13:13:18] 'C:\Alt\Git\ionide.github.io\_public\Editors/Code/generalInfo.html' generated in 187ms
[13:13:22] 'C:\Alt\Git\ionide.github.io\_public\Editors/Code/getting_started.html' generated in 566ms
[13:13:26] 'C:\Alt\Git\ionide.github.io\_public\Editors/Code/options.html' generated in 915ms
[13:13:30] 'C:\Alt\Git\ionide.github.io\_public\Editors/Code/overview.html' generated in 258ms
[13:13:33] 'C:\Alt\Git\ionide.github.io\_public\Editors/Vim/getting_started.html' generated in 785ms
[13:13:35] 'C:\Alt\Git\ionide.github.io\_public\Editors/Vim/overview.html' generated in 711ms
[13:13:38] 'C:\Alt\Git\ionide.github.io\_public\Editors/Vim/usage.html' generated in 770ms
[13:13:38] Watch mode started. Press any key to exit.
 Exiting...

Environment (please complete the following information):

OS: Windows 10
Ionide version: 4.7.0
VSCode version: 1.43.0
dotnet SDK version: 3.1.102

@mattherman Is this still happening?
I can't seem to recreate the issue on Windows or MacOS with either Fornax 11.0 version used locally or latest 13.1 used globally.

I'm not sure. I tried to reproduce on my Windows machine and ran into other issues now. When I run dotnet tool restore it says both tools were restored successfully, but then running dotnet tool run paket restore I get an error saying I need to run dotnet tool restore first. The only thing that has changed as far as I can tell is that I now have SDK version 3.1.401.
When I run through these steps on my Debian machine (with the same SDK version) it works fine, including the subsequent dotnet fornax watch.
I'm guessing something else is wrong with my Windows machine so this issue can probably be closed.


Adding an item to the world has a default velocity

Optional<Entity> optEntity = world.createEntity(EntityTypes.DROPPED_ITEM, pos);
if (optEntity.isPresent()) {
    Item item = (Item) optEntity.get();
    item.offer(item.getItemData().set(Keys.REPRESENTED_ITEM, stack));
    world.spawnEntity(item);
}

In a previous version of Sponge, the above code would result in an item being spawned at "pos" without being "thrown"/assigned a default velocity. Now, this results in the equivalent of a natural item spawn, where the item is thrown as if a block has been mined.
While both functionalities (no velocity, and vanilla random spawn velocity) are desirable, this behavior seems strange for the above case.

@DarkArc
Is this still valid?

As far as I've seen yes.

This would have been an effect of #37
The velocity can easily be set to zero after creating the entity,
item.offer(Keys.VELOCITY, Vector3d.ZERO);

The question is, when creating an entity should sensible defaults take priority over setting everything to zero?

@simon816
Not sensible defaults, whatever is happening in the entities' constructor by default in the base game is exactly the same we should be doing.
    public EntityItem(World worldIn, double x, double y, double z)
    {
        super(worldIn);
        this.health = 5;
        this.hoverStart = (float)(Math.random() * Math.PI * 2.0D);
        this.setSize(0.25F, 0.25F);
        this.setPosition(x, y, z);
        this.rotationYaw = (float)(Math.random() * 360.0D);
        this.motionX = (double)((float)(Math.random() * 0.20000000298023224D - 0.10000000149011612D));
        this.motionY = 0.20000000298023224D;
        this.motionZ = (double)((float)(Math.random() * 0.20000000298023224D - 0.10000000149011612D));
    }

Which means that you should set the velocity to ZERO as simon suggests @DarkArc after creating the Entity but before spawning it.

@Zidane I would like to bring a counter point that we should always go for the bare minimum behavior (nothing more than has been requested).
When we introduce the constraint that we're going to be relying on the default construction of the object, that opens us up for our behavior to change unexpectedly between versions of Minecraft.
A good example of how this can be a problem is one I encountered while using Bukkit. They "fixed" their mob spawning code by modifying the spawning methods to behave as a "normal" mob spawn would. However, this lead to problems with some of my plugins, as entities I intended -- and needed -- to be spawned "simple" were being spawned with potion effects/weapons/armor.
I would instead suggest extending the API in such a way that we must opt-in to "vanilla X" behavior, by doing this, we can -- at least in theory, we're a moding API so there's always that -- provide stronger guarantees about the future behavior of the API -- those which we explicitly state, not what we inherit.


Retrive notification by Email

Hi, is there any way to retrieve notifications based on user email.
In the doc it is using notification_id
/push/notifications/:notification_id
Thank you

No, this API endpoint doesn't exist (or at least isn't documented).

Can we make it as a feature.


Sources advertising no prestage key and not needing previous sources should skip tracking

See original issue on GitLab
In GitLab by [Gitlab user @traveltissues] on Oct 30, 2019, 12:39
When adding 33d62860244c1b8d44fc2aca53f003b9cb031877 it seemed that sources with BST_NO_PRESTAGE_KEY = True and BST_REQUIRES_PREVIOUS_SOURCES = False can skip tracking jobs to avoid re-calculating the source key. For sources without a pre-stage key and requiring previous sources it might be possible to cache the calculated source key in some way to communicate it to the build job.

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 10:28
assigned to [Gitlab user @traveltissues]

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 10:28
assigned to [Gitlab user @traveltissues]

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 15:51
[Gitlab user @juergbi] it seems like the best way to do this would be just to replace the assertion on the digest in Source._stage()
if self.BST_KEY_REQUIRES_STAGE:
    if self.__digest is None:
        self._get_unique_key()

or were you referring to something else in your comment?
you could also do something like: https://gitlab.com/BuildStream/buildstream/commit/6ee7ee2718981b0bb350c88e8bfd8769f48b6043
but I'd prefer to keep the logic encapsulated in Source._track and I'm not sure this would be more performant. Obviously if we could cache the key between processes then we could attempt to simply recover it here and skip calling either method. I'm not sure if that would be possible with the current scheduler.
Additionally we could take this opportunity to rethink the use of _get_unique_key outside of _generate_key. Calling just _get_unique_key would be more performant unless we can accurately memoize _generate_key

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 15:51
[Gitlab user @juergbi] it seems like the best way to do this would be just to replace the assertion on the digest in Source._stage()
if self.BST_KEY_REQUIRES_STAGE:
    if self.__digest is None:
        self._get_unique_key()

or were you referring to something else in your comment?
you could also do something like: https://gitlab.com/BuildStream/buildstream/commit/6ee7ee2718981b0bb350c88e8bfd8769f48b6043
but I'd prefer to keep the logic encapsulated in Source._track and I'm not sure this would be more performant. Obviously if we could cache the key between processes then we could attempt to simply recover it here and skip calling either method. I'm not sure if that would be possible with the current scheduler.
Additionally we could take this opportunity to rethink the use of _get_unique_key outside of _generate_key. Calling just _get_unique_key would be more performant unless we can accurately memoize _generate_key

In GitLab by [Gitlab user @juergbi] on Nov 5, 2019, 16:56

it seems like the best way to do this would be just to replace the assertion on the digest in Source._stage()

What would be the replacement? And how would that skip tracking?
I was thinking that we could completely skip tracking jobs for elements where none of their sources can track. We know that BST_KEY_REQUIRES_STAGE sources can't track but we could also generalize this and add a can_track() method such that sources can report whether they can track (e.g., the git plugin could return False if track is not set).
Element._schedule_tracking() could return False if none of its sources support tracking, in which case Stream could skip adding these elements to the track queue.
However, I don't know whether this optimization would be worthwhile for many users.

In GitLab by [Gitlab user @juergbi] on Nov 5, 2019, 16:56

it seems like the best way to do this would be just to replace the assertion on the digest in Source._stage()

What would be the replacement? And how would that skip tracking?
I was thinking that we could completely skip tracking jobs for elements where none of their sources can track. We know that BST_KEY_REQUIRES_STAGE sources can't track but we could also generalize this and add a can_track() method such that sources can report whether they can track (e.g., the git plugin could return False if track is not set).
Element._schedule_tracking() could return False if none of its sources support tracking, in which case Stream could skip adding these elements to the track queue.
However, I don't know whether this optimization would be worthwhile for many users.

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 17:01

What would be the replacement? And how would that skip tracking?

The replacement would be assert self.__digest is None
That was my question; I was trying to understand what you meant in your comment, skipping tracking jobs or skipping calls to tracking methods. So if it is the latter then yes, we could just skip it if the element sources don't track. It would make sense to avoid scheduling this at all

In GitLab by [Gitlab user @traveltissues] on Nov 5, 2019, 17:01

What would be the replacement? And how would that skip tracking?

The replacement would be assert self.__digest is None
That was my question; I was trying to understand what you meant in your comment, skipping tracking jobs or skipping calls to tracking methods. So if it is the latter then yes, we could just skip it if the element sources don't track. It would make sense to avoid scheduling this at all

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 11:53
mentioned in merge request !1689

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 11:53
mentioned in merge request !1689

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 12:06
mentioned in commit bc3e297d81fd46aa226008d311431f22b6dca043

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 12:06
mentioned in commit bc3e297d81fd46aa226008d311431f22b6dca043

In GitLab by [Gitlab user @BenjaminSchubert] on Nov 6, 2019, 12:15
I'm not sure this optimization actually buys anything, now that tracking is not allowed during the main pipeline anymore. Do you expect the tracking operation to be so much more efficient like that?

In GitLab by [Gitlab user @BenjaminSchubert] on Nov 6, 2019, 12:15
I'm not sure this optimization actually buys anything, now that tracking is not allowed during the main pipeline anymore. Do you expect the tracking operation to be so much more efficient like that?

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 12:34
we will have to benchmark and that will depend on a case-by-case but
initially it does seem that if we don't have to track then we should
just not. If it's cheaper to determine this than always scheduling tracking jobs then
it will be beneficial, otherwise it's not useful

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 12:34
we will have to benchmark and that will depend on a case-by-case but
initially it does seem that if we don't have to track then we should
just not. If it's cheaper to determine this than always scheduling tracking jobs then
it will be beneficial, otherwise it's not useful

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 17:25
mentioned in commit 5259a4c207cda08bf354b43dd8ff1b21805fca96

In GitLab by [Gitlab user @traveltissues] on Nov 6, 2019, 17:25
mentioned in commit 5259a4c207cda08bf354b43dd8ff1b21805fca96

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 09:52
mentioned in commit f568dc998d13a87f261b688fcd407bb242f333a2

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 09:52
mentioned in commit f568dc998d13a87f261b688fcd407bb242f333a2

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:06
mentioned in commit 17ddca2dcbae0b3f2d94e7580916026c1c33d204

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:06
mentioned in commit 17ddca2dcbae0b3f2d94e7580916026c1c33d204

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:06
mentioned in commit 9ce149a8f74b43eb7a10ca52f0f94382fc5d749b

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:06
mentioned in commit 9ce149a8f74b43eb7a10ca52f0f94382fc5d749b

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:10
mentioned in commit 4140cc66ae525b63dab2070241d5f3191c188f75

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:10
mentioned in commit 4140cc66ae525b63dab2070241d5f3191c188f75

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:13
mentioned in commit ddac68a9b264bb7e5844dc6e39663429b91a6b0e

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 10:13
mentioned in commit ddac68a9b264bb7e5844dc6e39663429b91a6b0e

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 11:16
mentioned in commit d4018620a627e322fd1e62bdcabf09341618e8bd

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 11:16
mentioned in commit d4018620a627e322fd1e62bdcabf09341618e8bd

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 12:25
mentioned in commit 7364a5b966a17d669d7a765e0e5ad3408fc9e8de

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 12:25
mentioned in commit 7364a5b966a17d669d7a765e0e5ad3408fc9e8de

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 12:57
mentioned in commit 5e6819e3377995b4e08fc0a8d434f363bcde7c7c

In GitLab by [Gitlab user @traveltissues] on Nov 7, 2019, 12:57
mentioned in commit 5e6819e3377995b4e08fc0a8d434f363bcde7c7c

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 08:45
mentioned in commit aa9af0a7e8dc253d91aad1edad2877d2ea2e9a9f

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 08:45
mentioned in commit aa9af0a7e8dc253d91aad1edad2877d2ea2e9a9f

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 09:15
mentioned in commit 0dd07312b47d33d385b69378b58f19aacdf09565

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 09:15
mentioned in commit 0dd07312b47d33d385b69378b58f19aacdf09565

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 10:52
mentioned in commit 1dc87347ab5bcb7e3cfd6b24ad171a92efe446bb

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 10:52
mentioned in commit 1dc87347ab5bcb7e3cfd6b24ad171a92efe446bb

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:11
mentioned in commit 9a42c36c135f4491dbc4120fa9bc8615ea3784d4

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:11
mentioned in commit 9a42c36c135f4491dbc4120fa9bc8615ea3784d4

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:15
mentioned in commit c274bdf4f3cb1443d56f0a1382077de7facf623f

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:15
mentioned in commit c274bdf4f3cb1443d56f0a1382077de7facf623f

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:19
mentioned in commit 1589ea0b5f00332a70f3420d6f28237f740e244d

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:19
mentioned in commit 1589ea0b5f00332a70f3420d6f28237f740e244d

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:27
mentioned in commit ecb2226d86844b56ef604cc00a07f19ece53ccc0

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 12:27
mentioned in commit ecb2226d86844b56ef604cc00a07f19ece53ccc0

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 13:26
mentioned in commit 54a597286dfc5e29f81deed0bd8890f7e5160d54

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 13:26
mentioned in commit 54a597286dfc5e29f81deed0bd8890f7e5160d54

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 13:45
mentioned in commit c76c918a7febda4ef4253f8a9a957276d8ebb614

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 13:45
mentioned in commit c76c918a7febda4ef4253f8a9a957276d8ebb614

In GitLab by [Gitlab user @marge-bot123] on Nov 11, 2019, 14:17
closed via merge request !1689

In GitLab by [Gitlab user @marge-bot123] on Nov 11, 2019, 14:17
closed via merge request !1689

In GitLab by [Gitlab user @marge-bot123] on Nov 11, 2019, 14:17
mentioned in commit b3270030b5a9e520fd6a2d9f7acd923e27e10139

In GitLab by [Gitlab user @marge-bot123] on Nov 11, 2019, 14:17
mentioned in commit b3270030b5a9e520fd6a2d9f7acd923e27e10139

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 16:43
mentioned in commit 078f8bae8af273490973aa86e867053b19cd7048

In GitLab by [Gitlab user @traveltissues] on Nov 11, 2019, 16:43
mentioned in commit 078f8bae8af273490973aa86e867053b19cd7048


node.rb

Wonder if you are able to move default_interface within deploy_dnsmasq block.  We have a closed environment with no default routes, so ip route get will fail.  We deploy dnsmasq differently in our closed environment.

@taylok Thanks for that PR.
The default interface is needed only when playing with 3.6+ as the node itself need to bind on port 53 from that version. Therefore we remove the <IP_ADDRESS> from the dnsmasq binding to a dedicated one (usually the default interface)
That being said, I believe the best for us is to add that logic to the code:

[ ] Target environment < 3.6 --> <IP_ADDRESS> or customer's choice via variable
[ ] Target environment >= 3.6 --> default GW interface or customer's choice via variable

https://github.com/IshentRas/cookbook-openshift3/commit/649cdcac240f8033818e4b85d1fe35f148de9046


zsh: command not found: appcenter

Description
Please describe the issue you are facing using the CLI.
i have used this command -- npm install --location=global appcenter-cli
i am using MAC M1 chip and nvm along with it and having this issue of "zsh: command not found: appcenter "
zsh: command not found: appcenter
node -- default -> 18.1.0 (-> v18.1.0)

Hello @vksgautam1986 and thanks for reaching out to us, it seems your problem is related to your nvm setup. For a more detailed investigation, it would be great to get the appcenter-cli installation logs.

Hi, it seems that your problem is duplicate of https://github.com/microsoft/appcenter-cli/issues/460, so i will close this issue, but if you still have some questions feel free to reopen it.

netsmartz@Netsmartzs-Mac-mini ~ % yarn global add appcenter-cli
yarn global v1.22.19
warning package.json: No license field
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "appcenter-cli ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| 5.x.x".
warning "appcenter-cli > webpack-cli ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| 5.x.x".
[4/4] 🔨  Building fresh packages...
success Installed<EMAIL_ADDRESS>with binaries:
- appcenter
✨  Done in 2.11s.
netsmartz@Netsmartzs-Mac-mini ~ % - appcenter
zsh: command not found: appcenter
netsmartz@Netsmartzs-Mac-mini ~ %

@vksgautam1986, this is definitely issue with your local environment, I've tried to install appcenter-cli via yarn, and all works as expected:
anatoly@MacBook-Pro ~ % yarn global add appcenter-cli
yarn global v1.22.19
warning package.json: No license field
[1/4] 🔍  Resolving packages...
warning appcenter-cli > ms-rest ><EMAIL_ADDRESS>request has been deprecated, see https://github.com/request/request/issues/3142
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "appcenter-cli ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| 5.x.x".
warning "appcenter-cli > webpack-cli ><EMAIL_ADDRESS>has unmet peer dependency<EMAIL_ADDRESS>|| 5.x.x".
[4/4] 🔨  Building fresh packages...
success Installed<EMAIL_ADDRESS>with binaries:
      - appcenter
✨  Done in 21.28s.
anatoly@MacBook-Pro ~ % appcenter -v
appcenter version 2.14.0

Please make sure that your ZSH configured properly.

Hi @AnatolyPristensky. Here is my  profile.i have tried things but it did not worked out
alias node16='export
PATH="/usr/local/bin:/usr/local/opt/node@16/bin:$PATH"; node -v'
export PATH="/opt/homebrew/bin:$PATH"
export PATH="/opt/homebrew/lib/ruby/gems/2.7.0/bin:$PATH"
export PATH=$PATH:/Users/netsmartz/Library/Android/sdk/platform-tools
export PATH="/opt/homebrew/Cellar/bin:$PATH"
if which rbenv > /dev/null; then eval "$(rbenv init -)"; fi
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && . "$NVM_DIR/bash_completion"  # This load$

thanks guys its looks like its working fine now.
i tried with this command --  npm install -g appcenter-cli
after that i ran the app center command , it returned--
Visual Studio App Center helps you build, test, distribute, and monitor mobile apps.
Version 2.14.0
Usage: appcenter
Commands:
analytics                      View events, audience info, sessions, and
other analytics for apps
apps                           View and manage apps
build                          Start builds, get their status, and
download artifacts
codepush                       View and manage CodePush deployments and
releases
crashes                        Upload symbols for better crash reports
distribute                     Send builds to testers and manage
distribution groups
orgs                           Manage organizations
profile                        Manage your profile
telemetry                      Manage telemetry preferences
test                           Start test runs and get their status
tokens                         Manage API tokens
help                           Get help using appcenter commands
login                          Log in
logout                         Log out
setup-autocomplete             Setup tab completion for your shell


Upgrade swarms to Flatcar 3227.2.1

Same process as #783 :
Describe the solution you'd like
Run the ripple upgrade scripts across staging and all production swarms.
Start with Canary.
Make sure to add an extra node to staging0001 before starting run, then remove it once done.

staging0001-01 upgraded to 3227.2.1
narendrakumar@staging0001-01:~/secondaries-scripts/flatcar_upgrade$ ./staging0001_ripple_restart.sh
Target version is: 3227.2.1
Existing host keys found in /home/narendrakumar/.ssh/google_compute_known_hosts
Processing staging0001-03
Found staging0001-03 at VERSION=3227.2.1
staging0001-03 is already at target version
Processing staging0001-04
Found staging0001-04 at VERSION=3227.2.1
staging0001-04 is already at target version
Processing staging0001-05
Found staging0001-05 at VERSION=3227.2.1
staging0001-05 is already at target version
Processing staging0001-06
Found staging0001-06 at VERSION=3227.2.1
staging0001-06 is already at target version
Processing staging0001-07
Found staging0001-07 at VERSION=3227.2.1
staging0001-07 is already at target version

canary upgraded to 3227.2.1
narendrakumar@staging0001-01:~/secondaries-scripts/flatcar_upgrade$ ./canary_ripple_restart.sh
Target version is: 3227.2.1
Existing host keys found in /home/narendrakumar/.ssh/google_compute_known_hosts
Processing canary-02
Found canary-02 at VERSION=3227.2.1
canary-02 is already at target version
Processing canary-03
Found canary-03 at VERSION=3227.2.1
canary-03 is already at target version

canary upgraded to 3227.2.1
Target version is: 3227.2.1
Existing host keys found in /home/narendrakumar/.ssh/google_compute_known_hosts
Processing swarm0001-04
Found swarm0001-04 at VERSION=3227.2.1
swarm0001-04 is already at target version
Processing swarm0001-05
Found swarm0001-05 at VERSION=3227.2.1
swarm0001-05 is already at target version
Processing swarm0001-06
Found swarm0001-06 at VERSION=3227.2.1
swarm0001-06 is already at target version
Processing swarm0001-07

Moving to PR44. These upgrades could go a lot quicker by doing swarms in parallel.

swarm0002 upgraded to 3227.2.1
Processing swarm0002-18
Found swarm0002-18 at VERSION=3227.2.1
swarm0002-18 is already at target version
Processing swarm0002-19
Found swarm0002-19 at VERSION=3227.2.1
swarm0002-19 is already at target version
Processing swarm0002-20
Found swarm0002-20 at VERSION=3227.2.1
swarm0002-20 is already at target version
Processing swarm0002-21
Found swarm0002-21 at VERSION=3227.2.1
swarm0002-21 is already at target version

swarm0003 upgraded to 3227.2.1
Target version is: 3227.2.1
Existing host keys found in /home/narendrakumar/.ssh/google_compute_known_hosts
Processing swarm0003-04
Found swarm0003-04 at VERSION=3227.2.1
swarm0003-04 is already at target version
Processing swarm0003-05
Found swarm0003-05 at VERSION=3227.2.1
swarm0003-05 is already at target version
Processing swarm0003-06
Found swarm0003-06 at VERSION=3227.2.1
swarm0003-06 is already at target version

swarm0003 upgraded to 3227.2.1
Processing swarm0004-04
Found swarm0004-04 at VERSION=3227.2.1
swarm0004-04 is already at target version
Processing swarm0004-05
Found swarm0004-05 at VERSION=3227.2.1
swarm0004-05 is already at target version
Processing swarm0004-06
Found swarm0004-06 at VERSION=3227.2.1
swarm0004-06 is already at target version

@cpswan - We can close this ticket.


Body size greater than 50mb

Can I send a POST body size greater than 50mb? example:
app.use(express.json({ limit: '50mb', extended: true }));
app.use(express.urlencoded({ limit: '50mb', extended: true }));

Hi, the official documentation states https://expressjs.com/en/api.html
Controls the maximum request body size. If this is a number, then the value specifies the number of bytes; if it is a string, the value is passed to the bytes library for parsing.
--

Also that is what the error could look like for a post greater then 50mb    https://github.com/expressjs/body-parser#request-entity-too-large
Does this help answer your initial question?

I also found a similar question raised earlier on the body parser module. and essentially express.json is exposing bodyParser, AS express has it as dependency of it.  https://github.com/expressjs/body-parser/issues/176#issuecomment-226683434
So from what I see, I think you would probably only need the first line.

Closing this issue

Since I feel that the question posed has been answered.
No further follow up on the ticket.


dependencies

Are there any dependencies that should be installed before running these scripts. I am getting errors at multiple locations

it also giving me a lot of errors


Modify serialization of Kusto dynamic (JSON) properties so keys are in a consistent order

Places that have undefined order (not an exhaustive list):

CatalogLeafItemRecord.Deprecation
CatalogLeafItemRecord.Vulnerabilities
PackageAssembly.CustomAttributes
PackageAssembly.CustomAttributesFailedDecode

Basically any property that has [KustoType("dynamic")] is suspect.

Done with https://github.com/NuGet/Insights/commit/2d3c3e9d9e21b2e3aa27eb31f62018fbff57f6ce


3041 REPL may lose intellisense after reset

Make sure we find proper R buffer if multiples are attached

👍


FileNotFoundError: [Errno 2] No such file or directory: '/media/jian/ourdata/LHT/IST-NET/IST-Net-main/data/obj_models/camera_train.pkl'

Where is the address  obj_models/camera_train.pkl'

This problem seems that you didn't put the data to the path "/media/jian/ourdata/LHT/IST-NET/IST-Net-main/data/", you can solve this problem by using soft link


Add HLI scripts

@MSSedusch,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot


category is not work

build a category at adminpanle, it worked very nice in my computer .
But,after deploy app in aws ec2 , it was broken.
My domian is 9turn.com
The category url is http://9turn.com/9turn.comcategory/ios , it will return a 404 page.
My nginx conf is
server {
  listen                *:80;

  server_name           9turn.com;
  location / {
    proxy_pass http://<IP_ADDRESS>:3000;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header X-Forwarded-For $remote_addr;
  }
}

What should i do ?

Sounds like your site URL is not configured properly in the settings. Is it missing the last "/"?

Oh, yes i miss it .let me have a try.
Thank you~
There is another question:
the url can't have a "http://" header. Shoud i change nginx conf ?
meow~

dear , after i remove my site url , the setting works.
but if i only add "/" ,it couldn't work too.

@SachaG I was also a victim of this ;) - it's a very easy mistake to make.
It's not great that the app only errors at this point - ideally it would happen earlier or not at all. I suggest either:

add the trailing / on save if it's not already there OR
add the trailing / wherever siteURL is used (if it's not already there) OR
throw a validation error if it's not present when siteURL is saved

I'm trying to work out how you would do this, but I'm not familiar with how meteor works yet.

Yes, I'm working on a fix.


Running "exec:nodemon" fails under Windows 8.1 x64

Hi,
I'd like to share my findings about the problem I encountered while playing around with Brisket 0.29.0.

After setting up a simple project by following the README the Grunt task exec:nodemon refused to work:

'node_modules' is not recognized as an internal or external command,
operable program or batch file.
Exited with code: 1.
Warning: Task "exec:nodemon" failed. Use --force to continue.

Well, node_modules is never a command, if I'm not mistaken. :smile_cat:
So, I checked the Gruntfile.js and my first assumption was that the the problem was in the path of nodemon server.js (line 49).
Therefore I changed it from

cmd: 'node_modules/.bin/nodemon server.js'

to

cmd: './node_modules/.bin/nodemon server.js'

But, no luck. :sob:
My "solution" was to uninstall the preinstalled nodemon package from my project's node_modules/.bin and install it again, globally.
npm i -g nodemon
Now, grunt works.

Running "concurrent:server" (concurrent) task
Running "exec:nodemon" (exec) task
Running "watch" task
Waiting...
22 Feb 01:10:18 - [nodemon] v1.3.7
22 Feb 01:10:18 - [nodemon] to restart at any time, enter rs
22 Feb 01:10:18 - [nodemon] watching: .
22 Feb 01:10:18 - [nodemon] starting node server.js
My favorite town is Brisket Town
My favorite server is a plate
ServerApp has started
Brisket app is listening on port: 8080

hi @brakmic that's interesting - when i look up my global npm modules on my Mac, I don't see nodemon:
Waynes-MBP:brisket wayne$ npm list -g --depth=0
/usr/local/lib
├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>├──<EMAIL_ADDRESS>└──<EMAIL_ADDRESS>
My test app has nodemon in node_modules/.bin. I wonder why it would work differently on Windows?
By the way, very cool to know that Brisket runs on Windows. I've never tried it.

:+1: for Brisket
Regards,
Harris


Initial unmovable Keypoint on Top left

Hi,
I'm not sure if this is expected, but my GUI has a keypoint on the top left, which I can't seem to remove.

I'm not sure if its a problem with my skeleton file. I'm trying to annotate it to the AnimalPose (https://sites.google.com/view/animal-pose/) standard.
Here's the csv i'm using .
name,parent,swap
head,NaN,NaN
eyeL,head,eyeR
eyeR,head,eyeL
earL,head,earR
earR,head,earL
nose,head,NaN
neck,head,NaN
throat,neck,NaN
body,NaN,NaN
withers,body,NaN
tailbase,body,NaN
elbowFR,body,elbowFL
elbowFL,body,elbowFR
elbowBR,body,elbowBL
elbowBL,body,elbowBR
kneeFR,body,kneeFL
kneeFL,body,kneeFR
kneeBR,body,kneeBL
kneeBL,body,kneeBR
pawFR,body,pawFL
pawFL,body,pawFR
pawBR,body,pawBL
pawBL,body,pawBR

Sorry, didn't realise this is the expected behaviour, once all the points for the region are annotated, the top left point is removed automatically.


Memory leak with jQuery cache

Hello
I do have a problem with memory leaks in my application which uses bootstrap and gwt. I don't use gwtbootstrap3 in this project but I see the same problem on your demo page.
For components which use bootstrap js (Tooltip, Modal, ... ) a object is created in $.cache the problem is that this entry prevents the DOM objects from being garbage collected even if they are detached.
Regards Kenny

@crevete Have you seen this? Is it because the *.cache.js files?

@jgodi Yeah it's really a bug. I've tested a simple case : after calling the destroy method, the tooltip cached in $.cache is never removed. I guess it's a bug in bootstrap.js. Not sure it's due to *.cache.js files.
Anyway, there is also a bug in Tooltip.java : when the wrapping widget is detached from DOM, we destroy the tooltip but still re-create a new one.

@crevete This is most likely a bug with either the bootstrap or javascript libraries.  Since we no longer call recreate with Tooltip and Popover I don't see there being much need to investigate this further.  Do you have any thoughts?

@sjardine I agree with you. The same issue appears in the bootstrap home page: every visited widget is cached in $.cache. As the tooltip issue was fixed (a real bug), this could also be closed. We could re-open this later if necessary.

For reference, I think I created a reproducer upstream, https://github.com/twbs/bootstrap/issues/17973#issuecomment-202648725


[WIP] Move unresolved imports to the language server + related bug fixes

Move unresolved imports to the analysis server
Fix #4349 for VS Code: LastReportedDiagnostics isn't cleared after document is closed and reopened bug
Fix #4351 for VS Code: OpenFile._pendingChanges isn't cleared after document is closed
Fix #4363: Inconsistent behavior when showing errors in project items
Fixed memory leaks in ErrorTaskProvider

Fixes for #4349 and #4351 for VS require better synchronization between BufferParser.AddBuffer and BufferParser.RemoveBuffer that has to be converted into async to send and await for textDocument/didOpen and textDocument/didClose.

LGTM apart from the options setting

Yep, but let’s make it “from true to false” on the client side. The language server should be able to introduce new warnings without a client update. But I’d still like to have a “give me experimental warnings” option that doesn’t refer to each warning code individually.

Matching PR in vscode: https://github.com/Microsoft/vscode-python/pull/2049


Better error msg when defining collimator camera

Main changes:

Hello @dvezinet! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file tofu/data/_class8_check.py:

Line 970:42: W292 no newline at end of file


Separate API router from server.

Implement basic interfaces to write custom routers that can be plugged
to the server. Remove server coupling with the daemon.
This is how you implement a new router:
type myRouter struct{}
func (myRouter) Routes() []router.Route {
    fooHander := func(context.Context, http.ResponseWriter, *http.Request, vars map[string]string) error {
        // Do something here
    }

    return []router.Route{
      local.NewRoute("GET", "/foo", fooHandler),
    }
}

And then you can add the router to the server in the InitRouters fuction:
func (s *Server) InitRouters(d *daemon.Daemon) {
  // Previous defined routers here
  s.addRouter(myRouter{})
}

Signed-off-by: David Calavera<EMAIL_ADDRESS>

LGTM
@calavera but next time probably better to split moving stuff around to different pull request to not intimidate reviewers with size of changes :)

@LK4D4 I totally agree with you. I've closed several of my own PRs because they were too big and intimidating to review, even for myself. Sometimes, there is no other way unfortunately.

So this change would also allow things like this:
func (s *Server) InitRouters(d *daemon.Daemon) {
    if d.discoveryWatcher.Peers() > 1 { // I just made this up and I know it's not valid
        s.addRouter(swarm.NewRouter(d))
    } else {
        s.addRouter(local.NewRouter(d))
    }
}

:scream_cat:

It looks like the attach test is slightly flaky :pensive:
INFO: Testing against a local daemon
PASS: docker_cli_attach_unix_test.go:17: DockerSuite.TestAttachClosedOnContainerStop	1.441s
OK: 1 passed
PASS
coverage: 11.2% of statements

LGTM
Like I said, I'm open to keep the name local for the local router package or rename it to default.


[chip dv] Fix chip level DV with Xcelium...again

The first 3 commits are fixes to 3 separate issues. The last commit adds a new regression target for Xcelium CI that will eventually be used to run chip level sims with Xcelium in presubmit CI so that we will no longer have Xcelium related fallouts. One test is added at the moment - it will be scaled up once the backend infra is ready to accept more tests.

@johngt
@moidx
@mundaym
i think it's probably being discussed alrady, but we should get the xcelium stuff into CI soon.

@johngt @moidx @mundaym
i think it's probably being discussed alrady, but we should get the xcelium stuff into CI soon.

One chip level test with Xcelium is actually already running in CI now. I made an announcement on this on Slack.


Improve searching for correct cop candidates

Follow up of https://github.com/bbatsov/rubocop/pull/5300#issuecomment-353717122.
This commit changed the search approach of the correct cop candidates from prefix-match by regular expression to StringUtil.similarity.
Before
When there is no prefix-match, candidates are not displayed.
% rubocop --only Lint/RedundanWithI
Inspecting 1083 files

0 files inspected, no offenses detected
Unrecognized cop or department: Lint/RedundanWithI.

After
When there is no prefix-match, candidate is displayed.
% rubocop --only Lint/RedundanWithI
Inspecting 1083 files

0 files inspected, no offenses detected
Unrecognized cop or department: Lint/RedundanWithI.
Did you mean? Lint/RedundantWithIndex

Cop candidates different from user's expectations may be displayed, however I think that there is no problem in cop names of similarity to typing miss.
% rubocop --only Lint/RedundanWithO
Inspecting 1083 files

0 files inspected, no offenses detected
Unrecognized cop or department: Lint/RedundanWithO.
Did you mean? Lint/RandOne, Lint/RedundantWithObject

Other Information
Initially I thought about using the module of did_you_mean which is a standard library of Ruby 2.3 or higher. However, since the following code does not work unless it is Ruby 2.4 or higher, I decided to useStringUtil.similarity.
DidYouMean::SpellChecker.new(dictionary: cop_names).correct(name)

Also, since it displays multiple cop candidates, this commit did not use NameSimilarity#find_similar_name.

Before submitting the PR make sure the following are checked:

[x] Wrote good commit messages.
[ ] Commit message starts with [Fix #issue-number] (if the related issue exists).
[x] Feature branch is up-to-date with master (if not - rebase it).
[x] Squashed related commits together.
[ ] Added tests.
[ ] Added an entry to the Changelog if the new code introduces user-observable changes. See changelog entry format.
[x] The PR relates to only one subject with a clear title
and description in grammatically correct, complete sentences.
[x] Run rake default or rake parallel. It executes all tests and RuboCop for itself, and generates the documentation.

Nice!

Unrecognized cop or department: Lint/RedundanWithI.

One random note - this should be a different message if it's an unrecognized department IMO. It looks a bit odd to me right now. Guess I didn't pay it much attention in the past.


GUI Improvements (URL in GUI + Masked Connect Code Input)

URL can be set in GUI
Added a Config class that can persist data such as the entered URL
Made Submit (now Connect) button respond to the Enter-key
Connect Code Input box is now a MaskedTextBox and Connect can only be clicked if it contains 6 valid characters
Set Tab order

URL -> Connect Code -> Submit -> ...

Sorry for making a pull request on your fork. I only added a few things to it that I feel would make it better.

Thanks for addressing my points @nzbr (not sure how to tag here, sorry). I will review your changes again tomorrow

I have now moved ClientSocket creation back into Program.cs
In doConnect() (UserForm.cs) SendConnectCode() is called directly after Connect(). The SocketIO socket connects asynchronously. Because of that I made Connect async, so that I could await socket.ConnectAsync() and then Connect() itself to make sure that the socket is actually connected when SendConnectCode() is called.
Another option to avoid the async entirely is to make Connect() accept a callback that would be added to the things executed in socket.OnConnected
SendConnectCode() didn't really need to be async, so I removed that.
As for the discards: VisualStudio complains if a function that returns a Task is called in an asynchronous context but not awaited. Explicitly discarding the Task was the fix VS recommended to me

Pretty sure you can move the creation of the GameMemReader back to Program.cs because the sockets arent subscribed to the events until after they are connected. Maybe you could create a Public Event in ClientSocket that fires when the private socket's OnConnected event is fired? so you can know if they connected or not. Just some ideas.

@Stenlan are you still interested in merging this PR or can I close it?