feat: output explanation

I like the content and structure of the explanation, I noticed a few wordings that might be a bit confusing, check it out this markdown with a few changes in the wordings, and let me know what do you think? @Whomy09
⬇
Group by Rule: In this view, the first line is highlighted in blue, representing the rule being evaluated. Following this, you'll see blocks of information structured as follows: file path, description, and message.
Group by File: In this view, the first line displays the file path being evaluated. Below this, you'll see blocks of information with the following structure: rule, description, and message.
Rule: The blue line that defines the specific rule being evaluated.
File: Shows the path to the file where the rule is applied.
Description: A concise explanation of the rule, often accompanied by a link to relevant documentation for further details.
Message: Indicates the line of code where the rule was violated. Yellow messages are warnings, suggesting best practices, while red messages highlight errors that must be corrected.
⬆

Also, this issue was about adding the output explanation to the docs, not to the README
Maybe we can leave it in both places? @rrd108

Yes I think README should contain all the basics, including this.


Added Edge Computing Technology and Applications

Hi,
Stjepan from Manning here. I thought this title might be a good match for your list. Thank you for considering it.
Best,

Hi stjepanjurekovic,
it looks like this book is still in early access and the security part may not be written yet. Do you have insight into the outline of the security aspects that are to be covered?
Thanks for the contribution!


System.Object being discovered as EntityType through navigation.

I tried today to update my solution to RC1 from beta8 and get the exception The derived type 'FieldDefinitionEntity' cannot have keys other than those declared on the root type.. The root type is not mapped, only the derivied type. Should the mapping throw this exception anyway?
public class FieldDefinitionEntity : FieldDefinitionEntityBase
{
    public virtual bool MultiCulture { get; set; }
}

public abstract class FieldDefinitionEntityBase
{
    public virtual string FieldType { get; set; }
    public virtual string Id { get; set; }
}

Mapping
builder.Entity<FieldDefinitionEntity>(b =>
{
    b.ToTable("FieldDefinition", "Products");
    b.HasKey(p => p.Id);
});

The derived type 'AssortmentEntity' cannot have keys other than those declared on the root type. even that the AssortmentEntity not derived from any object. Using EF from myget v7.0.0-rc1-16305.
public class AssortmentEntity
{
    public virtual string ExternalId { get; set; }
    public virtual Guid Id { get; set; }
    public virtual ICollection<string> FieldDatas { get; set; }
}

With the mapping
builder.Entity<AssortmentEntity>(b =>
{
    b.ToTable("Assortment", "Products");
    b.HasKey(p => p.Id); // <- this line will throw exception
    b.HasAlternateKey(p => p.ExternalId);

    b.Property(p => p.ExternalId).HasMaxLength(100);
});

Result StackTrace:
at Microsoft.Data.Entity.Metadata.Internal.EntityType.AddKey(IReadOnlyList`1 properties)
   at Microsoft.Data.Entity.Metadata.Internal.InternalEntityTypeBuilder.<>c__DisplayClass14_0.<HasKey>b__1()
   at Microsoft.Data.Entity.Metadata.Internal.MetadataDictionary`2.GetOrAdd(Func`1 getKey, Func`1 createKey, Func`2 createValue, Func`2 onNewKeyAdded, ConfigurationSource configurationSource)
   at Microsoft.Data.Entity.Metadata.Internal.InternalEntityTypeBuilder.HasKey(IReadOnlyList`1 properties, ConfigurationSource configurationSource)
   at Microsoft.Data.Entity.Metadata.Internal.InternalEntityTypeBuilder.PrimaryKey(IReadOnlyList`1 properties, ConfigurationSource configurationSource)
   at Microsoft.Data.Entity.Metadata.Internal.InternalEntityTypeBuilder.PrimaryKey(IReadOnlyList`1 clrProperties, ConfigurationSource configurationSource)
   at Microsoft.Data.Entity.Metadata.Builders.EntityTypeBuilder`1.HasKey(Expression`1 keyExpression)
   at App.DataModel.<>c.<OnModelCreating>b__1_0(EntityTypeBuilder`1 b) in \Core\Products\DataModel.cs:line 89
   at Microsoft.Data.Entity.ModelBuilder.Entity[TEntity](Action`1 buildAction)
   at App.DataModel.OnModelCreating(DbContext dbContext, ModelBuilder builder) in \Core\Products\DataModel.cs:line 87
   at AppDbContext.OnModelCreating(ModelBuilder builder) in \Data\AppDbContext.cs:line 113
   at Microsoft.Data.Entity.Infrastructure.ModelSource.CreateModel(DbContext context, IConventionSetBuilder conventionSetBuilder, IModelValidator validator)
   at Microsoft.Data.Entity.Infrastructure.ModelSource.<>c__DisplayClass8_0.<GetModel>b__0(Type k)
   at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
   at Microsoft.Data.Entity.Infrastructure.ModelSource.GetModel(DbContext context, IConventionSetBuilder conventionSetBuilder, IModelValidator validator)
   at Microsoft.Data.Entity.Internal.DbContextServices.CreateModel()
   at Microsoft.Data.Entity.Internal.LazyRef`1.get_Value()
   at Microsoft.Data.Entity.Internal.DbContextServices.get_Model()
   at Microsoft.Extensions.DependencyInjection.EntityFrameworkServiceCollectionExtensions.<>c.<AddEntityFramework>b__0_5(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.FactoryService.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetService[T](IServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.EntityFrameworkServiceCollectionExtensions.<>c.<AddEntityFramework>b__0_0(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.FactoryService.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ClosedIEnumerableService.CallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.TransientCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Data.Entity.Storage.DatabaseProviderServices.GetService[TService]()
   at Microsoft.Data.Entity.Storage.RelationalDatabaseProviderServices.get_QueryContextFactory()
   at Microsoft.Extensions.DependencyInjection.EntityFrameworkServiceCollectionExtensions.<>c.<AddQuery>b__1_0(IServiceProvider p)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.FactoryService.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceLookup.ConstructorCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.ScopedCallSite.Invoke(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.<>c__DisplayClass12_0.<RealizeService>b__0(ServiceProvider provider)
   at Microsoft.Extensions.DependencyInjection.ServiceProvider.GetService(Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
   at Microsoft.Extensions.DependencyInjection.ServiceProviderExtensions.GetRequiredService[T](IServiceProvider provider)
   at Microsoft.Data.Entity.Infrastructure.AccessorExtensions.GetService[TService](IInfrastructure`1 accessor)
   at Microsoft.Data.Entity.Internal.InternalDbSet`1.<.ctor>b__2_0()
   at Microsoft.Data.Entity.Internal.LazyRef`1.get_Value()
   at Microsoft.Data.Entity.Internal.InternalDbSet`1.System.Linq.IQueryable.get_Provider()
   at System.Linq.Queryable.Select[TSource,TResult](IQueryable`1 source, Expression`1 selector)

Tried running both repro using rc1-final packages. First one gave no errors. Second one throws exception that FieldDatas is not added the model. which is expected.
Can you provide full model and context for repro?

@smitpatel The second exceptions seems to be related to the first one. I have put out all related parts in a new project, Project is working in beta8, in later versions it will throw exception. Have tested with the rc1 from aspnetrelease and rc2 from aspnetvnext myget feeds.
beta8: https://dl.dropboxusercontent.com/u/16590863/EF-DeriviedExceptionTest-beta8.zip
rc1: https://dl.dropboxusercontent.com/u/16590863/EF-DeriviedExceptionTest-rc1.zip
rc2-16128: https://dl.dropboxusercontent.com/u/16590863/EF-DeriviedExceptionTest-rc2-16128.zip

Thanks for the codes. The only change I had to make was fixing xunit.runner.aspnet in rc1 bits since it had wrong reference to xunit runner.
Everything works fine in beta8, following exception is thrown in rc1 & rc2
 System.InvalidOperationException : The derived type 'TestProject.FieldDefinitionEntity+FieldDataEntity' cannot have keys other than those declared on the root type.
with rc1 we have base type discovery convention added so the convention is assigning base type to FieldDataEntity but looking at the model, it should not be assigned.
I will debug more into it.

@smitpatel

With rc1 we have base type discovery convention added

We should only include base types in the model if they are explicitly configured. Are you saying we automatically pull base types into the model now?

I am debugging into it. The exception is thrown because we have set base type to System.Object by convention which is a bug.

Found the issue. The model entity type FieldDataEntity has property of type object, since it cannot be mapped by type mapper, we try to add it as navigation which works and creates entity type of System.Object. Convention will put that as base type for every other entity type.
Fix should be - do not discover object as navigation type since the target type is never valid entity type.

@smitpatel Agreed that it is safe to always discard System.Object as invalid but shouldn't we have logic already that figures out that System.Object is invalid because it doesn't have keys? :smile:

@divega - Our logic for discard entities without keys happen in ModelValidator. Since System.Object is base type for all clr types, the entity type for it will be set as base type for all other entity types which will cause exceptions before even model validator stage is reached.

Ok, understood that it works that way, but how would this case be different from any other base type that doesn't have keys and appears as a property in in the model? Or are you saying a model in which there is a property of type object is valid without an Ignore() on it? I think that is debatable :smile:

In general we don't know whether an entity type will have a primary key before discovering it.
But, in my opinion it is safe to assume System.Object will not.

Or are you saying a model in which there is a property of type System.Object is valid without an Ignore() on it?

If type mapper supports mapping object to primitive type, then it should be valid model. Else it becomes invalid model and will throw that property is not added to the model or marked as ignored.

I don't disagree we can special case System.Object, I just want to make sure that we don't miss other examples of the same issue with other type.

If type mapper supports mapping  object  to primitive type, then it should be valid model

Agreed... Hang on, System.Object with a key explicitly configured in shadow could be a valid entity type too! (Not that I would recommend anyone doing that and not my highest concern to enable it).

System.Object  with a key explicitly configured in shadow could be a valid entity type too!

And that will be base type of all other entity types in the model. :smile:
Hence everything will just merge into 1 large table (at least till we have only TPH)

We could use KeyDiscoveryConvention to test whether the PK would be discovered before trying to add the entity type to the model, but we decided not to do that since it would make it harder to find out why an entity type wasn't discovered when the user expected that it would

@Tasteful  - for the issue you reported,
With rc1 we have full TPH support now. #3441 added base type discovery convention which sets hierarchy for entity types added in the model if they are in clr hierarchy.
The model has property of type object since it cannot be mapped by typemapper it is being added as entity type and since object is clr base type for any other type, any other entity type will get base type set on it and HasKey throws exception.
For resolution,
Since object is not mapped by type mapper as primitive, either you should ignore the property if not being used or change data type to something else which can be mapped as primitive by type mapper.
In rare case if you are trying to use object as entity type, let us know what is the requirement for you causing it.

@smitpatel
The object property is converted into a json-string property before saving and reverse after loading and the property is included in the mapping as b.Ignore(p => p.JsonValue); but the type discovery convention is run before this is executed. I don't need to be able to map an object directly with a sql-column, should be with ms-sql2016 when we have a json datatype on the serverside and maybe can map object directly to the column.
The fix you was doing, will that popup in the aspnetrelease or aspnetvnext myget channel?
Will the change be included in rc1-final-package?

@Tasteful There will not be any Json datatype in SQL 2016... http://blogs.msdn.com/b/jocapc/archive/2015/05/16/json-support-in-sql-server-2016.aspx

@ErikEJ SQL2016 will have JSON support, even if the datatype in SQL will be nvarchar (regarding to the link you was referring to). But that article will not handle how the conversion between object and JSON is made in the .net side, that can be manual part as today or full automatic. I haven't dig deeper into that than I have read that SQL2016 will support JSON.

@Tasteful - The change will appear in nightly builds - that is aspnetvnext feed in couple of days. It will not be part of rc1 release though.
To unblock yourself, Try using [NotMapped] attribute on the property or  call Ignore before HasKey in OnModelConfiguring) that should ignore it before running relationship discovery. That should avoid the exception.
EF supports SQL2008 onwards, further to support a datastore type, we need conversion from data store type to CLR type so mapping json datatype to object is not just matter of SQLServer supporting json data type. EF has plan to add support for user-defined type mapping (#242) which would allow user to map any CLR type to any data store using user provided conversion for mapping.

I have the same problem. beta 8 worked, rc1-final not working... Here is my model:
    public class AllergenQuantity
     {
                public decimal Quantity { get; set; }
                //...model simplified
     }

public class AllergenQuantityMaterial : AllergenQuantity
{
    public Guid MaterialId { get; set; }
    public virtual Material Material { get; set; }

    public Guid AllergenId { get; set; }
    public virtual Allergen Allergen { get; set; }
}

public class Material
   public Material()
    {
        this.Id = Guid.NewGuid();
    }

    public virtual ICollection<AllergenQuantityMaterial> AllergenQuantityMaterial { get; set; }

    [Key]
    public Guid Id { get; set; }

    [Display(Name = "Název")]
    public string Name { get; set; }
}

 public class Allergen
{
    public Allergen()
    {
         this.Id = Guid.NewGuid();
     }
     public virtual ICollection<AllergenQuantityMaterial> AllergenQuantityMaterial { get; set; }

    [Key]
    public Guid Id { get; set; }

    public string Name { get; set; }
}

Fluent API:
builder.Entity().HasKey(l => new { l.MaterialId, l.AllergenId });
...AND here is error message:

Why i need it:
I have in my model many customized join tables derived from one table: AllergenQuantity...
public class AllergenQuantityUsed : AllergenQuantity
    {
      //here is only other many-to-many mapping
    }
public class AllergenQuantityMaximum: AllergenQuantity
    {
         //here is only other many-to-many mapping
    }

Of coarse similar pattern has been used by me for many other tables. I have complex model... You can use in Beta 8 one table with defined columns and next tables derived from it, until you can specify keys in derived tables... Now is this not working. Is it bug or new feature please... ????
Of coarse, I can write each class like this:
public class AllergenQuantityUsed
{
        public decimal Quantity { get; set; }
        //...AND HERE COPY ALL OF PROPERTIES BY HAND...
}

 public class AllergenQuantityMaximum
{
        public decimal Quantity { get; set; }
        //...AND HERE COPY ALL OF PROPERTIES BY HAND TOO...
}

.....but previous pattern was better...
Thank you

@miroslavsiska this seems to be a different issue, even if the symptoms are similar.
I believe what is happening is that Code First is bringing AllergenQuantity as an entity type to the model, becoming the base class of the inheritance hierarchy. The whole hierarchy is supposed to use the shame key definition. Most likely things worked in Beta8 because inheritance support was incomplete.
Since you have simplified your model there are a couple of things that aren't clear:

How does the base type get pulled into the model? This shouldn't happen unless you have a DbSet<AllergenQuantity> property or you call ModelBuilder.Entity<AllergenQuantity>() or there is a navigation property of type AllergenQuantity (or some collection of element type AllergenQuantity)

Does AllergenQuantity have a property that EF would match as an entity key by convention? If it doesn't then I am afraid this exception isn't clear.

cc @smitpatel

@miroslavsiska - that is a different scenario that the one being discussed in this issue. Can you open a new issue with details of your scenario.
Please be sure to include a complete code listing to reproduce the issue as I tried creating a console app and everything worked as expected...
using Microsoft.Data.Entity;
using System;
using System.Collections.Generic;

namespace Repro
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var db = new MyContext())
            {
                db.Database.EnsureCreated();

                var peanutButter = new Material { Name = "Peanut Butter" };
                db.Materials.Add(peanutButter);

                var peanuts = new Allergen { Name = "Peanuts" };
                db.Allergens.Add(peanuts);

                db.AllergenQuantityMaterials.Add(new AllergenQuantityMaterial
                {
                    Allergen = peanuts,
                    Material = peanutButter,
                    Quantity = 1000
                });

                db.SaveChanges();
            }
        }
    }

    public class MyContext : DbContext
    {
        public DbSet<Material> Materials { get; set; }
        public DbSet<Allergen> Allergens { get; set; }
        public DbSet<AllergenQuantityMaterial> AllergenQuantityMaterials { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder
                .Entity<AllergenQuantityMaterial>()
                .HasKey(l => new { l.MaterialId, l.AllergenId });
        }

        protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
        {
            optionsBuilder.UseSqlServer(@"Server=(localdb)\mssqllocaldb;Database=Repro;Trusted_Connection=True;");
        }
    }

    public class AllergenQuantity
    {
        public decimal Quantity { get; set; }
    }

    public class AllergenQuantityMaterial : AllergenQuantity
    {
        public Guid MaterialId { get; set; }
        public virtual Material Material { get; set; }

        public Guid AllergenId { get; set; }
        public virtual Allergen Allergen { get; set; }
    }

    public class Material
    {
        public Material()
        {
            Id = Guid.NewGuid();
        }

        public Guid Id { get; set; }
        public string Name { get; set; }
        public virtual ICollection<AllergenQuantityMaterial> AllergenQuantityMaterial { get; set; }
    }

    public class Allergen
    {
        public Allergen()
        {
            Id = Guid.NewGuid();
        }

        public Guid Id { get; set; }
        public string Name { get; set; }
        public virtual ICollection<AllergenQuantityMaterial> AllergenQuantityMaterial { get; set; }
    }

}

@divega
My complete classes:
public class AllergenQuantity
{
    public Guid UnitId { get; set; }
    public virtual Unit Unit { get; set; }

    [Display(Name = "Množství obsahu")]
    public decimal Quantity { get; set; }
}

 public class Unit
{
    public Unit()
    {
        this.Id = Guid.NewGuid();
    }
    public virtual ICollection<AllergenQuantity> AllergenQuantity { get; set; }

    [Key]
    public Guid Id { get; set; }

    [Display(Name = "Jednotka")]
    public string Name { get; set; }

    [Display(Name = "Typ Jednotky")]
    public string UnitType { get; set; }

    [Display(Name = "Použité v kulturách")]
    public string Cultures { get; set; }
}

  public class TenantDbContext : IdentityDbContext<TenantUser, TenantRole, Guid>
{
    private string _connectionString { get; set; }

    public TenantDbContext(DbContextOptions<TenantDbContext> options) : base(options)
    {
        this._connectionString = "CON STRING";
    }

    public TenantDbContext()
    {
    }
    public TenantDbContext(string ConnectionString)
    {
        this._connectionString = ConnectionString;
    }
    public static TenantDbContext Create(string ConnectionString)
    {
        return new TenantDbContext(ConnectionString);
    }

    protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
    {
        optionsBuilder.UseSqlServer(_connectionString);
    }

    //  BASE
    public DbSet<Unit> Unit { get; set; }

    //  Stock
    public DbSet<Allergen> Allergen { get; set; }
    public DbSet<AllergenQuantityMaterial> AllergenQuantityMaterial { get; set; }        //Inherited from AllergenQuantity
    public DbSet<Material> Material { get; set; }

    protected override void OnModelCreating(ModelBuilder builder)
    {
        base.OnModelCreating(builder);

        //builder.Ignore<AllergenQuantity>();   // IF UNCOMENTED =  OTHER EXCEPTION: Object reference not set to object instance...

        //Composites keys mapping
        builder.Entity<AllergenQuantityMaterial>().HasKey(l => new { l.MaterialId, l.AllergenId });
    }
}

If is this new Issue, I send you my complete solution... But according your comment. maybe is problem this:
public Guid UnitId { get; set; }
public virtual Unit Unit { get; set; }

Yes, I think it would be very useful to have this as a new issue.
From what I see it must be the Unit.AllergenQuantity collection navigation property what is causing AllergenQuantity to be included in the model.
If you are actually planning to use that navigation property to hold instances of different derived types of AllergenQuantity then you cannot get away with not having AllergenQuantity as part of the model and define the entity key on it. The other alternatives you have is to ask EF to ignore the property or to configure it to hold instances of a more specific derived type.
Regardless of that, the exception you are getting should be improved.
The NullReferenceException from the call to Ignore<T>() sounds like a new issue as well.

Yes, I think it would be very useful to have this as a new issue.

@miroslavsiska Sorry, what I meant is to ask you to open a new bug with this information. But don't worry. I can do it myself.

I recently ran across this problem, and tried to solve it by calling Ignore() before doing anything else in OnModelCreating(). Unfortunately, that doesn't solve the problem -- System.Object still gets discovered.
However, applying the NotMapped annotation to the property in the class definition does solve the problem.
It seems odd that the annotation approach works but the fluent approach does not. Not sure why that is.

@markolbert what version of the bits are you using? This was fixed post RC1.

Diego,
I’m using 7.0.0-rc1-final (the “stock” configuration that comes up when you create a new ASPNET5 web project in VS2015).
I’m open to trying something more recent. What’s the easiest way to do that?

     Mark

From: Diego Vega<EMAIL_ADDRESS>Sent: Friday, April 22, 2016 3:43 PM
To: aspnet/EntityFramework<EMAIL_ADDRESS>Cc: Mark Olbert<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [aspnet/EntityFramework] System.Object being discovered as EntityType through navigation. (#3727)
@markolberthttps://github.com/markolbert what version of the bits are you using? This was fixed post RC1.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHubhttps://github.com/aspnet/EntityFramework/issues/3727#issuecomment-213614693

@markolbert RC2 hasn't shipped the closest we have is staged in this NuGet feed: https://www.myget.org/F/aspnetrelease/api/v3/index.json
(More information about the different feeds at https://github.com/aspnet/Home/wiki/NuGet-feeds)
I don't think we have any comprehensive guidance on how to move an existing RC1 project to RC2 yet.


CVE-2008-4610 & CVE-2017-16803

CVE-2008-4610 & CVE-2017-16803 commit

Is this Logan Pelletier? I think you need to take the survey on step 1 of the assignment because I'm not recognizing this username.

Mr. Meneely,
Sorry about that. I ran into an issue with my personal account and school
accounts conflicting so I just granted access to both of them on the same
fork. Originally I checked no in the survey, but I will let Mr. Meyers know.
Sorry again for the confusion.
Thank you,
Logan
On Mon, Mar 15, 2021 at 3:46 PM Andy Meneely @.***>
wrote:

Is this Logan Pelletier? I think you need to take the survey on step 1 of
the assignment because I'm not recognizing this username.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/VulnerabilityHistoryProject/ffmpeg-vulnerabilities/pull/35#issuecomment-799703431,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO3NQ2IRM4NP6OOH55PRUJDTDZPYPANCNFSM4ZHB63YQ
.


Illegal ambiguous match on configurable attribute "deps" in //tensorflow/tools/pip_package:included_headers_gather

Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information

Ubuntu 18.04:

TensorFlow installed from source
TensorFlow version:
Python version: 3.6.7
Installed using virtualenv:
Bazel version : 0.22.0
GCC/Compiler version : 7.3
CUDA/cuDNN version: 7.4
GPU model and memory: RTX 2080 Ti

./configure
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".
INFO: Invocation ID: 9cf9c46b-c379-4fdc-a9f5-13ad76711061
You have bazel 0.22.0 installed.
Please specify the location of python. [Default is /home/peter/PycharmProjects/Virtual_envs/tensorflow/bin/python]:
Found possible Python library paths:
/home/peter/PycharmProjects/Virtual_envs/tensorflow/lib/python3.6/site-packages
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
Please input the desired Python library path to use.  Default is [/home/peter/PycharmProjects/Virtual_envs/tensorflow/lib/python3.6/site-packages]
Do you wish to build TensorFlow with XLA JIT support? [Y/n]: y
XLA JIT support will be enabled for TensorFlow.
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Please specify the CUDA SDK version you want to use. [Leave empty to default to CUDA 10.0]: 10.0
Please specify the location where CUDA 10.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7]: 7.4
Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:
Do you wish to build TensorFlow with TensorRT support? [y/N]: n
No TensorRT support will be enabled for TensorFlow.
Please specify the locally installed NCCL version you want to use. [Default is to use https://github.com/nvidia/nccl]:
Please specify a list of comma-separated Cuda compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus.
Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 7.5]:
Do you want to use clang as CUDA compiler? [y/N]: y
Clang will be used as CUDA compiler.
Do you wish to download a fresh release of clang? (Experimental) [y/N]: y
Clang will be downloaded and used to compile tensorflow.
Do you wish to build TensorFlow with MPI support? [y/N]: n
No MPI support will be enabled for TensorFlow.
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]:
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl         	# Build with MKL support.
--config=monolithic  	# Config for mostly static monolithic build.
--config=gdr         	# Build with GDR support.
--config=verbs       	# Build with libverbs support.
--config=ngraph      	# Build with Intel nGraph support.
--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws       	# Disable AWS S3 filesystem support.
--config=nogcp       	# Disable GCP support.
--config=nohdfs      	# Disable HDFS support.
--config=noignite    	# Disable Apache Ignite support.
--config=nokafka     	# Disable Apache Kafka support.
--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished
(tensorflow) peter@Peter-Ubuntu:~/tensorflow$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
bash: ./bazel-bin/tensorflow/tools/pip_package/build_pip_package: No such file or directory
(tensorflow) peter@Peter-Ubuntu:~/tensorflow$ ./bazel-bin/tensorflow/tools/pip_package/build_pip_package/tmp/tensorflow_pkg
bash: ./bazel-bin/tensorflow/tools/pip_package/build_pip_package/tmp/tensorflow_pkg: No such file or directory
(tensorflow) peter@Peter-Ubuntu:~/tensorflow$ bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
Starting local Bazel server and connecting to it...
INFO: Invocation ID: e9e2dd08-b8a0-4807-b1fe-cd6842a67a51
DEBUG: Rule 'build_bazel_rules_swift' modified arguments {"commit": "001736d056d7eae20f1f4da41bc9e6f036857296", "shallow_since": "1547844730 -0800"} and dropped ["tag"]
DEBUG: /home/peter/.cache/bazel/_bazel_peter/21121c77bac67ad4a1aed68d97d6cb00/external/build_bazel_rules_apple/apple/repositories.bzl:35:5:
WARNING: build_bazel_rules_apple depends on bazel_skylib loaded from https://github.com/bazelbuild/bazel-skylib.git (tag 0.6.0), but we have detected it already loaded into your workspace from None (tag None). You may run into compatibility issues. To silence this warning, pass ignore_version_differences = True to apple_rules_dependencies().
ERROR: /home/peter/tensorflow/tensorflow/tools/pip_package/BUILD:34:1: Illegal ambiguous match on configurable attribute "deps" in //tensorflow/tools/pip_package:included_headers_gather:
@local_config_cuda//cuda:using_nvcc
@local_config_cuda//cuda:using_clang
Multiple matches are not allowed unless one is unambiguously more specialized.
ERROR: Analysis of target '//tensorflow/tools/pip_package:build_pip_package' failed; build aborted:
/home/peter/tensorflow/tensorflow/tools/pip_package/BUILD:34:1: Illegal ambiguous match on configurable attribute "deps" in //tensorflow/tools/pip_package:included_headers_gather:
@local_config_cuda//cuda:using_nvcc
@local_config_cuda//cuda:using_clang
Multiple matches are not allowed unless one is unambiguously more specialized.
INFO: Elapsed time: 41.489s
INFO: 0 processes.

Tensorflow: 1.12

Please find the tested build configurations. TF 1.12 was tested and supported with Bazel 0.15.0. Could you follow the instructions here and let us know how it works. I recently used those steps and successfully installed tensorflow-gpu. If you had already resolved the issue, please close the issue or let us know. Thanks!

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

Closing this issue due to lack of recent activity. Please open a new ticker when new information is available. Thanks!


Update to the latest upstream API

Signed-off-by: Mrunal Patel<EMAIL_ADDRESS>

@runcom @feiskyer PTAL

LGTM in general, just a nit: it's better to split vendor and code in different commits.

@feiskyer That breaks tests and git bisect. Hence I have clubbed everything into the same commit.

@mrunalp Reasonable. LGTM

@runcom ping

LGTM


Move activation/mocks into activation package

Motivation
The packages activation/mocks and activation are inherently dependent on each other, since the activation package uses the mocks and the mocks use types from the activation package. This leads to the necessity of moving types that would belong in the activation package to activation/types.
This practice creates a lot of small packages that all have the same generic name (types, mocks, config, util etc.) that makes code that uses multiple of them confusing. config.Config says considerably less about what it is compared to activation.Config or hare.Config. Another example is mocks.NewBuilderMock vs. activation.NewBuilderMock.
Some handwritten mocks exist purely for the reason that generating them in a dedicated mocks package would introduce a cyclic dependency that isn't allowed in Go.
This PR only changes the activation package in this regard but IMO we should slowly move away from packages that are split based their content (types, mocks, config, util) towards packages that are split on their intended use (activation, hare, crypto, grpc, etc.).
This also allows us to reduce the surface of a package by making types and functions private that currently are not, just because of the imo unnecessary package split.
Changes
Merge package activation/mocks into package activation.
Test Plan
no functionality was changed, just code re-arranged.
TODO

[X] Explain motivation or link existing issue(s)
[X] Test changes and document test plan
[X] Update documentation as needed

DevOps Notes

[x] This PR does not require configuration changes (e.g., environment variables, GitHub secrets, VM resources)
[x] This PR does not affect public APIs
[x] This PR does not rely on a new version of external services (PoET, elasticsearch, etc.)
[x] This PR does not make changes to log messages (which monitoring infrastructure may rely on)

not sure i agree. i think our top level pkgs are already split on their intended use. config here means top level app config. inside the config pkg it does refer to tortoise.Config, hare.Config etc. i assume you are referring to types and util inside common pkg? that just means data that's shared across packages.

Take a look at config/config.go its inside a config package and it imports another 4 packages called config. All of them contain a function called DefaultConfig() so config.DefaultConfig() instead of eligibility.DefaultConfig(), hare.DefaultConfig(). Some packages there do not have this "config" split: tortoise, beacon and vm don't have a sub-package called config. Why do the others?
This is not intended as criticism to anyone. I just think this is a bad practice that we should move away from.

not sure i agree. i think our top level pkgs are already split on their intended use. config here means top level app config. inside the config pkg it does refer to tortoise.Config, hare.Config etc. i assume you are referring to types and util inside common pkg? that just means data that's shared across packages.

Take a look at config/config.go its inside a config package and it imports another 4 packages called config. All of them contain a function called DefaultConfig() so config.DefaultConfig() instead of eligibility.DefaultConfig(), hare.DefaultConfig(). Some packages there do not have this "config" split: tortoise, beacon and vm don't have a sub-package called config. Why do the others?

yes. i agree that there shouldn't be config sub pkg within each package. i don't like sub packages in general.

This is not intended as criticism to anyone. I just think this is a bad practice that we should move away from.

lol. it would be a criticism for me anyway hahah.... like i said, i agree we should do without sub packages within packages.

it would be a criticism for me anyway hahah

i mean it would NOT be :P

bors merge


C200 home assistant not showing feed

Description
Camera feed not updating, various resets and power down complete. Got feed to eventually show but not a live feed and not the right orientation .
Reproduction Steps
Camera feed not updating, various resets and power down complete. Got feed to eventually show but not a live feed and not the right orientation .
Expected behavior
Camera feed not updating, various resets and power down complete. Got feed to eventually show but not a live feed and not the right orientation .
If applicable, add error logs.
Camera feed not updating, various resets and power down complete. Got feed to eventually show but not a live feed and not the right orientation .
Device Firmware
Latest HACS
Using stream component
Yes
Does camera work via official integrations?
Yes
Camera has all attributes filled out in developer tools
Yes
HASS Environment
Home assistant
Search for similar issues
Yes
Additional information
Camera feed not updating, various resets and power down complete. Got feed to eventually show but not a live feed and not the right orientation .

Provide more information. There is a reason why there are different sections in the template.


How a selected Pokemon's typing?

Ok so for my project I want the user to be able to pick a pokemon and have the typing for the pokemon displayed but I am having issues.

So in the image this is how I am attempting to get the typing for Clefairy. I know it is wrong but when I tried to write in the same way that Clefairy's species is found I am getting errors.

I've been at this for hours and I am just trying to learn for personal practice so if anyone can help and explain how to get the typings for a Pokemon I would greatly appreciate it!

Knowing exactly what type you need to use for GetResourceAsync<T> is a little tricky and does require you to both closely examine the return types and refer to PokeAPI's documentation. The model classes very closely match the naming patterns of the associated server endpoints.
You'll want to use this for getting a Pokemon's types:
var id = "clefairy";
var pokemon = await pokeClient.GetResourceAsync<Pokemon>(id);

// pokemon.Types and pokemon.PastTypes for historical typing
pokemon.Types[0].Type.Name == "fairy";

Thank you so much! I got it to work now! Learning different return types is quite tricky.


Path Finding Started to Return Invalid Paths in Horizon 1.4.0 (op_too_few_offers)

What version are you using?
Horizon 1.4.0
What did you do?
I'm obtaining paths from /paths/strict-receive and then use the most economic path in a path-payment-strict-receive operation. This has been working fine up until 1.3.0 but now we're intermittently (and increasingly often) seeing op_too_few_offers when the best path is used. Looking at the returned paths from /paths/strict-receive they seem implausible actually. I think there might be an issue with pathfinder, which started with version 1.4.0.
What did you expect to see?
I'd expect paths given be the pathfinder to work fine and not result in op_too_few_offers when used in a path payment operation.
What did you see instead?
Tx fails with op_too_few_offers.
Example. /paths/strict-receive finds this path, which then results in op_too_few_offers if used:
{
   "sourceAssetType":"credit_alphanum4",
   "sourceAssetCode":"ETH",
   "sourceAssetIssuer":"GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
   "sourceAmount":"0.0414144",
   "destinationAssetType":"credit_alphanum4",
   "destinationAssetCode":"USD",
   "destinationAssetIssuer":"GDUKMGUGDZQK6YHYA5Z6AY2G4XDSZPSZ3SW5UN3ARVMO6QSRDWP5YLEX",
   "destinationAmount":"10.0000000",
   "path":[
      {
         "assetType":"credit_alphanum4",
         "assetCode":"ETH",
         "assetIssuer":"GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5"
      },
      {
         "assetType":"credit_alphanum4",
         "assetCode":"ETH",
         "assetIssuer":"GCNSGHUCG5VMGLT5RIYYZSO7VQULQKAJ62QA33DBC5PPBSO57LFWVV6P"
      },
      {
         "assetType":"native",
         "assetCode":"XLM",
         "assetIssuer":null
      }
   ]
}

Upon manual inspection, this does not look like a plausible path: ETH -> {ETH -> ETH -> XLM} -> USD and correctly got rejected by the Network with tx_failed and op_too_few_offers.
Normally in 100% of the cases this would be a bad trade. Trading tethers horizontally across several issuers always is a loss of money. So it looks like pathfinder indeed finds invalid paths and then core rejects them because they don't actually exist and the tx fails.
Please let me know if you need more info. @bartekn @abuiles

@marcinx that's for the report we'll check it out. If you see something like this again, could you also add Latest-Ledger header from the HTTP response to the issue? It will help us bring orderbook graph to the ledger in which the path existed.

@tamirms if the fix is easy let's try to include it in 1.6.0.

Thanks, @bartekn! I added some additional logging to capture the ledger. Will post here when I capture some extra data.

Not sure if related or if I am wrong here but I seem to remember that the --max-path-length=4 setting previously used to count the source asset and destination asset. With above setting, the results used to have a maximum of two assets in the found paths: sourceAsset -> pathAsset1, -> pathAsset2 -> destinationAsset (=4 total assets; =max-path-length). I am almost 99% sure that this used to be the case in the past.
Now I am observing up to three path assets, i.e. sourceAsset -> pathAsset1 -> pathAsset2 -> pathAsset3 -> destinationAsset (see the path pasted in the issue description). Is this correct behavior? Is max-path-length supposed to count only the path assets or sourceAsset and destinationAsset as well ? @bartekn @tamirms @abuiles

@marcinx I think it doesn't count the source so the length is actually 4:
ETH -> {ETH -> ETH -> XLM} -> USD
        1      2      3       4

Caught another such case today. Here is some extra info:
Request: GET /stardust/v1/paths/strict-receive?sourceAccount=GBDHWMSXQ24FA2LJTPUH3HNK3WR7BJW3DUUBLMSNSJ7ST4XZK7WJHPCQ&destinationAssetType=credit_alphanum4&destinationAssetCode=EURT&destinationAssetIssuer=GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S&destinationAmount=10.0000000 (this proxies the params to Horizon)
Complete Horizon Response Body: https://gist.github.com/marcinx/f3b3fbfadf9535822f9e5ab6472f7bce
Most Economic Apay ETH -> Tempo EURT path: {    "sourceAssetType":"credit_alphanum4",    "sourceAssetCode":"ETH",    "sourceAssetIssuer":"GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",    "sourceAmount":"0.0476752",    "destinationAssetType":"credit_alphanum4",    "destinationAssetCode":"EURT",    "destinationAssetIssuer":"GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S",    "destinationAmount":"10.0000000",    "path":[       {          "assetType":"credit_alphanum4",          "assetCode":"ETH",          "assetIssuer":"GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5"       },       {          "assetType":"credit_alphanum4",          "assetCode":"ETH",          "assetIssuer":"GCNSGHUCG5VMGLT5RIYYZSO7VQULQKAJ62QA33DBC5PPBSO57LFWVV6P"       },       {          "assetType":"native",          "assetCode":"XLM",          "assetIssuer":null       }    ] }
Ledger from pathfinder http response headers: 'latest-ledger': '30609467'
When trying to use this path in ledger 30609468 with a path-payment-strict-receive, the response is: op_too_few_offers.
This happens intermittently.
Hope it helps! Please let me know when you have more questions. @bartekn @tamirms @abuiles

@marcinx I started v1.4.0 horizon locally and ingested ledgers up to 30609467. When I sent the following curl request:
curl -v "http://localhost:8000/paths/strict-receive?source_assets=ETH:GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR&destination_asset_type=credit_alphanum4&destination_asset_code=EURT&destination_asset_issuer=GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S&destination_amount=10.0000000"
I got a different result than the one you have. The first entry in my result was:
       {
        "source_asset_type": "credit_alphanum4",
        "source_asset_code": "ETH",
        "source_asset_issuer": "GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
        "source_amount": "0.0478696",
        "destination_asset_type": "credit_alphanum4",
        "destination_asset_code": "EURT",
        "destination_asset_issuer": "GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S",
        "destination_amount": "10.0000000",
        "path": [
          {
            "asset_type": "credit_alphanum4",
            "asset_code": "USDT",
            "asset_issuer": "GCQTGZQQ5G4PTM2GL7CDIFKUBIPEC52BROAQIAPW53XBRJVN6ZJVTG6V"
          },
          {
            "asset_type": "native"
          }
        ]
      }

Here is the full response from the curl request: https://gist.github.com/tamirms/d530f73c479886bffe0b14c1dd078e6d
are you doing any sort of caching when proxying requests to horizon? The path finding code should be completely deterministic. Given the same ledger state the path finder should always return the same set of results. So, I'm a bit confused as to why your response differs from mine

@tamirms No caching whatsoever.
Our request is different though because we query all source assets (trustlines) on a given source account. We take a source account and then do this:
            sourceAccount.balances.forEach(function(balance) {

                if (balance.asset_type == 'native') {
                    sourceAssets += 'native';
                } else {
                    sourceAssets += balance.asset_code + ':' + balance.asset_issuer
                }

            });

In your example source_assets is ETH:GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR, but in our request, source_assets are all trustlines (and XLM) on a given source account.
Can you re-create the issue when using the following as source_assets?
BRL:GDVKY2GU2DRXWTBEYJJWSFXIGBZV6AZNBVVSUHEPZI54LIS6BA7DVVSP,BTC:GAUTUYY2THLF7SGITDFMXJVYH3LHDSMGEAKSBU267M2K7A3W543CKUEF,BTC:GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5,CAD:GBLR5MVHI7PCN24JIHDR5LUYFBHSA2IOVPZP7F75L7PIF3WU3TQGQC3P,ETH:GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR,ETH:GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5,EURT:GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S,LTC:GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5,LTC:GC5LOR3BK6KIOK7GKAUD5EGHQCMFOGHJTC7I3ELB66PTDFXORC2VM5LP,NGNT:GAWODAROMJ33V5YDFY3NPYTHVYQG7MJXVJ2ND3AOGIHYRWINES6ACCPD,USD:GDSRCV5VTM3U7Y3L6DFRP3PEGBNQMGOWSRTGSBWX6Z3H6C7JHRI4XFJP,USD:GDUKMGUGDZQK6YHYA5Z6AY2G4XDSZPSZ3SW5UN3ARVMO6QSRDWP5YLEX,XRP:GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5,native

@tamirms In our response it finds the path you posted above as third most economic option: https://gist.github.com/marcinx/f3b3fbfadf9535822f9e5ab6472f7bce
Maybe the additional source assets (we have two source ETH with different issuers) are causing this somehow?

@marcinx here's my response using the full list of source assets: https://gist.github.com/tamirms/3778b07fd5c9b228f3d798bd922cd0f2
as you can see it still does not match your response. I cannot find any entry where the source amount is 0.0476752 with source asset ETH:GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR

@tamirms is it possible that OrderBookStream that was added in 1.4.0 is causing inconsistencies? Maybe the Latest-Ledger header is not updated properly (it's loaded from a DB, not a graph?). Maybe we should check a range of ledgers, like 10 before, 10 after: [30609457, 30609477].

@tamirms I'll add some more logging where I capture the exact request URL sent to Horizon along with the response headers and body. I'll post here when I catch another such case. They seem to appear daily.

@marcinx I just realized that I was using the default max-path-length which is 3 and you are using a max-path-length value of 4. After changing the configuration to match yours I'm able to observe the 0.0476752 ETH:GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR path in my response

@tamirms Okay, great! I guess now we just need to exclude the possibility of random front-running/out-running of the found path in the next ledger to be 100% sure this is a path finder issue? I find this explanation highly unlikely but it could theoretically be possible. From my manual observations and tests it is most likely a non-existent path.

@marcinx
do you have a link to the transaction containing your failed path payment? It would be helpful to know which ledger the network was at when your transaction was processed.
At ledger 30609467 the following path was available:
{
        "source_asset_type": "credit_alphanum4",
        "source_asset_code": "ETH",
        "source_asset_issuer": "GBDEVU63Y6NTHJQQZIKVTC23NWLQVP3WJ2RI2OTSJTNYOIGICST6DUXR",
        "source_amount": "0.0476752",
        "destination_asset_type": "credit_alphanum4",
        "destination_asset_code": "EURT",
        "destination_asset_issuer": "GAP5LETOV6YIE62YAM56STDANPRDO7ZFDBGSNHJQIYGGKSMOZAHOOS2S",
        "destination_amount": "10.0000000",
        "path": [
          {
            "asset_type": "credit_alphanum4",
            "asset_code": "ETH",
            "asset_issuer": "GBVOL67TMUQBGL4TZYNMY3ZQ5WGQYFPFD5VJRWXR72VA33VFNL225PL5"
          },
          {
            "asset_type": "credit_alphanum4",
            "asset_code": "ETH",
            "asset_issuer": "GCNSGHUCG5VMGLT5RIYYZSO7VQULQKAJ62QA33DBC5PPBSO57LFWVV6P"
          },
          {
            "asset_type": "native"
          }
        ]
 }

The source amount on the ETH -> {ETH -> ETH -> XLM} -> EURT path decreased from 0.0476752 to 0.0476656 on the next ledger 30609467.
That path remained available up until ledger 30609471.
On ledger 30609471 the source amount increased to 0.0476833.

@tamirms The tx should be sent in the ledger right after when this path was found. It's literally the next thing that happens in the code. As soon as we have a pathfinder response we take the best path and send the path-payment-strict-receive.
It's worth pointing out that we are adding 1.5% slippage padding to the source_amount in the path_payment operation. 0.0476752 found in the best path becomes 0.0483903 in the sendMax for the operation. So even if the required source amount increases somewhat to 0.0476752 to 0.0476656 the op should be able to execute.
I actually tried to find the failed tx manually a while ago but it doesn't seem to show up. Our Horizon doesn't ingest failed tx (yet) but I had no luck on horizon.stellar.org either. It should be somewhere on GBDHWMSXQ24FA2LJTPUH3HNK3WR7BJW3DUUBLMSNSJ7ST4XZK7WJHPCQ for the ledger(s) immediately after 30609467.

@tamirms PS: It's more complicated than that. The tx is sent from a channel account (which can be any of these):
GBBMI477CH2XZQ53R5PLTB5PUUPZEARARRF5LFEYOUEVWIC6J3WXWTN5
GBGG4PZQFXZYF3OUPB5P4Z5SDYY6QS3YAB3EQPIOBTG4J3BGA3JMXDOR
GBIWIMXSMFXDK44VUNFUHSCUJVK7UJUAD5J7TUJOYQIKRGHAHPWLSYZ6
GC2AEFWNZJ6YFZO4HGQ5DW3P6DBHJ6QDTPFSOTWOTSSHJZD56VC3XOSK
GC4WSVHVHYWR2KF4Z7AMTOB7M3C2P67TQ4JWWW44DSS7LUFNUT2VF6RL
GD4XE5WQTJLFDY67XDN5SOIQVPLWUCLZ5UPOFFKOZKNAC6GV3WRHCS6M
GD776TFY6ECG6K2SZC3Q26RZDIXF7KLRRNKGOWGUDRSNK4CUITLUPPZU
GDRLNPHYGF3AYG2Q2NXHMXVWGMRA6KIMAL7UJ2O3CDHSFZ2TWOQNMDS5
GDV3ZOYWTAUR2VLHB2DM77AM5EDC5KU6UKLC5VDPAVDRNQJYKELPUNK5
GDXMVK2NJRND72NIJ2XF37VWLKP2YNA64IBSSGVTY7B5KAIVECZCM3LT

So when looking for the failed tx, it should have either of above as source account. The source account for the op should be GBDHWMSXQ24FA2LJTPUH3HNK3WR7BJW3DUUBLMSNSJ7ST4XZK7WJHPCQ .
However, horizon.stellar.org doesn't seem to display failed transactions? When I go https://horizon.stellar.org/ledgers/30609467 it tells me that there are 31 successful and 10 failed txs. When I visit https://horizon.stellar.org/ledgers/30609467/transactions?limit=200 it only has the 31 successful txs but no fails.
I tried, but really don't know how to find the failed one without doing a major database query operation, which I would have to research how to do in the first place :) ¯_(ツ)_/¯ Maybe there is another way to verify that the suspicious path actually existed?

@marcinx you need to include_failed
https://horizon.stellar.org/ledgers/30609467/transactions?limit=200&include_failed=true
On Wed, Jul 15, 2020 at 2:34 PM marcinx<EMAIL_ADDRESS>wrote:

@tamirms https://github.com/tamirms PS: It's more complicated than
that. The tx is sent from a channel account (which can be any of these):
GBBMI477CH2XZQ53R5PLTB5PUUPZEARARRF5LFEYOUEVWIC6J3WXWTN5
GBGG4PZQFXZYF3OUPB5P4Z5SDYY6QS3YAB3EQPIOBTG4J3BGA3JMXDOR
GBIWIMXSMFXDK44VUNFUHSCUJVK7UJUAD5J7TUJOYQIKRGHAHPWLSYZ6
GC2AEFWNZJ6YFZO4HGQ5DW3P6DBHJ6QDTPFSOTWOTSSHJZD56VC3XOSK
GC4WSVHVHYWR2KF4Z7AMTOB7M3C2P67TQ4JWWW44DSS7LUFNUT2VF6RL
GD4XE5WQTJLFDY67XDN5SOIQVPLWUCLZ5UPOFFKOZKNAC6GV3WRHCS6M
GD776TFY6ECG6K2SZC3Q26RZDIXF7KLRRNKGOWGUDRSNK4CUITLUPPZU
GDRLNPHYGF3AYG2Q2NXHMXVWGMRA6KIMAL7UJ2O3CDHSFZ2TWOQNMDS5
GDV3ZOYWTAUR2VLHB2DM77AM5EDC5KU6UKLC5VDPAVDRNQJYKELPUNK5
GDXMVK2NJRND72NIJ2XF37VWLKP2YNA64IBSSGVTY7B5KAIVECZCM3LT
So when looking for the failed tx, it should have either of above as
source account. The source account for the op should be GBDHWMSXQ24FA2LJTPUH3HNK3WR7BJW3DUUBLMSNSJ7ST4XZK7WJHPCQ
.
However, horizon.stellar.org doesn't seem to display failed transactions?
When I go https://horizon.stellar.org/ledgers/30609467 it tells me that
there are 31 successful and 10 failed txs. When I visit
https://horizon.stellar.org/ledgers/30609467/transactions?limit=200 it
only has the 31 successful txs but no fails.
I tried, but really don't know how to find the failed one without doing a
major database query operation, which I would have to research how to do in
the first place :) ¯_(ツ)_/¯ Maybe there is another way to verify that the
suspicious path actually existed?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/stellar/go/issues/2810#issuecomment-658964666, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAFKDG7KLULPW2XRWVABD3R3YACTANCNFSM4OYRVMHA
.

 It's missing the first asset.
Sigh. It looks like it's an issue on our end then. Sorry for wasting your time. I'm glad it's not an issue on Horizon though 💪

@marcinx no worries :)
one tip which may be helpful:
you can log the transaction hash in your backend whenever a transaction fails. then, if you need to debug why the transaction failed, you will be able to easily look up the transaction details via horizon

@marcinx glad to hear this is not a Horizon bug. For the future, anything you can do to provide a repro out of the gate (e.g. exact Horizon call + failed txhash) would be awesome. We spent some time on this but turnaround could be a lot faster next time with a little more info up front.

@marcinx glad to hear this is not a Horizon bug. For the future, anything you can do to provide a repro out of the gate (e.g. exact Horizon call + failed txhash) would be awesome. We spent some time on this and turnaround could be a lot faster next time with a little more info up front.

Yeah, I can imagine. I feel bad for wasting your time. It seemed so obvious at first, considering that path finding gave wrong results in the very past. Contrary to that it looks very robust and fast now though 💯Will triple-check better and provide more meta out of the box next time I run into any issues. Thanks for helping to find the root cause of this!!


Replaced isinstance test by field attribute

Did you have a particular motivation? See #24227 for some discussion.

No, I just stumbled over this while browsing code for another issue. Do you think we could commit this (piece meal) or would you like to do at all at once with #24227? Feel free to close the PR in the latter case.

I don't know. I think it's difficult to make a decision about the replacements in general without seeing how custom fields with  use the flags. For example, it looks like this change would impose the requirement that a custom field with many_to_many=True have a value_from_object() method (currently undocumented) that return a queryset. Should we be concerned and/or document requirements like this? If we start changing some isinsatnce checks but not others, it could be confusing to authors trying to make custom fields work.

Let's close it for now.


Update open_source_dev_contract.md

@User2564 I think you missed my last comment.
"DeveloperConsulatnt" shouldn't it be written separately?
On that note wouldn't simply Developer be more accurate?


FMWK-393 Support of extra-ttl flag

added extra-ttl support

Codecov Report
Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Project coverage is 83.00%. Comparing base (45f53a9) to head (8d3edec).

Files
Patch %
Lines

config_restore.go
0.00%
1 Missing and 1 partial :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #127      +/-   ##
==========================================
- Coverage   83.01%   83.00%   -0.02%
==========================================
  Files          63       63
  Lines        3480     3483       +3
==========================================
+ Hits         2889     2891       +2
  Misses        372      372
- Partials      219      220       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Move tests into package to include them into distribution

Moving tests to test folder.

I think the problem is that there is no explicit broadbean in the calls to pytest.
e.g.
pytest --cov=broadbean --cov-report xml --hypothesis-profile ci

should be
pytest --cov=broadbean --cov-report xml --hypothesis-profile ci broadbean

Tried running locally but I can't recreate the test failure on my machine, @Dominik-Vogel we can dig into this when we chat tomorrow morning.

Codecov Report

Merging #108 (c712dcb) into master (0e57973) will increase coverage by 79.99%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           master     #108       +/-   ##
===========================================
+ Coverage    0.00%   79.99%   +79.99%
===========================================
  Files           8       15        +7
  Lines        1494     2359      +865
===========================================
+ Hits            0     1887     +1887
+ Misses       1494      472     -1022

Impacted Files
Coverage Δ

broadbean/tests/conftest.py
100.00% <ø> (ø)

broadbean/tests/test_awgfilegeneration.py
97.14% <ø> (ø)

broadbean/tests/test_blueprint.py
100.00% <ø> (ø)

broadbean/tests/test_sequence.py
100.00% <ø> (ø)

broadbean/tests/test_subsequences.py
100.00% <ø> (ø)

broadbean/tests/test_element.py
100.00% <100.00%> (ø)

broadbean/tests/test_forging.py
96.59% <100.00%> (ø)

... and 11 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

The conda test yaml also needs to be updated to explicitly include hypothesis


Deprecation warning pandas - Float

c:\users\sayala\documents\github\bifacial_radiance\bifacial_radiance\performance.py:70: FutureWarning: Calling float on a single element Series is deprecated and will raise a TypeError in the future. Use float(ser.iloc[0]) instead
Adjust=float(CECMod.Adjust)

This was fixed by #527


Material icons include incorrect spellings of 'fluorescent'

Our material icons constants include four icons with incorrect spellings:

flourescent
flourescent_sharp
flourescent_rounded
flourescent_outlined

Note the reversal of 'uo' for 'ou'. Correcting these would be a breaking change for any users of the existing constants. Instead we should add alternatives with the correct spellings and mark the incorrect ones deprecated. The file is automatically generated using the process described at go/effortless-flutter-font-updates (Google-only link), so this would likely involve supplying the tooling with a means to generate both names,with the misspelling marked as deprecated.
/cc @HansMuller @guidezpl

Filed internal bug b/227715546

Closing as obsolete, by https://github.com/flutter/flutter/issues/102560


DatasourceItem does not have webpage_url attribute

According to the documentation, DatasourceItem has a webpage_url attribute. The corresponding tableau server rest api docs also appears to return this field.
However, this attribute does not exist in the python client, even though the docs say it does:
>>> datasources, _ = server.datasources.get()
>>> datasources[0].website_url
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'DatasourceItem' object has no attribute 'website_url'

Hi @snazzyfox, apologies for the confusion, the documentation was updated a bit early. We are planning on a new release soon, which will support datasource.webpage_url.

This should now be available with v0.14.0.


[FLINK-13688][hive] Limit the parallelism/memory of HiveCatalogUseBlinkITCase

What is the purpose of the change
limit the parallelism of HiveCatalogUseBlinkITCase to avoid too many slot requests by default parallelism (use the core size of machine).
Verifying this change
This change is already covered by existing tests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no

Another way is to extend AbstractTestBase, which will configure mini cluster in a more predictable way.

And I'm not sure the root cause is the parallelism is too high for this case. We should support running very high parallelism batch job with only one slot by design.

@KurtYoung Yeah, the main problem is memory request is too high(128MB).  I think 8dacb85c1e1e1bbd7a31c91800b886eb62be2c80 will solve it.

How about we use a pre configured mini cluster instead of lower the memory request? Because it's not 100% safe to just rely on lowering the sort memory, because we might choose hash aggregate.

How about we use a pre configured mini cluster instead of lower the memory request? Because it's not 100% safe to just rely on lowering the sort memory, because we might choose hash aggregate.

Yeah, that is a way too, but in this case, I think use lower memory is OK. Because the data is too few.
1.It must rely on sort now, because UDAF only can be support by sort agg.
2.I think I can extract the configuration in BatchTestBase, to config all memory things.

I would like to repeat my first question: why not just extending  AbstractTestBase

I would like to repeat my first question: why not just extending  AbstractTestBase

OK.

This case makes me feel that the first time a user uses blink batch sql, he has to think about resources.....

+1 for fixing it. @KurtYoung @JingsongLi

Verified locally, +1.


Conda environment install not working

Tried installing conda environment on a new (Windows) computer and got:
Solving environment: failed

ResolvePackageNotFound:
  - conda-forge::scipy=1.4.1

It also gives this warning while attempting the install:
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.

The last warning is ok - I see it all the time even after adding pip - and it never breaks anything.
Also, looking at the conda-forge scipy distributions it looks like windows versions are pretty far behind.

Can you try replacing with - conda-forge::scipy=1.3.2?

Actually, after typing this I realize that we don't use scipy at all in the welding procedure. It is safe to remove to reduce dependencies!


first version of about page content

Overview
Adds about page content and link in navigation based on our Google Doc: https://docs.google.com/document/d/1qXuioHvuRF28oBrKD_iPPiQRbiQw_A4kx3FU245e3go/edit#heading=h.6gf8wfi5rz54
Closes #55
Testing Instructions

confirm the 'About' link displays and works in the top navigation
read through the content and confirm links/formatting look ok

Agreed with Victor's comments, beyond that lgtm!

@vkoves thx for the review - addressed your comments!


tab moves focus from editor area when 'Toggle Tab Key Moves Focus' is  disabled

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.85.0 (user setup)
OS Version: Windows_NT x64 10.0.19045

Steps to Reproduce:

Toggle off Toggle Tab Key Moves Focus
Press the Tab key in the editor area
The focus moves to other pane

I can't use the tab key to in the code area due to it changing focus to other panes even when the Tab Key moves focus option is toggled off.
I tried to disable all extensions and that didn't help either.
Output of Help: About:
Version: 1.85.0 (user setup)
Commit: af28b32d7e553898b2a91af498b1fb666fdebe0c
Date: 2023-12-06T20:48:09.019Z
Electron: 25.9.7
ElectronBuildId: 25551756
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: <IP_ADDRESS>-electron.0
OS: Windows_NT x64 10.0.19045

Output when I'm pressing tab using Keyboard Shortcuts Troubleshooting:
2023-12-12 15:06:14.558 [info] [KeybindingService]: / Soft dispatching keyboard event
2023-12-12 15:06:14.559 [info] [KeybindingService]: | Resolving Tab
2023-12-12 15:06:14.559 [info] [KeybindingService]: \ From 12 keybinding entries, no when clauses matched the context.
2023-12-12 15:06:14.559 [info] [KeybindingService]: / Received  keydown event - modifiers: [], code: Tab, keyCode: 9, key: Tab
2023-12-12 15:06:14.559 [info] [KeybindingService]: | Converted keydown event - modifiers: [], code: Tab, keyCode: 2 ('Tab')
2023-12-12 15:06:14.559 [info] [KeybindingService]: | Resolving Tab
2023-12-12 15:06:14.559 [info] [KeybindingService]: \ From 12 keybinding entries, no when clauses matched the context.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.85.1. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.85.1. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

I just updated to the latest version and the issues remained.

Related on Stack Overflow: VScode - tab moves focus from editor area even when disabled

Is there any more info I can provide ?

This is still happening

Just some feedback here: I think this "tab to focus" issue has manifested itself on macOS Sonoma recently. I'm using macOS Sonoma 14.4  which I think may have just recently or will be recently released to everyone soon. Either way, I noticed it after upgrading. VScode is 1.87.2.
It's refreshing to see that others are plagued by the same ailment, but this "feature" makes vscode almost unusable. I'm no UI/UX expert, but there should be NO Ctrl + keystroke that disables normal functionality of an app. Cuz that why we have the Command key. :-)
After reading, it does seem to have something to do wtih Ctrl +M but, there's more to it than that. For some reasons my windows (randomly) get this annoying light border around them, I guess indicating that the window has focus. I don't think this works or scales in multiple monitor situations, especially when displays can have separate spaces... In either event, I understand the need for accessibility and taking that seriously. However, Safari for Mac Doesn't even have this option enabled by default. By default it's Option + Tab to highlight each item which, and you can explicitly change this setting then use tab if necessary. so my 2 cents is not to remove the feature, just change the default behavior to match Safari, which is  to Option + Tab to highlight the window..., then have an option to enable Tab to focus.. however this Ctrl +M mumbo jumbo should be nixed (maybe keep it if it matches the default behavior of Microsoft Edge in windows land) :)
Cheers.

Any update on this ?

Happening to me today out of the blue

@pypp @digitigradeit @IordanisSap Perhaps it was a case of accidentally hitting CTRL+M like it was for myself, having just discovered this feature for which I had no idea existed.

@pypp @digitigradeit @IordanisSap Perhaps it was a case of accidentally hitting CTRL+M like it was for myself, having just discovered this feature for which I had no idea existed.

I don't think that's the case unfortunately, I disabled that keybinds, and  as you can see in the video the feature is not enabled for me.

I found the problem, for some reason I had the following keyboard shortcut in my keybindings.json file:
   {
     "key": "tab",
     "command": "-tab",
     "when": "editorTextFocus && !editorReadonly && !editorTabMovesFocus"
   },

I removed it and and it fixed the issue.

I found the problem,I had the following keyboard shortcut in my keybindings.json file:
   {
     "key": "tab",
     "command": "-tab",
     "when": "editorTextFocus && !editorReadonly && !editorTabMovesFocus"
   },

I removed it and and it fixed the issue.

I don't have a keybindings.json?

I found the problem,I had the following keyboard shortcut in my keybindings.json file:

{

 "key": "tab",

 "command": "-tab",

 "when": "editorTextFocus && !editorReadonly && !editorTabMovesFocus"

},

I removed it and and it fixed the issue.

I don't have a keybindings.json?

Press cmd/ctrl+shift+p and search for Open Keyboard Shortcuts (JSON)

right. when I do that, it says: // Place your key bindings in this file to override the defaults
[
]
so not sure if that is right. here's what I have:

Oh, well I guess you have a different issue.


Create an intro

Hello, I am a web developer. I focus on the area of full-stack development with ReactJS (front) and NodeJS (back) with TypeScript, I also have knowledge of Java and PHP with Laravel, and I have experience in MySQL, PostgreSQL and version control with Git. I also have solid experience in computer technical support in institutions.


RLMSyncUser:configurationWithPartitionValue not available in Swift

!!! MANDATORY TO FILL OUT !!!

Goals
Call RLMSyncUser:configurationWithPartitionValue from Swift. The method is marked as NS_REFINED_FOR_SWIFT but there is no equivalent Swift func to call
Expected Results
Be able to call the method or a redefined function in Swift.
Actual Results
Compiler error
Steps to Reproduce
Just use the Objective C SDK and try calling the function from Swift.
Code Sample

Version of Realm and Tooling
Realm framework version: ?
Realm (10.0.0-beta.2)
Realm Object Server version: ?
NA
Xcode version: ?
Xcode 11.5
iOS/OSX version: ?
ProductName:	Mac OS X
ProductVersion:	10.15.5
BuildVersion:	19F101
Dependency manager + version: ?
pod 1.9.3

Try myUser.configuration(partitionValue: "my partition value").

Doesn't work.

Can I see a larger code snippet?

Assuming you are calling the objc API from Swift, this is an issue. We will fix this in the next release.

Yes, I am. There are many items in the SDK that are marked as NS_REFINED_FOR_SWIFT. I don't know if those are problems as well. Also, the RLMSupport.swift file is missing the helpful extensions that were in the 5.0 version of the SDK.

Any ETA on his?
Is there a workaround (for Realm v10.1.1 in Swift) to be able to use Realm Sync Beta?

@andreasley
I'm working on this today. No exact ETA yet. I'll post updates in this thread.

@ericjordanmossman Actually, my problem seems to be a completely different one, despite resulting in the same symptoms.
I'm using the Swift Package Manager, which works fine for "regular" Realm. However, all symbols for Realm Sync seem to be missing (e.g. App). Some important Swift files are excluded from the target SwiftRealm in Package.swift and I suppose all the C++ sources for sync aren't referenced either. There's probably a reason for that which I have yet to find. :)

Yes, needing the binary library is why we can't support sync with swift package manager. We actually have now open sourced the sync client, but there's some work left to be done to build against that instead of the prebuilt library.

@levelchanger
The usage of NS_REFINED_FOR_SWIFT is defined in Apple's documentation as follows:
If you want to expose an Objective-C API to Swift with a different declaration,
but a similar underlying implementation, use the NS_REFINED_FOR_SWIFT macro.
When you import an Objective-C API into Swift, you can adopt Swift-only types
such as tuples. You can also reorder, combine, and rename parameters so the API
matches other Swift APIs.

This function has been re-written in the RealmSwift SDK and is available there:
func configuration<T: BSON>(partitionValue: T) -> Realm.Configuration {
    let config = self.__configuration(withPartitionValue: ObjectiveCSupport.convert(object: AnyBSON(partitionValue))!)
    return ObjectiveCSupport.convert(object: config)
}

However, if you use the Objective-C framework Realm instead of RealmSwift the function call changes slightly:
Applying the NS_REFINED_FOR_SWIFT macro exposes the existing Objective-C API
for reuse in your refined API. The existing API is renamed with double underscores (__)
when it's imported, to help prevent you from accidentally using the existing API elsewhere.

I have prepared an example repository that shows your reproduction steps and the resolution.
Your call from above
let configuration = user.configuration(partitionValue: user?.identity)

(see https://github.com/DominicFrei/Playground/commit/1f01f373604166f6060a45b1940ac5748c6b1ff9#diff-61a64f31044fbfd03856ead0b699077b15bc799953059f5dd6711e4c3d362ca3R22)
would look like this:
let configuration = user?.__configuration(withPartitionValue: "" as RLMBSON)

(see https://github.com/DominicFrei/Playground/commit/4d82ea953ac4139e8f61f2fc8d38548fc9595465#diff-61a64f31044fbfd03856ead0b699077b15bc799953059f5dd6711e4c3d362ca3R22)

@levelchanger
The usage of NS_REFINED_FOR_SWIFT is defined in Apple's documentation as follows:
If you want to expose an Objective-C API to Swift with a different declaration,
but a similar underlying implementation, use the NS_REFINED_FOR_SWIFT macro.
When you import an Objective-C API into Swift, you can adopt Swift-only types
such as tuples. You can also reorder, combine, and rename parameters so the API
matches other Swift APIs.

This function has been re-written in the RealmSwift SDK and is available there:
func configuration<T: BSON>(partitionValue: T) -> Realm.Configuration {
    let config = self.__configuration(withPartitionValue: ObjectiveCSupport.convert(object: AnyBSON(partitionValue))!)
    return ObjectiveCSupport.convert(object: config)
}

However, if you use the Objective-C framework Realm instead of RealmSwift the function call changes slightly:
Applying the NS_REFINED_FOR_SWIFT macro exposes the existing Objective-C API
for reuse in your refined API. The existing API is renamed with double underscores (__)
when it's imported, to help prevent you from accidentally using the existing API elsewhere.

I have prepared an example repository that shows your reproduction steps and the resolution.
Your call from above
let configuration = user.configuration(partitionValue: user?.identity)

(see https://github.com/DominicFrei/Playground/commit/1f01f373604166f6060a45b1940ac5748c6b1ff9#diff-61a64f31044fbfd03856ead0b699077b15bc799953059f5dd6711e4c3d362ca3R22)
would look like this:
let configuration = user?.__configuration(withPartitionValue: "" as RLMBSON)

(see https://github.com/DominicFrei/Playground/commit/4d82ea953ac4139e8f61f2fc8d38548fc9595465#diff-61a64f31044fbfd03856ead0b699077b15bc799953059f5dd6711e4c3d362ca3R22)

Sorry @DominicFrei, I forgot to mention. @levelchanger also had this related issue. Where this is also available.

Sorry @DominicFrei, I forgot to mention. @levelchanger also had this related issue. Where this is also available.

I've just spoken to @ericjordanmossman about this as a general solution:

The above mentioned way of using __ is a viable way to go.
However, our recommended solution for mixed projects is to use both Realm and RealmSwift.

@andreasley The solution shown in my example repository is probably also valid for you from what I can see. Using RLMApp instead of App since you are accessing the Objective-C layer.
@levelchanger @andreasley I will close this issue for now. Feel free to re-open it and provide further information if the above solutions don't fix your problems.

I've just spoken to @ericjordanmossman about this as a general solution:

The above mentioned way of using __ is a viable way to go.
However, our recommended solution for mixed projects is to use both Realm and RealmSwift.

@andreasley The solution shown in my example repository is probably also valid for you from what I can see. Using RLMApp instead of App since you are accessing the Objective-C layer.
@levelchanger @andreasley I will close this issue for now. Feel free to re-open it and provide further information if the above solutions don't fix your problems.


Blob Explorer Ctrl + F doesn't focus the search box when the side panel is open

Fix available in 20231020.5 or later

Edit, build 20231020.6 or later.


DDP training randomly stopping

🐛 Bug
Edit: it randomly stops in the middle of training epoch as well.
After validation ends (100%), the training process randomly stops without any error log. The stopping point changes randomly (sometimes after epoch 4 validation, sometimes after epoch 1 validation) and every time this happens, one of the machine shows 0% utilization while the others are consumed 100%. Memory is consumed as well from all gpus.
I have tried adding sync_dist=True in self.log and removed saving model checkpoint by top_k, referencing https://github.com/PyTorchLightning/pytorch-lightning/issues/5865. Following https://github.com/PyTorchLightning/pytorch-lightning/issues/9851, I already added seed_everything() as well. I checked that for training and validation, each gpu has same number of batches. However, the issue persists.
Any solution to this problem?

To Reproduce

I was unable to reproduce using the BoringModel, but as the stopping point is irregular even with same seed for pl.seed_everything, I believe it is a bug from ddp process itself.
Expected behavior
The training process should continue after validation.

Environment

PyTorch Lightning Version (e.g., 1.5.0): 1.4.9
PyTorch Version (e.g., 1.10): 1.9
Python version (e.g., 3.9): 3.8
OS (e.g., Linux): Linux
CUDA/cuDNN version: 11.2
GPU models and configuration: Google Cloud Platform A100 x8
How you installed PyTorch (conda, pip, source): pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
If compiling from source, the output of torch.__config__.show():
Any other relevant information:

Additional context

Here is my code for the trainer:
    checkpoint_callback = ModelCheckpoint(
        dirpath=log_dir,
        filename=cfg.exp_name + "-{epoch}-{val_auc:.3f}",
        every_n_epochs=1,
        save_top_k=-1,
    )

    trainer = pl.Trainer(
        callbacks=[
            checkpoint_callback,
            LearningRateMonitor(logging_interval="step"),
        ],
        max_epochs=100,
        accelerator="ddp",
        gpus=str(
            cfg.gpus
        ),
        logger=pl.loggers.WandbLogger(project="news_recommendation", name=cfg.exp_name),
        val_check_interval= cfg[cfg.experiment_type[cfg.current_stage]].val_check_interval, # cfg[cfg.experiment_type[cfg.current_stage]].val_check_interval,
        limit_train_batches=1.0,
        deterministic=True,
        num_sanity_val_steps=0,
        resume_from_checkpoint=cfg[cfg.experiment_type[cfg.current_stage]].load_ckpt
    )

cc @justusschock @kaushikb11 @awaelchli @akihironitta @rohitgr7

Hey @yoonseok312 ,
Would it be possible for you to reproduce this behavior with the BoringModel?
Best,
T.C

Hey, I'm having the same problem. Were you able to solve it?

Same here. Non DP/DDP training has no problem whatsoever.

#10947

Same problem, any solution?

I've been stuck with this last several days. The problem seems related to NCCL communication and surprisingly correlates with logging. The clue came from setting TORCH_DISTRIBUTED_DEBUG=DETAIL which makes training fail with meaningful error related to NCCL being not in sync and  tracebacks that some ranks are still writing logs and some ranks doing forward pass.
Then I've profiled line by line and it turned out that if I remove all self.log() entries from validation_epoch_end then training works fine!

@dselivanov It could be that we are missing a barrier at the end of the validation, I'm not sure. I'm a bit clueless though how that would relate to logging as by default Lightning does't do any syncing for self.log.

We're still experiencing this issue. Why isn't possible to log in the validation_epoch_end?

Could you try to set num_sanity_val_steps=False and see if it resolves the issue?
Also remove the if self.trainer.is_global_zero and rank_zero_only=True call. Only guard your calc_something(collected) with that and avoid any distributed calls in calc_something.

This is happening to me as well. Training hangs randomly during an epoch, sometimes restarts after an hour or I have to exit and continue training from last ckpt. I'm running on 4 V100 GPUs using DDP.

Same issue here with version 1.6.4

Same issue here too with 1.6.5. Training hangs at the beginning of a new epoch, stuck at 0%, all GPUs but one show usage at 100% and remaining one is at 0%.
The weirdest thing is it happens only when check_val_every_n_epoch is set to 5, when is set it to 1 it works fine...
Any chance this will be fixed in later releases?

the issue still persists in 1.7.0

@thiyagu-lily That's unfortunate. Without any further details we can only guess what might be the problem (see the comments above). Unfortunately, so far nobody was able to provide a reproducible case that we can work with.
This is essential for us to help, especially in these cases of distributed training, and we would be very thankful if anybody could provide us with this information.

hi @awaelchli
I have been able to narrow down the issue to torch metrics and val check interval.
I call metrtic.update() in val_step, and then metric.compute() and metric.reset() in val_epoch_end. When i reduce the val_check_interval, to something small I dont get any deadlocks. Im using Wandblogger with torch metrics.

Hi @thiyagu-lily , could you try whether this also happens if you use another logger or no logger at all?

It works without problem if I use tensor board logger directly
On Mon, 8 Aug 2022, 15:35 Justus Schock, @.***> wrote:

Hi @thiyagu-lily https://github.com/thiyagu-lily , could you try
whether this also happens if you use another logger or no logger at all?
—
Reply to this email directly, view it on GitHub
https://github.com/Lightning-AI/lightning/issues/11242#issuecomment-1207772273,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABHC5XM5DDTELHV7P6C4GADVYC2EPANCNFSM5KUXGI2Q
.
You are receiving this because you were mentioned.Message ID:
@.***>

hi @justusschock
It happens if i use any logger! I dont think the issue is because the logger!
Can it be an issue with the metrics not being reset?

I think it's more that there is something asynchronous with the metrics somehow that does result in the processes running out of sync.
@thiyagu-lily are you able to produce a minimal example we could debug? Preferably also with random data?

I miss this problem too. I think it relate to validate. I can train correctly when I remove all validate code.

I faced a similar issue and it was not related to PyTorch Lightning, it was in my case a deadlock issue as explained here: https://pytorch.org/docs/stable/notes/multiprocessing.html#avoiding-and-fighting-deadlocks
You could try amending your dataloader with pin_memory = False and reducing the allocated number of workers.
https://stackoverflow.com/questions/72183733/databricks-notebook-hanging-with-pytorch/72473053#72473053

Having recently encountered and resolved a very similar bug to this, I'd suggest looking at #10947 , for which the root cause was found to be the use of a batch sampler that was incorrectly seeded -- resulting in different replicas' dataloaders calculating a different number of distributed batches.

@arlofaria
Hi, thanks for your comments.
I was wondering where does the seed need to be added (or what has to be exactly done to solve the issue).
I was also thinking if you have an idea about my case. In particular, I observe that reducing the num_workers reduces the number of jobs get stuck, or delays the jobs that get stuck.
in particular, the chances of my job getting stuck is very low when num_workers=2, progressively increase as I set num_workers=4 or 8, i.e with num_workers=4 the jobs get stuck in a couple of hours to 5-6 hours, while, num_workers=8 leads to the jobs getting stuck in less than 1-2 hours.
Thanks for your help

I was wondering where does the seed need to be added (or what has to be exactly done to solve the issue).

In my situation, I was passing the optional batch_sampler argument to DataLoader, and that custom sampler was using an RNG that should have been seeded deterministically.  The details are implementation-specific, but for example you might seed it in the sampler's __init__ method, or perhaps in the __iter__ method if you derived from DistributedBatchSampler and were calling its .set_epoch().
In essence, you want to guarantee that each replica is getting the same number of batches at every epoch.  One way to debug this would be print LightningModule.trainer.num_training_batches and check if it ever differs between any two replicas.  If you do find that there's a difference, then you should figure out why -- or to quickly workaround such a bug you could set Trainer(limit_training_batches=N) where N is less than or equal to the smallest number of batches that any replica might produce.

I was also thinking if you have an idea about my case. In particular, I observe that reducing the num_workers reduces the number of jobs get stuck, or delays the jobs that get stuck.
in particular, the chances of my job getting stuck is very low when num_workers=2, progressively increase as I set num_workers=4 or 8, i.e with num_workers=4 the jobs get stuck in a couple of hours to 5-6 hours, while, num_workers=8 leads to the jobs getting stuck in less than 1-2 hours.

Hmm, this seems like you may have a different problem than I had.  My situation wasn't affected by the num_workers used by the DataLoader,  but rather by the number of replicas -- i.e. the DDP world size.  I always set num_workers=10 and found that I was most likely to get stuck with 2 replicas (and large datasets).  In your case, I think I read somewhere that setting DataLoader(pin_memory=True) might have helped some people.
Hope this helps!

@arlofaria
if I understand correctly, your issue was that each "rank" is having different number of batches to be processed and that was causing an issue in your case. I am assuming by replicas you mean "rank", is that right?
I used the Join() context manager link
to make sure I was having the same number of batches in each epoch.
I think you are right, my issue might be different.
Actually after a LOT of effort, I have failed to solve the issue in my case. The only observation that can lead me anywhere is the fact that less number of num_workers results in training failiing at a later time, or not failing at all (ie overall failure rate in a time span is greatly reduced). I have just one more hypothesis to test out.
@kayvane1  suggested that there can be deadlock issues. since each worker loads data, each of the worker is accessing my data files. And for debugging this problem, I was running 10-15 jobs at once (all jobs being the same). each job had 4 workers. total 60 workers. I am now starting to think if these are interfering with each other to cause a deadlock while accessing data.
I will now try to make a replica of the large dataset I am using, and will train just 1 model, run for many hours and see if it fails.
@kayvane1 if possible, could you let me know how did you find that it was a deadlock issue in your case, and do you know why was the deadlock happening. and does my case sound familiar?
Thanks!

Yes, I meant “replica” as in “rank”.  Rather than Join I had a call to self.all_gather which I think is what was ultimately blocking at the end of the epoch when some ranks had fewer batches and would never have reached that synchronization barrier.
Hope you can figure this one out — these kinds of bugs are incredibly painful to debug, but you’ll feel so relieved later.  The solution is always the last thing you think to try! 😉

This issue, unfortunately, still exists in 1.9.4, with CUDA 11.6, PyTorch 1.13.1
No way to replica, happened randomly

Pytorch_Lightning 2.0.4 also has this issue. After training the first epoch, the program stopped working, but the GPU usage rate reached 100% and there were no errors or warnings reported.

Pytorch_Lightning 2.0.4 also has this issue. After training the first epoch, the program stopped working, but the GPU usage rate reached 100% and there were no errors or warnings reported.

Having the same issue here....for the same dataset, only single GPU works, ddp hangs and the end of the first training epoch, while GPU usage is 100% while GPU power usage becomes lower.

Pytorch_Lightning 2.0.4 also has this issue. After training the first epoch, the program stopped working, but the GPU usage rate reached 100% and there were no errors or warnings reported.

Having the same issue here....for the same dataset, only single GPU works, ddp hangs and the end of the first training epoch, while GPU usage is 100% and GPU power usage becomes lower.

Having the same issue with Fabric DDP. 100% GPU usage but not training progress. Interestingly, the 100% usage only occurs on global rank 0 GPU.

@francotheengineer It was recently documented, in the method overview, and in the API docs. Please note that while we do our best to make the things less error prone, the choice of using Fabric's flexibility naturally also brings more responsibilities to the user handling certain things correctly that would otherwise be automated by the Lightning Trainer.

@awaelchli My mistake thanks. Great work on Fabric, loving using it!

Closing the issue due to its age. If you are experiencing issues similar to this one, please open a new ticket with the necessary details.
The most common reasons for "ddp randomly stopping" in my experience are incorrect implementation of custom samplers / batch samplers, incorrectly implemented iterable datasets, incorrect rank-zero-only guards that lead to racing conditions.

In my case, it is resulted during the logging synchronization.
In my code, I log sth with condition during ddp training, like:
if condition: self.log(some_values)
Since the condition is not always true, when there is the scenario some thread logs but some doesn't, the training hang.
So I modify it to log whenever condition is true to resolve my issue, like:
if condition: self.log(meaningful_values)
else: self.log(meaningless_values)


[ODR-356] Rename controller_ID to host_ID in smart catalog

This PR is part of change to fix https://hwjiraprd01.corp.emc.com/browse/ODR-356
The controller ID from get_smart script may vary for different drive connect to the same controller, so leave the controller ID to be defined in northbound API
@RackHD/corecommitters @cgx027 @sunnyqianzhang

:+1:

:+1:

Dependency PR https://github.com/RackHD/onserve/pull/95
Waiting for it approved and then merge together

dependency PR merged. So we could merge this PR


Support for Credentials

Summary of the new feature / enhancement
In order to be able to call certain types of resources, credential objects are required. I would be very useful to have an intuitive way to do this.
# example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2023/10/config/document.json
resources:
  - name: Get info from classic DSC resources
    type: DSC/PowerShellGroup
    properties:
      resources:
      - name: Get Teams App Permission Policy
        type: Microsoft365DSC/MSFT_TeamsAppPermissionPolicy
        properties:
          Identity: Global
          Credential:
            userName<EMAIL_ADDRESS>            Password: "MyPassword"

Being able to define and then reference would be great but maybe some interp. from dsc exec itself to get values from environment or a secret manager would be great too
Proposed technical implementation details (optional)
No response

Since credentials are sensitive, we wouldn't want the password to be in cleartext within the configuration itself.  Currently, you can use a SecretString parameter to pass it to the configuration and reference it via the parameter() function.  It would look something like:
# example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2023/10/config/document.json
parameters:
  globalCredential:
    type: secureObject
resources:
  - name: Get info from classic DSC resources
    type: DSC/PowerShellGroup
    properties:
      resources:
      - name: Get Teams App Permission Policy
        type: Microsoft365DSC/MSFT_TeamsAppPermissionPolicy
        properties:
          Identity: Global
          Credential:
            userName: "[parameters('globalCredential').username)]"
            Password: "[parameters('globalCredential').password]"

Where globalCredential would look like:
{
  "username"<EMAIL_ADDRESS>  "password": "MyPassword"
}

So the parameters input or file would look like:
parameters:
  globalCredential:
    username<EMAIL_ADDRESS>    password: MyPassword

And you would call dsc using:
dsc config -f ./globalCredentials.parameters.yaml get -p ./example.dsc.config.yaml

where -f specifies the file path to the parameters to be used and -p to the configuration file.


⚠ Samsung has degraded performance

In c25e2d0, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 1332 ms

Resolved: Samsung performance has improved in 182729d.


Namespace passed in place of subresource (and vice versa) to method call NewUpdateSubresourceAction in fakeNamespacedScaleClient.Update

Came across this bug in scale/fake/client.go. f.namespace and "scale" i.e 2nd and 3rd arg values to the method call testing.NewUpdateSubresourceAction are in inverted order.  "scale" should have been 2nd arg and f.namespace should have been 3rd.
https://github.com/kubernetes/client-go/blob/1e6150831a960cf38075bb8561e01f48837f8d74/scale/fake/client.go#L63

This is why structs or something type-safe is good when you have a bunch of arguments like this ;-).  Good catch.

/assign

I see this is fixed in this commit https://github.com/kubernetes/kubernetes/commit/34a446d812c423da7a306963f250f4002dac9be8#diff-1b24eea06678864c03f75811317ad446