subquery with ORDER BY leads to empty SQL

Good morning devs,
basically what I added as a comment in issue #411, but it's better keep track of issues separately:
A subquery with ORDER BY leads to an empty SQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
      ORDER BY DESC(?s)
      LIMIT   1
    }
  }
ORDER BY ?o

Here the debug output (testMode is enabled) which leads to an error because of what I learned from issue #405 the validation of the variable scopes:
DEBUG (QuestQueryProcessor.java:93) - SPARQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
      ORDER BY DESC(?s)
      LIMIT   1
    }
  }
ORDER BY ?o

WARN (QuestQueryProcessor.java:143) - Unexpected exception: The following non-projected variables [o] are appearing in different children of JOIN:
[
INTENSIONAL triple(s,<http://example.org/p3>,o),
SLICE limit=1
   DISTINCT
      CONSTRUCT [s] []
         ORDER BY [DESC(s)]
            INTENSIONAL triple(s,<http://example.org/p4>,o)
]
it.unibz.inf.ontop.iq.exception.InvalidIntermediateQueryException: The following non-projected variables [o] are appearing in different children of JOIN:
[
INTENSIONAL triple(s,<http://example.org/p3>,o),
SLICE limit=1
   DISTINCT
      CONSTRUCT [s] []
         ORDER BY [DESC(s)]
            INTENSIONAL triple(s,<http://example.org/p4>,o)
]
	at it.unibz.inf.ontop.iq.node.impl.JoinLikeNodeImpl.checkNonProjectedVariables(JoinLikeNodeImpl.java:80)
	at it.unibz.inf.ontop.iq.node.impl.InnerJoinNodeImpl.validateNode(InnerJoinNodeImpl.java:335)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.validateNode(NaryIQTreeImpl.java:64)
	at it.unibz.inf.ontop.iq.impl.AbstractCompositeIQTree.validate(AbstractCompositeIQTree.java:235)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.<init>(NaryIQTreeImpl.java:50)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.<init>(NaryIQTreeImpl.java:59)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at com.google.inject.internal.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:86)
	at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105)
	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
	at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:770)
	at com.sun.proxy.$Proxy48.createNaryIQTree(Unknown Source)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.getJoinTree(RDF4JInputQueryTranslatorImpl.java:763)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.buildJoinQuery(RDF4JInputQueryTranslatorImpl.java:735)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateJoinLikeNode(RDF4JInputQueryTranslatorImpl.java:633)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:161)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateOrder(RDF4JInputQueryTranslatorImpl.java:379)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:200)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateProjection(RDF4JInputQueryTranslatorImpl.java:774)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:176)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateDistinctOrReduced(RDF4JInputQueryTranslatorImpl.java:486)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:182)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:85)
	at it.unibz.inf.ontop.answering.reformulation.input.impl.RDF4JInputQueryImpl.translate(RDF4JInputQueryImpl.java:37)
	at it.unibz.inf.ontop.answering.reformulation.impl.QuestQueryProcessor.reformulateIntoNativeQuery(QuestQueryProcessor.java:94)

So far so "good", I know I can disable the test mode, but when doing this here, it leads to an empty query.
See the debug output (testMode disabled):
DEBUG (QuestQueryProcessor.java:93) - SPARQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
      ORDER BY DESC(?s)
      LIMIT   1
    }
  }
ORDER BY ?o

DEBUG (QuestQueryProcessor.java:95) - Parsed query converted into IQ (after normalization):
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            INTENSIONAL triple(s,<http://example.org/p3>,o)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  INTENSIONAL triple(s,<http://example.org/p4>,o)

DEBUG (QuestQueryProcessor.java:100) - Start the rewriting process...
DEBUG (QuestQueryProcessor.java:104) - Rewritten IQ:
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            INTENSIONAL triple(s,<http://example.org/p3>,o)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  INTENSIONAL triple(s,<http://example.org/p4>,o)

DEBUG (QuestQueryProcessor.java:106) - Start the unfolding...
DEBUG (QuestQueryProcessor.java:117) - Unfolded query:
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            CONSTRUCT [s, o] [s/RDF(VARCHARToSTRING(s1m3),IRI), o/RDF(VARCHARToSTRING(o1m3),IRI)]
               EXTENSIONAL "t1"(0:s1m3,1:o1m3)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  CONSTRUCT [s, o] [s/RDF(VARCHARToSTRING(s1m4),IRI), o/RDF(VARCHARToSTRING(o1m4),xsd:string)]
                     DISTINCT
                        EXTENSIONAL "t1"(0:s1m4,1:o1m4)

DEBUG (QuestQueryProcessor.java:122) - Planned query:
ans1(o)
EMPTY [o]
DEBUG (QuestQueryProcessor.java:152) - Producing the native query string...
DEBUG (QuestQueryProcessor.java:157) - Resulting native query:
ans1(o)
EMPTY [o]
DEBUG (QuestQueryProcessor.java:128) - Reformulation time: 148 ms

Any ideas what's going wrong here? Thank's in advance.
Cheers,
Lorenz

Hi Lorenz,
The bug comes from the SPARQL-to-IQ conversion, where the variable ?o should have been renamed in the sub-query. It is related to #405, but this time the query is invalid.
As you can see, both definitions of ?o after unfolding are incompatible, leading to the empty query.
We will look at it.
Best,
Benjamin

Hi Lorenz,
The fix for #405 (https://github.com/ontop/ontop/commit/9fe61eec6c941be14c48bb70fbb641621f6d8243) should a priori address this issue. Could you please give it a try (available in the version4 branch)?
Best,
Benjamin

Hi Benjamin,
thanks for the update. I can confirm that I do not get an empty query anymore,  so we can close this issue I guess. Thank you for the bug fix.
Cheers,
Lorenz

I'll attach the debug output for documentation purpose here:
21/08/17 11:01:57 DEBUG QuestQueryProcessor: SPARQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
      ORDER BY DESC(?s)
      LIMIT   1
    }
  }
ORDER BY ?o

21/08/17 11:01:57 DEBUG RDF4JInputQueryTranslatorImpl: Parsed query:
Distinct
   Projection
      ProjectionElemList
         ProjectionElem "o"
      Order
         OrderElem (ASC)
            Var (name=o)
         Join
            StatementPattern
               Var (name=s)
               Var (name=_const_5bbda064_uri, value=http://example.org/p3, anonymous)
               Var (name=o)
            Slice (limit=1) (new scope)
               Distinct
                  Projection
                     ProjectionElemList
                        ProjectionElem "s"
                     Order
                        OrderElem (DESC)
                           Var (name=s)
                        StatementPattern
                           Var (name=s)
                           Var (name=_const_5bbda065_uri, value=http://example.org/p4, anonymous)
                           Var (name=o)

21/08/17 11:01:57 DEBUG RDF4JInputQueryTranslatorImpl: IQTree (before normalization):
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            INTENSIONAL triple(s,<http://example.org/p3>,o)
            SLICE limit=1
               DISTINCT
                  CONSTRUCT [s] []
                     ORDER BY [DESC(s)]
                        INTENSIONAL triple(s,<http://example.org/p4>,o)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Parsed query converted into IQ (after normalization):
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            INTENSIONAL triple(s,<http://example.org/p3>,o)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  INTENSIONAL triple(s,<http://example.org/p4>,o)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Start the rewriting process...
21/08/17 11:01:57 DEBUG QuestQueryProcessor: Rewritten IQ:
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            INTENSIONAL triple(s,<http://example.org/p3>,o)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  INTENSIONAL triple(s,<http://example.org/p4>,o)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Start the unfolding...
21/08/17 11:01:57 DEBUG QuestQueryProcessor: Unfolded query:
ans1(o)
DISTINCT
   CONSTRUCT [o] []
      ORDER BY [ASC(o)]
         JOIN
            CONSTRUCT [s, o] [s/RDF(VARCHARToSTRING(s1m1),IRI), o/RDF(VARCHARToSTRING(o1m1),IRI)]
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(s)]
                  CONSTRUCT [s, o] [s/RDF(VARCHARToSTRING(s1m2),IRI), o/RDF(VARCHARToSTRING(o1m2),IRI)]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m2,1:o1m2)

21/08/17 11:01:57 DEBUG GeneralStructuralAndSemanticIQOptimizerImpl: New lifted query:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(RDF(VARCHARToSTRING(o1m1),IRI))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(RDF(VARCHARToSTRING(s1m1),IRI))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG FixedPointJoinLikeOptimizer: New query after left join mutable optimization:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(RDF(VARCHARToSTRING(o1m1),IRI))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(RDF(VARCHARToSTRING(s1m1),IRI))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG FixedPointJoinLikeOptimizer: New query after join mutable optimization:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(RDF(VARCHARToSTRING(o1m1),IRI))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(RDF(VARCHARToSTRING(s1m1),IRI))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG GeneralStructuralAndSemanticIQOptimizerImpl: New query after fixed point join optimization (45 ms):
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(RDF(VARCHARToSTRING(o1m1),IRI))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(RDF(VARCHARToSTRING(s1m1),IRI))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG GeneralStructuralAndSemanticIQOptimizerImpl: New query after simplifying the aggregation node:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(RDF(VARCHARToSTRING(o1m1),IRI))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(RDF(VARCHARToSTRING(s1m1),IRI))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG GeneralStructuralAndSemanticIQOptimizerImpl: New query after simplifying the order by node:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(VARCHARToSTRING(o1m1))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG AvoidJoinAboveUnionPlanner: Planned IQ: ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(VARCHARToSTRING(o1m1))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Planned query:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(VARCHARToSTRING(o1m1))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Producing the native query string...
21/08/17 11:01:57 DEBUG SQLGeneratorImpl: After lifting the RDF types:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(VARCHARToSTRING(o1m1))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: After lifting the post-processable function symbols :
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   DISTINCT
      CONSTRUCT [o1m1] []
         ORDER BY [ASC(VARCHARToSTRING(o1m1))]
            JOIN
               EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
               SLICE limit=1
                  ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                     EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: New query after lifting the slice:
DISTINCT
   CONSTRUCT [o1m1] []
      ORDER BY [ASC(VARCHARToSTRING(o1m1))]
         JOIN
            EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                  EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: New query after flattening the union:
DISTINCT
   CONSTRUCT [o1m1] []
      ORDER BY [ASC(VARCHARToSTRING(o1m1))]
         JOIN
            EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                  EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: New query after pushing down:
DISTINCT
   CONSTRUCT [o1m1] []
      ORDER BY [ASC(VARCHARToSTRING(o1m1))]
         JOIN
            EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(VARCHARToSTRING(s1m1))]
                  EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m1,1:o1m1)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: Query tree after pulling out equalities:
DISTINCT
   CONSTRUCT [o1m1] []
      ORDER BY [ASC(VARCHARToSTRING(o1m1))]
         JOIN AND2(STRICT_EQ2(s1m1,s1m0),STRICT_EQ2(o1m1,o1m2))
            EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(VARCHARToSTRING(s1m0))]
                  EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m0,1:o1m2)

21/08/17 11:01:57 DEBUG SQLGeneratorImpl: New query after the dialect-specific extra normalization:
DISTINCT
   CONSTRUCT [o1m1, v3] [v3/VARCHARToSTRING(o1m1)]
      ORDER BY [ASC(VARCHARToSTRING(o1m1))]
         JOIN AND2(STRICT_EQ2(s1m1,s1m0),STRICT_EQ2(o1m1,o1m2))
            EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp3"(0:s1m1,1:o1m1)
            SLICE limit=1
               ORDER BY [DESC(VARCHARToSTRING(s1m0))]
                  EXTENSIONAL "rdd184720884_httpp3ap2fp2fexamplecorgp2fp4"(0:s1m0,1:o1m2)

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Resulting native query:
ans1(o)
CONSTRUCT [o] [o/RDF(VARCHARToSTRING(o1m1),IRI)]
   NATIVE [o1m1, v3]
SELECT DISTINCT `v1`.`o` AS `o1m1`, `v1`.`o` AS `v3`
FROM `rdd184720884_httpp3ap2fp2fexamplecorgp2fp3` `v1`, (SELECT `v2`.`o` AS `o1m2`, `v2`.`s` AS `s1m0`
FROM `rdd184720884_httpp3ap2fp2fexamplecorgp2fp4` `v2`
ORDER BY `v2`.`s` DESC NULLS LAST
LIMIT 1) `v4`
WHERE (`v1`.`s` = `v4`.`s1m0` AND `v1`.`o` = `v4`.`o1m2`)
ORDER BY o1m1 NULLS FIRST

21/08/17 11:01:57 DEBUG QuestQueryProcessor: Reformulation time: 366 ms
21/08/17 11:01:57 INFO QueryEngineOntop: SQL query:
SELECT DISTINCT `v1`.`o` AS `o1m1`, `v1`.`o` AS `v3`
FROM `rdd184720884_httpp3ap2fp2fexamplecorgp2fp3` `v1`, (SELECT `v2`.`o` AS `o1m2`, `v2`.`s` AS `s1m0`
FROM `rdd184720884_httpp3ap2fp2fexamplecorgp2fp4` `v2`
ORDER BY `v2`.`s` DESC NULLS LAST
LIMIT 1) `v4`
WHERE (`v1`.`s` = `v4`.`s1m0` AND `v1`.`o` = `v4`.`o1m2`)
ORDER BY o1m1 NULLS FIRST

Fixed with commit https://github.com/ontop/ontop/commit/9fe61eec6c941be14c48bb70fbb641621f6d8243

@bcogrel Hi Benjamin,
looks like I did some wrong verification whether your changes do fix the issue here. Apparently it is not the case, it doesn't work for subqueries with a DISTINCT operator, which was also my initial query.
The reason why your freshRenaming fix doesn't work is mainly here I think:
@Override
    protected IQTree applyFreshRenaming(InjectiveVar2VarSubstitution renamingSubstitution, boolean alreadyNormalized) {
        InjectiveVar2VarSubstitution selectedSubstitution = alreadyNormalized
                ? renamingSubstitution
                : renamingSubstitution.filter(getVariables()::contains);

        return selectedSubstitution.isEmpty()
                ? this
                : getRootNode().applyFreshRenaming(renamingSubstitution, getChild(), getTreeCache());
    }

getVariables() does only return the projected variables of the subquery, thus, the selectedSubstitution will be empty and the renaming not applied recursively to the triple patterns of the subquery.
If we would use getKnownVariables() instead at least for my test query it works. But indeed I'm not sure about other consequences nor if also other cases of the code still have to be adapted to cover all algebra types for renaming.

@LorenzBuehmann Hi Lorenz,
Thanks for reopening this issue.
Ontop already has a mechanism for avoiding conflicts between variables when unfolding, i.e. replacing an intensional data node (e.g. a triple pattern) by an external IQ tree:
https://github.com/ontop/ontop/blob/8cb0129e591227971b406c21206f8d1c6bf81a06/core/optimization/src/main/java/it/unibz/inf/ontop/iq/optimizer/impl/AbstractIntensionalQueryMerger.java#L95
It is based on the QueryRenamer transformer, not on the fresh renaming. The former takes into account all known variables while the latter focuses on the projected ones, since at the stage it is normally used, there should not be variable conflicts within an IQ tree.
Could you please give us more context (logs) when the issue happens?
Best,
Benjamin

Hi Benjamin,
thanks for the fast response. So far it's basically the same issue as before in my opinion:
Ontop debug output:
INFO [ScalaTest-run] (QuestQueryProcessor.java:69) - Ontop has completed the setup and it is ready for query answering!

 INFO [ScalaTest-run-running-QuotationTests] (QueryEngineOntop.scala:274) - SPARQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
    }
  }
ORDER BY ?o

DEBUG [ScalaTest-run-running-QuotationTests] (QuestQueryProcessor.java:85) - SPARQL query:
PREFIX  :     <http://example.org/>

SELECT DISTINCT  ?o
WHERE
  { ?s  :p3  ?o
    { SELECT DISTINCT  ?s
      WHERE
        { ?s  :p4  ?o }
    }
  }
ORDER BY ?o

 WARN [ScalaTest-run-running-QuotationTests] (QuestQueryProcessor.java:129) - Unexpected exception: The following non-projected variables [o] are appearing in different children of JOIN:
[
INTENSIONAL triple(s,<http://example.org/p3>,o),
DISTINCT
   CONSTRUCT [s] []
      INTENSIONAL triple(s,<http://example.org/p4>,o)
]
it.unibz.inf.ontop.iq.exception.InvalidIntermediateQueryException: The following non-projected variables [o] are appearing in different children of JOIN:
[
INTENSIONAL triple(s,<http://example.org/p3>,o),
DISTINCT
   CONSTRUCT [s] []
      INTENSIONAL triple(s,<http://example.org/p4>,o)
]
	at it.unibz.inf.ontop.iq.node.impl.JoinLikeNodeImpl.checkNonProjectedVariables(JoinLikeNodeImpl.java:65)
	at it.unibz.inf.ontop.iq.node.impl.InnerJoinNodeImpl.validateNode(InnerJoinNodeImpl.java:298)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.validateNode(NaryIQTreeImpl.java:52)
	at it.unibz.inf.ontop.iq.impl.AbstractCompositeIQTree.validate(AbstractCompositeIQTree.java:210)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.<init>(NaryIQTreeImpl.java:38)
	at it.unibz.inf.ontop.iq.impl.NaryIQTreeImpl.<init>(NaryIQTreeImpl.java:47)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
	at com.google.inject.internal.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:86)
	at com.google.inject.internal.ConstructorInjector.provision(ConstructorInjector.java:105)
	at com.google.inject.internal.ConstructorInjector.construct(ConstructorInjector.java:85)
	at com.google.inject.internal.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:267)
	at com.google.inject.internal.InjectorImpl$2$1.call(InjectorImpl.java:1016)
	at com.google.inject.internal.InjectorImpl.callInContext(InjectorImpl.java:1092)
	at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1012)
	at com.google.inject.assistedinject.FactoryProvider2.invoke(FactoryProvider2.java:770)
	at com.sun.proxy.$Proxy71.createNaryIQTree(Unknown Source)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.getJoinTree(RDF4JInputQueryTranslatorImpl.java:707)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.buildJoinQuery(RDF4JInputQueryTranslatorImpl.java:678)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateJoinLikeNode(RDF4JInputQueryTranslatorImpl.java:621)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:162)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateOrder(RDF4JInputQueryTranslatorImpl.java:367)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:201)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateProjection(RDF4JInputQueryTranslatorImpl.java:718)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:177)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translateDistinctOrReduced(RDF4JInputQueryTranslatorImpl.java:474)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:183)
	at it.unibz.inf.ontop.answering.reformulation.input.translation.impl.RDF4JInputQueryTranslatorImpl.translate(RDF4JInputQueryTranslatorImpl.java:86)
	at it.unibz.inf.ontop.answering.reformulation.input.impl.RDF4JInputQueryImpl.translate(RDF4JInputQueryImpl.java:37)
	at it.unibz.inf.ontop.answering.reformulation.impl.QuestQueryProcessor.reformulateIntoNativeQuery(QuestQueryProcessor.java:86)

do I have to setup my reformulation config differently? Your commit fixed the issue for a subquery without DISTINCT operator, e.g. the aggregate I reported at https://github.com/ontop/ontop/issues/405
But with DISTINCT it fails to forward the substitution beyond the distinct tree.
Do I have to register or apply this QueryRenamer somewhere? If so, where? Ontop properties file? Reformulation config setup?

Thanks for the logs.
OK, so the bug comes from the SPARQL to IQ conversion, as the variable ?o should have been renamed in the sub-query.
We need to debug the class RDF4JInputQueryTranslatorImpl, it is probably a minor thing.

I managed to reproduce and fix:  https://github.com/ontop/ontop/commit/6ea5f8922facce4d13bb67878e19dcbd29469078.
@LorenzBuehmann, could you please double-check it?

@bcogrel Seems to be working at least for those simple subqueries and subsubqueries I have here for testing - corner cases or more complex queries I don't know, but according to your commit I guess this covers all. Thank you very much. Very helpful and fast support.
Cheers


it will coredump if senptr > recptr in memmove function in slutils.c

In the branch of develop.
(slconn->stat->slpack.reclen > 0 && slconn->stat->slpack.reclen > bufferlen))  should be (slconn->stat->slpack.reclen > 0 && slconn->stat->slpack.reclen + SLHEADSIZE > bufferlen))
otherwise  sendptr will be bigger than recptr
send:1560, rec:1556

Thanks @wangmm0220.  That should be protected against.
Out of curiosity, have you see this scenario occur?  If so, what were the circumstances?  Thanks.

This should be fixed in the new major release.


Too many states in FairseqLM decoder?

❓ Questions and Help
This question is about the FairseqLM class from : https://github.com/pytorch/fairseq/blob/3c4a8e41559fa50b6c907fbefa1dab55d57bda5c/examples/speech_recognition/w2l_decoder.py
I'm using the fairseqlm for inference and I've noticed that sometimes the memory goes really high. I suspect that the problem is in the LexiconDecoder. I'm not that familiar with C++ so I can't say for sure but from my findings I've noticed that the score method from the FairseqLM is called a lot (sometimes, for specific audio files). For an audio file of 21.75 seconds the self.states list holds about 10_001 states. Is this ok?
What is your question?
Is it ok to have 10_001 states decoded with FairseqLM for an audio file of 21.75 seconds duration?
What's your environment?

fairseq Version (e.g., 1.0 or master): 1.0.0a0+c8a0659
PyTorch Version (e.g., 1.0): 1.9
OS (e.g., Linux): Linux Mint
How you installed fairseq: source
Python version: 3.9.6
Any other relevant information: CPU

Ok. Thank you!


Release 1.12.0 - Update filenames that are uploaded to aws s3

What
Filenames are now human friendly:
The filtered csv downloads now use the dataset Id, edition, version and UTC timestamp so the user can easily distinguish the downloaded file to others they have previously downloaded, e.g. filtered-datasets/{filter_output_id}/{dataset_id}-{edition}-v{version}-filtered-2020-09-28T13-40-12Z.csv
The full csv download now looks like the following full-datasets/{dataset_id}-{edition}-v{version}.csv
Also includes:

CI configuration using latest go version - 1.15.2
go modules updated to use latest version of go 1.15
app dependencies updated

How to review

Sanity check changes
Unit tests pass
If possible check you can generate a csv using the cmd filter journey in develop <- if you do this then you will also prove the following pr works - https://github.com/ONSdigital/dp-filter-api/pull/164

Who can review
Anyone

Before releasing this, can we remove the debug logs, please? I will create a PR shortly against this release branch and develop


map Timestamp types to Date types

Objective
convertTimestampsPipe is mapping from a type A (with Timestamps) to a type B (with Dates). Therefore, a MonoTypeOperatorFunction (mapping from A to A) seems strange. With this PR I am introducing a simple type mapping to map Timestamp types to Date types.
Type

[X] Bug Fix
[X] New Feature

Work Done

add TS type mapping DeepReplace to replace occurrences of M[x][0] in T with M[x][1] recursively
add type casts
turn isTimestamp into a type guard

Tested

Previous tests still pass
No new tests added as the logic has not been changed
The type generic has been tested manually, see TSDoc example

@oberbeck I merged it too fast and just then I tried linter which throws me 2 errors. could you have a look at it, please?
only during npm install, not during npm run lint
error:
src/index.ts:23:10 - error TS2352: Conversion of type 'DeepReplace<any, [Timestamp, Date]>[]' to type 'DeepReplace<T, [Timestamp, Date]>' may be a mistake because neither type sufficiently overlaps with the other. If this was intentional, convert the expression to 'unknown' first.

23      return firebaseObject.map((item) => convertTimestamps(item)) as WithTimestampsAsDates<T>
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

src/index.ts:73:11 - error TS2352: Conversion of type 'Date' to type 'DeepReplace<T, [Timestamp, Date]>' may be a mistake because neither type sufficiently overlaps with the other. If this was intentional, convert the expression to 'unknown' first.

73              return value.toDate() as WithTimestampsAsDates<T>;
                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Hi, thanks for the quick response. I am sorry, I guess I was also to fast on my PR...
https://github.com/peterkracik/convert-firebase-timestamp/pull/12


HADOOP-19358. Update command usage of appendToFile.

In HDFS-16716, supported appending on file with new block. FileSystemShell.md should also be updated.

:confetti_ball: +1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 31s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  0s

codespell was not available.

+0 :ok:
detsecrets
0m  0s

detect-secrets was not available.

+0 :ok:
markdownlint
0m  0s

markdownlint was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

_ trunk Compile Tests _

+1 :green_heart:
mvninstall
43m  2s

trunk passed

+1 :green_heart:
mvnsite
1m 22s

trunk passed

+1 :green_heart:
shadedclient
76m 55s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+1 :green_heart:
mvninstall
0m 54s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
mvnsite
1m 12s

the patch passed

+1 :green_heart:
shadedclient
33m  5s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
asflicense
0m 39s

The patch does not generate ASF License warnings.

113m 41s

Subsystem
Report/Notes

Docker
ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7209/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/7209

Optional Tests
dupname asflicense mvnsite codespell detsecrets markdownlint

uname
Linux 26605a9254e0 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 3883e1865ce6fcf858f2f59bed7e124b0d0e6065

Max. process+thread count
700 (vs. ulimit of 5500)

modules
C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7209/1/console

versions
git=2.25.1 maven=3.6.3

Powered by
Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.


Set up a GHA to run turms-client-kotlin for e2e tests

Motivation

Validate the main method of clients and make sure they can work and be compatible with servers
Validate the main workflow of turms servers can work as expected

These e2e tests are very useful smoke tests indeed.
Note
We use turms-client-kotlin instead of other clients to reuse the testcontainers implementation that was written for server tests


Switch to pigz from gzip to enable multi-thread processing

This helps to make the docker build faster with using multi-core/thread.
Ubuntu18 has pigz installed by default, but still need test on other distro.

Messed up with the code, i will create another PR


Upgrade to match with ABP V 4.3

Could you please upgrade FileManagement to match with ABP V 4.3 .
UseVirtualFiles  is Obsolete in ABP V 4.3 .
The  current project includes code snippets  as:
https://github.com/EasyAbp/FileManagement/blob/2811b7547ddd75a8871e9e1125f264ab80ef008b/src/EasyAbp.FileManagement.Domain.Shared/EasyAbp/FileManagement/FileManagementDomainSharedModule.cs
Configure(options =>
{
options.FileSets.AddEmbedded();
});
https://github.com/EasyAbp/FileManagement/blob/2811b7547ddd75a8871e9e1125f264ab80ef008b/host/EasyAbp.FileManagement.Web.Host/FileManagementWebHostModule.cs
app.UseVirtualFiles();
https://github.com/EasyAbp/FileManagement/blob/2811b7547ddd75a8871e9e1125f264ab80ef008b/src/EasyAbp.FileManagement.Web/FileManagementWebModule.cs
Configure(options =>
{
options.FileSets.AddEmbedded();
});
https://github.com/EasyAbp/FileManagement/blob/2811b7547ddd75a8871e9e1125f264ab80ef008b/host/EasyAbp.FileManagement.HttpApi.Host/FileManagementHttpApiHostModule.cs
app.UseVirtualFiles();
Thanks

I plan to update all the modules 10 to 15 days later after a new ABP version is released. We should take time to observe the new ABP versions since nobody knows if they are stable enough. (We used to spend a lot of time on the patches, such as 4.3.1)


[BUG] tsduck identifies DTS-HD MA sound stream as SCTE-35 Splice Info

dts-hd-ma.ts.gz
Bug description:
tsduck identifies DTS-HD MA streams as SCTE-35 Splice Info
How to reproduce:
Run tsanalyze dts-hd-ma.ts
|  0x1001  PCR (not otherwise referenced) ............... C       18,080 b/s  |
|  0x1011  MPEG-2 Video (1920x1080i, @25 Hz, 16:9, 4:2:2  C   27,335,274 b/s  |
|  0x1100  SCTE 35 Splice Info (eng) .................... C    3,200,682 b/s  |
|  0x1101  MPEG-2 Audio (eng, Audio layer II, 384 kb/s,   C      428,663 b/s  |

Expected behavior:
Identify PID=0x1100 as 'DTS-HD MA' audio
Errors and logs:
no errors, no logs
Environment:

OS: macOS
OS version: 14.1
TSDuck full version: 3.35-3258
Installation type: homebrew

Additional information:
ffprobe (6.1) output:
Input #0, mpegts, from 'dts-hd-ma.ts':
  Duration: 00:00:02.70, start: 4200.000000, bitrate: 29599 kb/s
  Program 1
  Stream #0:0[0x1011]: Video: mpeg2video (4:2:2) ([2][0][0][0] / 0x0002), yuv422p(tv, top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn
    Side data:
      cpb: bitrate max/min/avg: 26580000/0/0 buffer size: 6651904 vbv_delay: N/A
  Stream #0:1[0x1100](eng): Audio: dts (DTS-HD MA) ([134][0][0][0] / 0x0086), 48000 Hz, 5.1(side), s32p (24 bit)
  Stream #0:2[0x1101](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, fltp, 384 kb/s

mediainfo output (audio stream related):
Audio #1
ID                                       : 4352 (0x1100)
Menu ID                                  : 1 (0x1)
Format                                   : DTS XLL
Format/Info                              : Digital Theater Systems
Commercial name                          : DTS-HD Master Audio
Muxing mode                              : Stream extension
Codec ID                                 : 134
Duration                                 : 2 s 507 ms
Bit rate mode                            : Variable
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 93.750 FPS (512 SPF)
Bit depth                                : 24 bits
Compression mode                         : Lossless
Delay relative to video                  : 21 ms
Language                                 : English

Hi @svlobanov
Which standard is this stream supposed to be? DVB?
MPEG-2 video for 1920x1080i at 27 Mb/s is not a common broadcast or streaming encoding. They all use AVC or HEVC. A master feed maybe?
The signalisation is quite minimal:
$ tspsi dts-hd-ma.ts

* PAT, TID 0 (0x00), PID 0 (0x0000)
  Version: 0, sections: 1, total size: 20 bytes
  - Section 0:
    TS id:       1 (0x0001)
    NIT:         0 (0x0000)  PID:   31 (0x001F)
    Program:     1 (0x0001)  PID:  256 (0x0100)

* PMT, TID 2 (0x02), PID 256 (0x0100)
  Version: 0, sections: 1, total size: 55 bytes
  - Section 0:
    Program: 1 (0x0001), PCR PID: 4097 (0x1001)
    Program information:
    - Descriptor 0: Registration (0x05, 5), 4 bytes
      Format identifier: 0x48444D56 ("HDMV")
    - Descriptor 1: unknown (0x88, 136), 4 bytes
      0000:  0F FF FC FC                                ....
    Elementary stream: type 0x02 (MPEG-2 Video), PID: 4113 (0x1011)
    Elementary stream: type 0x86 (SCTE 35 Splice Info), PID: 4352 (0x1100)
    - Descriptor 0: ISO-639 Language (0x0A, 10), 4 bytes
      Language: eng, Type: 0x00 (undefined)
    Elementary stream: type 0x04 (MPEG-2 Audio), PID: 4353 (0x1101)
    - Descriptor 0: ISO-639 Language (0x0A, 10), 4 bytes
      Language: eng, Type: 0x00 (undefined)

The PID has stream type 0x86, which is commonly used for SCTE 35. This is the reason why TSDuck identifies it as such.
Ffprobe and mediainfo are audio/video tools. They look into the audio data and identify it as DTS-HD. TSDuck is a transport stream tool. It looks into the signalisation and identifies it as SCTE 35.
This is of course DTS-HS audio because mediainfo and ffprobe say so, and the bitrate, and the language descriptor. But its signalisation is at best unusual and unknown to me.
On an MPEG standpoint, the stream type 0x86 is "user private". So, we have to look into higher-level standards. In DVB, the stream type for DTS-xx shall be 0x06 (not 0x86), see ETSI TS 101 154 section <IP_ADDRESS>.
For other standards, I do not know. This is why I asked which standard is that TS supposed to be compliant with.

The only thing I know regarding the source of this file is "C-band satellite capture", sorry no more info

After googling a bit, the registration descriptor with "HDMV" seems to indicate a BluRay disk. However, I am not aware of BluRay using MPEG2 encoding, they should use AVC/H.264.
Master Google also says that DTS-HD should have stream type 0x86 in the PMT of a BluRay disk with HDMV convention. Thus, your stream looks like a BDrip with the weird usage of a massive MPEG2 video encoding.
Anyway, not really a "standard" stuff.

Looks like, audio stream was transcoded and maybe some PIDs were deleted. If you don't have plans to support hdmv convention, then let's close the issue
I've just searched for yuv422 video and found this sample, then I observed this strange behaviour and decided and decided to report a bug.

I have added the specific interpretation of stream type per registration id, including "HDMV". However, there is no public specification of BluRay and no way to know the exact list of stream types for "HDMV" registration id. Based on several other open source projects, I have found partial lists, never the same and sometimes contradictory. So, the result is not 100% accurate.
In your case, DTS-HD Master Audio is now correctly identified.

Thanks a lot! Will compile from git and recheck next week

It works, thank you!


load slow

When I reached 500 interfaces, the Swagger document was slow to respond and sometimes timed out

Why wasn't there a caching mechanism here? What was the reason?

Dup of #1970


Rename Items

I would like to rename certain entries. Is it possible to implement this?

Unfortunately that isn't possible as it's not supported by the keys. You have to delete the credential with the incorrect name and re-add it with a correct name.


[fei4142.4] Support data validation and normalization

Summary:
This does two things:

It adds validation and normalization of Sentry data so that tag values are truncated meaningfully, and we avoid tag name clashes.
It removes most of the work done in the previous PR in favour of using Sentry's built-in normalizeDepth option.

Issue: FEI-4142
Test plan:
yarn test
Integrate into webapp, updating the normalizeDepth option, and very that it works with extra long tag names.

Seems ok! A couple tweaks inline - I'm worried about the increase in file size here, it's pretty substantial!

Agreed. I have a plan for that. I'm going to split this into dev vs prod builds. We only need the error checking in dev. So we can optionally include it. Keeping that to a separate PR.


Stuck in "Starting connection to server, please wait"

for the problem above, i search this article below:
https://dev.epicgames.com/community/learning/tutorials/VY4X/unreal-engine-where-s-my-stream-turn-server-debugging-for-pixel-streaming
and found out where is the bug. thx for that.
the turn server is indeed not relaying as expected,
even i force using turn in chrome,
and it doesnt work just as the article informed,
my ue project on cloud is 4.27 version.

So what is next, how can i fix this issue?
let me know if i miss anything that might help.🌹

is it possible that i dont set public ip manually in scripts or somewhere cause this bug?

it work somehow last night...wired...and unstable

This issue tracker is for tracking issues with the front end js/servers. For debugging ICE connections please use the official channels such as UDN.


Electrical Stimulator Fills your hunger bar when fully depleted.

Description (Required)

I crafted an electrical stimulator but did not charge it. I fell from about 15 blocks, enough to drain my hunger bar to repair my health. I was down 2 hunger hearts, then grabbed the stimulator. about 2 seconds later I was at full hunger.
Steps to reproduce the Issue (Required)

https://youtu.be/fykGOeBf2Kw
Expected behavior (Required)

to not fill my hunger bar when not charged.
Server Log / Error Report

no errors
Environment (Required)

Minecraft Version: 1.17.1
CS-CoreLib Version: n/a
Slimefun Version: 950
Plugin Version:  132

Fixed with f842cc0


FCL SDK: iOS

@lmcmz you guys should apply to this grant ;)

Good suggestion！！
let me apply the form.

grant accepted, closing this wishlist item!


azcopy sync --exclude-path flag is not working

Which version of the AzCopy was used?
version 10.16.0
Which platform are you using? (ex: Windows, Mac, Linux)
Linux
What command did you run?
SourceSASURI=https://$SourceStorageAccountName.file.core.windows.net/$SourceShareName/?$sas_source
TargetSASURI=https://$TargetStorageAccountName.file.core.windows.net/$TargetShareName/?$sas_target
azcopy sync $SourceSASURI $TargetSASURI --exclude-path="folder1" --exclude-path="folder2" --exclude-path="folder3" --delete-destination=true --recursive
What problem was encountered?
None of "folder1", "folder2", "folder3" were excluded from scanning and syncing.
Even when trying with 1 folder (azcopy sync $SourceSASURI $TargetSASURI --exclude-path="folder1"  --delete-destination=true --recursive).
I also to put the folder name without double quotes (--exclude-path=folder1), but same result.
How can we reproduce the problem in the simplest way?

Create an Azure file share (share1).
Add folders and files to share1.
Create a second Azure file share (share2).
Run AzCopy sync with --exclude-path flag to replicate the source (share1) to the destination (share2).

Have you found a mitigation/solution?
No.

Any updates on this please?

Hi @kibnelbachyr ! When using multiple folders with the exclude-path, you should list them as --exclude-path="folder1;folder2;folder3".
I am surprised that even just using one folder doesn't work for you as I was not able to reproduce your problem. If you are able to, could you please clarify the setup of the folders so I can verify if I have attempted the reproduction correctly?

Hi @siminsavani-msft, I apologize for the delay.
I was testing with a file share "share1" (source) with this setup of the folders:
=> folder1 {+files}, folder2 {+files}, folder3 {empty}
In my case, I have already folders and files in multiple target shares that I don't want to sync anymore with source shares.
I did a first copy, so a share2 (target) have the same content as share1. Then, I used azcopy sync with --exclude-path="folder1;folder3" and --delete-destination=true, I expected "folder1" and "folder3" and the files in "folder1" to be deleted in target share. But, I realised that --exclude-path option will just exclude the paths when comparing the source against the destination so nothing will change on the specified folders after the sync. I didn't find an option that can help cleaning up excluded folders and files at destination.
I confirm that --exclude-path option works with both single and multiple folders.

Hi @kibnelbachyr ! Thank you for your response!
You are correct, delete-destination would not delete "folder 1" and "folder 3" due to the exclude-path flag. Unfortunately, there is no option to clean up files that have been excluded so I would recommend using the azcopy rm command to delete the files in your destination share.
Please feel free to reopen this issue if you have any questions!


Pagination support in dbcore

The creation of this issue has been discussed in #736.
To implement pagination properly in the HTTP API, we need to add support for pagination in dbCore. Ideally the pagination mechanism should also be usable with the command-line interface.
Ideas about pagination have been discussed in #718. It seems were are basically 4 options :

the naive approach : using start/offset parameters in query. While this is trivial to implement it is definitely not very efficient.
use a seek-able id in the query ( i.e. "gives me the next items starting with this one") : this is the approach recommended by sqlite, we have to make a small prototype to make sure it's doable with complex queries.
when performing a query, store temporarily the result and return a token to the client, which can be used to walk through the ResultSet. The difficult part here is to maintain, and discard at some point, the ResultSets
last solution : only return id for complex queries. This looks nice but I'm afraid it might not play well with the current query mechanism, which fetches full rows anyway.

This would be especially useful for the Smart Playlist and Random plug-ins.  Right now there's no way to limit the number of tracks generated by the Smart Playlist plug-in.

Sorry guys, I'm not sure if I'm barking up the wrong tree here, and please tell me to get lost, but from a UI perspective, pagination or pager navigation isn't the most user friendly interface to have to use - we've done a lot of usability testing and found that they're not as straightforward as we'd like to think. You'd be surprised at the types of feedback, which I can share if you need me to.
A lot of the stuff mentioned here goes right over my head, but after reading https://github.com/sampsyo/beets/issues/738, it seems like pagers/paged navigation methods will play a part in the proposed UI implementations. Can we reconsider their use and look at alternative options before making a decision - like I said, in terms of UI. Many thanks.

Yes! I absolutely agree that it's not a foregone conclusion that the interface should paginate. But it's still useful for the API to admit pagination for efficiency reasons—independent of the interface.

That's good news, as long as we're on the same "page" ;-)
FWIW, our guys are still developing paging on the API too, makes sense.


Self-update

If at some point this becomes packaged for less-technical types (e.g. GUI accessible via PyInstaller), then it would be nice to:

[ ] check that the current d4m version is up to date (if using github, be wary of API limits since we run close to those already sometimes). Maybe if we have CI it could link up to that.
[ ] add an update button which shows either "d4m: check for updates" or "d4m: up-to-date!" depending on whether or not the check has been performed yet

should be clear that this is updating d4m and not dmm for example

[ ] check that the d4m version is a prepackaged version
[ ] If outdated and prepackaged, offer the user a self-update button which downloads the d4m exe, does a little shuffle to rename it away and put the new one in its place (beware windows file locking semantics), and restarts it. This might require a helper script that can be run between exit and restart to rename while the EXE is not in use.
[ ] If outdated and not prepackaged and pip is installed and the package is not an editable install and the user has write access to the package location, offer a button which does pip install -U or equivalent

Update check implemented in v0.3.1


Vulkan: fix crash in GVRRenderData.setMaterial

This PR makes gvr-controls work on OpenGL
GearVRF DCO signed of by: Nola Donato<EMAIL_ADDRESS>

Wrong branch

I'll fix it soon

Superseded by #1337


Return OpenSSH formatted values/attributes for items of type SSH Key

Summary
SSH Keys currently return the string value of the private key field. This feels a bit strange since the OP cli and UI would return the keys in OpenSSH format when revealing/copying the private key.
See https://1password.community/discussion/142994/beg-report-ed25519-ssh-private-keys-human-readable-value-differs-from-json-value for more information regarding the difference between the private key value field and the available values from ssh_formats.
Use cases
Consistent/WYSIWYG behaviour between terraform provider resources and UI/op-cli.
Proposed solution
Either always return the openssh formatted key for the private_key attribute or add another attribute that returns the openssh format - e.g. private_key_openssh? Or vice versa (e.g. private_key_pem_pkcs8).
Maybe use the tls providers attributes as a template to keep the resources alligned: https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key#read-only
Is there a workaround to accomplish this today?
Not sure. Feels kind of difficult/unnecessary to manually convert SSH keys in terraform.
References & Prior Work
OP-Cli also returns the OpenSSH format instead of the field value by default.

Hi @dominikmueller 👋 Thanks for your interest in our Terraform provider and for filing this issue.
Our team doesn’t currently have enough available bandwidth to prioritize this feature, but we really appreciate your detailed write-up here and your rationale for requesting this enhancement. We’ll keep an eye on this issue (and its emoji votes as indicators of interest from other customers) and possibly look into implementing this feature in the future.

I've just bumped in to this - I've pulled an SSH key from 1p to use as a deployment key in Argo, and the fact that it's not OpenSSH format has meant the sync with my GitHub repo fails. I now need to either inject that key a different way, or mangle it in Terraform somehow...

Damn, I just got hit by this too.
The ?format=openssh trick doesn't work, either :(
data "onepassword_item" "some_deploy_key" {
  # working, does correct thing:
  #   op read "op://VAULT/repo deploy key/private key?ssh-format=openssh"
  ##
  # Does not work, yields:
  # Unable to read item, got error: op error: "repo deploy key/private key?ssh-format=openssh" isn't an item in the "VAULT" vault. Specify the item with its UUID, name, or domain.

  title = "repo deploy key?ssh-format=openssh"
}

same here... please consider fixing this...

Bump. This needs fixing.

Bump. Here's my scenario:
I store the pub/priv keys in OP. When creating an EC2 in AWS, I provide the pub key so I can have SSH access, but I need to inject the private key (via a file provisioner (below)) since it is a GitHub deploy key to pull private repos. But as stated above, this is in the raw (non-openssh) format.
  provisioner "file" {
    content     = data.onepassword_item.ssh_key.private_key
    destination = "/home/ec2-user/.ssh/id_rsa"

    connection {
      type        = "ssh"
      user        = "ec2-user"
      private_key = data.onepassword_item.ssh_key.private_key
      host        = self.public_ip
    }
  }


Refactor modular

we were doing navigator.geolocation functions a lot so I thought we could definitely abstract that function and be more DRY.

all the navigator functions are located in utils.js and imported as geoAction in the files.

getCurrent = navigator.geolocation.getCurrentPosition
 - takes in a callback that is passed in successfully found coords (object w latitude and longitude)

setWatch= navigator.geolocation.watchPosition
- takes in a callback that is passed in successfully found coords (object w latitude and longitude)

clearWatch = navigator.geolocation.clearWatch
 - takes in watchID that you want to clear.

note: remember to save this binding by declaring self = this outside of the function and refer all 'this' as self. See the refactored functions for example.

call them by invoking eg:  geoAction.getCurrent()

resolves #71

./lib/orientation might be a better place for this to live, since all of the tools in there have to do with real world location and direction

@jake-shasteen moved it! :heavy_check_mark:

Found a bug introduced by a previous PR -- there is an if statement inside Map.js' render() that needs to be moved out to componentWillUpdate() or it will cause the app to crash when putting down a new pin and changing views.
Also, it seems like the 'go to shared pin' function when a friend shares a pin with you is not currently working as intended.

I'm going to close this one, since I had to merge two PR's and we shouldn't mess up my commit orders.


Fix loading when field is given with different representation

Issue happens when type supports non strict dump like naive_datetime which can be passed without ms, changes are then returned as is instead being loaded to correct form with ms in this case. This results in mismatch when returned from insert/update and get for example. Included test illustrates that (previously would fail).

Thank you @arcz although I am not sure we should merge this. There is no guarantee the database actually stores microseconds and we will have to investigate why Ecto is adding microseconds when none were specified.

Thanks for a quick response! Yeah, it might be that the problem is somewhere else. Here naive_datetime is always dumped to format with microseconds, regardless if they are set or not it matches. The same goes for utc_datetime.

Closing this in favor of general discussion in #1935.


Roadmap for operations console development

Hello,
from going through the issues I understood that you are currently in the process of setting up priorities for upcoming changes, resp. working on a roadmap.
Is this still a working in progress or do you have already a roadmap which you could publish?
I would find it really helpful to have an idea where the development of the console is heading.

@aMinuteMan the short-term road map is to add console support for Fabric system-less channel features. So specifically this is to:

create orderers that do not use a system channel (requires deployer)
add support to create/edit/view channels using the Fabric channel participation features
add support to join orderers to application channels
add support for orderers to "unjoin" application channels (aka leave a channel)

@dshuffma-ibm Got it, thank you!


Method to retrieve the matrix

Add a new method which will return the value of the matrix which we can reference in python code

I wonder why we didn't have it in the first case :)
Can you add the documentation for it, with a versionadded tag to 1.9 ?
Thanks!

I am happy to but where, looking in kivy/doc/sources i dont see an obvious place to add any docs or existing docs for matrix ? is this a different repo ?

figured out you just wanted a doc string, added :)

:+1:


bug: Make rehearsal executable

It seems rehearsal is not globally executable anymore (after migration o commander).
Need to fix this.

Yeah saw the same thing, will take a look and have a fix soon

@egorio @lynchbomb Can you help verify if this still happens to you guys? I just checked again and it's working for me know. My guess is before we didn't have the right permission for cli/bin/rehearsal.js.

@haochuan confirmed this is no longer an issue. @egorio run
chmod 755 ./bin/rehearsal.js && volta install @rehearsal/cli


Be able to consume data files

It would be great to create a folder of json/yml files and have them turned into a collection object so that we could arbitrarily loop through the data.
This may be out of scope for this project but I thought as Jekyll and other static site generators it might be useful to have.

You can already do that. Not straight forward but by consuming json/yml via the phenomic loader (the one used for the md files in the default boilerplate) and custom loaders plugins.
That said, I think we can improve this pretty standard use case by making this more simple.
Maybe data will need to have some standard fields?

Yeah I think standard fields maybe denoted with __ prefix, would be good. I can see how this would be implemented now with the current setup, it would be cool to get some official docs though.

Going to be fixed with #925


Tabs in Log modal are floating and too close to title text

Rancher Version: master
Steps to Reproduce:

Create a service
Go to the container and click on Log from the menu

Results: Notice the Combined, Standard Out & Standard Error tabs are floating and not connected to anything.  Also the tabs and the Logs: title text is too close.

Expected: Connect the tabs, move the ProTip to bottom, Move the tabs down a bit.

Version - master 10/20
Verified fixed


Upgrade to 3.5.0, ServiceAccount reverts back to default

Details
What steps did you take and what happened:
After upgrading HelmRelease to 3.5.0 for amd-gpu-node-labeler with a pre-defined RBAC configuration (ClusterRole, ClusterRoleBinding, and ServiceAccount), the serviceAccount configuration did not seem to be honored which caused the pods in the Daemonset to use the Default ServiceAccount instead of the custom defined one which has the correct permissions to read the node.
    serviceAccount:
      create: false
      name: amd-gpu-node-labeller

What did you expect to happen:
The Pods should use the defined service account

Anything else you would like to add:
Reverting the app-template to 3.4.0 fixes the issue.

Additional Information:

We've happened to identify this issue earlier today on Discord. @LarryGF has taken it upon himself to push a fix PR ❤ 3.5.1 should be rolling out soon afterwards


Setup artifacthub-repo.yml

This has to be pushed to both the registry for he oci releases as well to the gh-pages branch.
See https://artifacthub.io/docs/topics/repositories/helm-charts/

This merged.


Initial implementation of try_get_many

Attempt to implement get_many_mut with a variable number of keys, as requested in #332.
cc @Ten0

:umbrella: The latest upstream changes (presumably #466) made this pull request unmergeable. Please resolve the merge conflicts.


Ignore aws-java-sdk-bom at Dependabot.

@mshima OK to ignore it daily, but we should look at it at least weekly?

@DanielFran I fail to see the need of aws-java-sdk-bom in our bom.
IMO we should remove it.
Looking at https://github.com/jhipster/generator-jhipster/search?q=aws.
The only use I can find it is https://github.com/jhipster/generator-jhipster/blob/0c2b62991f86548b8aa60db9a6bf25e21abcd992/generators/aws-containers/templates/AwsSSMConfiguration.java.ejs
Other places uses node.js aws api.
So IMO it should be managed by aws-containers generator.
And there is a open issue to discuss aws-containers removal https://github.com/jhipster/generator-jhipster/issues/14264.

@mshima Yes I know, this is only used for that generator...
We already discuss it here: https://github.com/jhipster/jhipster-bom/pull/197
Maybe the trick would be to keep it and update only weekly or monthly until the generator is definitely deleted

Maybe the trick would be to keep it and update only weekly or monthly until the generator is definitely deleted

IMO we should:

remove from jhipster-bom
hardcode the version like others non tested features:
https://github.com/mshima/generator-jhipster/blob/5315227180209730d8110b044fdac0f2a65c6312/generators/generator-constants.js#L29
generate the version in the application pom https://github.com/mshima/generator-jhipster/blob/5315227180209730d8110b044fdac0f2a65c6312/generators/aws-containers/index.js#L343-L349.

@mshima OK.
@jhipster/developers If everyone agree, let's do it after V7

I'm ok to remove it from jHipster BOM
Thanks & Regards,
Deepu
On Fri, Mar 19, 2021 at 10:43 AM Daniel Franco @.***>
wrote:

@mshima https://github.com/mshima OK.
@jhipster/developers https://github.com/orgs/jhipster/teams/developers
If everyone agree, let's do it after V7
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jhipster/jhipster-bom/pull/223#issuecomment-802694251,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIOKFZKZBYGEPXTUZR5AFLTEMMDJANCNFSM4ZM25ZQQ
.


Sync with VS Code source

Is your feature request related to a problem? Please describe.
VS Code has received some major accessibility improvements in the last few months. Most notably, the settings UI is way more usable for blind or partially sighted people.
Describe the solution or feature you'd like
Sync with latest VS Code source. Also, please consider formalizing a strategy to do this on a regular basis, e.g. quarterly.

@leonardder we have a process to routinely update the vscode source we're build from.  Currently the insiders builds has an update through ~1.51.  We are planning to catchup with 1.53 in the next few weeks.  This is on-going engineering, so we don't track this through github issues.  Thanks!

Thanks for your explanation! Is there an easy way to find out what version of Code is targeted by a certain release of Data Studio?

@leonardder to get the VS Code engine version of your currently installed Azure Data Studio, click "About" in the "Help" menu

@dzsquared I believe @leonardder was asking for whether we plan out the syncs such that external users could see when we plan to be synced up to a certain point.
Currently we do not do this though - the VS Code merging isn't done on a completely regular basis at this time and depending on what changes have been made on their end some merges take longer than others. But typically we try to stay within a release or two of them when they do a release.
Looking through the commits to see when the merges happen is the only way to know for sure where we're currently at - look for commits such as this one https://github.com/microsoft/azuredatastudio/commit/ce612a3d9613a1857772150892e5f9abe1dd44f7 which will contain the commit hash that we'll be synced up to at that point.

This is also an item we have included in the release notes in the past, adding @yualan for awareness
For example: https://docs.microsoft.com/en-us/sql/azure-data-studio/release-notes-azure-data-studio?view=sql-server-ver15#march-2020

@leonardder to get the VS Code engine version of your currently installed Azure Data Studio, click "About" in the "Help" menu

Wow, I should have checked that myself, it is good to know that this info is available. Thanks for pointing this out.
Stable is now at 1.48, so that clarifies why the new settings editor behavior (introduced in 1.50) isn't yet there.

This is also an item we have included in the release notes in the past, adding @yualan for awareness

Yes, I recall that and it would be very helpful if this could be added in next release notes. For many people it might not be relevant, but as for accessibility, ADS heavily relies on improvements in VS Code.

I've gotten this extension to load in ADS 1.28.0 (VS Code 1.51.0) by changing the vscode version in package.json to match the version ADS is on. I assume this will cause problems for some dependencies that it appears ADS doesn't use. I also added the azdata engine with all versions because I don't know what I'm doing. :) But it compiled, and it's working fine in ADS -- whether it would cause issues in VS Code is unknown to me.
Changes:
	"engines": { 		"vscode": "^1.51.0", 		"azdata": "*" 	},
	"devDependencies": { 		"@types/chroma-js": "2.1.3", 		"@types/lodash-es": "4.17.4", 		"@types/node": "12.12.70", 		"@types/sortablejs": "1.10.6", 		"@types/vscode": "1.51.0",                 ... blah blah blah


Check entire port range implied by spacing instead of just selenium and mock ports

Currently the only ports within the base port spacing range that magellan actually checks are + 0 and + 1.
This PR:

updates magellan to check the entire range within the block given to the worker.
cleans up the code a bit and puts port range checking into its own module.
improves error messaging around occupied ports

/cc @geekdave

LGTM


Weight Converter

Project title
Project on weight converter
Project Description
The user will enter the weight in kilograms which will be converted into grams, pounds and ounces.
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS
[x] JavaScript
[] ReactJS

/assign


..

Description

Expected behavior

Actual behavior

Steps to Reproduce

[First Step]
[Second Step]
[and so on...]

Versions

Prezto commit:
ZSH version:
OS information:

It looks like this may have been opened by mistake. Please feel free to open another issue with the information requested in the template if you're still having trouble with something. Thanks!


An implementation of spark in C

Here's a C implementation of spark. Thanks for the neat idea.

This is rad. And though it's a bit neater and I'm sure much faster than my own spark, I think I want to keep it shell-only right now. I encourage you to split this off into its own project, though!


Update dg v1.3

Codecov Report
Base: 64.41% // Head: 64.41% // No change to project coverage :thumbsup:

Coverage data is based on head (8f2fb43) compared to base (35f5254).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##             master     #110   +/-   ##
=========================================
  Coverage     64.41%   64.41%
  Complexity      524      524
=========================================
  Files            94       94
  Lines          1759     1759
  Branches        204      204
=========================================
  Hits           1133     1133
  Misses          582      582
  Partials         44       44

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Remove React Devtools from debugging docs

Remove the react devtools mention until it works again.
Fixes #5918

@facebook-github-bot shipit

Closing this since it's merged already and the bot is not doing it's job!