Debugging quality

@ugermann @davidecaroselli @nicolabertoldi @mfederico
I have many indicators that there is something going wrong somewhere.
I was not able to spot the problem, but I report my findings and hope in your analytical skills.
I have training an engine with only CommonCrawl data from the WMT15 task, so it is just a 1 domain.
By design, I expect these 3 to return the same output, but it is not.
Context Passed
ubuntu@ip-172-30-0-162:~/mmt$ curl "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
    "context": [
        {
            "id": "commocrawl",
            "score": 0.017159795
        }
    ],
    "translation": "Je comme"
}

Context Not Passed
curl "http://localhost:8045/translate?q=I+like+it&context=" | python -mjson.tool
{
    "translation": "J'aime il"
}

Impossible Context
curl "http://localhost:8045/translate?q=I+like+it&context=XXYYSSDDSSWW" | python -mjson.tool
{
    "context": [],
    "translation": "J'aime il"
}

@cattoni
Assuming that cases 2 and 3 ("Context Not Passed" and "Impossible Context"
respectively) are ok, I've
tested case 1: what happens in a single-domain setup when translating the
same input but varying the context-weight?
Basically with the current adaptive system, it can happen that the output
translation changes.
The reason of this behaviour appears to be as follows: while the suffix
array does not change the list of hypotheses (because there is only a
single domain and varying its context-weight does not influence the
hypotheses score), the adaptive LM changes the score of the same hypothesis
if the context-weight changes.
In details, the LM multiplies the probability of each n-gram of an
hypothesis by the given context weight. So the LM feature score of a given
hypothesis is changed according to the context weight and the hypothesis
length.
This means that changing the context-weight may change the rank of the
hypotheses because the PT feature score does not change while the LM
feature score changes depending on the context weight AND the hypothesis
length.
Therefore changing the context weight may change the best translation
hypothesis.
This is what happens with the current adaptive system.
If this appears not compliant with the expectations, than solutions have to
be designed and implemented, such as:

imposing that in a single domain setup, the context weight must always be
1.0 ;
changing the adaptive model;
...

On Sat, Jan 16, 2016 at 11:01 AM, Marco Trombetti<EMAIL_ADDRESS>wrote:

@ugermann https://github.com/ugermann @davidecaroselli
https://github.com/davidecaroselli @nicolabertoldi
https://github.com/nicolabertoldi @mfederico
https://github.com/mfederico
I have many indicators that there is something going wrong somewhere.
I was not able to spot the problem, but I report my findings and hope in
your analytical skills.
I have training an engine with only CommonCrawl
http://www.statmt.org/wmt13/training-parallel-commoncrawl.tgz data from
the WMT15 task, so it is just a 1 context.
By design, I expect these 3 to return the same output, but it is not.
Context Passed
ubuntu@ip-172-30-0-162:~/mmt$ curl "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
"context": [
{
"id": "commocrawl",
"score": 0.017159795
}
],
"translation": "Je comme"
}
Context Not Passed
curl "http://localhost:8045/translate?q=I+like+it&context=" | python -mjson.tool
{
"translation": "J'aime il"
}
Impossible Context
curl "http://localhost:8045/translate?q=I+like+it&context=XXYYSSDDSSWW" | python -mjson.tool
{
"context": [],
"translation": "J'aime il"
}
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49.

Hi,
so a suggestion could be to always normalise the CA weights to sum up to 1.
Marcello

On 20 Jan 2016, at 10:29, Roldano<EMAIL_ADDRESS>wrote:
@cattoni
Assuming that cases 2 and 3 ("Context Not Passed" and "Impossible Context"
respectively) are ok, I've
tested case 1: what happens in a single-domain setup when translating the
same input but varying the context-weight?
Basically with the current adaptive system, it can happen that the output
translation changes.
The reason of this behaviour appears to be as follows: while the suffix
array does not change the list of hypotheses (because there is only a
single domain and varying its context-weight does not influence the
hypotheses score), the adaptive LM changes the score of the same hypothesis
if the context-weight changes.
In details, the LM multiplies the probability of each n-gram of an
hypothesis by the given context weight. So the LM feature score of a given
hypothesis is changed according to the context weight and the hypothesis
length.
This means that changing the context-weight may change the rank of the
hypotheses because the PT feature score does not change while the LM
feature score changes depending on the context weight AND the hypothesis
length.
Therefore changing the context weight may change the best translation
hypothesis.
This is what happens with the current adaptive system.
If this appears not compliant with the expectations, than solutions have to
be designed and implemented, such as:

imposing that in a single domain setup, the context weight must always be
1.0 ;
changing the adaptive model;
...

On Sat, Jan 16, 2016 at 11:01 AM, Marco Trombetti<EMAIL_ADDRESS>wrote:

@ugermann https://github.com/ugermann @davidecaroselli
https://github.com/davidecaroselli @nicolabertoldi
https://github.com/nicolabertoldi @mfederico
https://github.com/mfederico
I have many indicators that there is something going wrong somewhere.
I was not able to spot the problem, but I report my findings and hope in
your analytical skills.
I have training an engine with only CommonCrawl
http://www.statmt.org/wmt13/training-parallel-commoncrawl.tgz data from
the WMT15 task, so it is just a 1 context.
By design, I expect these 3 to return the same output, but it is not.
Context Passed
ubuntu@ip-172-30-0-162:~/mmt$ curl "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
"context": [
{
"id": "commocrawl",
"score": 0.017159795
}
],
"translation": "Je comme"
}
Context Not Passed
curl "http://localhost:8045/translate?q=I+like+it&context=" | python -mjson.tool
{
"translation": "J'aime il"
}
Impossible Context
curl "http://localhost:8045/translate?q=I+like+it&context=XXYYSSDDSSWW" | python -mjson.tool
{
"context": [],
"translation": "J'aime il"
}
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49.

—
Reply to this email directly or view it on GitHub https://github.com/ModernMT/MMT/issues/49#issuecomment-173143512.

Unfortunately I was not able to have a different translation changing the score.
Anyway@davidecaroselli can you release a new version with score normalization. We will try and test again.

@marcotrombetti: in your original post, the first two cases "Context
Passed" and "Context Not Passed" produced different translations ("Je
comme" and "J'aime il" respectively). In the first case the context-weight
was 0.017159795, in the second case 1.0 (implicitely). You can utilize such
example to test the next version.
On Wed, Jan 20, 2016 at 12:39 PM, Marco Trombetti<EMAIL_ADDRESS>wrote:

Unfortunately I was not able to have a different translation changing the
score.
Anyway@davidecaroselli can you release a new version with score
normalization. We will try and test again.
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173179484.

Using the standalone version of Moses provided by Davide,
querying in the corresponding way (see below),
and changing the context weights, I got the different translation (and scores) or maybe same translation but different scores.
The fact that two translations are equal, when using different contexts (and different weights) does not mean that the translation are actually the same; their scores may change.
May be that they are both he winner inside the corresponding competitors; we should check the actual score of the translations produced, or even better the nbest produced in the two conditions
Nicola
echo 'I like it' | /home/davide/moses -f  runtime/default/slave/moses.ini -v 1 2>&1 | grep BEST
BEST TRANSLATION: Desidero citare , ne ha [111]  [total=-1.462] core=(0.000,-5.000,0.000,3.000,-11.548,-13.474,-1.797,-19.591,-7.829,-2.708,0.000,0.000,0.000,-1.504,0.000,0.000,0.000,-15.851)
echo 'I like it' | /home/davide/moses -f  runtime/default/slave/moses.ini -v 1 2>&1 | grep BEST
BEST TRANSLATION: Desidero citare essa [111]  [total=-1.831] core=(0.000,-3.000,0.000,3.000,-11.320,-2.299,-0.894,-14.133,-4.437,-2.110,0.000,0.000,0.000,-1.504,0.000,0.000,0.000,-25.699)
echo 'Commission' | /home/davide/moses -f  runtime/default/slave/moses.ini -v 1 2>&1 | grep BEST
BEST TRANSLATION: Commissione [1]  [total=-1.009] core=(0.000,-1.000,0.000,1.000,-0.282,-0.078,-0.298,-0.251,-0.172,-0.547,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-15.528)
echo 'Commission' | /home/davide/moses -f  runtime/default/slave/moses.ini -v 1 2>&1 | grep BEST
BEST TRANSLATION: Commissione [1]  [total=-0.738] core=(0.000,-1.000,0.000,1.000,-0.282,-0.078,-0.298,-0.251,-0.172,-0.547,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-10.923)
On Jan 20, 2016, at 12:53 PM, Roldano wrote:

@marcotrombetti: in your original post, the first two cases "Context
Passed" and "Context Not Passed" produced different translations ("Je
comme" and "J'aime il" respectively). In the first case the context-weight
was 0.017159795, in the second case 1.0 (implicitely). You can utilize such
example to test the next version.
On Wed, Jan 20, 2016 at 12:39 PM, Marco Trombetti<EMAIL_ADDRESS>wrote:

Unfortunately I was not able to have a different translation changing the
score.
Anyway@davidecaroselli can you release a new version with score
normalization. We will try and test again.
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173179484.

—
Reply to this email directly or view it on GitHub.

@nicolabertoldi can you apply the weight normalization to IRSTLM? I would like to complete this task for today to let Marco test it asap. If you need any kind of support, please let me know.
Max priority for this task.

@davidecaroselli
I could, but would it be better that the context analizer provide normalized values? or alternatively, that Moses normalized the weights before SA and LM use them.
Moreover, are you sure that SA uses normalized weights; maybe it does not normalize weights, but it computes the same options and scores.
Let me know your preference.

My opinion is that the Context Analyzer provides its semantics to the
weight returned: they are similarity score. And for this reason they do not
sum up to 1.
If a specific implementation of a Moses feature (Adaptive LM in this case)
works better with some manipulation of the ca weights, it must be its
responsibility to apply that manipulation. In the future we could even
discover that for LM suites better another kind of manipulation that does
not work great with other Moses components.
Can you work on this?
Il giovedì 21 gennaio 2016, Nicola Bertoldi<EMAIL_ADDRESS>ha
scritto:

@davidecaroselli https://github.com/davidecaroselli
I could, but would it be better that the context analizer provide
normalized values? or alternatively, that Moses normalized the weights
before SA and LM use them.
Moreover, are you sure that SA uses normalized weights; maybe it does not
normalize weights, but it computes the same options and scores.
Let me know your preference.
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173479706.

Hi,
yes but keep in mind that a probabilistic/distributional interpretation
of the CA weights makes things much simpler. If are able to improve
the CA component (e.g. with some LM-based model)  we will not
have to retune the whole MT system.
Marcello

On 21 Jan 2016, at 08:23, Davide Caroselli<EMAIL_ADDRESS>wrote:
My opinion is that the Context Analyzer provides its semantics to the
weight returned: they are similarity score. And for this reason they do not
sum up to 1.
If a specific implementation of a Moses feature (Adaptive LM in this case)
works better with some manipulation of the ca weights, it must be its
responsibility to apply that manipulation. In the future we could even
discover that for LM suites better another kind of manipulation that does
not work great with other Moses components.
Can you work on this?
Il giovedì 21 gennaio 2016, Nicola Bertoldi<EMAIL_ADDRESS>ha
scritto:

@davidecaroselli https://github.com/davidecaroselli
I could, but would it be better that the context analizer provide
normalized values? or alternatively, that Moses normalized the weights
before SA and LM use them.
Moreover, are you sure that SA uses normalized weights; maybe it does not
normalize weights, but it computes the same options and scores.
Let me know your preference.
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173479706.

—
Reply to this email directly or view it on GitHub https://github.com/ModernMT/MMT/issues/49#issuecomment-173482033.

@mfederico good point. But I imagine that even a probabilistic CA could returns values that does not sum up to 1. That's because you're receiving only a portion of the whole distribution (only the top N) and even in this case, the LM should normalize to 1 because it just "work better" in a real case.
If you both agree @mfederico  and @nicolabertoldi I'll proceed in this way:

Update Interpolated IRSTLM to manipulate the weight in a way that perfectly fit its needs (aka normalize them)
At the same time we can work on a better and more advanced definition of the context weights, including also the results of you LM-based CA.

@davidecaroselli: sounds good!

@davidecaroselli
(Optional) normalization of the context weights
has been implemented in the new branch mmtdev_issue49
Currently only IRSTLM is able to enable the usage of normalized context weights
To do that, add the following parameter on the IRSTLM line  in the moses configuration file
"weight_normalization=yes"
Note that very little debugging was done; so report any strange behavior or error.
By the way LM scores change
with normalized weights
BEST TRANSLATION: Commissione [1]  [total=-15.155] core=(0.000,-1.000,0.000,1.000,-0.341,-0.080,-0.288,-0.299,-0.238,-0.578,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-13.331)
with not normalized weights
BEST TRANSLATION: Commissione [1]  [total=-17.353] core=(0.000,-1.000,0.000,1.000,-0.341,-0.080,-0.288,-0.299,-0.238,-0.578,0.000,0.000,0.000,0.000,0.000,0.000,0.000,-15.528)
Nicola

Hi ( @marcotrombetti @mfederico @nicolabertoldi @ugermann @cattoni )
I have included the last patch of Nicola to normalize CA weights for LM. Unfortunately the problem is not solved at all, it seems more deep. I've created a test that could replicate the bug and add some more details.

Download pre-release of mmt 0.11.1 at http://labs.mmt.rocks/builds/mmt-0.11.1-SNAPSHOT_201601211918-ubuntu14_04.tar.gz
Extract this file default.tar.gz inside the engines folder so that the path should become mmt/engines/default/...
Start mmt with command ./mmt start

Now lets try some translations:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool

{
    "context": [
        {
            "id": "all",
            "score": 0.051096026
        }
    ],
    "translation": ", al termine della, che viene"
}

curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool

{
    "translation": "In un'interrogazione successiva, non fa che"
}

curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool

{
    "context": [],
    "translation": "In un'interrogazione successiva, non fa che"
}

Again different translations where the same is expected.
Let's dig a little more, let's force a context weight of all: 1 (remember there's only one domain called all):
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=%5B%7B%22id%22%3A%20%22all%22%2C%22score%22%3A%201%7D%5D" | python -mjson.tool
{
    "translation": "In un', non fa che"
}

Another different translation, this should demonstrates that it's not a simple normalization problem.
Let's now try with a context weight of all: 0.4:
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=%5B%7B%22id%22%3A%20%22all%22%2C%22score%22%3A%200.4%7D%5D" | python -mjson.tool
{
    "translation": "In un'interrogazione successiva, che viene"
}

Again another (the 4th) different translation
This behavior appears to me inconsistent: we should always have the same translations. Event if I force a normalized Context Analyzer (test all: 1) I obtain a different translation.
Comments are welcome.

I fixed a bug
and now the standalone Moses )with your models) works as expected (see below).
All outputs of the normalized version are equal, while outputs of the un-normalized version differ.
I have not yet checked the MMT system.
I will do tomorrow.
I noticed that in the LM configuration file "default/data/lm/target.lm"
the pointed LM "all.alm" has no path.
Do you think this is correct?
cheers,
Nicola
Not normalized weight:0.2
BEST TRANSLATION: Mi come essa [111]  [total=-69.144] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-34.529)
Not normalized weight:1.0
BEST TRANSLATION: Mi come essa [111]  [total=-62.706] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-28.091)
Not normalized weight:2.4
BEST TRANSLATION: Mi come essa [111]  [total=-59.204] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-24.589)
Normalized weight:0.2
BEST TRANSLATION: Mi come essa [111]  [total=-62.706] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-28.091)
Normalized weight:1.0
BEST TRANSLATION: Mi come essa [111]  [total=-62.706] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-28.091)
Normalized weight:2.4
BEST TRANSLATION: Mi come essa [111]  [total=-62.706] core=(0.000,-3.000,0.000,3.000,-10.143,-4.907,-0.894,-10.713,-5.363,-2.015,0.000,0.000,0.000,-0.580,0.000,0.000,0.000,-28.091)
On Jan 21, 2016, at 8:04 PM, Nicola Bertoldi wrote:

I will debug tomorrow
Nicola
On Jan 21, 2016, at 7:41 PM, Davide Caroselli wrote:

Hi ( @marcotrombetti @mfederico @nicolabertoldi @ugermann @cattoni )
I have included the last patch of Nicola to normalize CA weights for LM. Unfortunately the problem is not solved at all, it seems more deep. I've created a test that could replicate the bug and add some more details.
Download pre-release of mmt 0.11.1 at http://labs.mmt.rocks/builds/mmt-0.11.1-SNAPSHOT_201601211918-ubuntu14_04.tar.gz
Extract this file default.tar.gz inside the engines folder so that the path should become mmt/engines/default/...
Start mmt with command ./mmt start
Now lets try some translations:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
"context": [
{
"id": "all",
"score": 0.051096026
}
],
"translation": ", al termine della, che viene"
}
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
{
"translation": "In un'interrogazione successiva, non fa che"
}
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool
{
"context": [],
"translation": "In un'interrogazione successiva, non fa che"
}
Again different translations where the same is expected.
Let's dig a little more, let's force a context weight of all: 1 (remember there's only one domain called all):
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]" | python -mjson.tool
{
"translation": "In un', non fa che"
}
Another different translation, this should demonstrates that it's not a simple normalization problem.
Let's now try with a context weight of all: 0.4:
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]" | python -mjson.tool
{
"translation": "In un'interrogazione successiva, che viene"
}
Again another (the 4th) different translation
This behavior appears to me inconsistent: we should always have the same translations. Event if I force a normalized Context Analyzer (test all: 1) I obtain a different translation.
Comments are welcome.
—
Reply to this email directly or view it on GitHub.

On Jan 21, 2016, at 7:41 PM, Davide Caroselli wrote:

Hi ( @marcotrombetti @mfederico @nicolabertoldi @ugermann @cattoni )
I have included the last patch of Nicola to normalize CA weights for LM. Unfortunately the problem is not solved at all, it seems more deep. I've created a test that could replicate the bug and add some more details.
Download pre-release of mmt 0.11.1 at http://labs.mmt.rocks/builds/mmt-0.11.1-SNAPSHOT_201601211918-ubuntu14_04.tar.gz
Extract this file default.tar.gz inside the engines folder so that the path should become mmt/engines/default/...
Start mmt with command ./mmt start
Now lets try some translations:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
"context": [
{
"id": "all",
"score": 0.051096026
}
],
"translation": ", al termine della, che viene"
}
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
{
"translation": "In un'interrogazione successiva, non fa che"
}
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool
{
"context": [],
"translation": "In un'interrogazione successiva, non fa che"
}
Again different translations where the same is expected.
Let's dig a little more, let's force a context weight of all: 1 (remember there's only one domain called all):
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]" | python -mjson.tool
{
"translation": "In un', non fa che"
}
Another different translation, this should demonstrates that it's not a simple normalization problem.
Let's now try with a context weight of all: 0.4:
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]" | python -mjson.tool
{
"translation": "In un'interrogazione successiva, che viene"
}
Again another (the 4th) different translation
This behavior appears to me inconsistent: we should always have the same translations. Event if I force a normalized Context Analyzer (test all: 1) I obtain a different translation.
Comments are welcome.
—
Reply to this email directly or view it on GitHub.

@nicolabertoldi you made a great job with very little time, what I tried to point out is the fact that the actual bug that this issue is about is still there. Even more: it does not depends on context weights normalization (test 4 is a totally new translation).
We made a guess of the cause of this bug, our tests suggest we were wrong. I need the collaboration of all of you to find out the real solution.

I tested     on  a FBK machine
I  am able to obtain expected results both with standalone Moses and with server-based Moses (XMLRPC version)
In this experiments I used directly the context weights without querying them to the context analizer
I did not try with the SNAPSHOT, because it does not include my last fixing of yesterday evening.
and I do not know how to create the library required by the MMT system, starting from the source code of Moses
Moreover,  by using the last available snapshot, I start the system
but I am not able to find the logs to inspect.
In particular I do not see any log about the CA module
Could you please give me any pointer?
Nicola
On Jan 22, 2016, at 12:58 AM, Davide Caroselli wrote:

@nicolabertoldi you made a great job with very little time, what I tried to point out is the fact that the actual bug that this issue is about is still there. Even more: it does not depends on context weights normalization (test 4 is a totally new translation).
We made a guess of the cause of this bug, our tests suggest we were wrong. I need the collaboration of all of you to find out the real solution.
—
Reply to this email directly or view it on GitHub.

I've recently updated the file INSTALL.md in the repo to include instructions to build MMT from sources, sorry I didn't mention it before.
When you say that you're able to obtain expected results, you mean that all the 5 translations I've reported, with the engine I've provided, are exactly the same right? I so, I would like to compare the Moses config.

Update
I've included the last fix of @nicolabertoldi into MMT and the results is that:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=%5B%7B%22id%22%3A%20%22all%22%2C%22score%22%3A%201%7D%5D" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=%5B%7B%22id%22%3A%20%22all%22%2C%22score%22%3A%200.4%7D%5D" | python -mjson.tool

Have the same translation (good!), but
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool

Have different translation.
It appears now to be a boolean condition that changes the translation: there is context, or not. Actually Nicola has observed different scores for Phrase Tables, @ugermann have you any idea of the reason?
By the way, we found a more worrying bug: Nicola was able to find better translations with the same models directly with Moses:
I like it -> Mi come essa (acceptable if we consider that the model is too small)
instead of
I like it -> In un' interrogazione successiva , non fa che (totally wrong)
I've attached the log files of Moses (verbose 3) with and without context, in both cases the translation is totally wrong:
with-context.log.zip
no-context.log.zip
We can observe that the Moses library receive the right sentence, and that the output is the same as MMT: the problem should be within Moses "space".
Comments would be much appreciated!

In order to investigate this I need access to the training data used, as
well as the word alignments, models and .ini files and the exact versions
(output of git describe in the respective repo) of the software used so
that I can replicate the system for debugging.

Uli

On Fri, Jan 22, 2016 at 10:16 AM, Davide Caroselli<EMAIL_ADDRESS>
wrote:

Update
I've included the last fix of @nicolabertoldi
https://github.com/nicolabertoldi into MMT and the results is that:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]" | python -mjson.tool
Have the same translation (good!), but
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool
Have different translation.
It appears now to be a boolean condition that changes the translation:
there is context, or not. Actually Nicola has observed different scores for
Phrase Tables, @ugermann https://github.com/ugermann have you any idea
of the reason?
By the way, we found a more worrying bug: Nicola was able to find
better translations with the same models directly with Moses:
I like it -> Mi come essa (acceptable if we consider that the model
is too small)
instead of
I like it -> In un' interrogazione successiva , non fa che (totally
wrong)
I've attached the log files of Moses (verbose 3) with and without context,
in both cases the translation is totally wrong:
with-context.log.zip
https://github.com/ModernMT/MMT/files/100610/with-context.log.zip
no-context.log.zip
https://github.com/ModernMT/MMT/files/100609/no-context.log.zip
We can observe that the Moses library receive the right sentence, and that
the output is the same as MMT: the problem should be within Moses "space".
Comments would be much appreciated!
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173869886.

--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh

Hi all,
after a bit of debugging to explain the strange behavior observed by marco and Davide
I found that random sampling behaves differently if context weights are specified or not.
No problem with ranked sampling
Assuming, we have just one training corpus (called "all")
it happens that the one of the scores provided by the SA differ in the two conditions (above)
Instead, there are no differences if different context weights are provided.
See the excerpt of the Moses log below (the input is simply "like")
the first option reported in all cases is "di"
but the 9th  feature value is
-11.639     if no context weights are given
-11.595     if the context weights is "all=1.0"
-11.595     if the context weights is "all=0.4"
The difference is small, but nevertheless can cause different translations
Using ranked sampling I obtain always the same options and same score  -11.639,   as expected
Note that this is true using the submodule from
https://github.com/ModernMT/moses-submodule
commit 1cf64e697edd55aa3f110253183d54e19f23fa19
Nicola
Biased sampling
no  ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.060 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.025 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.145 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.117 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.060 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.025 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.145 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.117 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
Ranked sampling
no  ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
On Jan 22, 2016, at 11:16 AM, Davide Caroselli wrote:

Update
I've included the last fix of @nicolabertoldi into MMT and the results is that:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]" | python -mjson.tool
Have the same translation (good!), but
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool
Have different translation.
It appears now to be a boolean condition that changes the translation: there is context, or not. Actually Nicola has observed different scores for Phrase Tables, @ugermann have you any idea of the reason?
By the way, we found a more worrying bug: Nicola was able to find better translations with the same models directly with Moses:
I like it -> Mi come essa (acceptable if we consider that the model is too small)
instead of
I like it -> In un' interrogazione successiva , non fa che (totally wrong)
I've attached the log files of Moses (verbose 3) with and without context, in both cases the translation is totally wrong:
with-context.log.zip
no-context.log.zip
We can observe that the Moses library receive the right sentence, and that the output is the same as MMT: the problem should be within Moses "space".
Comments would be much appreciated!
—
Reply to this email directly or view it on GitHub.

second update
which confirms my previous mail"

ranked sampling seems ok
biased sampling behaves "wrongly"
normalization of weights works always properly

Here are the outputs od different setting fo the input sentence "I like it"
with the weights provided by Davide
no context weights, biased sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=1.0), biased sampling
BEST TRANSLATION: , al , come , ne ha [111]  [total=-1.217] core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
normalized context weights (all=0.4), biased sampling
BEST TRANSLATION: , al , come , ne ha [111]  [total=-1.217] core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
not normalized context weights (all=0.4), biased sampling
BEST TRANSLATION: , al , come , ne ha [111]  [total=-1.489] core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-51.874)
not normalized context weights (all=1.0), biased sampling
BEST TRANSLATION: , al , come , ne ha [111]  [total=-1.217] core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
no context weights, ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=1.0), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=0.4), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
not normalized context weights (all=0.4), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.438] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-51.011)
not normalized context weights (all=1.0), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
Nicola
On Jan 22, 2016, at 5:04 PM, Nicola Bertoldi wrote:

Hi all,
after a bit of debugging to explain the strange behavior observed by marco and Davide
I found that random sampling behaves differently if context weights are specified or not.
No problem with ranked sampling
Assuming, we have just one training corpus (called "all")
it happens that the one of the scores provided by the SA differ in the two conditions (above)
Instead, there are no differences if different context weights are provided.
See the excerpt of the Moses log below (the input is simply "like")
the first option reported in all cases is "di"
but the 9th  feature value is
-11.639     if no context weights are given
-11.595     if the context weights is "all=1.0"
-11.595     if the context weights is "all=0.4"
The difference is small, but nevertheless can cause different translations
Using ranked sampling I obtain always the same options and same score  -11.639,   as expected
Note that this is true using the submodule from
https://github.com/ModernMT/moses-submodule
commit 1cf64e697edd55aa3f110253183d54e19f23fa19
Nicola
Biased sampling
no  ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.060 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.025 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.145 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.117 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.060 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.025 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.145 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.117 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
Ranked sampling
no  ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128 core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, :: term=0-0 : nonterm=: c=-0.061 core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, la :: term=0-0 : nonterm=: c=0.023 core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, come :: term=0-0 0-1 : nonterm=: c=0.143 core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
, desidero :: term=0-0 0-1 : nonterm=: c=0.097 core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
la :: term=0-0 : nonterm=: c=-0.119 core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)
On Jan 22, 2016, at 11:16 AM, Davide Caroselli wrote:

Update
I've included the last fix of @nicolabertoldi into MMT and the results is that:
curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]" | python -mjson.tool
Have the same translation (good!), but
curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
curl --silent "http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC" | python -mjson.tool
Have different translation.
It appears now to be a boolean condition that changes the translation: there is context, or not. Actually Nicola has observed different scores for Phrase Tables, @ugermann have you any idea of the reason?
By the way, we found a more worrying bug: Nicola was able to find better translations with the same models directly with Moses:
I like it -> Mi come essa (acceptable if we consider that the model is too small)
instead of
I like it -> In un' interrogazione successiva , non fa che (totally wrong)
I've attached the log files of Moses (verbose 3) with and without context, in both cases the translation is totally wrong:
with-context.log.zip
no-context.log.zip
We can observe that the Moses library receive the right sentence, and that the output is the same as MMT: the problem should be within Moses "space".
Comments would be much appreciated!
—
Reply to this email directly or view it on GitHub.

In order to look into this, I really need to see ALL the data, ALL the
models, and the .ini files used, as well as the git commit hashes of all
the software involved.

Uli

On Fri, Jan 22, 2016 at 4:12 PM, Nicola Bertoldi<EMAIL_ADDRESS>wrote:

second update
which confirms my previous mail"

ranked sampling seems ok
biased sampling behaves "wrongly"
normalization of weights works always properly

Here are the outputs od different setting fo the input sentence "I like it"
with the weights provided by Davide
no context weights, biased sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.098]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=1.0), biased sampling
BEST TRANSLATION: , al , come , ne ha [111] [total=-1.217]
core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
normalized context weights (all=0.4), biased sampling
BEST TRANSLATION: , al , come , ne ha [111] [total=-1.217]
core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
not normalized context weights (all=0.4), biased sampling
BEST TRANSLATION: , al , come , ne ha [111] [total=-1.489]
core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-51.874)
not normalized context weights (all=1.0), biased sampling
BEST TRANSLATION: , al , come , ne ha [111] [total=-1.217]
core=(0.000,-7.000,0.000,3.000,-10.279,-32.576,-2.634,-24.895,-16.055,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-44.544)
no context weights, ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.098]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=1.0), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.098]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
normalized context weights (all=0.4), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.098]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
not normalized context weights (all=0.4), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.438]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-51.011)
not normalized context weights (all=1.0), ranked sampling
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]
[total=-1.098]
core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
Nicola
On Jan 22, 2016, at 5:04 PM, Nicola Bertoldi wrote:

Hi all,
after a bit of debugging to explain the strange behavior observed by
marco and Davide
I found that random sampling behaves differently if context weights are
specified or not.
No problem with ranked sampling
Assuming, we have just one training corpus (called "all")
it happens that the one of the scores provided by the SA differ in the
two conditions (above)
Instead, there are no differences if different context weights are
provided.
See the excerpt of the Moses log below (the input is simply "like")
the first option reported in all cases is "di"
but the 9th feature value is
-11.639 if no context weights are given
-11.595 if the context weights is "all=1.0"
-11.595 if the context weights is "all=0.4"
The difference is small, but nevertheless can cause different
translations
Using ranked sampling I obtain always the same options and same score
-11.639, as expected
Note that this is true using the submodule from
https://github.com/ModernMT/moses-submodule
commit 1cf64e697edd55aa3f110253183d54e19f23fa19
Nicola
Biased sampling
no ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.061
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.023
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.143
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.119
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.060
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.025
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.145
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.117
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.126
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.595,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.060
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.752,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.025
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.211,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.145
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.652,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.117
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.535,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

Ranked sampling
no ctx_weight
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.061
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.023
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.143
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.119
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

ctx_weight:1.0
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.061
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.023
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.143
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.119
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

ctx_weight:0.4
[like ; 0-0]
di :: term=0-0 : nonterm=: c=-0.128
core=(0.000,-1.000,0.000,1.000,-4.710,-8.619,-0.298,-7.596,-11.639,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, :: term=0-0 : nonterm=: c=-0.061
core=(0.000,-1.000,0.000,1.000,-2.312,-6.219,-0.298,-3.785,-7.795,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, la :: term=0-0 : nonterm=: c=0.023
core=(0.000,-2.000,0.000,1.000,-2.312,-8.438,-0.717,-7.596,-8.264,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, come :: term=0-0 0-1 : nonterm=: c=0.143
core=(0.000,-2.000,0.000,1.000,-2.458,-9.823,-0.717,-4.881,-4.714,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

, desidero :: term=0-0 0-1 : nonterm=: c=0.097
core=(0.000,-2.000,0.000,1.000,-2.225,-7.546,-0.717,-7.596,-5.700,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

la :: term=0-0 : nonterm=: c=-0.119
core=(0.000,-1.000,0.000,1.000,-3.100,-6.031,-0.298,-7.596,-10.579,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000,0.000)

On Jan 22, 2016, at 11:16 AM, Davide Caroselli wrote:

Update
I've included the last fix of @nicolabertoldi into MMT and the results
is that:
curl --silent "
http://localhost:8045/translate?q=I+like+it&context=president" | python
-mjson.tool
curl --silent "
http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 1}]"
| python -mjson.tool
curl --silent "
http://localhost:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 0.4}]"
| python -mjson.tool
Have the same translation (good!), but
curl --silent "http://localhost:8045/translate?q=I+like+it" | python
-mjson.tool
curl --silent "
http://localhost:8045/translate?q=I+like+it&context=CJHKECJKHVECVOHUEVCHBVHCBLJHVSLKC"
| python -mjson.tool
Have different translation.
It appears now to be a boolean condition that changes the translation:
there is context, or not. Actually Nicola has observed different scores for
Phrase Tables, @ugermann have you any idea of the reason?
By the way, we found a more worrying bug: Nicola was able to find
better translations with the same models directly with Moses:
I like it -> Mi come essa (acceptable if we consider that the model is
too small)
instead of
I like it -> In un' interrogazione successiva , non fa che (totally
wrong)
I've attached the log files of Moses (verbose 3) with and without
context, in both cases the translation is totally wrong:
with-context.log.zip
no-context.log.zip
We can observe that the Moses library receive the right sentence, and
that the output is the same as MMT: the problem should be within Moses
"space".
Comments would be much appreciated!
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-173964666.

--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh

further NEGATIVE update
Always using ranked sampling and normalized context weights in LM.....
my previous experiments were done using this parameter "--max-partial-trans-opt 5", which should determine a smaller/larger amount of translation options
If I enlarge it (to 10), the difference re-appears again
without context weights
BEST TRANSLATION: In un' interrogazione successiva , non fa che [111]  [total=-1.045] core=(0.000,-8.000,0.000,3.000,-10.367,-29.457,-3.224,-23.799,-17.006,-3.085,0.000,0.000,0.000,-2.169,0.000,0.000,0.000,-34.023)
with context weight (all=1.0)
BEST TRANSLATION: In un' , non fa che [111]  [total=-1.090] core=(0.000,-6.000,0.000,3.000,-10.367,-25.794,-2.215,-23.799,-17.006,-3.085,0.000,0.000,0.000,-2.169,0.000,0.000,0.000,-35.003)
The issue is still there if I use a  smaller value (=4, i.e. smaller search space) for that parameter, I got different translations
without context weights
BEST TRANSLATION: In un' interrogazione successiva , come , ne ha [111]  [total=-1.098] core=(0.000,-9.000,0.000,3.000,-11.377,-34.203,-3.643,-24.895,-13.925,-4.317,0.000,0.000,0.000,-1.609,0.000,0.000,0.000,-41.848)
with context weight (all=1.0)
BEST TRANSLATION: , al termine , come , ne ha [111]  [total=-1.240] core=(0.000,-8.000,0.000,3.000,-10.279,-34.686,-3.117,-24.895,-13.925,-4.317,0.000,0.000,0.000,-2.303,0.000,0.000,0.000,-47.793)

@ugermann here's all the data involved and all the references you need:
Training data: training.tar.gz
Engine models: default.tar.gz
MMT system
branch: master
commit: latest (sha1: bd3906bf84116b3a61469fbfbb8d4029c904ee99)
Moses lib
repo: https://github.com/ModernMT/mosesdecoder.git
branch: mmt-dev_issue49
commit: c631c5fa39714620c3c85de9df3aa682b79605ba
IRSTLM lib
repo: https://github.com/irstlm-team/irstlm.git
branch: cd-correction-model
commit: 4c1a64aa053fde03f2d0f2ee6d49f69ac312fad9
If you need any other info, please don't hesitate to ask me.

@davidecaroselli the tuned weights for the phrase table look strange - it's unusual for them to be negative. I wonder if this is the reason why Moses is selecting bad translation options.
Could you post the tuning data set? I did not see it in either of the tarballs you posted. And do you have the intermediate files from the tuning run?
Having said that, the training data set is only 6000 parallel sentences, so it could simply be the small size of the training data which causes the strange behaviour.

Thanks all
We are making some progress finding other issues, but just to keep all people focused, our main problem for this threads is:
When passing or not passing the context on a single domain, the translation output must be the same.
By design they have to be equal.
We should solve that and move to the next.
@bhaddow good point, please open a separate issue for that. Unfortunately this is a new potential issue.

@bhaddow Good observation, but the training corpus for tracking this issue that Davide sent me (and that the logs appear to be based on) consists of only 6000 sentence pairs, which is great for debugging (easy to exchange and investigate data), but it will of course lead to a bad system (bad word alignments, and downhill from there). This explains some of the things that puzzled us yesterday when we were combing through the logs in search of evidence.

@bhaddow
I guess @marcotrombetti pointed out more a behavioural rather than performance issue.
So a very small data set is optimal to track and fix the problem. Hence, the
output can be totally wrong but it should deterministically be the same with
and without context.

Let me try to explain things a little bit.
@bhaddow and @ugermann the data you're using and the weights chosen has been created only with one purpose: point out this bug. As @mfederico said, we are not talking about quality of translations: they're awful, we expected that. The problem is that the same exact system is performing in two different ways when it shouldn't. The presence of the context should not impact the translation in any way when there is only one domain.
So no need to open a bug report, @marcotrombetti and @bhaddow, those weights has been modified to let the bug appear more clearly. They're not intended to be acceptable weights for a translation engine.

Fixed now in branch mmt-dev_issue49, hopefully.
On Sun, Jan 24, 2016 at 12:15 AM, Davide Caroselli<EMAIL_ADDRESS>
wrote:

Let me try to explain things a little bit.
@bhaddow https://github.com/bhaddow and @ugermann
https://github.com/ugermann the data you're using and the weights
chosen has been created only with one purpose: point out this bug. As
@mfederico https://github.com/mfederico said, we are not talking about
quality of translations: they're awful, we expected that. The problem is
that the same exact system is performing in two different ways when it
shouldn't. The presence of the context should not impact the translation in
any way when there is only one domain.
So no need to open a bug report, @marcotrombetti
https://github.com/marcotrombetti and @bhaddow
https://github.com/bhaddow, those weights has been modified to let the
bug appear more clearly. They're not intended to be acceptable weights for
a translation engine.
—
Reply to this email directly or view it on GitHub
https://github.com/ModernMT/MMT/issues/49#issuecomment-174236626.

--
Ulrich Germann
Senior Researcher
School of Informatics
University of Edinburgh

Hi @ugermann, unfortunately the bug seems unresolved yet.
I've compiled new version of moses with you're last commit https://github.com/ModernMT/mosesdecoder/commit/2bab48c9107af08d7795b35b6e5db8f0b45b005b
Then I've setup and engine with the data I've provided, the output of these 2 commands should be equal:
$ curl --silent "http://localhost:8045/translate?q=I+like+it&context=president" | python -mjson.tool
{
    "context": [
        {
            "id": "all",
            "score": 0.051096026
        }
    ],
    "translation": "In un', non fa che"
}

$ curl --silent "http://localhost:8045/translate?q=I+like+it" | python -mjson.tool
{
    "translation": "In un'interrogazione successiva, non fa che"
}

If you're not observing this problem anymore, could you please provide me instructions to replicate the right env?
Thank you!

Try branch mmt-merge from mosesdecoder repo. At least for the toy system you sent me, results now seem consistent. Related to problems with apparent deterioration of results with ranked sampling, I had to spend a considerable amount of effort rolling back the changes that were made by FBK in an effort to force parameter changes into moses at runtime during batch decoding via DLT, which broke the program logic (just for the record: batch decoding is currently always single-session, and you can't change context weights within a running session without wrecking havoc).
Most important changes in connection with this branch:

Different weights for LM and TM may lead to better translations, so it makes sense to offer both. So we now have separate weight sets for LM and Phrase table:

context-weights for the phrase table
lm-interpolation-weights for IRSTLM

The encoding format of the domain-to-weight-map is the same for both parameters. FBK's special map representation format for command line, a proper xmlrpc-encoded dictionary for server requests. So no change for (--)context-weights, except that IRSTLM will ignore them UNLESS ...

... the new parameter consider-context-weight={true|false} in the IRSTLM line in moses.ini  is set to true, in which case IRSTLM will look for context weights (and, currently, automatically normalize these irrespective of whether weight_normalization in the IRSTLM ini line is set to true or false), but only if no LM interpolation weights are provided. If consider-context-weights=false, IRSTLM won't provide adapted LM scores if no LM interpolation weights are provided.

Didn't mean to close the issue before the fix is confirmed.

Hi Uli
Could you please clarify which changes you did to ensure that sampling without passing context weights is identical to sampling passing context-weights equal to 1.0 in the case of one single domain?
I would like to understand which error I made, to avoid to repeat again in the future.
If I well understood (but I hope I didn't), you completed remove the possibility of changing (lm-)context weights by means of dlt.
This would break our possibility to run batch translation.
Sincerely, I do not get why you claim

force parameter changes into moses at runtime during batch decoding via DLT, which broke the program logic (just for the record: batch decoding is currently always single-session, and you can't change context weights within a running session without wrecking havoc

which is the "program logic"?
If I correctly interpret the log of branch mmt-merge, this was generated by cloning mmt-dev into mmt-merge and then merging master into mmt-dev.
Why did you do so? What portion of code of master branch did you need?
Furthermore, I have some comments about your changes:
X) My implementation to change LM interpolation weights included the possibility to specify which LM (lm_id) a specific set of weights refers to. In principle, we can have more than one interpolated LM, each with its own IDs, and you could decide to use different context weights for each of them.
This was implemented by means of a map of map of weights.
Actually, getting this map of map worked only in the batch case.
For the server case  we planned to do it soon, and currently we backoff-ed to use directly the context weights.
Actually, if I well interpreted the code,
your changes for modifying LM interpolation weights does not replicate our previous implementation in the batch case,
because it allows only one set of lm context weights shared among all LMs.
I would like to re-introduce this functionality.
X) Ideally, having different set of context weights could be implemented also for SA. In principle you could have many SAs generating translation options. What happens in this case with the context weights?
X) I would like to change the policy about "backoff" to context weights and normalization.
I would prefer this schema:

if lm-context weights are specified, use them
else if context-weights are specified, use them
else use the default weights specified in the LM (usually through the configuration file)
In any of the above cases
if weight_normalization is set, apply normalization
else use the weights as they are.

X) I would like to change the name of parameters, because context weights specific to LM could in principle used by other LM types and not only for the interpolated LM.
Hence I would go for renaming internal variables, functions and parameters name in this perspective.
replacing "lm-interpolation" with "lm-context"
The feedback of any partner is very welcome.
Nicola
On Feb 4, 2016, at 1:55 AM, Ulrich Germann wrote:

Try branch mmt-merge from mosesdecoder repo. At least for the toy system you sent me, results now seem consistent. Related to problems with apparent deterioration of results with ranked sampling, I had to spend a considerable amount of effort rolling back the changes that were made by FBK in an effort to force parameter changes into moses at runtime during batch decoding via DLT, which broke the program logic (just for the record: batch decoding is currently always single-session, and you can't change context weights within a running session without wrecking havoc).
Most important changes in connection with this branch:
Different weights for LM and TM may lead to better translations, so it makes sense to offer both. So we now have separate weight sets for LM and Phrase table:
context-weights for the phrase table
lm-interpolation-weights for IRSTLM
The encoding format of the domain-to-weight-map is the same for both parameters. FBK's special map representation format for command line, a proper xmlrpc-encoded dictionary for server requests. So no change for (--)context-weights, except that IRSTLM will ignore them UNLESS ...
... the new parameter consider-context-weight={true|false} in the IRSTLM line in moses.ini is set to true, in which case IRSTLM will look for context weights (and, currently, automatically normalize these irrespective of whether weight_normalization in the IRSTLM ini line is set to true or false), but only if no LM interpolation weights are provided. If consider-context-weights=false, IRSTLM won't provide adapted LM scores if no LM interpolation weights are provided.
—
Reply to this email directly or view it on GitHub.

@nicolabertoldi you're saying that the changes broke in some way the InterpoledIRSTLM correct functioning? Or is it just a matter of different design choices?
@ugermann do the changes made to the IRSTLM-Moses connector are necessary for the SA-problem fix? If not, is it possible to rollback to previous working version of this component?
Please note also that batch operation is not something MMT needs so any effort in that direction is not pertinent with the MMT development. In other words: I need the MMT system (that includes a modified version of Moses XMLRPC Server) to work correctly, and I did not understand if this fix has been tested on MMT.
Before close this issue I need to check that the whole system is still consistent also with the IRSTLM component, @nicolabertoldi can you check that please?
Thank you.

On Feb 4, 2016, at 9:41 AM, Davide Caroselli wrote:

@nicolabertoldi you're saying that the changes broke in some way the InterpoledIRSTLM correct functioning? Or is it just a matter of different design choices?

I have to wait some feedback from Uli in ordert to answer you

@ugermann do the changes made to the IRSTLM-Moses connector are necessary for the SA-problem fix? If not, is it possible to rollback to previous working version of this component?
Please note also that batch operation is not something MMT needs so any effort in that direction is not pertinent with the MMT development. In other words: I need the MMT system (that includes a modified version of Moses XMLRPC Server) to work correctly, and I did not understand if this fix has been tested on MMT.

I agree that batch translation  is not crucial for MMT, but consider that:

it can speed up experimentation
it could speed tuning if  done without the client-server infrastructure
actually, also in MMT one could think of having a batch version of the system for specific purposes (tuning)

Before close this issue I need to check that the whole system is still consistent also with the IRSTLM component, @nicolabertoldi can you check that please?

After getting some feedback from Uli, I will check.
Nicola

@nicolabertoldi ok then, @ugermann can we solve this aspect as soon as possible?
Regarding batch version of Moses, please consider that we've already succeed in speeding up tuning time by a factor of N with the distributed infrastructure, where N is the number of connected machine so currently batch Moses is not a part of MMT.

Using the code of moses from
https://github.com/ModernMT/mosesdecoder.git
branch mmt-merge
commit: c41eda494c0554a337925b462fcdf6fce5cf464a
Using the code of moses-submoduule from
https://github.com/ModernMT/moses-submodule
branch: master
commit: 1cf64e697edd55aa3f110253183d54e19f23fa19
Using the data prepared time ago by Marco ("default.tar.gz) which includes just one domain (called all)
Using a standard LM (all.alm, which is not interpolated and not adaptive); this is the only difference with respect to moses.ini in the archive.
Modifying the context-weights on the command line,
I got these outputs
echo "I like it" | moses -f default/moses.ini_runtime -v 3 --context-weights "all,0.4" >& LOG.dlt0.4
BEST TRANSLATION: In un' interrogazione successiva , non fa che [111]  [total=-1.043] core=(0.000,-8.000,0.000,3.000,-10.367,-29.457,-3.224,-23.799,-16.962,-3.085,0.000,0.000,0.000,-2.169,0.000,0.000,0.000,-34.023)
echo "I like it" | moses -f default/moses.ini_runtime -v 3 --context-weights "all,1.0"
BEST TRANSLATION: In un' interrogazione successiva , non fa che [111]  [total=-1.043] core=(0.000,-8.000,0.000,3.000,-10.367,-29.457,-3.224,-23.799,-16.962,-3.085,0.000,0.000,0.000,-2.169,0.000,0.000,0.000,-34.023)
echo "I like it" | moses -f default/moses.ini_runtime -v 3 >& LOG.nodlt
BEST TRANSLATION: In un' interrogazione successiva , non fa che [111]  [total=-1.045] core=(0.000,-8.000,0.000,3.000,-10.367,-29.457,-3.224,-23.799,-17.006,-3.085,0.000,0.000,0.000,-2.169,0.000,0.000,0.000,-34.023)
Outputs are always the same, but I think is only for chance, because the scores actually differ.
The feature scores in the case of translating without context weights, are different from those with context weights (regardless the value).
In particular, the only different score is the last (5th out of 5) produced by the SA.
I got the same results and scores if I translate using MMT system and the curl calls
as you can see here
Note that the command-line parameter "--context-weights" works properly in the sense that the weights are passed to the SA.
And it works exactly as in the xmlrpc communication
as you can see below where I got scores asking for the first best
with context weight: 0.4
$> curl "http://<IP_ADDRESS>:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 120.4}]&nbest=1" |  python -mjson.tool
"nbest": "translation": "In un'interrogazione successiva, non fa che" }
"scores": {
"DM0":  [ -3.08534, 0.0, 0.0, 0.0, -2.1690499999999999, 0.0, 0.0, 0.0 ],
"Distortion0":  [ 0.0 ],
"LM0":  [ -34.022599999999997 ],
"PT0":  [ -10.367000000000001, -29.4573, -3.2243900000000001, -23.798999999999999, -16.962199999999999 ],
"PhrasePenalty0":  [ 3.0 ],
"WordPenalty0":  [ -8.0 ] },
"totalScore":  -1.0430328,
],
with context weight: 1.0
$> curl "http://<IP_ADDRESS>:8045/translate?q=I+like+it&context_array=[{"id"%3A "all"%2C"score"%3A 121.0}]&nbest=1" |  python -mjson.tool
"nbest": "translation": "In un'interrogazione successiva, non fa che" }
"scores": {
"DM0":  [ -3.08534, 0.0, 0.0, 0.0, -2.1690499999999999, 0.0, 0.0, 0.0 ],
"Distortion0":  [ 0.0 ],
"LM0":  [ -34.022599999999997 ],
"PT0":  [ -10.367000000000001, -29.4573, -3.2243900000000001, -23.798999999999999, -16.962199999999999 ],
"PhrasePenalty0":  [ 3.0 ],
"WordPenalty0":  [ -8.0 ] },
"totalScore":  -1.0430328,
],
with context string "president"   which receives a weights of 0.051096026000000003
$> curl "http://<IP_ADDRESS>:8045/translate?q=I+like+it&context=president&nbest=1" |  python -mjson.tool
"nbest": "translation": "In un'interrogazione successiva, non fa che" }
"scores": {
"DM0":  [ -3.08534, 0.0, 0.0, 0.0, -2.1690499999999999, 0.0, 0.0, 0.0 ],
"Distortion0":  [ 0.0 ],
"LM0":  [ -34.022599999999997 ],
"PT0":  [ -10.367000000000001, -29.4573, -3.2243900000000001, -23.798999999999999, -16.962199999999999 ],
"PhrasePenalty0":  [ 3.0 ],
"WordPenalty0":  [ -8.0 ] },
"totalScore":  -1.0430328,
],
without context
$> curl "http://<IP_ADDRESS>:8045/translate?q=I+like+it&nbest=1" |  python -mjson.tool | perl -pe 's#\n##g;' | perl -pe 's#":#":\n#g;s# "#\n"#g;' |perl -pe 's#[\t ]+# #g;'
"nbest": "translation": "In un'interrogazione successiva, non fa che" }
"scores": {
"DM0":  [ -3.08534, 0.0, 0.0, 0.0, -2.1690499999999999, 0.0, 0.0, 0.0 ],
"Distortion0":  [ 0.0 ],
"LM0":  [ -34.022599999999997 ],
"PT0":  [ -10.367000000000001, -29.4573, -3.2243900000000001, -23.798999999999999, -17.005600000000001 ],
"PhrasePenalty0":  [ 3.0 ],
"WordPenalty0":  [ -8.0 ] },
"totalScore":  -1.0447181000000001,
],
Hopefully, I did not make mistakes.
SO, I suspect that the bug #49 is still there and it was not caused by dlt tag management.
Uli, please try again, and tell us if you get the same results as mine,
or something different
cheers,
Nicola

investigating deeper in the issue #49...
I translated the same sentence "I like it"
without passing any weight and passing the context string "president"
and I looked into the logs (verbose=3)  (in attachment)
and I  noticed that       (il numero iniziale indica la riga all'interno del rispettivo log)
1)
InitializeForInput   gives different scores; small differences but due to what?  what are exactly these scores?
without_context
102  InitializeForInput( UnknownWordPenalty0 )= 6.962e-06
103  InitializeForInput( WordPenalty0 )= 3.33e-06
104  InitializeForInput( Distortion0 )= 3.324e-06
105  InitializeForInput( PhrasePenalty0 )= 3.341e-06
106  InitializeForInput( PT0 )= 4.5108e-05
107  InitializeForInput( DM0 )= 3.357e-06
108  InitializeForInput( LM0 )= 3.638e-06
with_context
102  InitializeForInput( UnknownWordPenalty0 )= 6.81e-06
103  InitializeForInput( WordPenalty0 )= 3.012e-06
104  InitializeForInput( Distortion0 )= 2.989e-06
105  InitializeForInput( PhrasePenalty0 )= 3.126e-06
106  BIAS WEIGHTS GIVEN WITH INPUT at moses/TranslationModel/UG/mmsapt.cpp:831
107  BIAS all 1
108  InitializeForInput( PT0 )= 8.0332e-05
109  InitializeForInput( DM0 )= 3.005e-06
110  InitializeForInput( LM0 )= 3.327e-06

some translation options have different scores, in particular only the 9th values of the core scores  (i.e the 5th of the 5 scores provided by the SA)
without _context
131  [like ; 1-1]
132          di :: term=0-0 : nonterm=: c=-0.12768 core=(0,-1,0,1,-4.70953,-8.61867,-0.298077,-7.59613,-11.6389,0,0,0,0,0,0,0,0,0)
133          , :: term=0-0 : nonterm=: c=-0.061316 core=(0,-1,0,1,-2.31164,-6.21878,-0.298077,-3.78489,-7.7953,0,0,0,0,0,0,0,0,0)
134          , la :: term=0-0 : nonterm=: c=0.0232683 core=(0,-2,0,1,-2.31164,-8.43837,-0.716505,-7.59613,-8.26384,0,0,0,0,0,0,0,0,0)
135          , come :: term=0-0 0-1 : nonterm=: c=0.142837 core=(0,-2,0,1,-2.45824,-9.82292,-0.716505,-4.88068,-4.71418,0,0,0,0,0,0,0,0,0)
136          , desidero :: term=0-0 0-1 : nonterm=: c=0.0966651 core=(0,-2,0,1,-2.22462,-7.54565,-0.716505,-7.59613,-5.70044,0,0,0,0,0,0,0,0,0)
137          la :: term=0-0 : nonterm=: c=-0.118795 core=(0,-1,0,1,-3.10009,-6.03069,-0.298077,-7.59613,-10.579,0,0,0,0,0,0,0,0,0)
138          la disponibilità :: term=0-0 : nonterm=: c=0.087628 core=(0,-2,0,1,-3.10009,-6.81914,-0.716505,-7.59613,-4.60517,0,0,0,0,0,0,0,0,0)
139          non mi :: term=0-1 : nonterm=: c=-0.0170839 core=(0,-2,0,1,-4.01638,-6.14902,-0.716505,-7.59613,-5.9877,0,0,0,0,0,0,0,0,0)
140          come :: term=0-0 : nonterm=: c=-0.0188991 core=(0,-1,0,1,-2.63009,-3.60414,-0.298077,-2.32747,-3.45226,0,0,0,0,0,0,0,0,0)
with_context
133  [like ; 1-1]
134          di :: term=0-0 : nonterm=: c=-0.125974 core=(0,-1,0,1,-4.70953,-8.61867,-0.298077,-7.59613,-11.595,0,0,0,0,0,0,0,0,0)
135          , :: term=0-0 : nonterm=: c=-0.0596306 core=(0,-1,0,1,-2.31164,-6.21878,-0.298077,-3.78489,-7.75188,0,0,0,0,0,0,0,0,0)
136          , la :: term=0-0 : nonterm=: c=0.0253118 core=(0,-2,0,1,-2.31164,-8.43837,-0.716505,-7.59613,-8.2112,0,0,0,0,0,0,0,0,0)
137          , come :: term=0-0 0-1 : nonterm=: c=0.145256 core=(0,-2,0,1,-2.45824,-9.82292,-0.716505,-4.88068,-4.65189,0,0,0,0,0,0,0,0,0)
138          , desidero :: term=0-0 0-1 : nonterm=: c=0.0966651 core=(0,-2,0,1,-2.22462,-7.54565,-0.716505,-7.59613,-5.70044,0,0,0,0,0,0,0,0,0)
139          la :: term=0-0 : nonterm=: c=-0.117087 core=(0,-1,0,1,-3.10009,-6.03069,-0.298077,-7.59613,-10.5351,0,0,0,0,0,0,0,0,0)
140          la disponibilità :: term=0-0 : nonterm=: c=0.087628 core=(0,-2,0,1,-3.10009,-6.81914,-0.716505,-7.59613,-4.60517,0,0,0,0,0,0,0,0,0)
141          non mi :: term=0-1 : nonterm=: c=-0.00593171 core=(0,-2,0,1,-4.01638,-6.14902,-0.716505,-7.59613,-5.70044,0,0,0,0,0,0,0,0,0)
142          come :: term=0-0 : nonterm=: c=-0.0167746 core=(0,-1,0,1,-2.63009,-3.60414,-0.298077,-2.32747,-3.39754,0,0,0,0,0,0,0,0,0)
hope this help debugging....
Is it possible that there is some different random seed in the two cases?
Are we sure that the two cases should produce the same output?
Nicola
log_withcontext.txt
log_withoutcontext.txt

As far as I can tell, this last issue is fixed by commit 2ed99f94832686191e8a495f66f1e5a48e1138e3 (branch mmt-merge).

This issue has been fixed with the latest mosesdecoder commit https://github.com/ModernMT/mosesdecoder/commit/ed1910bcce8bed92b06d6f3c939ccad4ec78cb06 on branch mmt-dev.
Thanks to all for your collaboration.

As confirmed by @ugermann the fix consist in changing the way the decoder works when a single domain is present: if the SA detect there's only one single domain in the index, it will ignore any provided context-weights.


JsonFormatter: add option to ignore empty context and extra fields

Similar to efe572cb1074.

Also may I know what the use case is for this? For LineFormatter it kinda made sense to clean up the output as it's meant for human consumption, but in the json output I see less value.

@Seldaek We have an internal logging system that is manipulating each log record.
At the end of the process, extra is an empty array, and this system expects an object.
We'll force an object for it with something like {"extra": {"_force_object": true}}, but ideally we'd like to remove if it's empty :)

But the new stdClass code there was exactly about forcing it to be an object.. Are you decoding/re-encoding it somehow? Because it seems to me like the type should be correct when coming out of JsonFormatter

But the new stdClass code there was exactly about forcing it to be an object

Indeed: $normalized['extra'] = new \stdClass;

Are you decoding/re-encoding it somehow? Because it seems to me like the type should be correct when coming out of JsonFormatter

I'll check that, we have some custom Monolog Processors as well, they might be changing that 👍

Processors happen before the formatter. If you have problems I'd imagine it's something you do later with the log file, if you json_decode with $assoc=true, then you get an empty array, and if you json_encode that you'll encode it back into an array instead of an object.
Anyway I guess the option here doesn't hurt so I'll merge, but technically it sounds like you're doing something wrong somewhere.


Raise a warning when a private method exists with signature that is looks like overload of a public method

Is your feature request related to a problem? Please describe.
See sample: https://github.com/gradle/gradle/issues/15660
Users might think that private methods do not leak to the "public API" scope, however, Java Language Specification suggests that permissions check is the last step in the overload resolution: https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html#jvms-<IP_ADDRESS>
Describe the solution you'd like
No idea. Ideally, g-a-a-g-p could detect such cases (private method which looks like an overload and which uses one of "non-api" classes) and raise a suggestion to rename the private method.

Thanks for the issue. I was not aware of this fact:

Java Language Specification suggests that permissions check is the last step in the overload resolution

I do think it would be possible to detect this situation. I know you said you have "no idea" what a solution would look like, but if you find time to consider and propose one, it would be helpful and make it more likely a feature like this would be added sooner than later. I'm asking for the user-facing "API", not how it would be implemented.

Thanks for the issue. I was not aware of this fact:

Java Language Specification suggests that permissions check is the last step in the overload resolution

I do think it would be possible to detect this situation. I know you said you have "no idea" what a solution would look like, but if you find time to consider and propose one, it would be helpful and make it more likely a feature like this would be added sooner than later. I'm asking for the user-facing "API", not how it would be implemented.

I'm asking for the user-facing "API"

Frankly speaking, I think it would be nice if the feature could work automatically. There might be user-facing api to suppress the warning, however, I belive the feature does not need configuration and it should be pretty much safe to be activated by default.

I'm asking for the user-facing "API"

Frankly speaking, I think it would be nice if the feature could work automatically. There might be user-facing api to suppress the warning, however, I belive the feature does not need configuration and it should be pretty much safe to be activated by default.

I shouldn't have used the word "API", that was confusing. I meant "what should the warning look like, in the context of the other output generated by the plugin?"

I shouldn't have used the word "API", that was confusing. I meant "what should the warning look like, in the context of the other output generated by the plugin?"


set right cache strategy

Fixes #6
Please review
@HisashiHasebe @yabe-diverta san
For reviewers

Please delete the .next and node modules from root directory and run npm install and npm run dev again
This step is important because it will remove the existing API cache stored in the .next files and create a new cache instance

Updated cache strategy

The results for almost all GET requests will be cached for performance boost and since we expect the backend data to not change as frequently.

cache no-store is set only for :

/rcms-api/1/content/details/${id} : because fav_cnt count is updated frequently on details page
/rcms-api/1/ranking : dependent on number of likes
/rcms-api/1/my_favorite_list : dependent on logged in user's likes

No need no-store to /rcms-api/1/member/me ?
Other than that, I think it is OK if GithubActions are run when content is updated and reflected on the front page. (Next's cache does not misbehave).
I will check from the front end after the build error is resolved.

@HisashiHasebe san
On investigation I found that /rcms-api/1/member/me works fine without using no-store so I still think it won't be necessary.
Though I would like to request @yabe-diverta san to also confirm this once.

@VortexExpansion

On investigation I found that /rcms-api/1/member/me still works fine without using no-store
so I think it won't be necessary.

Is it okay to have no-store on /rcms-api/1/member/me?
If we want to cache, we don't set no-store; if not, we set no-store. I think it's better to unify the approach based on these reasons.

@HisashiHasebe san
Yes it is OK to have but not necessary to have.
I did not use no-store because we want to cache the current member info so that we don't have to fetch the same information on every request because this information is not being updated frequently so it is more optimal to fetch the request from database once and then use the cache for every other call to prevent calling overhead.
My understanding is that when different users are logged in, fetching the request once and then using their stored cache for every other call within their local is most optimum performance wise.
I'm sorry if I'm still missing your perspective. Please let me know. 🙇

@VortexExpansion @HisashiHasebe
Sry for you, due to manager's claim, we have to make all the frontend-fetch to be dynamic.
https://github.com/diverta/front_nuxt_corporate/issues/14
For novice users, we should not to have

server-side fetch / prefetch
client fetch cache
because some ppl would get confused that why A content is always fresh but B content has been stale unless build.
This case is A is client fetch without cache, B is prefetch on server-side.

To not to give frustrations to those users I think we need to make all fetch to be client fetch w/o cache.

@yabe-diverta san
Ok will make all fetch requests CSR

@VortexExpansion
Sry for waiting you so much, found this issue is unable to merge due to my update.
I had done drastically change for entire codebase, would you rework over it?  😅

Progress for client side fetching :

[ ] article/[id]
[x] contact
[ ] feature/[id]
[ ] login
[x] member/delete
[x] member/edit
[ ] member/limited[id]
[x] member/mypage
[ ] register
[ ] reminder
[ ] index

All dynamic pages to be handled separately in https://github.com/diverta/front_next_media/issues/21
Marking this ready for review now.
@yabe-diverta san


removed  space between hyphen and DisableCpuThrottleOnIdleScans

this is my own PR  , i checked this article and i found space should be removed between  hyphen and DisableCpuThrottleOnIdleScans

@e0i  @nam31, please look in this PR  #2332

Docs Build status updates of commit 201421d:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/defender/Set-MpPreference.md
:white_check_mark:Succeeded
View (win10-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

@opbld32. is any change needed from my side.

Hello, @RAJU2529 opbld32 is an automated bot, and nam31 does not coordinate this repository. It’s not necessary to tag us to move forward with the review. FYI and thank you for this contribution.

Docs Build status updates of commit 4599703:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/defender/Set-MpPreference.md
:white_check_mark:Succeeded
View (WindowsServer2019-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 0372a13:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docset/windows/defender/Set-MpPreference.md
:white_check_mark:Succeeded
View (WindowsServer2016-ps)

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Possible to use with one modality? What are the data size limits?

Hi @viktoriaschuster
Congratulations on getting this to a paper stage! Excited to try on my data. A few questions:

Is the package possible to use with one modality?
What are the data size limits?
A) The number of cells. Can I use 100k or 600k cells?
B) The number of DNA regions in ATAC modality - this value is commonly very large eg 300-600k regions or even 1mln regions.
Have you tried applying to data with 100-500 distinct cell populations? How should the hyperparameters of the decoder and latent space modified to accommodate?

Hi @vitkl,
Thanks a lot! To answer your questions:

It is designed to be flexible in terms of number of modalities, but I haven't tested it yet. The best setup is to either use a mudata object or an anndata object with a var column for 'modality'. There is also my older package scDGD (https://github.com/Center-for-Health-Data-Science/scDGD) for single cell RNA counts. All of this is still in development, so I am still working on harmonizing these two.
So far, we have worked with up to 100k cells and ~300k features. Based on this and previous work, I am expecting multiDGD to scale better with number of features than MultiVI, but MultiVI performing better when reaching 1M cells with <100k features.
The data sets I have applied so far only went up to ~50 cell types. In such a case as you are describing, it really depends on what level of clustering you would like to get. If you want these many cell populations to be as stringently separated as possible, go with the default setting of number of Gaussian Mixture components (taken from the 'observable' in the data set) and maybe set the scale of the mean prior to something larger ('softball_scale' > 2).

I see that it is currently hard to use the model with any customization. I will shortly add a preliminary documentation :)

Thanks for getting back. The first two points are clear but I don't fully understand what you suggest tuning in point 3. A documentation website would be great indeed!


Add a sample application for the combination of Microservice transaction and Spring Data JDBC for ScalarDB

Context
scalardb-samples doesn't have a sample application that uses microservice transaction and Spring Data JDBC for ScalarDB. It should be very helpful for users as it's a bit complicated to use Spring Data JDBC for ScalarDB to develop an actual microservice system.
Changes
This PR adds the sample application based on

https://github.com/scalar-labs/scalardb-samples/tree/main/microservice-transaction-sample
https://github.com/scalar-labs/scalardb-samples/tree/main/spring-data-multi-storage-transaction-sample

@reddikih Can you try running it when you get a chance? It can be after the PR is merged.

I'll revise the README based on the recent feedbacks from @josh-wong as a new PR. Thank you!


Reopen last edited files menu

I'd really love if something like that would get implemented, shouldn't be to hard I guess?

Would require saving metadata to the filesystem, which I'm not a fan of doing. Then again, if I reimplement themes in this coming 2.0 version, I'll be saving metadata anyway, so I could do it... But it would require extending the menu system to support submenus, which is much more complicated...


rdkit.Chem.rdfiltercatalog.FilterCatalogs.BRENK is still exists? or abounded. If abounded then in which version it is available?

I want to FilterCatalogs. RDKit 2022.09.1 does not have BRENK, NIH, PAINS filters. but in documentation (https://www.rdkit.org/docs/source/rdkit.Chem.rdfiltercatalog.html) it is still available. I want to know that rdkit.Chem.rdfiltercatalog.FilterCatalogs.BRENK is still exists? or abounded. If abounded then in which version it is available?

The issue template asks you a series of questions that make it easier for us to help.
The fact that you've either deleted those or only answered a subset of them makes it more difficult for us to help, so I'm closing this issue.
Please feel free to edit the issue description to include the requested information and reopen the issue and then we'll try to help.