[ALLUXIO-3050] Optimize performance of tiny read

https://alluxio.atlassian.net/browse/ALLUXIO-3050
This is the first PR to fix the tiny read performance issue, the goal of this PR is to reduce the number of computations for computing block IDs based on mPos for each call to updateStreams.
I did an experiment reading a 8GB file cached in Alluxio's memory tier with a 8B read buffer, the profiling of Alluxio client process shows that getBlockId and getCurrentBlockId takes more than 1/3 of the total time.

This PR changes the logic in updateStreams:

For streaming read, the block stream is updated based on whether the current block stream has all been consumed, we keep track of blockIdIndex instead of computing the index every time updateStreams is called.
For seek and skip, they can update streams based on computing the new block index from mPos.

I ran several experiments for reading a 4GB in memory file with an 8B read buffer with read type CACHE_PROMOTE. The improvement of read throughput for 1 thread is around 48% improvement.
But for 1 thread, comparing the read throughput with 1MB read buffer below, there is at least 2x slowdown for the 8B read buffer case, so there might still have space for optimization.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17334/
Test PASSed.

@calvinjia Could you explain a bit more? I'm a bit confused.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17342/
Test PASSed.

Thanks for the comments, I've updated the PR, PTAL.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17343/Failed Tests: 5org.alluxio:alluxio-core-client-fs: 1alluxio.client.file.FileInStreamTest.seekBackwardSmallSeekBuffer[2]org.alluxio:alluxio-tests: 4alluxio.client.FileInStreamIntegrationTest.eofSeekalluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[0]alluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[1]alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17344/Failed Tests: 5org.alluxio:alluxio-core-client-fs: 1alluxio.client.file.FileInStreamTest.seekBackwardSmallSeekBuffer[2]org.alluxio:alluxio-tests: 4alluxio.client.FileInStreamIntegrationTest.eofSeekalluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[0]alluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[1]alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17345/Failed Tests: 5org.alluxio:alluxio-core-client-fs: 1alluxio.client.file.FileInStreamTest.seekBackwardSmallSeekBuffer[2]org.alluxio:alluxio-tests: 4alluxio.client.FileInStreamIntegrationTest.eofSeekalluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[0]alluxio.client.RemoteReadIntegrationTest.seekAroundLocalBlock[1]alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17346/Failed Tests: 1org.alluxio:alluxio-tests: 1alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17347/Failed Tests: 1org.alluxio:alluxio-tests: 1alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17348/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17349/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17350/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17351/Build result: FAILURE[...truncated 819 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17352/Build result: FAILURE[...truncated 818 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17353/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17359/Build result: FAILURE[...truncated 817 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17360/Build result: FAILURE[...truncated 840 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17362/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17366/Build result: FAILURE[...truncated 818 lines...][JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving<EMAIL_ADDRESS>to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17364/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17365/
Test PASSed.

@uronce-cc What I meant was we currently have two methods update...onRead and update..onSeek. Instead, we could just have update and have it do the expensive computation if mCurrentBlockIndex == -1, then if we seek somewhere, we can set the index to -1

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17367/Build result: FAILURE[...truncated 819 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

@calvinjia I think your proposal is similar to have a boolean to switch between the two types of reads, but after further optimization, the two methods are quite different now, PTAL, thanks.

@apc999 I've updated the PR with further optimizations, I'm running the performance testing.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17370/
Test PASSed.

Merged build finished. Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17371/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17373/
Test PASSed.

1.7.0-SNAPSHOT represents the version with the latest PR, the performance is a bit better(2%) comparing to the initial PR, this is because the microbench read test contains no seek or skip operation, with seek or skip, the performance gain should be more obvious.

@uronce-cc I think unless we see enough performance gain, I would suggest not using bit-shift operations but still divisions to keep code easier to read and maintain. Otherwise, I am good with this PR.

@apc999 SG, I removed the division optimization since the performance gain is just 2%, which is nearly zero if we take variance into account.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17387/
Test PASSed.

@apc999 @gpang Thanks for the comments, I've updated the PR, PTAL.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17392/
Test PASSed.

Merged build finished. Test PASSed.

@gpang @apc999 @calvinjia I refactored the PR to put position related variables into a new class PositionState, this approach is inspired by Gene, thanks.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17403/Build result: FAILURE[...truncated 817 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

For the UnknownLengthFileInStream, I did some refactoring to make it explicit that the file is expected to have just one block, so the file length is determined by the first block's size.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17405/Failed Tests: 2org.alluxio:alluxio-tests: 2alluxio.client.UnderStorageReadIntegrationTest.seekalluxio.master.JournalShutdownIntegrationTest.multiMasterMountUnmountJournal
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17406/Failed Tests: 1org.alluxio:alluxio-tests: 1alluxio.client.UnderStorageReadIntegrationTest.seek
Test FAILed.

jenkins, test this please

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17412/Failed Tests: 3org.alluxio:alluxio-core-client-fs: 3alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[0]alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[1]alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[2]
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17413/Failed Tests: 3org.alluxio:alluxio-core-client-fs: 3alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[0]alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[1]alluxio.client.file.FileInStreamTest.seekToEOFBeforeReadingFirstBlock[2]
Test FAILed.

jenkins, test this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17414/
Test PASSed.

jenkins, test this please

@apc999 @calvinjia @gpang @yupeng9 PTAL, I'm blocked on this to do further improvements, thanks.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17415/
Test PASSed.

Merged build finished. Test PASSed.

jenkins, test this please

@apc999 @calvinjia @gpang any more comments?

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17419/
Test PASSed.

Merged build finished. Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17442/Build result: FAILURE[...truncated 817 lines...][JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/dependency-reduced-pom.xml to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/protobuf/target/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-protobuf/1.7.0-SNAPSHOT/alluxio-core-protobuf-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/pom.xml to org.alluxio/alluxio-keyvalue/1.7.0-SNAPSHOT/alluxio-keyvalue-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/pom.xml to org.alluxio/alluxio-parent/1.7.0-SNAPSHOT/alluxio-parent-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/server/pom.xml to org.alluxio/alluxio-keyvalue-server/1.7.0-SNAPSHOT/alluxio-keyvalue-server-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/underfs/oss/pom.xml to org.alluxio/alluxio-underfs-oss/1.7.0-SNAPSHOT/alluxio-underfs-oss-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/hdfs/pom.xml to org.alluxio/alluxio-core-client-hdfs/1.7.0-SNAPSHOT/alluxio-core-client-hdfs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/keyvalue/common/pom.xml to org.alluxio/alluxio-keyvalue-common/1.7.0-SNAPSHOT/alluxio-keyvalue-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/fs/pom.xml to org.alluxio/alluxio-core-client-fs/1.7.0-SNAPSHOT/alluxio-core-client-fs-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/pom.xml to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/common/target/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar to org.alluxio/alluxio-core-common/1.7.0-SNAPSHOT/alluxio-core-common-1.7.0-SNAPSHOT-tests.jar[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/worker/pom.xml to org.alluxio/alluxio-core-server-worker/1.7.0-SNAPSHOT/alluxio-core-server-worker-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/logserver/pom.xml to org.alluxio/alluxio-logserver/1.7.0-SNAPSHOT/alluxio-logserver-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/client/pom.xml to org.alluxio/alluxio-core-client/1.7.0-SNAPSHOT/alluxio-core-client-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/examples/pom.xml to org.alluxio/alluxio-examples/1.7.0-SNAPSHOT/alluxio-examples-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/pom.xml to org.alluxio/alluxio-core/1.7.0-SNAPSHOT/alluxio-core-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/core/server/proxy/pom.xml to org.alluxio/alluxio-core-server-proxy/1.7.0-SNAPSHOT/alluxio-core-server-proxy-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/shell/pom.xml to org.alluxio/alluxio-shell/1.7.0-SNAPSHOT/alluxio-shell-1.7.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Alluxio-Pull-Request-Builder/tests/pom.xml to org.alluxio/alluxio-tests/1.7.0-SNAPSHOT/alluxio-tests-1.7.0-SNAPSHOT.pomchannel stoppedArchiving artifacts
Test FAILed.

@gpang I removed mCurrentBlockId, mCurrentBlockStartPos, and mCurrentBlockEndPos, PTAL.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17443/
Test PASSed.

Merged build finished. Test PASSed.

@gpang @yupeng9 Comments are resolved, PTAL, thanks.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17445/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17449/
Test PASSed.

@calvinjia any more comments?

@uronce-cc I had a discussion with @calvinjia , let's have a in-person discussion on this PR to address some concerns.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17464/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/17465/
Test PASSed.

After discussion with @calvinjia and @apc999, we decided to hold this PR until another PR for refactoring the caching logic in FileInStream is merged.

@apc999 @calvinjia The issue we talked about the sync between the position states and the updateStreams also exists in the original codebase, to remove the manual sync of updateStreams, we almost need to rewrite the FileInStream class.
I think the current PR addresses the performance issue in maintaining position related states well, so I propose to get this PR in before @apc999 doing the refactoring of the caching logic.
@gpang @yupeng9 What do you think?

I prefer the current state of the PR, mainly because it is a less drastic change to each instream method. This PR optimizes the position accesses and updates by keeping track of some state instead of recomputing that state. It makes sense that those are all enclosed in the PosititionState class.
The actual streams can be updated according to the position, as it was done previously.
If you want to refactor it further, it might be easier to implement/review if it was done in a separate PR, and not this one. Those are my thoughts.

+1 to merging this PR. This is a strict improvement with better modularization, performance and code readability.

@uronce-cc LGTM


Generating safe movements

Dear all,
I have been told that a few years ago a "Virtual Skin" for the iCub was developed that basically "simulated" all movement commands in advance to avoid the iCub hitting itself.
I wanted to try and test it but I only found this page http://wiki.icub.org/wiki/MoBeE_installation which seems to be outdated (the source apparently fails to compile because something in Yarp has changed meanwhile).
So my question is: is the VirtualSkin still supported?
More generally how would you ensure that all movements from iCub are "safe" (meaning that it doesn't break by hitting itself)?

Hi @emilio-cartoni
I believe you're referring to our implementation of Peripersonal Space (PPS) on the iCub with the term "virtual skin". Actually, PPS does not account for a virtual skin but rather for a representation of the space surrounding the robot body that can be used in turn to predict contacts by merging visual and tactile information.
That being said, PPS equips the robot with a further perceptive layer, therefore you need to rely on a robust and reliable visual tracker to elicit "virtual" activation of the skin parts that we predict will be in touch with the approaching object (posing a threat to the safety of robot movements).
The key aspects here are thus: robust perception, reactive control.
By contrast, if you aim to achieve safe movements - as for the main point you made - without the need for complicated trackers and other perceptive magic, I'd recommend you to rely on planning components, among which MoBe is clearly a valid example, although we don't maintain it (you could reach out to the authors directly).
Alternatively, you might want to use our Cartesian Control system that is responsible for planning and performing movements of the iCub end-effectors (mostly hands) in the operational space. In particular, this component provides movements of the iCub arms that are guaranteed not to hit the torso with the upper arm covers as well as it takes into account mechanical limitations imposed by the length of the shoulder's cables.
What is left to the user is to check whether the hand will collide against the iCub body (torso + other hand). However, these checks under given conditions are fairly manageable boiling down to simple geometric verifications: e.g. the Cartesian location of the hand under use must be outside a cylinder centered around the torso.
Of course, we could also extend the inverse kinematics problem to account for such collisions at an earlier stage at the cost of degraded tracking performance; nonetheless, in our experience, this level of sophistication is not really required for a wide variety of Cartesian tasks.
Don't hesitate to ask for further details if you have other questions and/or doubts on the implementation.

Hi @pattacini,
thanks, we were thinking more along the planning components approach but PPS has some interesting features we might need, I'll look into the papers.
Our situation is that we have some projects starting that will all involve the iCub exploring a table with a few objects on it and we would like to have a general way to ensure that the iCub can freely explore any kind of movements but without the risk of breaking the iCub by hitting itself or other objects.
From what we have gathered so far, we could do the following:

Set iCub interaction to compliant mode
Enforce that all movements will be slow speed
Use Cartesian control to avoid hitting itself and the table (geometric checks against a cylinder for torso and a plane for the table)

We would still hit the objects but we do have to interact with them anyway and it shouldn't be a problem given that the collision will be slow speed (I hope!).
On this aspect, it might  be nice to have something like PPS that tracks (multiple) objects on the table and adjusts the speed when near them (so that we could go fast when in "free air" and automatically slow down near the objects).
Some of the projects would like to control the iCub in the joint space, so we would need a translation from the joint space to the Cartesian space (using iKin forward kinematics I guess) before sending their commands through the Cartesian controller.
(I have a feeling this translation could result in some weird behaviour and pose some challenges to the algorithms trying to control and learn in the joints space).
All of this should be as transparent as possible for each project.
So, from a software point of view, we would probably create some kind of proxy where the controller of each project does not interact directly with the iCub but sends the commands first to the "safety proxy" which does the checks and then executes the command (I think MoBe used this approach too).
The controllers will probably stream the desired positions so the "safety proxy" should be fast in doing the translation/checks.
Does it look like a reasonable approach?

That looks all good to me.
Let me only clarify one aspect: PPS does not track objects; it's merely responsible for eliciting virtual activation of the skin taxels involved in the interaction, whose inputs to do so are provided externally by any kind of tracking mechanism.
For tracking objects in a table-top scenario, I'd suggest you give a glance at our IOL component instead. There's an explanatory video showing IOL in action here: https://youtu.be/5PfqoNpPbAE.
Regarding IOL, we don't have a fully documented API yet, and it might be likely that we would need to iterate together to check the installation and to introduce you to its basic functionalities. However, it's doable, since we did it already in the past with several other partners, who can even chime in with their contributions too.

I have a feeling this translation could result in some weird behaviour and pose some challenges to the algorithms trying to control and learn in the joints space.

It's definitely true 😉

Closing for the time being. Feel free to open it again when needed.

That looks all good to me.
Closing for the time being. Feel free to open it again when needed.

@pattacini Ok, thanks for the help!
We'll have a look at IOL too.


Deal with case no result is returned

Not necessary to check the color argument in the loop every time

Thanks !

Actually, I messed up the committed message. Sorry about it.

No problem, we can keep this commit message ;)


Some pages are inverted in color

Has anyone ever had this issue? I get some pages inverted in color randomly (90% of the pages arent, 10% are and are scattered throughout the volume. )How to fix?

please give an input image with that problem

I bet you are in dark mode on your kindle, which is funky with manga.

closing this
reopen if needed


Add option to disregard original content inset when keyboard is visible, use KeyboardInfo for animationDuration

Going to go ahead and merge this so that we can get a version out that supports both Swift 4.1 & Swift 4.2.


cache-control error

Hi!
Thank you for creating Elysiajs & this static plugin!
I'm trying out Elysiajs with Bun (1.1.20) to see if I can setup a basic static file server + API.
However, using Elysia 1.1.3 and @elysiajs/static 1.1.0, I get the following error back in the browser
when trying to access a file from my public dir:
{"name":"TypeError","message":"undefined is not an object (evaluating 'headers[\"cache-control\"]')"}

It seems the Response object doesn't produce any headers. The relevant code:
export const app = new Elysia().use(staticPlugin({
    assets: 'public',
    alwaysStatic: true,
    indexHTML: true,
    headers: {},
})).listen(3030)

When I disable the cache, the error is gone:
export const app = new Elysia().use(staticPlugin({
    assets: 'public',
    alwaysStatic: true,
    indexHTML: true,
    headers: {},
    noCache: true,
})).listen(3030)

I'm not sure if this is a Bun related issue, or whether its something else, so I thought I post it here :)

Thanks for reporting this finding!
Can't reproduce it on my end. Can you give it a try with the latest verion?


Alternative camera modules

I've been looking into alternative camera modules which include UVC functionality. I found this in particular which looks like a good form factor: http://www.camera-module.com/product/autofocuscameramodule/small-auto-focus-8mp-usb-camera-module-sony-imx179.html
Unfortunately I can't find any resellers and direct purchase has a 2 unit minimum + freight.
Did you have any other cameras in mind?

I've found it on AliExpress https://www.aliexpress.com/item/1005001484224521.html


add career page with recent job post.for each job card like design

create career page

create career page


Alpha dependency on Perl IPC::Cmd

Testing 3.0.0alpha13, although the same issue happened with alpha12.
The build fails with Can't locate IPC/Cmd.pm in @INC.  NOTES.PERL says that Perl 5.10.0 is required (I have 5.10.1), and the only additional required module is Text::Template, which is also included in external/perl.
Is IPC.Cmd a requirement for building now?  If so, should it be documented and included in external/perl?

According to corelist, it should be available in a regular perl distribution:
$ corelist IPC::Cmd

Data for 2021-01-23
IPC::Cmd was first released with perl v5.9.5

Note that there's a snag, which is mentioned in NOTES-PERL.md as well:

on Linux distributions based on Debian, the package perl will
install the core Perl modules as well, so you will be fine.
on Linux distributions based on RPMs, you will need to install
perl-core rather than just perl.


update .gitignore and .gitattributes

Add .gitattributes file to remove test files when we are using it as a composer package
And Update .gitignore file not to listen on .idea directory

Thanks for the PR and sorry for the super late response! I'll release a new version in a few minutes with this included.


Prevent error when statuspage env vars are not set

Remove need for statuspage env vars
Fixes an issue where if you try to run the bot without statuspage variables set and the bot is not in a docker container, it will throw an error
Changes

simply return if either of the statuspage env vars are not set.

its not docker specific. but rather if I'm running the bot on my laptop outside of a container but also don't have the monitor values set, it shouldn't throw an error. smth wrong with it?

I'm thinking this should be broken out of the docker utils file.


ML-686: v3io_driver support for parallel array operations

Signed-off-by: Eyal Salomon<EMAIL_ADDRESS>

If the point of the change is to improve performance, we should probably benchmark it to make sure that it does so at least in some common cases. There is a lot of overhead that could potentially slow things down, so we should probably check this.


PP-4183 Multiple threads - CardCaptureProcess

WHAT
A brief description of the pull request:
HOW
Steps to test or reproduce:

I'd like some more explanation in the commit message about the rationale for the choice of queue type, and the logic around handling concurrent readers/writers and ensuring correct semantics.

@heathd @kbottla sorry to butt in here, but i think any solution here should warrant further discussion. i know i was quite up for using a real queue (e.g. SQS). though i'm not up for that anymore i recall @heathd saying we should consider using a real queue. and other people might possibly have more ideas for solutions.
granted, i haven't gone through this PR in detail but i'm not really sure about using the java.util.Queue.... if the connector app dies everything in the queue is gone.
i actually have a firebreak idea in mind to get rid of CardCaptureProcess. very briefly, the idea would be to perform the capture asynchronously as soon as the user clicks 'Confirm' on the payment. this is easily achieved using
CompletableFuture
.supplyAsync(() -> getPaymentProvider().capture(...))
.handle(
  //handle errors here
);

Using the CompletableFuture avoids the need for instantiating any sort of ExecutorService. very nice in my opinion.
in the event where the payment gateway is down, we'd have a cloudwatch (or some other thing) event to hit a connector endpoint which will retry the failed charges. this way only one instance of connector processes these and we have no conflict issues.
though i still think using a real queue would be great, if we can avoid adding infrastructure, we should do that.

@oswaldquek I haven't had time to fully digest your comments, but just to say the main reason for the background process is to ensure that capture is retried. More background here https://payments-platform.atlassian.net/browse/PP-1633

@oswaldquek Thanks for the comment.
I agree a real queue would serve the purpose better. The current solution is enhancement to what is currently in place and is quick win for short term solution.

granted, i haven't gone through this PR in detail but i'm not really sure about using the java.util.Queue.... if the connector app dies everything in the queue is gone.

Queue is used only to hold charges to capture. It doesn't hold real state. When connector dies we will end up with two scenarios

Orderly shutdown : Scheduler thread is hooked to dropwizard lifecycle. So it will wait for all threads to complete before shutdown.
Forced shutdown/killed : As the state is maintained in database another connector can pickup charges not processed

get rid of CardCaptureProcess.

As @heathd mentioned we need to retry captures (currently every 1hour for 48 hours). As per my current understanding this happens frequently for EPDQ. Background asnyc process sounds good though and combining with CardCaptureProcess to retry captures could lead to smaller queue size

@heathd yes, i've accounted for retrying the capture:
in the event where the payment gateway is down, we'd have a cloudwatch (or some other thing) event to hit a connector endpoint which will retry the failed charges. this way only one instance of connector processes these and we have no conflict issues.```


[FIX] add origins

Codecov Report

Merging #34 (899875b) into main (6d1fcf3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #34   +/-   ##
=======================================
  Coverage   70.04%   70.04%
=======================================
  Files          29       29
  Lines         601      601
  Branches       70       70
=======================================
  Hits          421      421
  Misses        180      180

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6d1fcf3...899875b. Read the comment docs.


Change to eventsPages

https://github.com/michael-small/Morris-MN-Public-Library/blob/8ca54f35e089a18db247c08ac65e234640b74966/src/components/Layout.js#L66
This is legit because there are posts, but it should actually be eventsPages.

Also change to events for
? data.eventsPages.edges.map(post => {
    return { ...post.node.fields, ...post.node.frontmatter }
})

Also, going to tack on this: https://github.com/michael-small/Morris-MN-Public-Library/commit/2997193f4730195b64ab7e05952100e8802fb889#r39555107


Add Solid and 3D Compound bug

The following:
p = Sphere(1) + Compound(Sphere(1))

gives this contradictory error:
ValueError: Only shapes with the same dimension can be added

This is a contrived example, however, such a scenario can often arise when using Algebra mode.

A Compound has no dimension as it can contain 1, 2 or 3 dimensional objects.  To "fix" this the Compound wrapper would have to be stripped off effectively reversing what the user did. Do you have a more realistic example?

A Compound is a generic container so its dimension is equal to the dimension of its contained shapes.
Compound<3>() + Part() = Compound<3>()
Compound<2>() + Sketch() = Compound<2>()
Compound<?>() + Sketch() = Compound<2>()
Compound<3>() + Sketch() = Error
Compound<?>() + Compound<?>() = Compound<?>()

A real example:
p = Sphere(1) + Sphere(1).fuse(Sphere(1))

Everything here is obviously 3D, yet i get ValueError: Only shapes with the same dimension can be added

A Compound is a generic container so its dimension is equal to the dimension of its contained shapes.

Not always, Compounds are more generic:
c = Compound(
    [
        Circle(1),
        Line((0, 0, 0), (0, 0, 5)).edge(),
        Pos(0, 0, 6) * Cone(1, 0, 2),
    ]
)

is a valid Compound. In ocp_vscode I have to treat them differently, i.e. unroll these "mixed" Compounds.
To achieve what you want, currently you have to wrap the fuse result into a Part:
a = Sphere(1) + Part(Box(3, 0.2, 0.2).fuse(Box(0.2, 0.2, 3)))

So issue to me is less that Part + Compound should work (it can't for mixed Compounds).
I am not sure fuse should automatically convert to a Part (it is on the direct API level), but maybe a best practice is to:
Either not mixin direct API calls and use proper algebra operators, i.e.  Sphere(1) + Box(3, 0.2, 0.2) + Box(0.2, 0.2, 3), or to wrap the result into a Part if using direct API is necessary.

How would I write a dimension agnostic operation then? For example, the builtin mirror function only returns the mirrored object but I want a version that returns the symmetric object:
def mirror_keep(obj: Union[Edge, Wire, Face, Compound, Curve, Sketch, Part]):
    return Compound(children=[obj, mirror(obj)])

This does not work due to the above issue. I could instead do
return obj + mirror(obj)

but that is an unneeded and much more expensive operation (in some cases this difference can be very significant)

You can do:

Part([b, mirror(b)])
Sketch([s, mirror(s)])
Curve([w, mirror(w)])
...
with b = Pos(0,1,0) * Box(1,1,1) and s = b.shell() and w = b.wire()

Not sure you need a function or an assembly. Compound (and Part, Sketch and Curve are Compounds) should do what you want.
If you want a function, check the type of the input object and depending on the type use the above

The dimension check is only done with the + operator not the fuse method so if you'd like to avoid the check just use the fuse method directly - assuming the specific situation is going to result in something that makes sense.

With commit 0b4b2b2b54422561e9c7b8b0d2e97cde7a16bbea the dimension (a property not a class variable) for a Compound is calculated from the dimension of its components. If the components are of mixed _dim then None is returned.
p = Sphere(1) + Compound(PolarLocations(0.5, 4) * Sphere(1))
print(p._dim) # 3


Topic/hpp joint configuration

Added methods in joint models to integrate, interpolate, differentiate , compute the distance between two joint configurations. Added corresponding algorithm to perform theses actions on a Model and Data and its python bindings & unittest.

@jcarpent Do not know what is happening with Travis.
See end of log at https://travis-ci.org/fvalenza/pinocchio/builds/117493884. Build stalls after all unittest passed. At the same time, on the stack-of-tasks travis repo, build stalls in the middle of unit testings..

Coverage increased (+1.2%) to 73.76% when pulling 242db36655d50876a7351c91dc72b14d71939437 on fvalenza:topic/hpp-joint-configuration into d6068a97444b6c3bdd186de8b48fe1fd6f2918e3 on stack-of-tasks:devel.

Coverage increased (+1.2%) to 73.806% when pulling 33b4e79aff768c3f78547aa65275d5d6b5e4e13a on fvalenza:topic/hpp-joint-configuration into d6068a97444b6c3bdd186de8b48fe1fd6f2918e3 on stack-of-tasks:devel.

Closing. See #141


[TLS: local_cert_path] unable to load PEM certs with .crt extensions

Note: If you have a feature request, you should contact support so the request can be properly tracked.
Output of the info page
Getting the status from the agent.

===============
Agent (v6.14.0)
===============

  Status date: 2020-01-31 17:19:17.071722 UTC
  Agent start: 2020-01-31 00:15:35.947147 UTC
  Pid: 339
  Go Version: go1.12.9
  Python Version: 2.7.16
  Check Runners: 4
  Log Level: WARNING

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: 1.29ms
    System UTC time: 2020-01-31 17:19:17.071722 UTC

  Host Info
  =========
    bootTime: 2020-01-24 17:51:55.000000 UTC
    kernelVersion: 3.10.0-1062.9.1.el7.x86_64
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 10.1
    procs: 69
    uptime: 150h23m45s
    virtualizationRole: host
    virtualizationSystem: kvm

  Hostnames
  =========
    host_aliases: [controller-1-opus-v1.staging.example.com]
    hostname: controller-1-opus-v1.staging.example.com
    socket-fqdn: datadog-2n5w6
    socket-hostname: datadog-2n5w6
    host tags:
      env:staging
      role:kubernetes
      cluster:opus-staging
    hostname provider: container
    unused hostname providers:
      aws: not retrieving hostname from AWS: the host is not an ECS instance, and other providers already retrieve non-default hostnames
      configuration/environment: hostname is empty
      gce: unable to retrieve hostname from GCE: Get http://<IP_ADDRESS>/computeMetadata/v1/instance/hostname: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

=========
Collector
=========

  Running Checks
  ==============

    coredns (1.2.0)
    ---------------
      Instance ID: coredns:e9e54b44455e0e34 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/coredns.d/auto_conf.yaml
      Total Runs: 4,095
      Metric Samples: Last Run: 137, Total: 561,015
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,095
      Average Execution Time : 30ms

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 4,095
      Metric Samples: Last Run: 6, Total: 24,564
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    disk (2.5.0)
    ------------
      Instance ID: disk:e5dffb8bef24336f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml.default
      Total Runs: 4,094
      Metric Samples: Last Run: 314, Total: 1 M
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 91ms

    docker
    ------
      Instance ID: docker [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/docker.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 241, Total: 986,654
      Events: Last Run: 0, Total: 1
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 24ms

    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 4,095
      Metric Samples: Last Run: 5, Total: 20,475
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 4,094
      Metric Samples: Last Run: 65, Total: 266,065
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    kubelet (3.3.2)
    ---------------
      Instance ID: kubelet:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/kubelet.d/conf.yaml.default
      Total Runs: 4,095
      Metric Samples: Last Run: 78, Total: 319,624
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 4, Total: 16,380
      Average Execution Time : 154ms

    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 4,094
      Metric Samples: Last Run: 6, Total: 24,564
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 4,094
      Metric Samples: Last Run: 17, Total: 69,598
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

    network (1.11.4)
    ----------------
      Instance ID: network:e0204ad63d43c949 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 4,095
      Metric Samples: Last Run: 61, Total: 249,795
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 2ms

    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 4,094
      Metric Samples: Last Run: 1, Total: 4,094
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 15ms

    tls (1.2.0)
    -----------
      Instance ID: tls:542c9cac6923a09f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 0s

      Instance ID: tls:67443a6e0603dad1 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,095
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,095
      Average Execution Time : 0s

      Instance ID: tls:6bdaabf90ec05a5a [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 0s

      Instance ID: tls:7a31a6a46b4dedde [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 0s

      Instance ID: tls:8a5b466f7b05d1b7 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,095
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,095
      Average Execution Time : 0s

      Instance ID: tls:9bffe636f25a9259 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 0s

      Instance ID: tls:e43d8794821c495e [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,095
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,095
      Average Execution Time : 0s

      Instance ID: tls:e4547a4365c5c93b [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,095
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,095
      Average Execution Time : 0s

      Instance ID: tls:efe8c9aca44a1a8e [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 4,094
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 4,094
      Average Execution Time : 0s

    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 4,095
      Metric Samples: Last Run: 1, Total: 4,095
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s

========
JMXFetch
========

  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    CheckRunsV1: 4,094
    Dropped: 0
    DroppedOnInput: 0
    Events: 0
    HostMetadata: 0
    IntakeV1: 342
    Metadata: 0
    Requeued: 0
    Retried: 0
    RetryQueueSize: 0
    Series: 0
    ServiceChecks: 0
    SketchSeries: 0
    Success: 8,530
    TimeseriesV1: 4,094

  API Keys status
  ===============
    API key ending with 70ee6: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.com - API Key ending with:
      - 70ee6

==========
Logs Agent
==========
    LogsProcessed: 7160
    LogsSent: 7160

  directory
  ---------
    Type: file
    Path: /hostvarlog/kubelet.log
    Status: OK
    Inputs: /hostvarlog/kubelet.log

=========
Aggregator
=========
  Checks Metric Sample: 3.9 M
  Dogstatsd Metric Sample: 298,877
  Event: 2
  Events Flushed: 2
  Number Of Flushes: 4,094
  Series Flushed: 2.8 M
  Service Check: 147,471
  Service Checks Flushed: 151,540

=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 298,876
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 19 M
  Udp Packet Reading Errors: 0
  Udp Packets: 298,877
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0

=====================
Datadog Cluster Agent
=====================

  - Datadog Cluster Agent endpoint detected: https://<IP_ADDRESS>:5005
  Successfully connected to the Datadog Cluster Agent.
  - Running: 1.3.2+commit.e3f5101

Additional environment details (Operating System, Cloud provider, etc):
Datadog cluster agent in kubernetes on bare metal (centos 7)
Steps to reproduce the issue:

Mount /etc/kubernetes/pki in the agent DaemonSet
Configure the tls integration to check a .crt file

          - local_cert_path: "/kube_pki/apiserver.crt"
            name: apiserver
            validate_hostname: false

Run the check

Describe the results you received:
[
  {
    "check": "tls.cert_validation",
    "host_name": "controller-1-opus-v1.staging.example.com",
    "timestamp":<PHONE_NUMBER>,
    "status": 2,
    "message": "Unable to parse the certificate: Unable to load certificate",
    "tags": []
  }
]

Describe the results you expected:
I would expect it to be able to load the cert. Permissions are correct and the general status of the tls integration is OK:
    tls (1.2.0)
    -----------
      Instance ID: tls:542c9cac6923a09f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tls.yaml
      Total Runs: 1
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 1
      Average Execution Time : 0s

Additional information you deem important (e.g. issue happens only occasionally):
I'm pretty sure this has to do with this line in tls.py. The function is trying to use load_der_x509_certificate to load a PEM formatted certificate because of its filename extension.
head -1 /kube_pki/apiserver.crt
-----BEGIN CERTIFICATE-----

Can someone at least confirm or deny that this is a bug? I've been trying to work with datadog support on this, but they're struggling to grasp what I'm claiming here.

@jslusher Hello! I can confirm as I wrote that. Would you recommend a new option or is .crt pretty much always PEM and that should be changed?

@ofek Thanks for the reply! In my experience .crt files are usually in PEM format, but ideally it shouldn't rely on the filename extension. Playing around with the binaryornot python module and it looks like you can might be able to use it to determine at least if the cert is binary and then assume that means it's DER format:
In [2]: from binaryornot.check import is_binary

In [4]: is_binary('apiserver.der')
Out[4]: True

In [5]: is_binary('apiserver.crt')
Out[5]: False

https://binaryornot.readthedocs.io/en/latest/index.html


Feature Request: Add continue button on long chat responses to load more.

Add a "Continue" button at the bottom of a generated response that may have been cut due to the length.
this allows to easily ask the chat for more text.

Is it possible to detect if a response was cut due network error or char limit? Then it might make sense if the app automatically prompted a "continue" to the API. But a button would be good either way, so that you can push the AI to continue writing stories, etc.

For what is worth, I realised that if you just type Continue it will understand and continue writing

Thats correct, but that might lead to weird responses from OpenAI on network errors as far I could see. But yeah, that's what you can do. Typing "continue" also works for continues writing of story chapters instead of "write the next chapter" etc.

Continue seems to be the best way to do this for now.


ci: add slither static analysis checks

Motivation
Describe why this issue should be fixed and link to any relevant design docs, issues or other relevant items.
Change Summary
Describe the changes being made in 1-2 concise sentences.
Merge Checklist
Choose all relevant options below by adding an x now or at any time before submitting for review

[ ] The PR title adheres to the conventional commits standard
[ ] The PR has been tagged with change type label(s) (i.e. documentation, feature, bugfix, or chore)
[ ] The PR's changes adhere to all the requirements in the contribution guidelines
[ ] This PR does not require changes to the Farcaster protocol

Additional Context
If this is a relatively large or complex change, provide more details here that will help reviewers.

closing for now, don't think we want slither in ci just yet


Question: How does options work?

I see that @options get passed for the command, but how do I get anything in there?
For example if I wanted to have following commands:
# using config-command to set value
app config foo=bar

# using config-command to read value
app config foo

# giving a flag for the command
app calculate_something --refresh-cache

Perhaps this is something that is documented somewhere, but I've missed that.

I sent out a pull request to explain this better see https://github.com/piotrmurach/tty/pull/40

@hasanen Apologies for not replying to you earlier. I have been sidetracked into fixing/improving few tty components. In future please ping me if I don't reply. I will write up a section addressing different types of flags and how to code them.
@charignon Thank you for submitting PR!  Unfortunately I won't merge it and here is why. I want to provide a through documentation on how to work with flags, options and arguments so that both beginners and advanced users know exactly how things are done. Referencing a file doesn't necessarily explain all available features. I'm happy for us to collaborate on the docs. If you have any ideas please submit them.

@piotrmurach no problem, I've been quite busy myself too, but I'll keep it mind. Thanks!


v0.30.1

See CHANGELOG

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license.

Found another issue in jsii, so I am closing this for now


Comma 2 crashes and becomes unresponsive with: echo -en "1" > /data/params/d/DisableUpdates

Describe the bug
I can confirm after much troubleshooting that this causes the Comma 2 to crash and become unresponsive on vehicle power down
echo -en "1" > /data/params/d/DisableUpdates
I was told that is the way to prevent auto updates, but it does not work.
Comma 2 will shutdown and become unresponsive after the car ignition is turned off, resetting the comma 2 will allow it to sleep and wake upon next using the car, but it will crash again at vehicle power off and won't come back on unless it is reset.
How to reproduce or log data
SSH into the Comma 2 and enter the command: echo -en "1" > /data/params/d/DisableUpdates
Expected behavior
Comma 2 will wake up the next time the car is used and will not crash every time the car is turned off.
Device/Version information (please complete the following information):

Device: Comma 2
Dongle ID: 783f2ad3c4e2874b
Version: Stock OP version 0.7.9
Car make/model: Honda CR-V 2019

Additional context
If anyone knows how to actually keep the Comma 2 from auto updating please tell me! I hate having to use Dragonpilot
Thank you

I can't reproduce this on 0.7.9, and there was a big params refactor done in 0.7.10. Feel free to reopen the issue if you have steps to reliably reproduce this or this isn't resolved in 0.7.10, which will be in release shortly.


Not working with IE 11

The problem seems to be that IE 11 doesn't support Uint8ClampedArray, which is needed in GlobalHistogramBinarizer.ts (and others).

Well actually that's a problem with the base-repo not your fork, but I thought we're gonna develop here. I'll create my own fork and make a PR to the base-repo instead

We can develop here, because this fork is too different to take updates (merges) from the base-repo. So we will need to manually implement any updates from there to here.


Focus on not hard-coding values.

The problem you're addressing (if any)
No response
Describe the solution you'd like
https://github.com/Dasharo/open-source-firmware-validation/pull/45/files#diff-2bd69584d51e03632b6b478cae660b9d98acb614bedfcf8ad005f876c9d6744eL498
Values like:
 Telnet.Read Until exit.
should not be hardcoded. If necessary, [Arguments] could be used to give a default value, so that such a keyword can also be used with arguments other than the default value.
Where is the value to a user, and who might that user be?
No response
Describe alternatives you've considered
No response
Additional context
No response

This is so open-ended, that we cannot define exit criteria here.


Taking sampling rate into account

I just realized that your functions did not take the sampling rate into account, and use a default of 1000Hz. I think it would be easy to fix to add a sampling_rate=1000 parameter to adjust the meaning of the RR intervals. Cheers!

Wait no my bad I didn't see that you adjust the RRis beforehand! Closing issue


Improper require of Hashie causing error (NameError: uninitialized constant Hashie::Extensions::Array)

See: https://github.com/intridea/hashie/issues/367
Hashie is being required (require "hashie/mash") here, but according to the above issue, it should also require "hashie" prior to that. The version of hashie is not restricted, so a bundle or gem update will cause this issue when using chef-zero (in my case, specifically, using Chefspec).
I suspect Hashie 3.4.5 will be fixed or reverted, but compatibility with newer versions will likely depend on fixing this.

I released hashie 3.4.6 so this shouldn't be necessary, but for making things future-proof I recommend changing this to a require 'hashie'.

tactically we should fix this.
strategically we should probably replace hashie with Chef::Mash to reduce external deps if possible.

I think this has cropped up again with Hashie 3.5.0:
https://github.com/intridea/hashie/issues/391

Yes, sigh. I'm working on a way to have a spec for this.

Resolved again by Hashie 3.5.1
Tested to verify.


Why OpenSearchStreamingLambdaFunction does not have app name prefix and current env suffix?

Which Category is your question related to?
API
Amplify CLI Version
7.6.7
What AWS Services are you utilizing?
Amplify
Provide additional details e.g. code snippets
Pushing minor updates in schema caused changing search lambda name to OpenSearchStreamingLambdaFunction - without app name or environment. Is this expected behaviour?
Trying to push to production now causes error:

OpenSearchStreamingLambdaFunction already exists in stack

I've got the same issue.
As of 7.6.7, it changes the previous OpenSearchStreamingLambda function from amplify-<appName>-<env>-OpenSearchStreamingLambd-<someId> to OpenSearchStreamingLambdaFunction, without any other identifier.
I updated my dev env, but I can no longer push to prod, due to the error:
Following resources failed
Resource Name: amplify---OpenSearchStreamingLambd- (AWS::Lambda::Function)
Event Type: update
Reason: OpenSearchStreamingLambdaFunction already exists in stack

If possible, can someone from Amplify team recommend a workaround, e.g. do we downgrade the CLI version?
Customers unable to deploy to production environments should be treated as a high-severity issue

@mattfysh downgrading to a version prior to v7.6.7 should let you workaround the issue.

If possible, can someone from Amplify team recommend a workaround, e.g. do we downgrade the CLI version?
Customers unable to deploy to production environments should be treated as a high-severity issue

@mattfysh I can confirm.  I downgraded to 7.6.5 (including in the Amplify console if you're using Amplify's git-based hosting) and it reverts back to the previous lambda name.  Test first in a non-production env to ensure that the OpenSearchStreamingLambdaFunction reverts back to the previous name.

Yea this is def a problem, my dev env had the prefixed es lambda, but my main is now OpenSearchStreamingLambdaFunction and I can't push any changes

When I push I get the following errors:
UPDATE_FAILED               OpenSearchStreamingLambdaFunction6ED4A1FD                               AWS::Lambda::Function      Wed Jan 12 2022 15:08:34 GMT-0800 (Pacific Standard Time) OpenSearchStreamingLambdaFunction already exists in stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM-SearchableStack-16OIOH0271T9U/63e749e0-263f-11ec-baa0-0a7614ed120d
UPDATE_FAILED               OpenSearchAccessIAMRole6A1D9CC5                                         AWS::IAM::Role             Wed Jan 12 2022 15:08:35 GMT-0800 (Pacific Standard Time) Resource creation cancelled
UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-dev-160955-apifox-Y0QXBP993EX1-SearchableStack-TV78E7975PIC AWS::CloudFormation::Stack Wed Jan 12 2022 15:08:36 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [OpenSearchAccessIAMRole6A1D9CC5, OpenSearchStreamingLambdaFunction6ED4A1FD].
⠸ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED SearchableStack AWS::CloudFormation::Stack Wed Jan 12 2022 15:08:40 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-dev-160955-apifox-Y0QXBP993EX1-SearchableStack-TV78E7975PIC/ec47cd50-263a-11ec-bcd6-0a9127743e0d was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [OpenSearchAccessIAMRole6A1D9CC5, OpenSearchStreamingLambdaFunction6ED4A1FD].
⠏ Updating resources in the cloud. This may take a few minutes...

UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-dev-160955-apifox-Y0QXBP993EX1 AWS::CloudFormation::Stack Wed Jan 12 2022 15:08:42 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [SearchableStack].
⠙ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED               apifox                 AWS::CloudFormation::Stack Wed Jan 12 2022 15:08:46 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-dev-160955-apifox-Y0QXBP993EX1/bbfb9af0-263a-11ec-9178-0682b490ec17 was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [SearchableStack].

@Rafcin there is a proposed fix in https://github.com/aws-amplify/amplify-cli/pull/9491. Until that is released, please downgrade to a version prior to v7.6.7.

I don't wanna be that guy, but 7.6.9 gave me so funky IAM/OS errors :(
UPDATE_FAILED               OpenSearchStreamingLambdaFunction6ED4A1FD                               AWS::Lambda::Function      Thu Jan 13 2022 16:56:13 GMT-0800 (Pacific Standard Time) OpenSearchStreamingLambdaFunction already exists in stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM-SearchableStack-16OIOH0271T9U/63e749e0-263f-11ec-baa0-0a7614ed120d
UPDATE_FAILED               OpenSearchAccessIAMRole6A1D9CC5                                         AWS::IAM::Role             Thu Jan 13 2022 16:56:13 GMT-0800 (Pacific Standard Time) Resource creation cancelled
UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-dev-160955-apifox-Y0QXBP993EX1-SearchableStack-TV78E7975PIC AWS::CloudFormation::Stack Thu Jan 13 2022 16:56:14 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [OpenSearchAccessIAMRole6A1D9CC5, OpenSearchStreamingLambdaFunction6ED4A1FD].
⠼ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED               SearchableStack                            AWS::CloudFormation::Stack Thu Jan 13 2022 16:56:19 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-dev-160955-apifox-Y0QXBP993EX1-SearchableStack-TV78E7975PIC/ec47cd50-263a-11ec-bcd6-0a9127743e0d was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [OpenSearchAccessIAMRole6A1D9CC5, OpenSearchStreamingLambdaFunction6ED4A1FD].
UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-dev-160955-apifox-Y0QXBP993EX1 AWS::CloudFormation::Stack Thu Jan 13 2022 16:56:20 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [SearchableStack].
⠙ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED               apifox                 AWS::CloudFormation::Stack Thu Jan 13 2022 16:56:34 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-dev-160955-apifox-Y0QXBP993EX1/bbfb9af0-263a-11ec-9178-0682b490ec17 was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [SearchableStack].

@Rafcin I don't think the fix has been released yet, 7.6.9 was released 21 hours ago, but the fix was only merged 9 hours ago

@ngnathan How did you get this to work when downgrading to 7.6.5? I tried and it's not reverting the lambda name to the old way.
@lazpavel Can you reopen this until the CLI receives the update? Just to make sure this issue isn't forgotten. It impacts all users using OS.

@Rafcin I downgraded to 7.6.5, and then ran amplify push --force to ensure it runs the update on the api.  Then if you're using the Amplify Github Hosted flow, in the Amplify console >> Build settings, you'll need to make sure you set the Amplify CLI version to 7.6.5 as well, so that when it runs its own push, it doesn't replace the current lambda again.

Thank you! I'll test this out now 😄

@ngnathan I tried that, and it gives me more errors :(
UPDATE_FAILED      SearchableRestaurantLambdaMappingB7879B15 AWS::Lambda::EventSourceMapping Fri Jan 14 2022 23:39:30 GMT-0800 (Pacific Standard Time) Resource handler returned message: "User: arn:aws:sts::707129987630:assumed-role/us-west-2_Dpvj9kGVp_Full-access/amplifyadmin is not authorized to perform: lambda:UpdateEventSourceMapping on resource: arn:aws:lambda:us-west-2:707129987630:event-source-mapping:a03daf78-583f-404a-bb80-e5ed1d2730b0 because no identity-based policy allows the lambda:UpdateEventSourceMapping action (Service: Lambda, Status Code: 403, Request ID: a7c21066-c8cb-4557-b8a5-46817cfb8dbe, Extended Request ID: null)" (RequestToken: cb96b09d-d01c-ba16-2496-5886ea2ea189, HandlerErrorCode: GeneralServiceException)
UPDATE_FAILED      SearchableUserLambdaMapping231B0978       AWS::Lambda::EventSourceMapping Fri Jan 14 2022 23:39:31 GMT-0800 (Pacific Standard Time) Resource handler returned message: "User: arn:aws:sts::707129987630:assumed-role/us-west-2_Dpvj9kGVp_Full-access/amplifyadmin is not authorized to perform: lambda:UpdateEventSourceMapping on resource: arn:aws:lambda:us-west-2:707129987630:event-source-mapping:ad6b2f45-06b0-4107-949b-cfb526363536 because no identity-based policy allows the lambda:UpdateEventSourceMapping action (Service: Lambda, Status Code: 403, Request ID: aa29c9d7-5528-4c8c-8038-5d30a4d0ebad, Extended Request ID: null)" (RequestToken: f553564a-5261-4594-1860-4a2652af1927, HandlerErrorCode: GeneralServiceException)
⠹ Updating resources in the cloud. This may take a few minutes...

UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM-SearchableStack-16OIOH0271T9U AWS::CloudFormation::Stack Fri Jan 14 2022 23:39:32 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [SearchableUserLambdaMapping231B0978, SearchableRestaurantLambdaMappingB7879B15].
⠏ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED               SearchableStack                              AWS::CloudFormation::Stack Fri Jan 14 2022 23:39:45 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM-SearchableStack-16OIOH0271T9U/63e749e0-263f-11ec-baa0-0a7614ed120d was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [SearchableUserLambdaMapping231B0978, SearchableRestaurantLambdaMappingB7879B15].
UPDATE_ROLLBACK_IN_PROGRESS amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM AWS::CloudFormation::Stack Fri Jan 14 2022 23:39:46 GMT-0800 (Pacific Standard Time) The following resource(s) failed to update: [SearchableStack].
⠴ Updating resources in the cloud. This may take a few minutes...

UPDATE_FAILED apifox AWS::CloudFormation::Stack Fri Jan 14 2022 23:40:01 GMT-0800 (Pacific Standard Time) Embedded stack arn:aws:cloudformation:us-west-2:707129987630:stack/amplify-fox-prod-174207-apifox-1XCP8S1CGPDTM/0ff51d80-263f-11ec-bda8-066ebf633925 was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to update: [SearchableStack].

It looks like you need permission to perform lambda:UpdateEventSourceMapping.

@cjihrig It's weird, the role I'm signed in with has full access, unless it just happens to omit that specific permission.

Please deploy the fix ASAP, downgrading to 7.6.5 introduces other issues and is not an acceptable workaround

update amplify Cli to beta version:
sudo npm install -g<EMAIL_ADDRESS>and it works
But it need extra permission for AdministratorAccess-Amplify
"Effect": "Allow"
"Action": "lambda:UpdateEventSourceMapping"
At the end its worked well

Fix released in 7.6.10