Memory leak and very low performance for import

OrientDB Version: 2.2.17
OS: Red Hat Enterprise Linux Server release 5.8 (Tikanga)
I'm using oriendb since 6 month and i have some problems/issues.
Here is my setup :

I have 20M Verticles / 40M Edges in Mysql and Elasticsearch.

I have done a multithread script to import all the data in OrientDB.

My server for OrientDB : 16CPU, 32GB RAM

My server for the script import : 8CPU, 16GB RAM

I'm using python and the driver pyorient

-> I had to separate the vertex and the edge import because it was impossible to do it with the batch import (lots of concurrency problems)
-> the RAM used on the server never stop increasing (I had to change OrientDB server three times to be able to import the whole database)

-> I'm seeing that the import is very slow : more than 2 days to import the entire database.
so :

How can i increase the performance? (i'm already doing bulk and multithread)

How can I fix the increasing RAM (memory leek) because i can't let it like this.Everytime my server start to swap after hours of importing... ?

StackOverflow issue : link
Thanks

Hi @oger-p, that is a quite small number of data.
Could send me configuration which you use on a server side?
I mean following:

Java settings.
The final size of the database on disk.
The server settings. The last one you can retrieve using http://orientdb.com/docs/2.2/Console-Command-Config.html . You should execute this command once you login on a remote server in a console application.

1/ Java settings
    /opt/OrientDB/jre1.8.0_111/bin/java -server -Dcom.sun.management.jmxremote.port=3333 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false     -server     -Xms24g -Xmx24g -Djna.nosys=true -XX:+HeapDumpOnOutOfMemoryError -XX:MaxDirectMemorySize=512g -Djava.awt.headless=true -Dfile.encoding=UTF8 -Drhino.opt.level=9     -Djava.util.logging.config.file=/opt/OrientDB/orientdb-community-2.2.17/config/orientdb-server-log.properties -Dorientdb.config.file=/opt/OrientDB/orientdb-community-2.2.17/config/    orientdb-server-config.xml -Dorientdb.www.path=/opt/OrientDB/orientdb-community-2.2.17/www -Dorientdb.build.number=2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16     17:20:27+0000 -cp /opt/OrientDB/orientdb-community-2.2.17/lib/orientdb-server-2.2.17.jar:/opt/OrientDB/orientdb-community-2.2.17/lib/*:/opt/OrientDB/orientdb-community-2.2.17/plugins/* com.orientechnologies.orient.server.OServerMain

4/ The final size of the database on disk.
    $ du -hs ~/OrientDB/orientdb-community-2.2.17/databases/*
    29G /opt/OrientDB/orientdb-community-2.2.17/databases/Delphes
    4,0K    /opt/OrientDB/orientdb-community-2.2.17/databases/GamesOfThrones
    27M /opt/OrientDB/orientdb-community-2.2.17/databases/GratefulDeadConcerts
    4,0K    /opt/OrientDB/orientdb-community-2.2.17/databases/OpenBeer
    20M /opt/OrientDB/orientdb-community-2.2.17/databases/OSystem
    22M /opt/OrientDB/orientdb-community-2.2.17/databases/test

3/ The server settings.
    $ cat /etc/*release*
    Red Hat Enterprise Linux Server release 5.8 (Tikanga)

    $ uname -a
    Linux foo.local 2.6.18-308.16.1.el5 #1 SMP Tue Sep 18 07:21:07 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

    $ JAVA_HOME=/opt/OrientDB/jre1.8.0_111/ bin/console.sh

    OrientDB console v.2.2.17 (build 2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16 17:20:27+0000) www.orientdb.com
    Type 'help' to display all the supported commands.
    Installing extensions for GREMLIN language v.2.6.0

    orientdb> CONFIG

    LOCAL SERVER CONFIGURATION
    +----+----------------------------------------------+------------------------------------------------------------------------------------------------+
    |#   |NAME                                          |VALUE                                                                                |
    +----+----------------------------------------------+------------------------------------------------------------------------------------------------+
    |0   |environment.dumpCfgAtStartup                  |false                                                                                |
    |1   |environment.concurrent                        |true                                                                                |
    |2   |environment.lockManager.concurrency.level     |128                                                                                |
    |3   |environment.allowJVMShutdown                  |true                                                                                |
    |4   |script.pool.maxSize                           |20                                                                                |
    |5   |memory.useUnsafe                              |true                                                                                |
    |6   |memory.chunk.size                             |2147483647                                                                                |
    |7   |memory.directMemory.safeMode                  |true                                                                                |
    |8   |memory.directMemory.trackMode                 |false                                                                                |
    |9   |memory.directMemory.onlyAlignedMemoryAccess   |true                                                                                |
    |10  |jvm.gc.delayForOptimize                       |600                                                                                |
    |11  |storage.openFiles.limit                       |512                                                                                |
    |12  |storage.componentsLock.cache                  |10000                                                                                |
    |13  |storage.diskCache.pinnedPages                 |20                                                                                |
    |14  |storage.diskCache.bufferSize                  |4096                                                                                |
    |15  |storage.diskCache.writeCachePart              |15                                                                                |
    |16  |storage.diskCache.writeCachePageTTL           |86400                                                                                |
    |17  |storage.diskCache.writeCachePageFlushInterval |25                                                                                |
    |18  |storage.diskCache.writeCacheFlushInactivity...|60000                                                                                |
    |19  |storage.diskCache.writeCacheFlushLockTimeout  |-1                                                                                |
    |20  |storage.diskCache.diskFreeSpaceCheckInterval  |5                                                                                |
    |21  |storage.diskCache.diskFreeSpaceCheckInterva...|2048                                                                                |
    |22  |storage.diskCache.keepState                   |true                                                                                |
    |23  |storage.configuration.syncOnUpdate            |true                                                                                |
    |24  |storage.compressionMethod                     |nothing                                                                                |
    |25  |storage.encryptionMethod                      |nothing                                                                                |
    |26  |storage.encryptionKey                         |                                                                                |
    |27  |storage.makeFullCheckpointAfterCreate         |false                                                                                |
    |28  |storage.makeFullCheckpointAfterOpen           |true                                                                                |
    |29  |storage.makeFullCheckpointAfterClusterCreate  |true                                                                                |
    |30  |storage.trackChangedRecordsInWAL              |false                                                                                |
    |31  |storage.useWAL                                |true                                                                                |
    |32  |storage.wal.syncOnPageFlush                   |true                                                                                |
    |33  |storage.wal.cacheSize                         |3000                                                                                |
    |34  |storage.wal.fileAutoCloseInterval             |10                                                                                |
    |35  |storage.wal.maxSegmentSize                    |128                                                                                |
    |36  |storage.wal.maxSize                           |4096                                                                                |
    |37  |storage.wal.commitTimeout                     |1000                                                                                |
    |38  |storage.wal.shutdownTimeout                   |10000                                                                                |
    |39  |storage.wal.fuzzyCheckpointInterval           |300                                                                                |
    |40  |storage.wal.reportAfterOperationsDuringRestore|10000                                                                                |
    |41  |storage.wal.restore.batchSize                 |1000                                                                                |
    |42  |storage.wal.readCacheSize                     |1000                                                                                |
    |43  |storage.wal.fuzzyCheckpointShutdownWait       |600                                                                                |
    |44  |storage.wal.fullCheckpointShutdownTimeout     |600                                                                                |
    |45  |storage.wal.path                              |                                                                                |
    |46  |storage.diskCache.pageSize                    |64                                                                                |
    |47  |storage.diskCache.diskFreeSpaceLimit          |256                                                                                |
    |48  |storage.lowestFreeListBound                   |16                                                                                |
    |49  |storage.lockTimeout                           |0                                                                                |
    |50  |storage.record.lockTimeout                    |2000                                                                                |
    |51  |storage.useTombstones                         |false                                                                                |
    |52  |record.downsizing.enabled                     |true                                                                                |
    |53  |object.saveOnlyDirty                          |false                                                                                |
    |54  |db.pool.min                                   |1                                                                                |
    |55  |db.pool.max                                   |100                                                                                |
    |56  |db.pool.idleTimeout                           |0                                                                                |
    |57  |db.pool.idleCheckDelay                        |0                                                                                |
    |58  |db.mvcc.throwfast                             |false                                                                                |
    |59  |db.validation                                 |true                                                                                |
    |60  |nonTX.recordUpdate.synch                      |false                                                                                |
    |61  |nonTX.clusters.sync.immediately               |manindex                                                                                |
    |62  |tx.trackAtomicOperations                      |false                                                                                |
    |63  |tx.pageCacheSize                              |12                                                                                |
    |64  |index.embeddedToSbtreeBonsaiThreshold         |40                                                                                |
    |65  |index.sbtreeBonsaiToEmbeddedThreshold         |-1                                                                                |
    |66  |hashTable.slitBucketsBuffer.length            |1500                                                                                |
    |67  |index.auto.synchronousAutoRebuild             |true                                                                                |
    |68  |index.auto.lazyUpdates                        |10000                                                                                |
    |69  |index.flushAfterCreate                        |true                                                                                |
    |70  |index.manual.lazyUpdates                      |1                                                                                |
    |71  |index.durableInNonTxMode                      |true                                                                                |
    |72  |index.ignoreNullValuesDefault                 |false                                                                                |
    |73  |index.txMode                                  |FULL                                                                                |
    |74  |index.cursor.prefetchSize                     |500000                                                                                |
    |75  |sbtree.maxDepth                               |64                                                                                |
    |76  |sbtree.maxKeySize                             |10240                                                                                |
    |77  |sbtree.maxEmbeddedValueSize                   |40960                                                                                |
    |78  |sbtreebonsai.bucketSize                       |2                                                                                |
    |79  |sbtreebonsai.linkBagCache.size                |100000                                                                                |
    |80  |sbtreebonsai.linkBagCache.evictionSize        |1000                                                                                |
    |81  |sbtreebonsai.freeSpaceReuseTrigger            |0.5                                                                                |
    |82  |ridBag.embeddedDefaultSize                    |4                                                                                |
    |83  |ridBag.embeddedToSbtreeBonsaiThreshold        |40                                                                                |
    |84  |ridBag.sbtreeBonsaiToEmbeddedToThreshold      |-1                                                                                |
    |85  |collections.preferSBTreeSet                   |false                                                                                |
    |86  |file.trackFileClose                           |false                                                                                |
    |87  |file.lock                                     |true                                                                                |
    |88  |file.deleteDelay                              |10                                                                                |
    |89  |file.deleteRetry                              |50                                                                                |
    |90  |security.userPasswordSaltIterations           |65536                                                                                |
    |91  |security.userPasswordSaltCacheSize            |500                                                                                |
    |92  |security.userPasswordDefaultAlgorithm         |PBKDF2WithHmacSHA256                                                                            |
    |93  |network.maxConcurrentSessions                 |1000                                                                                |
    |94  |network.socketBufferSize                      |0                                                                                |
    |95  |network.lockTimeout                           |15000                                                                                |
    |96  |network.socketTimeout                         |15000                                                                                |
    |97  |network.requestTimeout                        |3600000                                                                                |
    |98  |network.retry                                 |5                                                                                |
    |99  |network.retryDelay                            |500                                                                                |
    |100 |network.binary.loadBalancing.enabled          |false                                                                                |
    |101 |network.binary.loadBalancing.timeout          |2000                                                                                |
    |102 |network.binary.maxLength                      |16384                                                                                |
    |103 |network.binary.readResponse.maxTimes          |20                                                                                |
    |104 |network.binary.debug                          |false                                                                                |
    |105 |network.http.installDefaultCommands           |true                                                                                |
    |106 |network.http.serverInfo                       |OrientDB Server v.2.2.17 (build 2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16 1...|
    |107 |network.http.maxLength                        |1000000                                                                                |
    |108 |network.http.streaming                        |false                                                                                |
    |109 |network.http.charset                          |utf-8                                                                                |
    |110 |network.http.jsonResponseError                |true                                                                                |
    |111 |network.http.jsonp                            |false                                                                                |
    |112 |network.http.sessionExpireTimeout             |300                                                                                |
    |113 |network.http.useToken                         |false                                                                                |
    |114 |network.token.secretKey                       |                                                                                |
    |115 |network.token.encryptionAlgorithm             |HmacSHA256                                                                                |
    |116 |network.token.expireTimeout                   |60                                                                                |
    |117 |profiler.enabled                              |false                                                                                |
    |118 |profiler.config                               |                                                                                |
    |119 |profiler.autoDump.interval                    |0                                                                                |
    |120 |profiler.maxValues                            |200                                                                                |
    |121 |profiler.memoryCheckInterval                  |120000                                                                                |
    |122 |sequence.maxRetry                             |100                                                                                |
    |123 |sequence.retryDelay                           |200                                                                                |
    |124 |storageProfiler.intervalBetweenSnapshots      |100                                                                                |
    |125 |storageProfiler.cleanUpInterval               |5000                                                                                |
    |126 |log.console.level                             |info                                                                                |
    |127 |log.file.level                                |info                                                                                |
    |128 |class.minimumClusters                         |0                                                                                |
    |129 |log.console.ansi                              |auto                                                                                |
    |130 |cache.local.impl                              |com.orientechnologies.orient.core.cache.ORecordCacheWeakRefs                                    |
    |131 |command.timeout                               |0                                                                                |
    |132 |command.cache.enabled                         |false                                                                                |
    |133 |command.cache.evictStrategy                   |PER_CLUSTER                                                                                |
    |134 |command.cache.minExecutionTime                |10                                                                                |
    |135 |command.cache.maxResultsetSize                |500                                                                                |
    |136 |query.parallelAuto                            |false                                                                                |
    |137 |query.parallelMinimumRecords                  |300000                                                                                |
    |138 |query.parallelResultQueueSize                 |20000                                                                                |
    |139 |query.scanPrefetchPages                       |20                                                                                |
    |140 |query.scanBatchSize                           |1000                                                                                |
    |141 |query.scanThresholdTip                        |50000                                                                                |
    |142 |query.limitThresholdTip                       |10000                                                                                |
    |143 |query.live.support                            |true                                                                                |
    |144 |statement.cacheSize                           |100                                                                                |
    |145 |sql.graphConsistencyMode                      |tx                                                                                |
    |146 |client.channel.maxPool                        |100                                                                                |
    |147 |client.connectionPool.waitTimeout             |5000                                                                                |
    |148 |client.channel.dbReleaseWaitTimeout           |10000                                                                                |
    |149 |client.ssl.enabled                            |false                                                                                |
    |150 |client.ssl.keyStore                           |                                                                                |
    |151 |client.ssl.keyStorePass                       |                                                                                |
    |152 |client.ssl.trustStore                         |                                                                                |
    |153 |client.ssl.trustStorePass                     |                                                                                |
    |154 |server.openAllDatabasesAtStartup              |false                                                                                |
    |155 |server.channel.cleanDelay                     |5000                                                                                |
    |156 |server.cache.staticFile                       |false                                                                                |
    |157 |server.log.dumpClientExceptionLevel           |FINE                                                                                |
    |158 |server.log.dumpClientExceptionFullStackTrace  |false                                                                                |
    |159 |distributed.crudTaskTimeout                   |10000                                                                                |
    |160 |distributed.maxStartupDelay                   |10000                                                                                |
    |161 |distributed.commandTaskTimeout                |120000                                                                                |
    |162 |distributed.commandQuickTaskTimeout           |5000                                                                                |
    |163 |distributed.commandLongTaskTimeout            |86400000                                                                                |
    |164 |distributed.deployDbTaskTimeout               |1200000                                                                                |
    |165 |distributed.deployChunkTaskTimeout            |15000                                                                                |
    |166 |distributed.deployDbTaskCompression           |7                                                                                |
    |167 |distributed.asynchQueueSize                   |0                                                                                |
    |168 |distributed.asynchResponsesTimeout            |15000                                                                                |
    |169 |distributed.purgeResponsesTimerDelay          |15000                                                                                |
    |170 |distributed.conflictResolverRepairerChain     |majority,content,version                                                                        |
    |171 |distributed.conflictResolverRepairerCheckEvery|5000                                                                                |
    |172 |distributed.conflictResolverRepairerBatch     |100                                                                                |
    |173 |distributed.txAliveTimeout                    |30000                                                                                |
    |174 |distributed.requestChannels                   |1                                                                                |
    |175 |distributed.responseChannels                  |1                                                                                |
    |176 |distributed.heartbeatTimeout                  |10000                                                                                |
    |177 |distributed.checkHealthCanOfflineServer       |false                                                                                |
    |178 |distributed.checkHealthEvery                  |10000                                                                                |
    |179 |distributed.autoRemoveOfflineServers          |0                                                                                |
    |180 |distributed.publishNodeStatusEvery            |5000                                                                                |
    |181 |distributed.localQueueSize                    |10000                                                                                |
    |182 |distributed.dbWorkerThreads                   |8                                                                                |
    |183 |distributed.queueMaxSize                      |10000                                                                                |
    |184 |distributed.backupDirectory                   |../backup/databases                                                                             |
    |185 |distributed.backupTryIncrementalFirst         |true                                                                                |
    |186 |distributed.concurrentTxMaxAutoRetry          |10                                                                                |
    |187 |distributed.atomicLockTimeout                 |300                                                                                |
    |188 |distributed.concurrentTxAutoRetryDelay        |100                                                                                |
    |189 |db.document.serializer                        |ORecordSerializerBinary                                                                         |
    |190 |client.krb5.config                            |                                                                                |
    |191 |client.krb5.ccname                            |                                                                                |
    |192 |client.krb5.ktname                            |                                                                                |
    |193 |client.credentialinterceptor                  |                                                                                |
    |194 |client.ci.keyalgorithm                        |AES                                                                                |
    |195 |client.ci.ciphertransform                     |AES/CBC/PKCS5Padding                                                                            |
    |196 |client.ci.keystore.file                       |                                                                                |
    |197 |client.ci.keystore.password                   |                                                                                |
    |198 |security.createDefaultUsers                   |true                                                                                |
    |199 |server.security.file                          |                                                                                |
    |200 |jna.disable.system.library                    |true                                                                                |
    |201 |distributed.queueTimeout                      |500000                                                                                |
    |202 |db.makeFullCheckpointOnIndexChange            |true                                                                                |
    |203 |db.makeFullCheckpointOnSchemaChange           |true                                                                                |
    |204 |client.session.tokenBased                     |true                                                                                |
    |205 |oauth2.secretkey                              |                                                                                |
    |206 |storage.cluster.usecrc32                      |false                                                                                |
    |207 |lazyset.workOnStream                          |true                                                                                |
    |208 |db.mvcc                                       |true                                                                                |
    |209 |db.use.distributedVersion                     |false                                                                                |
    |210 |mvrbtree.timeout                              |0                                                                                |
    |211 |mvrbtree.nodePageSize                         |256                                                                                |
    |212 |mvrbtree.loadFactor                           |0.7                                                                                |
    |213 |mvrbtree.optimizeThreshold                    |100000                                                                                |
    |214 |mvrbtree.entryPoints                          |64                                                                                |
    |215 |mvrbtree.optimizeEntryPointsFactor            |1.0                                                                                |
    |216 |mvrbtree.entryKeysInMemory                    |false                                                                                |
    |217 |mvrbtree.entryValuesInMemory                  |false                                                                                |
    |218 |mvrbtree.ridBinaryThreshold                   |-1                                                                                |
    |219 |mvrbtree.ridNodePageSize                      |64                                                                                |
    |220 |mvrbtree.ridNodeSaveMemory                    |false                                                                                |
    |221 |tx.commit.synch                               |false                                                                                |
    |222 |tx.autoRetry                                  |1                                                                                |
    |223 |tx.log.fileType                               |classic                                                                                |
    |224 |tx.log.synch                                  |false                                                                                |
    |225 |tx.useLog                                     |true                                                                                |
    |226 |index.auto.rebuildAfterNotSoftClose           |true                                                                                |
    |227 |client.channel.minPool                        |1                                                                                |
    |228 |storage.keepOpen                              |true                                                                                |
    |229 |cache.local.enabled                           |true                                                                                |
    +----+----------------------------------------------+------------------------------------------------------------------------------------------------+

    orientdb>

@oger-p you sent me "local" configuration, but you should see "REMOTE SERVER CONFIGURATION"  in the command output. Could you login into the server http://orientdb.com/docs/2.2/Console-Command-Connect.html and execute the same command after that.

@laa I have the same thing even if i'm connected :
$ orientdb-community-2.2.17$ JAVA_HOME=/opt/OrientDB/jre1.8.0_111/ bin/console.sh
OrientDB console v.2.2.17 (build 2.2.x@rd9bace82ea8437117fd48114fc255e791056014b; 2017-02-16 17:20:27+0000) www.orientdb.com
Type 'help' to display all the supported commands.
Installing extensions for GREMLIN language v.2.6.0

orientdb> CONNECT REMOTE:<IP_ADDRESS>/Delphes root root

Connecting to database [REMOTE:<IP_ADDRESS>/Delphes] with user 'root'...OK
orientdb {db=Delphes}> CONFIG

LOCAL SERVER CONFIGURATION
+----+----------------------------------------------+------------------------------------------------------------------------------------------------+
|#   |NAME                                          |VALUE                                                                                |
+----+----------------------------------------------+------------------------------------------------------------------------------------------------+
|0   |environment.dumpCfgAtStartup                  |false                                                                                |
|1   |environment.concurrent                        |true                                                                                |
|2   |environment.lockManager.concurrency.level     |128                                                                                |
|3   |environment.allowJVMShutdown                  |true                                                                                |
|4   |script.pool.maxSize                           |20                                                                                |
|5   |memory.useUnsafe                              |true                                                                                |
|6   |memory.chunk.size                             |2147483647                                                                                |
|7   |memory.directMemory.safeMode                  |true
....

Hi @oger-p I see. I think first your mistake is that  you set -Xms24g  we do not use Jave heap memory, much instead we use direct memory, but when you assign 24g nothing left for direct memory. So I ask you to lower heap memory to 4g and repeat the import.

Hi,
Ok, I'll do it and i'll update this topic when it's done !
Thanks

Hello, @laa ,
So i did the import again with 1M Vertex / 20M Edge (first part of my dataset)
Here is the result :

22minutes for 1M Vertex
14 hours for 20M Edges

The graph of the memory (you can see it doesn't go down after the end of the import (3/25 00:00) :

@oger-p memory consumption will not go down after insertion because it is kept in disk cache to further next request in a quick manner. Do not think that we have a memory leak here because memory consumption becomes stable after some time. About insertion speed of edges, you may consider usage of lightweight edges   http://orientdb.com/docs/2.2/Lightweight-Edges.html if that is OK for  you.

@laa,
I can't use lightweight edges.
I agree with you for the stable memory but it's just because the import was finished. What happen if I continue to import and arrive at 48/48GB of memory?
I'll try a new full import and keep you updated.
Thanks

@oger-p it depends on your value of disk cache size. Because you use the default value, I suppose memory consumption will grow till about 2 GB will be left free for OS.
Could you meanwhile give me the code which you use for data import or outline your algorithm using pseudocode?

Ok i keep you updated with the next import.
I'm using swallow (https://github.com/pagesjaunes/swallow).
I have separated the importation of Edge and Vertex.
Edge import was not working with batch because of the concurrency.
Vertex:
Configuration

1 processor
15 writters
orientdb batch

Process
One example :

I do a loop on values :
results.append('UPDATE Expression SET libelle = "{0}" UPSERT WHERE libelle = "{0}"'.format(value))
I insert every 2500 command with a batch

Edge
Configuration

5 processor
10 writters
orientdb command

Process
One example :

I do a loop on values :
results.append( 'CREATE EDGE Facultatif FROM (SELECT FROM Regle WHERE id = "{0}") to (SELECT FROM Ontologie WHERE id = "{1}") set type = {2}'.format( regle_id, regle_ontologie_id, regle_ontologie_type))
I insert every iteration

@oger-p do you have any index for Regle.id and Ontologie.id fields ?

@laa, yes :
session_orientdb.command("CREATE INDEX indexOntologie ON Ontologie (id) UNIQUE")
session_orientdb.command("CREATE INDEX indexRegle ON Regle (id) UNIQUE")

Could you add following settings to the server -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc: and send me content of GC log file ?

Did you forget something after -Xloggc?
I send it to you after the full import? partial import?

I suppose that partial import is enough. I have fixed command line settings above.

Ok, i send it to you when it's done.
Thanks

hello @laa,
Here is the result of the import :
Starting time

15h20

Time for importation

20 minutes for all Vertex
13 hours for all Edges

The memory graph

The GCFile

gc.log.gz

Hi @oger-p . I checked GC seems all is OK.
But your throughput is very low, though, we never experienced such small throughput )).
Is it possible for you to create a script which imports from OrientDB to OrientDB and send me this script with a partially imported database which you created and step by step instructions how to run it?
I want to repeat your steps on my side.
To check why throughput numbers are small.
WDYT ?

Hello,
I think the memory problem is fixed.
I let the low performance issue at the moment. My script import all the database successfuly  and it will be a one shot import so it's not the priority to focus on this import.
Thank you for your time !
(I'll keep you updated if needed)

cool @oger-p , I close issue then. Please do not hesitate to contact us if you have new ones.


Start on Bulk API testset

Can you add the description of this directory and file structure as a README somewhere?

Will do

I'm not sure whether we want one file per test case listing all steps, or one file per step with the directory being the test case.
Pro file per test:

Easier to execute, no file system shenanigans

Pro file per step:

Easier to navigate, read, write

Con storing tests themselves as model:

We would need to package the serialization chunk as property, as we want to test invalid chunks

I foresee a maintenance problem with embedding chunks in the JSON test files.
We are using different tools (Freon, MPS, Java/Kotlin, ...) to read and write chunks, because we don't want to write bare JSON. These tools typically read/write chunks from a file. It is going to be hard to read/write the chunks embedded in the test files using these tools.
Maybe we should put the chunks in separate files and use the $variable syntax or something similar as values for chunks referring to a file containing a chunk.

I think the Java lib should mention the LionCore Built-ins as used language.

Maybe we should put the chunks in separate files and use the $variable syntax or something similar as values for chunks referring to a file containing a chunk.

How does this solve the problem that we have to refer to node ids we've got from a previous request?


[.NET] Korean DateTime code skeleton + Holiday support

Base code skeleton for Korean DateTime branching from English.
Added support for flag IgnoreResolution in DateTime tests in TestBase following Number implementation.
Support for Korean Holiday (only extraction), all test cases pass.
Added new test cases in HolidayExtractor corresponding to Korean holidays.

Sorry, shouldn't it branch from Chinese as they have similar root and to encourage reuse between Chinese/Japanese/Korean (CJK) like Numbers?

Sorry, shouldn't it branch from Chinese as they have similar root and to encourage reuse between Chinese/Japanese/Korean (CJK) like Numbers?

Since we are not completely aware of the differences between English and Chinese implementations at the moment, we decided to consider English first because following the standard implementation seems preferable (if possible). In case it is not, we will switch to Chinese implementation.
Or is it better in this case to start directly from Chinese?

This PR should be abandoned in favour of #2566, right?

yes

Closing as superseded by #2566.


ARM ComputeLibrary is slower than NNPACK?

I have the code as it is written below:
high_resolution_clock::time_point t1 = high_resolution_clock::now();
layer1->run();
layer2->run();
layer3->run();
...
layern->run();

arm_compute::CLScheduler::get().sync();
high_resolution_clock::time_point t2 = high_resolution_clock::now();

CLTensor tensor;
tensor.allocator()->init(TensorInfo(TensorShape(3U, 3U, 16U, 1U), 1, DataType::F32));
tensor.allocator()->allocate();
tensor.map();
tensor.unmap();

auto time = duration_cast<microseconds>( t2 - t1).count()/1e6;

I achieve worse execution time using OpenCL GPU ARM Compute Library engine than using NNPACK's CPU engine on mobile phone device (S. Galaxy S7) for the same NN architecture. Is there any obvious missing point that I am not aware of? Optimization flag or something else for mobile phone's GPU? Do you have time executions comparisons by layers between OpenCL GPU ARM Compute Library and NNPACK for CPU? I believe that GPU engine should be much more faster than CPU engine even though we talk about CPUs and GPUs on mobile phones.

Just a couple of things:

With 18.01 and previous releases for NEON you might want to manually set the number of threads used by the scheduler (Otherwise by default on big LITTLE systems it will defaults to the number total of cores rather than just the number of big cores)
Both NEON and CL performance should have greatly improved since this issue was open.
On the S8 the GPU cores are partitioned, and compute workloads can't use all the cores which is why in some cases you might not get the expected performance.

Have you tried to collect data with a more recent version of the library ?

I think because of bottleneck of copying data from cpu to gpu.


[SVCS-496] Fix filestystems folder name

Ticket
https://openscience.atlassian.net/browse/SVCS-496
Purpose
Filesystems folder metadata name was returning an empty string. This is a bug and should be fixed.
Changes
Fixes file metadata property and add root, folder and subfolder test cases.
Side effects
None that I know of.
QA Notes
This is not a user-facing bug
Deployment Notes
None.

Coverage remained the same at 89.773% when pulling 481c2091319e8ee03acfca6a18c866688ccb6772 on Johnetordoff:feature/fix-filestystems-folder-name into 3ec764ae879c107c405d67b1e2d60d1018eafa96 on CenterForOpenScience:develop.


[Discussion] Re-write all APIs

What would be your ideal Geolocator plugin?
I would love to re-think the entire API. Thoughts?

Hi James, any particular reason you want to rewrite it again ? I think the current one work fine. In fact have a suggestion for a few new plugins like

Custom Font Selector
SearchBar with Custom Icon
Maps with Maps App selector where instead of platform specific maps we can open map of our own choice google maps (if available).

Regards
S Ravi Kumar

Xamarin.Forms support for iOS and Android included.
3 modes:

A one-shot mode async mode which can be awaited, where you want to get one coordinate for one location. You set a threshold of time and accuracy, and the result is returned when one of the two is reached. -- get me one coordinate -> stop at deadline
A one-location continuous async mode with events and/or a returned object with bindable properties. You want to get one coordinate for one location, you can turn it on and leave it on, keeping the GPS on until you are happy with the returned coordinate and accuracy, and then turn it off again. -- get me one coordinate, I continue until it's good enough, and I want feedback on progress -> continue until accuracy reached
A continuous async mode with events and/or bindable properties. The GPS can be turned on/off at will (suspend/resume or in background), any change in location is registered, timestamped and stored by the Geolocation manager. If there is no coordinate update, or GPS signal is lost, you can query the last known coordinate. As a bonus there could be 2 timestamps, when a coordinate was first found, and when it was most recently found (how long the device has not moved). Bonus 2 would be to set thresholds on the sensitivity of changes before they are stored as the new coordinate. Bonus 3 would be for the Geolocation manager to store not only the last, but a (limited?) history of coordinates. -- get me any change in coordinate/orientation/accuracy/etc. as I am moving about and/or tracking my movements and/or getting multiple locations, I want to get a result at any time I want, even if there is no current coordinate (timestamp) -> continuous tracking

This is what we are looking for in geolocation, hope this is of any use to you.

Would also be really great if it could optionally interface with an external bluetooth GPS module and use that as a source instead of the internal GPS.

How about combining this plugin with the Geofence plugin https://github.com/domaven/xamarin-plugins/tree/master/Geofence.
In a current project I need both the current location and fences setup

Hi James,
Thanks for asking.
How about full background support?  Not saying it is easy, but when yoiu're asking the king of plugins....


⚠ Bofrost.be has degraded performance

In d957069, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1144 ms

Resolved: Bofrost.be performance has improved in 63dacda.


Fix eq and ne tpl function issue

eq and ne template functions don't work as expected when those are
used with a raw number and a calculated value by add, sub etc. It's
caused by both numbers type differences. For example, eq 5 (add 2 3)
returns false because raw 5 is int while add 2 3 returns 5 with
int64
This normalizes int, uint and float type values to int64,
uint64 and float64 before comparing them. Other type of value is
passed to comparing function without any changes.
Fix #961

That was fast. Merged in 44cdb37


Add option to build old androidndk r8e.

This version is the one required to build mobile Firefox. See build
documentation here:
https://wiki.mozilla.org/Mobile/Fennec/Android/Detailed_build_instructions#Install_Android_NDK
@svanderburg

Thanks -- I've pulled the r8e version into a separate file. Let me know if I should factor common stuff into an androidndk_common.nix file, or if you want me to squash those two commits together.


Avoid using "sudo pip" for installation

The instructions for installation suggest using "sudo pip" . OS X El Capitan will display a warning when doing so. For details : OSX El Capitan: sudo pip install OSError:

Okay, historically that was there because pythonpy would try to automatically install it's bash completion to /etc/bash_completion.d. But this has been so sketch for cross platform compatibility that I've revised how that is done and now one has a normal pip install without sudo. source $(find_pycompleter.py) will allow separate adding of bash completion if that is desired.


Add install targets to the sidebar menu

Brief Issue Summary
The extension should display all types of targets.
CMake Tools Diagnostics
No response
Debug Log
No response
Additional Information
No response

Is this specific to the Project Outline view? If so, I believe we now support this. In order to best process issues, I'm going to close this issue, but if you still have desired features, please create a new issue. THanks!


Error while compiling.

vue.common.js?e881:560 [Vue warn]: Error compiling template:

    <canvas
      :style="{height: size + 'px', width: size + 'px'}"
      height={{size}}
      width={{size}}
      v-el:qr
    ></canvas>

- height="{{size}}": Interpolation inside attributes has been removed. Use v-bind or the colon shorthand instead. For example, instead of
, use
.
- width="{{size}}": Interpolation inside attributes has been removed. Use v-bind or the colon shorthand instead. For example, instead of
, use
.

(found in <Qrcode>)

in vue 2.1.10   "vue-qrcode": "0.0.2",

un support for vue2.0

@asd8855 Please upgrade to 0.2.0, which is a complete rewrite from scratch, which supports<EMAIL_ADDRESS>Online DEMO: https://rx-vue.now.sh


Noisy output when using the provided checkpoints

Hello, Thanks for providing this repo. I get a very noisy output when using the checkpoints you provided (https://drive.google.com/drive/folders/1Fh7zr8zoTydNpD6hTNBPKUGN_s93Bqrs) to do the synthesis (using the synthesis.py code). I have also trained a fastspeech2 model using your code and I am getting a noisy output using my checkpoints as well. I have attached the output when using the checkpoints you've provided (checkpoint_model_150k_steps.pyt).
output.zip. I would be grateful if I could know what causes the difference between the attached generated output and the outputs provided in the sample folder.

@marzieh-razavi yes that actual pitch/f0 and energy are overfitted that's why you get high amplitude noisy output use 58k checkpoint with waveglow vocoder you feel less noise. I am currently working on that part. do share your tensorboard, as your test_tts.wav is too noisy due to very high amplitude..

I find similar results as @marzieh-razavi. This is my tensorboard screenshot for training/validation.

I think the model starts to overfit over the training set at about 12k steps, but I get noisy output when I use my reproduced  checkpoint at 10k-th step to generate audio as below. I am not sure whether the noise is due to overfitting or any other reason.
test_tts.zip

@ming024 are you using synthesis.py to generate output? If yes then please use colab notebook and check your checkpoint there because their are some issues with MelGAN when I am saving the audio in synthesis.py or to get rid of that use waveglow.
Actually, Whenever I play generated audio in IPython.display.Audio it's playing well but when I save the same audio
(wav) using librosa.output.write_wav or scipy it gets noisy I don't know why? especially when audio generated from MelGAN and this problem is not with Waveglow.

@rishikksh20
Save the wav signal as 16-bit integers with scipy.io.wavfile.write  fix the problem on my machine.
from scipy.io.wavfile import write

write(path, hp.sample_rate, x.astype('int16'))

@ming024 thanks

@marzieh-razavi take pull of latest commit and then use your checkpoint and run synthesis.py . Please report here if issue still exist.

@rishikksh20 Thanks a lot for providing the update. It works now.


Hl Groups

The plugins lacks highlight groups which makes it harder to use it with multiple colorschemes

Thanks for reporting @UTFeight
Can you give an example (ideally including a screenshot)?
I didn't have any issues and were also using multiple colorschemes.

Sorry...
Somehow I thought this plugin did not implement colorscheme specific attributes.
Here:
Gruvbox Dark

Ayu Light

[!NOTE]
As you can see link highlights differ by the color scheme (which is how it should)

@UTFeight you were right when you said highlight groups are not taken into account in the code.
So far this plugin's approach is to present the information as a specific filetype which will then be presented with filetype specific highlighting.


Fix deprecation messages for Rails 5

this makes it ONLY compatible with Rails 5.1+, right? should we update the gemspec to make that clear, so that we don't accidentally apply it to a project which doesn't support it...also, in case someone in the public-land does this?


Automated Package Upgrade

:tada: This PR is included in version 27.1.57 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Flow react ref on custom component?

So, I've read this page (https://flow.org/en/docs/react/refs/) which works for the simple case (button), but how do you do it on a custom react component?
I've tried something like:
import * as React from 'react';
import SomeLibrary from 'some-library';

type SomeLibraryAPI = {
  foo: () => void
};

class MyComponent extends React.Component<{}> {
  lib: ?SomeLibraryAPI;

  render() {
    return (
      <SomeLibrary ref={(lib: SomeLibraryAPI) => this.lib = lib} />
    );
  }
}

https://flow.org/try/#0JYWwDg9gTgLgBAKjgQwM5wEoFNkGN4BmUEIcA5FDvmQNwBQdMAnmFnAMolYAywARlGRQmAQQAKASTgBeOAG86cOAQgQAXHAAUAShkA+OADcIwACZ0AvvTq4ANmnScQPfoOFwsADxhYAdqfRsPBgAOgBhEkhfPxgAHjkLAwUlFQgtXWSlOFwIX1QIWywQ2wgAc00yVLJteiULRTgGyn8sKB15BqVKGABXKF84WNNgQz0ACSxbEtiAemHR2rh6+pt7VHQAWSYI8FyYj28-AMwqUJ2omPjEjqVbfg0AficXASFRSWsuo9b2zK-e-paTpZWLPXivdyUAjSOSaO58DRg1xvcQSXTSAwwAAWwFQxX4Mjg8IscBmemBNQayyAA
and I ran into a bunch of errors. Am I missing something?

ref function takes as argument the instance of SomeLibrary and it needs to also handle null since on unmount it will be called with null. SomeLibraryAPI does not match SomeLibrary since for classes flow applies nominal typing.
Overall, this works (only changes are (lib: ?SomeLibraryAPI) => /* ... */ and lib: ?SomeLibrary;):
https://flow.org/try/#0JYWwDg9gTgLgBAKjgQwM5wEoFNkGN4BmUEIcA5FDvmQNwBQdMAnmFnAMolYAywARlGRQmAQQAKASTgBeOAG86cOAQgQAXHAAUAShkA+OADcIwACZ0AvvTq4ANmnScQPfoOFwsADxhYAdqfRsPBgAOgBhEkhfPxgAHjkLAwUlFQgtXWSlOFwIX1QIWywQ2wgAc00yVLJteiULRTgGyn8sKB15BqVKGABXKF84WNNgQz0ACSxbEtiAemHR2rh6+pt7VHQAWSYI8FyYj28-AMwqUJ2omPjEjqVbfg0AficXASEmay6j1vbMz97+rSdLKxZ68V7uSgEaRyTR3PiPUGuN66aQGGAAC2AqGK-BkcDhFjgMz0QJqDWWQA

@cprodescu Ah interesting, thanks! So, do I even need SomeLibraryAPI anymore? I don't have to explicitly tell flow what methods are in the class?

hmm I guess not, everything works properly! thanks!


Compile python requirements

Automated changes by create-pull-request GitHub action

Seems like moto deprecated mock_s3 and now uses mock_aws instead

I forgot that running the compile action (even against a branch that isn't main) would force push onto this branch!
so the mock_aws fixes got blown away


Istio mTLS is enabled by default even when set to False

PR https://github.com/istio/istio/pull/6395 creates a config map that is not gated by the global auth flag. As a result, the default destination rule for all services has tls mode set to Istio mutual, there by enabling istio auth for all clients (even though the server side is not enabled).
I encountered this error by looking at Pilot's cluster output where I saw TLS settings for istio-egressgateway even though I had explicitly specified mTLS: false in the helm templates.

You can see the in the daily release : https://gcsweb.istio.io/gcs/istio-prerelease/daily-build/release-1.0-20180711-09-15/ where istio-demo.yaml (no auth) has a default destination rule that enables mTLS for all.

please ignore.. I can't reproduce the issue with a fresh install.


subversion17 can't access the repository via https

Built the 1.7.14_1 bottle with --with-unicode-path option in OS X 10.10.3.
$ brew install subversion17 --with-unicode-path

And then tried to checkout the repository via https, but occurred error:
$ svn checkout https://example.com/foobar/
svn: E175000: Unable to connect to a repository at URL 'https://example.com/foobar/'
svn: E175000: SSL is not supported

I guess that the cause of this is neon is not built with SSL option.
$ svn --version
svn, version 1.7.14 (r1542130)
   compiled Apr 23 2015, 22:22:39

Copyright (C) 2013 The Apache Software Foundation.
This software consists of contributions made by many people; see the NOTICE
file for more information.
Subversion is open source software, see http://subversion.apache.org/

The following repository access (RA) modules are available:

* ra_neon : Module for accessing a repository via WebDAV protocol using Neon.
  - handles 'http' scheme
* ra_svn : Module for accessing a repository using the svn network protocol.
  - with Cyrus SASL authentication
  - handles 'svn' scheme
* ra_local : Module for accessing a repository on local disk.
  - handles 'file' scheme
* ra_serf : Module for accessing a repository via WebDAV protocol using serf.
  - handles 'http' scheme
  - handles 'https' scheme

Aye. This is a known thing, sadly. Subversion17 (and 16) refuse to recognise a Neon newer than the ones they shipped with, and disabling the version check on that has led to other problems in the past, such as the whole thing throwing fatal failures at runtime and rendering the formulae involved useless, heh.
So we're tied to an old neon on that - But trying to compile that old neon with Homebrew's newer OpenSSL led to compile failure galore. I'll look into it some more, but I'm not super super hopeful on any reasonable solution here.

Aye. We're not going to be able to do anything on this by the looks of it. It's looking for SSLv2 in compile, which you won't find in any OpenSSL shipped by Homebrew (Or Debian, or Ubuntu, etc) -
Undefined symbols for architecture x86_64:
  "_SSLv2_server_method", referenced from:
      _ne_ssl_context_create in ne_openssl.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [libneon.la] Error 1
make: *** [subdirs] Error 2

I would be uncomfortable vendoring an SSLv2 enabled OpenSSL for the older Subversions and Neons. CC @Homebrew/owners for thoughts though.

Yeh, feels a bit weird.

Can it use system openssl?

Can it use system openssl?

Should be possible, theoretically. Apple still ships some SSLv2 ciphers.

That seems like a good idea :+1:

Do we want to hurl a caveat or anything in given the system OpenSSL situation?

If you have specific concerns you could enumerate them in a caveat (but I think probably not?).

Well, I have specific concerns that it's using outdated, well-regarded-insecure ciphers which share roughly the security benefit of yelling your bank details out of an open window, but :wink:.

It probably isn't, though; that's a server configuration issue. If system OpenSSL were e.g. vulnerable to downgrade attacks maybe that's another story.

System OpenSSL is going to die in terms of upstream support at the end of this year. I guess my question is, after the first major security problem is announced post December 2015 for OpenSSL, what do we do with the subversion16 and subversion17 formulae?

I'd spike 'em.

Alright. I can live with supporting them until that point.

Got it. Thanks! :smile:

It's fixed for now. brew update && brew reinstall subversion17 should allow you to checkout SSL/TLS repos again.


⚠ ClamAV has degraded performance

In ab1cbef, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 120 ms

Resolved: ClamAV performance has improved in a3f1251 after 1 hour, 19 minutes.


Error when trying to install displaylink on zen kernel

Describe the bug
An error is happening when I'm trying to install displaylink on a zen kernel
Steps To Reproduce
Steps to reproduce the behavior:

Inside configuration.nix

services.xserver.videoDrivers = [ "displaylink" "modsetting" ];

nix-prefetch the displaylink driver from their website
Rebuild it

# nixos-rebuild switch

Expected behavior
Getting displaylink driver to be installed and to be used for an Asus portable monitor
Screenshots

Additional context
It works fine with the vanilla kernel
Notify maintainers
@nshalman
@abbradar
@peterhoeg
@eyJhb
Metadata
# nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 5.15.3-zen1, NixOS, 21.11 (Porcupine)`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(root): `"nixos-21.11.335820.521e4d7d13b, nixos-hardware, nixos-unstable-22.05pre351891.fbd030fb2d8"`
 - nixpkgs: `/etc/.nixpkgs`

Are you sure this isn't an upstream issue?

I'm not really sure if this is an upstream issue, that's why I reported it first here...


Completed Kata; solutions still locked

Completed Kata: https://www.codewars.com/kata/simple-encryption-number-2-index-difference in Python.
Shows up in my profile's Completed list at https://www.codewars.com/users/kjmosher .
Still getting "Solutions have been withheld" message when attempting to view Kata's solutions.
Attempted to resubmit solution; Basic tests passed, attempted final tests.  Output window freezes on "Executing" , but allowed me to submit ..  Took me right back to the "Solutions have been withheld" screen.

Same thing just happened on kata: https://www.codewars.com/kata/sum-up-the-random-string

I had the same issue on https://www.codewars.com/kata/sum-the-repeats, over a month ago.

Just had the same issue with https://www.codewars.com/kata/create-phone-number, clicked on "forfeit eligibility" after submitting and getting accepted twice, and my code wasn't there. Submitted it a third time, and it's still not there.

I submitted my solution two times, but got on "Solutions" tab:
"Solutions have been withheld
Since you have not yet solved this kata we have hidden the solutions from you. If you choose to view the solutions you will forfeit your eligibility to earn honor/rank progress for this kata."

You can use preview.codewars.com. The issue is fixed there. It will be released to www next week along with a big update.

@jhoffner was this issue resolved?

Similar issue: #314, #441, #553, #617, #665

Same thing with Katas
https://www.codewars.com/kata/5596f6e9529e9ab6fb000014
https://www.codewars.com/kata/557f6437bf8dcdd135000010

Completed this Katas
https://www.codewars.com/kata/5596f6e9529e9ab6fb000014
https://www.codewars.com/kata/557f6437bf8dcdd135000010
They added to my list of finished in my status. But I didn't get honor for them and they didn't mark with tick in list of all Katas(All other my finished Katas marked with tick).

Time to time I have to duplicate sending correct solutions get solutions list open. Because after fist successful Submit Final a solutions stays closed. Today this happens twice on the baby-shark-lyrics-generator kata when I solve it in JS and Ruby.  Also very often need to click Attempt several times before Submit Final appears.
#617 #1683 - relates issues.

@lbvf50mobile That's https://github.com/Codewars/codewars.com/issues/1683

I had the same issue today. I think i got the honour points twice.
https://www.codewars.com/kata/a-string-of-sorts/train/haskell

Same here. Is there are any updates?

i don't mind the locked solutions bug at all, i got in the habit of backing up my solutions. but i would really appreciate not getting the extra honour / rank, that's just unfair

@falcowinkler, it didn't "exist for 3 years". The bug reappeared a few months ago.

No, it exists for at least 3 years. But the frequency seemed to have increased this year, either because of updating the database driver at the end of last year or the increased amount of data.
Also, there are many other symptoms caused by the same bug that's been reported (solution not shown, solution locked, solution appears multiple times under profile, failing to submit after passing, not getting honor, getting honor multiple times, etc).
Codewars has many bugs that existed for many years. This is because no one was working on the project for few years until I started to work on it part time last year (spent most of the time just updating EOLed components from 6 years ago :().
This is all because we needed to focus on Qualified which is required for Codewars to exist. We've reached the point that we can start working on Codewars more so we'll make a post explaining the current situation and our future plans soon.

The issue should be fixed (the affected data from the past is not yet fixed). If this happens again, please let me know by commenting on #1683.


[BUG] Remix (SPA) Support

Describe the bug
The current build process of refine packages makes it incompatible with Remix SPA, and possibly? more.
This has been tested on Remix 2.8.0.
The following steps need to be taken:

Disable minify on all repos in tsup esbuildOptions via options.minify = false; (remix SPA does a sort-of 1 time SSR eval, running Refine through a node-js server which needs the ESM import, unminified)
Disable tsbuild --legacyOutput.
Add export maps with the esm import pointing to a .mjs file (this actually really matters)

Ensure export maps have typings as well.

Replace lodash-es and papaparse at a minimum to use .js extensions when imported
Update commands in packages/cli/src/commands/runner as remix-serve no longer exists and needs to use remix vite:*?

You can see a debug branch with many of these changes at https://github.com/LumeWeb/refine/tree/remix for reference.
Steps To Reproduce
Try to build a remix SPA with any basic refine example.
Expected behavior
It should run the dev server and build correctly.
Packages
All refine packages are likely impacted if they are to be used within remix.
Additional Context
No response

Thanks for the explanation @pcfreak30! About the lodash imports, we have an esbuild plugin to replace them in cjs and esm builds which is shared between packages, I think updating it to include .js so we don't have to worry about cjs builds 😅
About the types in the package.json#exports, do you think we should also create ones with .mts extensions or is it not required?

Thanks for the explanation @pcfreak30! About the lodash imports, we have an esbuild plugin to replace them in cjs and esm builds which is shared between packages, I think updating it to include .js so we don't have to worry about cjs builds 😅
About the types in the package.json#exports, do you think we should also create ones with .mts extensions or is it not required?

The .mts I don't know, but Webstorm definitely complained about types in general (while honoring exports) (prob from tsc). That is mostly an IDE thing, IIRC.


Adds missing metric in Android module

Description
sdkVersion metrics must be added by the JavaScript interface, however for explicitness it should be added in the Android module. This way if for some reason the value is not added by JS code, the element will still be there in the metrics payload.
The way this metrics is set is something like:
function sendAppAndDeviceMetrics() {
  this.getAppAndDeviceMetricsFromNativeModule()
    .then(metrics => {
      metrics.app.sdkVersion = this.getPluginVersionFromWherever();

      // publish
    });
}

I see that change is not complete. I guess we can merge it, but I think that we can easily provide complete implementation as you have mentioned. I'm worried if we start PR with some unfinished changes we may then forget about this or leave some things without implementation.

@wtrocki It is neither incomplete nor failing, this change is optional and doesn't have anything to do with JS implementation. It doesn't change the behaviour of the SDK.


Fix/advanced menu delay

1. Why is this change necessary?
The advanced menu hover effect does not work correctly. This pull request faces/solves multiple issues:

The "Hover delay (ms)", that can be configured in the settings, is not respected on first hover of the navigation bar after loading a page. The advanced menu instantly opens. The delay is respected on following (2nd, 3rd, ...) hovers.
When moving the cursor downwards moving over the navigation bar, the advanced menu always opens. Even when the cursor leaves the navigation bar in less time than configured as "Hover delay (ms)". This happens when the cursor stays in the area where the advanced menu is about to be displayed. This is an annoying behavior, because the advanced menu covers the first part of the content section - every time a user moves down the cursor from the header to the content section (without the intention to open the menu).
The performance impact of the used mousemove function on body is bad. This problem has partially been addressed with this contribution: https://github.com/shopware/shopware/pull/2153 https://issues.shopware.com/issues/SW-24297 https://github.com/shopware/shopware/commit/06eb4d0b23548848da0b1ed011aee1790ab67408# But this "fix" broke the hover delay resulting in the issue described above.

2. What does this change do, exactly?
This change solves the three issues described above by

reverting the commit https://github.com/shopware/shopware/commit/06eb4d0b23548848da0b1ed011aee1790ab67408#
replacing the mousemove listener on the body element with mouseleave listeners only on the relevant elements
initially calling closeMenu() to fix the first issue (hover delay not working on first hover)

3. Describe each step to reproduce the issue or behaviour.
Description under '1. Why is this change necessary?' should contain all information to reproduce the issues.
4. Please link to the relevant issues (if any).
N/A
5. Which documentation changes (if any) need to be made because of this PR?
N/A
6. Checklist

[x] I have written tests and verified that they fail without my change
[x] I have squashed any insignificant commits
[x] This change has comments for package types, values, functions, and non-obvious lines of code
[x] I have read the contribution requirements and fulfil them.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hey @favor-it ,
Can you push your changes in two commits? First reverting that commit and then applying your fixes?
Thanks! :blue_heart:

Hey @shyim, I have just updated my pull request accordingly.

Hi,
looks interesting. I wrote the first patch from a modified version used in production. Basically the patch moved the events to better position and nowhere near from the setTimeout or delay.
In my version i use throttle from underscore to handle the events and delay to avoid rapid firing the event.
I currently testing your version and looks fine with my patches applied.

Hello,
thank you for creating this pull request.
I have opened an issue on our Issue Tracker for you. See the issue link: https://issues.shopware.com/issues/SW-26263
Please use this issue to track the state of your pull request.

@favor-it i noticed touch won't work, maybe i messed up something, but it looks like touch won't open the menu. e.g. Tablet.
Fix for me
                me._on($el, 'touchstart', $.proxy(me.onTouchStart, me, i, $el));
                me._on($el, 'mouseenter touchstart', $.proxy(me.onListItemEnter, me, i, $el));

@mitelg this should be fine now and also work in responsive mode


[Erlang] Added an Erlang implementation.

some tips
You are measuring time by hand with get_timestamp/0. You can user timer:tc/2 to do this for you.
In next/4 you should utilize pattern matching on the first character of Start and than construct a list recursively.
Extracting n-th charachter by hand, and than recreating the whole string with string:left(Start, I - 1) ++ New ++ string:right(Start, N - I), is O(n), so basically next is O(n^2).
Last parameter of next is always 1 character long. Do you even need it to be a list?
Same for convert, why not using just a character? I guess this is some genetic computing, so if you don’t expect input that is not A, C, G or T, just let it crash. Also, you can use pattern matching in the function argument.
Also, you are using some obsolete functions from string module. This surely works, but take a look into the documentation for substitute functions.
you can use pattern matching in calculate/4 in a following way:
calculate(_, Stop, Stop, C) -> C; %% This ensures 2nd and 3rd argument is equal
calculate(N, Start,Stop,C) -> calcualate(N, next(N,Start), Stop, C+1).

so the "next" function can be written like this
next(N, Start) -> next(N,1,Start,'T').

next(N, I, [C|Rest], 'T') ->
    [convert(C) | next(N, I+1,Rest,C)];
next(_, _, Start, _) ->
    Start.

convert('A') -> 'C';
convert('C') -> 'G';
convert('G') -> 'T';
convert('T') -> 'A'.

gl!


Modernizing for Azure Functions OpenAI extension bindings, Flex Consumption, and Managed Identity

Purpose

Updated the code to use the Azure Functions OpenAI extension bindings
Bicep now uses Azure Functions Flex Consumption and Managed Identity (no more keys/secrets)
Added Azure Functions DevContainer/Codespaces support

Does this introduce a breaking change?

[ ] Yes
[x] No

Pull Request Type
What kind of change does this Pull Request introduce?

[x] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

How to Test
Follow the readme

@dm-chelupati could you please review?

@vrdmr could you please review?


sqs input does not work after a while

From https://logstash.jira.com/browse/LOGSTASH-2255
Hi,
We have been using sqs output plugin on different logstash version. At last, It it is used on logstash 1.4.1. In that version, per every 2-3 weeks, the sqs plugin has been crashed. It has been stopping to receive from sqs queue but logstash has been continued to work.. After logstash restarted, everything goes fine. Yesterday it repeated, we catched below log.
Note: We are using IAM roles on Aws to authenticate aws sqs service.
{:timestamp=>"2014-06-22T08:10:43.067000+0000", :message=>"Error reading SQS queue.", :error=>#<AWS::SQS::Errors::SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'POST
/544087101480/WsServiceLogs
content-length:231
content-type:application/x-www-form-urlencoded; charset=utf-8
host:sqs.eu-west-1.amazonaws.com
user-agent:aws-sdk-ruby/1.35.0 jruby/1.9.3 java
x-amz-content-sha256:e55e8ad53e44c3f51337e79c5be913b4980ecf21b7fdfd4f7bad182a2bb3b8bc
x-amz-date:20140622T081042Z
x-amz-security-token:AQoDYXdzECka0AOo3QtT3g/……………………………………………. EIk1IAokX2D53/F0qXjvND4XbEyCykZqdBQ==
content-length;content-type;host;user-agent;x-amz-content-sha256;x-amz-date;x-amz-security-token
e55e8ad53e44c3f5…………………………b7fdfd4f7bad182a2bb3b8bc'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20140622T081042Z
20140622/eu-west-1/sqs/aws4_request
4100c5a8468784……………………………………..77c7794563d201'
thanks,

Running into same exact issue. Any estimates when a fix for it will be available? THx

Any errors in the SQS Output plugin will cause logstash to crash. It needs better error handling around the 'receive' event. Personally I'd rather see this kind of handling make it in to the core framework so one misbehaving plugin doesn't risk the stability of the entire app.

Can anyone comment on if this is still an issue with the latest version of logstash (5) and the latest version of this plugin (4) ?

this is still happening in
logstash 7.16.1
logstash-output-sqs (6.0.0)
but it makes logstash restart itself, so we got that going for us


Error when running composer install --no-dev

When running composer install --no-dev you receive the following error message:
  Class "AaronFrancis\Solo\Providers\SoloApplicationServiceProvider" not found

Which prevents the application from building. I'm not sure if a Deferred Provider (https://laravel.com/docs/11.x/providers#deferred-providers) would be a solution for this? Or if I'm missing a more obvious solution. If you know generally how to fix this, then I'd be happy to get a pointer or two and then to contribute to the package as it's already been massively useful in local development! 🙌

I think that's fixed in 0.2.1! Seems like the same thing as #19

Amazing, yes that works now for me! I did need to re-install solo but I think it's a niche issue there 😄 Thanks for the quick response.


Fix error where datasource input can't be deleted

There's a problem with the Grafana dashboard image when deployed on OpenShift. The container is run with a random UID (not grafana) and GID 0. This was causing it to crash due to being unable to delete the datasource.yaml.in file. This PR changes ownership of our datasource/dashboard files to have GID 0. This allows them to be edited in OpenShift, as they would be with Podman. I also changed the input file in /tmp to the actual destination, and just use in-place sed. This was because /tmp's sticky bit still prevented deletion, even with the GID fix.

Image pushed to quay.io/rh-jmc-team/container-jfr-grafana-dashboard:1.0.0-BETA2


GraphQL filter design

After discussing the design today with @laura-ham and @JeroenStravers1, I realized that #438 is superfluous.
There still two things that I'd like to suggest:

Make AND and OR take a list of clauses
Add a NOT, useful to flip selections in the Playground.
Merge the IE and EQ  into one (also the boolean operators such as and, or, not)
Remove operator from the value string
Add typed values.
Have only one single top level filter; use and/or to have multiple filters.

These will IMHO still have the effect of making the queries more algebraic (read: like a structured tree), and therefore easier to build/consume in e.g. the Playground.
Some examples
ConvertedFetch(where: { path: ["Things", "person", "age], operator: GreaterThan, valueInt: 18 })

ConvertedFetch(where:
  { operator: Or, clauses: [
    { operator: And, clauses: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 21},
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "male" }
    ]},
    { operator: And, clauses: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 18 },
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "female" }
    ]},
    { path: ["Things", "Person", "hasDrivingLicense"], operator: Equal, valueBool: true }
  ]}
)

Graphical representation of how these filters can be constructed

Extensibility
This is an extensible scheme. We could also for example add a "CloseTo" operator, or an "InRange" operator, that would internally be desugred to simpler queries. For example, the "CloseTo" and "InRange" would resolve to  x >= expected - epsilon && x <= expected + epsilon and x >= minRange && x <= maxRange respectively.

@laura-ham, what are your thoughts here? Would be nice to add this to the wiki as "filter design pattern".

@moretea  Thanks for the great suggestions. Although I see that this is a more robust and extensible scheme, I have some things to mention:
A very simple filter will look like this according to your design, right? I'm not sure whether 'clauses', with a list, should be the right way to do this.
ConvertedFetch(where:
    { clauses: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 21}
    ]}
)

ConvertedFetch(where:
    { operator: Not, clauses: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 21}
    ]}
)

In the original design, operators are fields themselves, rather than calling them as operator: Or, clauses: [{}]. Also, you are using the argument 'operator' in two different layers: one where you expect a list of clauses and one where you expect a path and value. What do you think of the following design? In my opinion, this is more user friendly, but maybe you could give your thoughts about the extensibility of this?
ConvertedFetch(where:
  OR: [
    { AND: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 21},
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "male" }
    ]},
    { AND: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 18 },
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "female" }
    ]},
    { path: ["Things", "Person", "hasDrivingLicense"], operator: Equal, valueBool: true }
  ]
)

And this will look like the following with a 'NOT' operator:
ConvertedFetch(where:
  NOT: { OR: [
    { AND: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 21},
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "male" }
    ]},
    { AND: [
      { path: ["Things", "Person", "age], operator: GreaterThan, valueInt: 18 },
      { path: ["Things", "Person", "gender"], operator: Equal, valueString: "female" }
    ]},
    { path: ["Things", "Person", "hasDrivingLicense"], operator: Equal, valueBool: true }
  ]}
)

@bobvanluijt I'll add this to the wiki in a separate page.

The clauses are only supposed to be filled for AND and OR. See the first snippet in my opening comment in this issue, where I only filter on people above the age of 18.
To be perfectly clear, I'm fine with any syntactical version; I do care about the semantics. I believe we really should have a proper tree structure for the formulas; it will make the filters unambiguous and easy to mechanically manipulate by e.g. the playground.
Consider a simple formula like x^2 * 4 + 3. What kind of formula is this? According to the precedence rules, it should be parsed like ((x^2) * 4) + 3, so this formula, at it's top level is an addition.
I'd like to have the same assurances about the filters.
Another way than preventing this from being possible by construction, as I'm proposing here now, we could just return errors if users specify both an AND and OR at the same level in the tree.
W.r.t. the actual design of the filter, something like this might suffice for my proposal,
input Filter {
    operator FilterOperator!

    clauses: [Filter] # Probably needs better name

    path String
    valueInt int
    valueBool bool
    valueDouble double
    #etc
}

enum FIlterOperator {
    And
    Or
    Equal
    NotEqual
    GreaterThan
    GreaterThanEqual
    /// etc
}

We could also solve this by adding extra "type-checks" to the resolver for the GraphQL queries. That it would only accept queries if and only if they are well formed (e.g. the represent something in the form of x && y at one level, not x && y p || q, which would be the case if we allow both AND and OR to be set in the same object.

@laura-ham is this still relevant?

@laura-ham is this still relevant?

@bobvanluijt I think this issue can be closed because the where filter is designed according to these suggestions a while ago.


Draft: Resolve Errors Around Failed PDB Updates

This PR resolves the two following errors we were seeing in production logs:
The first occurs when an object is updated in the middle of the reconcile loop causing the controller's knowledge of the object to be out of date.
time="2024-07-17T14:20:46Z" level=error msg="Failed to update PDB: Operation cannot be fulfilled on poddisruptionbudgets.policy \"my-cool-deployment-pdb-controller\": the object has been modified; please apply your changes to the latest version and try again"

and the second is related to https://github.com/mikkeloscar/pdb-controller/issues/54. The issue is occurs because the loop deletes the PDB then attempts to update it after it is deleted.
time="2024-07-17T14:10:42Z" level=error msg="Failed to update PDB: Operation cannot be fulfilled on poddisruptionbudgets.policy \"my-cool-deployment-pdb-controller\": StorageError: invalid object, Code: 4, Key: /registry/poddisruptionbudgets/my-namespace/my-cool-deployment-pdb-controller, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: <uid redacted>, UID in object meta: "

The first was resolved by adding a RetryOnConflict wrapper around the update and create methods.
The second was resolved by ending the loop early if we delete the PDB.

After some more testing it seems this PR does resolve the second error message but it does not yet resolve the first one:
time="2024-07-17T14:20:46Z" level=error msg="Failed to update PDB: Operation cannot be fulfilled on poddisruptionbudgets.policy \"my-cool-deployment-pdb-controller\": the object has been modified; please apply your changes to the latest version and try again"

Even with a longer backoff than the default it doesn't seem to care much for the update we're trying to make. I'm going to dig some more and try and find where the conflict is coming from.

After some more testing it seems this PR does resolve the second error message but it does not yet resolve the first one:
time="2024-07-17T14:20:46Z" level=error msg="Failed to update PDB: Operation cannot be fulfilled on poddisruptionbudgets.policy \"my-cool-deployment-pdb-controller\": the object has been modified; please apply your changes to the latest version and try again"

Even with a longer backoff than the default it doesn't seem to care much for the update we're trying to make. I'm going to dig some more and try and find where the conflict is coming from.

After digging some more into this I still can't find which fields are exactly causing issues but because we're doing a .Update() instead of a .Patch() I think we're getting caught up in some dynamic fields being changed which causes sporadic failures.
I was able to work around this in our own clusters by adding a step after the .Update() method to do another .Get() of the same object and then applying the desired state over that fresh copy of the managedPDB.
I think the PR can still go as it is now and then I can come back with a second PR with the described logic for a separate review. I don't want to slow down the fix for the error around the controller attempting to update deleted PDBs. So I will modify the description of this PR to be more focused on that specific issue and will leave the failed updates for another PR after.

How the retryOnConflict is used doesn't seem right/have any effect according to how it's documented here: https://pkg.go.dev/k8s.io/client-go/util/retry#RetryOnConflict
I would propose to remove those changes from this PR, then we can merge the continue fix to avoid updating deleted resources.

How the retryOnConflict is used doesn't seem right/have any effect according to how it's documented here: https://pkg.go.dev/k8s.io/client-go/util/retry#RetryOnConflict
I would propose to remove those changes from this PR, then we can merge the continue fix to avoid updating deleted resources.
Also it's not really a problem that an update fails because the controller will automatically retry in next reconciliation loop.

Agreed on removing the RetryOnConflict since it's not relevant to this PR now in it's more limited scope. I have updated my PR to reflect this change.
I have been running a local fork of it in our clusters with a RetryOnConflict though and it does resolve the failed update errors I mentioned in my previous comment. We can have that conversation in a more relevant PR though after this :)

👍

:+1:


Error while adding OneDrive - System.Runtime.InteropServices.COMException (0xC00D3E85)

Description
Error while adding OneDrive - System.Runtime.InteropServices.COMException (0xC00D3E85)
Steps To Reproduce
Requirements
Fix error pop-up / provide more info on how to resolve this OneDrive error
Files Version
<IP_ADDRESS>
Windows Version
10.0.22631.2792
Log File
debug.log

@Sacchid according to the log file, it looks like OneDrive was added without any issues.

Sorry I might have read it wrong. I still got 2-3 notifications to report errors, is there some other error?

Yes, It does crash but opened all tabs immediately.
debug.log

At which point is it crashing, right away on startup, or is it after you already start using the app?

While using app or even in the background when I have minimised it & using someother app.

Can you confirm if it's the same as #13995?


Add Cargo manifest for monorepo

Creates top-level Cargo.toml that declares a workspace
Moves workspace members into top-level

Most of the profile settings for legacy workspaces are put into the individual crates in their manifest, and at the moment are being ignored by Cargo.
Removes legacy workspaces by removing the Cargo manifest that defines them.

Still needs some cleanup of inner Cargo.lock files

I guess we also want to think about how this impacts the nix build. IIRC some of the crates have their own workspace-aware nix build scripts, and we probably don't want to break those

I guess we also want to think about how this impacts the nix build. IIRC some of the crates have their own workspace-aware nix build scripts, and we probably don't want to break those

Yeah, I'm doing it because the nix flakes were having problems, mainly (e.g. flake file needs to be on same directory as Cargo.lock). I guess that the next step is to refactor nix for the monorepo.

LGTM

I guess we also want to think about how this impacts the nix build. IIRC some of the crates have their own workspace-aware nix build scripts, and we probably don't want to break those

Yeah, I'm doing it because the nix flakes were having problems, mainly (e.g. flake file needs to be on same directory as Cargo.lock). I guess that the next step is to refactor nix for the monorepo.

Correct, this will make nix packaging much easier.


Adding data for testing legacy index file formats used by WebDataset

This PR adds index files for the WDS data format. The point is that we would like to test the legacy file formats and backwards compatibility with them. Added directory will contain all legacy WDS index file formats.
Signed-off-by: Michał Szołucha<EMAIL_ADDRESS>

Maybe you should add a readme with a link to a particular version of idx generator script used to obtain this data?

Sure, I'll add it in this PR as soon as the NVIDIA/DALI#3614 is approved (right now there will be circular commit dependency...)


Improve ClassLoader to load jars in jars.

Applied as new commit
https://github.com/seeburger-ag/karaf/commit/4ceb6e09907ab5c31f1afef241bcec707b893183


Using lower frequency data(monthly, quarterly,annually) for generating tear sheet

use alphalens.tears.create_returns_tear_sheet with quarterly/annually dataseries
Since I have access to return data, I didn't use utils.get_clean_factor_and_forward_returns but created dataframe as suggested. I also set df.index.levels[0].freq = pd.DateOffset(months = 1)
df.head()
                          1M        3M       12M    factor  group  \
date       asset
2010-04-30 0028262 -0.087310  0.085591  0.599461  0.459896    3.0
           0045614 -0.149729  0.027014  0.516753 -0.088869    8.0
           0056650 -0.107866 -0.014220  0.395736 -0.970670    8.0
           0096162 -0.136532 -0.080226  0.344415 -1.018054    6.0
           0118271 -0.080590 -0.064945 -0.025323 -0.055454    7.0

                    factor_quantile
date       asset
2010-04-30 0028262              4.0
           0045614              3.0
           0056650              1.0
           0096162              1.0
           0118271              3.0

But I am not getting graphs. Is there some way to generate quantile performance charts using alphalens using longer frequency dataseries

Please have a look at get_clean_factor_and_forward_returns implementation, You will see that it computes forward returns, which you already have and then it calls get_clean_factor, that is what you need to call.  You only need to make sure your forward returns are properly formatted (the format is documented in get_clean_factor function).
Once you are sure that your input data is properly formatted everything should work fine. If that's not the case please report the error together with a minimal example that is able to reproduce the error.
thanks.

My forward returns column names are ('1M','3M','12M') respectively for monthly/quarterly/annual returns. Will this work? I can see in "get_clean_factor" documentation  column names should follow pd.Timedelta format. When I try pd.Timedelta('1M') it returns Timedelta('0 days 00:01:00') i.e. 1 minute object

When I try pd.Timedelta('1M') it returns Timedelta('0 days 00:01:00') i.e. 1 minute object

Exactly, that's the problem. 'M' refers to minute. Also, it wouldn't be a good a idea to specify the forward return period as number of month or years, since those don't represent a specific number of days. I would suggest you to use the exact number of trading days your returns represent (e.g. "252D" for a year or "21D" for a month). Then make sure to call infer_trading_calendar like here, so that Alphalens will be able to handle holidays, weekends and all the non-trading days correctly

Feel free to re-open the issue if you still have problems with this

Hi, I run into similar problems here as I would like to analyze 1 month forward returns at month ends. I used '21D' as the forwarding return name, and it works well. However, may I ask how you solve the freq issue if you have any? I have a month-end price dataset, and the df.index.levels[0].freq = M as I have month-end dates. I don't think using Bday=21 would solve this problem as adding 21 business days does not always return month ends.


Fix typo in partybus upgrade output

Database was in this text twice
Signed-off-by: Tim Smith<EMAIL_ADDRESS>

https://buildkite.com/chef/chef-chef-server-master-omnibus-adhoc/builds/3098#2f30351b-0cf9-4e94-a5f9-e2d24b75dc17


Create bot_pr_approval.yaml

Applicable spec: N/A
Overview
Enables the auto-approval of bot PRs. @yhaliaw could you please get this PR merged?
Checklist

[x] The charm style guide was applied
[x] The contributing guide was applied
[x] The changes are compliant with ISD054 - Managing Charm Complexity
[x] The documentation is generated using src-docs
[x] The documentation for charmhub is updated.
[x] The PR is tagged with appropriate label (urgent, trivial, complex)

Ok.


fixes for 12.1 windows compiler

had to make these changes to get visual studio to compile deepspeed as well as some changes to the torch cpp_extension.py (which didn't include c10 lib in the linker args)

@microsoft-github-policy-service agree

@microsoft-github-policy-service agree


correct examples to use valid js

I'm not familiar enough with JS to judge the changes myself but I was being told they make sense so I'm all for it 👍