Possibility to block youtube shorts

Newsboat version (copy the output of newsboat -v or the first line of git show):
Newsboat 2.29.0
System: Linux 5.17.0-1019-oem (x86_64)
Compiler: g++ 11.2.0
ncurses: ncurses 6.3.20211021 (compiled with 6.3)
libcurl: libcurl/7.85.0 OpenSSL/1.1.1q zlib/1.2.13 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.4 libssh2/1.10.0 nghttp2/1.50.0 librtmp/2.3 (compiled with 7.85.0)
SQLite: 3.39.4 (compiled with 3.39.4)
libxml2: compiled with 2.10.3

Is it possible to block youtube shorts or filter out from RSS feeds  video with certain defined length (such as less than 2 minutes)?
May be its possible to have a special treatment of youtube video feeds? youtube-dl could be used to extract the duration of the video and that can be displayed in the feed. This can be helpful to remove the shorts. May be this can be done while loading the feed  and the output of the youtube-dl can be stored in the cache of newsboat?

If you're using the original YouTube feeds, then you're out of luck, since the feeds do not contain the video lengths. I create my own feeds based on the original ones. Coincidentally, I've written an article last week on how that works in a bit more detail. It's a manual process though, but it could be automated to drop videos from the rewritten feed automatically if they are too short.

It is not always the case, but many Youtube channel feeds with shorts have the text "#shorts" in the description.
You could maybe filter them using the newsboat filter: functions.
https://newsboat.org/releases/2.21/docs/newsboat.html#_scripts_and_filters_snownews_extensions
Hope this helps,

I'd prefer not to put site-specific hacks into Newsboat. Please use filter: for this. Lyse's article has some useful bits, like using a cache to avoid fetching the durations over and over again. Finally, if you do create such a filter, consider submitting a pull request to add it to the contrib/ directory so other users can use it too.


Fix guest update

Update to #70
Pending CodeCov to see if additional testing is required

Codecov Report

Merging #71 (5ef207e) into test-checkin-checkout (79e2159) will increase coverage by 5.92%.
The diff coverage is 84.23%.

@@                     Coverage Diff                     @@
##             test-checkin-checkout      #71      +/-   ##
===========================================================
+ Coverage                    65.99%   71.92%   +5.92%
- Complexity                     411      500      +89
===========================================================
  Files                           77       85       +8
  Lines                         1394     1535     +141
  Branches                       138      170      +32
===========================================================
+ Hits                           920     1104     +184
+ Misses                         438      388      -50
- Partials                        36       43       +7

Impacted Files
Coverage Δ

src/main/java/seedu/address/MainApp.java
0.00% <ø> (ø)

...main/java/seedu/address/commons/core/Messages.java
0.00% <ø> (ø)

...rc/main/java/seedu/address/logic/LogicManager.java
71.42% <0.00%> (-3.58%)
:arrow_down:

src/main/java/seedu/address/model/Model.java
100.00% <ø> (ø)

.../java/seedu/address/model/util/SampleDataUtil.java
18.75% <0.00%> (-1.25%)
:arrow_down:

...in/java/seedu/address/storage/JsonAdaptedRoom.java
0.00% <0.00%> (ø)

src/main/java/seedu/address/ui/MainWindow.java
0.00% <0.00%> (ø)

src/main/java/seedu/address/ui/RoomCard.java
0.00% <0.00%> (ø)

src/main/java/seedu/address/ui/RoomListPanel.java
0.00% <ø> (ø)

src/main/java/seedu/address/model/AddressBook.java
72.72% <33.33%> (-9.10%)
:arrow_down:

... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79e2159...5ef207e. Read the comment docs.


Improve publishing setup

This PR contains changes made to update and improve android library template example with removing the buildSrc groovy based setup and replacing it with env var based sonatype config setup.
At first I wanted to create a separate repo just for this type of setup (so you can decide which one to use for yourself) but decided against it so we can discuss the pros and cons of each approach. So far I haven't encountered a con with this one(used on JsonApiX) and I do obviously prefer it.
Also, I took the liberty to enrich the example with multi module based setup so it's easier for people to understand the publishing feature from a more "real world scenario" example.
Note*
If we decide to go with this approach, then the handbook would need to be updated/tweaked as well.

@antunflas @schalke95 I would like to start with implementing @JonatanPlesko suggestions so would like to check if you're gonna have a look before I do so?

@antunflas @JonatanPlesko I've also added a sample publish.properties file and added a comment explaining the need based on Option 3 described in maven.gradle file. Also it's ignored so you will not be able to see it here.

@antunflas @JonatanPlesko I've also added a sample publish.properties file and added a comment explaining the need based on Option 3 described in maven.gradle file. Also it's ignored so you will not be able to see it here.

If it's ignored, where exactly are we supposed to see it? 🤔

Approved, but I believe this should be merged together with #3 when that one is ready.

@JonatanPlesko #3 Is merged, I would say this one is now ready to be merged into main.

Sure 👍


Address many long-standing LGTM complaints

References

Fixes #8320

Description
Short summary of changes (1-2 sentences).
Instructions for Reviewers
Please add a more detailed description of the changes made by your PR. At a minimum, providing a bulleted list of changes in your PR is helpful to reviewers.
List of changes in this PR:

First, ...
Second, ...

Include guidance for how to test or review your PR. This may include: steps to reproduce a bug, screenshots or description of a new feature, or reasons behind specific changes.
Checklist

[ ] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & integration tests). Exceptions may be made if previously agreed upon.
[ ] My PR passes Checkstyle validation based on the Code Style Guide.
[ ] My PR includes Javadoc for all new (or modified) public methods and classes. It also includes Javadoc for large or complex private methods.
[ ] My PR passes all tests and includes new/updated Unit or Integration Tests based on the Code Testing Guide.
[ ] If my PR includes new, third-party dependencies (in any pom.xml), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
[x] If my PR modifies the REST API, I've linked to the REST Contract page (or open PR) related to this change.

@mwoodiupui : Wow! 101 alerts already fixed.   Just wanted to note that, rather than trying to strive for "perfection", you might want to consider what is a "reasonable stopping point" for this PR.  I'd hate for this to wait for too long, or wait so long that it grows and grows (in terms of lines of code changed).
I'll leave it up to you when you want to move this out of "draft" status. Just wanted to note that we may not want this to turn into a several thousand line PR, as the larger it gets, the more complex the review/testing process may be.


Change event?

Is it possible to add a change event? So when a visitor uses a filter there should be a change event triggered? I want to give the visitor some visual feedback, like a loading button or so. The search is now too fast, visitors don't notice changes :P
var onChangeHandler = function() {};
search.on('change', onChangeHandler);

You could use the searchFunction for that:
var search = instantsearch({
  searchFunction(helper) {
    // do something like showing a loading thing
    // when ready, use
    helper.search();
  }
});

instantsearch.on('render', function() {
  // remove the loading indicator
});

Although we do not recommend this, maybe show a very small fake progression bar showing everything is allright to the user if he needs some progress.
It also depends if those users were used to your old possibly slow search, those may be a bit lost but new users won't notice.


[VL] Use Velox's monolithic build

What changes were proposed in this pull request?
Monolithic build was introduced into velox by https://github.com/facebookincubator/velox/pull/9948. This pr enables it. With it, linking velox libs in Gluten becomes less complex and easier to maintain.
How was this patch tested?
CI.

cool feature related to build!

@zhztheplayer, @xumingming, do you have any comments?
I checked the overall size of all separate velox libs when monolithic build is off. It roughly equals to the size of libvelox.a.


Outdated collectable coordinates

recently, several collectables were moved, including lover's laurel. addon hasn't been updated to account and still points at the old locations.

You can fix this by just going to the right nodes and either pressing the snapshot button in the interface or just generally having recording on.
You could also record the correct locations and post them here, or just research them and post them, so I can change the location in the plugin itself for the next version, and don't have to do it myself :D

due to my memory, I wouldnt be able to record and post 'em here. I forget too easily. I've tried snapshot, and sometimes it doesnt detect anything despite me standing on the node with it targeted. is there some form of printout for recorded locations?

It doesn't add anything because it was already recorded (I hope). I have
the lover's laurel thingies done myself now anyway, but the next
GatherBuddy update is a big one so I'm not yet ready to release.

it wasnt recorded, it took a few tries to get it to record


Expand RackspaceLBDriver attributes, add error page/access list getters

This patch adds a number of new attributes to the Rackspace LB Driver's "extra" dict, and two new extended actions for getting the load balancer access list and error page.

Weighted algorithms
New LB states (error and deleted) + translation into Libcloud "Pending" for PENDING_UPDATE/PENDING_DELETE Rackspace states
Health Monitors (Connect, HTTP, HTTPS)
Connection Throttle
Session Persistence Type
Connection Logging
Error page
Access list

Thanks for the feedback, I'll update this with the suggested changes.

Thanks for the feedback so far.  This should address the remaining comments.

Merged in r1213364 (https://svn.apache.org/viewvc?view=revision&revision=1213364).
Thanks!


Large merge transactions hang

For details see https://github.com/nigelsmall/py2neo/issues/484

With py2neo V3 B2 and neo4J 3.0.0 RC1, we are getting less hangs with Bolt but it still hangs on occasion.  If we disable Bolt transactions, the script runs to completion without complaint.  You can find our test script @ http://urban-warrior.org/neo-ingestion-rate.py.  You may need to run it multiple times before it hangs.  Note, it calls graph.delete_all() so be sure to remove these statements otherwise it will wipe your existing database!   When it hangs our stack trace shows its hanging during the commit:
python3 neo-ingestion-rate.py
Number of batches: 8
Entity transactions per batch: 6144
Merge entities: 2016-04-22 13:24:18.449533
^CTraceback (most recent call last):
  File "neo-ingestion-rate.py", line 143, in <module>
    transaction.commit()
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 999, in commit
    self._post(commit=True)
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 1228, in _post
    self.finish()
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 1233, in finish
    self._sync()
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 1223, in _sync
    connection.fetch()
  File "/usr/lib/python3.4/site-packages/py2neo/packages/neo4j/v1/connection.py", line 292, in fetch
    raw.writelines(self.channel.chunk_reader())
  File "/usr/lib/python3.4/site-packages/py2neo/packages/neo4j/v1/connection.py", line 167, in chunk_reader
    chunk_header = self._recv(2)
  File "/usr/lib/python3.4/site-packages/py2neo/packages/neo4j/v1/connection.py", line 145, in _recv
    b = self.socket.recv(8192)
  File "/usr/lib64/python3.4/ssl.py", line 736, in recv
    return self.read(buflen)
  File "/usr/lib64/python3.4/ssl.py", line 625, in read
    v = self._sslobj.read(len or 1024)
KeyboardInterrupt

Bolt transactions work properly now with the released version of Neo4J 3.0.0.   No more hangs.

Closing for now as problem has not been seen in final release. Can be reopened if this changes.

We're seeing hangs with the Bolt protocol.  It consistently happens with Neo4J 3.0.3 and py2neo 3 but only after loading several million nodes and relationships.  When it does hang, other Neo4J activities can continue such as backups or browser queries.  There is nothing referencing the hang in the Neo4J logs.  If we start another script, it hangs when we call Graph().  It does not timeout, it just sits there for hours until we kill the script or restart Neo4J.    In the mean-time we switched to a HTTP connection and so far we have not seen any hangs.   Perhaps the problem is in Neo4J instead of py2neo.

@mikayla-grace can you provide a thread-dump with kill -3 <pid> or jstack `

Unfortunately it takes up to 2 days before the hang occurs and we can't afford any more downtime populating our graph database.  However, we do have a smaller host with less memory.  We'll see if we can reproduce the problem there without running out of memory and get a stack dump.  We'll post here if we can reproduce the Bolt protocol hang.
In the mean-time, since we switched to HTTP protocol, most of our ingestion scripts appear to be working but one is failing with a HTTP exception.  We have now increased  py2neo http.socket_timeout hoping that will fix the problem.
We also received yet another exception just now since we switched to the HTTP protocol:
File "/usr/lib/python3.5/site-packages/py2neo/database/__init__.py", line 1152, in _post
raise GraphError.hydrate(raw_error)
py2neo.database.status.DatabaseError: File has been unmapped: /<path>/data/databases/graph.db/neostore.counts.db.a

For that last error, could you check if there is a stack trace in logs/debug.log or logs/neo4j.log with more details on that error?
Also, as an alternative to providing thread dumps, could you give a bit more detail on what you are doing, and we can try and replicate it on our end? For example:

What do the queries look like
Maybe a small python code sample of what you do in the client
How many times do you run the queries before hitting issues
What does uname -r say
What python version, and what specific py2neo version?

And so on; anything you think could be helpful to try and replicate it.

We're ingesting data from a variety of public sources using a Python class we wrote.  We're using Python 3.5 and Neo4J 3.0.3 Enterprise and py2neo 3.0.0, we just upgraded to py2neo 3.1.0.  We need to ingest around 25 million records and the scripts have failed repeatedly with either a hang with Bolt, or we get a 404 for HTTP, or we get a graph.hydrate exception with HTTP.
The failures are during the ingestion process.  We have not queried much because we have been unable to create a stable graph database.  We use py2neo merge() with transactions.  At first we thought the transactions might be too big so we cut back from 1000 transactions before a commit to 64 now.
We typically ingest nearly 2.5 million records from 4 different scripts ingesting from 4 different public data sources.  With Bolt, the scripts typically hangs in the same or nearly the same place usually around 20 hours or so after the ingestion starts.    With HTTP, one or more scripts will typically fail with a graph.hydrate or HTTP 404 exception usually within a few hours.
The code is pretty straightforward:
    node=Node(label)
    <add a few properties to the Node>
    <check the transaction limit, if more than 64, commit>
    self.transaction.merge(node,label,{ key: value})

Here is our kernel: 4.5.7-200.fc23.x86_64+debug.
We have a 1.4MB log file.  One exception is:
2016-06-29 18:24:30.755+0000 ERROR [o.n.b.v.r.i.ErrorReporter] Client triggered an unexpected error [UnknownError]: null. See debug.log for more details, reference f9f73c55-1502-4ed1-b644-9185eff49375.
2016-06-29 18:24:30.756+0000 ERROR [o.n.b.v.r.i.ErrorReporter] Client triggered an unexpected error [UnknownError]: null, reference f9f73c55-1502-4ed1-b644-9185eff49375.
java.lang.NullPointerException
        at org.neo4j.bolt.v1.transport.ChunkedOutput.ensure(ChunkedOutput.java:156)
        at org.neo4j.bolt.v1.transport.ChunkedOutput.writeShort(ChunkedOutput.java:90)
        at org.neo4j.bolt.v1.packstream.PackStream$Packer.packStructHeader(PackStream.java:304)
        at org.neo4j.bolt.v1.messaging.PackStreamMessageFormatV1$Writer.handleRecordMessage(PackStreamMessageFormatV1.java:132)
        at org.neo4j.bolt.v1.messaging.msgprocess.RecordStreamCallback$1.visit(RecordStreamCallback.java:47)
        at org.neo4j.bolt.v1.runtime.internal.CypherAdapterStream.lambda$accept$9(CypherAdapterStream.java:68)
        at org.neo4j.cypher.internal.compatibility.ExecutionResultWrapperFor3_0$$anon$5.visit(CompatibilityFor3_0.scala:410)
        at org.neo4j.cypher.internal.compiler.v3_0.helpers.RuntimeJavaValueConverter$feedIteratorToVisitable.accept(RuntimeJavaValueConverter.scala:54)
        at org.neo4j.cypher.internal.compiler.v3_0.PipeExecutionResult.accept(PipeExecutionResult.scala:109)

We also got a ' Discarded stale query from the query cache' perhaps expected after a log rotation.
We could make the scripts and the debug logs available privately to you if you prefer.  If so, send a private email to our temporary e-mail address at neophite at virginimage dot org.

Can you share the concrete code?
I'm not sure what py2neo's merge does under the hood.
I'd go with a plain unwind + merge statement (I presume you have the necessary constraint(s) in place).
Something like: (pass a list of size 10k to 100k  of dictionaries as "rows" parameter))
UNWIND {rows} as row
    MERGE (n:Label {id: row.id}) ON CREATE SET n += row

you can also spell out the individual properties in the SET clause

A subset of the concrete code is:
    self.graph=Graph(graphURL,user=graphUser,password=graphPassphrase,bolt=False)
    self.transaction=self.graph.begin()
    ...
  def mergeEntity(self,label,key,value,labels,**properties):
    '''
      Merge an entity graph node;  label is required, labels are optional.
    '''
    if value == None or value == '': return
    node=Node(label)
    if labels != None:
      if not isinstance(labels,list):
        node.add_label(labels)
      else:
        node.update_labels(labels)
    if "name" not in properties:
      node["name"]=value
    for _property in properties:
      node[_property]=properties[_property]
    node[key]=value
    name=label+'-'+key+'-'+value
    node['type']=self.slugify(label)
    node['id']=self.slugify(value)
    node['nameHash']=self.slugify(node["name"])
    if node["uuid"]!= False:
      node['uuid']=str(uuid.uuid5(uuid.NAMESPACE_DNS,name))
    node["source"]=self.source
    node['importDate']=datetime.datetime.utcnow().isoformat()+'Z'
    self.checkTransactionLimits('update','entity',name,node)
    self.transaction.merge(node,label,{ key: value})
    return(node)
...
  self.transaction.commit()

A client looks like this:
  neoPhite.mergeEntity('Article',"id",_id,subType,
    url='http://mysite.com/'+entity['id'],
    name=neoPhite.canonicalProperty(entity,'title'),
    description=neoPhite.canonicalProperty(entity,'text'),
    title=neoPhite.canonicalProperty(entity,'title''),
   ...

Each script might have 10 to 40 different properties.
We're fairly invested in py2neo.  Our scripts were developed on a smaller set of data and it always seemed to work.  It only started to fail when we scaled.   Rather than completely rewriting our 20 plus scripts, maybe it would work if we reduced our dataset to batches of 100 records or so and run them as separate script instances.  If all else fails, we might revert to Neo4J version 2 series with fingers crossed that that will work.

We're now having some success with the py2neo 3.1 HTTP protocol ingesting large amounts of data into Neo4J 3.0.3.  Graph() does not accept a timeout parameter, it probably should.  Instead we set http.socket_timeout to a large value and our ingestion scripts have not thrown any exceptions.  If they do, we'll let you know.  Some of our scripts run for several days and it seems likely a socket timeout may throw spurious exceptions making it difficult to track.  Perhaps the Bolt transport is timing out as well on long runs.  However, we cannot find a way to set a timeout for Bolt.  Bolt works for us for any script that runs for less than 20 hours or so.  It hangs when a script runs for several days.

Thanks for the detailed report. I'm going to be looking at timeout configuration across Bolt and all the drivers over the next few weeks since this issue has surfaced a few times in different places. Stay tuned for some improvements here.

Nigel, looking forward toward a patch to this problem.  In the mean-time, my scripts returned another hydrate exception.  Not sure its related:
Traceback (most recent call last):
  File "/knovo/knovo/Fusion/KnovoGraph.py", line 240, in commitTransaction
    self.transaction.commit()
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 1022, in commit
    self._post(commit=True)
  File "/usr/lib/python3.4/site-packages/py2neo/database/__init__.py", line 1193, in _post
    raise GraphError.hydrate(raw_error)
py2neo.database.status.TransientError: ForsetiClient[5] can't acquire ExclusiveLock{owner=ForsetiClient[3]} on NODE(1552), because holders of that lock are waiting for ForsetiClient[5].
 Wait list:ExclusiveLock[
Client[3] waits for [5]]

Perhaps we just need to retry when this exception is thrown.

We thought our HTTP troubles were behind us when we set http.socket_timeout to 2592000 but our script just failed with a 404:
Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/py2neo/database/http.py", line 203, in post
    response = self.__base.post(body, headers, **kwargs)
  File "/usr/lib/python3.4/site-packages/py2neo/packages/httpstream/http.py", line 984, in post
    return rq.submit(**kwargs)
  File "/usr/lib/python3.4/site-packages/py2neo/packages/httpstream/http.py", line 452, in submit
    return Response.wrap(http, uri, self, rs, **response_kwargs)
  File "/usr/lib/python3.4/site-packages/py2neo/packages/httpstream/http.py", line 489, in wrap
    raise inst
py2neo.packages.httpstream.http.ClientError: 404 Not Found

@nigelsmall does this funcitionality use cypher under the hood? I presume it must when it's usable via bolt?

@mikayla-grace it's not so much the number of transactions but the number of entities per transaction. How much memory (heap) is your server configured with (neo4j-wrapper.conf)?

We use the default configuration which as you know is dynamically calculated based on available system resources.  We have a dedicated 32GB 12 core Linux host (only the OS and neo4J runs on the system) so one would think that 64 transactions before a commit is sufficiently small enough to not cause heap problems.  However, if you have a recommendation on heap size or transaction size, let us know and we'll try our scripts with your recommendations.

There is only automatic computation for the page-cache (i.e. 50% of total RAM).
JVM heap is allocated by java if not configured, but  it usually only takes a small amount of the RAM as heap. So I presume the heap is just too small.
You can check the actual size of the heap and page cache in debug.log

Here is what we found in debug.log:
 System memory information:
 Total Physical memory: 31.27 GB
 Free Physical memory: 188.66 MB
 Committed virtual memory: 11.99 GB
 Total swap space: 15.69 GB
 Free swap space: 15.16 GB
 JVM memory information:
 Free  memory: 390.97 MB
 Total memory: 502.00 MB
 Max   memory: 7.82 GB
 Garbage Collector: G1 Young Generation: [G1 Eden Space, G1 Survivor Space]
 Garbage Collector: G1 Old Generation: [G1 Eden Space, G1 Survivor Space, G1 Old Gen]
 Memory Pool: Code Cache (Non-heap memory): committed=4.44 MB, used=4.41 MB, max=240.00 MB, threshold=0.00 B
 Memory Pool: Metaspace (Non-heap memory): committed=13.25 MB, used=12.55 MB, max=-1.00 B, threshold=0.00 B
 Memory Pool: Compressed Class Space (Non-heap memory): committed=1.75 MB, used=1.55 MB, max=1.00 GB, threshold=0.00 B
 Memory Pool: G1 Eden Space (Heap memory): committed=34.00 MB, used=14.00 MB, max=-1.00 B, threshold=?
 Memory Pool: G1 Survivor Space (Heap memory): committed=4.00 MB, used=4.00 MB, max=-1.00 B, threshold=?
 Memory Pool: G1 Old Gen (Heap memory): committed=464.00 MB, used=92.40 MB, max=7.82 GB, threshold=0.00 B
 Operating system information:
 Operating System: Linux; version: 4.5.6-200.fc23.x86_64+debug; arch: amd64; cpus: 12
 Max number of file descriptors: 327680
 Number of open file descriptors: 101
 Byte order: LITTLE_ENDIAN
 Local timezone: America/New_York
 JVM information:
 VM Name: OpenJDK 64-Bit Server VM
 VM Vendor: Oracle Corporation
 VM Version: 25.92-b14
 JIT compiler: HotSpot 64-Bit Tiered Compilers
 VM Arguments: [-XX:+UseG1GC, -XX:-OmitStackTraceInFastThrow, -XX:hashCode=5, -XX:+AlwaysPreTouch, -XX:+UnlockExperimentalVMOptions, -XX:+TrustFinalNonStaticFields, -XX:+DisableExplicitGC, -Dunsupported.dbms.udc.source=tarball, -Dfile.encoding=UTF-8]
 Java classpath:
  [bootstrap] /usr/lib/jvm/java-1.8.0-openjdk-<IP_ADDRESS>-1.b14.fc23.x86_64/jre/lib/jsse.jar
  [bootstrap] /usr/lib/jvm/java-1.8.0-openjdk-<IP_ADDRESS>-1.b14.fc23.x86_64/jre/lib/charsets.jar

Recommendations for configuring memory is welcome.  However, would memory problems cause the Bolt protocol to hang, exclusive lock exceptions, or HTTP post status of 404?

@mikayla-grace , did you figure this out?
I also have a script which is failing with
java.lang.IllegalStateException: too much data after closed for HttpChannelOverHttp

did you figure this out?

We continue to have numerous problems with Neo4J enterprise 3.0.4 and py2neo 3.1.  Just today we got another deadlock during the ingestion.  We had to restart Neo4J and restart our ingestion scripts to recover.   We also have slow query responses.  As previously mentioned, we can ingest 20 million articles into Solr within 2 days.  With Neo4J and py2neo, we project it will take close to a year.  We've been ingesting for months now and we're up to 4 million articles.  It seems that ingestion is slowing down as the number of articles increases.  We keep hoping that as each new release of Neo4J and py2neo are posted, that these problems will disappear.  If that does not happen soon, we will likely need to change to a different graph database.

Could you report in detail what you're doing, share you code, config an setup.
Also raise a support ticket for your enterprise license. Neo4j can easily ingest 20m nodes within a few minutes.
Von meinem iPhone gesendet

Am 03.09.2016 um 00:54 schrieb mikayla-grace<EMAIL_ADDRESS>did you figure this out?
We continue to have numerous problems with Neo4J enterprise 3.0.4 and py2neo 3.1. Just today we got another deadlock during the ingestion. We had to restart Neo4J and restart our ingestion scripts to recover. We also have slow query responses. As previously mentioned, we can ingest 20 million articles into Solr within 2 days. With Neo4J and py2neo, we project it will take close to a year. We've been ingesting for months now and we're up to 4 million articles. It seems that ingestion is slowing down as the number of articles increases. We keep hoping that as each new release of Neo4J and py2neo are posted, that these problems will disappear. If that does not happen soon, we will likely need to change to a different graph database.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.

Also raise a support ticket for your enterprise license.

We have a start-up enterprise license.  Not sure support is covered under that license since it was free.

Could you report in detail what you're doing, share your code, config an setup

We're using a Python class and an ingestion script to load PubMed citations into Neo4j with py2neo on a single 8 core Linux host with 32GB of memory and 6GB Sata.  We're ingesting about 6 thousand citations an hour.  Given we have 25 million citations to load,  it will take many months of continuous processing to finish.
In the past, while ingesting the citations, we've had Neo4J hang on multiple occasions.  The only way to get it working again was to restart Neo4J and then restart the ingestion scripts.  We switched from Bolt to HTTP and the hang mostly went away, although we still get them on occasion.  With HTTP we get hydrate errors due to deadlocks.  We wait 30 seconds and try the transaction commit again but it never works-- another exception is thrown.
You can review our Python scripts and reproduce the slow ingestion rate and maybe the deadlocks we're been getting:  http://urban-warrior.org/citations.tar.gz.

2016-06-30 22:31:09.676+0000 WARN  badMessage: java.lang.IllegalStateException: too much data after closed for HttpChannelOverHttp@185b8e50{r=1,c=false,a=IDLE,uri=-}

I have the same issue in two different setups.
First setup :
Neo4j php client ingesting data under heavy load, sometimes 300-400 transactions per second.
It happens only when two requests in a row returns a ConstraintViolation exception.
Second setup, java process running http requests to an unmanaged extension

can you share more information about your data model
and the schema (indexes and constraints you have in place)
Thanks
On Sun, Sep 4, 2016 at 10:24 PM, mikayla-grace<EMAIL_ADDRESS>wrote:

Also raise a support ticket for your enterprise license.
We have a start-up enterprise license. Not sure support is covered under
that license since it was free.
Could you report in detail what you're doing, share your code, config an
setup
We're using a Python class and an ingestion script to load PubMed
citations into Neo4j with py2neo on a single 8 core Linux host with 32GB of
memory and 6GB Sata. We're ingesting about 6 thousand citations an hour.
Given we have 25 million citations to load, it will take many months of
continuous processing to finish.
In the past, while ingesting the citations, we've had Neo4J hang on
multiple occasions. The only way to get it working again was to restart
Neo4J and then restart the ingestion scripts. We switched from Bolt to HTTP
and the hang mostly went away, although we still get them on occasion. With
HTTP we get hydrate errors due to deadlocks. We wait 30 seconds and try the
transaction commit again but it never works-- another exception is thrown.
You can review our Python scripts and reproduce the slow ingestion rate
and maybe the deadlocks we're been getting: http://urban-warrior.org/
citations.tar.gz.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/neo4j/neo4j-python-driver/issues/47#issuecomment-244626572,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEHY1qikvYGvXQlhnIhacJQUeD7SRbcks5qmykPgaJpZM4H9_u8
.

While ingesting, we just got another exception: py2neo.database.status.TransientError: ForsetiClient[1] can't acquire ExclusiveLock{owner=ForsetiClient[2]} on NODE(2335), because holders of that lock are waiting for ForsetiClient[1].  We tried to commit again after 30 seconds and the second commit failed as well.
The data model revolves around citations which we call articles.  The article ID is our only unique constraint.  Each article has an ID, name, description, source, publication date, etc.  In addition we have other nodes that each article might be associated with.  They include author, grant, chemicals, and journals.  We first create the articles, authors, grants, chemicals and journal nodes.  Next we re-read the citations and create relationships from the articles to the associated authors, grants, chemicals, and journals.  We use merge() instead of create() because we so often need to restart the ingestion and this permits us to initially ingest the nodes as well as update them later using the same class.  We did try create() at some point and did not get the performance increase we were expecting.

But if you don't have any other constraints / indexes, it has to do a full
graph/label scan e.g. to find a single author, grant, chemical or journal
node, and that for each relationship that you're trying to create?
Please make sure that your query is using indexes for all node lookups with
EXPLAIN.
On Sun, Sep 4, 2016 at 11:41 PM, mikayla-grace<EMAIL_ADDRESS>wrote:

While ingesting, we just got another exception: py2neo.database.status.TransientError:
ForsetiClient[1] can't acquire ExclusiveLock{owner=ForsetiClient[2]} on
NODE(2335), because holders of that lock are waiting for ForsetiClient[1].
We tried to commit again after 30 seconds and the second commit failed as
well.
The data model revolves around citations which we call articles. The
article ID is our only unique constraint. Each article has an ID, name,
description, source, publication date, etc. In addition we have other nodes
that each article might be associated with. They include author, grant,
chemicals, and journals. We first create the articles, authors, grants,
chemicals and journal nodes. Next we re-read the citations and create
relationships from the articles to the associated authors, grants,
chemicals, and journals. We use merge() instead of create() because we so
often need to restart the ingestion and this permits us to initially ingest
the nodes as well as update them later using the same class. We did try
create() at some point and did not get the performance increase we were
expecting.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/neo4j/neo4j-python-driver/issues/47#issuecomment-244630397,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEHY3yq9w3IlI86s3386bhWiNzQLS9xks5qmzsAgaJpZM4H9_u8
.

it has to do a full graph/label scan

Kudos.  That made a significant difference.  That increased performance from 6,000 articles an hour to 75,000  (projected).  Let's see if that gets rid of the periodic Neo4J hangs.  Apologies for not spotting that problem ourselves.    We're benchmarking 300,000 articles now.  As soon as it completes, we'll  post the ingestion rate here.

Good luck.
How many relationships are created per article?
With a heap size of 6-8G you can create 1M relationships per transaction.

How many relationships are created per article?

It varies of course but its likely no one article would have more than one hundred relationships.  Will have better metrics as soon as the benchmark completes.

@mikayla-grace maybe you should consider to make custom cypher queries for importing your data. With Cypher I have a write rate of approx 6k nodes, 27k rels per second (standard benchmark script I use)

We thought of that-- custom cypher queries, however, we use the same class for about 20 different public datasets we ingest with additional disparate datasets in the works.    We can also use this single elegant object-oriented Python class whenever we need to update already ingested nodes.   We have custom cypher queries on the back burner if all else fails.  Thanks.

@jexp.  That did it.  We ingested 750,000 nodes and relationships in 1 hour without any exceptions or hangs.  Apologies for not detecting this flaw earlier.  It is likely that all the exceptions and hangs we encountered in the past were related to searching un-indexed fields.  It also explains why our ingestion process got slower as we ingested more and more data.   If I may, we'll use the start-up card.  Always too much to do-- always too little time to do things right.

Great it's resolved. Good luck with the next steps. I'll close the issue then.


Fix sorting

Versions
0.7.0
What did you do?
Added compat sort test TestQueryCompatSort
https://github.com/FerretDB/FerretDB/blob/5920ba1b88ac40135c40b7b2024ae66b70bbeecc/integration/query_compat_test.go#L153
What did you expect to see?
The sorting returns the same as compat.
What did you see instead?
The sort order for array for composite is not the same. It's because array sort order should check the elements within the array for sorting. https://www.mongodb.com/docs/manual/reference/bson-type-comparison-order/#arrays
https://github.com/FerretDB/FerretDB/blob/05950cb4ffb0604e96d51312747faad6afac143c/integration/query_compat_test.go#L154
        --- FAIL: TestQueryCompatSort/Asc/TestQueryCompatSort_Composites (0.01s)
            query_compat_test.go:168: Compat (expected) IDs: [array-empty array-composite array-null array-three array-three-reverse array array-numbers-asc array-two array-strings-desc document-empty document-null array-documents document document-composite document-composite-reverse]
            query_compat_test.go:168: Target (actual)   IDs: [document-empty document-null document document-composite document-composite-reverse array-empty array-null array-three-reverse array array-numbers-asc array-three array-two array-composite array-strings-desc array-documents]
            query_compat_test.go:168:

https://github.com/FerretDB/FerretDB/blob/05950cb4ffb0604e96d51312747faad6afac143c/integration/update_field_compat_test.go#L121
        	            	--- expected
        	            	+++ actual
        	            	@@ -3,8 +3,8 @@
        	            	     "_id",
        	            	-    "a",
        	            	-    "v"
        	            	+    "v",
        	            	+    "a"
        	            	   ],
        	            	   "_id": "unset",
        	            	-  "a": 30,
        	            	-  "v": 39
        	            	+  "v": 39,
        	            	+  "a": 30
        	            	 }

Part of this is fixed in #1634 and remaining maybe fixed in #1652.

Fixed by #1652


code_ptr validity in ElfDisassembler::disassembleSectionUsingSymbols

There appear to be some assumptions in ElfDisassembler::disassembleSectionUsingSymbols, and these are:
a) the first .text symbol in the ELF symbol table originates at the same address as the start of the .text section
b) the .text symbols are contiguous throughout the .text section
c) none of the .text symbols have addresses with the least significant bit (indicating THUMB mode)
d) the .text symbols are encoded into the ELF in address order
e) all of the .text symbol addresses are within the address range of the section
(a) through (d) are very possible, if not likely.  (e) is unlikely, but was remedied with an additional three lines of code.
What happens in the existing code is that code_ptr is set to the origin of the .text section.  However, if the first symbol is not the same address as the origin, the calls to the Capstone Engine library pass data from the beginning of the section instead of the address of the symbol.  (There is some code that tries to fix the code_ptr in such a scenario, but it only works if the proceeding data is marked with symbol(s) that are indicated to be data.)
Even if the first symbol is the same address as the origin, if there are discontinuities between the end of one symbol and the beginning of the next, the same sort of misalignment occurs.
The end result is that spedi tries to decode data as instructions, and this doesn't go well.
The second commit addresses a segfault caused by PLTProcedureMap.  It invokes MCParser, but that class's deconstructor assumes its initialize() member function has already been called to open the Captone Engine library.

Happy to accept commit 2 as it is. I have few comments about commit 1 though. Actually, my main assumption is that we can map every address in .text to one, and exactly one, code mapping symbol. Based on that, I collect the code symbols and sort them here. Let me now address your points.

a) the first .text symbol in the ELF symbol table originates at the same address as the start of the .text section

Can a single code symbol span two adjacent sections? for example, .plt and .text? if not, then this assumption should hold.

b) the .text symbols are contiguous throughout the .text section

assuming that code symbols are complete (main assumption) and sorted then this should follow.

c) none of the .text symbols have addresses with the least significant bit (indicating THUMB mode)

THUMB instructions are 2 byte aligned and can not be found on odd addresses. Odd addresses are only used to signal mode changing in an instruction like bx. I would be a bit surprised to find a code symbol with an odd address.

d) the .text symbols are encoded into the ELF in address order

they are explicitly sorted.

e) all of the .text symbol addresses are within the address range of the section

yes, because I collect code symbols per individual section here
That said, disassembly by adjusting offsets, as you have implemented it, is certainly more robust. It works with less assumptions than what we already have. I'm only curious about the code mapping symbols you have encountered in the wild.

I saw the goal of spedi to be recovery of information about the firmware execution structure without the aid of prior knowledge of the code.
At the moment, if the ELF is stripped of its debug information (such as with "strip"), spedi reports that there isn't a symbol table and gives up.  (Note that the entry address encoded into the ELF ought to be treated like a symbol, but that is not addressed in these two commits; however, this should allow a stripped ELF to work.)
I agree that a symbol should not span a section, but I would argue that it is not a given that every ELF will have every byte of a section documented with debugging information.  It is not an ELF requirement.  The tool creating the ELF can create as little or as much information as it wants.
Certainly, objdump will disassemble without any symbol tables.  However, for mixed THUMB/ARM, it seems to need to be given a "$t" or "$a" symbol for the entry address to be reasonably successful.  (Side note: I have an armbin2elf tool available on my page that I've been using to encapsulate THUMB binaries for analysis by objdump.)
Regarding bit 0, when I do a "readelf -a" of Cortex-M firmware ELF images (created by industry tools), the THUMB symbols have bit 0 set.  (The same convention is used for the entry address.) This is why I added code to zero-ize bit 0 of the address.  However, perhaps some C++ library is already zero-izing that bit?
The address bound checking that I included is not something that I feel strongly about and wouldn't be concerned if it were removed.  It was added out of (excessive) cautiousness.
Of the changes, the update of code_ptr on a per symbol basis is, I believe, the most important one.  It is the only way to ensure operation when not every byte in the section has debugging information.

pruning ancient PRs


pull request for e2e test at 11/30/2016 07:56:04

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Places support and ability to limit what comes up in a search.

Potential to cause problems for some users:
Use of http requests
Having the placeholder only being updated if it is set to the .description
(from UPDATES.md)
Has the options of searching for places now
Requires ion-google-place element to have google-key. (google-key="your key")
Option of search-type in ion-google-place element (geocode, address, establishment, (cities), or (regions))
Placeholder now must be the ng-model dot description to be displayed. (if ng-model is address, placeholder ={{address.description}})

Hey Pete, this looks very promising, thanks a lot. I tried implementing it and the http requests are giving me CORS errors. Any idea how to avoid that? Thanks!

Either cross origin resource sharing has to be enabled or you can use the server-side-code-required branch of my fork instead. I recommend the second option. If you want to improve the first option, use the Google Maps JavaScript API instead of the Google Places API Web Service. I missed the note in the documentation that that library is for server usage. I'll try to fix that when I get some free time to do so.

Hmm interesting. I sort of moved on and coded it myself, but I ended up using the places API and didn't have any issues with CORS, and I'm doing it client side. Not sure what the difference is...
Sent from my iPhone

On Mar 27, 2016, at 3:01 PM, Peter Keller<EMAIL_ADDRESS>wrote:
Either cross origin resource sharing has to be enabled or you can use the server-side-code-required branch of my fork instead. I recommend the second option. If you want to improve the first option, use the Google Maps JavaScript API instead of the Google Places API Web Service. I missed the note in the documentation that that library is for server usage. I'll try to fix that when I get some free time to do so.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub


Fix plugin version info

Changes

Properly display the version of the plugin

This probably should be merged into the release-0.1 branch as well.
/kind bug
Fixes #51
Release Note
Properly display the version of the plugin

/lgtm
/approve

/cherry-pick release-0.1


HttpServer

Hello,
you might want to check HttpServer library, that provides
http requests, http responses, cookies, sessions, timers, json, two templating engines, one based on DotLiquid pot to Universal Apps and another based on simple string substitutions, dynamic web pages with template engines, custom URL listeners,
To integrate with your rest api, minimal code would be
HttpServer server = new HttpServer();
server.AddPath("/api", restRouteHandler);
server.start();

There is demo project that shows more functionality.

Hey strehar,
Thanks for the link, from the features you mentioned I think we'd want to support cookies in restup, we already support custom url listeners, http requests, http responses and json.
The other features seem more like they shouldn't be part of a http server implementation but rather part of a framework and would most likely not be implemented in the core bit of restup but maybe as an extension of restup.
I am interested in seeing how you prove that your http server implements the http spec properly, do you use some kind of external testing framework for this? I couldn't discover any kind of tests in your repo.
Cheers,
Jark

Sessions should be part of server too, it's usefull to have them supported out of the box.
You can test server on the REDbot. I check it there, very usefull tool.

Had a look at it, seems quite useful, thanks :)
Closing this issue, since it's not really a problem with restup. Feel free to link to this repo from your repo as a possible integration.
Cheers,
Jark


in detail views choice fields are rendered using dot notation instead of string representation

apis_ontology.Profession.None instead of an empty field

Created acdh-oeaw/apis-core-rdf#331 for that

Closing, as we default to the edit view now


Add unittests for GHRepo reader and fix filter logic.

I was able to mock some parts of the GithubRepoReader. However, I am open to feedback on how to test it. I am not expert on pytest and best practices on how to mock things.
I also discovered that filter logic was not working properly while writing the unittest especially for test_load_data_* and test_filter_* test cases and also somebody mentioned it on discord.
I believe these unit tests should be good starting point for testin the GH repo reader with filter.

seems like there's some merge conflicts (maybe due to revert), could you repull main and fix conflicts?
this is a tricky merge conflict. it doesn't give me chance to select between incoming and current changes for some parts of the code and just overwrites them.


Can gfastats convert fastg to gfa?

Hello - is gfastats able to convert fastg to gfa?

Hi @tarunaaggarwal
Thanks for reaching out. At present it does not. If there are useful use cases and conversion is unambigous we could consider it though.


Logic Apps Swagger fixes

The PR includes logic app swagger fixes and updated examples to match the Logic apps service RP.
Latest improvements:
MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and have all been fixed in this PR.
[ ] The OpenAPI Hub was used for checking validation status and next steps.

ARM API Review Checklist

[ ] Service team MUST add the "WaitForARMFeedback" label if the management plane API changes fall into one of the below categories.
adding/removing APIs.
adding/removing properties.
adding/removing API-version.
adding a new service in Azure.

Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs.

[ ] If you are blocked on ARM review and want to get the PR merged urgently, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.
Please follow the link to find more details on API review process.

You don't have permission to trigger SDK Automation.
Please add yourself to Azure group from opensource portal if you are MSFT employee,
or please ask reviewer to add comment *** /openapibot sdkautomation ***.
Please ask<EMAIL_ADDRESS>(or NullMDR in github) for additional help.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: nidpathak sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Can one of the admins verify this patch?


Could not get WebGPU adapter: LimitsExceeded

https://github.com/eliemichel/LearnWebGPU-Code/blob/f84c28fcc7710013faddede13c3bd225a31b9ce9/main.cpp#LL87C31-L87C31
Hi Élie, thanks for your tutorial on WebGPU! I've been going through the steps, and I found that I get the following error (on macOS 13.3, Xcode 14.3) at runtime:
Requesting adapter...
Got adapter: <wgpu::Adapter 0x7f8f7af06600>
Requesting device...
Could not get WebGPU adapter: LimitsExceeded(FailedLimit { name: "min_uniform_buffer_offset_alignment", requested: 64, allowed: 256 })
Got device: <wgpu::Device 0x0>
adapter.maxVertexAttributes: 31
thread '<unnamed>' panicked at 'invalid DeviceId', src/lib.rs:210:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
fatal runtime error: failed to initiate panic, error 5
[1]    6198 abort      build/Debug/App

When I comment out line 87 deviceDesc.requiredLimits = &requiredLimits;, it runs fine.
Can you help me figure out this issue?

Also encountering this issue on macOS 13.3 using wgpu-native, clang and C. Verified that all of my requiredLimits were within the supported limits.
Looks like this might be an issue with wgpu? I haven't tried Dawn.

Do not copy all adapter limits to the device limit, this defeats the prupose of having a limit management mechanism (and if this is really what you want, then just set the device limits to nullptr in the device descriptor ;) ). If you do so, you open the door to the "it works on my hardware" problem where you cannot reproduce your user's issues.
Forwarding limit does make sense for the 2 of them that start with "min" though, so in this very case:
deviceLimits.minUniformBufferOffsetAlignment = adapterLimits.minUniformBufferOffsetAlignment;

Sorry I forgot it it should indeed be in the example code! I'll leave this issue open to remaind me to do it.

Why does it make sense to forward the two "min*" limits in this case?
Anyway, your suggestion fixed the issue on my end. I added this to line 83:
requiredLimits.limits.minUniformBufferOffsetAlignment = supportedLimits.limits.minUniformBufferOffsetAlignment;
Thanks for your quick reply and your help!

Fixed up to step044, next steps are WIP for many other reasons!


write file question

hello,when i write file,how can i update a node,if the node is set in the file?

Im not sure what you mean. Can you describe your issue with a small code example?

in my toml file
[test]
a="123"
b="456"
i want to change the  a node "123"  to   "789",  then write to the toml file.
how can i change it?

I think what you try to do is:
TomlTable tbl = Toml.ReadFile("myfile.toml");
Assert(tbl.Get<string>("a") == "123");
tbl["a"] = "789"
Toml.WriteFile(tbl, "myfile.toml") // overwrite with modified content

I inspected my API and this currently is not possible in a clean manner. Currently there is no API to change the value of TOML table row (at least when using the generic TomlTable data structure instead of a strongly typed CLR object). A possible workaround would be:
var tbl = Toml.ReadFile("myfile.toml");
var dict = tbl as IDictionary<string, TomlObject>;
dict["a"] = tbl.CreateAttachedValue("123");

or
var tbl = Toml.ReadFile("myfile.toml");
tbl.Remove("a");
tbl.AddValue("a", "123");

But I will definitely add a dedicated update method in the near future.

I don't know if this is doable, but it would be nice if there also was an option to also preserve whitespaces when writing back to the file. Or at least a few options to specify how to format the output
Toml is a pretty nicely human-readable format, but it must be formatted accordingly for that.
Currently formatted toml gets squached together, are out of order and some comments get lost.

API for v0.8.0 got new methods that allow to update table rows (values)
@Splamy This does not belong into this issue and is a large topic on its own (will be my workload for the upcoming months). In fact nearly all of my 0.9.0 backlog items are linked to the topic of formatting the TOML file contents in both directions (read & write). So I'll close this issue here.


Floating utility bar

@msankaran0712 I will discuss with David Silcock how they submit

Need more details.

There are four issues separately and this is double. Terminology clash.
Submission for Floating:

Navbar - Variant
Side Nav - Variant
Button (single button) Variant
Toolbar (container with several buttons, floating). New component.

https://github.com/trimble-oss/modus-web-components/issues/1417
https://github.com/trimble-oss/modus-web-components/issues/1418
https://github.com/trimble-oss/modus-web-components/issues/1419
https://github.com/trimble-oss/modus-web-components/issues/1420
Closing this double.


docs: update link to React Query

updating the link to react-query to point to the latest docs. https://react-query.tanstack.com/ is a legacy link that re-directs to our v3 docs, while https://tanstack.com/query/latest/ is the entrypoint for the latest docs on TanStack Query (currently v5)

FYI, I did sign the CLA. Shouldn't the check be ✅ by now ?

thanks!


Inconsistent Popup Window Size Transition

Currently, the popup window's size changes when switching from break to focused mode. It would improve visual consistency and user experience if the popup window size remained constant.

I would like to take this issue

Closing this as #14 is merged now, thank you Henry.


cannot install FRT with Xilinx OpenCL ext 2022

Hi,
We recently upgraded to XRT/Vitis 2022.1. It seems that Xilinx has made a lot of changes to their OpenCL extension APIs, and as a result, FRT cannot be compiled. Here is an excerpt of the error when I was compiling FRT from source
[ 66%] Building CXX object CMakeFiles/frt_static.dir/src/frt/devices/xilinx_opencl_device.cpp.o
/scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp: In destructor ‘virtual fpga::internal::XilinxOpenclStream::~XilinxOpenclStream()’:
/scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp:22:39: error: too many arguments to function ‘void clReleaseStream()’
     auto err = clReleaseStream(stream_);
                                       ^
In file included from /opt/xilinx/xrt/include/CL/cl_ext.h:39:0,
                 from /usr/include/CL/opencl.h:50,
                 from /usr/include/CL/cl2.hpp:504,
                 from /scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.h:6,
                 from /scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp:1:
/opt/xilinx/xrt/include/CL/cl_ext_xilinx.h:342:13: note: declared here
 extern void clReleaseStream();
             ^~~~~~~~~~~~~~~
/scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp:22:10: error: ‘void err’ has incomplete type
     auto err = clReleaseStream(stream_);
          ^~~
/scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp: In constructor ‘fpga::internal::XilinxOpenclStream::XilinxOpenclStream(const string&, cl::Device, cl::Kernel, int, fpga::internal::Tag)’
:
/scratch/users/sx233/fpga-runtime/src/frt/devices/xilinx_opencl_stream.cpp:33:3: error: ‘cl_stream_flags’ was not declared in this scope
   cl_stream_flags flags;

Hi Shaojie, you are right that FRT requires an old XRT to compile because Xilinx removed the streaming APIs since XRT 2.13 in https://github.com/Xilinx/XRT/commit/d2540202aba4f402519b663612fe7d1c5b459196. I'll make xilinx_opencl_device an optional feature.

@hecmay Could you try a6c4f884d4bda35ab18197bc028dc0a78cf2e3e0 from the latest master branch? It will disable xilinx_opencl_stream if XRT_VERSION >= 2.13. You can disable it manually with cmake -DENABLE_XOCL_STREAM=OFF ..., too.
BTW, this is a private, compile-time dependency; as long as you don't actually use the streaming APIs from FRT, any library or binary files compiled against the old XRT will still work.

@Blaok thanks Yueze! Sure I will it a try.


dev: using Array as buffer in encode trait

[x] resolves #298
[x] follows contribution guide
[ ] code change includes tests

Hey @maciejka , could you take a look at the changes in the encode traits? I’d like to confirm if the core logic seems solid to you. If it looks good, I'll start updating the tests and addressing any minor errors. Thanks for your help!

@maciejka I think this would not work because of compact size encoding and pubkey script (and also segwit flags).
Maybe we can instead use a more efficient ByteArray alternative?
struct WordArray {
    buffer: Array<u32>,
    pending_word_len: usize
}

In case the pending word is partially filled we apply inefficient algorithm involving mult/div, if we are lucky and pending word len is 0 and our item happened to be of u32 - then we just append.
Will need to benchmark this to make sure we are actually gaining something.

Isn't ByteArray doing the same but with 8 bits of granularity? What we save here is that we don't need to deserialize from array of felts to an array of u32 before hashing.

Hey! @m-kus @maciejka , what idea should we move forward with?

Isn't ByteArray doing the same but with 8 bits of granularity? What we save here is that we don't need to deserialize from array of felts to an array of u32 before hashing.

Yes, ByteArray packs data into 31-byte chunks which we later convert to Array, here we use 4-byte chunks from the beginning.

Isn't ByteArray doing the same but with 8 bits of granularity? What we save here is that we don't need to deserialize from array of felts to an array of u32 before hashing.
@PavitraAgarwal21we are busy at devcon. Sorry for the delay. I will have a look today.

See: https://github.com/keep-starknet-strange/raito/pull/306


sdk/state: support single asset only

What
Change the SDK back to supporting a single asset only.
Why
We resolved to focus on single asset payment channels because someone made a good point that if we aren't supporting exchange/swaps, then there isn't huge value in having multiple assets in the one channel vs multiple single-asset channels.
Incorporates the changes proposed by SEP changes #146 and #147.
Close #72

It looks like we're keeping the Trustline struct

It looks like I missed removing a ton more functionality 😄. I just pushed 1601e42 that should remove the remainder of the multi asset support from the formation and open code.


Make Validator a singleton?

Validator gets passed around a lot now. It doesn't have any writable state, so it could become a Singleton. On the other hand, that explicit passing around means that the dependencies are explicitly documented, and making Validator a singleton would hide that dependency, making the code harder to maintain, see e.g. https://docs.microsoft.com/en-gb/archive/blogs/scottdensmore/why-singletons-are-evil. Let's revisit and decide at some point.

Oh hey, this got fixed by #72.


Support for listings package

Dear @ashthespy,
I am an intensive user of LaTeX (actually, pdflatex from the texlive distribution) to write papers about programming.
My LaTeX code includes a large number of code snippets, formatted with the listings package, see https://en.wikibooks.org/wiki/LaTeX/Source_Code_Listings.
It looks like the lstlisting environment is not recognized by your Atom style.  Is it handled?
It is especially important that the beautifier does not modifies the indentation there, and that the LaTeX keywords are not highlighted within this environment, as any arbitrary language can be used there.
Best regards,
Luc.

Hi Luc,
The package already supports lstlisting environment.
Could I request you to:

Provide a screenshot of the issue you are facing?
With your cursor in the  lstlisting environment, launch the Command pallet (Ctr+ Shift + P) and run Editor: Log Cursor Scope and share the scopes you see?

@lucbouge any updates? I am closing this issue as inactive for now - feel free to comment, and shall open it again.

@ashthespy  I am currently facing an issue with the lstlisting environment. The formatting of the environment does not follow the same rules as say the equation environment . See the screenshot below.

@ashthespy Hi, I am currently having the same problem.

`Scopes at Cursor
text.tex.latex
meta.embedded.block.generic
meta.function.embedded.latex
`
I wonder if I haven't configured the right settings.


formValues' values seem one step behind in some cases

I have a weird problem once again :D And I was not able to reproduce it in a simple CodeSandbox example, so I am not sure what is causing it in our code.
To summarize, I have a part of my form that updates a value (via the props change method, if that matters), and another part that displays that value, via formValues. Both are encapsulated in a FormSection (with name='my_array[0]'). When I update the value, the formValues component gets updated with the previous value.
For example:
Initial value is null => formValues value is null
I update the value from null to foo => nothing happens
I update the value from foo to bar => formValues value gets updated to foo
I update the value from bar to toto => formValues value gets updated to bar
I tried reproducing the problem in CodeSandbox but wasn't able to, so the problem is probably coming from somewhere in our code, but I don't know where, and I hope someone here can give me indications on what to look for, or better, maybe someone had this very problem and solved it :)

Got it ! And the problem I had on #3228 is very likely the same cause.
I tracked down the problem to the context updating with the wrong values, because I was calling change in a componentWillReceiveProps.
What happened was that an update on the store (unrelated to the form) would trigger reduxForm to update its values, triggering a cascade of context update for all its descendants, among which, the component where I called change. I would then call change, which would update the store, but the process of updating all of the childrens' context was still ongoing, with values corresponding to the previous state. I added a setTimeout to my change and the problem is solved. I should probably move it to componentDidUpdate instead.


AXON-368 fix in master (3.x) branch

This is a PR as requested  in http://issues.axonframework.org/youtrack/issue/AXON-368 for the master branch

Does it not make sense to deserialize MetaData values into their proper types instead of a generic Map, just as we do with the commands?
I've just built a MetaDataSerializer/Deserializer which wraps all values into containers with one field for the type and one field for the value to be able to achieve that. It's a bit messy, but so far it seems to be working (unit tests only, not deployed yet 😛). Now the only thing that remains is to try and override this default serializer :)

https://gist.github.com/sganslandt/0ada2f7b6462f878176514ce01bb2360 is the current PoC.

The problem of this approach is that it would in some ways invert the dependency between the data and the runtime system that deserializes it in making the data depend on the runtime system and not the other way round. It is problematic in a similar way it would be to use java binary serialization. It may work now, but what do you do if your classes change structurally or their names change down the road?
Other things to think about:
What if you are sharing your data with another system for integration? Do they need to know your java based implementation details or are you going to strip it out?
What do you do with existing production data?
What does this approach actually buy you? You can already deserialize "actual" objects with the current approach, but you have to make it explicit in your implementation instead of in your data.
Do not get me wrong: I think it is an interesting idea, just not a very good default. As an option it might be helpful though.

I see and agree with all of your points.
Just to clarify, my use case was about passing commands between service (ye, we haven't really dared to try the DistributedCommandBus), so the APIs would only live until all clients have been upgraded. Structural changes could be dealt with by versioning the endpoints.
Upcasters would be another option of course.
My main reason for going down this route dispite better knowledge is that commands in the CommandMessage already have handled in this way. Or least they they're passed down with a payloadType and payload to enable it.

What if you are sharing your data with another system for integration? Do they need to know your java based implementation details or are you going to strip it out?

No they don't. They "just" need a class with the same fully qualified name on the classpath that can be used to deserialize the payload. And if you're sharing API-pojos, they should be pojos, data classes.

What do you do with existing production data?

Well, my use case was far passing commands between servers so I don't have to care about that. If you would use this default for the event store you'd need upcasters.

What does this approach actually buy you? You can already deserialize "actual" objects with the current approach, but you have to make it explicit in your implementation instead of in your data.

The benefit of not having to do it explicitly, it just works. Obv. With all your mentioned downsides :)
Also, the XStream serialiser has all of these problems too, so in this particular context, I would still argue that it might be a sensible default still :)

My point is not that it cannot be done or that it does not have its merits but rather that it should not be the default because it breaks a lot of things (stored event data, which is the single most important kind of data in an event sourced system) and would require a lot of infrastructure within the framework (upcasters, versioning, etc.) that is not there yet.
The payload part of a message is completely different insofar as it has a type by definition. Meta data fields do not. They are actually defined by the serialization format (in case of JSON: arrays, objects, scalars or in case of XML element, attributes and text). Using java objects instead of more primitive constructs like lists, maps and scalars to interpret your meta data is the perspective of your implementation, not of the data.

What if you are sharing your data with another system for integration? Do they need to know your java based implementation details or are you going to strip it out?

No they don't. They "just" need a class with the same fully qualified name on the classpath that can be used to deserialize the payload. And if you're sharing API-pojos, they should be pojos, data classes.

This misses my point as the emphasis is on "java based implementation": what if they are not java based systems at all? If all you ever need is java, why bother using an implementation independent serialization format at all and waste cpu, memory, bandwith, storage, etc.? I do not mean that literally of course, but you see what I'm getting at :)

What do you do with existing production data?

Well, my use case was far passing commands between servers so I don't have to care about that. If you would use this default for the event store you'd need upcasters.

The thing about frameworks is that just caring for your use case, declaring it the default and ignoring others is usually a very good way to break stuff ;) Upcasters for meta data do not exist yet AFAIK and you haven't shown an example of what they might look like.

What does this approach actually buy you? You can already deserialize "actual" objects with the current approach, but you have to make it explicit in your implementation instead of in your data.

The benefit of not having to do it explicitly, it just works. Obv. With all your mentioned downsides :)
Also, the XStream serialiser has all of these problems too, so in this particular context, I would still argue that it might be a sensible default still :)

I am always very cautious of "it just works" as that mostly points to some form of probably unnecessary hidden complexity or another which at least in my mind goes against the fundamental principles of the whole DDD+CQRS+ES pattern (making implicit things explicit comes to mind), but let's not get too philosophical here :)
As for XStream: there is actually a fundamental difference between XML and JSON: XML has the ability to qualify elements by namespaces which enables a deserializer to do what you propose without introducing implementation specific information within the data. JSON does not currently have that ability, although there are things like JSON-LD out there. My point is: it might be better to explore options that do not structurally change the data in an incompatible way. Introducing additional fields (as JSON-LD does) is way less invasive and can be ignored by old clients. Changing the structure introduces a whole set of problems as we agree :)
I think it might be sensible to provide a pull request and or an issue of your own to discuss these things instead of highjacking this one as these are two distinct changes. But I am not a maintainer, just another user, so I am not sure what might be the preferred way here.

Upcasting metadata is supported in Axon 3, so the proposed solution of Sebastian could also work for persistent data like events.
However, I agree with Klaus that it is preferable not to modify metadata values structurally. To me the best way would be to configure Jackson to add the class name of non-primitive Metadata values using an additional property.
We could achieve this using Jackson’s @JsonTypeInfo annotation and Jackson Mixin annotations.

Upcasting metadata is supported in Axon 3, so the proposed solution of Sebastian could also work for persistent data like events.

Thanks for the clarification. I did not know that. Meta data upcasters should probably be used carefully because in my understanding payload data specifications and meta data specifications do not share the same lifecycle. Payload data specifications change per "message type" while as meta data specifications change more by "context boundary" (or "service", "component", "system" depending on wording and architecture) for a whole set of message types. That might be worth mentioning in the docs.

We could achieve this using Jackson’s @JsonTypeInfo annotation and Jackson Mixin annotations.

That should work and without breaking things :)
Another way might be to leverage more implementation independent ways like JSON-LD with Jackson which is more like namespaces in XML. I have not used this particular project myself but it looks promising.
There might be a different solution:
Currently the default JacksonSerializer has a hard wired dependency on the MetaDataSerializer-Implementation. It might be useful to be able to specify a custom JsonDeserializer instead of having to find a one-size-fits-all default.

We could achieve this using Jackson’s @JsonTypeInfo annotation and Jackson Mixin annotations.

Or just a custom Serializer to not have to sprinkle annotations everywhere? :)

Currently the default JacksonSerializer has a hard wired dependency on the MetaDataSerializer-Implementation. It might be useful to be able to specify a custom JsonDeserializer instead of having to find a one-size-fits-all default.

.... and that.


Mirroring workflow is broken

Example error:
To https://mscodehub.visualstudio.com/_git/win-net-test
 ! [remote rejected] main -> main (TF402[45](https://github.com/microsoft/win-net-test/actions/runs/8397088781/job/22999774886#step:3:46)5: Pushes to this branch are not permitted; you must use a pull request to update this branch.)
error: failed to push some refs to 'https://mscodehub.visualstudio.com/_git/win-net-test'
Successfully mirrored latest changes
Error: Process completed with exit code 1.

I fixed it. I had to give myself permissions to push and bypass policy, since my PAT is the one being used. Rerun passed.


Keep references to the callback functions

@piiswrong I only saw your comment after I was already done and I used a slightly different approach.
ref: #587  #554

@starimpact Could you test if either my PR or @piiswrong PR solves your problem?

ok, let me have a try.

I am sorry, but where can I get your commit?

cd to/your/local/mxnet/version
git remote add vchuravy https://github.com/vchuravy/mxnet.git
git fetch vchuravy
git checkout vc/py_gc

# make sure that your version of  the python bindings is up to date.
cd python
python setup.py install

ok , :smile:

do not need to do make ?

Sine there are only changes to the python side of things no. But you should make sure that you then have the correct version installed

Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 314, in 'calling callback function'
  File "/home/mzhang/anaconda/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/operator.py", line 167, in backward_entry
    self.backward(in_data=tensors[0], out_data=tensors[1],
SystemError: Objects/cellobject.c:24: bad argument to internal function

maybe I should use C++ to implement the perators.

I must use C++ now, since I must finish my project firstly.
Thank you very much. :smiley:

We can do more try after my project.

@piiswrong I guess this PR can be closed due to #587? Or is there anything that can be incorporated?

If @starimpact problem has been solved with #587 than we can indeed close this.

I am sorry. I just right have completed the CPP version , it can work well .
the python version, I will take a time to do a test, but not now.
Really busy.
Really sorry.

I am sorry. I just now have completed the CPP version , it can work well .
the python version, I will take a time to do a test, but not now.
Really busy.
Really sorry.

Closing for now


Should there be Microsoft Word substitute as epub is not sufficient as substitute ?

NewDraftMicrosoftWordSubstitute.txt
Scenario/Intent
[What you are trying to achieve but can't?]
Environment Details

OneView SDK Version: [Version of this SDK for which you are encountering the issue]
OneView Appliance Version: [Version of the OneView appliance you're interacting with]
OneView Client API Version: [API version listed in your configuration file or dictionary]
Python Version: [Version of Python in your environment]
Platform: [OS distribution and release version]

Steps to Reproduce
[What are the complete steps needed to do in order to reproduce your problem?]
Expected Result
[What do you expect to happen after taking the steps above?]
Actual Result
[What actually happens after the steps above? Include error output or a link to a gist.]

Suggestion  to file an issue at HPEDEV Slack

@vidushidewan - I am not sure what you are suggesting here. We are not using any editor in this library. This library provides a Python interface to the HPE OneView REST APIs.
If you are facing any issue with the HPE OneView Python sdk, then please elaborate the issue in detail.

Sorry Mam,
I didn't get you right maybe.
I am not talking about editor please.
Sorry for misconception.
On Mon 8 Apr, 2019, 11:48 Priyanka Sood<EMAIL_ADDRESS>wrote:

@vidushidewan https://github.com/vidushidewan - I am not sure what you
are suggesting here. We are not using any editor in this library. This
library provides a Python interface to the HPE OneView REST APIs.
If you are facing any issue with the HPE OneView Python sdk, then please
elaborate the issue in detail.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HewlettPackard/python-hpOneView/issues/427#issuecomment-480698306,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhTE0KgjG9LQUhnNAQDMEvwTtMoc74c3ks5vet8lgaJpZM4cgraR
.

@vidushidewan - Np. Please elaborate the issue by providing us the error you are getting.

No
It was just a suggestion
On Mon 8 Apr, 2019, 12:40 Priyanka Sood<EMAIL_ADDRESS>wrote:

@vidushidewan https://github.com/vidushidewan - Np. Please elaborate
the issue by providing us the error you are getting.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HewlettPackard/python-hpOneView/issues/427#issuecomment-480711169,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AhTE0Fe-sOD72f99b05sd9w1xm6kn03Fks5veutQgaJpZM4cgraR
.

@vidushidewan This project does not use epub document format for any assets.  As @soodpr asked, are you sure you are asking this question to the correct GitHub project?

Hello Chris
I am not sure if I wrote on the correct project at GitHub.
But, I wasn't sure which project should be approached for this
As for my books at Divine Publishers, I faced some trouble uploading ebooks
for sale
Later, I realised that the format is wrong , EPUB is the format usually
used for ebooks
And I might be wrong , not sure to say that EPUB is used exclusively by
Kindle
So. Later tried to find about EPUB and realised that it's Microsoft who
makes EPUB format.
That is the reason I tried to add a project to you that it is something I
don't know and it's not approachable to me
Please guide on this as I am not sure still
Can I try to code for type text document with HPE
Thanks
Vidushi Dewan
Proprietor - DAKSH GROUP
On Fri 10 May, 2019, 11:42 PM Chris Lynch<EMAIL_ADDRESS>wrote:

@vidushidewan https://github.com/vidushidewan This project does not use
epub document format for any assets. As @soodpr
https://github.com/soodpr asked, are you sure you are asking this
question to the correct GitHub project?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HewlettPackard/python-hpOneView/issues/427#issuecomment-491382645,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIKMJUDJIMT3QCIFWXTT7JDPUW3JFANCNFSM4HECW2IQ
.

Please close, this is obviously not related to the OneView library.

closing this issue as this is not related to OneView python sdk.

@Daniel,  thanks
May I ask you if any action needs to be taken from my end ?
Since the wrong project was approached earlier ?
On Thu 4 Jul, 2019, 8:55 PM Priyanka Sood<EMAIL_ADDRESS>wrote:

Closed #427
https://github.com/HewlettPackard/python-hpOneView/issues/427.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/HewlettPackard/python-hpOneView/issues/427?email_source=notifications&email_token=AIKMJUHF7BB5TKOVSGLX36TP5YI5JA5CNFSM4HECW2I2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOSKXJVRQ#event-2460916422,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIKMJUEFO5AFKY2WKS3GQ7TP5YI5JANCNFSM4HECW2IQ
.


Catching mouse events

Hi,
first of all, congratulations on such a great tool.
I was wondering, it is possible to catch mouse events? More specifically, I would like to set a callback function for mouse hovering and clicking (so that I know what residue that mouse is hovering above, and the same for features if possible). You must already handle in the zoom thank to the zoom feature. It there a public
API as well?
Something like:
protael.onMouseHoverCallback(function (residueIndex) {
   console.log(residueIndex);
});

To give an example, I need something like this - the website currently uses pViz, but I think protaeljs is better.
Thank you very much,
Lukas

Hi Lukas,
As of now, you can use following for mouseOver:
protael.onMouseOver(function (e) {
   console.log(protael.toOriginalX(protael.mouseToSvgXY(e).x));
});

Mayya

Hi Mayya,
it works great. Thank you!
Lukas


Sometimes npm complains about /home/docker/.config

I'm not sure why it happens but sometimes after fin rm and fun up cli gets created in a weird manner and npm complains about that file. (docksal/cli:1.2-php7)
Creating this issue to track this behavior and possible issue. Output is below. You might notice that owner of some files is not docker but 501. Which is weird, because docker user should have had id equal to 501 after "Changing uid/gid in cli to 501/20 to match the host". But it did not. That explains the nature of the error, but does not explain how did we get here.
08:34:35 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin rm -f
Removing containers...
Stopping demo_web_1 ... done
Stopping demo_cli_1 ... done
Stopping demo_mail_1 ... done
Stopping demo_db_1 ... done
Removing demo_web_1 ... done
Removing demo_cli_1 ... done
Removing demo_mail_1 ... done
Removing demo_db_1 ... done
Removing network demo_default
Removing volume demo_project_root
Volume docksal_ssh_agent is external, skipping
Removing volume demo_host_home

08:35:05 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin up
Starting services...
Creating network "demo_default" with the default driver
Creating volume "demo_project_root" with local driver
Creating volume "demo_host_home" with local driver
Creating demo_mail_1
Creating demo_cli_1
Creating demo_db_1
Creating demo_web_1
Changing uid/gid in cli to 501/20 to match the host
Resetting permissions on /var/www...
chown: changing ownership of ‘/var/www/node_modules/.bin/esparse’: Input/output error
chown: changing ownership of ‘/var/www/node_modules/.bin/esvalidate’: Input/output error
chown: changing ownership of ‘/var/www/node_modules/.bin/js-yaml’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/php-parse’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/psysh’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/robo’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/terminus’: Input/output error
Restarting php daemon...
Connected vhost-proxy to "demo_default" network.

08:35:26 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin run ls -la /home/docker

┌──────────────────────────────────────────────────────────┐
│                 npm update check failed                  │
│           Try running with sudo or get access            │
│           to the local update config store via           │
│ sudo chown -R $USER:$(id -gn $USER) /home/docker/.config │
└──────────────────────────────────────────────────────────┘
total 64
drwxr-xr-x  24<PHONE_NUMBER> May  4 12:35 .
drwxr-xr-x  12 root   root 4096 May  4 12:35 ..
drwxr-xr-x   2 docker  100 4096 May  4 12:35 .acquia
-rw-r--r--   1 docker  100  140 Apr  7 17:41 .bash_aliases
-rw-r--r--   1    501  100  220 Nov  5 21:22 .bash_logout
-rw-r--r--   1<PHONE_NUMBER> Apr  7 17:39 .bashrc
drwxr-xr-x   6 docker  100 4096 May  4 12:35 .composer
-rw-r--r--   1 docker  100  377 Apr  7 17:29 .docksalrc
drwxr-xr-x   7 docker  100 4096 May  4 12:35 .drush
drwxr-xr-x 340 docker  100 4096 May  4 12:35 .npm
drwxr-xr-x  14<PHONE_NUMBER> May  4 12:35 .nvm
-rw-r--r--   1 docker  100  716 Apr  7 17:41 .profile
drwxr-xr-x   2 docker  100 4096 May  4 12:35 .ssh
lrwxrwxrwx   1    501  100   36 Apr  7 17:38 .zlogin -> /home/docker/.zprezto/runcoms/zlogin
lrwxrwxrwx   1    501  100   37 Apr  7 17:38 .zlogout -> /home/docker/.zprezto/runcoms/zlogout
drwxr-xr-x  10 docker  100 4096 May  4 12:35 .zprezto
lrwxrwxrwx   1    501  100   39 Apr  7 17:38 .zpreztorc -> /home/docker/.zprezto/runcoms/zpreztorc
lrwxrwxrwx   1    501  100   38 Apr  7 17:38 .zprofile -> /home/docker/.zprezto/runcoms/zprofile
lrwxrwxrwx   1    501  100   36 Apr  7 17:38 .zshenv -> /home/docker/.zprezto/runcoms/zshenv
lrwxrwxrwx   1    501  100   35 Apr  7 17:38 .zshrc -> /home/docker/.zprezto/runcoms/zshrc
drwxr-xr-x   4 docker  100 4096 May  4 12:35 drush6
drwxr-xr-x   4 docker  100 4096 May  4 12:35 drush7

Notice: no .config dir at all!

After I got to this situation I got confused, so I did fin rm and fin up once again. And it just went back to normal.
08:37:03 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin rm
WARNING: All containers and volumes for the current project will be removed!
Continue? [y/n]: y
Removing containers...
Stopping demo_web_1 ... done
Stopping demo_db_1 ... done
Stopping demo_cli_1 ... done
Stopping demo_mail_1 ... done
Removing demo_web_1 ... done
Removing demo_db_1 ... done
Removing demo_cli_1 ... done
Removing demo_mail_1 ... done
Removing network demo_default
Removing volume demo_project_root
Volume docksal_ssh_agent is external, skipping
Removing volume demo_host_home

08:37:23 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin up
Starting services...
Creating network "demo_default" with the default driver
Creating volume "demo_project_root" with local driver
Creating volume "demo_host_home" with local driver
Creating demo_cli_1
Creating demo_mail_1
Creating demo_db_1
Creating demo_web_1
Changing uid/gid in cli to 501/20 to match the host
Resetting permissions on /var/www...
chown: changing ownership of ‘/var/www/node_modules/.bin/esparse’: Input/output error
chown: changing ownership of ‘/var/www/node_modules/.bin/esvalidate’: Input/output error
chown: changing ownership of ‘/var/www/node_modules/.bin/js-yaml’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/php-parse’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/psysh’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/robo’: Input/output error
chown: changing ownership of ‘/var/www/vendor/bin/terminus’: Input/output error
Restarting php daemon...
Connected vhost-proxy to "demo_default" network.

08:37:45 Alexei-Workstation /Users/alexei.chekulaev/Projects/demo
$ fin exec ls
222  app.js  composer.json  composer.lock  docker-compose.yml  docroot  installer.phar  node_modules  vendor

BTW home folder permissions are a bit off by default because we never set group id that would match host on them.

This hasn't happened for a while so I'll close this one.

how to fix

audited 10051 packages in 27.678s
found 30 vulnerabilities (3 low, 26 moderate, 1 high)
run npm audit fix to fix them, or npm audit for details
┌────────────────────────────────────────────────────────────┐
│                  npm update check failed                   │
│            Try running with sudo or get access             │
│            to the local update config store via            │
│ sudo chown -R $USER:$(id -gn $USER) /home/satheesh/.config │
└────────────────────────────────────────────────────────────┘
how to fix this anyone help me

@Satheesh10173 have you read the message you posted before posting it?


Convo.next()- how to make the bot listen carefully?

I'm having issues with creating a way where the bot correctly listens within a slack channel without replying to everything being said in the channel.
So, here is the way I currently approached this:
controller.hears(['You\'re awesome','Help',"What would Jeremy say?", "Hello","Top 5", "Hi", "I love you", /^.{0,}jirabot.{0,}$/], ['direct_message','direct_mention','mention','ambient'],function(bot,message) {

    console.log(message);

  // This will show the last 5 created tickets in Zendesk
    if(message.text === "Top 5"){
      try{
          Do this thing
      });
    }catch(err){
      bot.reply(message, 'I\'m sorry I did not get that. Please try again.');
    }

  }

    else if (message.text === "You're Awesome"){
        bot.reply(message, 'Nah, You\'re Awesome <@'+message.user+'>');
      }
    else {
         bot.reply(message, 'I\'m sorry I don\'t understand');
         }

This method works cause the bot is able to hear things like you're awesome and top 5. It does the action it is told to do so and then it waits to listen for another key word.
Now, if a user says something within the slack channel that doesn't match any of the keywords like 'Where's Jim?' The bot will always reply I'm sorry I don't get that. I understand why it's doing that, but I'm having issues in finding a way around it.
I also tried to do it this way:
var hi = 'HI';
var love = 'I LOVE YOU';

controller.hears([hi, /^.{0,}jirabot.{0,}$/],['direct_message','direct_mention','mention','ambient'],function(bot,message) {

  // start a conversation to handle this response.
  bot.startConversation(message,function(err,convo) {

      if (message.text.toLowerCase() === hi.toLowerCase()){
        bot.reply(message, 'What can I do for you? <@'+message.user+'>');
      convo.next();
    }else{
      bot.reply(message, 'Sorry, I don\'t understand');
      convo.next();
    }

  });
});
controller.hears([love, /^.{0,}jirabot.{0,}$/],['direct_message','direct_mention','mention','ambient'],function(bot,message) {

  // start a conversation to handle this response.
  bot.startConversation(message,function(err,convo) {

      if (message.text.toLowerCase() === love.toLowerCase()){
        bot.reply(message, 'No, I love you more <@'+message.user+'>');
      convo.next();
    }else{
      bot.reply(message, 'Sorry, I don\'t understand');
      convo.next();
    }

  });
});

The problem with this method is that the bot does the one action and doesn't do anything. So, if it hears Hi it will do the action. If it hears love it won't do anything. I put convo.next() in the controller for each specific action, However, it doesn't work.
For some reason this way seems to work:
controller.hears([love, /^.{0,}jirabot.{0,}$/],['direct_message','direct_mention','mention','ambient'],function(bot,message) {

  // start a conversation to handle this response.
  bot.startConversation(message,function(err,convo) {
      convo.ask('did you mean to say you love me?'function(response,convo) {
      if (message.text.toLowerCase() === love.toLowerCase()){
        bot.reply(message, 'No, I love you more <@'+message.user+'>');
      convo.next();
    }else{
      bot.reply(message, 'Sorry, I don\'t understand');
      convo.next();
    }
     })

  });
});

But, it makes no sense to have the bot ask the user for some input in order for it to keep the conversation going (meaning- be ready to hear the next keyword to be said).
Any feedback would be appreciated it, thanks!

I think you have to use convo.say() for startConversation and you don't need the command convo.next().
The bot.reply() is used outside the method startConversation().
Let me know if I helped you!


Removal of resources from the codebase

This issue is opened for further discussion about the removal of the fields resources and respective supports.
Thanks to @pritidesai @vdemeester and @lbernick for the discussion at the v1-CRD-release PR https://github.com/tektoncd/pipeline/pull/5579#discussion_r1029711364_
Related discussions:

https://github.com/tektoncd/pipeline/issues/4546
TEP-0074
TEP-0105 to remove v1alpha1
were proposed in API WG

AI:

[ ] Validations
[ ] Comments/ documents on the conversion

Closing as the consensus is reached during WG and resources will be kept supported in the code base while being serialized into annotations when converted to v1.
https://github.com/tektoncd/pipeline/issues/5805 is created as a follow up issue of the AI

Closing as the consensus is reached during WG and resources will be kept supported in the code base while being serialized into annotations when converted to v1.
https://github.com/tektoncd/pipeline/issues/5805 is created as a follow up issue of the AI


[1.0.0] Should we clone position on the PositionComponent setter?

So currently if you do:
  component.position = position

For a Position component it does not create a clone. I just wanted to discuss if this is our desired behaviour. These are the side effects:
No clone
If you do
  crate.position = car.position

They will be linked forever. Which might be intended or not, or might be a bit confusing.
Cloning
If you do
  final p = Vector2.all(10)
  crate.position = p
  p.x = 100

It won't move the box. Which might be intended or not, or might be a bit confusing.

Either way I don't think it's a bit clear, it just crossed my mind while I ported the old flame_crates_example and I wanted to flag. Regardless of the choice it's abundantly clear what will happen just looking at the code (which is what I did).

I wrote up a PR for it so that it sets the values to position instead of changing the reference, but I think it might be more confusing.
Because then we should also do the same on size and then I thought there are probably cases where you want to control the sizes of many components at the same time by just changing the same Vector2. And then I thought maybe the same is true for position, for example if you have several sprites on top of each other?
My suggestion is that we keep it like it is now and let the user handle cloning of the vector2, or to use position.setFrom(v).

is this discussion still valid after all the changes we did?

It is, we still don't have a setter which does clone on the argument to position on PositionComponent, we just have to decide whether that is something we want or not. If we do add it the user doesn't accidentally tie that Vector2 to something else, but it also restricts the user from controlling the position of multiple components at the same time through one single Vector2.

I see, IMO we shouldn't clone, so we create less instances, and hopefully get a better performance?

Let's keep it as is then, I don't think @luanpotter had any strong opinions on the matter either.
(Otherwise we can just re-open it)

SGTM


Updating dependencies to get rid of CVEs brought in with kafka and log4j-1.2 libs

Updating dependencies to get rid of CVEs brought in with kafka and log4j-1.2 libs
CVE-2020-27218,
CVE-2021-38153,
CVE-2021-44228,
CVE-2021-44832,
CVE-2021-45046,
CVE-2021-45105,
CVE-2020-9488,
CVE-2019-17571,
CVE-2021-4104
Motivation
mvn clean install verify -Powasp-dependency-check -DskipTests found various CVEs
Modifications
Upgraded dependencies/excluded transitive dependencies that were bringing in libraries with CVEs.
Verifying this change
This change is already covered by existing tests
Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): YES
The public API: (yes / no)
The schema: (yes / no / don't know)
The default values of configurations: (yes / no)
The wire protocol: (yes / no)
The rest endpoints: (yes / no)
The admin cli options: (yes / no)
Anything that affects deployment: (yes / no / don't know)

Documentation
Check the box below or label this PR directly (if you have committer privilege).
Need to update docs?

[ ] doc-required
(If you need help on updating docs, create a doc issue)

[x] no-need-doc
(Please explain why)

[ ] doc
(If this PR contains doc changes)

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

At this point I believe that the test failures are unrelated, I see similar flakiness on other PRs and the tests don't fail on local runs on the branch.

Closing and re-opening to pick up fix for CI.


iperf2 2.2.0 update

Description of changes

Update iperf2 2.1.4 (18 August 2021) to latest 2.2.0 version
-- Please note the existing version of iperf2 crashes for me on Linux x86
Prefer iperf2 over iper3, which is potentially slightly controversial, so here's the reasons why:
-- https://iperf2.sourceforge.io/IperfCompare.html

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

This is my first nixpkgs pull request, so I tried to follow the instructions.  I cloned the repo into a folder and did the rebuild, and it looks good.
sudo nixos-rebuild switch -I nixpkgs=/home/das/nixpkgs

[das@nixos:~]$ iperf2 --version
iperf version 2.2.0 (10 April 2024) pthreads

[das@nixos:~]$ which iperf2
/run/current-system/sw/bin/iperf2

[das@nixos:~]$ iperf --version
iperf version 2.2.0 (10 April 2024) pthreads

[das@nixos:~]$ uname -a
Linux nixos 6.6.32 #1-NixOS SMP PREEMPT_DYNAMIC Sat May 25 14:22:56 UTC 2024 x86_64 GNU/Linux

[das@nixos:~]$ date
Sun Jun  9 12:58:29 PM PDT 2024

[das@nixos:~]$ cd nixpkgs/

[das@nixos:~/nixpkgs]$ git rev-parse HEAD
1b8293b45bf9064bf871e24ae6aebbb2b7eb1793

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

Test on linux only
I have been testing iperf2 2.20 on linux for several weeks now, doing a manual install.
#!/usr/bin/bash

wget https://sourceforge.net/projects/iperf2/files/iperf-2.2.0.tar.gz/download
mv ./download ./iperf-2.2.0.tar.gz

tar zxf ./iperf-2.2.0.tar.gz
cd iperf-2.2.0 || exit

# git clone https://git.code.sf.net/p/iperf2/code iperf2-code
# cd iperf2-code || exit
./configure
make
echo "sudo make install"       <--- manual install

Thanks in advance
Dave

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4122


WIP: Issue 439 correct representation of datasetID

Please link to the issue #439.
Improve/correct the representation of Dataset ID in the Adobe Experience namespace.
Please review this WIP

@cdegroot-adobe for the schema "https://ns.adobe.com/experience/profile/experienceevent", I am not seeing "meta:extends": ["https://ns.adobe.com/xdm/context/experienceevent"] included. If you need this profile/experienceevent schema to be included in "context/experienceevent" as well, you will need this meta:extends. Or it will be only in "adobe/experience/experienceevent" schema.

👍